Rediscovering Transaction Processing From History and First Principles
How TigerBeetle discovered that row locks held across network round-trips are a fundamental throughput ceiling no amount of hardware can fix.
Every database scaling problem eventually comes down to buying more hardware. Until the bottleneck isn’t compute or storage at all — it’s a row lock held open while bytes travel across a wire.
Source
Rediscovering Transaction Processing From History and First Principles
By: Joran Dirk Greef, TigerBeetle
Published: Jul 23 2024
Type: Engineering article
Worth saving because the diagnosis here — not the product — is what most engineers working on payment or ledger systems need first. The argument traces a current architectural bottleneck back to Jim Gray’s 1985 benchmark and arrives somewhere counterintuitive: the OLTP problem was named correctly forty years ago, and general-purpose databases were never the right answer for it.
Why row locks and network round-trips define your throughput ceiling
In 1985, Jim Gray published “A Measure of Transaction Processing Power,” a paper that launched what collaborators later called a benchmark war driving the database industry forward for fifteen years. Gray defined the standard transaction not as a SQL operation but as a DebitCredit: debit a bank account, update a history record, rewrite teller and branch balances, commit. That choice is striking because SQL had existed for a decade by then. Gray could have defined the benchmark transaction in SQL. He defined it as debit/credit because that was the textbook example of why atomicity, consistency, isolation, and durability existed in the first place — the transfer of value between two accounts, nothing more.
Gray also insisted on cost-efficiency alongside raw throughput. He noted that some systems required six months of expert time to implement the benchmark while others took a day or two. This dual frame — cost per transaction, not just transactions per second — shapes TigerBeetle’s argument four decades later.
In 2020, Joran Dirk Greef was consulting on a central bank payments switch funded by the Bill and Melinda Gates Foundation. The switch processed financial transfers through a general-purpose relational database. Tracing its SQL queries for a single financial transaction showed the problem immediately: 10–20 SQL queries per transfer, all executing while row locks were held. For one debit/credit, the database selected account balances and locked those rows, shipped the data across the network to application code where the funds check happened, wrote the new balances back, and released the locks. Every step crossed the wire while locks stayed open.
The arithmetic is brutal. If a transaction requires at minimum two network round-trips while holding locks, and a typical round-trip time is 0.5 milliseconds, then competing transactions cannot acquire those locks for at least one millisecond. One millisecond per transaction, one thousand milliseconds per second: you are capped at roughly 1,000 transactions per second on any contended row regardless of the hardware behind it. Most OLTP workloads have hot rows — branch accounts, global counters, shared inventory — that a large fraction of transactions touch. Adding more CPUs, faster disks, or additional replicas doesn’t help because the lock serializes access to that row for every machine. The bottleneck isn’t throughput; it’s contention. They’re different problems.
Greef’s response was to ask what a database would look like if debit/credit were a first-class primitive instead of something assembled from SQL. The key measurement: a single debit/credit is 128 bytes — two CPU cache lines. TigerBeetle packs up to 8,000 debit/credits into a single 1 MiB batch, processes the entire batch in one round-trip to the database, and eliminates row locks because the transfer semantics are enforced inside the engine rather than by application code making repeated trips. What previously required 8,000 SQL queries — each acquiring locks across the wire — becomes one operation. The claim is a 1,000× improvement in that specific scenario, and it follows directly from the arithmetic of the bottleneck, not from clever engineering.
The generalisable idea here is impedance mismatch. When the database’s unit of work — a row lock, a query, a commit — is smaller than the application’s unit of work — a transfer, a two-phase reservation, a batch settlement — every semantic operation becomes a sequence of round-trips while holding state. The only escape is either to push the decision logic into the database, or to redesign what the database treats as atomic. Stored procedures push logic in but don’t eliminate the lock-plus-round-trip problem for workloads with intrinsic contention across a small number of hot rows. TigerBeetle takes the second path: make the domain primitive atomic, and the impedance gap closes.
Safety improvements came alongside the performance redesign. TigerBeetle applies 2018 Protocol-Aware Recovery to handle storage faults that standard crash-recovery algorithms miss, borrows NASA’s Power of Ten rules for safety-critical code (including static memory allocation and 6,000+ runtime assertions), and runs a deterministic simulator that does model checking against the actual production code rather than a separate representation. The motivation for that level of rigor: India’s instant payments volume grew 7,000× in seven years to 14 billion transactions per month, and the world’s payments infrastructure is accelerating faster than the databases underneath it were designed for.
What I find most striking is that the architecture Greef arrived at from first principles — by tracing a bottleneck in a central bank switch — is exactly the primitive Jim Gray named the canonical transaction forty years earlier. He didn’t start from Gray’s history; he started from a profiling session. The convergence is real: debit/credit is the minimal unit of financial state change, 128 bytes, and any database that handles a transfer as multiple smaller operations will serialize on hot rows at scale.
My take
This post reveals how much generality costs. Row locks and network round-trips are perfectly reasonable abstractions when the database can’t know what the application will do. When the domain is fixed — financial ledgers, energy arbitrage, gaming economies with shared pools — that abstraction is pure overhead. The honest version of TigerBeetle’s argument isn’t “we’re faster”; it’s “the contract between database and application was drawn at the wrong level of abstraction for this workload, and no amount of hardware fixes a design mismatch.”
Where this is useful
- When diagnosing an OLTP throughput ceiling that doesn’t improve with additional hardware — contention on hot rows is almost always the root cause, not raw compute.
- When designing a payment or ledger system and reaching for Postgres with stored procedures: this is why stored procedures reduce round-trips but don’t fully resolve the lock-RTT problem on hot accounts.
- When evaluating purpose-built OLTP systems for workloads where the transaction schema is known and fixed.
- Not applicable to analytics, general-purpose web applications, or any workload where diverse schemas and ad hoc queries are genuinely necessary.
Apply this: Count the SQL queries your application sends per business transaction, and identify which of them acquire locks. Each locked round-trip is a cap on your maximum TPS for that row, independent of hardware.
What I want to remember
Remember: When throughput plateaus and hardware doesn’t help, look for a lock held across a network round-trip. That lock is the ceiling. Hardware scales compute; nothing scales away contention.