Why does a payment system barely need to scale for throughput?

Because the numbers are small. A million transactions a day is 1,000,000 ÷ 86,400 seconds ≈ 12 per second on average, and even a generous 5× peak is under 60 per second — a load a single modern database handles without breaking a sweat. The engineering difficulty in payments lives somewhere else entirely: correctness. A message queue that drops one message in a million is fine; a payment system that drops or duplicates one charge in a million has lost or double-billed real money, and someone will notice. So the whole design optimizes for never losing and never duplicating a cent — idempotency, a double-entry ledger, and reconciliation — rather than for requests per second.

How do payment systems avoid storing credit card numbers?

By never letting the card number touch their own servers. In the hosted-payment-page (or hosted-fields) flow, the customer’s card details go straight from their browser to a payment service provider (PSP) like Stripe or Adyen, which returns a single-use token — sometimes called a nonce — that stands in for the card. Your server only ever sees that token, and charges the card by sending the token plus an amount back to the PSP. This keeps the card number inside the PSP’s PCI-compliant environment and drops your own systems to the lightest tier of PCI DSS scope, because raw card data is never received, processed, or stored by you. It is the reason a startup can accept cards without building a vault: the PSP is the vault.

What is a double-entry ledger and why do payment systems use one?

A double-entry ledger records every movement of money as two matching legs — a debit on one account and a credit on another — that always sum to zero, so the books can never silently drift out of balance. Square’s ledger service, Books, describes the rule directly: "The accounting equation states that all transactions (which we call 'journal entries') must balance to 0, so each cent lost is matched with a cent gained." A single balance field can be corrupted by one bad update and you would never know; a double-entry ledger makes an illogical state — Square’s example is "$100 of paid-out ledger entries but $50 of payouts" — structurally impossible to represent. Because the entries are append-only and immutable, the ledger doubles as a complete audit log of every operation that ever happened.

Why should money be stored as integers instead of floating-point numbers?

Because binary floating point cannot represent most decimal fractions exactly, so arithmetic on dollars-as-floats accumulates rounding error — the classic demonstration is that 0.1 + 0.2 evaluates to 0.30000000000000004, not 0.3. On a single transaction that error is invisible; summed across millions of ledger entries it drifts, and a ledger that does not sum to exactly zero is a ledger you cannot trust. The fix is to store amounts as integers in the currency’s smallest unit — cents, not dollars. This is exactly what Stripe’s API requires: it "expects currency values using the given denomination’s smallest unit represented without decimals. For example, enter 1099 to charge 10.99 USD." Integer cents are exact under addition and subtraction, so the ledger balances to the penny.

What is payment reconciliation and why is it the last line of defense?

Reconciliation is a periodic job — usually nightly — that compares your own ledger against the settlement file the bank or PSP sends, line by line, to catch any place the two disagree. It exists because every earlier defense can still fail in ways only the source of truth reveals: a webhook that never arrived, a charge that succeeded at the PSP but whose ledger write was lost, a refund double-counted. The diff sorts mismatches into three classes — auto-fixable (a missing webhook you can now apply), classifiable-but-manual (a known discrepancy an operator must resolve), and unclassifiable (something genuinely wrong that pages a human). When your ledger and the bank disagree, the bank’s settlement file wins, because that is where the real money actually moved.

Design a Payment System

A million transactions a day is only about ten a second — so the hard part was never throughput, it was never losing or duplicating a single cent. Built from zero: the hosted-payment-page flow that keeps card numbers off your servers entirely (nonce → token → charge → webhook, with the PCI reality stated plainly), idempotency keys that survive both a double-click and a payment-processor retry, the double-entry ledger where every movement is a debit and a credit that sum to zero, why money is integers and never floats, a retry queue with a dead-letter tail for the charges that fail, and reconciliation as the last line of defense — the nightly diff against the bank’s settlement file and the three classes of mismatch it turns up. The signature exhibit sends one payment across the whole system with its idempotency key visible, replays it to watch every stage no-op, then breaks the ledger write and lets reconciliation catch the gap the next morning.

System design · Systems. The source ↗

A free, interactive, animated visual explainer of Design a Payment System — built to be understood, not skimmed.

Questions

Why does a payment system barely need to scale for throughput?: Because the numbers are small. A million transactions a day is 1,000,000 ÷ 86,400 seconds ≈ 12 per second on average, and even a generous 5× peak is under 60 per second — a load a single modern database handles without breaking a sweat. The engineering difficulty in payments lives somewhere else entirely: correctness. A message queue that drops one message in a million is fine; a payment system that drops or duplicates one charge in a million has lost or double-billed real money, and someone will notice. So the whole design optimizes for never losing and never duplicating a cent — idempotency, a double-entry ledger, and reconciliation — rather than for requests per second.
How do payment systems avoid storing credit card numbers?: By never letting the card number touch their own servers. In the hosted-payment-page (or hosted-fields) flow, the customer’s card details go straight from their browser to a payment service provider (PSP) like Stripe or Adyen, which returns a single-use token — sometimes called a nonce — that stands in for the card. Your server only ever sees that token, and charges the card by sending the token plus an amount back to the PSP. This keeps the card number inside the PSP’s PCI-compliant environment and drops your own systems to the lightest tier of PCI DSS scope, because raw card data is never received, processed, or stored by you. It is the reason a startup can accept cards without building a vault: the PSP is the vault.
What is a double-entry ledger and why do payment systems use one?: A double-entry ledger records every movement of money as two matching legs — a debit on one account and a credit on another — that always sum to zero, so the books can never silently drift out of balance. Square’s ledger service, Books, describes the rule directly: "The accounting equation states that all transactions (which we call 'journal entries') must balance to 0, so each cent lost is matched with a cent gained." A single balance field can be corrupted by one bad update and you would never know; a double-entry ledger makes an illogical state — Square’s example is "$100 of paid-out ledger entries but $50 of payouts" — structurally impossible to represent. Because the entries are append-only and immutable, the ledger doubles as a complete audit log of every operation that ever happened.
Why should money be stored as integers instead of floating-point numbers?: Because binary floating point cannot represent most decimal fractions exactly, so arithmetic on dollars-as-floats accumulates rounding error — the classic demonstration is that 0.1 + 0.2 evaluates to 0.30000000000000004, not 0.3. On a single transaction that error is invisible; summed across millions of ledger entries it drifts, and a ledger that does not sum to exactly zero is a ledger you cannot trust. The fix is to store amounts as integers in the currency’s smallest unit — cents, not dollars. This is exactly what Stripe’s API requires: it "expects currency values using the given denomination’s smallest unit represented without decimals. For example, enter 1099 to charge 10.99 USD." Integer cents are exact under addition and subtraction, so the ledger balances to the penny.
What is payment reconciliation and why is it the last line of defense?: Reconciliation is a periodic job — usually nightly — that compares your own ledger against the settlement file the bank or PSP sends, line by line, to catch any place the two disagree. It exists because every earlier defense can still fail in ways only the source of truth reveals: a webhook that never arrived, a charge that succeeded at the PSP but whose ledger write was lost, a refund double-counted. The diff sorts mismatches into three classes — auto-fixable (a missing webhook you can now apply), classifiable-but-manual (a known discrepancy an operator must resolve), and unclassifiable (something genuinely wrong that pages a human). When your ledger and the bank disagree, the bank’s settlement file wins, because that is where the real money actually moved.

Design a Payment System

Questions

Related explainers