How does a notification system send to millions of devices across push, SMS, and email?

It never delivers the last mile itself — it hands each message to a third party that owns the pipe to the device. An iOS push goes through Apple Push Notification service (APNs), an Android push through Firebase Cloud Messaging (FCM), a text through an SMS aggregator like Twilio, and an email through a provider like Amazon SES or SendGrid. The system you design is the fan-out layer in front of them: it takes one logical event — “your order shipped” — decides which channels a given user should get it on, renders the right template for each, and enqueues one job per channel. Workers pull those jobs and call the provider APIs. The scale comes from that queue-and-worker shape: 10 million pushes a day is only about 116 sends a second on average, but real traffic is spiky, so the queues absorb the peaks and the worker pools drain them at whatever rate the providers allow.

Why does each notification channel get its own queue?

So one channel’s bad day cannot become every channel’s bad day. Each provider fails and throttles independently — Twilio queues outbound SMS and drains it at your account’s messages-per-second limit, APNs and FCM have their own store-and-retry behavior — so if all four channels shared one queue and one worker pool, a Twilio outage would let SMS jobs pile up at the head of the line and starve the pushes and emails behind them. Give each channel its own queue and its own workers and the blast radius is contained: the SMS queue backs up and alarms, while push and email keep flowing untouched. When Twilio recovers, its dedicated workers drain the backlog at the capped rate. This is the same isolation a distributed message queue buys with partitions, applied one queue per channel.

Can a notification system guarantee a notification is delivered exactly once?

No — and the honest design says so out loud. The providers themselves refuse the guarantee: FCM’s own docs state that getting a message ID back “doesn’t mean that the message was already delivered to the device… it means that it was accepted for delivery,” and Apple says “delivery of notifications is a best effort, not guaranteed.” On top of that, your own workers can crash after calling the provider but before recording that they did, so on retry they call it again. The best you can build is at-least-once: never silently drop a notification, and accept that a small rate of duplicates is possible. You shrink the duplicate rate by deduplicating on a stable event ID — if a job for (event 91237, user 8, channel push) has already been sent, skip the resend — but a dedupe window reduces duplicates, it cannot erase them, because the very act of recording “sent” can itself fail. Idempotency is the tool; honesty about at-least-once is the mindset.

How does a notification system avoid spamming users?

With three first-class layers between the event and the send. User settings are the hard gate: a per-user, per-category, per-channel preference table that says this person turned marketing push off and only wants transactional email — checked on every send, never skipped, because skipping it is a compliance violation, not a glitch. Rate caps are the volume gate: a cap like “no more than 5 marketing pushes per user per day” and a global per-provider ceiling so a runaway loop can’t fire a million texts. And templates keep the content consistent and localized so you’re not hand-building message bodies in the hot path. The settings check is the one that has to hold even when the system is overloaded — the failure mode of skipping it “to keep up” is sending a notification to someone who explicitly opted out.

What happens when a provider like Twilio or APNs goes down?

The channel’s queue stops draining and starts growing; every other channel is unaffected because they’re isolated behind their own queues. The workers for the down channel keep retrying with exponential backoff — wait 1s, then 2s, 4s, 8s, with a random jitter so a fleet of workers doesn’t retry in lockstep — and the messages sit safely in the queue because you persisted them before enqueuing. Two things bound the damage: the queue has a retention window and a depth alarm, so operators know within minutes that SMS is backing up; and each message has a time-to-live, because a “your car is here” alert is worthless an hour late and should expire rather than deliver stale. When the provider recovers, the dedicated workers drain the backlog at the provider’s rate limit, dedupe suppresses the retries that had actually succeeded before the ack was lost, and the channel catches up without a single lost message.

Design a Notification System

One event — “your driver is arriving” — has to reach a phone, and you own almost none of the road: the last mile belongs to Apple, Google, Twilio, and an email provider, and none of them promises the message arrives. Built from zero: a commit-first envelope on 10M push + 1M SMS + 5M email a day, why one inline sender collapses, the fan-out core where a single event lands in one queue per channel so a Twilio outage never blocks a push, the device-token table and the templates / user-settings / rate-cap layers that keep you from spamming a real person, the reliability spine (persist-then-queue, retry with backoff, and the honest at-least-once truth that dedupe reduces but cannot erase), the open/click tracking loop — then the failure sweep whose quietest box is a compliance bug that ships a notification to someone who opted out.

System design · Systems. The source ↗

A free, interactive, animated visual explainer of Design a Notification System — built to be understood, not skimmed.

Questions

How does a notification system send to millions of devices across push, SMS, and email?: It never delivers the last mile itself — it hands each message to a third party that owns the pipe to the device. An iOS push goes through Apple Push Notification service (APNs), an Android push through Firebase Cloud Messaging (FCM), a text through an SMS aggregator like Twilio, and an email through a provider like Amazon SES or SendGrid. The system you design is the fan-out layer in front of them: it takes one logical event — “your order shipped” — decides which channels a given user should get it on, renders the right template for each, and enqueues one job per channel. Workers pull those jobs and call the provider APIs. The scale comes from that queue-and-worker shape: 10 million pushes a day is only about 116 sends a second on average, but real traffic is spiky, so the queues absorb the peaks and the worker pools drain them at whatever rate the providers allow.
Why does each notification channel get its own queue?: So one channel’s bad day cannot become every channel’s bad day. Each provider fails and throttles independently — Twilio queues outbound SMS and drains it at your account’s messages-per-second limit, APNs and FCM have their own store-and-retry behavior — so if all four channels shared one queue and one worker pool, a Twilio outage would let SMS jobs pile up at the head of the line and starve the pushes and emails behind them. Give each channel its own queue and its own workers and the blast radius is contained: the SMS queue backs up and alarms, while push and email keep flowing untouched. When Twilio recovers, its dedicated workers drain the backlog at the capped rate. This is the same isolation a distributed message queue buys with partitions, applied one queue per channel.
Can a notification system guarantee a notification is delivered exactly once?: No — and the honest design says so out loud. The providers themselves refuse the guarantee: FCM’s own docs state that getting a message ID back “doesn’t mean that the message was already delivered to the device… it means that it was accepted for delivery,” and Apple says “delivery of notifications is a best effort, not guaranteed.” On top of that, your own workers can crash after calling the provider but before recording that they did, so on retry they call it again. The best you can build is at-least-once: never silently drop a notification, and accept that a small rate of duplicates is possible. You shrink the duplicate rate by deduplicating on a stable event ID — if a job for (event 91237, user 8, channel push) has already been sent, skip the resend — but a dedupe window reduces duplicates, it cannot erase them, because the very act of recording “sent” can itself fail. Idempotency is the tool; honesty about at-least-once is the mindset.
How does a notification system avoid spamming users?: With three first-class layers between the event and the send. User settings are the hard gate: a per-user, per-category, per-channel preference table that says this person turned marketing push off and only wants transactional email — checked on every send, never skipped, because skipping it is a compliance violation, not a glitch. Rate caps are the volume gate: a cap like “no more than 5 marketing pushes per user per day” and a global per-provider ceiling so a runaway loop can’t fire a million texts. And templates keep the content consistent and localized so you’re not hand-building message bodies in the hot path. The settings check is the one that has to hold even when the system is overloaded — the failure mode of skipping it “to keep up” is sending a notification to someone who explicitly opted out.
What happens when a provider like Twilio or APNs goes down?: The channel’s queue stops draining and starts growing; every other channel is unaffected because they’re isolated behind their own queues. The workers for the down channel keep retrying with exponential backoff — wait 1s, then 2s, 4s, 8s, with a random jitter so a fleet of workers doesn’t retry in lockstep — and the messages sit safely in the queue because you persisted them before enqueuing. Two things bound the damage: the queue has a retention window and a depth alarm, so operators know within minutes that SMS is backing up; and each message has a time-to-live, because a “your car is here” alert is worthless an hour late and should expire rather than deliver stale. When the provider recovers, the dedicated workers drain the backlog at the provider’s rate limit, dedupe suppresses the retries that had actually succeeded before the ack was lost, and the channel catches up without a single lost message.

Design a Notification System

Questions

Related explainers