System Design curriculum

Clients are many and independent, so nothing stops them from arriving at once, while the server's threads, connections and memory are finite. Protecting it means admission control: per-client rate limits, bounded queues, and rejecting excess work quickly. Turning a few requests away keeps the rest fast; admitting every one of them makes all clients wait together.

The state still exists; it moves to a shared store such as a database or cache, found by a session token the client sends. That lets a load balancer pick any instance and lets you replace or terminate instances without logging anyone out. The cost is a network read for that state on each request, and the shared store becomes the part you must scale and make redundant.

A web tier often spends its effort on connections and TLS, an application tier on CPU for business logic, and a data tier on disk I/O, memory for its working set, and locks. Find which one is saturated before adding machines: more app servers in front of a busy database only add database connections. Stateless tiers grow by cloning; the data tier grows by replicas or partitions.

Hard dependencies multiply: three components at 99.99% each give about 99.97% together, so one unduplicated piece caps everything that depends on it. Two independent copies at 99.9% give about 99.9999%, because both must fail at once. The catch is independence. Replicas that share a power feed, a switch or a config push fail together, and the math stops holding.

The classic list has eight: the four in the title plus a secure network, a topology that never changes, zero transport cost and a homogeneous network. Each becomes a design rule — set timeouts, budget for slow and lossy links, expect routes and peers to change, and authenticate traffic between your own services. Code written as if a remote call were local quietly assumes all eight.

In one process, a crash takes caller and callee down together, so you either get an answer or nothing ran. Across a network they fail independently: the request can be lost, the server can crash mid-work, or the reply can vanish after the work committed. That is why every remote call needs a timeout, and why the caller must plan for an outcome it cannot observe.

A timeout covers three cases you cannot tell apart: the request never arrived, the server failed partway, or the work finished and the reply was lost. Retrying blindly can charge a card twice. Make the operation idempotent, usually with a client-generated request ID the server records, or read back the resulting state before deciding whether to send it again.

The gaps are uneven. RAM to SSD is about 1,000×, SSD to a same-datacenter round trip only about 5×, and one cross-continent round trip costs about 200 same-datacenter ones. Physics sets that last floor: light in fiber covers roughly 200 km per millisecond, so a 5,000 km path costs at least 50 ms there and back before any queueing. Treat these as orders of magnitude, not benchmarks.

A batch spreads a fixed cost, such as a disk sync, a network round trip or a lock, over many items, so more items finish per second. But the first item waits for the batch to fill or a timer to fire, so its latency grows by up to that wait. Flushing at N items or T milliseconds, whichever comes first, caps the latency you give up.

Speeding up another stage can still cut latency for the work passing through it, but output stays the same; that stage just waits more. Find the bottleneck where work queues up or where one stage stays near 100% busy. Once you widen it, the next slowest stage becomes the ceiling, so measure again rather than assuming the old limit still applies.

Worked example: 10 million daily users × 20 requests each is 200 million a day, and 200,000,000 ÷ 86,400 is about 2,300 per second. Rounding a day up to 100,000 seconds keeps the math mental but lands about 14% low, so add margin. Then multiply by your measured peak factor and by fan-out, since one user action often triggers several backend calls.

The mean hides the daily cycle: traffic follows waking hours, and launches, sales or news add spikes on top. Take the factor from your own history, using the busiest minute rather than the busiest hour, because an hourly average smooths away the burst that actually saturates you. An audience spread across time zones flattens the curve; one region sharpens it.

Reads can be answered by any copy, so caches and replicas absorb them and a high hit rate removes most of the load. Every write must reach the authoritative copy and then every replica and cache holding the item, so extra copies multiply write work instead of dividing it. Heavy writes need splitting across partitions, each owning a slice of the keys.

Worked example: 1 million rows a day × 1 KB × 365 days is about 365 GB, and three replicas make it about 1.1 TB before indexes. Write-ahead logs, backups and free headroom add more on top. Keep units straight, bytes versus bits and KB versus KiB, and round up generously so a sloppy assumption shows up on paper instead of as a full disk.

Worked example: 2,000 requests per second × 500 KB each is 1 GB per second, which is 8 Gbps, since links are rated in bits. Cloud pricing is lopsided: AWS, for instance, does not charge for data arriving from the internet but bills data sent out to it. A design that serves large responses pays per byte, which is what compression and edge caching cut.

Read it in reverse during an incident: if a dependency slows from 100 ms to 1 s while traffic stays at 1,000 per second, in-flight requests jump from 100 to 1,000. Thread pools, connection pools and memory sized for 100 then run out, and the queueing adds still more latency. The law holds for long-run averages of any stable system, whatever the arrival pattern.

For one server with random arrivals, average time in the system is service time ÷ (1 − utilization): 2× at 50% busy, 5× at 80%, 10× at 90% and 20× at 95%. The last few points of utilization cost the most latency, so plan to run well below the knee. Arrivals or request costs more variable than random push waits higher still, at the same utilization.

Speedup on N machines is 1 ÷ (serial share + parallel share ÷ N). With 5% serial work, 100 machines give about 16.8×, and no number of machines can pass 20×. In distributed systems the serial share hides in a single coordinator, a global lock or a final merge step, so shrinking that part usually beats adding nodes.

In its usual form, capacity at N nodes is N ÷ (1 + a(N − 1) + bN(N − 1)). The a term is waiting on a shared resource, and on its own it only flattens growth toward a ceiling of 1 ÷ a. The b term is the cost of keeping nodes in agreement, which grows with every pair, so capacity peaks near N = sqrt((1 − a) ÷ b) and then falls. Fit a and b from load tests.

With three zones each carrying a third of peak, losing one hands the other two half of peak each, 1.5× their normal share, so each can run at no more than about two-thirds of its tested capacity at peak. Four zones allow 75%. Planning to launch capacity during the failure ties recovery to a control plane that the same event may have impaired.

Scaling up is often the quickest fix for a database because the code stays unchanged, but a resize often needs a restart and the largest size is a wall. Scaling out asks for work the app may not have done: requests any node can serve, state moved out of the process, and a load balancer in front. Below the ceiling, a bigger box can be cheaper than the coordination.

Cloning is the cheapest and needs a stateless app, but every copy still sees all the data, so it shrinks neither the database nor each cache's working set. Splitting by function lets each service scale and deploy alone, at the price of network calls between them. Splitting by data runs identical code on slices of the data, which needs a router and makes repartitioning hard.

Each node owns its slice of data plus its own CPU, memory and disk, so adding a node adds all three. The ceiling returns wherever nodes meet: a shared database, a global lock, a central cache, or any request that must touch many partitions. Near-linear growth holds only while most requests stay inside one node, which is why partition design matters more than node count.

Latency rises only after a queue has formed, so a latency trigger fires when users already wait, and it also rises when a downstream service slows, where more instances do not help. Backlog divided by the workers serving it says directly how far behind you are: with a target of 100 messages per worker, 1,500 queued across 10 workers calls for 5 more.

A new instance must boot, start the app, warm its caches and pass health checks before it helps, and scaling policies also wait for its metrics to settle. Burst absorption therefore comes from capacity you already hold: steady headroom, scheduled or predictive scaling ahead of known peaks, queues that let work wait, and load shedding that rejects the excess quickly.

Draining has three steps: the load balancer stops sending new requests, in-flight requests finish, and only then does the process exit. Set the window longer than your slowest normal request. WebSockets and other long-lived streams outlast any reasonable window, so clients must expect to reconnect. Queue workers need the same care: stop taking new work first.

In AWS Lambda's standard model each concurrent request gets its own execution environment, and a new one runs your setup code before the handler, which is the cold start. Because environments multiply with concurrency, a spike can open hundreds of database connections at once and exhaust the database. A connection pooler in front of it, and pre-warmed environments for latency-critical paths, are the usual fixes.

Every replica must apply every write, so adding replicas adds read capacity but no write capacity, and each one adds replication traffic the primary must send. Write capacity grows only when different nodes own different keys, so each write lands on one partition. Replicas also lag, so a read routed to one right after a write may miss it.

A global auto-increment counter, one lock guarding a hot table, or one leader that every write passes through serializes work no matter how many nodes surround it. Common fixes: hand out IDs in blocks, or build them from a node ID plus a timestamp and counter; lock per row or key; and split one leader into many by partitioning, so each owns a range.

An L4 balancer picks a backend once per connection and copies bytes, so it cannot route by path or header and cannot retry a single failed request. That bites with HTTP/2 and gRPC, which carry many requests on one long-lived connection: L4 pins them all to one backend. L7 terminates the protocol and balances each request, paying for the parsing and usually TLS too.

Rotation counts requests, not work. When the costliest queries take 1,000 to 10,000 times the CPU of the cheapest, a backend that draws a few heavy ones runs hot while its peers idle; Google's SRE book reports up to a 2× CPU spread under plain round robin. Differences between machines and noisy neighbors widen the gap further.

It has a trap: a broken backend that returns errors instantly always looks idle, so it draws even more traffic. Count recent errors as if they were active requests, or slow failed responses down to normal latency. Each client also sees only its own in-flight requests, so with many clients the balance it achieves is local rather than global.

Always picking the single least-loaded backend works badly when load data is slightly stale: every balancer sees the same 'best' target and they all pile onto it. Picking two at random and taking the lighter keeps most of the benefit while spreading choices, because different callers sample different pairs. It needs only two load readings, never a scan of the whole fleet.

Requests for the same key keep landing where that key's data is already in memory, and adding or removing one of N backends moves only about 1/N of keys instead of reshuffling everything. The price is balance: one hot key overloads its owner, and a backend's load follows its keys rather than the traffic mix. Spread a known hot key across several backends.

An active probe finds a dead backend even when no traffic flows, but it tests only the path the probe exercises. Passive checking watches real responses, so it catches failures probes miss, yet it learns only after users hit errors and needs enough traffic to judge. Use both, and cap how much of the fleet passive ejection may remove so one shared fault cannot empty the pool.

The opposite mistake costs as much. If every server's check calls a shared database, one database blip fails every check at once and the balancer pulls servers that could still serve cached or partial answers. A common split keeps the routing check to local resources such as disk and the process, and alerts on dependencies separately. Many balancers fail open when every target looks unhealthy.

Each connection costs memory and health-check work on both ends, so 1,000 clients each talking to 1,000 servers means a million connections. A subset of 20 to 100 backends per client bounds that, but random subsets leave some backends with far more clients than others. Deterministic assignment, with a fresh shuffle for each round of clients, gives every backend an equal share.

Users pinned to an instance stay there, so a newly added instance gets only new sessions and idles while the old ones stay hot, which makes scale-out look ineffective. When the pinned instance dies or drains, the balancer sends the user elsewhere and any session state held in that instance's memory is gone. Keeping sessions in a shared store removes the need for affinity.

Recursive resolvers cache answers for the TTL, and some ignore the TTL, so a change spreads gradually and never fully on schedule. One resolver can serve thousands of users, so a single cached answer moves a large block of traffic at once. The authoritative server also sees the resolver's address, not the user's, unless the resolver forwards a client subnet hint.

Light in fiber travels about 200 km per millisecond and real routes run longer than the straight line, so a user 8,000 km away waits at least 80 ms per round trip. A fresh connection over TCP with TLS 1.3 spends two of those round trips before the request goes out. A nearby edge site terminates the connection and serves cached bytes, so the distant trips happen only on misses.

With pull, the first request at each edge site misses and fetches from the origin, then later requests hit, so rarely requested files cost nothing and nothing has to be uploaded ahead. With push, you upload ahead, which avoids that first miss for a known large release but makes you manage what sits at every site and pay to store files nobody asks for.

max-age sets freshness for every cache, s-maxage overrides it for shared caches only, private forbids shared caches from storing the response, and no-cache allows storing but requires revalidation before each reuse; no-store is the one that forbids storage. Many CDNs also offer minimum and maximum TTL settings that override the origin, so check both layers.

When a cached copy goes stale, the cache sends If-None-Match with the stored ETag, or If-Modified-Since with the date; if nothing changed, a 304 comes back carrying headers and no body. For a large file that saves most of the transfer, but the round trip remains, so small objects gain little. Dates have one-second resolution, a gap ETags do not have.

A cache keys on the URL by default; every header, cookie or query parameter added to the key splits one object into many copies. A tracking parameter or a per-user session cookie can make every request unique. Vary: Accept-Encoding is cheap because it has few values, while Vary: User-Agent has thousands. Normalize or drop any value the response does not depend on.

Put a hash of the file's contents in its name, serve it with a long max-age plus immutable, and reference it from an HTML page that is cached briefly or revalidated. Changing the file changes the name, so every cache fetches the new one. A purge, by contrast, takes time to spread across edge sites and cannot reach copies already sitting in users' browsers.

Without one, each regional cache that misses sends its own request, so a popular new object can reach the origin once per region. With a shield, all misses funnel through one layer, and simultaneous misses for the same object collapse into one fetch. Place it where latency to the origin is lowest; it adds a hop on misses and helps little for content that barely caches.

TCP needs one round trip before the request can leave, and a full TLS 1.3 handshake adds another, so over a 100 ms link a fresh HTTPS request waits about 200 ms before it can even be sent. Reusing a kept-alive connection skips both. TLS 1.3 0-RTT resumption sends data in the first flight, but that early data can be replayed, so limit it to idempotent requests.

BGP picks the entry point by network topology and routing policy, which usually, but not always, tracks distance or latency. Failover is automatic: withdraw a site's route and its traffic shifts elsewhere, and attack traffic is soaked up near where it starts. The risk is a route change mid-connection, which can send later TCP packets to a site holding no state for them.

Empty is safe for correctness, not for load: after a flush, restart or deploy, every request misses at once and the database takes the full read rate. Each miss also costs three steps: check, query, populate. On writes, delete the cached key instead of updating it, since two racing writers can leave the older value cached, and set a TTL to cap how long any mistake survives.

Write-through refreshes the cached copy on each write, so reads stay current only while every write goes through the cache; a batch job or another service writing the database directly leaves it stale. Each write pays two trips, a newly added cache node starts empty, and much written data is never read, so pair it with lazy loading and a TTL.

Writes land in the cache and reach the database later, often batched, so repeated updates to one key collapse into one database write and write latency is just cache latency. Anything not yet flushed is lost if the cache node fails, and anyone reading the database directly sees older data until the flush. Use it where losing the last few seconds of writes is acceptable.

Choose the TTL from how much staleness a reader tolerates: a price might allow seconds, a country list days. Keys written together expire together, so add random jitter to spread their refreshes. A soft TTL that triggers a refresh, plus a longer hard TTL during which the old value is still served if the source fails, keeps reads working through an outage.

LRU evicts whatever was used longest ago, so one pass over a large table can push out every hot entry, since each scanned item is briefly the most recent. LFU keeps often-used items but can cling to ones that were popular yesterday. Scan-resistant designs admit new items to a probation area and promote only those used again, so a one-off scan never reaches the protected set.

Miss rate, not hit rate, sizes the origin: going from 95% to 99% hits cuts origin traffic fivefold, from 5 in 100 requests to 1. Turned around, losing the cache at 95% sends the origin 20 times its usual load. Size the origin for the miss stream at peak, and either for a cold-cache event or for shedding load while the cache refills.

It happens whenever a popular entry vanishes: expiry, eviction, a node restart, or a deploy that empties local caches. Hundreds of requests miss in the same instant, each runs the same expensive query, and the slowed database makes every miss take longer, so more pile up. Many keys set at once with the same TTL turn one stampede into a wave.

With coalescing, the first miss fetches while concurrent requests for the same key wait for its result. It works per process, so 50 app servers can still send 50 fetches; a shared lock or shared cache tier closes that gap. Early refresh renews a hot key in the background before it expires, and serving the stale value while one refresh runs keeps waits near zero.

Give cached misses a shorter TTL than hits, and delete the cached miss when the key is created, or a new record stays invisible until the miss expires. It protects against repeated lookups of the same absent key, not against a flood of distinct random keys, each of which misses once; rate limits or a membership filter handle that case.

A user's display name may live in their profile entry, in many feed entries and in search results, so renaming them means expiring every derived copy. Track which keys derive from which records, tag entries so one purge clears a group, or put a version number in the key so a write makes old entries unreachable. TTLs are the backstop for anything the logic misses.

Keeping keys in order means 'all orders from March' reads a few adjacent partitions. But a timestamp or sequence key sends every insert to the partition holding the highest values, so one node takes all writes while the rest idle. Leading with a hashed or high-cardinality column spreads inserts, at the cost of turning those time-range reads into reads across many partitions.

The hash scatters neighboring keys, so a query over a range of IDs or dates must ask every partition. A compound key keeps both: hash one part, such as a customer ID, to pick the partition, then sort by another, such as a timestamp, inside it, so one customer's history still scans in order. Hashing cannot help when one key value is itself hot.

With plain modulo hashing, growing from 10 to 11 nodes leaves a key in place only when both remainders agree, about 1 time in 11, so roughly 91% of keys move and caches go cold everywhere. On a ring, the new node takes over only the arc next to its position, about 1/11 of the keys, and every other assignment stays where it was.

With one token per node and only a few nodes, random positions leave some nodes owning arcs several times larger than others. Many tokens per node average that out, and when a node joins or leaves, its data moves to or from many peers rather than one neighbor, spreading the rebuild work. More tokens also mean more neighbors per node and more repair work, so the count is a trade-off.

If orders are sharded by order ID but the common query is 'orders for customer X', every lookup must ask all shards and wait for the slowest. Choose the key from the most frequent and latency-sensitive queries, and keep data that is read together on one shard. Queries on other fields need a secondary index or a second copy of the data keyed their way.

A celebrity's profile or a flash-sale item draws far more traffic to one key than hashing can spread, since every request for it lands on one partition. Writes split by appending a suffix, item#1 through item#N, at the cost of reading all N and merging the results. Reads spread by caching or replicating that one key on many nodes.

If each shard is slow 1% of the time, a request that waits for 100 shards meets at least one slow shard about 63% of the time, since 0.99^100 is about 0.37. Hedging, sending a backup request when the first runs slow, trims the tail, and returning partial results after a deadline caps it. Touching fewer shards per request helps most.

A safe split copies existing rows to the new shards while streaming ongoing changes to them, verifies that source and target match, moves reads first and writes second, and keeps the old shards until the move is proven. The write switch still needs a brief pause or careful ordering so no write lands on an old shard afterward. A big-bang rehash moves everything at once and stops service while it runs.

A local index sits beside its shard's rows, so a write updates it in the same place, but a query by the indexed field must ask every shard unless it also names the partition key. A global index is partitioned by the indexed field, so such a query goes to one place, but a write may touch another node, and many systems update it asynchronously, so index reads can lag.

Inside one node, a commit is atomic for free. Across shards you need two-phase commit, which leaves participants that voted yes blocked if the coordinator fails, or a saga of local steps with compensating actions that undo earlier ones, which lets others see intermediate states. The cheapest fix is a shard key that keeps data changed together on one shard.

Because one node orders every write, there are no conflicting writes to resolve and followers apply the same log in the same order. The costs: all write throughput must fit on one machine, followers lag, and the hard part moves to failover, which means detecting a dead leader, picking a follower, and making sure the old leader cannot keep accepting writes.

With synchronous replication a commit waits for a follower's confirmation, so a slow or dead follower can stall commits indefinitely unless another standby takes its place. With asynchronous replication commits return at once, and a leader crash loses whatever had not shipped, an amount that grows with the lag. A common compromise makes one follower synchronous and the rest asynchronous.

Write latency is set by the k-th fastest follower, so one slow follower cannot hold up commits. Once k followers confirm, the write exists on k + 1 nodes counting the leader, so it survives losing any k of them. After the leader fails, the new leader must be a follower that confirmed, which means checking at least n − k + 1 followers to be sure of finding one.

A follower applies the leader's changes after the fact, so its view is always somewhat old, and the delay grows when writes surge, when the follower is busy with heavy queries, or after a network blip. Watch lag in both time and bytes. Any read sent to a follower can be stale by the current lag, which breaks flows like 'save, then reload the page'.

Two common approaches: send a user's reads to the leader for a short window after they write, or have the client remember the log position of its last write and read only from replicas that have applied that far, otherwise waiting or falling back to the leader. It is a per-user promise, so others may still see the old value, and it gets harder when one user writes from two devices.

If one replica is current and another lags, a user who refreshes twice can watch a new comment appear and then vanish. Pinning each user to one replica, for example by hashing the user ID, prevents it while that replica stays healthy; carrying the version of the last read and skipping older replicas also survives failover. It promises no freshness, only no going backward.

The old leader acknowledged writes that never reached the follower now in charge. When the old leader returns, its extra writes conflict with ones the new leader has accepted since, and they are usually thrown away. Worse, if IDs came from a counter, the new leader may reissue IDs the lost writes used, so caches or other systems holding those IDs now point at the wrong rows.

A leader paused by garbage collection or a network stall can wake after its lease expired and a successor took over, still believing it leads. Each lease grant carries a higher number; the storage being written remembers the highest number it has seen and rejects anything lower, so the stale leader's writes bounce. Fencing works only if the resource itself checks the token.

Two leaders can each accept an edit to the same record before hearing about the other. Last-writer-wins keeps one by timestamp and silently drops the other, and clock skew can make it drop the newer one. Version vectors detect true concurrency, and resolving it needs a merge rule: application logic, a data type built to merge, or asking the user.

Read repair fixes a stale replica when a read notices the copies disagree, so data nobody reads can stay wrong indefinitely. Background repair compares replicas by exchanging trees of hashes and streams only the ranges that differ. It must run more often than deletion markers are purged, or a replica that missed a delete can bring the deleted data back.

When a replica is down, a peer stores the write as a hint and replays it once the replica returns, so writes keep succeeding. Hints are best-effort: they are kept only for a limited window, and if the replica stays down longer, or the node holding the hints is lost, only a full repair brings the replica back in sync.

The real-time part is what makes it strong: once a write returns, every read that starts afterward must see it or something newer, as if only one copy existed. Reading from an asynchronous replica breaks it. It applies to single objects, and it cannot stay available on both sides of a network partition, so it costs coordination on every operation.

A serializable database promises only that transactions match some one-at-a-time order, not the order they actually happened in, so a read that starts after a write finished may still miss it, and a client can fail to see its own earlier transaction. Linearizability adds real-time order but covers single operations on one object. Neither implies the other.

Transactions behave as if run one at a time, in an order that respects real time: if one commits before another starts, it comes first. You can picture the whole database as a single linearizable object. Every commit must be coordinated across replicas, often across regions, and the model cannot stay available during a partition, so latency is the price.

A reply never shows up before the message it answers, because the system tracks which writes each write had seen. Two unrelated posts may appear in different orders to different readers. It can keep working during a partition as long as each client keeps talking to the same server, which linearizability cannot do.

It is a liveness promise only: given no new writes, replicas will eventually agree. It says nothing about what you read meanwhile, so a value can appear to go backward, and it does not say which concurrent write wins. Systems relying on last-writer-wins can converge by quietly discarding writes, so check the merge rule, not just the label.

The first two cover reads; the other two cover writes. Monotonic writes means two writes from one client are seen by everyone in the order the client made them. Writes-follow-reads means that if you read a value and then write, your write is ordered after the one you read. Three of the four survive a network partition; read-your-writes needs each client to stay with one server.

The C in CAP means every read sees the most recent write, which is linearizability, not the consistency of ACID. Partitions are not optional in a real network, so the real choice is what happens during one: refuse some requests to stay consistent, or answer from whatever data each side holds. When there is no partition, a system can offer both.

With no partition, a strongly consistent write must reach other replicas before returning, so every request pays at least one replica round trip, which across regions means tens of milliseconds. PA/EL systems take lower latency and accept staleness; PC/EC systems pay the latency always. Many stores let you choose per request, such as an eventually consistent read that skips the wait.

With N = 3, W = 2 and R = 2, a write that reaches only one replica before failing is reported as an error yet stays there. A reader whose quorum includes that replica sees the new value; a later reader hitting the other two sees the old one, so time appears to run backward. Having the reader write the newest value back to a quorum before replying closes that gap.

Each replica applies updates locally and merges others' states with a function that ignores order, grouping and duplicates, so replicas that have seen the same updates hold the same value. A grow-only counter keeps one slot per replica, merges by taking each slot's maximum and reports the sum. The limits: metadata can grow, and an invariant such as 'balance never below zero' still needs coordination.

Applied to a replicated log, it lets several servers behave as one reliable state machine: each applies the same commands in the same order. Safety holds through delays, lost or duplicated messages and partitions, while progress needs a majority up and talking. It assumes nodes fail by stopping, so it cannot protect against a bug or bad input that every replica executes faithfully.

Two groups that each hold more than half the nodes must share a member, so a later leader or read always meets a node that saw every committed write. Five nodes need three for a quorum and survive two failures; six need four and still survive only two, so an even size adds cost without tolerance. Three tolerates one, which a planned restart can already use up.

A term is a counter, not a time: each election starts a new term, and some terms end with no leader after a split vote. Servers exchange terms on every message; one that sees a higher term adopts it and, if it was leader or candidate, steps down at once. That is how a deposed leader returning from a partition learns it is out of date.

The candidate increments its term, votes for itself and asks the others; each server grants at most one vote per term, first come first served, so two leaders in one term cannot happen. Voters also refuse a candidate whose log is less up to date, comparing the last entry's term and then log length, so only a server holding every committed entry can win.

If every follower timed out together, each would vote for itself and none would reach a majority, again and again. Timers drawn from a range, 150 to 300 ms in the Raft paper's example, let one server usually time out first and win before the others wake. The timeout must sit well above the time to reach all peers and well below the time between crashes.

Once a leader has stored an entry from its own term on a majority, that entry and everything before it are committed. A new leader cannot commit an older term's entry by counting replicas, since it can still be overwritten; it commits by replicating a fresh entry from its own term, which carries the older ones along. Followers learn the commit index from the leader's messages.

Each append carries the index and term of the entry just before the new ones, and a follower refuses it unless its own log holds that same entry. On refusal the leader backs up and retries until the logs agree, then the follower drops everything after that point and takes the leader's entries. Only uncommitted entries are ever discarded, because a leader already holds every committed one.

Answering a read straight from a leader is unsafe if a newer leader has been elected without its knowledge. The safe default confirms leadership by exchanging heartbeats with a majority before answering. A lease skips that round by assuming no successor can be elected until the lease ends, which holds only if clock drift and process pauses stay bounded.

The result concerns progress, not safety: a deterministic protocol can be kept from ever deciding, because it cannot tell a crashed node from a slow one. Practical systems keep safety unconditional and get progress from timeouts and randomized retries, so they decide whenever the network stays calm long enough. During long instability they may stall, but they never disagree.

A lying node can tell different peers different things, so every quorum needs enough honest members to outvote liars. With 3f + 1 nodes and quorums of 2f + 1, any two quorums share at least f + 1 nodes, so at least one honest node sits in both. Raft and Paxos assume nodes only stop, which is why they get by with 2f + 1.

On Linux, CLOCK_REALTIME jumps whenever an admin or a time daemon steps the clock, which daemons such as ntpd do for a large offset instead of slewing it gradually, so end - start measured with it can even come out negative. CLOCK_MONOTONIC never jumps, though NTP may still adjust its rate, and it pauses during suspend. Its values mean nothing on another machine or after a reboot, so use it only for durations on one host.

The NTP project's FAQ puts typical accuracy over the public internet at 5 to 100 ms, and about a millisecond on a quiet LAN. Two events on different machines closer together than the current skew can therefore get timestamps in the wrong order. Skew drifts between syncs and widens under network congestion, so treat any timestamp as an interval of uncertainty rather than an exact instant.

Lamport built the relation from three facts: events inside one process are ordered, a send precedes its receipt, and the relation is transitive. Two events where neither can reach the other are concurrent, and no clock can honestly rank them. The trap is causality that travels outside the system: if a user phones a friend who then acts, the system never saw that link and may order the second action first.

Each process bumps its counter between events and stamps every outgoing message; a receiver moves its counter past any stamp it sees. That guarantees one direction only: if a caused b, then C(a) < C(b). The converse fails, so C(a) < C(b) proves nothing about causation. Breaking ties by process id yields a total order that respects causality but ranks concurrent events arbitrarily.

Each node increments its own slot on every event, ships its whole vector with each message, and on receipt takes the element-wise maximum. Event x precedes y exactly when every slot of x is at most y's and at least one is smaller; if neither vector dominates, the writes were concurrent and something must reconcile them. The vector gains a slot for every node that ever writes the key.

Under last-write-wins, two concurrent writes are both acknowledged, then only the one with the higher timestamp survives and the other vanishes with no error. Skewed clocks make it worse: a write that causally followed another can carry an earlier timestamp and lose to it. LWW is safe only where losing a concurrent update is acceptable; otherwise keep both versions or use a mergeable data type.

An HLC timestamp pairs a physical part l with a small counter c. On each event, l becomes the largest of the local clock, the previous l and any received l, and c breaks ties. Kulkarni and colleagues proved l never falls behind physical time and stays within the clock-skew bound ahead of it, and it fits a 64-bit NTP-format field. Like a Lamport clock, it still cannot detect concurrency.

Spanner reads time as an interval [earliest, latest] with an error bound, epsilon. A leader picks a commit timestamp no lower than TT.now().latest, then keeps the write invisible until that timestamp has surely passed, an expected wait of at least twice the average epsilon. The 2012 paper reports epsilon of roughly 1 to 7 ms; a looser clock means a slower commit, though the wait overlaps replication.

Twitter's original Snowflake packed 41 bits of milliseconds since a custom epoch, a 10-bit machine id and a 12-bit per-millisecond sequence. Ids sort only roughly, since each machine's clock differs, and Twitter promised order only within about a second. If the clock steps back, a generator can mint duplicates or out-of-order ids, so Snowflake refuses to issue any until time passes its last stamp.

Read committed still allows non-repeatable reads: the same row read twice can change. Repeatable read, as the SQL standard defines it, still allows phantoms, new rows matching an earlier predicate. Snapshot isolation never shows phantom reads but allows write skew, so it and standard repeatable read are incomparable. Serializable allows none. In PostgreSQL 18, repeatable read is implemented as snapshot isolation.

Snapshot isolation aborts a transaction only when another has written the same row, so disjoint writes both commit. Berenson and colleagues' 1995 example: a rule caps a set of tasks at 8 hours, each transaction sees 7 and inserts its own 1-hour task, and the total reaches 9. Row locks cannot help, because the conflicting rows did not exist yet; use serializable isolation and retry, or lock one shared row.

In the prepare round each participant does its work, forces a prepare record to stable storage and votes; after voting yes it can no longer abort on its own. If every vote is yes, the coordinator logs commit and tells everyone; a single no, or a timeout before the decision, means abort. The result is atomic, all commit or none do, but it says nothing about isolation from other transactions.

A participant that voted yes is in doubt: it can neither commit nor abort until it hears the decision, so its locks stay held. PostgreSQL's PREPARE TRANSACTION docs warn that a transaction left prepared keeps its locks and also stops VACUUM from reclaiming space. If the coordinator dies for good, those participants wait until an operator resolves them by hand, and every request needing those rows queues behind them.

Each step commits locally and immediately, so a saga never holds locks across services. If step three fails, compensations for steps two and one run in reverse order. Because steps and compensations can be invoked again after a crash, each must be idempotent. The price is isolation: other requests see half-finished sagas, which AWS guidance counters with semantic locks such as a pending status on the record.

A database rollback erases changes nobody else saw. A saga's earlier steps were committed and visible, so a confirmation email may already be sent and the reserved stock resold before the failure. A compensation is a new forward transaction, such as issuing a refund rather than deleting the charge, and some effects cannot be undone at all, which is why steps that cannot be compensated belong as late in the saga as possible.

With an orchestrator the workflow lives in one place, so timeouts, retries and progress tracking are centralized, but that coordinator must itself be highly available. With choreography each service reacts to events and nothing central can fail, yet the flow is implicit and cycles between services can creep in. AWS guidance says choreography suits sagas with few participants; as steps multiply, orchestration is easier to follow.

If the service commits to the database and then crashes before publishing, downstream systems never hear about the change. Publish first and then fail the commit, and consumers act on a change that never happened. Retrying the publish cannot close the gap, because the crash can land between any two steps. The fix is to make one write the source of truth and derive the other from it.

Because the outbox row commits atomically with the business change, the event exists exactly when the change does. A relay then reads committed rows, publishes them, and deletes or marks them; if it crashes after publishing but before deleting, the event goes out again. Delivery is therefore at-least-once and consumers must deduplicate. A per-aggregate sequence number in each row preserves commit order.

A connector such as Debezium 3.7 reads the database's own log, the MySQL binlog or a PostgreSQL logical replication stream, so it sees only committed changes and needs no second write path. Debezium's FAQ says consumers see every event at least once and may get duplicates after a crash. The stream also exposes your table layout to every consumer, so renaming a column becomes a breaking change for them.

A queue helps only while the average arrival rate stays below what consumers can process; it buffers peaks, it never adds capacity. A backlog drains at the difference between the two rates, so 100,000 queued messages with 50 per second of spare capacity take over half an hour to clear. Every message ages meanwhile, so alert on the age of the oldest message, not only on queue length.

With a queue, adding consumers splits the work; with a topic, adding a subscriber adds another full copy of the stream. A common hybrid is a topic that fans out into one queue per subscribing service, giving each service its own copy and its own pace. Kafka builds both into consumer groups: within a group each partition goes to exactly one consumer, while every separate group receives all records.

The choice comes down to when the consumer records progress. Commit the offset or acknowledge before processing, and a crash mid-work loses that message: at-most-once. Process first and commit afterward, and a crash between the two replays it: at-least-once. Because a crash can fall at any instant, one of those gaps always exists; systems advertising exactly-once add deduplication on top.

Give every message a stable id and record the ids you have applied in the same database transaction as the effect, skipping any id already present. Kafka's idempotent producer does the same inside the log, deduplicating resends by producer id and sequence number. Kafka's docs add that exactly-once into an outside system needs that system's cooperation, typically storing the consumer offset alongside the output.

In Amazon SQS a received message is hidden, not removed, until the consumer deletes it. If processing outlasts the visibility timeout, the message reappears and a second consumer starts on it while the first is still working. Set the timeout above your slowest normal processing time, extend it with ChangeMessageVisibility as a heartbeat, and still deduplicate, since standard queues deliver at least once.

Kafka guarantees order only within a partition, and records with the same key land in the same partition, so per-key order comes free. Each partition is read by one consumer per group, which caps a single key's throughput at what one consumer can handle. Adding partitions later changes the key-to-partition mapping for new records, so one key's events can split across partitions and lose relative order.

Under Kafka's classic eager protocol, every member revokes all its partitions before a rebalance, so the whole group stops. Incremental cooperative rebalancing, shipped in Kafka 2.4, revokes only the partitions that move, and the consumer protocol made generally available in Kafka 4.0 drops the group-wide barrier entirely. Either way, a consumer that misses max.poll.interval.ms is ejected and its partitions reassigned.

A log keeps records for a configured retention period whether or not anyone reads them, and a consumer group's progress is a single offset per partition, so replay just means moving that offset back. A queue tracks each message separately and deletes it on acknowledgment, which suits handing out work but leaves no history to reread. Kafka's log compaction sits between them, keeping at least the latest record per key.

In an unordered queue a poison message does not block others; it keeps reappearing and burning receives until it expires. In an ordered stream, a Kafka partition or an SQS FIFO message group, it halts everything behind it. SQS moves a message aside after maxReceiveCount receives, and AWS warns this breaks strict order in FIFO queues. Alert on any DLQ arrival and give the DLQ longer retention than its source.

In an SQS FIFO queue, while one message from a message group is in flight, later messages in that group stay unavailable until it is deleted or its visibility timeout lapses. The cure is narrower ordering keys: order only what truly depends on each other, such as one customer's events, so a stuck message delays its own key instead of a whole shard. Per-message timeouts bound how long the wait can last.

A thin notification carries an id, so every consumer needing details calls the producer back, coupling them at runtime and multiplying load on each change. A fat event lets consumers keep their own copy and keep working while the producer is down, at the cost of duplicated data and copies that lag. Choose by whether consumers can tolerate staleness, and include a version so they can drop out-of-order updates.

A command goes to one handler that checks it against current state and may refuse it; it reads as an imperative, like BookRide. An event is broadcast after the fact, reads in the past tense, like RideBooked, and subscribers must cope with it rather than veto it. Publishing an event while quietly expecting one specific reaction is a disguised command, which Martin Fowler calls passive-aggressive.

Writes go to a model built to enforce invariants, while reads come from separately stored views, each denormalized for one screen or query, so the two sides can scale and change independently. The cost is a second model plus the sync between them, which is usually asynchronous. Martin Fowler warns that CQRS adds risky complexity and should be applied to specific bounded contexts, never to a whole system by default.

A user who saves and immediately reloads may read a projection that has not yet processed the event, and the change looks lost. Two common remedies: return a version number from the write and have the read wait until the view reaches it, or render the change at once, marked as pending. Track projection lag as a metric with an alert, because a stalled projector looks exactly like data loss to users.

To get current state, replay an entity's events through a deterministic apply function, oldest first. Concurrent writers are handled with optimistic concurrency: each append names the version it expects, and a mismatch is rejected rather than silently interleaved. You gain a full audit trail and state as of any past moment; you pay with harder queries, which is why event sourcing usually pairs with CQRS.

A snapshot stores the folded state together with the position of the last event it includes; loading then means reading the snapshot and replaying only later events. A snapshot is a cache, never the source of truth: if the fold logic changes or had a bug, old snapshots hold the old result and must be rebuilt from events. Set snapshot frequency from how long a cold load is allowed to take.

Stored events get replayed for years, so every reader must understand every version ever written. Add fields with safe defaults, never change an existing field's type or meaning, and carry a version number so replay code can convert old shapes. Correct a wrong event by appending a compensating one, not by editing history; an edit silently diverges from every projection and audit that already consumed the original.

Because a read model is derived, you can build a new one, or repair a buggy one, by replaying the log into a fresh store and switching reads over once it catches up. Replay must be side-effect free: Martin Fowler's advice is to route external calls through gateways that can be switched off during replay, or old emails and charges go out again. On a large log a rebuild can be slow, so plan it like a migration.

With choreography, the end-to-end process exists only as the sum of each service's subscriptions; no single file or diagram in the code states it. Answering 'where is order 123 stuck?' means correlating logs across services, so propagate a correlation id through every event and use distributed tracing. AWS guidance notes that end-to-end monitoring is harder with choreography than with orchestration.

Because standard methods have published meanings, a cache can key on method plus URI and serve repeat GETs, and RFC 9111 makes a cache invalidate a URI after a successful unsafe request to it. An RPC-style API that tunnels every action through POST to one endpoint gives intermediaries nothing to recognize, so caching, safe retries and per-resource access logs all have to be rebuilt by hand.

Idempotent describes the intended effect on the server, not the response: a second DELETE may return 404 where the first returned 204, yet the end state matches. RFC 9110 lets a client repeat such a request after a dropped connection, but forbids proxies from automatically retrying non-idempotent ones like POST. A PUT handler that appends or increments breaks the contract and makes those retries unsafe.

RFC 9110 calls GET, HEAD, OPTIONS and TRACE safe: the client asks for no state change. Crawlers, link checkers and browser prefetchers follow GET links freely on that promise, so a link like ?do=delete gets triggered by bots. Logging is an acceptable side effect because the client never requested it; any change the user is asking for and accountable for belongs behind POST, PUT or DELETE.

Per RFC 9110, a 4xx means the client seems to have erred, so resending the identical request usually fails the same way. A few invite a retry: 408, 429 (RFC 6585) and 425 Too Early (RFC 8470), ideally after any Retry-After delay. A 5xx means the server failed or is overloaded: 503 is explicitly temporary, while 501 is not worth retrying. After a 5xx on a non-idempotent call, the work may or may not have happened.

With OFFSET 10000 the database still computes and discards the first 10,000 rows, as PostgreSQL's docs warn, and an insert or delete between page loads shifts every later page by a row. A cursor encodes the last sort key seen and asks for rows after it, served from an index; the key must be unique, so add the id as a tiebreaker. The trade-off is that a cursor cannot jump straight to page 50.

Martin Fowler calls this parallel change: expand by adding the new field or endpoint beside the old one, migrate every client, then contract by removing the old. Repurposing a field breaks any consumer still reading the old meaning, which is why protobuf's guidance says never reuse a field number and reserve deleted ones. The usual failure is skipping the contract step and carrying both shapes forever.

gRPC defines services in protobuf, generates client stubs, supports unary plus client, server and bidirectional streaming, and builds in deadlines. It runs on HTTP/2 framing that browsers never expose to scripts, so browser clients need gRPC-Web through a translating proxy, which carries only unary and server-streaming calls. Binary protobuf payloads also mean you need gRPC-aware tools to read traffic or try a call by hand.

A single query can nest relations and alias the same field many times, so its cost is invisible until analyzed. The GraphQL project's security guide recommends depth and alias limits, cost estimates charged against each client's budget, and, for first-party apps, an allowlist of trusted documents. Queries usually arrive as POST to one /graphql endpoint, so URL-keyed HTTP caches help only if queries are sent by GET.

Once another service reads your tables directly, your schema is its API: you cannot rename a column, split a table or change storage engines without a coordinated release. Expose data through an API or published events instead, and enforce it with a database user per service whose grants cover only its own schema. A consumer that needs joined data can keep its own read model built from your events.

Doing edge work once removes slightly different copies of authentication and throttling scattered through every service, and gives clients one endpoint; AWS API Gateway, for example, throttles with a token bucket and answers 429. The cost is an extra network hop and a component every request depends on, so it needs real redundancy. Keep business logic out, and have services still authorize each action themselves.

A B-tree finds a key by walking one root-to-leaf path and rewrites that page in place, so random inserts touch random pages; WiredTiger's docs note insert speed drops sharply once the tree outgrows cache. An LSM tree buffers writes in memory, flushes sorted immutable files sequentially and merges them later, so writes stay fast, but a read may check several files and merges compete for disk.

Write amplification is bytes written to storage divided by bytes the application wrote. RocksDB's tuning guide works out about 33x for leveled compaction where each level is ten times the previous, because data is rewritten each time it merges down a level. On SSDs that rewriting wears flash and competes with foreground I/O; universal compaction writes less but uses more space and makes reads costlier.

Read amplification is disk reads per query. A RocksDB point lookup checks every level-0 file, whose key ranges overlap, plus at most one file per deeper level, but a per-file Bloom filter answers 'definitely not here' from memory and skips most of those reads. Whole-key filters do nothing for range scans and prefix filters help only within one prefix, so a scan still pays about one seek per level-0 file plus one per non-empty deeper level.

Insert by setting k hashed bit positions; a lookup that finds any of them zero proves absence, while all ones only means maybe. With the optimal number of hashes, a 1% false-positive rate costs about 9.6 bits per element, and each tenfold improvement adds roughly 4.8 more. A standard Bloom filter cannot delete, since clearing a bit may erase other members, and overfilling it drives the error rate up.

PostgreSQL's rule is that a data page may reach disk only after the log records describing its change have been flushed. A commit then needs one sequential log flush instead of random writes to every touched page, and a single flush can cover many concurrent commits. After a crash, recovery replays the log forward to redo changes the pages missed, so losing the log loses committed work.

When write() returns, the kernel holds the bytes, so they survive your process dying but not a power cut or kernel crash. fsync() pushes them to the device, and a newly created file also needs an fsync on its directory or the name can vanish. Even then, PostgreSQL's docs warn that drives with volatile write-back caches may acknowledge early, so durability is only as honest as the hardware.

In MySQL 8.4's InnoDB the primary key is the clustered index and holds the full row, while each secondary index entry stores the row's primary key, so a secondary lookup searches two B-trees and a long primary key bloats every secondary index. PostgreSQL 18 has no clustered index: every index points into a separate heap, and an index-only scan skips the heap only for pages marked all-visible.

A row store keeps each record's fields together, so fetching or updating one order touches one page. A column store keeps each column together, so summing one field across a billion rows reads only that column, and neighboring values of one type compress far better, especially once sorted. The cost is the reverse case: assembling or changing one full row touches every column's storage.

Indexing splits text into terms and stores, for each term, a postings list of the documents containing it, with frequencies and positions, as Lucene 10.3's file format documents. A query intersects postings lists and then scores the matches; Elasticsearch scores with BM25 unless told otherwise. Tokenization decides what can match: 'running' finds 'run' only if the analyzer stems both to the same term.

Feed the index from the primary database, ideally through change data capture, and treat it as rebuildable at any time. Even after an indexing call succeeds, Elasticsearch makes a document searchable only at the next periodic refresh, so a just-saved item can briefly be missing from results. Never let the search copy decide correctness: confirm ownership, price or stock against the primary.

A request-count limit treats a cheap lookup and an expensive report as equal, so it either throttles harmless traffic or lets a few costly calls saturate the database. Google's SRE book argues for measuring capacity in the actual resource, such as CPU, rather than queries per second. Stripe runs a concurrent-request limiter for the same reason, capping in-flight work on CPU-heavy endpoints.

The refill rate sets the long-run average and the bucket size sets the largest burst a client can send after a quiet spell. A request that finds too few tokens is rejected or delayed, and tokens arriving at a full bucket are discarded, so idle time cannot be banked beyond the cap. AWS API Gateway configures throttling exactly this way, as a rate and a burst, and returns 429 once both are exceeded.

In its queue form, requests enter a fixed-size buffer and leave at a constant rate, so downstream sees perfectly smooth traffic, every burst turns into queueing delay, and arrivals that find the buffer full are dropped. The meter form, which tracks a level instead of holding requests, makes the same accept-or-reject decisions as a token bucket with matching parameters. Pick the queue form when smoothness beats latency.

A counter per client per clock window, reset at each boundary, costs one atomic increment. With a limit of 100 per minute, a client can send 100 requests in the last second of one window and 100 more in the first second of the next: 200 in two seconds. Windows aligned to the clock also reset for every client at the same instant, which lines up traffic spikes.

The approximation keeps just two counters per client: the previous window's count, weighted by how much of that window still overlaps the sliding one, plus the current count. It assumes the previous window's requests were evenly spread. Cloudflare measured it on 400 million requests and found 0.003% wrongly allowed or limited, while a timestamp log needs one entry for every request.

Splitting a global limit of 1,000 per second across 10 nodes gives each 100, but autoscaling to 20 nodes silently doubles the effective limit, and uneven load balancing lets a client exceed it on one node. A central counter, as Stripe runs in Redis, is exact but adds a network call per request and a dependency that should fail open. Local counters synced periodically trade a small overshoot for no per-request hop.

Past capacity, a server that accepts everything slows down for everyone until responses outlast the clients' timeouts and useful throughput collapses. Rejecting excess early, with a cheap error, keeps latency normal for whatever is admitted. Google's SRE book describes per-task rejection once utilization crosses a threshold, and dropping queued requests whose caller has already given up, since finishing them serves nobody.

Google's SRE book describes four levels, CRITICAL_PLUS, CRITICAL, SHEDDABLE_PLUS and SHEDDABLE, where production traffic defaults to CRITICAL and batch work to SHEDDABLE_PLUS. Higher classes get higher rejection thresholds, so batch is turned away first. The label is set close to the user and inherited by every downstream call, or a backend could not tell a user's request from a nightly job.

Backpressure lets a slow consumer cap how much it is sent, by granting credits, blocking the producer, or rejecting with an overload status. Without it, a faster producer forces someone to buffer without limit; the Reactive Streams specification exists to keep the buffers between asynchronous stages bounded. The signal must reach the true source, ultimately the user, or the unbounded queue has merely moved upstream.

Google's SRE book shows the cost: with a queue ten times the thread count and 100 ms of work per request, a request at the back waits about 1.1 seconds, nearly all of it idle. It suggests queues no larger than about half the thread pool for steady traffic, so servers reject early. A full bounded queue returns an error the caller can act on; an unbounded one grows until memory runs out.

Some clients wait forever by default; the docs for Python's requests 2.34 state that a call never times out unless you pass timeout. Choose the value from the dependency's latency distribution: AWS suggests picking an acceptable false-timeout rate, such as 0.1%, and using that percentile, p99.9, plus padding. Set it too short and a slight slowdown becomes a wave of failures and retries.

Instead of each hop applying its own fixed timeout, the edge sets one deadline and every call carries what remains: in the SRE book's example, a 30-second deadline with 7 seconds spent leaves 23 for the next RPC. Each stage checks the remaining budget before starting work and cancels child calls once it expires. Reserve a margin for network and queueing, or replies land just after the caller quit.

With up to three attempts per call and half of all attempts failing, a client sends 1 + 0.5 + 0.25 = 1.75 times its normal traffic. Retries stacked in layers multiply: in the SRE book's example, three layers each making 4 attempts produce up to 64 database attempts for one user action. Retry at one layer only, and have an overloaded server return an error that tells callers not to retry.

Each failed attempt multiplies the wait, typically doubling up to a cap, so the retry rate falls quickly while an outage lasts. A fixed interval keeps load stacking up: in the SRE book's example, a backend just over its limit sees retries every second push the excess from 100 to 200 to 300 QPS. A cap without an attempt limit leaves clients retrying at the cap indefinitely, so bound the attempts too.

Backoff alone still leaves clients that failed together retrying together, because identical schedules stay aligned. AWS's simulations of clients contending for one row found that full jitter, which randomizes the entire wait, used the least total client work. Jitter belongs on any synchronized timer too, such as jobs that all fire at midnight; for scheduled work, AWS prefers a fixed per-host offset that stays predictable.

Google's SRE book caps retries per request at three attempts and per client at about 10% of requests. In its analysis, the per-request cap alone lets traffic toward an overloaded datacenter climb to just under 3x, while the 10% budget holds growth near 1.1x. AWS SDKs keep a similar local token bucket in standard retry mode: each retry spends tokens, successes refill them, and once it is empty the SDK returns the error without retrying.

The client generates a unique key, Stripe suggests a V4 UUID, and resends it with every retry of that request. Stripe's v1 API saves the status and body of the first attempt once execution begins and replays them, even a 500, and rejects a reused key with different parameters. Store the key in the same transaction as the side effect, or a crash between the two still duplicates work.

Closed, the breaker passes calls and counts failures; past a threshold it opens and fails calls instantly without contacting the dependency. After a reset timeout it goes half-open and lets a trial call through: success closes it, failure reopens it. Count timeouts as failures, decide per call what the fallback is, and log every state change, since a breaker that trips often points to a deeper problem.

If every outbound call shares one thread or connection pool, a dependency that starts hanging fills the pool and the caller can reach nothing at all. A separate, capped pool per dependency confines the damage to calls that needed the broken service. Microsoft's pattern guide notes the cost: partitioned capacity sits idle while a neighbor is starved, so size each pool from that dependency's concurrency.

The AWS Reliability Pillar frames it as turning hard dependencies into soft ones: if recommendations fail, render the page without them rather than an error. Decide in advance which feature is core, test the degraded path regularly, and keep it far simpler than the main one. Never let a failed refresh wipe the last good cache or config, which converts a dependency blip into an outage.

Google's SRE book walks through it: cluster A serves 1,000 QPS, cluster B fails, A receives 1,200 and starts failing, and its load spills further. Health checks often spread it: overloaded servers fail them, get pulled, and leave even fewer servers to absorb the load. Keep headroom for the loss of a failure domain, and make servers reject excess load instead of slowing until checks fail.

Typical triggers are a popular cache entry expiring, a fleet reconnecting after a network blip, and clients retrying on identical schedules. Google's SRE workbook describes Pokemon GO clients whose immediate retry plus constant backoff synchronized into spikes 20 times the previous global peak. The cures are jittered timers, letting one request refill an expired entry while others wait, and staggered reconnects.

Bronson and colleagues describe a web app at 280 QPS against a database that handles 300: a 10-second network outage releases a surge, latency climbs, every query times out and is retried, and load sits at 560 QPS with zero useful work. Because the feedback loop is the real cause, ending the trigger fixes nothing, and the system may have run safely for months in that vulnerable state first.

Bronson and colleagues found that leaving a metastable state takes a strong push, such as a reboot or a deep cut in load. In their retry example, load had to fall below 150 QPS or retries below 20 QPS. Servers restarted into the same retry storm can fall straight back in, and cold caches add load. The SRE book's last resort is to drop to a sliver of traffic, then ramp back up.

Averages hide it: a fleet averaging 35% can contain one shard pinned at 100% and failing, because a single key, such as a celebrity account or today's date, draws a large share of traffic. Graph the busiest shard, not the fleet mean. DynamoDB's guidance shows the write-side fix: append a suffix to spread one hot key, at the price of reading every suffix and merging the results.

Huang and colleagues at Microsoft define it by differential observability: the failure detector sees a healthy component while the applications using it see it failing, through slowness, packet loss or flaky I/O. A liveness probe that returns 200 from a process whose disk has died is a typical case. Detect it from the caller's side, with error rate and latency as clients measure them, not self-reported health.

Redundancy helps only if replicas fail independently, and shared inputs break that: one bad config pushed everywhere, certificates issued together that expire together, one software bug triggered on every replica by the same input, a shared DNS or auth service. Google's SRE workbook concludes that no load shedding or autoscaling saves services that all fail in sync, so stagger rollouts and expiry dates by failure domain.

Google's SRE book describes the loop for garbage-collected runtimes: less spare CPU slows requests, so more are in flight holding memory, which triggers more collection, which takes even more CPU. Adding instances may not break it, since each new one meets the same load. A restart clears the spiral, but a restarted server facing the same load re-enters it, so cap in-flight requests and queue length.

Each cell is a complete, independent copy of the workload that shares no state with other cells, and a thin router sends each request to its cell by a partition key such as customer id. In AWS's guidance, with 10 cells a failure in one leaves 90% of requests unaffected. Roll changes out cell by cell. The router is the remaining shared component, so keep it as simple as possible.

Each tenant gets a random small combination of nodes rather than a fixed shard. In AWS's example, 8 workers in fixed pairs make 4 shards, so one bad tenant hurts a quarter of customers; random pairs give 28 combinations, affecting about 1 in 28. Route 53 gives each domain 4 of 2,048 virtual name servers, about 730 billion combinations. It works only if clients retry across their shard's nodes.

Write it as good events over valid events, such as successful requests over well-formed ones, so every SLI runs from 0% to 100% and reads the same way. Excluding malformed client requests keeps one broken caller from burning your budget. Measure as close to the user as you can, at the load balancer or in the client, since server-side metrics miss failures that never reached the server.

'99.9% available' means nothing until you say over what period and how it is counted: per request or per minute, at the server or the load balancer, over a rolling window or a calendar month. Google's SRE workbook recommends a rolling four-week window as a general default, since it tracks what users actually experienced, with calendar periods kept for planning and reporting.

The test from Google's SRE book: if missing the target triggers an explicit consequence, such as a refund, it is an SLA; if not, it is an SLO. Keep the internal SLO tighter so you see and fix trouble before it costs money. Over-delivering has a cost too: users come to depend on the higher level, which is why Google deliberately takes global Chubby down when it runs far above target.

With a 99.9% SLO over four weeks and 3 million requests, the budget is 3,000 failed requests, and one outage failing 1,500 spends half of it. The SRE workbook pairs the budget with a policy agreed in advance: while budget remains, ship; once it is spent, freeze changes and work on reliability until the service is back within SLO. Release speed versus reliability becomes arithmetic.

From the SRE book's availability table, 99.9% allows 43.2 minutes in a 30-day month, 99.99% allows 4.32 minutes and 99.999% about 26 seconds, so each nine cuts the allowance tenfold. Cost rises faster than that: the book warns one increment can cost 100 times the previous one, while a user on a 99%-reliable phone cannot tell 99.99% from 99.999%.

The SRE book's example: typical requests take about 50 ms while 5% take 20 times longer, which the mean blurs into a mildly raised number. Track p50 for the typical case and p99 or p99.9 for the worst plausible one. Percentiles cannot be averaged across servers or time windows; merge the underlying histograms and recompute, or the result means nothing.

If each backend is slow on 1% of requests, a request waiting on 100 of them is slow with probability 1 - 0.99^100, about 63%, as Dean and Barroso's 'The Tail at Scale' computes. In one Google service they measured, waiting for the slowest 5% of leaf responses accounted for half the 99th-percentile latency. Fan-out turns a rare per-server delay into the common user experience.

Send to the best replica, and only if no answer arrives by the 95th-percentile expected latency, send a copy elsewhere, take whichever replies first and cancel the other. Dean and Barroso report this keeps extra load near 5%; in a Google BigTable benchmark, hedging after 10 ms cut 99.9th-percentile latency from 1,800 ms to 74 ms for 2% more requests. Hedge only requests that are safe to run twice.

Metrics are cheap numeric aggregates you alert on, but aggregation discards which request failed. Logs record individual events in full detail, at a cost that grows with traffic. Traces stitch one request's spans across services using a propagated trace id and parent span ids. Put the trace id in every log line, so an alert from a metric leads to the exact trace and then its logs.

Burn rate is how fast you consume error budget relative to spending it evenly across the window. Google's SRE workbook suggests paging at 14.4x over one hour, which is 2% of a 30-day budget, or 6x over six hours, and filing a ticket at 1x over three days. Pair each long window with a short one about 1/12 its length, so an alert clears soon after the problem ends.

Ship the new code dark, with its flag off, and turning the behavior on becomes a config change you can make, watch and undo without touching binaries. The trap is treating flag-off as already tested: both paths are now production code, so test the configuration you plan to switch on and the fallback with it switched back off.

The controller retires a slice of old instances and starts a slice of new ones, so capacity dips by at most one batch, and a surge allowance of extra instances can keep it from dipping at all. Old and new serve side by side for the whole rollout, so every change must be safe in a mixed fleet. Rolling back is another rolling update and takes about as long.

One color serves while the other holds the new version fully deployed and idle, so cutover and rollback are each a single router change. The doubling is the price, and it buys less for stateful parts: both colors usually share one database, so a schema change the old color cannot read breaks the instant rollback. Shifting traffic gradually turns the pair into a canary.

Comparing the canary with last week, or with itself before the deploy, mixes the release's effect with everything time changes: load, traffic mix, the day of the week. Run a baseline of the old version deployed at the same moment, at the same size, on the same traffic slice, and compare the two in parallel. A fresh baseline also matches the cache warmup and heap growth a new canary has.

A small canary still catches loud failures: a crash loop or an error spike shows up on a handful of instances. What it cannot do is tell a real half-percent rise in errors from noise; the rarer the regression and the noisier the metric, the more traffic or time it needs. Spinnaker's canary guide wants at least 50 data points per metric. A pass from an underpowered canary means not catastrophic, never safe.

A kill switch helps only if flipping it needs no build or deploy, so its value lives in a runtime config store and the code falls back to a safe default when that store is unreachable. Each flag is also a branch you carry: delete release flags once a launch settles, keep the long-lived switches that shed non-essential features under load, and exercise those before an incident needs them.

Every step but the last is separately deployable and reversible: after the expand, old code still works; while both columns are written, you can roll the application back freely; the backfill can run slowly in batches. Skipping the dual write leaves rows created during the backfill missing from the new column, and contracting early breaks any old instance still running.

Mid-rollout, an old instance can read a row or message a new one wrote, and the reverse. Formats therefore evolve additively: new fields are optional with sensible defaults, a removed field's number is reserved and never reused, and readers skip what they do not recognize. Queued messages outlive the rollout, so a consumer must accept the previous format until its queue has fully drained.

Rolling back code restores a binary, not data. Once a column is dropped or a topic deleted, the only way back is a restore, which loses everything written since the backup and can take hours or days. So first ship the code that stops using the thing, let it run long enough to prove nothing still reads it, and only then delete it, as a separate change.

Zones in one region sit within about 100 km of each other on links fast enough for synchronous replication, so a database can commit synchronously to two zones at a small latency cost. Regions are separate geographic areas, and a cross-continent round trip is ~100ms, so most designs replicate between regions asynchronously and accept a nonzero recovery point. Multiple zones already cover most single-site disasters.

In active-passive, all writes go to one region and the other waits, so there are no write conflicts, but failover is a rare, risky event whose readiness is only known when tested. Active-active serves users from every region, which removes failover as an event and forces a choice for concurrent writes: send all writes to one region, partition them by key, or let the last writer win.

Recovery point is set mostly by how often data leaves the primary: nightly backups risk up to a day of writes, continuous replication risks seconds. Recovery time is set by how much must be rebuilt, from redeploying everything to promoting a running replica. Tightening either one costs more, and when recovery would cost more than the outage, a looser target is the honest choice.

A replica stays seconds behind the primary, which is exactly why it cannot save you from a bad migration, an errant DELETE or a bug that corrupts rows: the damage reaches every copy almost at once. A backup is a frozen past state you can return to. Point-in-time recovery and object versioning give you both: continuous capture plus a way to rewind to before the mistake.

Backups fail silently: an empty archive, a restore that depends on the very system that just failed, or missing configuration needed to redeploy. Only an end-to-end restore, run on a schedule and timed, proves the data comes back. Time it at full size: restoring petabytes can take days, which can blow the recovery time objective even though no data was lost.

If records must stay inside a jurisdiction, each copy you make has to stay there too: replicas, backups, logs, caches, analytics extracts and the disaster recovery site. When the jurisdiction holds only one region, a cross-region standby is ruled out and multi-zone redundancy is the ceiling. The usual design gives each user a home region and routes their requests and storage there.

Each record, usually keyed by user or tenant, has one region that accepts its writes; other regions serve reads from replicas or forward writes home, so there is never a second writer to reconcile with. The costs are a cross-region round trip when someone writes from far away, and lost write availability for those records while their home region is down, until you re-home them.

With asynchronous replication the standby lags, so promoting it discards whatever had not arrived yet. To lose nothing, every commit must first be acknowledged by the remote region, adding a cross-region round trip to each write, or the system must refuse writes until the old primary returns. A failover on a false alarm pays that price for nothing, which is why many teams trigger it by hand.

Backup-and-restore keeps only copies, so recovery means rebuilding everything. Pilot light keeps data replicated and live with servers switched off, ready to start and scale out. Warm standby runs a smaller, fully working copy that can take traffic at once and only needs to scale up. Multi-site serves from every region, cutting recovery near zero except for corrupted data, which still needs backups.

A relational schema lets you ask questions you did not plan for, because joins and constraints live in the engine. Most NoSQL stores trade that for easy partitioning: you model around known access patterns, duplicate data across tables, and join in your own code. The usual mistake is picking a store for scale you do not have, then finding each new query needs a new copy of the data.

If a department's name is stored on every employee row, renaming it means finding every row, and missing one leaves the data contradicting itself. Third normal form moves each non-key fact to the table whose key it depends on, so there is one place to change it. The price shows up on reads, which rebuild the picture with joins across many small tables.

Copying a customer's name into every order row lets an order page render from one read with no join, which suits read-heavy data that rarely changes. Each copy is now a write you must remember, ideally in the same transaction, and a missed one is a silent inconsistency. Denormalize for a measured read path, and keep one authoritative copy to rebuild the others from.

A relational table rejects a malformed row at insert, so readers can trust its shape. A data lake that accepts raw files skips that check: loading is fast and flexible, but each reader must parse, validate and handle bad records itself, and two readers may interpret one field differently. A common compromise lands raw data cheaply, then publishes validated, typed tables for wider use.

Atomicity is all-or-nothing, consistency means constraints hold when a transaction commits, isolation hides uncommitted work from concurrent transactions, and durability means committed changes survive a crash. They come apart in practice: an engine can be atomic and durable while running at an isolation level that still permits anomalies, so check the level before assuming.

If a transfer debits one account and the server crashes before the credit, recovery undoes the debit, so no reader ever sees money vanish. The guarantee covers one database's transaction only: writing a row and publishing a message, or updating two services' databases, is not atomic unless you add a pattern such as an outbox or a saga.

An update writes a new row version instead of overwriting, and each query reads the versions committed as of its snapshot, so readers never wait on writers and writers never wait on readers. The cost is garbage: old versions pile up until cleanup, such as VACUUM in PostgreSQL 18, removes them, and one long-open transaction keeps alive every version it might still see.

Every request names its key, so hashing the key picks the partition, and adding nodes just spreads keys thinner. Anything else is the expensive case: finding all items with some attribute means a full scan or a second structure you maintain yourself. A very popular key is also a hot partition that extra nodes cannot fix, since one key always lives in one place.

Storing an order with its line items and shipping address in one document means one read fetches everything the page needs, and the shape can vary per record. Trouble starts when the data must be seen from another angle: totals for one product across all orders, or an address embedded in thousands of documents. Embed what is read together; reference what is shared or grows without bound.

In Cassandra 5.0, the partition key decides which nodes hold the data and the clustering key fixes the sort order within that partition, so the last 50 events for one device is one sequential read. A query that omits the partition key must ask every node, which is why you design one table per query and accept duplicated data. An unbounded partition keeps growing until it hurts.

In a relational schema, every extra hop, such as friends of friends, is another self-join, and the engine rebuilds each relationship from foreign keys at every step. A graph engine stores edges as first-class records and follows them directly, so cost tracks the part of the graph visited rather than table size. For one-hop lookups and wide aggregates, a relational database usually does as well and is easier to run.

Points arrive in time order and are almost never updated, so storage is cut into time-based chunks. Expiring old data then means dropping a whole chunk, a file deletion, instead of slow row-by-row deletes. Downsampling first rolls raw points into per-minute or per-hour summaries, so history stays queryable for less. In TimescaleDB, a chunk is dropped only when all of its rows are past retention.

A hash index stores a hash code of the value, which scatters neighbors, so it can find email = 'x' but cannot help with >, BETWEEN or ORDER BY. A B-tree keeps keys sorted, so the same index serves equality, ranges and sorted output. That is why the B-tree is the default and a hash index is a narrow optimization, as the PostgreSQL 18 docs describe.

Entries sort by a, then by b within each a, then by c, so a filter on a and b lands on one contiguous slice while a filter on c alone has no slice to jump to. A range on b ends the guaranteed narrowing: conditions on c are checked in the index but do not necessarily shrink the scan. PostgreSQL 18 can skip-scan when a has few distinct values; with many, expect a sequential scan.

Putting the selected columns in the index, as keys or as non-key payload with INCLUDE, lets the engine answer from the index alone. Each extra column makes the index larger and slower to maintain. PostgreSQL 18, like earlier releases, has a catch: an index-only scan must still visit any table page not yet marked all-visible, so on heavily updated tables much of the saving disappears.

Each index match costs a lookup back into the table at a random page, while a sequential scan reads pages in order. Filtering a status column with three values matches about a third of the rows, which touches nearly every page anyway, so the planner scans. The exception is a skewed column where the value you want is rare; table statistics reveal that, and a partial index can target it.

An application database is tuned for many short transactions that read or write a few whole rows. A warehouse holds history from many sources, loaded in periodic batches and modeled as a star schema: a large table of facts joined to descriptive dimension tables, so a query scans millions of rows across a few columns. Running reports on the live database competes with customer traffic.

ETL cleans and reshapes data on a separate system before it reaches the warehouse, so only conforming rows land, but every new question means changing the pipeline upstream. ELT copies raw data in first and transforms it with SQL inside the warehouse, keeping raw history for reprocessing and using the warehouse's own compute. The catch is that raw, possibly sensitive data now sits there.

An exact search compares the query with every stored vector, so cost grows linearly with the collection. Approximate indexes such as HNSW and IVFFlat examine only a promising fraction, which is fast but can miss true neighbors; the share found is called recall. Knobs like hnsw.ef_search in pgvector buy recall with latency, so measure recall on real queries instead of assuming exact results.

Each embedding model places text in its own space, so a distance between an old-model vector and a new-model vector means nothing, even when the dimensions match. Mixing them in one index quietly returns wrong neighbors. Run a model change like a schema migration: add a new vector column, backfill it while the old index serves, compare recall on test queries, switch reads, then drop the old column.

Autoscaling, deploys and failures keep changing instance addresses, so callers look a name up when they need it, through DNS or a registry API. DNS needs no client library, but answers are cached for the record's TTL, so a dead address can linger; an API can propagate changes quickly but couples every client to it. Either way, the lookup path becomes a dependency of every call.

Each instance reports in periodically, and if no report arrives within the time-to-live, the registry marks it unhealthy and stops handing it out. The TTL sets a trade: short catches crashes fast but evicts healthy instances during a garbage-collection pause or a network blip, while long keeps routing to dead ones. A heartbeat proves the process can reach the registry, not that it serves correctly.

With server-side balancing, clients call one stable address and a load balancer picks the instance: simple clients, one extra hop, and one more component to scale and keep available. With client-side balancing, each caller fetches the instance list and chooses itself, which saves the hop, but every client library in every language must implement health checks and balancing policy identically.

Each instance's traffic passes through a local proxy, Envoy in Istio 1.31, which applies load balancing, retries, circuit breaking, mutual TLS and metrics from central configuration instead of per-codebase libraries. The cost is an extra hop on every call, the proxy's CPU and memory on every instance, and one more layer to debug. Set retries in the mesh or the app, not both, or the attempts multiply.

Code at a CDN location can rewrite URLs, normalize cache keys or verify a signed token a few milliseconds from the user. What it cannot do cheaply is reach your data: the lightest edge runtimes forbid network calls entirely, and a call back to a database in one region spends the latency you saved. Keep edge logic to decisions made from the request itself or small replicated config.

A browser cache is private to one user, and you cannot purge a single entry in it: only expiry, the user, or a Clear-Site-Data response that wipes your whole origin's cache clears it. A CDN is shared and usually offers a purge API, and gateway or application caches are yours to invalidate in code. That is why static assets get versioned file names, making each release a new URL, while HTML stays short-lived and revalidated.

The real difference is whether the input is bounded. A batch job knows its input is complete, so it can sort, join and aggregate everything at once and rerun with the same result. A stream job never sees the end, so it emits results incrementally and must decide when a partial answer is good enough. Apache Beam runs one pipeline definition over either kind, which shows the logic need not differ.

If a report is read once each morning, computing it overnight over a complete day avoids every streaming problem: late events, watermarks, checkpointed state and always-on workers. Batch also suits jobs that need the whole dataset at once, such as retraining a model or a full deduplication, and recovering from a bug is just a rerun. Choose streaming when someone acts on the result within minutes.

A count or sum over a stream never has a final value, because more events can always arrive. A window groups events by timestamp into finite buckets, such as each five minutes, so every bucket can produce an answer. Apache Beam puts unwindowed data in one global window whose default trigger waits for all the data, which an unbounded source never delivers, so Beam rejects that grouping when the pipeline is built.

A tumbling window, such as each minute, puts every event in exactly one bucket, which suits billing totals. A sliding window has a length and a period, say 60 seconds starting every 30, so each event lands in more than one window; it suits running averages. A session window is cut separately per key after a gap with no events, so its length follows each user's activity.

A phone that was offline uploads an hour of taps at once: each tap's event time is when it happened, and its processing time is when your pipeline saw it. Grouping by processing time files those taps under the wrong hour, and replaying the same log would file them differently again. Grouping by event time gives the same answer on every run, at the cost of handling late data.

Once the watermark passes the end of a window, the window can emit its result, and anything that arrives later with a timestamp inside it is late. An eager watermark drops or later corrects real data; a cautious one delays every result. Allowed lateness keeps a window open for corrections, at the cost of holding its state longer. Apache Beam's default configuration discards late data.

The batch layer periodically recomputes accurate views from the full, immutable history, while the speed layer fills the recent gap with fast, possibly approximate results, and queries merge the two. The appeal is that a mistake in the fast path heals at the next batch run. The cost is the same logic written twice on two frameworks, which drift apart and double testing and operations.

Keep the input in a log retained long enough to cover any reprocessing you expect. To change the logic, start a second instance of the new job from the start of the log, writing to a new output table; when it catches up, point readers at the new table and delete the old one. The log's retention bounds how far back you can recompute, and output storage doubles during the switch.

Ten thousand users and a hundred million users lead to different designs, as do mostly-reads and every-user-writes-constantly. Before drawing boxes, pin down the core operations, daily active users, read-to-write ratio, data size and latency target, then convert them into requests per second and storage. A design built on an unstated guess can be elegant and still answer the wrong question.

Two systems with identical features, post and read a feed, can need opposite architectures when one must respond within 200 milliseconds at four nines of availability and the other may lag a minute. The non-functional targets are where trade-offs live: strong consistency costs latency, durability costs write speed, availability costs money. Write them as numbers, or no design can be judged against them.

An application tier behind a load balancer and one database with a replica can carry a surprising amount of traffic. Add a cache, a queue, sharding or a second region only when a stated number demands it, and say which number. Every component brings its own failure modes and operating cost, so an unneeded one makes the system less reliable, not more. Name the signal that would trigger the next step.

Encoding an auto-increment counter in base62 never collides but yields sequential codes, so anyone can enumerate every link. Random codes resist guessing yet can collide: seven base62 characters give about 3.5 trillion codes, but by the birthday bound some collision is about as likely as not after roughly 2.2 million random codes, so each insert must check. A truncated hash of the URL collides the same way.

Each node stands for a prefix, so finding every word that starts with 'cat' means following three edges, then listing that subtree, in time set by the prefix length rather than the dictionary size. For autocomplete, you can store the few best completions at each node so a keystroke needs no subtree walk. The cost is memory, a node per character, which a radix tree cuts by merging single-child chains.

Geohash interleaves longitude and latitude bits and writes them in base32, so a longer shared prefix guarantees two points are close. The reverse fails: two points a meter apart across a cell edge, such as the equator or the prime meridian, may share no prefix at all. A nearby search therefore checks the target cell and its eight neighbors, and cells shrink in ground width toward the poles.

Each cell holds up to a fixed number of points and splits into four quadrants when it overflows, so a dense city center subdivides deeply while open ocean stays one large cell. A range query skips every quadrant that misses the search area. The weak spots are heavy clustering, which forces a deep, lopsided tree, and fast-moving objects, which cause constant splits and merges.

Redis stores a sorted set as a skip list plus a hash table, so updating a score, finding a member's rank and reading the top 10 each cost about O(log N). Two catches: equal scores are ordered by the member string's bytes, not by who got there first, and scores are 64-bit floats, exact for integers only up to 2^53, which matters if you pack a timestamp into the score to break ties.

Instead of keeping each value, it hashes values and keeps a fixed array of small registers summarizing those hashes, so memory stays constant: Redis uses at most 12 KB per counter with a 0.81% standard error. You cannot ask whether one item was seen, or list items. Counters merge into an estimate of the union, so daily counts combine into a weekly unique count, but Redis offers no intersection.

Each item bumps one counter in each of several hashed rows, and a query returns the smallest of its counters; collisions only add, so the estimate never falls below the true count. The error is a fraction of total stream volume, so heavy hitters read accurately while rare items drown in noise. Width comes from the error you accept, depth from its probability. Negative increments break the guarantee.

Every connected client sends heartbeats, and every change fans out to everyone watching that user. A million users beating every 30 seconds is about 33,000 writes per second before any status changes, and one flip for a user with 500 watching contacts is 500 deliveries. Designs keep presence in memory with expiry, batch updates, and push only to watchers who are online themselves.

Megabyte-sized images in table rows bloat backups, replication traffic and the buffer cache, and push every byte through database connections. Object storage is built for large, immutable files and can serve them through a CDN or presigned URLs, while the row keeps the key and metadata. No transaction spans both systems, so upload the object first and sweep up objects whose row never committed.

Lifecycle rules move rarely read objects to cheaper tiers, and the catches are in the fine print. Infrequent-access tiers charge per retrieval and bill a minimum storage duration. Some archive tiers must be restored before you can read an object, which takes minutes to hours, while others serve reads in milliseconds. Tier on measured access, since demoting data that turns out to be hot can cost more than it saves.

The same video is encoded at several bitrates, cut into short segments and listed in a manifest, such as an HLS playlist or a DASH description, with each rendition's bandwidth. Before fetching each segment, the player compares its measured throughput with those figures and picks a rung. Switching happens only at segment boundaries, so a bandwidth drop costs visible sharpness instead of a frozen picture.

Fetch a host's robots.txt before its pages: RFC 9309 says not to use a cached copy for more than 24 hours, and to treat a server error on that file as disallow-everything. Rate limits apply per host, so the frontier queues URLs by host. Normalize URLs before checking the seen-set; at web scale that can be a Bloom filter, whose false positives skip a few pages never actually fetched.

In HDFS 3.3.5, a file is cut into large blocks, typically 128 MB, each stored on several DataNodes, while the NameNode keeps the whole namespace and block map in memory. Large blocks keep that map small and let clients stream at disk speed. Because every file and block costs NameNode memory, millions of tiny files hurt more than a few huge ones, and missed heartbeats trigger re-replication elsewhere.

Writing one row per click turns a traffic spike into a database outage. Instead, events land in a durable log, a stream job counts them per ad per minute by event time, and only rollups are stored, so a minute with a million clicks becomes a few thousand rows. The job may retry, so rollup writes must be idempotent, keyed by window and ad, and the raw log is kept for recomputing after a bug.

Kubernetes documents that a CronJob creates a job about once per scheduled time, sometimes twice and sometimes not at all, so jobs must be idempotent. Decide what a missed run means, catch up once, run every missed slot, or skip, and whether a new run may start while the last continues. A worker that dies mid-job needs a lease that expires, so another worker can take the job over.

A pooled table keyed by tenant_id is cheapest and scales with total load, but one missing filter leaks data between tenants, one heavy tenant slows everyone, and an outage hits them all. A database per tenant contains the blast radius and eases per-tenant cost tracking and compliance, while multiplying migrations and idle capacity. Many products pool small tenants and silo the large or regulated ones.

A managed database takes patching, backups, failover and capacity work off your team, and the provider's operators handle the underlying hosts. In exchange you accept the versions, extensions, settings and maintenance windows it offers, along with its quotas and outages. Self-hosting pays off when you need a control the service hides, or at a scale where its markup outweighs the cost of a team.

A volumetric flood can exceed any single site's links, so it must be absorbed by a network spread across many edge locations that filters layer 3 and 4 floods, using SYN cookies for example, before traffic reaches you. The shield fails if attackers find the origin's real address and hit it directly, so let the origin accept traffic only from the CDN. Application-layer floods also need rate rules.

Rules match request patterns such as SQL injection or cross-site scripting payloads and block or log them before the app sees them, which can buy time while a flaw is fixed. A WAF judges requests, not meaning: it cannot tell that user 42 should not read user 43's invoice. Its rules need tuning against real traffic as the app changes, because a false positive blocks a real customer.

A password baked into an image or committed to a repository spreads to every copy, survives in history after you delete it, and cannot rotate without a rebuild. A secrets service hands credentials out at runtime through an API, logs each read, and rotates them on schedule. Better still is a credential you never store: short-lived tokens issued to the workload's own identity, such as a cloud role.

Ordinary TLS proves the server's identity to the client; with mTLS the client also presents a certificate, so the server learns which workload is calling instead of trusting the subnet it came from. Identity is not permission: mTLS performs no authorization, so the server must still check that this caller may take this action. It also needs a private CA and automated certificate rotation.

A perimeter firewall assumes everything inside is friendly, so one compromised laptop or service can reach the rest. Under zero trust, as NIST SP 800-207 frames it, location grants no implicit trust: each request carries an identity that is authenticated and authorized for that resource before access is allowed. Network segmentation remains one layer of defense, never the only check.

Encryption in transit stops someone on the network path from reading or altering traffic. Encryption at rest protects disks, snapshots and backups that leave your control. Neither stops an attacker with a valid application credential, because the database decrypts for any authorized query. TLS that ends at a load balancer also leaves the hop behind it in plaintext unless you encrypt that hop too.

Deleting the primary row is the easy part; copies sit in replicas, caches, search indexes, analytics tables, logs and backups designed never to change. The UK ICO's guidance accepts putting backup data 'beyond use' when it cannot be erased at once, and expects you to tell each recipient you disclosed it to, unless that proves impossible or disproportionate. Some designs encrypt each user's data under its own key and delete the key.

On a miss, a recursive resolver walks from the root servers to the top-level domain's servers to your authoritative servers, then caches the answer for its TTL, so a change spreads only as caches expire. Before a planned move, lower the TTL to something like 300 seconds and wait out the old TTL, switch, then raise it again. A long TTL means fewer lookups but slower changes.

On one machine cron is simple; on a fleet, every instance with the same crontab runs the job once each. Run it from one scheduler or take a lock, decide whether a run may start while the previous one is still going, and decide whether runs missed during downtime are caught up or skipped. Pin the time zone, since an unset one follows whatever the scheduler's host uses.

Tests run against mocks and fixtures, so they miss a wrong environment variable, a missing permission, a migration that times out at realistic row counts, or a third-party API that behaves differently from its stub. A staging environment built from the same infrastructure code as production catches those. It still misses whatever depends on real traffic or full scale, which is why canaries remain.

Text such as HTML, JSON and JavaScript can shrink by up to about 70% with gzip or Brotli, negotiated through Accept-Encoding and Content-Encoding. Already-compressed formats such as JPEG, PNG and video gain nothing and can grow. Send Vary: Accept-Encoding so shared caches keep compressed and uncompressed copies apart, and skip responses small enough to fit in one packet anyway.

After an HTTP upgrade handshake, a WebSocket stays open and either side can send at any moment, so a chat server delivers a new message as it arrives instead of waiting to be asked. The trade: many long-lived connections pin server memory, complicate load balancing and deploys, and need reconnect logic. The browser's WebSocket API also lacks backpressure, so a fast sender can swamp a slow client.

The client sends a request, the server holds it until there is news or a timeout, answers, and the client asks again at once. It passes through anything that speaks HTTP, but each message carries full headers, worst-case delivery takes more than three network transits, and proxies may cut off a request held too long, so RFC 6202 treats a hold of about 30 seconds as the safer choice.

Your server signs a URL that allows one operation, such as a PUT to one object key, until it expires, and the client sends the file straight to object storage while your app never handles the bytes. The URL acts as a bearer credential: whoever holds it can use it until expiry, it carries only the signer's permissions, and an upload to an existing key replaces that object.

The detector records the gaps between recent heartbeats and, from their mean and spread, converts the time since the last one into phi, a suspicion score on a log scale. On a steady network a short delay already looks suspicious; on a jittery one the same delay does not. You set a phi threshold instead of a timeout: lower catches crashes sooner with more false alarms.

Each update enters at the head and passes through every server to the tail, and only the tail replies and serves reads, so a read never sees a write that is not fully replicated. With t servers, up to t − 1 can fail. A separate master detects failures and splices nodes out, assuming servers halt cleanly rather than misbehave. Write latency grows with the length of the chain.

Sources inject numbered barriers into the stream, and each operator snapshots its state as a barrier passes, so records before it belong to snapshot n and those after to n+1. An operator with several inputs must align, holding back inputs whose barrier came early until the rest arrive, which adds delay under backpressure. Flink 2.3 offers unaligned checkpoints that store in-flight records instead.

A monotone program only accumulates conclusions: more input never retracts an earlier output, so replicas can apply updates in any order and still agree without coordinating. Set union and reachability qualify. Anything that concludes from absence does not, such as an item not being in a cart or no path existing, and must coordinate to know that all the input has arrived.

A sequencing layer batches incoming transactions and fixes their global order before any runs; each replica then executes that order deterministically, so replicas agree without two-phase commit and a node failure aborts nothing. The catch is that a transaction must declare its read and write sets up front, so one whose reads decide its writes needs a scouting read and restarts if that goes stale.

A patch or config fix ships as a new image that replaces instances, never as an edit on a live server, so servers cannot drift apart and the previous version stays ready for rollback. The discipline breaks on state: anything that must outlive an instance belongs in external storage, and an emergency hand fix vanishes at the next deploy unless it is built into the image.

Declaring networks, databases and permissions in versioned templates means a new environment is one more run of the same code, and each change arrives as a reviewable diff with history. The value leaks away when someone edits a resource by hand: the live system drifts from its declaration, and the next apply may quietly revert the fix. Detect drift, and route emergency changes back through the code.

An image bundles the application with its libraries and runtime, and a built image never changes, so the artifact you tested is the one you ship. Two gaps remain: containers share the host's kernel, so kernel-level differences still leak through, and a tag such as latest can be repointed to another image, so deploy by digest when you need the exact bytes you tested.

With settings read from the environment, one build runs everywhere; with no local state, any instance can serve any request and be killed at will; with fast startup and clean shutdown on SIGTERM, the scheduler can add and remove instances freely. Sticky sessions break this, which is why session data moves to a shared store with expiry. A quick test: could the code be published without leaking a credential?