Replicas, lag and read routing

Copying the primary's history

The primary accepts the price update and records its outcome. A replica obtains the history needed to reproduce that outcome and applies it to its own copy. In PostgreSQL physical replication, that history is the write-ahead log, or WAL. The standby starts from a compatible base copy and replays later WAL. It does not run the application's price-setting request again.

A read replica can take catalogue lookups or reports off the primary. It still has to store the copied data and keep up with incoming changes. Adding readers does not remove the work of applying writes at each copy, and a busy reporting workload can compete with that work.

Replicas and shards solve different problems. A replica is another copy of the same data; a shard holds a portion of the data. Our primary, A and B each hold the shop, rather than dividing its products between them. Sharding and distributed SQL develops that division and agreement within a replicated partition. This article follows one copied history.

A live copy also follows committed mistakes. If a separate mistaken update sets P7 to 2.20, replaying it faithfully spreads the mistake. Replication does not preserve an earlier recovery point just because another machine holds the data. Backup, restore and reconciliation explains how an earlier recoverable copy changes that choice.

Received, durable and replayed

“A has the change” hides three questions. Have its records arrived? Are they stored so that they survive the promised failure? Have they been applied far enough for a new read snapshot to include the commit? A can pass one of these boundaries without passing the next.

Once C is durable, A can recover it from its surviving log, yet its applied price may still be 20.00. Durable describes survival; replayed describes application of the copied history. Receipt is earlier still: bytes in volatile memory are not a second durable copy.

Replaying U alone does not make an unfinished price visible. In PostgreSQL, once the transaction's commit record has been replayed, its changes can appear in new snapshots on the standby. A report retaining a snapshot from before C can continue seeing 20.00. Catching up the replica does not refresh the report's established view.

These progress points need not advance together. Real send, flush and replay work overlaps. A position is useful only with its meaning: a receipt position cannot be substituted for a replay position, and a log position is not by itself evidence of a transaction’s commit outcome.

What a commit waits for

Under asynchronous replication, the primary does not wait for a replica's acknowledgement before answering the write. That can still be a fully durable local commit. “Asynchronous” here describes the remote wait; it does not say that the primary skipped its own log flush.

With synchronous replication, the reply waits for selected remote progress as well. Name both the required copies and the required progress point. Waiting for C to be durable on A gives a different promise from waiting for A to replay it too.

PostgreSQL makes this distinction explicit: with synchronous standbys configured, synchronous_commit = on waits for their durable flush, while remote_apply waits for replay too. Its remote_write level waits for writing to the standby's operating system, which is weaker than surviving that operating system's failure. PostgreSQL physical replicas and promotion shows how to configure and measure those waits.

Waiting makes the selected remote work part of write latency and availability. If the required acknowledgements cannot arrive, a write can remain waiting even while the primary's local storage works. Relaxing that rule changes the failure promise. Waiting for A also says nothing about an arbitrary read sent to B: the write's acknowledgement policy and the read's routing policy must fit together.

Reading your own writes

Suppose the application confirms the price edit, then reloads the catalogue page. The editor expects the reload to include the write it just confirmed. This is a read-your-writes requirement. It does not require every reader everywhere to be current.

One route is to read from the primary after the commit, using a new snapshot that includes it. Another is to use a replica shown to include the required history. To explain that second route, give the application an illustrative token H:C: “this read must include history H through C”. A real engine or routing layer needs a reliable way to obtain and enforce such a requirement; a successful SQL update does not universally return this token.

Suppose the next read is routed to A while it is behind C. The application waits for A to qualify, then takes a new snapshot there. If the wait expires, it can use a qualified primary or another qualifying copy under its fallback policy. If none is available, it must report that this read promise cannot be met.

The check belongs to the actual target. Checking A and then allowing a load balancer to send the query to B loses the proof. So does retaining a snapshot established before the check and assuming it now includes C. Keep the qualified target and the read together, and establish the required new snapshot after qualification. If a connection is replaced or the topology changes, validate again.

A fixed sleep cannot provide that proof. Waiting a second may usually be enough, until replay pauses or the link breaks. A bounded wait for the required condition has a different meaning: either the condition holds before the deadline, or the application takes its declared fallback. If no eligible copy is reachable, return an explicit inability to meet the promise. Serving an older price requires an agreed weaker contract.

Including C is a lower bound, not a demand to freeze the catalogue at 22.00. A later legitimate price change may also be visible. Read-your-writes says the reader must not lose its write from the history it observes; it does not promise that nobody has since changed the same row.

Stale reads and changing replicas

A customer browsing the catalogue may be allowed an older view. That is a stale read: a read from history behind the current writer. Whether it is acceptable comes from what the request means. A browsing page, a price-edit confirmation and a stock-changing purchase need not have the same read promise.

There is another surprise even without a new write. A reader sees 22.00 on A, then a later request sees 20.00 on B. Neither copy moved backwards: the session changed targets without preserving the history it had already observed.

Monotonic reads prevent this retreat to history that omits an earlier read's relevant changes. They differ from read-your-writes: the reader may never have written anything. In our single ordered history, retaining an observed-history requirement can rule out B until it catches up. Keeping the session on A can help while A retains that history, but a routing change must preserve the requirement too.

Neither requirement establishes a time-based freshness bound. “Includes C” tells us about a particular commit. “No more than two seconds behind” needs trustworthy evidence relating the served view to time and a rule for refusing reads when that evidence is insufficient. A server being reachable, a recent network reply or a small-looking backlog does not establish that bound.

A fresh read also does not reserve its answer. Another transaction can change stock between a read and a purchase. Enforce the purchase rule at the write authority, through the transaction and concurrency protocol described in Transactions and isolation. Reading a newer copy is no replacement for that protocol.

These examples qualify one copy of one history. A read that joins data owned by different shards needs a compatible view across them; Time, snapshots and distributed reads develops that extra problem.

Lag and retained history

Replication lag is a gap between progress points. Locate the gap before treating it as one number. If the primary has generated history that has not been sent, inspect the sending path. If sent history has not arrived, inspect transport and receipt. If receipt is advancing but durable progress is not, inspect the receiving storage path. If durable history is advancing while replay falls behind, inspect application of the log and the work competing with it.

PostgreSQL's reported write, flush and replay lag times describe recent progress and the sender learning about it. They are not catch-up predictions. On an idle, fully caught-up system, a last measured delay may remain briefly before becoming null; subtracting the last transaction's time from now would instead make an idle database appear progressively further behind.

Catching up while writes continue requires spare capacity. Suppose 600 MiB remains to replay and replay capacity is 20 MiB/s. With no new writes, the ideal drain time is 30 seconds. Continuing writes use part of that same capacity.

At 20 MiB/s of replay and 10 MiB/s of new log, only 10 MiB/s drains the backlog. The ideal estimate is 60 seconds, twice the estimate with writes stopped. Rates can change, and another stage can become the bottleneck, so the arithmetic is a way to test an assumption rather than promise a deadline.

Extra reporting work can reduce that spare capacity. PostgreSQL standby queries can also conflict with recovery work: allowing a query to continue or cancelling it to let replay advance is an operational trade, not free read capacity.

Keeping enough log to catch up

A disconnected replica cannot ask for missing history after every surviving copy of that history has been removed. It needs a complete usable chain from its current state. Retaining that chain lets it resume later, but the retained log takes space while the primary continues writing.

PostgreSQL replication slots can retain WAL needed by a standby. An abandoned or slow consumer can therefore make retained WAL grow; limits can protect storage at the cost of losing the consumer's ability to resume from that position. If the missing files remain in a usable archive, the replica may obtain them there. Otherwise it needs a fresh base copy rather than a missing stretch guessed from the current rows.

Monitor the history still required alongside the space and time available to retain it. Local crash recovery no longer needing a log segment does not mean every replica or backup has finished with it. The PostgreSQL replication article develops the native positions, retention controls and rebuild procedure.

When the primary is unavailable

Suppose the primary disappears while A has C durably stored but not yet replayed. If A’s storage survives, it can recover the 22.00 change, although reads may need to wait for replay. B, still before C, has not shown that it can recover that acknowledged change. The last primary reply alone does not tell us which surviving copy contains it.

A read can continue only under a promise the surviving target can meet. A reachable B may serve a permitted older catalogue view. It cannot satisfy the H:C request without obtaining the missing history. When no qualifying copy is available, preserving that promise costs read availability.

Reachability and recent data do not grant permission to accept writes. The former primary may still be running beyond a broken link. Failover and fencing follows the choice of a surviving write authority, exclusion of the former writer and handling of any lost or divergent work.

History requirements must survive that decision honestly too. A restored copy or a promoted branch may not contain C, even if its numeric log position looks larger. Revalidate the history identity and its relationship to the required commit before comparing positions. A read promise follows the required history, not whichever server has inherited the primary's address.