Skip to content
All notes

Keep Slow Work Off the Real-Time Path

The shared action should not inherit the latency of every feature it triggers.

Software engineering3 min read
Contents

One user action can trigger a surprising amount of work:

  • validate a command;
  • save shared state;
  • notify connected clients;
  • call an AI model;
  • update analytics;
  • send a push notification;
  • write an audit record.

If the product waits for all of it before confirming the action, the slowest side effect becomes the user's latency.

Define the critical event

Ask: what is the first fact every participant must observe?

For chess, it is the accepted move.

For chat, it is the accepted message.

For a collaborative editor, it is the accepted operation.

Everything else should be classified against that fact:

required before acceptance
required before broadcast
allowed after broadcast
optional

That classification is more useful than calling everything "async."

A move should not wait on commentary

Real-Time Chess generates a short commentary line after every accepted move. The model call can fail or take seconds.

The event order is:

validate move
save room
emit move
generate commentary
save commentary
emit commentary

The board updates from the move event. The commentary panel renders a separate loading state and later consumes commentary.

If generation fails, a local rule-based generator produces the line instead. Gameplay does not need to know which generator won.

Give side effects their own state

Separating the events is only half the work. The interface needs to represent each stage.

For commentary:

move accepted → commentary pending → commentary ready
                                  ↘ fallback ready

For chat enrichment:

message accepted → link preview pending → preview ready | unavailable

For file processing:

upload accepted → scan pending → usable | quarantined

Do not make a secondary failure look like the primary action failed.

Use bounded time

Every external call on a side-effect path needs a deadline.

The chess commentary layer uses:

  • a four-second abort timer around the provider request;
  • an eight-second outer race around the generator;
  • a client-side fallback message if commentary still does not arrive.

The exact numbers depend on the product. The principle is stable: a pending branch must eventually become success, fallback, or failure. It cannot remain pending forever.

Persist before or after broadcast?

For the primary shared action, saving before broadcast is usually the safer default:

validate → persist → broadcast

If the server broadcasts first and the write fails, every client may display state the system cannot recover.

For a secondary result, the same question depends on its value:

  • persist before broadcast when clients expect it in future snapshots;
  • broadcast without persistence for disposable presence or typing indicators;
  • queue durable work when retries must survive process failure.

In the chess project, both the move and final commentary are saved before their corresponding event is emitted.

Fire-and-forget is not durable background work

Calling an async function without awaiting it keeps the request path short. It does not guarantee completion.

If the process crashes after the move broadcast but before commentary is saved, that commentary disappears. For a demo, this trade-off is acceptable. For important work, use a durable job:

transaction commits primary state + outbox record
worker claims outbox record
worker performs side effect
worker records completion
server emits or clients poll the result

The outbox pattern ties the primary write and the intent to do background work into one transaction.

Preserve causality

A side effect needs a stable reference to the action that caused it.

The chess commentary event uses the move's SAN value. That is readable, but not globally unique. A production event would carry:

{
  eventId: "01J...",
  causedBy: "move_01J...",
  roomId: "abc123",
  roomVersion: 18,
  text: "..."
}

Now logging, retries, deduplication, and UI reconciliation all share the same causal chain.

Decide what the user loses

The boundary between critical and secondary work becomes clear when you ask what failure means.

If the move is lost, the game is wrong.

If commentary is lost, the game is still playable.

That difference should control ordering, persistence, retry effort, alerting, and UI language.

The reusable pattern

For any real-time action with enrichment:

  1. validate the core command;
  2. persist the authoritative state;
  3. broadcast the accepted event;
  4. expose a pending state for enrichment;
  5. run slow work with a deadline;
  6. persist and emit the result;
  7. fall back without undoing the accepted action;
  8. move to a durable queue when losing the side effect is no longer acceptable.

Fast real-time products are not systems where every task is fast. They are systems that refuse to make the user wait for work that does not belong on the critical path.

Notes from the build

Get more AI engineering insights

Follow the work: AI tools, browser products, product decisions, and honest lessons from the build.

By subscribing, you agree to receive Orlando's emails. No spam. Unsubscribe anytime.

Keep Slow Work Off the Real-Time Path | Orlando Ascanio