Realtime

A collection that declares realtime: true can be followed live. A client opens one stream for the whole realm with POST /v1/{realm}/_realtime, names what it follows, and receives the changes the caller may read as server-sent events.

POST /v1/shop/_realtime
Authorization: Bearer <session or API key>
Content-Type: application/json

{
  "subscriptions": [
    { "id": "mine",  "db": "main", "collection": "orders", "where": { "status": "open" } },
    { "id": "order", "db": "main", "collection": "orders", "doc": "o_17" }
  ]
}

Read it with fetch and a streaming body: EventSource can’t send an Authorization header, and sessions never go in URLs.

Live queries, not a change log#

A stream is a live query: it tells a client how its own view of a collection changes, not what happened to the collection. That is why events are per subscriber:

• A document that becomes readable to you (a teammate assigns it to you, an update makes it match your where) arrives as created, even though it was only updated.
• A document that stops matching arrives as removed, even though it still exists. removed means deleted or no longer visible to you, never “deleted for everyone”.
• A change to a document you never could read produces nothing: you can’t learn that it exists.

If you need an audit of everything that happened, use triggers or the audit trail; a stream shows state.

Events#

event: ready
data: {"subscriptions":["mine","order"]}

id: eyJw…
event: created
data: {"sub":"mine","doc":{"id":"o_9","status":"open","_meta":{"version":1}}}

event: removed
data: {"sub":"mine","id":"o_9","version":3}

event: reset
data: {"subs":["mine"],"reason":"slow_consumer"}

: heartbeat

• Documents are what a read returns for this caller: the same fields, _meta.version included, no _expanded.
• Strangers learn nothing. A change to a document the caller couldn’t read before or after it produces no event at all, not even an id.
• reset reasons. history_expired (the change history no longer reaches the stream, or a resume token is too old), slow_consumer (the client didn’t keep up and its buffer was dropped), preimage_missing (a change arrived without the image needed to tell who could see it, so nothing is guessed), config_changed (the configuration the subscription depended on changed: the subscription is gone and must be made again; a collection that lost realtime: answers realtime_not_enabled) and access_changed (reserved for access changes).

Subscriptions#

order_by, limit, page and expand are not part of a stream: ordering and loading belong to the client.

Following the configuration#

A subscription with config instead of db and collection follows the realm’s app configuration. It needs a realm that declares one, not a realtime: true collection, and may be the only subscription of the stream. A stream has at most one, and it counts toward the limit of subscriptions.

{ "subscriptions": [{ "id": "cfg", "config": { "paths": ["maintenance", "features"] } }] }

When the configuration changes, or the caller’s roles or consents do, the stream sends:

event: config_changed
data: {"id":"cfg","version":"c42","paths":["maintenance.enabled"]}

• paths are the keys whose value as this caller reads it differs from the last time they were told: a key their rules hide never appears, and a change that concerns nothing they read sends no event. An empty list means “read it all again”: backd could not tell what changed, and nothing is guessed.
• version is the configuration version the paths were computed against.
• Not the reset reason of the same name. That one (reset with reason: "config_changed") says a collection’s rules or realtime: opt-in changed; this event is about the app configuration’s values.
• Not resumable: config_changed carries no resume token and is not replayed. After a reconnection, read the configuration again (with If-None-Match, usually a 304); the JavaScript client does it for you.

Start from a known state#

1. Open the stream and wait for ready.
2. Load each subscription’s data with normal list or get calls, keeping the events that arrive meanwhile.
3. Apply the buffered events, then the live ones, ignoring any event whose _meta.version is not newer than the document you hold. Replays and races are then harmless.

Handling reset#

reset says the stream can no longer promise the client has the right state for some subscriptions. The remedy is always the same: read those subscriptions again. The JavaScript client does it for you, with a random wait so that many clients reset by one cause don’t read at once.

Resuming#

The last event of every change carries an SSE id: a resume token. After a disconnect, open a stream again with resume_from (or the Last-Event-ID header) set to the last token you saw, and the stream continues after it.

• A token is signed and bound to the realm, the caller and what their access depended on when it was issued: after a role, membership or team change a token is not replayed (the stream says reset with history_expired: reload). Another caller’s, another realm’s or a forged token is 400 invalid_resume_token.
• Replayed events are evaluated with the caller’s current access, so a resume can never show more than a list would.
• If the history no longer reaches the token (MongoDB’s oplog moved on, or the token is more than a day old) the stream says reset with history_expired for every subscription: reload them.
• The token isn’t tied to the set of subscriptions: you may add or drop subscriptions when you resume; reconcile new ones by version.

Errors#

Limits#

An instance and a realm set them with the BACKD_REALTIME_* variables and, lowering only, the realtime: section of realm.yaml. A stream never stalls the change feed: a client that can’t keep up is reset, not waited for.

What a stream costs#

The cost has three parts, and only the first depends on your subscriptions:

• Checks per change. Each change is evaluated against the subscriptions it can matter to; the guide below shows how filters keep that small.
• Pre-change images. A realtime collection keeps the previous version of every updated or deleted document in MongoDB’s change log, for as long as the log is kept: storage and some write time that grow with document size and change rate (measured numbers). Don’t mark very large, hot collections realtime: true.
• The read of the log. One stream per instance reads the whole deployment’s changes, so it costs as much as the cluster’s write rate, whatever you subscribe to.

Every change to a realtime collection is checked once against the subscriptions it can matter to, with the caller’s where and read rule evaluated on the document before and after the change. Subscriptions are filed by the equalities their filter can only be visible under, so a change finds its subscribers without asking all of them:

• A subscription to one document (doc) is found by its id.
• A where (combined with the read rule) with an equality or $in on a field, such as _meta.org, _meta.team or status, is filed under that value. inOrg, hasOrgRole, inTeam and hasTeamRole rules become _meta.org/_meta.team memberships per caller, so they are filed like plain equalities.
• A subscription with no such equality is checked on every change of its collection on that instance.

When access changes#

A stream is tied to the session, user or API key that opened it, and to the roles, memberships and teams its rules depend on.

• Sign-out, revocation, deletion or disabling of the session, user or key: session_ended and the stream closes, within the time it takes the change to reach the instance. Reconnecting with the same credential fails with 401.
• Roles, organization memberships, team roles, organization claims or require_mfa: the stream reads the caller’s access again. If the effective access to a subscription differs, that subscription gets reset with access_changed and follows the new access from then on; if it is the same, nothing is sent. Changes of an organization are coalesced into at most one reload per stream every quarter second.
• The app configuration: a stream whose rules read it (appConfig(...)) reloads when the configuration changes. Time: a role grant’s window (from and until of a team role, or of an organization role) opening or closing has no event in the database, so the stream reads the caller’s access again at that moment: the subscription that changes gets its reset within a fraction of a second of the boundary. A rule that reads now has no boundary to tell, so open streams also read their access every five minutes.
• A parent changed. A stream on a collection whose read rule follows a reference with parent() (the items of a list) reads the caller’s access again when a document of the parent collection changes: the stream is told with reset (access_changed) when the lists that qualify for the caller are no longer the same, for example when they are added to a list or removed from it, and the items of the new lists are what it follows from then on. Both collections need realtime: true: the parent’s changes arrive through the same change feed. Reloads are coalesced like those of an organization.
• Credential lifetime: a stream never outlives its session or key: at its expiry the stream says reauth and closes.

A rule that depends on the clock (now) is a different filter at each reload, so such a stream may get a reset it didn’t need: it is harmless and costs one reload of that subscription.

The JavaScript and Dart clients do all of the above for you.