Operations
MongoDB, provisioning, health checks, logging, timeouts, shutdown, and running backd in production.
MongoDB
backd needs MongoDB 8.2 running as a replica set. A single node is fine. Replica sets support transactions, which atomic batch writes rely on, and every setup backd is tested with uses one. Sharded clusters (through mongos) work too.
At startup, backd asks the server what it is. It refuses to start against a standalone server:
mongodb: the server is a standalone instance; backd needs a replica set (a single node is fine): start mongod with --replSet and run rs.initiate(), then add ?replicaSet=<name> to MONGO_URI
Managed services such as MongoDB Atlas always run replica sets. To run a single node yourself:
docker run -d --name mongo -p 127.0.0.1:27017:27017 docker.io/library/mongo:8.2 --replSet rs0
docker exec mongo mongosh --quiet --eval \
"rs.initiate({_id: 'rs0', members: [{_id: 0, host: 'localhost:27017'}]})"
export MONGO_URI='mongodb://localhost:27017/?replicaSet=rs0'
- The member’s
hostis the address clients use to reach it:localhost:27017for a server on your machine, the service name (such asmongo:27017) inside a compose network. replicaSet=rs0inMONGO_URInames the set. Without it, the driver still discovers the set from the host you give.- With authentication, a replica set also needs a key file (
--keyFile), even with one node. The production reference shows how. make exampleand the test setups start a single-node replica set and initiate it from the container’s health check.
Upgrading an existing standalone server
Restart it as a replica set and initiate it once; the data stays in place:
- Stop
backd. - Restart
mongodwith--replSet rs0(and--keyFileif authentication is on). - Run
rs.initiate({_id: 'rs0', members: [{_id: 0, host: '<host>:27017'}]})once, as the admin user if authentication is on. - Add
replicaSet=rs0toMONGO_URIand startbackd.
Provisioning
At startup, backd makes sure MongoDB matches the config. What it does depends on PROVISION_MODE.
apply (default)
- Creates each missing collection with its
$jsonSchemavalidator. MongoDB creates the database implicitly. - Updates (
collMod) validators that differ fromschema.json. - Creates indexes declared in
indexes.jsonthat don’t exist yet. - Creates or updates the system database of every realm with
auth: enabled. - Records the config’s fingerprint for each realm (see Deploying config).
- Is idempotent and safe to run on every start.
- If a declared index exists with a different
uniquesetting,applystops with an error rather than dropping it. Drop the index manually and restart. - Creating a unique index fails if the collection already holds duplicate values. The error names the index.
verify
Only compares collections, validators and indexes with the config, including the system databases.
If anything differs, it refuses to start and lists every difference. For example:
backd serve: MongoDB does not match the config (run `backd provision`): shop__orders.items: collection missing shop__orders.clients: validator differs from schema.json shop__orders.clients: index (email) missingThen compares the config’s fingerprint with the one provisioning recorded for each realm, and refuses to start if it differs or none was recorded: every instance must run the config that was provisioned (see Deploying config).
Needs only read/write privileges on the data, and read access to
backd___deployment.realms.
Common to both modes
- Validators always use
validationLevel: "moderate"andvalidationAction: "error". - Nothing is ever dropped. Some databases, collections and indexes exist in MongoDB but not in the config: databases named like
<realm>__<database>, collections inside configured databases, or indexes not inindexes.json.backddoesn’t serve those databases and collections, keeps those indexes, and logs a warning for each. The same goes for a<realm>___systemdatabase whose realm is gone or hasauth: disabled: it is kept and reported, never used. - Schema keywords that MongoDB can’t enforce are logged at
infolevel (see Schema validation).
backd provision runs apply once and exits.
Both backd provision and backd serve then apply the role assignments seeded in each realm’s realm.yaml.
Realm system databases
Each realm with auth: enabled has a system database named <realm>___system (three underscores). It holds the realm’s users and credentials, managed through the admin API (for example with backd user); only backd bootstrap writes to it directly. Realm names must be short enough for this name to stay under MongoDB’s 64-character limit, so a realm name can be at most 54 characters.
| Collection | Holds | Indexes |
|---|---|---|
users | email, verified flag, roles, disabled flag | email (unique, sparse) |
identities | sign-in methods per user: a password hash today, external providers later | provider + subject (unique), user_id |
sessions | sessions: public id, hashed token, last use and expiry | token_hash (unique), user_id, expires_at (TTL) |
api_keys | hashed API keys with name, first characters, last use and expiry | name (unique) |
invitations | invitations: public id, hashed token, optional email | token_hash (unique), expires_at (TTL) |
login_attempts | failed-login counters per email and client address; reused, under their own key namespace, for function rate limits | expires_at (TTL) |
audit | the audit trail: append-only records of security-sensitive actions | at, target + at, expires_at (TTL) |
secrets | function secrets, encrypted at rest (never their plaintext values) | database + name (unique) |
invocations | function invocation history: append-only, what happened — never a call’s input or output | function + at, request_id (sparse), expires_at (TTL) |
jobs | async function jobs: input, status, and once done, result — the one exception to never storing input/output, since there’s nowhere else to keep it until read | status + lease_expires + created_at, expires_at (TTL) |
idempotency | Idempotency-Key claims: scoped to the function and caller, a sync call’s stored response or an async call’s job id | expires_at (TTL) |
Each collection has a $jsonSchema validator that checks its required fields and their types. The TTL indexes make MongoDB delete expired sessions, invitations, counters, audit records, invocation records, jobs and idempotency claims automatically. If one of these indexes exists with different options (for example expires_at without TTL, so sessions would never expire), apply and verify fail and name it; drop the index and let backd recreate it.
Treat the system databases as sensitive: they contain password hashes and token hashes. Back them up with the rest of the realm’s data, encrypted: only role seeds can be rebuilt from the config. Tokens and keys are stored only as hashes, so a leaked backup doesn’t reveal usable credentials, but password hashes can still be attacked offline. See Backup and restore.
Least-privilege deployments
The service’s MongoDB user doesn’t need to create collections, change validators or build indexes. Run provisioning as a separate deploy step instead:
- Run
backd provisionwith an admin-privilegedMONGO_URI. - Grant the service’s user
find,insert,update,removeandlistIndexeson each collection thatbackd databases --collectionsprints, andlistCollectionson their databases. MongoDB privileges can’t match names by pattern, so repeat this step whenever the config changes. - Start the service with
PROVISION_MODE=verifyand that user.
Grant collections rather than MongoDB’s built-in readWrite role on whole databases: readWrite also allows creating, dropping and re-indexing collections, and MongoDB lets insert on a database create any collection in it. The production reference automates all three steps.
backd databases needs only CONFIG_DIR. It prints every data database (<realm>__<database>) and the system database (<realm>___system) of every realm with auth: enabled. With --collections, it prints every collection as <database>.<collection>, including the system collections:
$ CONFIG_DIR=./config backd databases
blog___system
blog__main
shop__orders
$ CONFIG_DIR=./config backd databases --collections
blog___system.api_keys
…
blog__main.posts
shop__orders.items
Upgrading to v0.9.0
backdwithout a command prints the help; it no longer serves. Before,backdalone ranserve. Anything that starts the binary or the container image with no command must sayservenow: acommand: ["serve"]in a compose file,args: ["serve"]in Kubernetes,docker run … backd serve,ExecStart=/usr/local/bin/backd serve. The images now haveCMD ["serve"], sodocker run backdand a container that overrides nothing still serve, but a service that sets its ownentrypointor runs the binary directly does not. The shipped compose files say it explicitly.rules.yamlis gone: the rules are therules:section of the collection’scollection.yaml. A configuration directory with arules.yamlno longer loads:backdstops at startup and inbackd config check, naming the file. Migrate each one by moving its content, indented by two spaces, under arules:key in the collection’scollection.yaml(create the file if there is none; its other sections,files:,soft_delete:andon_owner_delete:, stay as they are) and deletingrules.yaml. The rules mean exactly what they meant.rules.test.yamlstays where it is. The config fingerprint changes (it hashescollection.yamlnow), and the admin API’s configuration view givescollection_fileandcollection_yamlwhere it gaverules_file; the admin interface prepares a change tocollection.yaml.backd files regenerateis new (see File fields);backdandbackd helpshow the commands grouped by who uses them.
Upgrading to v0.8.1
v0.8.1 adds image versions, storage quotas and direct uploads in parts, and it has breaking changes: the first two points need your action, the rest are worth a look. Nothing else that worked on v0.8.0 changes. What to check:
- Move
rules.yamlintocollection.yaml. The rules are now itsrules:section (same keys). A collection directory that still has arules.yamldoes not load, so a forgotten file can’t leave a collection without rules: indent the file’s content two spaces underrules:incollection.yamland delete it.rules.test.yamlstays where it is. See rules. backdwithout a command no longer serves. It prints its help and exits 0. The images already runserve; a Dockerfile, compose file or unit that startedbackdalone must saybackd serve.- JPEG and PNG originals lose their metadata by default (EXIF with the GPS position, XMP, comments, PNG text), pixels untouched and orientation kept. Set
keep_metadata: trueon a field to keep it. A picture too damaged to be copied is refused with422 invalid_image. Files stored before the upgrade are not changed. See Metadata and the GPS risk. - Image versions and quotas are opt-in:
versions:under a file field (workers make the copies, so run them;?version=serves them andbackd files regenerateremakes them after a declaration changes) andstorage.quotainrealm.yaml. - Direct uploads above 5 GiB need a provider that checks a SHA-256 per part: AWS S3, Cloudflare R2 and MinIO. With DigitalOcean Spaces
max_sizeof a direct field stays at 5 GiB. A file sent in parts has asha256that is the digest of its parts’ digests, not of its bytes (why). Workers abort the uploads left unfinished. - Sizes accept
TBandTiB.GET …/_files/{field}addsmultipart_abovefor the fields that can hold more; the start of a direct upload takespart_sha256instead ofsha256for such a file. - API additions:
POST /_admin/config/check(a change checked against the realm’s whole configuration, without applying it) and the image version routes. The admin UI’s schema and rules editors use the first. - The JavaScript client is 0.8.0 (
npm install backd-js@latest):uploadFilesends large files in parts by itself, plus the version helpers andadmin.checkConfig(). It works with a v0.8.0 server for everything that existed before.
Upgrading to v0.8.0
v0.8.0 adds files, schema checks, on_complete and job steps, and more control over schedules (pausing, timezone, overlap: skip). Configuration that worked on v0.7.0 still loads, except for the one case in point 2; each new feature is off until you use it. What to check:
- Provision before rolling out, as always. New system collections are created in each realm’s
___systemdatabase:file_journal,file_deletions,storage_usage,schema_checksandschedules.PROVISION_MODE=verifyrefuses to start untilbackd provisionhas made them. - A file field can’t be declared in
schema.json. Files are declared underfiles:incollection.yaml(backd adds their schema), and startup refuses a collection that has the same property in both, and one withfiles:in a realm withoutstorage:. If you used a property to hold file details, move it. The tutorial’sassets.fileplaceholder moved this way. - Files are opt-in, and need more than configuration. A realm declares
storage:inrealm.yamland sets its two keys as realm secrets (backd secret set, withBACKD_SECRETS_KEYon the instance); without them the realm’s file endpoints answer503 storage_unavailableand reads of its documents come without links. Run workers (backd worker, or--with-worker): they delete the objects of replaced, removed and abandoned files and finish uploads a crash interrupted. SetBACKD_URLin production, since backd-signed links name it.BACKD_MAX_UPLOAD_BYTES(default 100 MiB) caps proxy uploads. - Give each backd instance its own storage
prefix, and read Security of files: links and caches outlive later rule changes, and a function can’t declare the storage secrets orBACKD_FILES_LINK_KEYas its own (startup refuses it). - Erasing a user now also queues the files of the documents the policy deletes or anonymizes (a
removeof a file field). ADELETE, a batch delete and a purge queue the document’s files too. MongoDB’s own retention of a soft-deleted document can’t:backd storage reconcilefinds those objects. - The admin UI gains the file controls and schema checks. Its
Content-Security-Policynow also allows images and direct uploads from the origin of the realms’ file storage (and from no other origin besides its own); a stack without storage has the policy it had. - API and CLI additions: the
_filesroutes and?file_links=trueon reads;GET /_admin/storage,POST /_admin/storage/checkand/reconcile;/_admin/data-checks;/_admin/schedules;backd storage check|usage|reconcile,backd data check|checks,backd functions schedules|pause|resume. New error codes:too_many_files,upload_mode_mismatch,unsupported_file_type,file_missing,invalid_file_link,upload_mismatch,file_not_uploadedandstorage_unavailable. Theownedreport listsreplaceas[], no longernull, for a policy that replaces nothing. - The tutorial’s stack now runs a MinIO: fetch
compose.yamlandnginx.confagain before chapter 13. - The JavaScript client is 0.7.0 (
npm install backd-js@latest):files(id, field),uploadFile,prepareUpload,fileUrl,downloadFile,deleteFile,fileLinks,admin.storage.status()andreconcile(),admin.dataChecks,admin.schedules,job.stepsandwait({ onProgress }). It works with a v0.7.0 server for everything that existed before.
Upgrading to v0.7.0
v0.7.0 adds the admin web interface, a read-only admin level with admin.read_access, an audited admin data route, a read-only configuration view in the admin API, and a switch that turns the admin API off per instance. Configuration that worked on v0.6.0 still loads; the interface is off until you switch it on. What to check:
admin: trueand admin API keys now reach documents. The new admin data route (/v1/{realm}/_admin/data/…) lists, reads and changes documents past the collection’s access rules, and it belongs to everyadmin: truerole and every admin API key. Writes are audited without their content; reads are not (they carry the actor in the access log). Before upgrading, look at who holds those roles and keys: give people who don’t need documents an area list (admin: [users, invitations]), which doesn’t include the newdataarea, and keep admin keys to the tooling that needs them.- Two new areas. An area list can now also name
dataandconfig; a role that lists areas and doesn’t name them doesn’t get them.configshows what the instance runs for the realm (GET /_admin/config) and every read-only level has it. - The read-only level is new.
admin: read(or[read, secrets]) reads every area except users and data, which a realm opts in to withadmin.read_access.usersandadmin.read_access.data(bothfalseby default).GET /_admin/whoamitells a client what the signed-in level may do. - The admin API can be switched off per instance:
BACKD_ADMIN_API=falseleaves every/_adminroute out (404). The production reference now splits a publicbackd(no admin API) from an internalbackd-admin, and the CLI’sBACKD_URLpoints at the internal one: if you built ondeploy/production, compare your compose file with it. - The admin web interface is off by default.
BACKD_ADMIN_UI=trueserves it at/_ui/from an instance that also has the admin API (the combination withBACKD_ADMIN_API=falsestops startup);BACKD_ADMIN_UI_IDLE(default30m) signs an idle administrator out. Release images and binaries embed it; a binary built from source needsmake ui(Node) first, and without it/_ui/answers404and startup warns. Serve it only from the internal instance, and read its security review and the hardening checklist first. - Smaller changes:
GET /_admin/userstakesq(a search of at most 254 characters),GET /_admin/users/{id}/sessionsandDELETE …/sessions/{session_id}list and end a user’s sessions. The example stack serves/_ui/and has anadminuirealm for it. - The JavaScript client is 0.6.0 (
npm install backd-js@latest):admin.whoami(),admin.config(),admin.data(database, collection),admin.users.sessions()andrevokeSession(), theqsearch onadmin.users.list()and theoriginfilter onadmin.jobs.list(). It works with a v0.6.0 server for everything that existed before.
Upgrading to v0.6.0
v0.6.0 adds soft delete, admin rights per role, Prefer: respond-async and login_throttle. Configuration that worked on v0.5.0 still loads and behaves the same; each new feature is off until you use it. What to check:
- Provision before rolling out, as always. A collection with
soft_deletegets its unique indexes rebuilt with_meta.deleted_atas the last key and, with aretention, a TTL index on_meta.purge_at;PROVISION_MODE=verifyrefuses to start untilbackd provisionhas created them. Collections that don’t opt in are untouched. - Turning soft delete on for a collection that already has a unique index: the old index stays (
backdnever drops one) and also covers deleted documents, so a deleted document’s values stay taken until you drop it.backd provisionwarns about it by name; drop it once the new index exists. - Admin roles:
admin: trueis what it was. A role can now list areas (users,invitations,apikeys,secrets,audit,functions) and an administrator can no longer hand out, or change a user who holds, rights they don’t have themselves; admin API keys are created and revoked only by full administrators. Keep one role withadmin: true:backd bootstrapneeds it, and startup warns when a realm has none. restoreandpurgeare new rule keys for collections that soft-delete. They are denied unless declared andwritedoesn’t cover them; declaring one in a collection withoutsoft_deletestops startup.backd rules testfixtures takerestoreandpurgeassertions and_meta.deleted_at.- Two query parameters are now meaningful:
deletedon list and get, andpurgeonDELETE. On a collection that doesn’t soft-delete they still answer400, with a message that says so. Prefer: respond-asyncis per call. Nothing changes for callers who don’t send it. Browser apps on another origin can now sendIdempotency-KeyandPreferand readPreference-AppliedandIdempotent-Replayed: the CORS allow and expose lists lacked them.login_throttleis optional; the defaults are the values the throttle always had.- The JavaScript client is 0.5.0 (
npm install backd-js@latest):deleted,purgeandrestoreon collections,respondAsynconfn(). It works with a v0.5.0 server for everything that existed before.
Upgrading to v0.5.0
v0.5.0 adds cursor pagination, API key scopes, session cookies, Idempotency-Key on creates, dates stored as dates and backd rules test. Configuration that worked on v0.4.0 still loads; every new feature is off until you use it. What to check:
- Provision before rolling out, as always: the
api_keyscollection of each realm’s system database gains thescopesfield in its validator, and each collection with a field marked"x-backd-store": "date"getsbsonType: datefor it.PROVISION_MODE=verifyrefuses to start untilbackd provisionhas run. - API key scopes are opt-in. Existing keys have no scopes and keep full access. A key created with
scopescan only do what they grant (read,writeorcall, on everything, a database or a collection or function), and answers403otherwise. x-backd-store: datedoes not convert existing data. Documents written before keep text in that field until they are written again, and date queries skip them until then; the dates section has amongoshone-liner to convert in one go.- Session cookies are opt-in per realm and per login (
sessions.cookie.enabled, then"cookie": trueon the login). A cookie login turns on an origin check for writes that carry only the cookie, so list your app’s origin incors.origins. Idempotency-Keyon creates needs authentication. A create or batch with the header answers400for anonymous callers and in realms withauth: disabled; without the header nothing changes. Answers are kept for 24 hours in the realm’s existingidempotencycollection.- The admin user list is cut by MongoDB now (
GET /_admin/usersreturnsnext_cursor, withafterto continue;skipstill works). The command line and the JavaScript client already follow it; a script that read the whole list in one call must follownext_cursorfor realms with more users than one page. - Two limits are stricter, one is new. The login throttle now serializes the password checks of an account across instances, and answers
429withRetry-After: 1if it waits more than 5 seconds for the lock.sessions.admin_idle_timeoutandsessions.admin_max_lifetime(unset: nothing changes) shorten the sessions of users who hold an admin role. See the hardening checklist. - The JavaScript client is 0.4.0 (
npm install backd-js@latest):afteranditeratefollow cursors,cookies: true,scopeswhen creating keys, andidempotencyKeyoncreateandbatch(keyed requests are retried by the client). It is versioned apart from the server, and works with a v0.4.0 server for everything that existed before. - If you copied the template CI (
backd template project), add a stepbackd rules test(withCONFIG_DIR: ./config) afterbackd config check, as the template now does: it checks your access rules against the cases in eachrules.test.yaml, with no database.
Upgrading to v0.4.0
v0.4.0 adds email flows, erasure, metrics and the npm client. Configuration that worked on v0.3.0 still loads and nothing new is required, but two behaviors change, so read the first two items.
- Deleting a user means something else.
DELETE /_auth/me(a user deleting their own account) now deactivates it: the account is disabled and its sessions end, but the data and the email address are kept, and an administrator can reactivate it.DELETE /_admin/users/{id}andbackd user deletenow erase: the user becomes a tombstone (erased-<id>@erased.invalid, so the real address is free again) and a worker applies thecollection.yamlpolicy of each collection that declares one; they answer202with an erase job, andbackd user deletewaits for it. Before, both removed the user and left every document alone. To get that effect now, declare no policy: an erase without policies leaves the documents as they are. A worker must be running (backd worker, orserve --with-worker); see Deleting and erasing users. - Provision before rolling out, as always: each realm’s system database gains collections and indexes (email tokens, an index of erase jobs), and
PROVISION_MODE=verifyrefuses to start without them. The worker now claims jobs in every realm with authentication, not only those with functions. - Some addresses are refused. Addresses on the reserved
.invaliddomain, with a comma, semicolon, colon, angle bracket, parenthesis, square bracket, quote or backslash, or longer than 254 characters no longer pass, at sign-up and when looked up. An existing account with such an address can’t sign in by email until an administrator changes the address in the database. Ordinary addresses are not affected. - If you copied the production reference: the stricter rate-limit zone now covers password reset, verification, invitations and address changes (
nginx/backd.conf.template), and it gained a Prometheus. Run itssetup.shagain: it addsBACKD_METRICS_TOKENto your.envandsecrets/metrics-token. - The JavaScript client is on npm as
backd-js(npm install backd-js). It was never published as@backd/client: if you installed it from a checkout, change the import. Versions ofbackd-jsbefore 0.2 were an unrelated, older library. - Metrics, email and the
accountsettings are all off by default. Turn them on when you want them:METRICS_ADDR, theemailsection ofrealm.yamlandaccount.*.
Upgrading to v0.2.0
v0.2.0 adds users and access rules and is a breaking release: every realm needs a realm.yaml, and authentication is on by default. To keep an existing realm working as before on a private network, give it a realm.yaml with auth: disabled; to protect it, provision it and give server-side clients API keys.
- Add a
realm.yamlto every realm. To keep a realm working exactly as before, without authentication, putauth: disabledin it. Only do this for realms that stay on a private network: anyone who can reach them can read and modify their data. - Or turn authentication on (the default,
auth: enabled), declare an admin role inrealm.yaml(roles: {admin: {admin: true}}), then:- run
backd provision(or startbackd servewithPROVISION_MODE=apply) to create the realm’s system database; - create the realm’s first administrator with
backd bootstrap --realm <realm> --email <email>, and log in as them withbackd login --realm <realm> --url <server>; - give server-side clients an API key:
backd apikey create --realm <realm> --name <name> --expires 90d, sent asAuthorization: Bearer <key>; - add a
rules:section to thecollection.yamlof collections that users or anonymous callers should reach. Without one, only API keys can.
- run
- Existing documents don’t have ownership fields. They keep working:
_meta.ownerreads asnull, so owner-based rules don’t match them, and their next write adds_meta.updated_by. API keys can still manage them. - Behind a reverse proxy, set
TRUSTED_PROXIESso login throttling sees real client addresses. - Browser apps on another origin need that origin under
cors.originsinrealm.yaml.
Health checks
Use GET /healthz as the liveness probe and GET /readyz as the readiness probe (see HTTP API).
Logging
Logs are JSON lines on standard error, filtered by LOG_LEVEL.
- At startup,
backdlogs aWARNline for each realm withauth: disabled(its data is open to anyone who can reach the service), and for each realm withauth: enabledbut no rules in any collection (only API keys can reach its data). It warns about realms withauth: enabledthat nobody can administer with a session (no admin role, or no user holding one). It also lists API keys that never expire or expire within 14 days (see the expiry policy). - Each request produces one access-log line (
"msg":"request"). It includes the method, route pattern, realm, database, collection, status, duration, bytes,request_idand the client’s address (client, followingTRUSTED_PROXIES), plusactorin realms withauth: enabled:user:<id>for a session,key:<name>for an API key,key:<name> as user:<id>for an API key acting on behalf of a user, oranonymous. - At
debuglevel, requests refused by access rules are logged ("msg":"access denied") with the collection, operation and reason. - Request bodies, document contents, passwords and tokens are never logged.
Timeouts
| Timeout | Value |
|---|---|
| Storage work per document request | MONGO_OP_TIMEOUT (default 10s). Exceeding it returns 503 unavailable |
| Reading request headers | 10s |
| Reading the whole request | 30s |
| Writing the response | MONGO_OP_TIMEOUT + 10s, at least 30s |
| Idle keep-alive connections | 120s |
Shutdown
On SIGINT or SIGTERM, backd:
- closes its listener at once, so new connections and readiness probes are refused;
- lets in-flight requests finish, for up to
SHUTDOWN_TIMEOUT(default 15s); - closes the MongoDB client and exits with status 0.
Requests still running after the timeout are aborted, and backd exits with a non-zero status.
Give your process manager a stop grace period longer than SHUTDOWN_TIMEOUT:
- the included
docker-compose.ymlusesstop_grace_period: 20s; - Docker’s default is 10s;
- Kubernetes’ default
terminationGracePeriodSecondsis 30s.
backd serves plain HTTP. Terminate TLS in front of it with a reverse proxy, ingress or load balancer. The production reference deployment does this and everything else below. Before going public, go through the hardening checklist. Before exposing a realm, also go through the security checklist.
Rate limiting and abuse
backd doesn’t limit request rates, except for failed logins. Put a reverse proxy, API gateway or CDN in front of it and limit there (the production reference has a tested nginx configuration):
- Sign-up: in realms with
signup: open, limitPOST /v1/{realm}/_auth/signupper client address, or usesignup: invite. - Public data: collections whose
readrule lets anonymous callers in can be scraped; limit or cache them. - Everything else: a general per-address limit protects MongoDB from floods.
Password hashing is bounded by PASSWORD_HASH_CONCURRENCY whatever the traffic: excess logins queue, then answer 503 at their deadline, instead of exhausting memory.
Client addresses behind a proxy
Login throttling counts failures per client address. Behind a proxy, every connection comes from the proxy, so set TRUSTED_PROXIES to the proxies’ exact addresses:
TRUSTED_PROXIES=172.30.80.10
List networks only if every address in them is one of your proxies. Whole private ranges such as 10.0.0.0/8 would let any other container or host in them forge X-Forwarded-For (see client addresses in the production reference).
X-Forwarded-Foris only believed when the connection comes from a trusted proxy. Otherwise clients could pick any address they like.- The client is the right-most address in
X-Forwarded-Forthat isn’t a trusted proxy. Entries further left were supplied by the client and are ignored. - The proxy in front of
backdmust overwriteX-Forwarded-Forwith the address it sees, unless it trusts a load balancer in front of it. The production reference shows how. The local stack (make example) trusts all private ranges for convenience: don’t copy that. - Without
TRUSTED_PROXIES,backduses the connection’s address. Behind a proxy, all clients then share one counter, and a flood of failures from anyone slows down logins for everyone behind that proxy.
Build CONFIG_DIR once from a tag of your config repository, deploy the same artifact everywhere, and let backd refuse instances that run anything else.
Production deploymentA tested reference deployment: TLS, rate limits, client addresses, MongoDB with authentication and TLS, least-privilege provisioning, and a hardening checklist.
MetricsPrometheus metrics from every backd process on a private port: how to turn them on, what they hold, and why the port must never be public.
Backup and restoreWhat must be backed up, encrypted backups with mongodump and age, restoring, and what to do after a restore.
Hardening checklistOne page to go through before a backd deployment faces the internet: realms, credentials, the edge, MongoDB and operations.