Exporting and importing on back(d)

Moving an app between instances

Mon, 01 Jan 0001 00:00:00 +0000

This is the whole journey of moving an app, users included, from an instance you call old to one you call new.

Prepare the new instance. Its CONFIG_DIR needs the same databases and collections as the old one (the schemas may differ: the import tells you what the new ones refuse). Provision it (backd-server provision, or start it) and make sure the realm has authentication enabled and, if documents hold files, a storage: section (and a public_storage: one if they hold public files) whose secrets are set.

Seeding staging

Mon, 01 Jan 0001 00:00:00 +0000

A staging or test instance is only as safe as the data in it. This page is about making that data with backd export --anonymize; the import is the one of moving an app.

A staging or test instance is only as safe as the data in it. --anonymize turns an export into a copy that holds no real person’s details, using what you already declared for erasing a user:

backd export --realm clinic --include auth --anonymize --anonymize-config staging.yaml \
 --staging-password env:STAGING_PASS --out ./clinic-staging
backd import --realm clinic --from ./clinic-staging # on the staging instance

Collections. Each collection’s on_owner_delete remove and replace are applied to every document, whoever owns it. They were already checked against the schema, so the documents stay valid. The fields of pull and unset that hold an email are rewritten with the same addresses as the users’ (an address that belongs to no user becomes unknown-<hash>@example.invalid). A collection without a policy is exported as it is: write one, or use the override file.

Encrypted fields

Mon, 01 Jan 0001 00:00:00 +0000

Encrypted fields are there so that a dump, a backup or a package can’t be read without their keys. An export keeps that promise: the server never decrypts for an export. Whatever you can read of an encrypted field in a package, you read because you supplied its key, in the command on your machine.

What a package holds

backd export --realm clinic --out ./clinic

Every encrypted value is in the package as it is stored (ciphertext), with _system/data_keys.jsonl, the wrapped data and index keys of those fields, and the manifest lists each field with the name and fingerprint of its key and its form:

Reporting extracts and CSV

Mon, 01 Jan 0001 00:00:00 +0000

To hand an analyst a table, not a package, narrow the export and ask for CSV:

backd export --realm clinic --db main --collection patients \
 --fields name,birth_year,address --where '{"status": "active"}' \
 --format csv --out ./active-patients

--fields keeps only those paths of every document (id is always there; _meta.owner and the other _meta fields can be asked for, and are left out otherwise). A name that is an object, like address, keeps all of it. A path that exists in none of the exported collections is refused. A package made with --fields is marked partial in its manifest.
--where takes the same JSON filter as the data API’s where and needs --db and exactly one --collection. It is checked against the collection’s schema before anything is read; documents in the trash match like any other.
--format csv writes <database>/<collection>.csv, one RFC 4180 file per collection, and a manifest with their checksums. No system data and no files go in it.

The CSV is a flat view and loses information, so it can never be imported:

Direct mode: without a running backd

Mon, 01 Jan 0001 00:00:00 +0000

backd-server’s export and import run the same export and import as the CLI’s in the command itself, straight on MongoDB and CONFIG_DIR, with no running server: use them when the server is stopped or can’t be reached, or to read a backup. They take the CLI’s flags, plus these:

backd-server export --mongo-uri env:MONGO_URI --realm clinic --no-files --out ./clinic-offline
backd-server import --mongo-uri env:MONGO_URI --realm clinic --from ./clinic-offline

Flag	Meaning
`--mongo-uri`	The MongoDB URI, or `env:<VAR>` / `file:<path>`; default `BACKD_MONGO_URI`, then `MONGO_URI` (`CONFIG_DIR` is the environment variable, as for `serve`)
`--master-key`	The secrets master key (`env:` / `file:`); default `BACKD_SECRETS_KEY`
`--storage-key`, `--storage-secret`	The storage credentials, instead of the realm’s secrets (`env:` / `file:`)
`--public-storage-key`, `--public-storage-secret`	The same for the realm’s public storage, if it has one
`--force`	Import while an instance serves the realm

What it needs. CONFIG_DIR is read to know the schemas, the file fields and the indexes; the functions’ bundles are not built, so no Deno is needed. The realm must have authentication enabled. The command validates and writes exactly as the network path does, and nothing is provisioned or changed in MongoDB’s structure.
The master key is not needed for data. It is needed for two things: files, because the storage credentials are realm secrets (give --storage-key and --storage-secret, and the --public-storage-* pair for public files, instead, or leave files out with --no-files), and the authentication data sealed by the realm (authenticator secrets, push device tokens, Apple tokens) with --include auth: without --master-key they stay behind and the command says how many.
Refuses while the realm is served. Every backd-server serve records a heartbeat in MongoDB (backd___deployment.instances, every 15 seconds); an offline import will not write while an instance that serves the realm was heard from in the last 45 seconds, because two writers could clash. --force writes anyway, with a warning. An export only reads and never refuses, but it is not a snapshot while the realm is being written.
Audited, as offline. The records go into the realm’s audit trail with the actor cli:offline and offline: true, like any other.
No credentials are asked for. Whoever holds MongoDB and the configuration already holds more than any admin account: protect that access accordingly.

Reading an old backup

You do not need a separate tool to read a mongodump: restore it into any MongoDB with mongorestore, point --mongo-uri at it and run backd-server export with the configuration the backup was made under. The result is an ordinary package you can inspect, import into another instance or hand to an analyst as CSV.

Reference

Mon, 01 Jan 0001 00:00:00 +0000

Everything about the commands, in one place: the package format, every flag, the admin API and what to do about each error.

The package

A package is a directory, or one .tar.gz of it (--archive). All paths are relative, with no .., and the whole is verified against manifest.json before an import uses any of it.

clinic-2026-10-09/
 manifest.json
 main/patients.jsonl one file per collection
 main/visits.jsonl
 main/_files/<file id> the object of every file the documents reference (private and public)
 _system/data_keys.jsonl wrapped keys of encrypted fields (empty without any)
 _system/auth/users.jsonl only with --include auth
 _system/auth/identities.jsonl … orgs, org_members, org_teams, org_connections, mfa, devices

manifest.json

{
 "format": 1,
 "backd_version": "v0.10.0",
 "realm": "clinic",
 "created_at": "2026-10-09T10:15:00Z",
 "operator": "key:svc",
 "config_fingerprint": "9f2c…",
 "flags": {"partial": false, "anonymized": false, "auth": true, "no_files": false, "csv": false},
 "databases": [
 {"name": "main", "collections": [
 {"name": "patients", "documents": 1204, "part": "main/patients.jsonl"}
 ]}
 ],
 "encrypted_fields": [{"database": "main", "collection": "patients", "path": "notes", "key": "HEALTH", "fingerprint": "9f2c41d0", "form": "ciphertext"}],
 "auth": [{"name": "users", "documents": 31, "part": "_system/auth/users.jsonl"}],
 "auth_key_fingerprint": "6f42f0e47d8b8cc9",
 "files": 87,
 "checksums": {"main/patients.jsonl": "sha256:ba78…", "main/_files/fl_0123456789abcdefghij": "sha256:2c26…"}
}

Field	Meaning
`format`	The package layout version. An import refuses a format it doesn’t know
`backd_version`, `realm`, `created_at`, `operator`	Where and by whom it was made; `operator` is `cli:offline` for a direct export
`config_fingerprint`	Of the configuration it was made under; a different one on import is a warning
`flags.partial`	Made with `--fields`: an import refuses it
`flags.csv`	A CSV extract: an import refuses it
`flags.anonymized`	A staging copy; an import compares it with `BACKD_ENV`
`flags.auth`	Holds authentication data (the `auth` parts)
`flags.no_files`	Made without files (`--no-files`, an anonymized or CSV export)
`databases[].collections[]`	Each collection’s document count and the package path of its documents
`encrypted_fields[]`	Each encrypted field: its `database`, `collection`, `path`, the field key’s `key` name and `fingerprint`, and `form` (`ciphertext`, or `plaintext` once someone opened it with the key)
`auth`, `auth_key_fingerprint`	The authentication data parts, and the fingerprint of the one-time export key their sealed fields are sealed under
`checksums`	The SHA-256 of every other file, as `sha256:<hex>`

Documents

One document per line, in the API representation: id, the fields and _meta, soft-deleted documents included. Typed values use MongoDB’s relaxed Extended JSON, so nothing is lost:

Encrypted fields format

Mon, 01 Jan 0001 00:00:00 +0000

This page is the specification of field encryption’s storage format, version 1. It is complete enough to write a tool that reads an export package or a database dump with the field keys alone, with no backd and no master key, and the test vectors at the end let that tool prove it. backd’s own tests check the implementation against the same vectors, so this page and the code cannot drift apart without a test failing. How to use encrypted fields is in Field keys and encrypted fields.

Keeping packages safe

Mon, 01 Jan 0001 00:00:00 +0000

A package holds the data it was made from, and a key can open part of it. These notes are about where to keep them, how to give them to the commands and when to throw them away.

The package

Treat it like the database it came from. Even with ciphertext and anonymization it holds data. Keep it where only the people who may read the realm can, not in a shared folder, a ticket or a repository.
Permissions. Files backd writes are readable only by their owner (0600, directories 0700), and --archive makes a .tar.gz with the same permissions. Keep them that way when you copy it; copy it over an encrypted channel (scp, rclone to a private bucket with encryption), not by email.
Verify before use. An import checks every checksum before it trusts anything, so a package changed in transit or at rest is refused. That protects against damage, not against someone who can change the package (they can change the manifest and its checksums too) and not against reading it: get a package over a channel you trust.
Delete it when you are done. A package is a copy of production that nobody backs up or audits. Remove the directory or archive (and any copy) as soon as the move or report is finished; the audit trail already says what was exported and when.
Authentication data is the most sensitive. With --include auth the package holds password hashes, which let anyone with the package guess passwords offline, plus the sealed second factors and push device tokens. Keep that package for as short a time as you can.
An extract is still data. A CSV or a --fields package is a lossy view, but it can hold names and addresses: apply the same care, and use --anonymize for anything that leaves the team.

Keys

The one-time export key opens the authenticator secrets, device tokens and Apple tokens of a package made with --include auth. It is shown once and backd keeps no copy; the manifest holds only its fingerprint. Store it apart from the package (a password manager, a secrets store), never inside the same archive or the same ticket, and delete it with the package.
Raw keys in commands. key:<value> puts a key in your shell history and in the process list, where other users of the machine can read it. Prefer env:<VAR>, which keeps it out of both (set it in the shell, not on the command line), or file:<path> with permissions for your user only: a file readable by others draws a warning. The same applies to --staging-password, --mongo-uri (which holds credentials) and --master-key.
--auth-key-out writes the key to a file readable only by you instead of printing it, so it never lands in a terminal’s scrollback or a CI log. Use it in scripts.
The master key (BACKD_SECRETS_KEY) never leaves an instance and is never in a package. A direct export needs it only for what the realm sealed; give it as env: or file:.
In CI, pass keys as masked secrets through environment variables, never on the command line, and make sure the job doesn’t store the package as an artifact.

What an operator can do

Through the admin API, only a full administrator can export or import, with a fresh proof of identity for sessions, and every start and end is audited with counts, never content. Authentication data is a separate, explicit choice audited as export.auth.
In direct mode nobody is asked for a credential, because whoever holds MongoDB and CONFIG_DIR already holds more than an admin account. The records are still written, as cli:offline. Protect that access like the database itself.
An anonymized export applies only the rules you declared; see Seeding staging before sharing a copy.

Exporting and importing on back(d)

Moving an app between instances

Seeding staging

Encrypted fields

What a package holds#

Reporting extracts and CSV

Direct mode: without a running backd

Reading an old backup#

Reference

The package#

manifest.json#

Documents#

Encrypted fields format

Keeping packages safe

The package#

Keys#

What an operator can do#

What a package holds

Reading an old backup

The package

manifest.json

Documents

The package

Keys

What an operator can do