Scan Upload Wire Contract

The HTTP contract between beebox's scan routes (server) and its two clients: the stand-alone scan-uploader/ package (laptop client) and the Mac app's upload client (mac-app/Sources/BeeBoxMacCore/Scan/). The sides share no code — this document is the single coordination point. Every implementation site that encodes part of this contract carries the breadcrumb comment:

// WIRE CONTRACT (scan-upload): must match docs/scan-upload-contract.md — change both sides together.

Breadcrumbed sites: the server routes (src/webapp/routes/scan-upload/*.ts), the Node client's HTTP layer (scan-uploader/src/), the Mac client (mac-app/Sources/BeeBoxMacCore/Scan/ScanWire.swift, ScanSweep.swift, ScanFiles.swift and ContractVersion.swift), and the server route tests (test/webapp/routes/scan-upload.test.ts) — the tests exercise this contract exactly as the client sends it and are its executable form. Design history: implemented-plans/scanner-ingest.md.

Auth

Every request: Authorization: Bearer <scan-token>.

Scan tokens are per-box credentials minted by the boxholder (scanTokens tRPC procedures), stored hashed in .beebox/scan-tokens.secret.json. They authorize only the two routes below — verified independently by the hub (which also requires the path to be exactly one of those two routes under /<slug>/, hash form included; nothing else in the /api/scan/ subtree is reachable with a scan token) and by the box child. They are not mobile device tokens; they cannot mint session cookies, open WebSockets, or reach any other surface. A full owner identity (browser session) is also accepted on these routes.

Failures: 401 invalid/revoked token; 401 from the hub for a scan token on any non-scan path.

Hashes

SHA-256 of the file bytes, lowercase hex, 64 chars. Used as the PUT path segment, the check key, and the server's dedup/idempotency key.

POST /<box>/api/scan/check

Request body (JSON): { "hashes": ["<sha256>", …] } — batch, ≤500 entries.

Response 200 (JSON):

{ "contractVersion": 1,
  "states": { "<sha256>": { "state": "unknown" }, "<sha256>": { "state": "rejected", "reason": "…" } } }

contractVersion is the box's own contract version (see "Client identity"). It is a sibling of states, and a client that does not know about it ignores it.

Per-hash state:

pending/imported count as confirmed so a client that crashed between PUT and disposition converges on the next run. A server-side entry mid-promotion is reported as pending — the wire vocabulary stays at these four states.

An empty hashes array is legal (200, empty states). A malformed body, a malformed hash (not 64 lowercase hex), or >500 hashes → 400.

A box that cannot receive scans at all answers every request on both routes — check included — with the retryable 503 { "status": "server-error", … } below, so a client learns the box is unavailable on its first call rather than on its first upload. The one condition today is a box that has not been converted to git-annex: promotion stages raw asset bytes, which a manifest-scheme box gitignores, so an accepted there would be a lie the client acts on by deleting its only copy.

PUT /<box>/api/scan/files/<sha256>

Body: the raw file bytes, Content-Type: application/octet-stream, streamed. Content-Length required (411 without it). Malformed path hash or missing/unusable X-Upload-Filename → 400.

Headers:

Server behavior: streams to quarantine while metering bytes (over-limit → 413, partial file deleted), re-hashes, then validates (magic-byte sniff vs extension allowlist; qpdf --check for PDFs, image decode for images). Image uploads may be JPEG, PNG, TIFF, WebP, or AVIF. For .avif, the sniffer may identify the generic HEIF container (image/heif); both recognized AVIF/HEIF container labels still require a complete image decode. Other MIME/extension mismatches are rejected.

Responses (JSON, status field is the vocabulary):

Re-PUT of a rejected hash re-runs validation (the retry path after a validator fix). Re-PUT of pending/imported is a no-op duplicate. PUT is idempotent throughout — retrying any response is safe.

Server-side lifecycle (why check answers change on their own)

Accepted files sit in the box's scan quarantine until the promote worker (src/core/scan/promote/core.ts) runs — debounced two minutes after the last PUT (accepted or rejected), once at box-serve startup, and on a slow GC sweep for boxes that stop scanning — under a per-box cross-process lock. A pass imports pending files through bbx upload --as scan (materialized under their original filenames, tagged --source scan-upload/<token-name>), raises one question card per rejection, runs a supervised full bbx wakeup recorded by a durable marker until it succeeds, and garbage-collects quarantine. The GC is what makes the answers below stable rather than eventually-empty:

Client consequence: a rejected hash can eventually return to unknown. That is deliberate, and far enough out that re-sending deserves fresh validation.

Limits

50 MB per file; 60 requests/min per token; 500 hashes per check call.

Client obligations

The uploader must: apply a settle gate (skip files modified <10 s ago), snapshot file identity (dev, inode, size, mtime-ns) before hashing, and restat before any disposition (archive/trash), skipping the disposition if identity changed. Disposition only after accepted/duplicate on PUT or pending/imported from check. rejected files are never moved or deleted.

Client identity

The uploader is a stand-alone package that never updates itself: a copied dist/scan-uploader.mjs sits at whatever revision it was built from until somebody copies a new one over it. So it volunteers who it is, and the box reports an uploader that has fallen behind.

SCAN_CONTRACT_VERSION is one monotonic integer with three spellings: the box (beebox/src/core/scan/contract-version.ts), the Node uploader (scan-uploader/src/cli/contract-version.ts) and the Mac app (mac-app/Sources/BeeBoxMacCore/Scan/ContractVersion.swift, as scanContractVersion). Each client sends it as X-Scan-Contract; the box returns its own as contractVersion on the check response; the client compares them every sweep and reports which side is behind.

When to bump SCAN_CONTRACT_VERSION

The test: would a client built before this change now do the wrong thing, or fail to do a right thing, even though it still parses every response? If yes, bump. The version exists for exactly the changes that do not announce themselves — a client that can no longer parse the wire already throws a protocol error and needs no version to make that visible.

Bump for:

Do not bump for:

The row above is the one to read carefully, because it is where a plausible reading goes wrong: the question is never "does this touch a limit or a header the contract mentions", it is "can a client built before this change still do the right thing". Go and read the client's handling before deciding.

When it is genuinely unclear, bump. The cost of a needless bump is one spurious "out of date" line telling someone to re-copy a file they could have kept. The cost of a missed bump is a stale client reported as current, which is worse than having no version at all, because it answers the question wrongly rather than not at all. Nothing can test that a human bumped this, so the bump is part of the change discipline below rather than a separate obligation.

Mechanically, a bump is: +1 to all three constants in the same change (they are three spellings of one number, never per-side versions), this document, and the route tests. Bumping does not refuse anything — an old client keeps uploading and starts being reported. And never bump the build stamp: it is a build artifact, not a hand-maintained number.

Adding something the client must now send

A bump reports; it never refuses. But a change that makes the server require something new — a new mandatory header, a stricter body shape — refuses on its own, and every copied bundle in the field fails until somebody walks to that machine and copies a new file over. Nothing here auto-updates. So a bump is not sufficient for this shape of change, and the ordering is part of the change:

1. Ship the server side accepting but not requiring it, and the client side sending it. Verify-if-present; a missing value stays legal.
2. Wait until every uploader has actually been re-copied. The box can tell you: the scan-uploaders health check reports each uploader's build, so "have they all been updated" is a question with an answer rather than a guess.
3. Only then make it mandatory — and bump again, because that is the change that alters what a correct client must do.

Collapsing this into one step is the mistake, and it fails in the field rather than in a test: the tests exercise a current client against a current box, which is the one pairing that cannot show the breakage.

The build stamp (X-Scan-Client-Build, X-Scan-Client-Built-At) answers a different question: not "does this client still speak the protocol" but "how old is it". A client can be current on the contract and still be missing features. A checkout sends source, because it runs current source on every sweep and cannot drift; only a copied bundle can. The box records both on the scan token's record and compares the build time against its own deploy time — two timestamps from the same monorepo, which is ordered in a way comparing git revisions could not be.

All three request headers ride both routes (the table above lists them under PUT, but the check request sends them too), so a box that only ever sees uploads still learns what is talking to it.

Neither is a credential and neither gates anything. The box never refuses an old client: a contract change that genuinely breaks a client already fails loudly at parse (an unknown state or an unexpected status is a protocol error), and refusing a client that still works would strand scans on the laptop with nothing to show for it. A box reporting no contractVersion, and a client sending no identity headers, both mean "no opinion" — never "drifted". Every one of these fields is additive over a parser that ignores unknown keys and unknown headers, in both directions, so no flag day is needed.

Note the prefix: these must not be named x-bbx-*. The hub deletes every client-supplied header in that namespace before it reaches a box (its spoof wall), so such a header would silently never arrive.

Change discipline

Any change to routes, headers, states, statuses, limits, or hashing updates this doc, every breadcrumbed implementation (box, Node client, Mac client), and the route tests in the same change. A change that alters the client obligations, the state vocabulary, or a route's shape also bumps SCAN_CONTRACT_VERSION in all three spellings (see "Client identity") — that bump is part of this same change, not a follow-up.