​

Capacity model (preliminary)

Status: preliminary estimate, 2026-07-14. The design ceiling from the reference architecture (§14.1) is ~100,000 directory identities and ~500 million file-like assets per organization. The prototype has NOT validated this scale. Everything below is arithmetic over the actual slice schema plus small slice-scale measurements on one developer machine. Numbers at 1M+ assets are extrapolations; numbers at 500M are naive linear extrapolations recorded so they can be falsified by the validation ladder (§7), not claims. Related: ADR-0003 (single SQLite behind role interfaces), ADR-0007 (stage caching), ADR-0011 (minimized-evidence retention), ADR-0012 (CDX strategy), handoff reference architecture §14.

​

1. Model variables

Per handoff ARCH §14.1. Baselines are assumptions to be replaced by pilot measurements; none of them has been observed in a real tenant. Byte figures include SQLite row and index overhead as measured, not just payload.

​

What one asset costs in the actual schema

From packages/server/migrations/0002 and 0003, one fully processed asset writes: Not modeled per asset: audit events (per governed action, ~400 B hash-chained row), activity aggregates (per asset per window, not_enabled in the slice), messages (out of slice scope).

​

2. Daily steady-state load

Per day, after bootstrap:

• changed items: c · A → new asset_versions + permission/label observations (~1 KB/item retained history before retention sweeps);
• content-changed subset (assume ~⅓ of changes touch content, i.e. cTag changes): new fingerprints + features for ~0.33 · c · A · r… in practice the stage-cache keys (ADR-0007) guarantee rename/permission/label-only changes cost no content fetch, hash, or extraction — this is tested in the slice;
• model calls: m × content-changed items. At 500M assets, 2%/day, ~⅓ content changes, m=1–2 → 3–7M model calls/day. This, not storage, is the dominant live operating cost and must be bounded by per-tenant budgets and workload priorities (ARCH §14.2). The exercised slice path uses the deterministic mock (m=0); Azure OpenAI/Anthropic adapters exist but are config-gated.

​

3. Worked example: 1M assets

Assumptions: r=30%, d≥10:1, one full scan pass of telemetry retained, B_text≈5 KB typical. Total: roughly 8–15 GB. Bytes are not the problem at 1M assets. The constraint is the single SQLite writer (§5) and bootstrap wall-clock, not storage.

​

4. Worked example: 500M assets (naive linear extrapolation)

Explicitly unvalidated; recorded to size the target architecture, not to claim it. Total order: ~2.5–5 TB across stores. Feasible for Postgres + Blob + a search service with partitioning; the hard problems are bootstrap throughput against provider rate limits, model-call budget, permission-set dedup holding up in hostile tenants (unique direct shares defeat d), and index growth — all of which are exactly what the validation ladder (§7) must measure before any 500M claim.

​

5. Why single-writer SQLite tops out (~1–5M assets, single node)

The slice intentionally runs one SQLite file (WAL) behind role interfaces (ADR-0003). Where that stops working:

• One writer. WAL allows concurrent readers but exactly one writer. The API process, scan worker, projections, audit appends, and queue lease/heartbeat updates all serialize on the same writer (busy_timeout 5 s). Loadgen’s ~220k rows/s (§8) is tight batched transactions with no readers; the real pipeline’s effective sustained rate is far lower and degrades as interactive load appears.
• Queue on the same writer. Work-unit leasing and heartbeats compete with observation inserts — the first visible symptom of contention.
• Operational limits, not size limits. 1M assets ≈ 3–15 GB is comfortable for SQLite; ~5M ≈ 15–60 GB is still storable. But there is no online replication, no HA, backup is whole-file, checkpoint pauses grow with write volume, and query p95s hold only while the working set fits one node’s page cache.
• No horizontal read scaling for many concurrent domain-owner queries.

So: comfortable at slice scale, plausible to low single-digit millions of assets on one strong node with careful batching, structurally wrong beyond that.

​

6. Migration triggers per role interface (ADR-0003)

Each role moves independently; the code already speaks the role interface, so substitution is additive. Cross-cutting at any migration point: keep every row tenant-scoped (already enforced), keep projections rebuildable with watermarks, and keep stage telemetry (stage_attempts) on a TTL or in an operational log store rather than the system of record.

​

7. Staged validation ladder (ARCH §14.3)

​

8. First measurements (slice scale, this dev machine)

Harness: bun run loadgen [assetCount] — isolated packages/server/data/loadgen.db, fresh migrations, same INSERT patterns as the pipeline’s observe/permissions stages (including upsertPermissionSet dedup, imported from the pipeline itself), fingerprint clusters for ~30% of assets, no network, no content bytes. Machine: developer laptop (Apple M4, 16 GB, macOS, Bun 1.3.14). Numbers below are slice-scale harness measurements, not a scale validation. They measure raw batched insert cost and one bounded query shape — no connector latency, no extraction/inference stages, no concurrent readers or writers. Observations:

• Measured ~2.6–2.8 KB/asset is the basis for B_meta ≈ 1.2 KB and B_fp ≈ 5.5 KB in §1 (4 canonical rows per asset + 19 fingerprint rows for 30% of assets; averages ~290 B/row including index overhead).
• Dedup ratio improves with asset count as expected (the distinct-set pool saturates); real-tenant inherited permissions should dedup much harder, and hostile cases (per-item unique shares) much worse. Measure in stage 2.
• Index discipline is load-bearing: the container-listing query ran at ~14 ms p95 until the asset_versions join was tenant-scoped so it could use the partial index idx_asset_versions_current (tenant_id, asset_id) WHERE is_current = 1; with tenant scoping it is ~0.2–0.5 ms. Every query must carry tenant_id — the repository convention (ADR-0003) is also the performance-correct one.
• Bytes/asset here excludes features, evidence, assertions, stage telemetry, and object-store text — the loadgen number is a floor, not the full per-asset cost modeled in §1.