Skip to main content

​
ADR-0014: Roadmap features — connectors, notifications, model governance, embeddings, control plane

Status: Accepted · 2026-07-14 (roadmap follow-up session)

​
Context

Eight roadmap items were built on top of the validated vertical slice. Each required a design decision that this ADR records, all sharing the invariants: mock-first (runnable without credentials), honest live gating, secrets out of source control and logs, deny-by-default authorization, human authority over material changes.

​
Decisions

​
1. Change notifications are a hint, verified by clientState HMAC

A Graph change notification never carries trusted content. On receipt the webhook verifies a per-subscription clientState — derived as HMAC(sessionSecret, subscriptionId), stored only as a sha256, compared in constant time — before any side effect, then reconciles the affected scope through the existing delta feed. The lifecycle receiver applies the same verification (a review found and we fixed an early bypass when the field was absent). Mismatches are metered, not persisted, so the unauthenticated endpoint can’t be used for write amplification. Live subscriptions need a public HTTPS notificationUrl; absent it, the path is honestly blocked and a dev simulator exercises the receiver + reconcile flow. Freshness is measured receive→reconcile.

​
2. Teams files canonicalize to the driveItem; Exchange attachments are similarity

Both are distinct modalities modeled in dedicated tables (not forced into physical_assets, whose kind CHECK is deliberately narrow and FK-referenced by 11 tables). The load-bearing distinction:
  • A Teams channel file is the backing SharePoint driveItem, so an attachment resolves to the existing inventoried asset — canonicalization, no second identity. Verified: sync creates zero new physical_assets.
  • An Exchange attachment is a copy, so a content match (identical raw-byte or normalized-text hash, via the existing fingerprints table) is recorded as similarity, not identity — an attachment_similar_to edge, never a merge.
Both live paths are gated behind feature flags + protected-API consent (ChannelMessage.Read.All, Mail.Read scoped to selected mailboxes) and reported honestly. Message bodies are reduced to bounded, identifier-redacted previews; only detector signals (which information type, count) are retained.

​
3. Model releases are human-approved with separation of duties

The learning loop turns owner review decisions into dataset candidates → human curation → a consented, cluster-aware train/eval snapshot (whole near-duplicate clusters stay on one side; a balancing rule guarantees a non-empty eval set) → a proposed ensemble config → evaluation on the held-out split → approval by a different principal than the proposer (enforced in code, not just by capability split) → promote (bump the active ensemble + rescore) → rollback. Nothing material auto-applies. Evaluation currently sweeps the ensemble decision threshold (the primary tunable) over stored confidences; detector-definition releases would add a feature recompute (documented as follow-on).

​
4. Embeddings: deterministic default, cosine LSH, per-model namespaces

Embeddings are provider-neutral through the AI gateway. The default is a deterministic feature-hashing embedding that runs credential-free (the “mock keeps dev/tests runnable” invariant); Azure OpenAI is a config-gated adapter; Anthropic honestly reports no first-party embeddings. Embeddings are a retained feature computed once per asset version inside inference, so re-scoring is idempotent and source-free. Approximate retrieval uses random-hyperplane cosine LSH (8 bands / 24 bits) + an exact rerank — sublinear, no external vector store. Different embedding models write disjoint namespaces (queries filter by model), so mock and provider vectors never mix. The semantic-neighbors API re-applies the same asset-visibility filter as every other owner-facing read.

​
5. Control plane carries licensing + health only, allowlisted

The entitlement is a signed token verified locally (HMAC in this build; asymmetric in production — identical interface) with an offline grace window so the product survives an unreachable control plane, plus usage-vs-plan-limit evaluation. Fleet-health telemetry is a .strict() Zod schema of licensing + operational health bands only — deployment id (hashed), version, scanner counts, queue/dead-letter/scale bands — never content, names, paths, domains, or raw asset counts. The schema fails closed on any extra field; a denylist-backed test guards the allowlist. A transparency endpoint shows the customer the exact payload, and telemetry is opt-outable at any time (default on).

​
6. Azure deployment keeps secrets out of the template

infra/main.bicep provisions a managed identity, Key Vault (RBAC), storage + file share, Log Analytics, and a Container App. Key Vault secrets are empty placeholders set out-of-band; the app reads them via Key-Vault-backed Container App secrets resolved by the managed identity, preserving the app’s *_REF indirection. The honest scale path (Postgres, separate worker app, private endpoints) is documented, not implied as done.

​
Consequences

  • Eight capabilities the design docs called for are runnable mock-first and honestly gated for live; several were validated against the live CDX tenant (admin probe, change-notification reconcile, model-release loop on real data, 172 semantic embeddings).
  • Two new modalities (Teams, Exchange) are modeled without destabilizing the tuned SharePoint pipeline, and without the risky recreation of the heavily-referenced physical_assets table.
  • The readiness manifest now distinguishes live / blocked / disabled by whether a secret reference actually resolves — no capability claims to be live until its credentials are present.