> ## Documentation Index
> Fetch the complete documentation index at: https://docs.findmydata.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Roadmap

# Find My Data — roadmap

What is built, and what the design documents call for next. Grounded in
`find_my_data_handoff/FIND_MY_DATA_MVP_DELIVERY_PLAN.md` §6 (backlog) and §10
(releases), reconciled against the actual code. Priority key: **P0** in the
coherent slice, **P1** next MVP work, **P2** future.

## Built (this repository)

* Platform kernel (auth, capability/domain authz, hash-chained audit, redacting
  logger, **metrics registry**), governance/onboarding, provider-neutral AI
  gateway, durable staged scan pipeline with cache invalidation, explainable
  ensemble + clusters + risk + findings, review loop with propagation, typed
  Analyst, remediation state machine.
* **Live Microsoft Graph connector (read)** — validated read-only against CDX
  (ADR-0013).
* **Broad extraction** — text family, DOCX/XLSX/PPTX, HTML, RTF, best-effort PDF,
  behind a magic-byte-sniffing registry (validated live: 172/174 CDX files).
* **Live Purview label read/write path** — gated, with a separate remediation
  identity (assignSensitivityLabel).
* **Distributed scanner fleet** — registration, heartbeat, capability
  advertisement, offline takeover, fleet health.
* **Observability** — counters/gauges/latency histograms surfaced in Operations.
* **Admin setup wizard + settings + connector probe** (`/setup`) — validated live.
* **Change notifications** — subscription lifecycle, clientState-verified webhook
  receiver, delta reconciliation, freshness metrics; dev simulator validated live.
* **Governed model-release pipeline** (`/models`) — dataset curation → consented
  cluster-aware snapshot → evaluate → SoD approve → promote/rollback; ran on live
  CDX data.
* **Embeddings + approximate vector index** — provider-neutral embeddings, cosine
  LSH, owner-scoped semantic-neighbors; validated live (172 embeddings).
* **Teams connector** — channels/messages, file→driveItem canonicalization (no
  double identity); mock-validated, live gated.
* **Exchange connector** — selected mailboxes, per-folder delta, attachment↔
  SharePoint **similarity** (not identity); mock-validated, live gated.
* **Azure deployment IaC** (`infra/main.bicep`) — managed identity, Key Vault,
  Container App; secrets never in the template.
* **Control plane** — signed entitlement (offline grace + usage limits) +
  allowlisted fleet-health telemetry with opt-out.

## Release A — Microsoft 365 pilot hardening (P1)

| Item                                                                                                    | Design ref                      | Current hook                                                                       |
| ------------------------------------------------------------------------------------------------------- | ------------------------------- | ---------------------------------------------------------------------------------- |
| Change-notification receiver + subscription lifecycle/renewal; reconcile with delta                     | ARCH §5.7                       | `/api/ready` reports `changeNotifications: disabled`; delta is authoritative today |
| Migrate live connector to `Sites.Selected` (least privilege) + per-site grant                           | ARCH §5.2, permissions-manifest | Connector uses `Sites.Read.All` for the read pilot                                 |
| Validate live change-sequence deltas (edit/rename/permission/label/delete/resync) against CDX           | MVP §5.2                        | Fully exercised against the mock; connector `changes()` ready                      |
| Teams connector (channels, channel messages, files) — canonicalize files to `driveItem`                 | ARCH §5.3                       | Connector contract admits it; `teamsConnector: disabled`                           |
| Selected-mailbox Exchange connector (per-folder message delta, app RBAC)                                | ARCH §5.4                       | Contract admits it; `exchangeConnector: disabled`                                  |
| Purview label catalog operational sync (`Get-Label` import / supported API) while Graph listing is beta | ARCH §5.6, ADR (label catalog)  | Fixture catalog + `catalog_source` field; import path documented                   |
| Azure deployment IaC + managed identities + Key Vault secret refs                                       | ARCH §16, ADR-0011              | Config resolves secrets by env reference; ready for Key Vault                      |
| Live-tenant load/throttle measurement                                                                   | ARCH §14.3                      | `graph_throttled` metric + Retry-After handling in place                           |

## Release B — governed learning & enterprise operations (P1)

| Item                                                                              | Design ref                 | Current hook                                                                  |
| --------------------------------------------------------------------------------- | -------------------------- | ----------------------------------------------------------------------------- |
| Dataset/evaluation/model-release UI + approval/rollback over `dataset_candidates` | PRODUCT §8, ARCH §8.4      | `dataset_candidates` populated on review decisions; releases versioned        |
| Owner-feedback release pipeline (label-quality, leakage-safe splits, gold sets)   | PRODUCT §8                 | Decisions stored with training-eligibility flags                              |
| Embeddings + approximate vector candidate index                                   | ARCH §7.2, DATA §8         | `FingerprintIndex` role + namespaces; SimHash today                           |
| Drift/calibration dashboards                                                      | ARCH §8.4                  | Metrics layer + ensemble release versioning                                   |
| Distributed scanner packaging (container/K8s) + signed work envelopes + SBOM      | ARCH §3.3, SECURITY §9     | Fleet registration/heartbeat/takeover built; `worker-main.ts` runs standalone |
| Commercial control plane (signed entitlement, fleet heartbeat/status bands)       | ARCH §3.2, MVP Epic H      | Fleet health model; no telemetry leaves the boundary                          |
| Attestation workflow + orphan-domain queue; source-request workflow               | SECURITY §13, PRODUCT §6.7 | `ownership_assignments` carry attestation dates; `ScanTargetRequest` modeled  |

## Release C — remediation & broader evidence (P2)

* Ticket integrations; **permission-change proposals/approvals** (draft-only in MVP);
  external ticketing (PRODUCT §6.6, SECURITY §15.2).
* Richer activity/security telemetry; DLP/Purview/SIEM signal ingestion
  (activity currently surfaced as `not_enabled` in risk coverage).
* Authoritative-copy & ROT campaigns; regulatory/GRC validation views.
* **OCR / image understanding** for scanned PDFs and images (today: honest
  `unsupported`); layout/visual fingerprints (ARCH §7.1–7.2).
* Opt-in advanced/customer-defined Analyst skills.

## Release D — structured & hybrid ecosystem (P2)

* Database/warehouse/object/file-system **connector SDK**; structured
  schema/sampling/synthetic tables (ARCH §7.3, DATA §4.3). Canonical contracts
  already admit structured sources.
* Workday/PeopleSoft & other SaaS connectors; on-prem/AWS/GCP scanner footprints.
* Lineage integrations; modality-specific models.

## Cross-cutting platform work

* **Admin setup wizard UI** + in-UI connector/source configuration + settings
  (retention profile, AI provider, evidence level). Onboarding is API-driven today.
* Distributed traces across job/item/stage/model/source (metrics + correlation IDs
  exist; traces do not).
* Backup/restore + projection-rebuild runbooks; documented RPO/RTO; customer-managed
  keys; private endpoints (ARCH §15).
* Postgres / Blob / Service Bus / AI Search implementations of the store role
  interfaces (ADR-0003) as scale demands — triggers in `docs/capacity-model.md`.

## Commercial web & control plane (handoff update-001)

Applying [update-001](../find_my_data_handoff/FIND_MY_DATA_CLAUDE_UPDATE_001/)
(ADR-0015). The deployment-side licensing, the vendor Control service (ADR-0016),
the Account portal (ADR-0017), entitled downloads + supply chain (ADR-0018),
**and the support portal + diagnostic bundles** (ADR-0019) are **built**; the
remaining surfaces (isolated demo, protected dev env) and a real ticketing
integration are staged. **Production IdP** for the portal is now built (Clerk,
ADR-0020) — it needs only the operator's Clerk app + `FMD_CLERK_*` config.

| Item                                                                                                                 | Design ref      | Status                                                                                                                              |
| -------------------------------------------------------------------------------------------------------------------- | --------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| Signed, versioned entitlement (edition/features/capacity/term/offline policy/kid) verified locally (Ed25519 or HMAC) | web doc §6      | **built** (`controlplane/entitlement.ts`)                                                                                           |
| Offline activation + renewal + deactivation; append-only licensing events                                            | web doc §4, §6  | **built** (`controlplane/licensing.ts`, migration 0013)                                                                             |
| Non-destructive license enforcement (new-scan gate; reads/audit/export always available)                             | web doc §6      | **built** (`assertLicensedOperation`)                                                                                               |
| Strict-allowlisted fleet-health telemetry + customer opt-out                                                         | web doc §6      | **built** (`controlplane/telemetry.ts`)                                                                                             |
| Marketing landing page (canonical `findmydata.io`)                                                                   | web doc §3, §11 | **built (minimal)** (`site/index.html`)                                                                                             |
| Live outbound activation/renewal to `control.findmydata.io`                                                          | web doc §6      | **built** — `activate-online` + online renewal, verified locally (ADR-0016)                                                         |
| Vendor Control service holding signing private keys (issues + signs entitlements)                                    | web doc §6      | **built** — separate `control` deployable, `packages/server/src/vendor/` (ADR-0016)                                                 |
| Account portal (`account.findmydata.io`): register, evaluations, licenses, deployment status                         | web doc §5      | **built** — `account` deployable, `packages/server/src/account/` (ADR-0017); member management remains                              |
| Entitled downloads + software supply chain (short-lived URLs, checksums, SBOM)                                       | web doc §7      | **built** — release catalog + signed URLs (ADR-0018); SBOM + SHA256SUMS in `release.yml`; object-storage origin + signatures remain |
| Paid conversion / billing (evaluation → subscription without reinstall)                                              | web doc §5, §9  | **built** — Stripe Checkout + signed webhook upgrades the registry edition (ADR-0021); operator sets Products/Prices                |
| Support portal + consent-based redacted diagnostic bundles                                                           | web doc §8      | **built** — deployment `/api/support/diagnostics/preview` + Account support cases (ADR-0019); real ticketing integration remains    |
| Isolated synthetic `demo.findmydata.io`; protected `dev.findmydata.io`                                               | web doc §9      | **built** — fail-closed demo mode + banner (ADR-0022); Cloudflare Access/Tunnel runbook; provisioning is ops                        |
| Cloudflare DNS/redirects/Access                                                                                      | web doc §12     | **runbook only** (`docs/cloudflare-runbook.md`); no live changes                                                                    |

## Immediate next (recommended order)

The eight cross-cutting/Release-A/B items above are now built. The remaining
highest-value work, per the design docs:

1. **Sites.Selected migration + live change-sequence validation** on CDX — the
   one read-connector item still on `Sites.Read.All`; validate live deltas
   (edit/rename/permission/label/delete/resync) and, with a public webhook URL,
   real Graph change-notification subscriptions end-to-end.
2. **Live Teams/Exchange validation** — the connectors are built and gated; the
   next step is consenting `ChannelMessage.Read.All` / `Mail.Read` (protected
   APIs) on a test tenant and running them live, as was done for SharePoint.
3. **Entra OIDC sign-in** on the existing production-lockout seam + directory
   sync for owner suggestions.
4. **Postgres / Blob / Service Bus** implementations of the store-role interfaces
   (ADR-0003) + a separate worker Container App, per the scale triggers in
   `docs/capacity-model.md` and the `infra/` scale-path notes.
5. **OCR / image understanding** for scanned PDFs and images (today: honest
   `unsupported`), and the structured/DB **connector SDK** (Release D).