Platform overview

This document is the starting point for anyone new to Find My Data — evaluators and buyers who want to know what the product does and how far along it is, engineers who need a map of the concepts before reading code, and administrators preparing a deployment. It summarizes the product, the roles, the vocabulary, the honest state of every capability, and the invariants the platform is built on. Facts current as of release 26.7.15.0 (2026-07-15; versioning is year.month.day.x, so a same-day follow-up release would be 26.7.15.1), with 196 tests passing on Bun 1.3.14.

What Find My Data is

Find My Data is a data ownership and sensitive-data intelligence platform. It finds enterprise content that may belong to a business domain, asks the people who understand that data to confirm or correct it, connects those decisions to related assets and risks, and enables governed remediation. It combines policy-grounded AI, conventional detectors, metadata, identity and permission signals, exact and near-duplicate fingerprints, human confirmation, and graph-based relationship analysis. The Microsoft-focused MVP covers SharePoint, OneDrive, Teams, Exchange, Entra ID, and Microsoft Purview sensitivity labels. The product loop — and tagline — is:

Find it. Confirm it. Connect it. Govern it.

Step	What happens
Find it	Durable, incremental scans inventory source content and an explainable classification ensemble proposes which business domain and information type each asset belongs to — with evidence.
Confirm it	The “Is this yours?” review queue asks domain owners to confirm or correct proposals. Decisions create new assertions; the original inference is never destroyed.
Connect it	Confirmations propagate to related assets (byte-identical copies, same-template near-duplicates, container neighbors) with stated reasons, and feed clusters, risk scoring, and the relationship graph.
Govern it	Findings drive approval-gated remediation actions (e.g., Purview label changes) with preview, justification, separation of duties, idempotent execution, source verification, and a hash-chained audit trail.

The end-to-end loop runs locally with zero credentials against a deterministic Microsoft 365–shaped fixture tenant for the fictional demo organization Meridian Grove Holdings (all fixture people and documents are synthetic). See the README for the quick start and demo script, and STATUS.md for the authoritative build state.

Personas and visible roles

The UI presents five understandable roles; enforcement underneath uses granular capabilities with scopes (tenant or domain), not roles. The role and capability definitions live in packages/shared/src/capabilities.ts.

Visible role	Demo persona	What they do
`domain_owner`	Priya Raman (HR), Diego Alvarez (Finance)	Understands the meaning and legitimate use of a domain’s information. Reviews “Is this yours?” candidates, explores the data landscape and evidence, queries the Analyst, and drafts/submits remediation actions — all scoped to their assigned business domains only.
`governance_admin`	Sam Whitfield	Manages taxonomy, policies, owner assignments and attestations, and model-release governance: proposes and approves taxonomy, ingests policy documents, curates datasets, and proposes/approves ensemble releases.
`platform_admin`	Avery Chen	Manages deployment, connectors, scans, and operational health. Deliberately gets no evidence, excerpt, or review capabilities — operating the platform does not grant access to business content.
`remediation_approver`	Morgan Ellis	Reviews consequential actions (impact, justification), approves, rejects, or cancels them. Approves but does not draft, and does not explore evidence broadly. Separation of duties is enforced and tested.
`auditor`	Ren Nakamura	Reads the audit trail, policies, taxonomy, findings, and scan/operations state without any mutation capability and without excerpt access.

A person may hold multiple roles, subject to policy. Domain isolation is enforced and tested: a Finance owner probing an HR asset gets a 404 with no existence leak.

Core concepts glossary

Terminology is defined canonically in the handoff product and data model documents; enumerations live in packages/shared/src/domain-enums.ts.

Term	Meaning
Business domain	An accountable business area (e.g., Human Resources). One of the two primary taxonomy anchors; ownership, review scope, and Analyst scope all resolve against it.
Information type	A recognizable business artifact or semantic kind of information (e.g., employee background-check report). The primary classification target and the second taxonomy anchor.
Asset	A physical asset: an observed object in a source system (file, email, attachment, list item), with versions, locations, permissions, and label observations tracked over time.
Evidence	A typed, immutable, bounded observation supporting or contradicting an assertion — a phrase hit, label GUID, fingerprint match, permission exposure, path context, keyphrase overlap, policy citation, or human decision — with provenance, versions, and access classification.
Assertion	A versioned claim (e.g., asset `belongs_to_domain`, asset `is_information_type`) with confidence, provenance kind, evidence links, and supersession chains. Confirmation never overwrites inference; a confirming or correcting assertion is linked to the original so model evaluation remains possible.
Provenance	The explicit origin kind of evidence or an assertion, never collapsed into a single confidence number: `Observed`, `RuleDerived`, `ModelInferred`, `PolicyDerived`, `OwnerConfirmed`, `GovernanceValidated`, `RemediationVerified`, `Imported`.
Cluster	A versioned grouping of assets connected by similarity or context (exact duplicate, near-duplicate/template, shared context), with a per-member score and reason. An asset may belong to multiple clusters.
Finding	A typed, governable, explainable issue or opportunity — overexposure, unlabeled or inconsistent label, unexpected location, ROT candidate, duplicate sprawl — with severity, evidence, and a lifecycle (`open`, `acknowledged`, `planned`, `remediating`, `resolved`, `reopened`, `suppressed`). Findings resolve automatically on recompute when the underlying condition clears.
Review candidate	A prioritized item in the “Is this yours?” queue: a candidate assertion plus its evidence bundle awaiting a domain-owner decision. Six decision kinds: `mine`, `mine_different_type`, `another_domain`, `mine_not_sensitive`, `not_a_match`, `need_more_evidence`.
Action	A proposed, approved, executed, and verified change or task (kinds: `internal_task`, `purview_label_set`, `purview_label_remove`) moving through a state machine from `draft` through approval, idempotent execution, and source verification, including `drifted` (target changed since preview) and `blocked` states.
Scan campaign	A durable scan run over a connector scope, decomposed into leased work units with retry classes, backoff, dead letters, and pause/resume/cancel. Resuming never duplicates logical work (tested).
Delta	An incremental change feed from a source. A delta scan processes only what changed; stage-cache keys implement the invalidation matrix, so a permission-only change triggers zero content fetches (tested) and a rename preserves identity and history.
Ensemble release	A versioned, governed release of the classification ensemble configuration: curate datasets → snapshot → evaluate against held-out labels → approve (approver ≠ proposer) → promote → rescore, with rollback. Promotion re-scores from retained features with no connector dependency (tested).

Capability matrix — what is built, honestly

Find My Data never overstates its state. Every capability is in one of these states, surfaced in the UI and the /api/ready readiness endpoint as an integration mode (mock, live, blocked, disabled):

Built + validated live (CDX) — implemented and exercised against the live Contoso CDX tenant (read-only inventory path only; see ADR-0013 and the CDX test runbook).
Built, mock-validated, live path feature-gated — implemented and tested against deterministic fixtures; the live path refuses to run (with an explicit error, never a fake integration) until configuration/permissions are provided.
Not built — a seam, contract, or backlog item only.

The table below reproduces the ground truth from STATUS.md:

Capability	State	Unblock path
Microsoft Graph connector (read)	Built + validated live (CDX), read-only (ADR-0013). Mock remains the default/CI path.	Migrate to `Sites.Selected`; validate live deltas/change sequence
Microsoft Graph connector (write)	Not built — read-only app	Separate remediation identity + write permissions
Entra ID sign-in	Not built — seam + production lockout only; OIDC flow returns 501	Entra app registration; implement code flow against the documented seam
Purview label read (live)	Built, gated `FMD_FEATURE_PURVIEW_LABEL_READ`; honest `unknown` when off	Grant `InformationProtectionContent.Read.All` + enable flag
Purview label write (live)	Built, gated + separate remediation identity; refuses if unconfigured	Protected/metered `assignSensitivityLabel` enablement + remediation app
File-type extraction	Built + validated live (CDX): text/DOCX/XLSX/PPTX/HTML/RTF/PDF (172/174 CDX files extracted)	OCR/image for scanned PDFs & images (backlog P2)
Distributed scanners	Built: fleet registration/heartbeat/takeover; runs in-process or standalone	Container/K8s packaging + signed work envelopes (P1)
Metrics / observability	Built: counters/gauges/histograms in Operations	Distributed traces (P1)
Azure OpenAI / Anthropic AI providers	Adapters implemented, config-gated, not exercised by tests (deterministic mock is the default)	Provide endpoint/key refs; contract identical to mock
Change notifications	Built: subscription lifecycle + webhook receiver (clientState HMAC) + reconcile; dev simulator validated live against CDX	Public HTTPS webhook URL (`FMD_WEBHOOK_PUBLIC_URL`) + flag for live Graph subscriptions
Teams connector	Built, mock-validated (file attachments canonicalize to the backing SharePoint driveItem); live gated	`FMD_FEATURE_TEAMS_CONNECTOR` + `ChannelMessage.Read.All` (protected API)
Exchange connector	Built, mock-validated (selected mailboxes, per-folder delta; attachment↔SharePoint match is similarity, not identity); live gated	`FMD_FEATURE_EXCHANGE_CONNECTOR` + `Mail.Read` scoped to selected mailboxes
Embeddings / vector index	Built + validated live (CDX): deterministic default + cosine LSH; semantic-neighbors API; 172 embeddings indexed	Azure OpenAI embeddings adapter for provider vectors (config-gated)
Model-release pipeline	Built + validated live (CDX): curate → snapshot → evaluate → SoD approve → promote/rollback, ran on live CDX data	— (governed loop complete)
Admin setup/config UI	Built + validated live (CDX): `/setup` wizard + settings + connector probe	—
Control-plane telemetry	Built: signed entitlement + strictly allowlisted fleet health + opt-out	Actual Control endpoint to receive telemetry (out of scope; no content leaves)
Azure deployment	Built: `infra/main.bicep` (managed identity, Key Vault, Container App)	`az bicep build` in a CLI env; Postgres + separate worker for scale
Activity signals	Not built — `not_enabled` state surfaced honestly in risk coverage	Graph activity APIs evaluation (backlog P1)

Beyond that table, the core product loop — kernel (config, capability authorization, hash-chained audit, redacting logger), governance and taxonomy, fixture connector and staged scan pipeline, intelligence (detectors, similarity, ensemble, risk, findings), review with propagation, the scoped Analyst, and the action state machine — is implemented and tested end-to-end (196 tests: unit, integration, security, acceptance). Only the read-only inventory path has ever touched a live tenant. Remaining work is prioritized in roadmap.md and the STATUS.md backlog.

Architecture at a glance

Find My Data is a Bun/TypeScript modular monolith (ADR-0001, ADR-0002): one deployable server process hosts the HTTP API and an embedded durable worker (the worker can also run standalone via bun run worker against the same queue). All persistence is a single SQLite file (WAL) behind logical role interfaces — system of record, full-text search projection, typed graph nodes/edges, durable work queue with leases and dead letters, and a purpose-flagged artifact store (82 tables across 12 migrations) — so each role can later migrate to a dedicated technology without rewriting callers (ADR-0003). Sources plug in through a versioned connector contract (ADR-0008) with a deterministic mock M365 tenant as the default and the live Graph connector config-gated; AI goes through a provider-neutral gateway (deterministic mock default; Azure OpenAI and Anthropic adapters config-gated, ADR-0005); and the web UI is a React (Vite) app in packages/web/. The full actual-code diagram and staged-pipeline detail are in architecture.md; consequential decisions are recorded in docs/adr/.

Product invariants

These properties come from the original handoff package and do not change without an explicit, documented decision:

Data-domain-owner first. The main experience serves people who understand the business meaning of data, not only security engineers. The home page, review queue, landscape, and Analyst are all built for the domain owner.
Evidence before assertion. Every finding, classification, relationship, and AI answer exposes its evidence, confidence, provenance, and version. The Analyst answers with interpretation, scope, evidence-row IDs, coverage, freshness, and blind spots (ADR-0009).
Human authority. AI proposes; authorized humans confirm ownership, material classifications, and remediation. Propagation reprioritizes related candidates but never auto-confirms; actions require separate approval (ADR-0010).
Honest staging. The platform reports what is mocked, gated, blocked, or unknown — in the UI, in /api/ready, and in STATUS.md. Unknown, unsupported, and inaccessible states stay distinct; coverage, confidence, and risk are reported separately; gated integrations fail with explicit errors rather than pretending to work.
Deny-by-default authorization. A request is allowed only when the session principal holds the required capability at a scope covering the resource. Roles are a UI convenience; capabilities and domain scopes are the enforcement, with deliberate exclusions (e.g., platform admins get no evidence access).

The handoff README lists the full set of ten invariants, including customer control of content, incremental-by-design scanning, provider-neutral AI, and approved remediation. Related reading: threat model, permissions manifest, capacity model.

​Platform overview

​What Find My Data is

​Personas and visible roles

​Core concepts glossary

​Capability matrix — what is built, honestly

​Architecture at a glance

​Product invariants