> ## Documentation Index > Fetch the complete documentation index at: https://docs.findmydata.io/llms.txt > Use this file to discover all available pages before exploring further. # Platform overview # Platform overview This document is the starting point for anyone new to Find My Data — evaluators and buyers who want to know what the product does and how far along it is, engineers who need a map of the concepts before reading code, and administrators preparing a deployment. It summarizes the product, the roles, the vocabulary, the honest state of every capability, and the invariants the platform is built on. Facts current as of release **26.7.15.0** (2026-07-15; versioning is `year.month.day.x`, so a same-day follow-up release would be 26.7.15.1), with **196 tests passing** on **Bun 1.3.14**. ## What Find My Data is Find My Data is a **data ownership and sensitive-data intelligence platform**. It finds enterprise content that may belong to a business domain, asks the people who understand that data to confirm or correct it, connects those decisions to related assets and risks, and enables governed remediation. It combines policy-grounded AI, conventional detectors, metadata, identity and permission signals, exact and near-duplicate fingerprints, human confirmation, and graph-based relationship analysis. The Microsoft-focused MVP covers SharePoint, OneDrive, Teams, Exchange, Entra ID, and Microsoft Purview sensitivity labels. The product loop — and tagline — is: > **Find it. Confirm it. Connect it. Govern it.** | Step | What happens | | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Find it** | Durable, incremental scans inventory source content and an explainable classification ensemble proposes which business domain and information type each asset belongs to — with evidence. | | **Confirm it** | The "Is this yours?" review queue asks domain owners to confirm or correct proposals. Decisions create new assertions; the original inference is never destroyed. | | **Connect it** | Confirmations propagate to related assets (byte-identical copies, same-template near-duplicates, container neighbors) with stated reasons, and feed clusters, risk scoring, and the relationship graph. | | **Govern it** | Findings drive approval-gated remediation actions (e.g., Purview label changes) with preview, justification, separation of duties, idempotent execution, source verification, and a hash-chained audit trail. | The end-to-end loop runs locally with zero credentials against a deterministic Microsoft 365–shaped fixture tenant for the fictional demo organization **Meridian Grove Holdings** (all fixture people and documents are synthetic). See the [README](../README.md) for the quick start and demo script, and [STATUS.md](../STATUS.md) for the authoritative build state. ## Personas and visible roles The UI presents five understandable roles; enforcement underneath uses **granular capabilities with scopes** (`tenant` or `domain`), not roles. The role and capability definitions live in [`packages/shared/src/capabilities.ts`](../packages/shared/src/capabilities.ts). | Visible role | Demo persona | What they do | | ---------------------- | ----------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `domain_owner` | Priya Raman (HR), Diego Alvarez (Finance) | Understands the meaning and legitimate use of a domain's information. Reviews "Is this yours?" candidates, explores the data landscape and evidence, queries the Analyst, and drafts/submits remediation actions — **all scoped to their assigned business domains only**. | | `governance_admin` | Sam Whitfield | Manages taxonomy, policies, owner assignments and attestations, and model-release governance: proposes and approves taxonomy, ingests policy documents, curates datasets, and proposes/approves ensemble releases. | | `platform_admin` | Avery Chen | Manages deployment, connectors, scans, and operational health. Deliberately gets **no** evidence, excerpt, or review capabilities — operating the platform does not grant access to business content. | | `remediation_approver` | Morgan Ellis | Reviews consequential actions (impact, justification), approves, rejects, or cancels them. Approves but does not draft, and does not explore evidence broadly. Separation of duties is enforced and tested. | | `auditor` | Ren Nakamura | Reads the audit trail, policies, taxonomy, findings, and scan/operations state without any mutation capability and without excerpt access. | A person may hold multiple roles, subject to policy. Domain isolation is enforced and tested: a Finance owner probing an HR asset gets a 404 with no existence leak. ## Core concepts glossary Terminology is defined canonically in the handoff [product](../find_my_data_handoff/FIND_MY_DATA_PRODUCT_AND_UX.md) and [data model](../find_my_data_handoff/FIND_MY_DATA_DATA_AND_GRAPH_MODEL.md) documents; enumerations live in [`packages/shared/src/domain-enums.ts`](../packages/shared/src/domain-enums.ts). | Term | Meaning | | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Business domain** | An accountable business area (e.g., Human Resources). One of the two primary taxonomy anchors; ownership, review scope, and Analyst scope all resolve against it. | | **Information type** | A recognizable business artifact or semantic kind of information (e.g., employee background-check report). The primary classification target and the second taxonomy anchor. | | **Asset** | A physical asset: an observed object in a source system (file, email, attachment, list item), with versions, locations, permissions, and label observations tracked over time. | | **Evidence** | A typed, immutable, bounded observation supporting or contradicting an assertion — a phrase hit, label GUID, fingerprint match, permission exposure, path context, keyphrase overlap, policy citation, or human decision — with provenance, versions, and access classification. | | **Assertion** | A versioned claim (e.g., asset `belongs_to_domain`, asset `is_information_type`) with confidence, provenance kind, evidence links, and supersession chains. Confirmation never overwrites inference; a confirming or correcting assertion is linked to the original so model evaluation remains possible. | | **Provenance** | The explicit origin kind of evidence or an assertion, never collapsed into a single confidence number: `Observed`, `RuleDerived`, `ModelInferred`, `PolicyDerived`, `OwnerConfirmed`, `GovernanceValidated`, `RemediationVerified`, `Imported`. | | **Cluster** | A versioned grouping of assets connected by similarity or context (exact duplicate, near-duplicate/template, shared context), with a per-member score and reason. An asset may belong to multiple clusters. | | **Finding** | A typed, governable, explainable issue or opportunity — overexposure, unlabeled or inconsistent label, unexpected location, ROT candidate, duplicate sprawl — with severity, evidence, and a lifecycle (`open`, `acknowledged`, `planned`, `remediating`, `resolved`, `reopened`, `suppressed`). Findings resolve automatically on recompute when the underlying condition clears. | | **Review candidate** | A prioritized item in the "Is this yours?" queue: a candidate assertion plus its evidence bundle awaiting a domain-owner decision. Six decision kinds: `mine`, `mine_different_type`, `another_domain`, `mine_not_sensitive`, `not_a_match`, `need_more_evidence`. | | **Action** | A proposed, approved, executed, and verified change or task (kinds: `internal_task`, `purview_label_set`, `purview_label_remove`) moving through a state machine from `draft` through approval, idempotent execution, and source verification, including `drifted` (target changed since preview) and `blocked` states. | | **Scan campaign** | A durable scan run over a connector scope, decomposed into leased work units with retry classes, backoff, dead letters, and pause/resume/cancel. Resuming never duplicates logical work (tested). | | **Delta** | An incremental change feed from a source. A delta scan processes only what changed; stage-cache keys implement the invalidation matrix, so a permission-only change triggers **zero content fetches** (tested) and a rename preserves identity and history. | | **Ensemble release** | A versioned, governed release of the classification ensemble configuration: curate datasets → snapshot → evaluate against held-out labels → approve (approver ≠ proposer) → promote → rescore, with rollback. Promotion re-scores from retained features with no connector dependency (tested). | ## Capability matrix — what is built, honestly Find My Data never overstates its state. Every capability is in one of these states, surfaced in the UI and the `/api/ready` readiness endpoint as an integration mode (`mock`, `live`, `blocked`, `disabled`): * **Built + validated live (CDX)** — implemented and exercised against the live Contoso CDX tenant (read-only inventory path only; see [ADR-0013](adr/ADR-0013-live-graph-connector.md) and the [CDX test runbook](cdx-test-runbook.md)). * **Built, mock-validated, live path feature-gated** — implemented and tested against deterministic fixtures; the live path refuses to run (with an explicit error, never a fake integration) until configuration/permissions are provided. * **Not built** — a seam, contract, or backlog item only. The table below reproduces the ground truth from [STATUS.md](../STATUS.md): | Capability | State | Unblock path | | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | | Microsoft Graph connector (read) | **Built + validated live (CDX)**, read-only ([ADR-0013](adr/ADR-0013-live-graph-connector.md)). Mock remains the default/CI path. | Migrate to `Sites.Selected`; validate live deltas/change sequence | | Microsoft Graph connector (write) | **Not built** — read-only app | Separate remediation identity + write permissions | | Entra ID sign-in | **Not built** — seam + production lockout only; OIDC flow returns 501 | Entra app registration; implement code flow against the documented seam | | Purview label read (live) | Built, gated `FMD_FEATURE_PURVIEW_LABEL_READ`; honest `unknown` when off | Grant `InformationProtectionContent.Read.All` + enable flag | | Purview label write (live) | Built, gated + separate remediation identity; refuses if unconfigured | Protected/metered `assignSensitivityLabel` enablement + remediation app | | File-type extraction | **Built + validated live (CDX)**: text/DOCX/XLSX/PPTX/HTML/RTF/PDF (172/174 CDX files extracted) | OCR/image for scanned PDFs & images (backlog P2) | | Distributed scanners | Built: fleet registration/heartbeat/takeover; runs in-process or standalone | Container/K8s packaging + signed work envelopes (P1) | | Metrics / observability | Built: counters/gauges/histograms in Operations | Distributed traces (P1) | | Azure OpenAI / Anthropic AI providers | Adapters implemented, config-gated, **not exercised by tests** (deterministic mock is the default) | Provide endpoint/key refs; contract identical to mock | | Change notifications | Built: subscription lifecycle + webhook receiver (clientState HMAC) + reconcile; dev simulator validated live against CDX | Public HTTPS webhook URL (`FMD_WEBHOOK_PUBLIC_URL`) + flag for live Graph subscriptions | | Teams connector | Built, mock-validated (file attachments canonicalize to the backing SharePoint driveItem); live gated | `FMD_FEATURE_TEAMS_CONNECTOR` + `ChannelMessage.Read.All` (protected API) | | Exchange connector | Built, mock-validated (selected mailboxes, per-folder delta; attachment↔SharePoint match is **similarity, not identity**); live gated | `FMD_FEATURE_EXCHANGE_CONNECTOR` + `Mail.Read` scoped to selected mailboxes | | Embeddings / vector index | **Built + validated live (CDX)**: deterministic default + cosine LSH; semantic-neighbors API; 172 embeddings indexed | Azure OpenAI embeddings adapter for provider vectors (config-gated) | | Model-release pipeline | **Built + validated live (CDX)**: curate → snapshot → evaluate → SoD approve → promote/rollback, ran on live CDX data | — (governed loop complete) | | Admin setup/config UI | **Built + validated live (CDX)**: `/setup` wizard + settings + connector probe | — | | Control-plane telemetry | Built: signed entitlement + strictly allowlisted fleet health + opt-out | Actual Control endpoint to receive telemetry (out of scope; no content leaves) | | Azure deployment | Built: `infra/main.bicep` (managed identity, Key Vault, Container App) | `az bicep build` in a CLI env; Postgres + separate worker for scale | | Activity signals | **Not built** — `not_enabled` state surfaced honestly in risk coverage | Graph activity APIs evaluation (backlog P1) | Beyond that table, the core product loop — kernel (config, capability authorization, hash-chained audit, redacting logger), governance and taxonomy, fixture connector and staged scan pipeline, intelligence (detectors, similarity, ensemble, risk, findings), review with propagation, the scoped Analyst, and the action state machine — is implemented and tested end-to-end (196 tests: unit, integration, security, acceptance). Only the **read-only inventory path** has ever touched a live tenant. Remaining work is prioritized in [roadmap.md](roadmap.md) and the STATUS.md backlog. ## Architecture at a glance Find My Data is a **Bun/TypeScript modular monolith** ([ADR-0001](adr/ADR-0001-runtime-language-repo-layout.md), [ADR-0002](adr/ADR-0002-modular-monolith-and-worker.md)): one deployable server process hosts the HTTP API and an embedded durable worker (the worker can also run standalone via `bun run worker` against the same queue). All persistence is a single SQLite file (WAL) behind logical **role interfaces** — system of record, full-text search projection, typed graph nodes/edges, durable work queue with leases and dead letters, and a purpose-flagged artifact store (82 tables across 12 migrations) — so each role can later migrate to a dedicated technology without rewriting callers ([ADR-0003](adr/ADR-0003-persistence-single-sqlite-behind-role-interfaces.md)). Sources plug in through a versioned **connector contract** ([ADR-0008](adr/ADR-0008-connector-contract-and-mock-m365.md)) with a deterministic mock M365 tenant as the default and the live Graph connector config-gated; AI goes through a **provider-neutral gateway** (deterministic mock default; Azure OpenAI and Anthropic adapters config-gated, [ADR-0005](adr/ADR-0005-ai-provider-gateway.md)); and the web UI is a React (Vite) app in `packages/web/`. The full actual-code diagram and staged-pipeline detail are in [architecture.md](architecture.md); consequential decisions are recorded in [docs/adr/](adr/). ## Product invariants These properties come from the original [handoff package](../find_my_data_handoff/README_FIND_MY_DATA_HANDOFF.md) and do not change without an explicit, documented decision: 1. **Data-domain-owner first.** The main experience serves people who understand the business meaning of data, not only security engineers. The home page, review queue, landscape, and Analyst are all built for the domain owner. 2. **Evidence before assertion.** Every finding, classification, relationship, and AI answer exposes its evidence, confidence, provenance, and version. The Analyst answers with interpretation, scope, evidence-row IDs, coverage, freshness, and blind spots ([ADR-0009](adr/ADR-0009-analyst-typed-plan.md)). 3. **Human authority.** AI proposes; authorized humans confirm ownership, material classifications, and remediation. Propagation reprioritizes related candidates but never auto-confirms; actions require separate approval ([ADR-0010](adr/ADR-0010-remediation-state-machine.md)). 4. **Honest staging.** The platform reports what is mocked, gated, blocked, or unknown — in the UI, in `/api/ready`, and in [STATUS.md](../STATUS.md). Unknown, unsupported, and inaccessible states stay distinct; coverage, confidence, and risk are reported separately; gated integrations fail with explicit errors rather than pretending to work. 5. **Deny-by-default authorization.** A request is allowed only when the session principal holds the required capability at a scope covering the resource. Roles are a UI convenience; capabilities and domain scopes are the enforcement, with deliberate exclusions (e.g., platform admins get no evidence access). The handoff README lists the full set of ten invariants, including customer control of content, incremental-by-design scanning, provider-neutral AI, and approved remediation. Related reading: [threat model](threat-model.md), [permissions manifest](permissions-manifest.md), [capacity model](capacity-model.md).