How to read this catalog
Validation states. Find My Data never overstates its state. Each category below is marked with one of:- Live-validated (CDX) — exercised against a live Microsoft 365 demo tenant (Contoso CDX), read-only (ADR-0013).
- Mock-validated, live gated — built and tested against the deterministic mock M365 fixture; the live path exists but is behind an explicit feature flag and has not been run against a real tenant.
- Not built — recorded intent only; called out explicitly.
FMD_DB_PATH, default ./data/fmd.db), or an optional PostgreSQL 16 backend (FMD_DB_BACKEND=postgres + FMD_DB_URL) with identical role interfaces. Content-derived blobs live in the artifact store — local disk by default (FMD_ARTIFACT_DIR, default ./data/artifacts, at <FMD_ARTIFACT_DIR>/<tenantId>/<artifactId>), or a customer-owned S3-compatible bucket or Azure Blob container via FMD_OBJECT_STORE (credentials by reference; content never leaves your deployment) — indexed by the artifacts table, whose backend column (migration 0022) records which backend holds each artifact’s bytes. Every tenant-scoped business table carries a NOT NULL tenant_id; the deliberate exception is the deployment-level scanners table (§7), which is infrastructure, not tenant data.
Retention. Retention durations are not enforced in this build. Artifacts carry a retention_class column (default standard) and ADR-0011 records per-class TTLs as a follow-up, but no scheduled deletion sweep exists (threat model §4.3). Where a row below says “until superseded,” the schema keeps temporal history (is_current, valid_from/valid_to, supersession chains) rather than deleting.
1. Evidence access levels
Evidence rows carry an explicit access classification, and reads are gated by distinct capabilities so that “can see that an asset exists” never implies “can read content excerpts.”
Enforcement facts (all tested):
- The capability list is defined in
packages/shared/src/capabilities.ts;platform_admindeliberately holds noasset.evidence.*,review.*, oranalyst.*capabilities. - A caller with summary access but not excerpt access receives evidence with
excerpts: null— never a lesser-redacted fallback (packages/server/src/review/service.ts). - Every evidence read is audited under category
evidence_access, with distinct actions for reads that included excerpts (candidate_evidence_with_excerpts_read) vs. reads that did not. - Artifact-store reads are additionally purpose-checked:
ObjectStore.getrequires a declared purpose against the artifact’s content-use flags (use_flags_json, §3) and throws when the artifact does not permit it.
2. Data-handling profiles
tenants.data_handling_profile accepts three values (migration 0001). The exact profile notes shipped in packages/server/src/admin/settings.ts:
Honest implementation state: only
minimized_evidence (the default) drives actual pipeline behavior, and the current build deviates from its own profile note in one documented way: bounded extracted text (≤ 20,000 characters) is retained, because source-free re-scoring on detector releases requires a text representation. This deviation, its rationale, and its follow-ups (envelope encryption, per-class TTLs, a true drop-text option) are recorded in ADR-0011 and the threat model §4.3. The metadata_only and enhanced_evidence values are schema-ready configuration only — validated, audited, and cache-invalidating (a profile change bumps the tenant config_version, which participates in the evidence-bearing stage cache keys — content, features, inference — so the next scan/rescore re-derives those stages; tested in scan.test.ts and admin.test.ts), but they do not yet change what the pipeline persists. A reviewer should treat the platform’s actual retention posture as: minimized evidence, plus bounded extracted text as the ADR-0011 deviation.
3. Catalog: content-derived data (highest sensitivity)
Everything in this section is derived from customer file content. Derivative data is still sensitive — none of these tables is “safe” merely because it holds no full text.
Validation state: extraction, fingerprints, keyphrases, detector hits, excerpts, and embeddings are live-validated (CDX) — 172/174 real files extracted, 172 embeddings indexed, re-score-without-refetch proven live. Azure OpenAI/Anthropic AI-provider paths are config-gated and unexercised; the deterministic mock is the only exercised provider.
4. Catalog: source observations (metadata acquired from connectors)
Source families. These observation tables are populated not only by the SharePoint/Graph connector but also by the OneDrive, Azure Blob Storage / Azure Files (incl. ADLS Gen2), on-prem SMB, SQL, Google Drive, and Box connectors, all through the same pipeline. Permission-observation completeness is honest per source: Google Drive sharing is fully observable (“anyone with the link”, domain-wide, and group shares are mapped as exposure, not guessed); Box collaborations and shared links are mapped, with an open link surfacing as anonymous exposure; Azure Blob container public-access levels surface as anonymous-exposure signals, and storage objects carry no Purview labels, so label observations honestly report
unsupported; SMB mounts cannot see NTFS ACLs, so permission observations are honestly reported as not_observable; SQL tables readable by the database’s public/guest role surface as broad exposure.
Validation state: the read-only Graph inventory/delta/permission/label-observation path is live-validated (CDX): 34 drives discovered, 174 assets via driveItem delta, deduplicated permission sets, labels honestly unknown under a read-only app. The mock connector remains the default and CI path (FMD_CONNECTOR_MODE=mock).
5. Catalog: messaging (Teams / Exchange)
Raw message bodies are never persisted. Both connectors persist the same bounded shape (packages/server/src/teams/service.ts, packages/server/src/exchange/service.ts):
- Preview: whitespace-normalized body, passed through
redactSensitivePreview, truncated to 200 chars (BODY_PREVIEW_MAX). - Redaction (
packages/server/src/intelligence/redact.ts) masks, in order: US SSNs (123-45-6789/ space-separated) →[redacted-ssn]; card-like 13–16 digit runs (optionally space/dash grouped) →[redacted-card]; bare digit runs of 9+ →[redacted-number]. Detector signals are kept separately; only raw values are masked. - Body hash:
sha256of the full body (body_hash) for change detection — non-reversible. - Detector signals:
detector_hits_jsonholds only{detectorKey, informationTypeName, count}(SafeDetectorHit) — no matched text.
Access: Teams/Exchange messages are not yet mapped to business-domain ownership, so their read views are admin/audit-scoped, deny-by-default — a known limitation tracked in STATUS.md.
Validation state: mock-validated, live gated. Both connectors are built and tested against fixtures (including a regression test that the preview redaction holds). Live paths require
FMD_FEATURE_TEAMS_CONNECTOR (+ ChannelMessage.Read.All, a protected API) and FMD_FEATURE_EXCHANGE_CONNECTOR (+ Mail.Read scoped by application access policy) and have not been run against a real tenant.
6. Catalog: governance and human-decision records
7. Catalog: operational data
8. Catalog: audit
audit_events (migration 0004) is the platform’s tamper-evident record:
- Append-only with a per-tenant SHA-256 hash chain (
prev_hash→hash, genesis-anchored);verifyAuditChainrecomputes the chain and reports the first broken link (packages/server/src/kernel/audit.ts, tested). - Categories:
auth,config,governance,scan,evidence_access,review,analyst,action,model,operations. details_safe_jsonis restricted to structured references (ids, versions, outcomes, booleans) by theappendAuditcontract and typed input — never excerpts, paths, prompts, or names. Honest nuance: unlike the logger (§10), this is a caller contract backed by tests, not a runtime key allowlist.- Read access requires the
audit.readcapability (held byauditor). - Limit (threat model T15): the chain detects tampering but does not prevent it against an attacker with direct DB write access; external anchoring is a documented follow-up.
9. What is never stored / never leaves the boundary
10. Log redaction policy
The logger (packages/server/src/kernel/log.ts) enforces redaction structurally, not best-effort:
- Only field keys on the
SAFE_KEYSallowlist are emitted (correlation/tenant/principal/work-unit ids, stage, status, outcome, error category, HTTP route/method/status, duration, counts,sizeBand, etc.). Unknown keys are dropped and counted (_droppedFields). - String values must match a conservative opaque-token pattern (
^[A-Za-z0-9_.:/-]{1,120}$) or are replaced with[redacted]— so free-text content (excerpts, prompts, multi-word names or titles) cannot pass even under an allowlisted key. The pattern alone would admit a single-token value such as a URL or hex hash; the end-to-end scan test below is what verifies those never appear in default logs. - Sizes are logged as bands (
lt10k…gte10m), not exact values, viasizeBand. - A ring buffer of the last 500 already-sanitized entries backs the operations UI.
- An end-to-end log-scan test asserts that default logs contain no names, paths, URLs, phrases, GUIDs, or hashes (
packages/server/src/testing/acceptance.test.ts; unit tests inpackages/server/src/kernel/log.test.ts; ADR-0011).
AppError messages, and runtime crash output to stderr is outside the logger’s control.
11. Fleet telemetry (strict allowlist)
Defined inpackages/server/src/controlplane/telemetry.ts (fleet-telemetry-v1), covering licensing + fleet health only. The Zod schema is .strict() — any field not on this list fails validation, so the payload fails closed rather than over-sharing:
schemaVersion, deploymentId (a non-reversible sha256 of the internal tenant id, truncated to 24 hex chars — never the tenant name or id), productVersion, collectedAt, entitlementId, entitlementState, edition (free/paid; the retired evaluation/standard/enterprise values are still accepted from older deployments), scannerCount, activeScannerCount, activeUserCount90d (v2 licensing meter — distinct signed-in principals over the trailing 90 days; optional for v1 compatibility), queueDepthBand (empty/low/moderate/high), deadLetterBand (none/low/elevated), scaleBand (small/medium/large/xlarge — raw asset counts are never reported), telemetryEnabled.
A SENSITIVE_FIELD_DENYLIST (name, path, excerpt, content, domain, informationtype, asset, email, principal, tenantid, secret, token, body, subject, count) backs a CI tripwire test so a future edit adding e.g. assetCount fails the build. scannerCount/activeScannerCount/activeUserCount90d are the only deliberately allowed *count fields (operational and licensing meters — activeUserCount90d counts distinct signed-in principals over the trailing 90 days and carries no identity, name, or content).
Customer controls: control_plane_state.telemetry_enabled is opt-outable at any time, and the UI shows a transparency preview of the exact payload. Honest state: the payload builder, transparency preview, and opt-out are built and tested; the vendor Control service (26.7.15.2) receives telemetry at a strict-allowlist ingest endpoint, and a deployment transmits only when a control-plane URL is configured — an offline deployment sends nothing.
12. Retention summary and open items
Open retention items a reviewer should track: retention durations and sweeps (not built), envelope encryption of evidence artifacts (not built), a true text-dropping
metadata_only profile (config-only today), and audit chain external anchoring (not built). Each is recorded in ADR-0011, the threat model, and the roadmap.