At 09:14 on Wednesday May 13, 2026 the Chief AI Officer of Acme Inc. opened a Grafana dashboard the Head of AI Operations had built four months earlier for ServiceAssist v1.0, the company's Annex III high-risk customer-service copilot, placed on the EU market in February. A single panel told the story: retrieval relevance for the production RAG corpus had dropped from a 30-day rolling baseline of 87% to 76% between May 1 and May 11. The MTTD, mean time to detect, read 38 hours. Not minutes. Hours. The triggering event, a routine knowledge-base document refresh authored on April 30, had quietly degraded the assistant's grounding quality for ten days before the drift detector crossed its alerting threshold. No Article 73 serious-incident threshold crossed. No Annex XII deployer complaint. The system still within its declared Article 15 accuracy band. But the CAIO's question, typed into the PMM Slack channel at 09:17, was the question every L4 reader of this lesson must answer in 2026: "Why wasn't MTTD 38 minutes?" This lesson is the answer: the bridge between the Article 72 post-market monitoring system as a design artefact and the run-state operational discipline that makes it defensible at audit, at incident, and at the AIGC table: KPIs, drift formulas, escalation tiers, CAPA workflow, deployer-loop closure, tooling stack, and the integration with Articles 26(5), 43(4), and 73 that turns the PMM plan from compliance paperwork into a regulator-grade operational system.

Article 72 - The Nine-Element Plan, the Deployer Article 26(5) Counterpart, and the MDR Article 83/86 Pattern

Article 72(1) requires every provider of a high-risk AI system to "establish and document a post-market monitoring system in a manner that is proportionate to the nature of the artificial intelligence technologies and the risks." Article 72(2) requires "active and systematic" collection, documentation, and analysis of relevant data "which may be provided by deployers or which may be collected through other sources on the performance of high-risk AI systems throughout their lifetime." Article 72(3) authorises the Commission to specify the post-market monitoring plan's contents via implementing act and provides a template, in development with an expected publication horizon of late-2026 / early-2027. Article 72(4) provides a regulated-sector deeming for institutions covered by Union financial-services law where existing PMM regimes already satisfy the AI Act's requirements. Article 26(5) is the deployer counterpart: deployers monitor the operation on the basis of the instructions for use, and where the deployer "has reasons to consider that the use of the high-risk AI system in accordance with the instructions for use may result in that AI system presenting a risk within the meaning of Article 79(1)," informs the provider, the distributor, and the relevant market surveillance authority "without undue delay." Together Articles 72 and 26(5) form the closed monitoring loop: provider monitors, deployer monitors, provider receives deployer signals, both escalate to the MSA when thresholds cross.

The pattern underlying Article 72, the most useful reference for any L4 provider building a PMM system before the implementing act lands, is the EU medical device MDR Articles 83-86. MDR Article 83 requires manufacturers to "plan, establish, document, implement, maintain and update a post-market surveillance system" proportionate to risk class. MDR Article 84 requires a written post-market surveillance plan as part of the technical documentation. MDR Article 85 governs the PMS report for class IIa devices; MDR Article 86 governs the periodic safety update report (PSUR) for class IIb and III. The pattern is forty years old in medical-device regulation: documented plan, active and systematic data collection, periodic analysis, defined corrective action, defined regulator reporting, defined feedback into the risk-management system. Article 72 borrows the pattern wholesale. Providers who follow the MDR pattern as named reference build defensible systems. Providers who improvise build systems that fail conformity assessment when the implementing act lands and the auditor opens the gap-analysis page.

The Article 72 plan must include nine elements, derived from the regulation, the implementing-act draft circulated by DG CNECT in early 2026, and the MDR Article 84 analogue. (1) Description of intended purpose and how monitoring is keyed to it. Per Article 13's instructions-for-use definition, the intended purpose names the deployment context; PMM indicators are keyed to it. (2) Indicators collected. Performance (accuracy, latency, throughput); fairness (slice metrics under Article 10(5)); robustness (drift signals, adversarial-input rate); security (OWASP LLM Top 10, Agentic Top 10, MITRE ATLAS observations). (3) Methods and frequency of analysis. Continuous (streaming detectors with sub-hour latency for safety-critical signals); daily (drift, slice metrics); weekly (eval-suite full runs); monthly (KPI dashboard review with AIGC); quarterly (full PMM report); annual (plan review). (4) Trigger thresholds for corrective action. Per indicator, red-amber-green levels. (5) Personnel and responsibilities. Named owner (typically Head of AI Operations); analyst rotation; on-call cover; escalation paths. (6) Reporting cadence, internal and external. Internal monthly AIGC; quarterly executive; biennial board AI subcommittee. External: Annex XII quarterly to deployers; MSA on demand under Article 74(3); AI Office for GPAI under Article 55. (7) Article 73 serious-incident integration. PMM feeds the serious-incident detector; classification crosses to the Article 73 reporting clock with documented handoff. (8) Article 43(4) substantial-modification trigger workflow. Where PMM data indicates intended-purpose drift, training-data domain shift, or retrain producing different conformity profile, the change-control ticket opens. (9) Annex XII downstream-deployer information loop. GPAI providers and Annex III deployer-supplying providers route PMM information back to deployers, the bidirectional flow that keeps the deployer's Article 26(5) obligation grounded in current provider data.

The Article 17 QMS build (lesson 079) holds the Article 72 plan as Section 9 (operations) and Section 10 (performance evaluation) operating evidence. The plan is itself a controlled document under Section 16 - Doc ID PMM-PLAN-{system-id}-v{n}, owner Head of AI Operations, approver CAIO + CQO, retention 10 years per Article 18, review semi-annual with material-change updates. Referenced from Annex IV §6 (monitoring description) and §9 (changes), keeping the technical file and the PMM plan synchronised. Article 26(5) deployer obligations sit one bracket downstream: the provider's Annex XII package tells the deployer what monitoring the provider performs, what data the deployer should collect, how to report material observations back. The Article 72(4) deeming only applies where an existing financial-services PMM regime is "equivalent", the burden sits with the provider; the cross-walk register is the discharge document.

The Eight-Pillar PMM Operations Architecture - Telemetry, Drift, Fairness, Security, Feedback, KPIs, CAPA, Reporting

Translating the nine-element plan into running operations requires eight architectural pillars. Each pillar maps to defined tooling, defined ownership, defined cadence, and defined evidence. The pillars build in dependency order, telemetry first, dashboards last, but operate in parallel once stood up.

Pillar 1 - Telemetry layer. The foundation. Captures every production inference's structured signal: timestamp, model version (semver + content hash), prompt fingerprint (hash, length, tokens, language), retrieval-source IDs (RAG), tool-call sequences (agents), output fingerprint (hash, length, tokens, refusal flag, judge-model rating), latency components (p50/p95/p99 per layer), salted-hash user-session ID, deployer-context ID. OpenTelemetry spans across the inference pipeline; Loki / ELK structured logs with cardinality limits; Prometheus / Mimir metrics; S3 / Azure Blob archival with content-hashed object names. Privacy: PII never logged; user-prompt-text retention follows Article 19 (6 months minimum, longer where law requires); Article 18 ten-year retention applies to aggregate metrics and analytical artefacts, not per-request raw prompts. The CycloneDX 1.7 ML-BoM (lesson 029) tracks every model version run; the telemetry layer tags every span with version for regression and rollback. Pillar 2 - Drift detection. Four signal families. Input-distribution drift via PSI = Σ (P_current[i] - P_baseline[i]) · ln(P_current[i] / P_baseline[i]) over input-feature bins; PSI < 0.1 stable, 0.1-0.25 minor, > 0.25 significant. Output-distribution drift via KL divergence on judge-rated buckets, KL = Σ P_current[i] · ln(P_current[i] / P_baseline[i]); thresholds calibrated empirically. Performance drift via rolling 30-day accuracy vs at-deployment baseline; alert when delta exceeds 2σ of baseline noise floor or a fixed business threshold (typically -5 to -10 pp). Refusal / abstention shift via percent-refused queries by category week-over-week; inflation indicates prompt-template degradation, RAG corruption, safety-classifier mis-calibration, or upstream model behaviour change. Latency drift via p95/p99 vs SLA; leading indicator of vendor or infrastructure change.

Pillar 3 - Fairness slice monitoring. Continuous, not point-in-time. Intersectional cohort drift across protected classes (where applicable under Article 10(5)). Disparity ratio per the four-fifths rule (lowest-group selection rate ≥ 80% of highest-group rate; lessons 047-048); slicing keyed to deployment context (employment by gender × age × ethnicity where data permits; lending by gender × race × age). Slicing-dimension list and disparity-ratio thresholds are controlled-document artefacts approved by AIGC; ad-hoc production slicing is forbidden because cherry-picked slices are statistical lying. Pillar 4 - Safety / security incident detection. Pattern detectors for the OWASP LLM Top 10 (prompt injection, insecure output handling, training-data poisoning, MDoS, supply-chain, sensitive-info disclosure, insecure plugin, excessive agency, overreliance, model theft); OWASP Agentic Top 10 (memory poisoning, tool misuse, intent breaking, identity spoofing, unexpected RCE, prompt-injection cascade, hallucination chain, repudiation, overwhelming HITL, agent rogue action ASI10); MITRE ATLAS techniques. Detector outputs route to SIEM (Splunk, Sentinel) for correlation. Pillar 5 - User-feedback loop. In-product feedback (thumbs-up/down with comment; report-a-problem; survey samples), support-ticket signals (AI-taxonomy categorised), deployer escalations (Annex XII channel), regulator signals (MSA inquiries, Article 86 whistleblower channel). Every feedback item gets ID, classification, owner, resolution status, and CAPA bridge where corrective-action criteria are met.

Pillar 6 - KPI dashboard with red-amber-green thresholds. The single integrated view the CAIO opens every morning and the AIGC reviews every month. Fourteen metrics with explicit thresholds, detailed next section. Pillar 7 - Corrective-action workflow (CAPA). The medical-device-borrowed pattern. Issue intake → root-cause analysis (5-Why / Fishbone / FRACAS for AI - Failure Reporting, Analysis, and Corrective Action System, the engineering discipline most useful for AI failure-mode tracking) → containment → corrective action → preventive action → verification of effectiveness → AIMS record update → Annex XII downstream notification. Each CAPA gets CAR-ID, owner, target date, status, evidence link, signed-by, retention. Pillar 8 - Reporting cadence. Monthly AIGC briefing (KPI dashboard + exceptions narrative); quarterly executive PMM report; biennial board AI subcommittee; quarterly Annex XII deployer package; Article 74(3) MSA on-demand disclosure; Article 73 serious-incident reporting on the applicable clock (10/2/15 days per lesson 025); Article 55(1)(c) AI Office reporting where the system uses a GPAI-with-systemic-risk model.

Pillars build in dependency. Pillar 1 without Pillar 2 is data lake without analysis. Pillar 2 without Pillar 6 is alerts without visibility. Pillar 6 without Pillar 7 is visibility without action. Pillar 7 without Pillar 8 is action without governance accountability. The complete eight-pillar architecture is what an external auditor, notified body Module H, certification body ISO 42001 surveillance, or MSA Article 74 inspection, expects when asking "show me your post-market monitoring system."

The 14-Metric KPI Dashboard, Drift Formulas, and Threshold Calibration

The KPI dashboard is the artefact that makes PMM legible to non-engineers: the CAIO, the AIGC, the audit committee, the regulator. Fourteen metrics, organised in four tiers, each with explicit red-amber-green thresholds and explicit data sources.

Tier 1 - Detection performance (4 metrics). (1) Drift-detection rate. Percent of weekly drift checks that completed successfully (not failed due to pipeline error). Green ≥ 99%; amber 95-99%; red < 95%. Source: drift-detection pipeline logs. (2) Mean time to detect (MTTD). Median elapsed time from drift-event onset to alert. Green < 6 hours; amber 6-24 hours; red > 24 hours. Source: synthetic drift-injection tests run weekly to calibrate MTTD; the Acme May-2026 38-hour reading sat solid red, hence the CAIO's question. (3) Mean time to repair (MTTR). Median elapsed time from alert to verified resolution. Green < 7 days; amber 7-14 days; red > 14 days. Source: CAPA workflow records. (4) Incident rate per million inferences. Count of P1/P2 incidents per million production inferences. Green < 5; amber 5-20; red > 20. Source: incident dossier aggregated to the inference-volume denominator.

Tier 2 - Quality and fairness (4 metrics). (5) Refusal-rate by topic category. Percent of queries refused, sliced by topic taxonomy. Green within ±2 pp of 30-day baseline per category; amber ±2-5 pp; red > ±5 pp. Source: refusal-classifier output joined to prompt-category classifier. (6) Fairness disparity ratio. Lowest-group selection rate ÷ highest-group selection rate across protected-class slices. Green ≥ 0.80 (four-fifths rule satisfied); amber 0.70-0.80; red < 0.70. Source: slice-monitoring pipeline. (7) Eval-pipeline pass-rate. Percent of weekly Promptfoo + Inspect eval-suite assertions passing. Green ≥ 98%; amber 95-98%; red < 95%. Source: CI eval-pipeline outputs. (8) User-feedback resolution percent. Percent of in-product feedback items closed within SLA (typically 14 days). Green ≥ 90%; amber 80-90%; red < 80%. Source: feedback-management system.

Tier 3 - Governance and change control (3 metrics). (9) Corrective-action closure percent. Percent of CAPAs closed within target date. Green ≥ 90%; amber 80-90%; red < 80%. Source: CAPA workflow records. (10) Substantial-modification trigger count. Count of Article 43(4) substantial-modification tickets opened in the last 90 days. Tracked trend, sudden spikes indicate either improving sensitivity (good) or fast-changing scope (concerning). Source: change-control system. (11) Vendor-model-version-change notification SLA. Percent of upstream foundation-model version changes (OpenAI, Anthropic, Google, Meta, Mistral, etc.) notified to AI Operations within the contractual SLA (typically 14 days). Green ≥ 95%; amber 80-95%; red < 80%. Source: vendor-notification log integrated to the ML-BoM update workflow.

Tier 4 - Incident, escalation, deployer (3 metrics). (12) Serious-incident count. Count of Article 73-reportable incidents in the trailing 12 months. Green 0; amber 1-2; red ≥ 3. Source: Article 73 reporting log. (13) Red-team finding closure percent. Percent of red-team findings (lessons 059-066) closed within severity-based SLA (Critical 30 days; High 60 days; Medium 90 days). Green ≥ 90%; amber 75-90%; red < 75%. Source: red-team finding register. (14) Deployer-loop escalation count. Count of Annex XII deployer feedback items received in the trailing 90 days, with sub-counts for severity. Tracked trend; sudden spikes indicate either improving deployer engagement (good) or deteriorating system performance in production fleets (concerning). Source: Annex XII feedback channel.

Threshold calibration is the discipline most providers underinvest in. Thresholds set too high (green-on-everything) generate silent failures, the system degrades but the dashboard reassures. Thresholds set too low generate alert fatigue and the cry-wolf failure mode. The defensible approach: thresholds set at deployment from the baseline plus noise-floor analysis; reviewed quarterly; re-calibrated semi-annually with AIGC sign-off; documented in PMM plan version-control. The Acme May-2026 incident had a contributory calibration cause: the retrieval-relevance drift threshold was set at deployment baseline (87%) minus 10 pp; 77% triggered alert; no amber band (77-85%) configured; the system went red at 76% without earlier warning. Post-incident: green ≥ 85%, amber 80-85%, red < 80%. The CAPA's preventive action was the threshold-band policy itself, every drift metric must have amber as well as red, every band reviewed quarterly.

The dashboard's secondary discipline is provenance: every metric click-throughs to source: drift-detection rate to pipeline logs, fairness-disparity ratio to slice-monitoring numerator/denominator, serious-incident count to the Article 73 log with classification memos. Auditors who can click-through form one impression; auditors who see numbers nobody can explain form another. Grafana with Loki/ELK panels is the 2026 reference implementation; commercial stacks (DataDog, Arize, WhyLabs, Fiddler) ship the same pattern with pre-built integrations.

CAPA Workflow, 4-Tier Escalation, and Integration with Article 73, Article 43(4), and Annex XII

The CAPA workflow operationalises Pillar 7, what happens between detection and verified resolution. Discipline borrowed from medical-device, aerospace, and automotive quality engineering with AI-specific extensions. Eight-step pattern. Step 1 - Issue intake. Drift alert, fairness breach, security event, user-feedback signal, deployer escalation, internal-audit finding, or regulator inquiry opens a CAR (Corrective Action Request) with ID, owner, triage SLA (typically 24 hours), routed to the responsible team. Step 2 - Triage and classification. Severity (Critical/High/Medium/Low); Article 73 reportability evaluation (does this cross Article 3(49)?); Article 43(4) substantial-modification evaluation (intended-purpose or design change?); tier-1 escalation evaluation. Triage signed by Head of AI Operations. Step 3 - Containment. Immediate measures: model rollback, traffic-routing override, feature flag, tool-allowlist tightening, HITL addition, system pause. Runs parallel with later steps; not gated by RCA. Step 4 - Root-cause analysis. 5-Why, Fishbone/Ishikawa (method, machine, material, manpower, measurement, environment), or FRACAS for high-severity. Output: documented root cause distinguishing immediate cause (proximate failure) from systemic cause (underlying condition). Step 5 - Corrective action. Fix the immediate cause; specific, measurable, time-bound; verification criteria defined. Step 6 - Preventive action. Fix the systemic cause so the issue cannot recur. Often a policy, control, training, or procurement-clause change. Step 7 - Verification of effectiveness. Evidence both actions worked, follow-up measurements at 30/60/90 days showing no recurrence. Step 8 - AIMS record update + Annex XII notification. Controlled documents reflecting new state updated; Annex XII deployer package updated where deployer-relevant; CAR closed with evidence retained 10 years per Article 18.

The four-tier escalation matrix triggers convening based on severity and reportability. Tier 1 - Model-owner only. Routine drift, in-tolerance variance, low-severity feedback. Standard CAPA cadence; recorded in the PMM log but not escalated. Tier 2 - Cross-functional + AIGC monthly. Sustained drift, fairness slice approaching threshold, repeated user complaints in one category, security signal warranting investigation. CAF (lesson 077) convenes; AIGC briefed at the next monthly cadence; CAPA opened with target-date discipline. Tier 3 - Board AI subcommittee + AI Office notification. Multiple drift signals correlated; fairness threshold breached on protected-class slice; security incident with material customer impact; substantial-modification indicator triggered. Board AI subcommittee convenes within 5 business days; AI Office notified where the system uses a GPAI-with-systemic-risk model (Article 55(1)(c)); CAPA with executive sponsorship. Tier 4 - Article 73 clock + Article 86 stakeholder notification. The event crosses Article 3(49): death or serious harm to health (73(1)(a)), serious-and-irreversible critical-infrastructure disruption (73(1)(b)), widespread or single-instance fundamental-rights infringement (73(1)(c)), serious harm to property or environment (73(1)(d)). The 10/2/15-day clock starts (lesson 025); Article 86 deployer and user notification flows in parallel; AIGC meets immediately; General Counsel owns the reporting decision; full incident response policy runs. Tier escalation is a one-way ratchet, events move up as facts develop, cannot move down without documented rationale signed by the AIGC chair.

Three regulatory integration points deserve named coverage. Article 73 integration. Pillar 4 (safety/security) and Pillar 5 (user feedback) feed the serious-incident detector. CAPA step 2 explicitly asks: does this meet Article 3(49)? If yes, the 10/2/15-day clock starts; the incident response policy runs; the General Counsel signs the reporting decision; the MSA receives the initial report. The PMM dossier and the Article 73 incident dossier are the same dossier, unified record from detection through reporting through closure, retained 10 years per Article 18. Article 43(4) substantial-modification integration. Where PMM data indicates intended purpose or design has substantively changed, domain shift sufficient to alter safety profile, retrain producing materially different conformity behaviour, intended-purpose creep where deployer use patterns sit outside the declared purpose, the substantial-modification ticket opens. This may trigger re-conformity-assessment, Annex IV technical-documentation update, Article 47 declaration refresh, notified-body re-engagement. The PMM system is the early-warning detector; missing the warning means discovering the substantial modification at the next surveillance audit, an expensive surprise. Annex XII downstream-deployer loop. Bidirectional. Provider → deployer: instructions-for-use updates, performance bulletins, security advisories, version-change notifications, end-of-life. Deployer → provider: Article 26(5) operational observations, Article 79(1) risk signals, Article 73 serious-incident escalations, deployer-specific performance feedback. The Annex XII package as a controlled document (versioned, 10-year retention, deployer-acknowledgment tracked) closes the loop. Providers who fail to operationalise discover the failure when a deployer escalation arrives via a regulator letter rather than the contracted channel, at which point the regulator has a parallel question about why the channel didn't work.

Tooling Stack 2026, Common Failure Modes, and the Acme ServiceAssist Worked Example

The 2026 tooling stack has settled. Open-source spine: Promptfoo CI eval-pipeline runs (every PR/release/weekly cron); Garak nightly red-team probing (lesson 062); PyRIT scheduled scenario attacks (lesson 063); Inspect periodic structured evaluations (lesson 061); OpenTelemetry trace instrumentation; Loki/ELK log aggregation; Prometheus/Mimir metrics; Grafana visualisation; Trivy/Syft ML-BoM scanning; CycloneDX 1.7 ML-BoM authoring (lesson 029). Commercial AI observability: DataDog AI observability; Arize AI (embedding drift); WhyLabs (auto-baseline-learning); Fiddler AI (explainability + monitoring); Patronus/Galileo (LLM-specific). SIEM: Splunk or Sentinel for security correlation; PagerDuty for on-call; SOC threat-intel integration. CAPA/QM: ETQ, MasterControl, Jama Connect, or the QMS Section 16 controls (lesson 079). Dossier: Confluence/SharePoint controlled spaces; specialised AI-QMS platforms emerging in 2026 (Credo AI, Holistic AI, Trustible). Budget for a mid-size provider: tooling €150K-€500K/year, personnel 2-5 FTE running €400K-€1.5M/year for 3-10 high-risk systems.

Six recurring failure modes in first-generation 2026 PMM implementations. (1) Build at go-live, never refresh thresholds. Thresholds drift over time; without quarterly review, the dashboard reassures while the system degrades. (2) Telemetry without analyst time budget. Beautiful dashboards nobody reads. The 2-5 FTE budget is the difference between a PMM system and a PMM screensaver. (3) Thresholds too high (silent failure). Dashboard stays green; failure surfaces as a regulator inquiry or deployer escalation rather than internal alert. (4) No Article 73 integration. PMM detects serious-incident-class event; classification doesn't reach the General Counsel; the clock starts unreported; Article 99(3) €15M/3% exposure amplified by the timeline breach. (5) No Annex XII deployer-loop closure. Deployer feedback never reaches PMM analysts; provider feedback never reaches deployers; the bidirectional expectation unmet. (6) Eval-pipeline drift. The Q1 2026 Promptfoo suite never refreshed; the pipeline reports green while testing yesterday's failure modes against today's model. Mitigation: explicit refresh cadence (monthly review, quarterly substantive update, semi-annual full rebuild with red-team finding integration) plus the eval-pipeline-pass-rate metric on the dashboard.

Acme's ServiceAssist v1.0 worked example, Q2 2026 PMM operating record. System: Annex III high-risk customer-service AI copilot placed on the EU market February 14, 2026 by Acme AG (Germany), deployed to 87 enterprise customers across 14 Member States. Foundation: domain-fine-tuned variant of an upstream GPAI flagship reasoning model; RAG over 1.4M documents; tool-call access to ticket-management, knowledge-base-search, email-draft (constrained scope, no autonomous execution). 9-element plan: PMM-PLAN-SVCAST-v1.0 approved Feb 8; owner Head of AI Operations Sven Kraus; approvers CAIO + CQO; semi-annual review. 8-pillar architecture stood up January 5 - February 12. Telemetry: OpenTelemetry → Loki + Prometheus → Grafana; S3 archival with content-hashed objects. Drift: PSI on input-language and topic-category distributions (daily); KL divergence on judge-rated output buckets (daily); rolling 30-day accuracy on 500-question reference set (weekly); refusal-rate by category (continuous). Fairness: language × tenant × topic-category (no protected-class data; system out of Article 10(5) scope). Security: Promptfoo CI; Garak nightly; PyRIT weekly; Splunk SIEM. Feedback: thumbs/comment, report-a-problem, quarterly NPS. Dashboard: 14-metric standard. CAPA: ETQ. Reporting: monthly AIGC; quarterly Annex XII via provider portal.

Drift event May 1-11. Trigger: April 30 KB refresh: 412 new articles, 1,180 updates, 287 retirements; embedding indices rebuilt overnight, completing 02:14 UTC May 1. Retrieval-relevance metric (% of production queries where top-3 retrieved chunks include the gold-standard reference passage from a 2,400-question suite) dropped: 87.1% April 30 → 84.7% May 1 → 82.3% May 4 → 79.8% May 7 → 76.4% May 11. Threshold (77%, deployment baseline 87% minus 10 pp) crossed May 11 14:38 UTC. Alert fired; PagerDuty paged Kraus; CAR-2026-0387 opened. MTTD from drift-event onset (May 1 02:14) to alert (May 11 14:38) = 260 hours = 10.85 days. Reported in the dashboard as "38 hours" because the dashboard's MTTD calculation used the prior daily reading's date (May 7 → May 11 ≈ 4 days), not the May 1 actual onset. The May 13 CAIO question, "why wasn't MTTD 38 minutes?", corrected to 260 hours, intensified.

CAPA execution May 11-19. Containment May 11 14:38-18:00: model rollback rejected (model not broken; retrieval was); embedding-index rollback to April 30 02:00 snapshot; retrieval-relevance back to 86.4% within 6 hours. RCA May 12-14: 5-Why traced "76%" → "doc update broke embeddings" → "new content used different terminology than the eval suite expected" → "eval-suite gold passages never re-indexed against the doc-update" → "eval-suite refresh process didn't trigger from doc-update process" → "two pipelines stood up independently with no integration". Systemic cause: pipeline integration gap. Corrective May 14-17: eval-suite gold-passage regeneration auto-triggered by any doc-update. Preventive May 17-19: doc-update playbook updated; integration test added to doc-update CI; threshold-band recalibration policy adopted (every drift metric needs amber as well as red, quarterly review). Verification May 19: synthetic doc-update test triggered eval-suite refresh; retrieval-relevance held at 86.4%. CAR closed May 19 16:22 UTC. MTTR from alert to verified resolution = 8 days 1 hour 44 minutes, solid green (< 14 days).

No Article 73 trigger: retrieval-relevance drift, not a fundamental-rights infringement, critical-infrastructure disruption, death-or-harm event, or serious property/environmental harm. No clock started. No Article 43(4) trigger: intended purpose, model architecture, and conformity profile unchanged after rollback. Annex XII deployer notification May 20: Q2 bulletin disclosed event with full root-cause and CAPA evidence. Two deployers acknowledged no follow-up; one requested confirmation eval-suite was version-controlled (provided May 22). AIGC briefing June 4: learning case; cross-system audit of every Acme high-risk AI's eval-suite refresh process surfaced two further pipelines with similar gaps (both CAPA'd by end-June). The CAIO's question, "why wasn't MTTD 38 minutes?", became the June 4 paper title; the answer: "because the drift detector and the eval-pipeline ran on separate baselines and neither watched the other." Sub-hour MTTD requires either real-time embedding-quality monitoring (Q3 2026 roadmap) or a cross-pipeline integration test that fails the doc-update CI before merge. Both controls were added.

Penalty Exposure and Cross-Walk - Articles 9, 11, 15, 17, 18, 26(5), 43(4), 47, 71, 72, 73, 86; ISO 42001 A.6.2.7 + A.9; NIST AI RMF; SR 11-7 Pillar 5

The penalty cascade for Article 72 failure concentrates in Article 99(3). Article 99(3), €15M or 3% of worldwide annual turnover (whichever higher) for Article 72 post-market monitoring failures and Article 26(5) deployer monitoring failures. Material gaps, no documented plan; plan with missing elements; no operational data collection; no analysis cadence; no thresholds; no CAPA; no Annex XII deployer-loop closure, sit squarely in this tier. The Article 73 timeline-breach amplifier adds to the exposure where a PMM detection fails to escalate to the serious-incident reporting clock; the missed-clock failure under Article 73 is itself an Article 99(3) breach. Article 99(5), €7.5M or 1% applies where the misleading-record-keeping standard is met: misrepresenting what PMM data exists, what thresholds were configured, what alerts fired, what corrective actions were taken. The combination is concerning: a provider with weak PMM that also misrepresents the weakness to a regulator faces stacked exposure.

The full cross-walk for an Article 72 PMM system, the audit-day reference the Head of AI Operations should be able to recite, spans AI Act, standards, and U.S. equivalents. EU AI Act: Article 9 (risk management, PMM data feeds back into Article 9 risk register); Article 11 (technical documentation Annex IV §6 monitoring description and §9 changes); Article 15 (accuracy, robustness, cybersecurity, PMM detects breaches of declared performance); Article 17 (QMS, PMM as one operational system within the QMS, lesson 079); Article 18 (record retention, 10 years for PMM dossier); Article 26(5) (deployer monitoring counterpart); Article 43(4) (substantial-modification integration); Article 47 (declaration refresh on substantial modification); Article 71 (EU database registration and post-market data); Article 72 (the post-market monitoring system itself, the subject of this lesson); Article 73 (serious-incident reporting integration); Article 74 (market surveillance authority cooperation including Article 74(3) data access); Article 86 (whistleblower / stakeholder notification); Article 99(3) (€15M / 3% penalty tier). Annexes: Annex IV §6 (post-market monitoring description), §9 (changes description); Annex XII (downstream-deployer information loop for GPAI; analogue practice for Annex III deployer-supplying providers). Standards: ISO/IEC 42001:2023 A.6.2.7 (post-deployment monitoring), A.9 (performance evaluation), Clause 9.1 (monitoring, measurement, analysis, evaluation), Clause 9.3 (management review). NIST AI RMF 1.0 (2023): Manage 1.1 (AI risk and benefit assessment), Manage 1.3 (prioritisation), Manage 2.1 (resource allocation), Manage 4.1 (post-deployment monitoring documented), Manage 4.3 (incident response procedures); Measure 2.7 (security and resilience), 2.8 (interpretability), 2.10 (privacy), 2.11 (fairness and bias), 2.12 (environmental impact), 3.1 (performance metrics), 3.2 (functionality measurement), 4.1 (regular tracking of approved metrics), 4.2 (measurement results re-evaluated periodically). NIST AI 600-1 GenAI Profile (July 2024): applicable to the 12 GenAI risks identified: confabulation; dangerous, violent, or hateful content; data privacy; environmental impacts; harmful bias or homogenisation; human-AI configuration; information integrity; information security; intellectual property; obscene, degrading, and/or abusive content; value chain and component integration; CBRN information or capabilities. U.S. prudential: SR 11-7 ongoing-monitoring pillar (Pillar 5) for U.S.-supervised banks; OCC Bulletin 2011-12; FRB SR 15-19; PRA SS1/23 principle 5 ongoing monitoring; FCA AI governance expectations.

The Acme ServiceAssist worked example, 38-hour MTTD reading the CAIO questioned, 260-hour actual MTTD, 8-day MTTR, full CAPA, no Article 73 trigger, Annex XII notification on cadence, AIGC learning case June 4, illustrates the run-state discipline that distinguishes a PMM plan from a PMM system. The plan is a controlled document; the system is what operates while the document sits on a shelf. The eight pillars, fourteen metrics, four escalation tiers, eight-step CAPA, three regulatory integration points, and 2026 tooling stack, assembled with documented thresholds, named owners, defined cadence, and the budget to staff it, are what the notified body finds in surveillance, what the MSA finds in Article 74 inspection, what the AIGC reviews monthly, what the board AI subcommittee briefs quarterly. Build the system, not just the plan. The Commission's Article 72(3) template, when it lands, will codify what MDR Articles 83-86 have shown for forty years: monitor actively, systematically, with documented thresholds, cross-functional accountability, the deployer-loop closed, and the corrective-action discipline that turns drift into resolved CAR records rather than regulator letters.

Key Takeaways

• Article 72 mandates a documented post-market monitoring system with nine plan elements, intended-purpose mapping; indicators; methods/frequency; trigger thresholds; personnel; reporting cadence; Article 73 integration; Article 43(4) integration; Annex XII deployer-loop. Article 26(5) is the deployer counterpart. MDR Articles 83-86 are the named precedent until the Commission's Article 72(3) implementing-act template lands.
• The 8-pillar PMM operations architecture: telemetry; drift detection; fairness slice monitoring; safety/security incident detection (OWASP LLM Top 10, Agentic Top 10, MITRE ATLAS); user-feedback loop; KPI dashboard with red-amber-green thresholds; CAPA workflow; reporting cadence to AIGC and competent authorities. Pillars build in dependency; the complete architecture is what auditors expect to see.
• Drift formulas with concrete thresholds, PSI for input-distribution drift (< 0.1 stable, > 0.25 significant); KL divergence on judge-rated output buckets; rolling 30-day accuracy vs deployment baseline (alert at -5 to -10 pp); refusal-rate by category (±2-5 pp bands); latency p95/p99 vs SLA. Thresholds calibrated at deployment, reviewed quarterly, re-calibrated semi-annually with AIGC sign-off.
• The 14-metric KPI dashboard spans four tiers: detection performance (drift-detection rate, MTTD, MTTR, incident rate per million inferences); quality and fairness (refusal-rate by category, fairness disparity ratio, eval-pipeline pass-rate, user-feedback resolution percent); governance and change control (CAPA closure percent, substantial-modification trigger count, vendor-version notification SLA); incident, escalation, deployer (serious-incident count, red-team finding closure percent, deployer-loop escalation count). Every metric click-throughs to source.
• CAPA workflow follows the medical-device pattern: issue intake; triage with Article 73 and Article 43(4) reportability evaluation; containment; root-cause analysis (5-Why, Fishbone, FRACAS for AI); corrective action; preventive action; verification of effectiveness; AIMS record update plus Annex XII downstream notification. Each CAR retained 10 years per Article 18.
• The 4-tier escalation matrix, T1 model-owner-only routine drift; T2 cross-functional + AIGC monthly; T3 board AI subcommittee + AI Office notification; T4 Article 73 serious-incident clock (10/2/15 days) + Article 86 stakeholder notification. Escalation is a one-way ratchet.
• The 2026 tooling stack has settled, Promptfoo + Garak + PyRIT + Inspect open-source eval and red-team; OpenTelemetry + Loki + Prometheus + Grafana telemetry spine; DataDog/Arize/WhyLabs/Fiddler commercial AI observability; Splunk/Sentinel SIEM integration; ETQ/MasterControl CAPA management; CycloneDX 1.7 ML-BoM versioning. Personnel 2-5 FTE for mid-size provider; budget €150K-€500K tooling plus €400K-€1.5M personnel annually.
• Penalty exposure concentrates in Article 99(3) €15M / 3% for Article 72 + Article 26(5) failures, amplified by Article 73 timeline breach where missed serious-incident reporting compounds the PMM failure. Six recurring failure modes, go-live thresholds never refreshed; telemetry without analyst budget; thresholds too high (silent failure); no Article 73 integration; no Annex XII deployer-loop closure; eval-pipeline drift. The Acme worked example, 38-hour reported MTTD vs 260-hour actual, 8-day MTTR, full CAPA, no Article 73 trigger, AIGC learning case, illustrates the run-state discipline that distinguishes a PMM plan from a PMM system.