The Compliance Deadline You Cannot Ignore

In August 2026, the EU AI Act's high-risk system requirements take full effect. Organizations deploying AI in healthcare, hiring, credit scoring, law enforcement, or critical infrastructure face fines of up to 35 million euros or 7% of global annual revenue for non-compliance. This is not a distant threat: GPAI model obligations already became enforceable in August 2025, and prohibited AI practices have been banned since August 2024. Whether you are building foundation models or deploying domain-specific AI, you need a concrete evaluation program that maps directly to the Act's articles. This lesson gives you the technical evaluation playbook to achieve and demonstrate compliance.

Risk Classification and Its Evaluation Implications

The EU AI Act classifies AI systems into four risk tiers: unacceptable (banned), high-risk, limited-risk, and minimal-risk. Your first evaluation task is determining where your system falls. High-risk includes AI used in biometric identification, critical infrastructure management, education and vocational training access, employment decisions, essential services access, law enforcement, migration management, and administration of justice. Each tier triggers different evaluation obligations. A hiring-screening tool, for instance, falls squarely into high-risk and must satisfy Articles 9 through 15. A chatbot answering FAQs about store hours is minimal-risk with only transparency obligations. Map every AI system in your portfolio to its risk tier before designing evaluation procedures. Misclassification is itself a compliance violation.

Article 9: Building a Continuous Risk Management Evaluation

Article 9 mandates a risk management system that operates throughout the AI system's lifecycle. This is not a one-time audit. You must identify and analyze known and reasonably foreseeable risks, estimate those risks with available data, evaluate risks from intended use and reasonably foreseeable misuse, and adopt risk mitigation measures verified through testing. In practice, build a risk registry for each high-risk system. For a credit-scoring model, your registry might include entries like: discriminatory outcomes by protected class (severity: high, likelihood: medium), adversarial manipulation of input features (severity: high, likelihood: low), and model drift under macroeconomic shifts (severity: medium, likelihood: high). Each risk needs a quantified mitigation target and an evaluation metric proving the mitigation works.

Article 10: Data Quality Evaluation Requirements

Article 10 requires that training, validation, and testing datasets meet specific quality criteria. You must evaluate datasets for relevance, representativeness, freedom from errors, and completeness. Concretely, this means running demographic parity analyses across protected attributes, measuring label noise rates (aim for under 2% disagreement on gold-standard subsets), documenting data provenance chains, and assessing temporal validity. Build a data card for every dataset your high-risk system touches. Include distribution statistics, collection methodology, annotation guidelines, inter-annotator agreement scores (Cohen's kappa above 0.8 for high-risk applications), and known gaps. The Act specifically requires that datasets account for the specific geographical, contextual, and behavioral setting in which the system will be used.

Article 15: Accuracy, Robustness, and Cybersecurity Evaluation

Article 15 demands that high-risk AI systems achieve appropriate levels of accuracy, robustness, and cybersecurity. 'Appropriate' means relative to the system's intended purpose and the state of the art. For a medical diagnostic AI, you need to report sensitivity, specificity, positive predictive value, and negative predictive value across demographic subgroups. For robustness, run adversarial testing: perturb inputs with noise, edge cases, and distribution shifts, then measure performance degradation. A compliant robustness evaluation might show that your system maintains F1 above 0.92 even when 10% of input features are corrupted. For cybersecurity, evaluate resistance to data poisoning, model extraction, and prompt injection attacks. Document all results with confidence intervals.

General-Purpose AI Model Evaluation Obligations

If you provide a GPAI model (foundation model, large language model), Article 53 imposes additional obligations already in force since August 2025. You must perform and document model evaluations including adversarial testing, publish evaluation results on standardized benchmarks, and report known limitations. For systemic-risk GPAI models (those trained with more than 10^25 FLOPs), you must additionally conduct and report adversarial red-teaming results. In practice, this means running your model through benchmarks like HELM, MMLU, and domain-specific evaluations, then publishing results publicly. Track your evaluations against the emerging EU AI Office standards and the Code of Practice for GPAI providers. Use LLM-as-judge evaluations alongside human assessment to cover both scale and depth.

Conformity Assessment: The Evaluation of Your Evaluations

High-risk AI systems must undergo conformity assessment before market placement. For most categories, this is a self-assessment following Annex VI procedures. For remote biometric identification and critical infrastructure AI, you need third-party assessment by a notified body. Your conformity assessment documentation typically spans 50 to 150 pages and must include: a detailed description of the system and its intended purpose, your risk management evaluation results, data governance documentation, test logs and metrics, human oversight procedures, and accuracy and robustness test reports. Structure your evaluation pipeline to generate this documentation automatically. Build templates that map each Article's requirements to specific test suites, and ensure your CI/CD pipeline produces audit-ready artifacts with every model release.

Post-Market Monitoring and Continuous Evaluation

Compliance does not end at deployment. Article 72 requires a post-market monitoring system proportionate to the AI system's risk. For high-risk systems, you must actively collect and analyze performance data from real-world use. Build automated monitoring dashboards that track prediction accuracy, fairness metrics, and distribution drift in production. Set alert thresholds: if demographic parity difference exceeds 0.05, or if overall accuracy drops more than 3 percentage points from your conformity assessment baseline, trigger a review. Log all incidents and near-misses. The Act requires reporting serious incidents to market surveillance authorities within defined timeframes. Your evaluation infrastructure must support rapid root-cause analysis when issues arise.

Building Your Technical Documentation Stack

Annex IV specifies the required contents of technical documentation. Every evaluation you run must produce traceable, versioned artifacts. Use model cards (following the Mitchell et al. framework extended for EU AI Act fields) for each model version. Maintain dataset documentation using datasheets or data cards. Version your evaluation scripts alongside model code so that any historical result can be reproduced. Store evaluation results in structured formats (JSON or database records, not just PDFs) so they can be queried and compared across versions. A practical stack: MLflow or Weights & Biases for experiment tracking, a dedicated compliance database mapping each Article requirement to test results, and automated report generation that produces both machine-readable and human-readable compliance documents.

Try This Now: Compliance Gap Analysis

Pick one AI system your organization deploys or is developing. Walk through this checklist: (1) Classify its risk tier under the EU AI Act. (2) For each applicable Article (9 through 15), list what evaluation evidence you currently have and what is missing. (3) For Article 10, check whether your training data has documented provenance, demographic breakdowns, and inter-annotator agreement scores. (4) For Article 15, identify your accuracy metrics and whether you have adversarial robustness test results. (5) Estimate the gap: how many months of work would full compliance require? Most organizations find that data governance documentation (Article 10) and robustness testing (Article 15) are their largest gaps. Starting this analysis now gives you the lead time you need before August 2026 enforcement.

Key Takeaways

The EU AI Act creates legally binding evaluation requirements for AI systems deployed in or affecting EU citizens. High-risk systems must satisfy Articles 9 through 15 with documented, reproducible evaluation evidence. GPAI model providers must already publish benchmark evaluations and adversarial testing results under obligations effective since August 2025. Compliance is not a one-time certification but a continuous evaluation lifecycle including post-market monitoring. Build your evaluation infrastructure to produce audit-ready artifacts automatically. Start your compliance gap analysis immediately; the August 2026 deadline for high-risk systems requires 6 to 12 months of preparation. Penalties are severe: up to 35 million euros or 7% of global revenue.