You Cannot Improve What You Cannot Measure

When a CRO at a large healthcare system was asked by the board in Q4 2025 to rate the organization's AI governance maturity on a scale of one to five, she paused. She knew they had policies, a governance committee, and some documentation standards. But she could not honestly distinguish between 'adequate' and 'good,' between performing governance activities and performing them effectively. She did not have a maturity model to assess against, benchmarks to compare against, or metrics to demonstrate progress over time. She was not alone -- a 2025 ISACA survey found that 82 percent of organizations had no formal method for assessing their AI governance maturity. This is a strategic problem, not just a measurement problem. Without maturity assessment, you cannot identify your most critical governance gaps, prioritize improvement investments, demonstrate progress to regulators and boards, or benchmark against peers. This chapter provides you with the maturity models, KPIs, and benchmarking techniques to answer the board's question with precision and confidence -- and more importantly, to drive continuous improvement in your AI governance program.

AI Governance Maturity Models: Choosing the Right Framework

Several maturity models have emerged for AI governance, each with different strengths. The CMMI-style five-level model (Initial, Managed, Defined, Quantitatively Managed, Optimizing) is the most familiar structure for audit and risk professionals. Applied to AI governance, Level 1 (Initial) means AI governance is ad hoc and reactive -- no formal policies, no designated oversight, individual teams managing AI independently. Level 2 (Managed) means basic governance elements exist -- an AI inventory, acceptable-use policy, and designated oversight body -- but implementation is inconsistent across the organization. Level 3 (Defined) means standardized processes are documented and applied consistently -- risk classification, documentation standards, testing requirements, and reporting are uniform across all AI deployments. Level 4 (Quantitatively Managed) means governance effectiveness is measured through KPIs, and data-driven decisions guide improvement. Level 5 (Optimizing) means the organization uses measurement data to continuously refine governance processes, proactively adapts to emerging risks, and is recognized as a leader by peers and regulators. Alternative frameworks include the NIST AI RMF Profiles approach, which assesses maturity across the Govern, Map, Measure, and Manage functions independently, and the ISO 42001 AI Management System standard published in late 2023, which provides certification-ready governance requirements. Choose the model that best aligns with your existing governance assessment approach to minimize adoption friction.

Defining Assessment Domains: What to Measure

A comprehensive AI governance maturity assessment evaluates capability across eight domains. First, governance structure: the maturity of oversight bodies, charters, roles, accountability, and decision-making authority. Second, policy and standards: the comprehensiveness, currency, and enforcement of AI policies, acceptable-use guidelines, and quality standards. Third, risk management: the integration of AI risk into ERM, the sophistication of AI risk assessment methods, and the effectiveness of risk mitigation. Fourth, inventory and classification: the completeness and accuracy of the AI system inventory and the rigor of risk classification. Fifth, development and deployment controls: the quality of controls over model development, validation, testing, and production deployment. Sixth, monitoring and assurance: the maturity of continuous monitoring, performance tracking, and internal audit coverage of AI systems. Seventh, talent and competency: the availability of AI governance expertise across the organization, from board members to first-line model owners. Eighth, stakeholder engagement: the effectiveness of communication, training, and coordination across functions and with external stakeholders. For each domain, define four to six specific capability indicators that can be assessed objectively. For example, under inventory and classification, indicators might include: (a) percentage of AI systems registered in the centralized inventory, (b) percentage of registered systems with current risk classifications, (c) frequency of inventory validation, and (d) process for detecting unregistered AI usage. Assess each indicator on your chosen maturity scale.

Designing Key Performance Indicators for AI Governance

KPIs transform subjective maturity assessments into objective, trackable measurements. Design your AI governance KPI framework around three categories that mirror the policy effectiveness levels from the previous chapter. Leading indicators measure governance activity and predict future effectiveness. Examples include: number of AI systems with current risk assessments divided by total AI systems (target: above 95 percent); percentage of high-risk AI deployments that completed the full governance review process before production (target: 100 percent); AI governance training completion rate across target populations (target: above 90 percent); and mean time from AI deployment proposal to governance approval (target: decreasing trend, indicating process maturity). Lagging indicators measure governance outcomes. Examples include: number of AI-related incidents or control failures per quarter (target: decreasing trend); regulatory examination findings related to AI governance (target: zero repeat findings); number of unauthorized or unregistered AI deployments discovered (target: decreasing trend); and AI system downtime or performance degradation events (target: within defined tolerance). Operational indicators measure governance process efficiency. Examples include: cost of AI governance per AI system managed (target: stable or decreasing as portfolio scales); governance staff hours per AI risk assessment (target: decreasing through standardization); and time from AI incident detection to resolution (target: decreasing). Select ten to fifteen KPIs total, assign an owner to each, define data sources and calculation methods, and establish quarterly reporting cadence.

Building the AI Governance Dashboard

Your KPIs are only useful if they are visible to the right audiences in the right format. Design a tiered dashboard architecture. The board-level dashboard is a single page showing aggregate maturity score, trend over the past four quarters, three to five critical KPIs in traffic-light format, and a brief narrative highlighting the most significant changes since the last report. This dashboard should be interpretable in under two minutes. The executive-level dashboard provides more granularity: maturity scores by domain, KPI trends with sparkline charts, top five governance gaps with remediation status, and regulatory compliance posture by jurisdiction. This is the working dashboard for the AI Governance Council, typically two to three pages. The operational-level dashboard provides full KPI detail: all indicators with current values, targets, trends, and drill-down capability to individual AI systems and risk assessments. This is the daily tool for the AI Governance Office. Design principles for all three tiers: use consistent color coding (red, amber, green with clearly defined thresholds), show trends rather than static snapshots, provide context for every metric (what is the target? what does amber mean?), and automate data collection wherever possible. Manual dashboard updates are unsustainable and introduce lag that reduces the dashboard's value for decision-making. Connect your dashboard to your GRC platform, AI inventory database, and incident management system for automated refresh.

Benchmarking AI Governance Against Peers and Industry Standards

Internal maturity assessment tells you where you are; benchmarking tells you where you should be. Effective benchmarking requires careful peer selection and appropriate comparison dimensions. Identify your peer group along three axes: industry (financial services peers face different AI regulatory requirements than manufacturing peers), organizational size and complexity (a 50,000-employee multinational needs different governance than a 500-person firm), and AI adoption maturity (benchmark against organizations at a similar stage of AI deployment, not against AI-native technology companies). Access benchmarking data through several channels. Industry associations like the IIA, ISACA, and RIMS publish periodic surveys on AI governance practices. The Big Four consulting firms and specialized AI governance advisory firms publish benchmarking reports that compare maturity levels, staffing ratios, and governance investments across industries. Regulatory examination feedback provides indirect benchmarking -- if examiners cite your AI governance as deficient relative to peer expectations, that is a powerful data point. Professional networking through NACD director forums, IIA chapters, and industry-specific governance roundtables provides qualitative benchmarking. When benchmarking, avoid two common errors. First, benchmarking against aspirational peers rather than actual peers inflates the perceived gap and can lead to over-investment in governance at the expense of AI innovation. Second, benchmarking only on structural elements (committee existence, policy publication) rather than effectiveness measures (incident rates, compliance levels) can mask poor governance behind impressive-looking structures.

Driving Continuous Improvement: From Assessment to Action

Maturity assessment without improvement action is an academic exercise. Convert assessment findings into a governance improvement roadmap using a disciplined prioritization approach. First, gap analysis: for each assessment domain, calculate the gap between your current maturity level and your target maturity level. Your target should be informed by regulatory expectations, peer benchmarks, organizational risk appetite, and strategic priorities -- not every domain needs to reach Level 5. Second, impact-effort prioritization: plot each improvement initiative on a two-by-two matrix. Quick wins (high impact, low effort) should be implemented immediately to build momentum and demonstrate value. Strategic investments (high impact, high effort) should be planned and resourced through the annual budgeting process. Incremental improvements (low impact, low effort) can be incorporated into ongoing operations. Low-priority items (low impact, high effort) should be deferred or reconsidered. Third, roadmap development: sequence prioritized initiatives into a twelve-to-eighteen-month improvement roadmap with quarterly milestones and defined deliverables. Assign an owner to each initiative. Fourth, progress tracking: reassess maturity quarterly against the same framework to measure progress. Report improvement trajectories to the AI Governance Council and board. The most effective continuous improvement programs establish a governance improvement sprint cycle -- typically quarterly -- where the governance team selects two to three improvement initiatives, implements them, measures the impact, and reports results before selecting the next round.

Avoiding Common Maturity Assessment Pitfalls

Maturity assessments go wrong in predictable ways. The most common pitfall is self-assessment inflation: teams rate their own maturity higher than an independent assessment would. Combat this by calibrating self-assessments with independent validation -- have internal audit or an external assessor independently rate two or three domains and compare scores. If self-assessed scores are consistently one or more levels higher than independent scores, apply a systematic adjustment. The second pitfall is measuring structure rather than effectiveness. An organization can have a beautifully chartered AI Governance Council that meets quarterly, publishes comprehensive policies, and maintains a complete AI inventory -- and still have poor AI governance because the Council lacks enforcement authority, policies are not followed, and the inventory is inaccurate. Your assessment must include effectiveness indicators, not just existence indicators. Ask not just 'do you have an AI risk assessment process?' but 'what percentage of AI systems have completed their required risk assessment on schedule?' The third pitfall is infrequent assessment. Annual assessment is insufficient for a rapidly evolving domain. Conduct comprehensive assessments annually but supplement them with quarterly pulse assessments on three to four priority domains. The fourth pitfall is assessment without context. A Level 2 maturity score is concerning for a financial services firm with 200 AI models in production but perfectly reasonable for a mid-sized manufacturer that deployed its first AI system six months ago. Always interpret maturity scores relative to the organization's AI adoption stage, regulatory environment, and risk profile.

Try This Now: Conduct a Rapid AI Governance Maturity Self-Assessment

Perform a focused self-assessment across the eight governance domains using a simplified three-level scale: Developing (ad hoc, inconsistent, or absent), Established (documented, implemented, and generally followed), and Advanced (measured, continuously improved, and benchmarked). For each domain, answer the key question: (1) Governance Structure -- is there a chartered body with decision-making authority over AI deployments? (2) Policy and Standards -- are there published, current AI policies that employees can reference and are trained on? (3) Risk Management -- are AI risks formally identified, assessed, and integrated into your ERM framework? (4) Inventory and Classification -- can you produce a complete, accurate list of all AI systems with risk classifications within 24 hours? (5) Development and Deployment Controls -- are there mandatory gates (documentation, testing, approval) that AI systems must pass before production deployment? (6) Monitoring and Assurance -- are deployed AI systems subject to ongoing performance monitoring and periodic independent review? (7) Talent and Competency -- do your governance, risk, and audit teams have sufficient AI knowledge to effectively oversee AI systems? (8) Stakeholder Engagement -- are cross-functional coordination mechanisms active and producing tangible governance improvements? Score each domain, calculate your overall profile, and identify the two domains with the largest gap between current state and where you believe your organization needs to be. These two domains should be the focus of your next governance improvement sprint. Share the results with your CAE or CRO along with a specific proposal for addressing the top two gaps within the next 90 days.

Key Takeaways

• Eighty-two percent of organizations lack formal AI governance maturity assessment, creating a strategic blind spot that prevents targeted improvement, regulatory demonstration, and peer benchmarking.
• Choose a maturity model (CMMI five-level, NIST AI RMF Profiles, or ISO 42001) that aligns with your existing governance assessment approach to minimize adoption friction.
• Assess maturity across eight domains -- governance structure, policy, risk management, inventory, controls, monitoring, talent, and stakeholder engagement -- using specific, objectively assessable capability indicators.
• Design ten to fifteen KPIs across three categories -- leading indicators (governance activity), lagging indicators (governance outcomes), and operational indicators (process efficiency) -- with defined owners, targets, and quarterly reporting cadence.
• Build a tiered dashboard architecture: one-page board view, two-to-three-page executive view, and detailed operational view, with automated data collection wherever possible.
• Benchmark against carefully selected peers along industry, size, and AI adoption maturity dimensions, avoiding the traps of aspirational benchmarking and structure-only comparison.
• Convert maturity assessments into action through gap analysis, impact-effort prioritization, quarterly improvement sprints, and regular reassessment that demonstrates measurable governance progress over time.