Overview

Dr. Amara Osei-Bonsu leads digital services at a large urban library system. In 2024, her team deployed an AI-powered catalog search tool - and then, three months later, received a formal complaint. A patron with low vision reported that the AI interface was not compatible with her screen reader. A second patron, a recent immigrant with limited English literacy, reported that the system's error messages were written at a reading level she couldn't parse. The tool had been through legal review, UX testing, and a pilot with staff. Nobody had tested it with the people most likely to need accommodation. "We thought we were making things better," Amara told me. "We hadn't even thought to ask who we were leaving out."

Inclusive AI design is not a finishing touch you add after building something. It is a set of decisions woven through every stage of design, development, and deployment - decisions about who the system is built for, how it behaves when those users show up, and what happens when it fails them.

Who Gets Excluded and How

Exclusion in AI systems happens in at least three ways, often simultaneously.

Interface exclusion is the most visible: the system isn't accessible to users with disabilities. Screen readers can't parse dynamic AI outputs. Voice interfaces fail users with speech impairments. Text-heavy interfaces exclude users with low literacy or who rely on translation. These failures often violate legal requirements - in many jurisdictions, digital public services must meet Web Content Accessibility Guidelines (WCAG), and AI-powered services are not exempt.

Training data exclusion is less visible: the model was trained on data that underrepresents certain populations. A healthcare triage AI trained predominantly on records from one demographic may perform worse for patients from other groups. A hiring AI trained on historical resumes may systematically undervalue applicants from non-traditional educational backgrounds. The model isn't broken. It reflects the data it learned from.

Interaction design exclusion is subtler still: the system works technically but assumes a level of AI familiarity, vocabulary, or interaction style that not all users share. Prompt-based interfaces reward users who already know how to frame requests to AI systems. This advantage compounds: sophisticated users get better outputs; less experienced users get worse ones; the gap widens over time.

The Inclusive Design Principles That Apply to AI

Inclusive design started in the physical world - curb cuts, captioning, ramps. The same underlying logic applies to AI systems. The three most relevant principles are:

Equitable use: The system should be usable by people with diverse abilities, and it should not stigmatize or segregate any group. In practice, this means building accessibility into the core interface rather than providing a "basic version" as an afterthought. Amara's library now tests every AI-powered feature with screen reader software before launch - not as a final check but as a standard step in the build process.

Flexibility in use: The system accommodates a wide range of individual preferences and abilities. For AI interfaces, this means supporting multiple input modalities (typed, spoken, uploaded), adjustable reading levels in outputs, and graceful degradation when a user doesn't engage in the expected way.

Low physical effort with perceptive information: The system communicates necessary information clearly, regardless of ambient conditions or the user's sensory abilities. Error messages written at a graduate reading level, AI outputs returned without structure, and color-coded status indicators without text labels all fail this principle.

Practical Checklist for AI Program Design

Inclusive design is not a single step. It is embedded in these five decision points:

• Who is in the room during design? If your design and testing team doesn't include people with the characteristics of your most vulnerable or least-served users, you will miss problems. Disability, language background, age, and digital literacy should all be represented in user research, not just as data but as voices in the room.
  - What does the training or input data represent? For any AI system that makes decisions about or for people, audit the demographic representation of the training data. Document known gaps. Establish a plan for monitoring performance disparities post-deployment.
  - Does the interface meet WCAG 2.1 AA standards? This is the baseline legal requirement in most jurisdictions for digital services. AI-generated content is particularly tricky because it is dynamic - static accessibility testing tools may not catch real-time output failures.
  - What is the failure mode? When the AI system can't help a user, what happens? An inclusive failure mode provides clear, plain-language guidance and a path to a human. An exclusive failure mode leaves the user stranded.
  - How do you monitor equity in outcomes? After deployment, track whether outcomes differ by user group. If the AI catalog search returns fewer relevant results for users searching in non-English terms, that is a measurable equity gap you can act on.

Accessibility Standards Beyond WCAG

WCAG is the web accessibility standard. For AI programs, two additional frameworks are increasingly relevant.

The EU AI Act, which took effect in 2024, requires that high-risk AI systems be designed to allow natural persons to oversee and intervene - with specific attention to the needs of people with disabilities and low digital literacy. Organizations deploying AI in education, employment, essential services, or credit scoring are in scope.

Section 508 of the US Rehabilitation Act requires that federal agencies and recipients of federal funding ensure their electronic and information technology is accessible. AI-powered tools used in federal agencies or in federally funded programs fall under this requirement.

Neither framework is complete or final - regulators are actively developing AI-specific accessibility guidance. Staying current means treating compliance as an ongoing discipline, not a one-time audit.

Key Takeaways

• AI exclusion happens three ways: interface inaccessibility, training data underrepresentation, and interaction design that rewards already-sophisticated users.
  - Inclusive design is embedded throughout the process, not added at the end - the five critical decision points are design team composition, training data audits, WCAG compliance, failure mode design, and post-deployment equity monitoring.
  - WCAG 2.1 AA is the baseline interface standard - dynamic AI outputs require active rather than static accessibility testing.
  - The EU AI Act and Section 508 introduce specific accessibility obligations for AI systems in high-risk or federally funded contexts - these are evolving and require ongoing compliance attention.
  - Test with representative users before launch, not just user types you can easily recruit - the users most likely to be excluded are often the least likely to be in a default test panel.
  - Design failure modes inclusively: when the AI can't help, the user should find a clear, plain-language path to a human - not an impenetrable error message.