Overview

Paulo Bianchi had been the HR director at a regional bank for eight years when his CEO asked him to lead the AI upskilling program. His first instinct was to buy a well-reviewed online course and send the link to everyone. His second instinct - the better one - was to pause. "I realized I had no idea what people already knew," he told me. "I was about to spend €200,000 on training and I couldn't tell you whether we needed AI basics for everyone or advanced prompt engineering for a subset." That pause saved the bank from a very expensive mistake, and it led to a skills assessment that shaped the next two years of their AI capability building.

Skills assessment and gap analysis is the practice of finding out what AI capabilities exist in your organization, which are missing, and which gaps are actually preventing you from executing your strategy. The emphasis on that last part matters. Every organization has gaps. The question is which ones to close first.

What You Are Trying to Learn

A useful skills assessment answers three distinct questions. They look similar but require different methods.

What do people know? This is baseline literacy - can employees explain what a large language model does, spot AI-generated content, use a prompt effectively? This distributes very unevenly and surprises most leaders in both directions. Some people with no formal AI background have been experimenting for months. Some people with technical titles have never opened a commercial AI tool.

What can people do? This is practical capability - can the marketing team actually use AI to speed up campaign content, or do they know the theory but struggle when they try? Knowing and doing are different. Assessment methods for practical capability require observation or task completion, not just surveys.

What does the organization need? This is the strategic layer - given where the organization is headed and what AI projects are planned, what capabilities are required? This question links the assessment to business outcomes rather than treating AI skills as an end in themselves.

Paulo's team asked all three questions. The answers were different in important ways: the bank had reasonable baseline literacy, weaker practical capability, and a specific strategic gap around data interpretation skills needed for their credit risk automation project.

The Four Skill Categories

Organizing your assessment around skill categories prevents you from treating "AI skills" as a single thing when it is actually several distinct things.

AI fundamentals

Understanding what AI is, how language models work at a conceptual level, what AI can and cannot do reliably, and how to evaluate AI output critically. Every employee in an AI-adopting organization needs some version of this.

Tool proficiency

The ability to use specific AI tools effectively in a job context. This is role-specific. A legal team's tool proficiency needs are different from an operations team's. Assess this per function, not organization-wide.

Domain-specific application

The ability to apply AI to the specific problems of a business domain - understanding what data is relevant, what outputs can be trusted, how to integrate AI assistance into existing workflows. This is where most value gets created and where most training programs underinvest.

Governance and responsible use

Understanding what data can be used with which tools, how to handle AI-generated content appropriately, when to apply human oversight, and how to report concerns. This is not optional for any employee who will use AI tools in their work.

Assessment Methods

Different questions call for different methods. A portfolio of three or four data sources is more reliable than any single approach.

Self-assessment surveys

Fast to deploy, easy to aggregate, and deeply unreliable as a standalone source. People systematically overestimate skills they have not tested and underestimate skills they use without naming. Use surveys to identify populations of interest and prioritize where to do deeper investigation - not to draw final conclusions.

Manager interviews

Fifteen-minute conversations with managers in each function surface two things surveys miss: what the work actually requires and what barriers to learning exist. A manager can tell you that her team needs to use AI for contract review but that the legal team has not approved the tool yet - information that should shape the training plan before the training is designed.

Practical tasks

Give a representative sample of employees a realistic task - a prompt to improve, an AI output to evaluate, a scenario to work through - and observe the results. This is the most reliable method for assessing practical capability and the most time-intensive. A sample of 20-30 people per function is usually enough to reveal patterns.

Usage analytics

If your organization has licensed AI tools with usage logging, this data is gold. Who is actually using the tools, how often, and for what? High-frequency users are potential internal experts. Nonusers in functions where AI should be useful are a gap worth understanding - they often have a barrier (tool access, time, uncertainty about policy) that is easier to remove than a skills gap to close.

Identifying Priority Gaps

Paulo's assessment identified fourteen distinct skill gaps across the bank's six business units. He could not address all fourteen simultaneously. The prioritization framework that worked for them asked three questions for each gap:

• Is this gap blocking a planned AI project? If yes, it is high priority regardless of how many people are affected.
  - How many people are affected? A gap that affects 800 people in customer service has more organizational impact than one affecting 12 people in treasury.
  - How quickly can it be closed? Some gaps close with a two-hour workshop. Others require months of practice and coaching. Sequencing matters - quick wins build momentum for harder investments.

The bank's top priority turned out to be neither the largest gap nor the most urgent one. It was the data interpretation gap affecting the 40-person team implementing the credit risk model - a gap that was both blocking a strategic project and addressable in eight weeks with targeted training.

Distinguishing Universal from Role-Specific Skills

Not everyone needs the same AI skills. Conflating universal and role-specific skills is one of the most common design errors in organizational AI training programs. It produces courses that are too basic for some learners and too advanced for others, with everyone feeling their time was wasted.

Universal skills - AI literacy, responsible use, and basic prompt construction - belong in a program that reaches every employee. Make it short (two to four hours total) and practical.

Role-specific skills belong in targeted programs built with specific functions. The marketing team's AI curriculum should use marketing examples, marketing tools, and marketing workflows. The same module will not serve them and the finance team equally well. Build the universal program first. Then build role-specific layers for the functions where AI will have the highest impact.

Involving Employees in the Assessment

The best source of information about what skills are needed is often the employees who will need them. They know what their day actually looks like, what slows them down, and where they feel uncertain when using AI tools.

Paulo ran a series of 45-minute group sessions - six to eight people at a time - where he asked two questions: "Where in your work do you wish AI could help you more?" and "What stops you from using AI tools you already have access to?" The answers shaped the training priorities more than any survey data.

This approach also builds buy-in. People who were asked what they needed are more likely to engage with the training that results. Assessment is not just data collection - it is the beginning of the relationship between the training program and the people it serves.

Key Takeaways

• Know what you are measuring before you measure it. Literacy (what people know), practical capability (what they can do), and strategic need (what the organization requires) are three different questions requiring different methods.
  - Use multiple data sources. Self-assessment surveys reveal populations of interest. Manager interviews surface barriers. Practical tasks measure real capability. Usage analytics show adoption patterns. No single method is sufficient.
  - Prioritize gaps by strategic impact, not size alone. A gap affecting 20 people on a critical AI project can be more important to close than a gap affecting 200 people in a function where AI is not yet deployed.
  - Separate universal from role-specific skills. Build a short, practical baseline program for everyone, then design role-specific curricula for the functions where AI impact will be highest.
  - Involve employees in the assessment. Group sessions asking what would help and what is stopping them generate better priorities than surveys - and build the buy-in that makes training stick.
  - The assessment is the beginning, not the end. Repeat it annually at minimum. Skills change, the tool landscape changes, and strategic priorities shift. A static gap analysis becomes outdated within months.