Overview

Lecture URL: https://skill.re/learn/recruiting/ai-summarization-capabilities-and-typical-errors.php

TRANSCRIPT: AI Summarization: Capabilities and Typical Errors

Course: AI for Recruiters - Professional Credential

Module: Level 2: Hands-On Foundations

Section: Chapter 8 -- Note Synthesis and Summary Support

Theme: note-synthesis-and-summary-support

Lecture: 8.1

Duration: 60 min

Format: Workshop + Hands-On

Audience: Recruiters beginning to use AI tools

Prerequisites: L1 Certification

What you will learn: You'll understand what AI does well when summarizing interview notes and

candidate information, and what errors to watch for. By the end, you'll know the strengths and

limitations of AI summaries and how to use them as decision support, not decision substitutes.

INTRODUCTION

Interview notes are messy. You're trying to listen, take notes, and assess simultaneously. Your

notes capture some key moments, miss others, and sometimes misinterpret what was said. Then you

have 10 notes from different interviewers, and you need to synthesize them into a hiring decision.

This is where AI summaries are incredibly useful. AI can read through messy notes and extract

patterns, signals, and key information. But AI also makes consistent, systematic errors that you

need to watch for. Today, we're learning what AI is actually good at, what it's bad at, and how

to use it accordingly.

WHAT AI DOES WELL IN SUMMARIZATION

AI is excellent at:

1. PATTERN RECOGNITION ACROSS MULTIPLE SOURCES

You have 5 interview notes from different people. One note mentions "works well with ambiguity."

Another says "comfortable with change." A third says "doesn't need a lot of structure." AI can

recognize these are the same signal across different sources. Humans miss this pattern-matching

across documents.

2. EXTRACTING AND ORGANIZING INFORMATION

AI can turn messy notes into clean, structured information. Instead of reading through five

paragraphs of notes, you get a bulleted list of key signals by category: technical skills, soft

skills, concerns, red flags, cultural fit signals.

3. DISTINGUISHING INFORMATION LEVELS

AI can identify what's explicitly stated vs. what's inferred. "She said she knows Python" is

explicit. "She probably has database experience" is inferred. Good summaries mark the difference.

4. IDENTIFYING EVIDENCE FOR CLAIMS

AI can point to specific moments in notes that support conclusions. Instead of "This person

communicates well," it can say "Evidence: She explained her architecture decision clearly in the

technical round, and she asked three clarifying questions in the domain expert interview."

5. FLAGGING MISSING INFORMATION

AI can identify gaps. "We have strong signals on technical skills, but limited information on

leadership experience." This helps you know what you still need to learn.

WHAT AI GETS WRONG IN SUMMARIZATION

AI makes consistent, predictable errors:

ERROR 1: HALLUCINATIONS (MAKING UP DETAILS)

AI might claim the candidate said something or did something that they didn't.

Example:

Notes say: "She has backend experience in Java."

AI summary might say: "10 years of backend experience in Java" (inventing the "10 years").

Why it happens: AI is trained to be helpful and complete. If you ask "How many years?" and the

notes don't say, AI might infer or generalize based on patterns.

How to catch it: Review specific facts in AI summaries against original notes. If the summary

says "5 years of experience" and the notes say "several years," that's hallucination.

ERROR 2: OVERWEIGHTING RECENCY

AI tends to overweight the most recent information in notes, even if earlier information is more

important.

Example:

Notes from 5 interviewers: 4 mention strong technical skills, 1 mentions a communication concern

from the final interview. The AI summary might lead with "Communication concerns" because it's last.

How to catch it: Check whether the summary reflects the balance of evidence or over-emphasizes

recent information.

ERROR 3: MISSING CONTEXT

AI might extract a quote or concern without understanding the context that makes it meaningful or

meaningless.

Example:

Notes say: "Candidate mentioned they've switched jobs 3 times in 5 years. But they explained each

move was because they wanted to learn new technologies."

AI summary might flag "Job hopping" without capturing the explanation that gives it context.

How to catch it: Review summaries for context. If a red flag is flagged, is the explanation

included?

ERROR 4: MISINTERPRETING SOFT SKILLS

AI struggles with soft skills and interpersonal signals because they're more subtle than technical

skills.

Example:

Candidate was quiet in the meeting, asked few questions. Notes say "reserved but thoughtful."

AI might interpret this as "not comfortable engaging with the team."

How to catch it: Compare AI's soft skills assessment with your notes. Does it match your

impression?

ERROR 5: TONE MISMATCHES

AI might miss tone or context that changes meaning.

Example:

Notes say: "I'm not sure about their experience with Python" (meaning uncertain, should clarify).

AI might interpret: "They don't have experience with Python."

How to catch it: Read the original notes. Did the interviewer use tentative language (I think,

might, possibly)? Did the AI summary treat it as definite?

USING AI SUMMARIES EFFECTIVELY

AI summaries are decision-support tools, not decision substitutes. Use them to:

1. ORGANIZE INFORMATION QUICKLY

Instead of reading 5 pages of notes, get a 1-page summary that's structured by key dimensions.

2. IDENTIFY PATTERNS ACROSS INTERVIEWERS

"Three people mentioned his communication skills, one person mentioned database depth."

3. SURFACE WHAT'S MISSING

"We have strong information on technical skills but limited data on leadership or team influence."

4. SUPPORT YOUR DECISION

"Based on the summary, here are the three key questions we should resolve before making a decision."

Don't use AI summaries to:

1. REPLACE YOUR OWN READING OF NOTES

You should still read the original notes, especially for candidates you're seriously considering.

2. SUBSTITUTE FOR YOUR JUDGMENT

"The summary says they're a strong fit" doesn't mean they are. You interpret the summary in

context of your needs.

3. BYPASS STRUCTURED INTERVIEWING

Good interviewing captures the information you need. AI can't fix bad interviewing with good

summarization.

ANTI-PATTERNS

ANTI-PATTERN 1: TRUSTING SUMMARIES WITHOUT VERIFICATION

Description: Using AI summary as the source of truth without checking against original notes.

Example: Making a decision based on the summary without reading the notes.

Why it fails: You might be acting on hallucinated or misinterpreted information.

How to avoid: Always spot-check AI summaries. Read the original notes on candidates you're

considering closely.

ANTI-PATTERN 2: LETTING SUMMARIES DRIVE THE NARRATIVE

Description: The summary becomes the canonical version, and the notes become secondary.

Example: In a team meeting, you present the summary, and people make decisions based on it

without seeing the notes.

Why it fails: Errors or misinterpretations in the summary get amplified. Team members don't have

access to the full context.

How to avoid: Keep notes as primary. Use summaries to support, not replace.

ANTI-PATTERN 3: ASSUMING COMPLETENESS

Description: Assuming the summary captures everything important from the notes.

Example: Notes have nuance and context about concerns. Summary flags the concern but loses the

nuance.

Why it fails: You're missing important context that would change how you interpret the signal.

How to avoid: Always read original notes, especially for serious candidates. Summaries are tools,

not complete representations.

PRACTICE PROMPTS

Exercise 1: Compare Summary to Notes

Find an AI summary (or generate one using interview notes). Compare it to the original notes. What

did the summary capture well? What did it miss or misinterpret?

Exercise 2: Spot Check for Hallucinations

Review an AI summary. For every specific fact (years of experience, specific skills, specific

achievements), verify it against the notes. Find any hallucinations.

Exercise 3: Assess Balance

Review an AI summary. Does it reflect the balance of evidence from the notes? Or does it over-

weight certain sources or recent information?

Exercise 4: Context Evaluation

Review how AI handled soft skills or concerns. Did it include enough context? Did it lose important

nuance?

Exercise 5: Decision Support, Not Decision

Use an AI summary to identify questions you need answered. What's missing? What would you want to

clarify? Now go back to notes and see if the answer is there.

KEY TAKEAWAYS

1. AI excels at: pattern recognition across sources, organizing information, extracting and

distinguishing evidence, identifying missing information.

1. AI struggles with: avoiding hallucinations, context and nuance, soft skills assessment,

understanding tone, weighing information appropriately.

1. AI summaries are excellent decision-support tools, not decision substitutes. Use them to organize

information and surface patterns, not to replace your judgment.

1. Always spot-check AI summaries against original notes, especially for candidates you're seriously

considering.

1. The more structured your original notes, the better the AI summary. Good notes -> good summaries.

Bad notes -> bad summaries (or hallucinations to fill gaps).

1. Use summaries to identify what information you're missing. "Limited data on leadership" tells

you what follow-up you need.

GLOSSARY

Hallucination: When AI generates information that wasn't in the source material and might not be

true. Always verify specific facts in AI summaries against original notes.

Pattern Recognition: AI's ability to identify the same signal expressed different ways across

multiple notes. This is one of AI's great strengths.

Context Loss: When AI extracts information but loses the surrounding context that gives it meaning.

"Job hopping" loses meaning without "explained by growth-seeking."

Soft Skills: Interpersonal and personal qualities like communication, leadership, adaptability. AI

struggles to assess these accurately from notes.

Tone Misalignment: When AI misinterprets the tone or confidence level of statements in notes. "I'm

not sure" might be interpreted as "definitely no."

SYNTHESIS AND APPLICATION

AI summarization is most useful when you use it for what it does well (organizing information,

finding patterns) and least useful when you use it to replace human judgment. Your job is to know

the difference.

This week, use AI summaries on your interview notes. Check them against the notes for accuracy.

Build a sense of where they're reliable and where you need to verify. Over time, you'll develop

intuition about which summaries to trust and which ones need deeper review.

REFLECTION EXERCISE

1. When you read interview notes from multiple people, what do you find hard? Organizing? Finding

patterns? Synthesizing conflicting information? What would AI summaries help with most?

1. Have you ever caught an AI making up details or misinterpreting something? What was it? How did

you catch it?

1. What information from interviews do you most often lose track of? Technical skills? Soft skills?

Cultural fit signals? How might summaries help?

1. For your next hiring decision, would an AI summary of interview notes actually help you decide?

Or would it just be extra work to verify?

CLOSING REMARKS

AI summaries are powerful tools when used right. They're terrible substitutes for human judgment.

Know the difference, and you'll get real value from them. In the next session, we're going to learn

how to extract specific information and signals from summaries for decision-making.

AI for Recruiters Certification Program

Level 2: Hands-On Foundations | Note Synthesis and Summary Support | Lecture 8.1

A SkillsClinic initiative.

Duration: ~60 minutes | Word Count: ~2,450