Overview

Lecture URL: https://skill.re/learn/recruiting/hands-on-practice-review-ai-summaries-identify-errors-revise.php

TRANSCRIPT: Hands-On Practice: Review AI Summaries, Identify Errors, Revise

Course: AI for Recruiters - Professional Credential

Module: Level 2: Hands-On Foundations

Section: Chapter 8 -- Note Synthesis and Summary Support

Theme: note-synthesis-and-summary-support

Lecture: 8.5

Duration: 60 min

Format: Workshop + Hands-On

Audience: Recruiters beginning to use AI tools

Prerequisites: L1 Certification

What you will learn: You'll practice reviewing AI-generated candidate summaries, spotting errors

and hallucinations, and revising outputs for accuracy and usefulness through realistic scenarios.

By the end, you'll feel confident reviewing and improving AI summaries before using them for

decisions.

INTRODUCTION

Reading an AI summary is one thing. Critically reviewing it, finding errors, and improving it is

another. That's what separates AI that supports decision-making from AI that misleads you.

In this session, we're doing hands-on practice. You'll review real (or realistic) AI summaries,

identify problems, understand where things went wrong, and practice fixing them. By the end, you'll

have a checklist for reviewing every summary before it influences a hiring decision.

THE REVIEW CHECKLIST

Before using an AI summary for any hiring decision, check:

1. ACCURACY OF FACTS

Are all explicit facts correct? Years of experience, specific projects, specific companies,

specific technologies.

2. DISTINCTION BETWEEN EXPLICIT AND INFERRED

Does the summary mark what's directly stated vs. what's inferred? "Said they know Python"

(explicit) vs. "Likely comfortable with frontend" (inferred)?

3. CONTEXT PRESERVATION

Are concerns explained with context, or just flagged? "Job hopping" without context is

meaningless. "Job hopping but each move was geographic/personal circumstances" gives context.

4. EVIDENCE CITATION

Are major claims supported by specific examples? "Strong problem-solver" without evidence is

vague.

5. BALANCE

Does the summary reflect the balance of evidence from the interview, or does it overweight

certain parts?

6. TONE ACCURACY

Does the summary preserve the tone and confidence of the original notes? "I'm not sure about

their experience with X" should be marked as uncertain, not definite.

7. MISSING INFORMATION

What's missing that you wished was there? Soft skills? Technical depth? Team dynamics?

SCENARIO 1: REVIEWING FOR HALLUCINATIONS

THE SUMMARY:

"Alex has 8 years of backend engineering experience, with deep expertise in distributed systems.

He architected our company's real-time payment system and led a team of 5 engineers. Strong

communicator, explains technical decisions clearly."

THE ORIGINAL NOTES (from interview):

"Alex has worked in backend engineering for several years. Mostly worked on monolithic services

at previous roles. At current company, worked on payment processing system. Explains things fairly

clearly in technical discussions."

THE PROBLEM:

• "8 years" (hallucinated -- notes say "several years")
  - "Deep expertise in distributed systems" (hallucinated -- notes don't mention this)
  - "Architected the real-time payment system" (hallucinated -- notes say "worked on," not architected)
  - "Led a team of 5" (hallucinated -- no mention of team leadership)
  - "Explains technical decisions clearly" is accurate

HOW TO CATCH IT:

Check every specific claim against the notes. If the summary says "8 years" and the notes say

"several years," that's hallucination.

HOW TO FIX IT:

Revise to match notes: "Alex has worked in backend engineering for several years, mostly on

monolithic services. At current company, contributed to payment processing system development.

Communicates fairly clearly in technical discussions."

SCENARIO 2: REVIEWING FOR LOST CONTEXT

THE SUMMARY:

"Red flag: candidate changed jobs 4 times in 6 years."

THE ORIGINAL NOTES:

"Changed jobs 4 times in 6 years. First move: company acquired. Second move: wanted to learn new

tech stack. Third move: spouse relocated. Current role: 3 years, no plans to leave."

THE PROBLEM:

Context is completely lost. "Changing jobs 4 times" becomes concerning without understanding that

three of the moves had external reasons.

HOW TO CATCH IT:

Review red flags. Do they include context or explanation? If just flagged, ask: what explains this?

HOW TO FIX IT:

Revise to include context: "Changed jobs 4 times in 6 years. Context: acquisition, intentional

growth-seeking, spouse relocation. Currently at stable role for 3 years with no plans to leave."

SCENARIO 3: REVIEWING FOR LOST SOFT SKILLS

THE SUMMARY:

"Communication: Adequate. Team collaboration: Adequate. Leadership: Not assessed."

THE ORIGINAL NOTES:

"In technical round, very clearly explained his debugging approach and asked clarifying questions

from the system designer. In team round, facilitated discussion when two engineers disagreed,

acknowledged both sides, helped reach consensus. Quiet overall, but thoughtful."

THE PROBLEM:

The summary reduces rich observations ("clearly explained," "facilitated discussion") to vague

ratings ("adequate"). Soft skills are underrepresented.

HOW TO CATCH IT:

Compare the summary's soft skills section to the original notes. Are detailed observations

reduced to vague ratings?

HOW TO FIX IT:

Revise with evidence: "Communication: Strong. Clearly explained debugging approach. Asked

clarifying questions. Team collaboration: Strong. Facilitated discussion between disagreeing

engineers. Leadership: Not directly assessed."

THE REVIEW AND REVISION PROCESS

When you review a summary:

STEP 1: SCAN FOR OBVIOUS HALLUCINATIONS

Are there specific facts that seem inflated or that you don't remember from the notes?

STEP 2: CHECK CLAIMS AGAINST NOTES

For every major claim (years of experience, specific achievements, skill assessments), verify

against the notes.

STEP 3: ASSESS CONTEXT

Do red flags include context, or just the flag? Do strengths include evidence, or just ratings?

STEP 4: IDENTIFY MISSING INFORMATION

What would you want to know that's not in the summary? Soft skills? Technical depth? Team

dynamics?

STEP 5: FIX OR REVISE

For problems you find, either: (a) edit the summary directly, or (b) ask AI to revise with

specific feedback.

STEP 6: FINAL REVIEW

Reread your revised summary. Does it feel accurate? Does it represent the candidate fairly?

ANTI-PATTERNS

ANTI-PATTERN 1: ACCEPTING SUMMARIES WITHOUT VERIFICATION

Description: Receiving an AI summary and using it without checking facts.

Example: Summary says "12 years of experience." You never verify this against the resume.

Why it fails: You base decisions on hallucinated information.

How to avoid: Always spot-check facts in summaries, especially specific numbers and achievements.

ANTI-PATTERN 2: NOT REVISING INADEQUATE SUMMARIES

Description: Summary is 80% there, but missing key information. You use it anyway.

Example: Summary doesn't capture soft skills. You use it to evaluate technical fit. You discover

later that communication is a problem you missed.

Why it fails: You're making decisions without complete information.

How to avoid: If a summary is missing something important, revise it or ask AI to expand that

section.

ANTI-PATTERN 3: ASSUMING THE SUMMARY TONE MATCHES REALITY

Description: Summary says "weak communicator," but the notes actually show adequate communication

with a different style.

Example: "Quiet in the meeting" gets translated to "poor communication."

Why it fails: You're mischaracterizing candidates based on personality rather than actual

communication ability.

How to avoid: Compare the summary's tone assessments to your recollection of the interview.

PRACTICE PROMPTS

Exercise 1: Spot Hallucinations

Take an AI summary (real or example). Identify three claims. Check each against the original notes.

Find any hallucinations or inflations.

Exercise 2: Add Missing Context

Take a summary that flags a concern (job hopping, communication style, etc.). Compare to the

original notes. What context is missing? Revise the flag to include context.

Exercise 3: Verify Soft Skills

Take the soft skills section of a summary. Compare to the original notes. Are detailed observations

preserved, or reduced to vague ratings? Revise to include evidence.

Exercise 4: Full Review and Revision

Take a complete AI summary. Go through the review checklist (accuracy, distinction, context,

evidence, balance, tone, missing info). Document all issues. Revise the summary.

Exercise 5: Create Your Review Checklist

Based on the issues you've identified in practice, create a personal checklist for reviewing

summaries. What are the most common errors you catch? Build your checklist around those.

KEY TAKEAWAYS

1. Always spot-check AI summaries for hallucinations. Verify specific facts, years of experience,

and claimed achievements against notes.

1. Red flags should include context. "Job hopping" without explanation is incomplete information.
2. Soft skills assessment should include evidence. "Strong communicator: clearly explained,

asked clarifying questions" is better than "Strong communicator."

1. Use the review checklist: accuracy, distinction explicit/inferred, context, evidence, balance,

tone, missing info.

1. Don't accept summaries at 80%. If they're missing important information, revise them before

using them for decisions.

1. When you catch issues, document them. Over time, you'll identify patterns in how AI summarizes

for your specific interview context.

GLOSSARY

Hallucination: When AI generates information that wasn't in the source material. Always verify

specific facts in summaries against original notes.

Context Loss: When summary flags a concern without explaining the circumstances that give it

meaning.

Soft Skills: Interpersonal qualities like communication, collaboration, leadership. Often

underrepresented in AI summaries.

Spot-Checking: Verifying key facts in a summary against the original source. Essential for catching

hallucinations.

Revision: Editing or asking AI to improve a summary based on identified issues.

SYNTHESIS AND APPLICATION

The difference between AI-supported recruiting and AI-mislead recruiting is one step: verification.

When you always review summaries against original notes, you catch hallucinations and misinterpretations

before they influence your decisions. That one step protects your recruiting quality.

This week, use AI to summarize interview notes. Review each summary using the checklist. Document

issues you find. Build confidence in your ability to catch and fix problems.

REFLECTION EXERCISE

1. In your experience with AI, what kinds of errors do you most often encounter? Hallucinations?

Lost context? Reduced soft skills assessments?

1. How much time would it take you to review each summary? Is that time investment worth the

quality gain?

1. What's one error pattern you could prevent by improving your summary prompt?
2. If you reviewed all your recent hiring decisions with this verification checklist, what might

you discover about your hiring patterns?

CLOSING REMARKS

Verification is where AI-assisted recruiting becomes trustworthy recruiting. Once you have a

strong review process, you can use AI confidently. In the next chapter, we move from supporting

good decisions to preventing bad ones -- reviewing AI outputs critically for bias and errors.

AI for Recruiters Certification Program

Level 2: Hands-On Foundations | Note Synthesis and Summary Support | Lecture 8.5

A SkillsClinic initiative.

Duration: ~60 minutes | Word Count: ~2,420