Overview

Lecture URL: https://skill.re/learn/recruiting/scenario-practice-review-and-critique.php

TRANSCRIPT: Scenario Practice: Review and Critique

Course: AI for Recruiters - Professional Credential

Module: Level 2: Hands-On Foundations

Section: Chapter 9 -- Reviewing AI Output Critically

Theme: reviewing-ai-output-critically

Lecture: 9.5

Duration: 60 min

Format: Workshop + Hands-On

Audience: Recruiters beginning to use AI tools

Prerequisites: L1 Certification

What you will learn: You'll practice critically reviewing AI outputs in realistic scenarios.

By the end, you'll develop expertise in identifying errors and bias, deciding when to accept or

reject, and writing feedback that improves outputs.

INTRODUCTION

Theory is one thing. Practicing on real scenarios is another. In this session, we're doing

hands-on critique of AI-generated candidate materials. You'll practice the full cycle: review,

identify issues, decide accept/reject/escalate, and provide feedback.

SCENARIO 1: CRITICAL REVIEW OF A SUMMARY

THE OUTPUT:

"Sarah is an excellent product manager with 12 years of experience. She's passionate about

building world-class products and has strong leadership skills. She's led teams of up to 8 people

and has excellent communication skills. Strong cultural fit for our dynamic, fast-moving team."

YOUR TASK:

Review this using the frameworks from Chapters 9.1-9.4. Identify: (1) hallucinations or bias, (2)

missing evidence, (3) vague language, (4) unclear claims.

ISSUES TO IDENTIFY:

• "Excellent" and "world-class" are vague
  - "Passionate" is unsubstantiated
  - "12 years" needs verification
  - "Led teams of up to 8 people" is vague and unsubstantiated
  - "Excellent communication" needs evidence
  - "Strong cultural fit" is biased (could mean "like us")
  - No specific examples or evidence cited
  - Reads like a sales pitch, not an objective assessment

DECISION:

This output has multiple red flags: missing evidence, vague claims, potential bias, no citations.

Should be REJECTED or completely rewritten.

SCENARIO 2: SPOT HALLUCINATIONS

THE OUTPUT:

"Alex graduated from Stanford with a degree in Computer Science. He has 8 years of backend

engineering experience, with deep expertise in distributed systems and microservices. He led a

team of 5 engineers for 2 years. His GitHub shows 150 repositories with significant open source

contributions."

THE ACTUAL RESUME:

"BS in Computer Science, State University. 6 years backend engineering experience. Worked on

distributed systems projects at previous role. Contributions to open source frameworks."

ISSUES TO IDENTIFY:

• Stanford -> State University (hallucinated prestige)
  - 8 years -> 6 years (hallucinated additional experience)
  - "Deep expertise in distributed systems" -> "worked on" (inflated)
  - "Led a team of 5 for 2 years" -> no mention (completely hallucinated)
  - "150 repositories" -> "contributions to frameworks" (numbers hallucinated)
  - "Significant open source contributions" -> vague in original

DECISION:

Multiple hallucinations. Output is unreliable. REJECT.

SCENARIO 3: IDENTIFYING BIAS

THE OUTPUT:

"Jessica is a strong candidate but seems like she might prioritize family over work. She mentioned

wanting flexibility, which suggests she's not serious about a senior role with demanding hours."

ISSUES TO IDENTIFY:

• "Might prioritize family" is assumption about motherhood (gender bias)
  - "Wanting flexibility" is coded as lack of commitment (gender bias, assumes working parents are

less committed)

• "Not serious about a senior role" is an inference with no evidence
  - Biased conclusions that could disqualify an otherwise qualified candidate

DECISION:

Clear, unfixable bias. Output should be REJECTED. Candidate shouldn't be evaluated based on

assumptions about family/flexibility.

SCENARIO 4: BORDERLINE -- ACCEPT, ESCALATE, OR REJECT?

THE OUTPUT:

"Tech fit: Strong in Python and backend systems. Moderate experience with our specific cloud

platform (AWS). Communication: Good in technical discussions, quieter in group settings. Cultural

fit: Unclear. Concerns: Limited experience with the specific tech stack we use. May need 2-3 months

to ramp up."

ISSUES TO IDENTIFY:

• Technical fit is clear and well-supported
  - Communication is described with evidence ("good in technical," "quieter in group")
  - Cultural fit is explicitly marked "unclear" (honest)
  - Concerns are specific and reasonable

DECISION:

This output has escalation flags, not reject flags. The assessment is honest and highlights

genuine questions that need human discussion. ESCALATE to hiring team for discussion about

ramp-up time and team dynamics.

YOUR CRITIQUE FRAMEWORK

For each scenario:

1. IDENTIFY ISSUES:

• Hallucinations (facts inflated or made up)
  - Bias (discriminatory language or assumptions)
  - Missing evidence (claims without support)
  - Vague language (unclear assessments)
  - Omissions (missing important information)

1. ASSESS SEVERITY:

• Critical (affects decision significantly)
  - Major (notable but might be fixable)
  - Minor (doesn't affect decision)

1. DECIDE:

• REJECT: Multiple critical issues, unfixable bias, unreliable
  - ESCALATE: Contradictions, unclear signals, fairness questions, decisions needing human judgment
  - USE: Clean output with clear information and no red flags

ANTI-PATTERNS

ANTI-PATTERN 1: ACCEPTING "GOOD ENOUGH"

Description: Accepting output that's 70-80% good when it should be 90%+.

Why it fails: Over time, small issues compound. Quality degrades.

How to avoid: Hold output to consistent standards. Use reject/escalate criteria.

ANTI-PATTERN 2: BEING TOO PERMISSIVE ON BIAS

Description: Seeing bias and deciding it's "close enough" or "not that bad."

Why it fails: Bias is never acceptable. You're compromising on fairness.

How to avoid: If it has bias flags, reject or escalate. Don't accept biased output.

ANTI-PATTERN 3: NOT PROVIDING FEEDBACK

Description: Rejecting output but not explaining why or what to fix.

Why it fails: AI doesn't improve. You just keep getting bad output.

How to avoid: When you reject, explain specifically what's wrong.

PRACTICE PROMPTS

Exercise 1: Critique These Outputs

Work through Scenarios 1-4 above. Identify issues. Decide accept/reject/escalate.

Exercise 2: Your Own Scenario

Take a real AI-generated candidate assessment from your work. Critique it using the same framework.

Exercise 3: Provide Improvement Feedback

For an output you rejected, write specific feedback about what to improve.

Exercise 4: Team Calibration

Share your critiques with colleagues. Do you agree on what should be rejected vs. escalated?

Exercise 5: Build Your Critique Checklist

Based on the issues you've identified, create a personal critique checklist.

KEY TAKEAWAYS

1. Use your review frameworks to systematically critique AI output.
2. Identify: hallucinations, bias, missing evidence, vague language, omissions.
3. Assess severity: is it critical, major, or minor?
4. Decide: REJECT (multiple critical issues, unfixable bias), ESCALATE (contradictions, fairness

questions), or USE (clean output).

1. Always explain your critique. Don't just reject -- explain what needs to change.
2. When you see patterns in your rejections (e.g., always rejecting vague cultural fit language),

improve your prompts to prevent those issues.

GLOSSARY

Critique: Systematic evaluation of AI output for accuracy, bias, completeness, and reliability.

Hallucination: AI-generated information not in source material, often inflated or false.

Bias: Discriminatory language or assumptions based on protected characteristics.

Missing Evidence: Claims made without supporting examples or citations.

SYNTHESIS AND APPLICATION

Critical review is a skill that develops through practice. The more you critique, the faster you

spot issues. The faster you spot issues, the less time wasted on low-quality outputs.

This week, critique 10 AI-generated candidate assessments. Document your feedback. Notice what

patterns emerge in what you reject. Use those patterns to improve your prompts.

REFLECTION EXERCISE

1. What types of issues do you most often find in AI output? Hallucinations? Bias? Missing evidence?
2. When you critique output, are you usually rejecting, escalating, or accepting? What does that

pattern tell you?

1. How much time do you spend reviewing vs. using AI output? Is that ratio right?
2. If you shared your critiques with the person/team generating the outputs, what would they learn?

CLOSING REMARKS

You're now equipped to critically review AI output and make principled decisions about when to use

it. In our final chapters, we're scaling this to teams and building systems that support responsible

AI use at scale.

AI for Recruiters Certification Program

Level 2: Hands-On Foundations | Reviewing AI Output Critically | Lecture 9.5

A SkillsClinic initiative.

Duration: ~60 minutes | Word Count: ~2,480