Overview

A recruiter asks a text-generation tool to rewrite a job description. Three hours later, the marketing team posts the rewritten version. A hiring manager asks the same tool the next day for help with interview questions. The tool generates questions that are eerily similar to something it suggested three months ago, but the manager doesn't know that. Every time someone interacts with these systems, they're working with black-box machinery they don't understand. This lesson removes some of the mystery.

Purpose

You don't need to be able to build an AI system, but you do need to understand how it works well enough to use it intelligently. This lesson walks through the mechanics of generative AI, what happens when you type a prompt, why it sometimes hallucinates, why temperature and context window matter, and what tokens are. You'll come away with practical knowledge that helps you get better output and spot when something is wrong.

The stakes are real. If you don't understand how these systems work, you'll miss obvious problems. A prompt that seems perfectly clear to you might produce terrible output because you didn't understand how the system processes language. You might rely too heavily on a feature that is fundamentally limited. You might trust output you shouldn't trust.

Why This Matters for HR Professionals

HR professionals increasingly use generative AI tools. Your recruiters are using them to write job descriptions and interview questions. Your compensation team might be using them to draft pay band definitions. Your learning teams are using them to create training content. L&D is using them to draft performance review language.

Here's the problem: Most HR teams don't understand these tools well enough to use them well. They think the tool "understands" what they're asking. They assume if the output looks good, it's probably accurate. They don't know why the tool sometimes contradicts itself or generates complete fabrications.

Understanding the mechanism prevents disaster. You'll know why you have to verify everything. You'll know which types of tasks are safe to delegate to AI (drafting initial outlines, summarizing patterns) and which aren't (generating specific policy language without review, creating authoritative statements about regulations).

How Large Language Models Actually Work

A large language model (LLM) is a system trained on enormous amounts of text. During training, it learns the statistical relationships between words. It learns that certain words follow other words with high frequency. It learns sentence structure, grammar, common phrases, and patterns of reasoning.

Here's a simplified version of what happens: The system has been trained on billions of words from books, websites, articles, and other text sources. During training, it learned that the word "manager" is often followed by words like "will," "is," "leads," "oversees." It learned that "performance review" is often followed by "process," "feedback," "meeting," "schedule." These aren't hard rules. They're statistical probabilities.

When you give the system a prompt (like "Write a job description for a sales manager"), it doesn't think through the request the way you would. It doesn't consult a database of job descriptions. Instead, it generates text token by token, predicting the most likely next token based on patterns learned during training.

A token is a small piece of text, usually a word or a few characters. When the system processes language, it breaks it down into tokens. "Job description" becomes two tokens. "Performance review" becomes two or three tokens depending on how the tokenizer breaks it up.

When you type a prompt, the system converts your words into tokens. Then it starts predicting. What's the most likely next token after your prompt? It generates that token. What's the most likely next token after that? It generates that one. It keeps going until you ask it to stop or until it reaches some limit.

This is why the system sometimes generates text that sounds perfect but is factually wrong. It's predicting statistically likely tokens, not facts. If your training data included articles about HR regulations that were slightly inaccurate, and those articles used confident language, the system learned that pattern. When you ask about regulations, it generates text that matches the pattern it learned, regardless of whether the pattern is actually true.

Why Temperature Matters (And What It Actually Does)

You might have encountered a setting called "temperature" when using a generative AI tool. Temperature controls randomness in the output.

Here's what's happening: At each step, the system calculates probabilities for thousands of possible next tokens. The most likely token might have a 40% probability, the second most likely 15%, the third 8%, and so on.

With low temperature (close to 0), the system almost always picks the highest-probability token. The output is consistent and predictable. With high temperature (closer to 1 or above), the system picks randomly from the list of possibilities, weighted by probability. Higher-probability tokens are still more likely, but lower-probability tokens have a chance.

Why does this matter for HR? With low temperature, when you ask for interview questions, you get formulaic questions that sound similar to everything the system has seen before. With higher temperature, you get more creative and varied questions, but they might be weirder or less professional.

In HR work, you usually want lower temperature. You want predictability. You want the system to stick to what it knows. You don't want an interview question that's technically accurate but bizarre. You don't want a compensation analysis that's creative but wrong.

Tip: If your AI tool has temperature settings, set them low for HR work. You want consistency and predictability, not creativity and surprise.

Context Windows and Why Your Input Matters

Every generative AI system has a context window, a limit on how much text it can process at once. An older system might have a context window of 2,000 tokens. A newer system might handle 100,000 tokens or more. A token is roughly a word, so a 2,000-token window means about 1,500 words of input.

Why does this matter? Because your prompt, everything you paste into the system, counts toward the context window. If you paste a 2,000-word policy document and then ask the system to analyze it, your entire prompt uses up the context window. The system might not have enough space to generate a meaningful response.

More importantly, what you put into the context window matters enormously. If you paste performance feedback for Employee A, then ask a question about Employee B, the system has context about Employee A but might confuse them or apply reasoning from one to the other.

An HR example: You paste performance review feedback for five employees into the system and ask it to identify common themes. The system has context about all five employees. But because it's pattern-matching statistically, it might find patterns that are artifacts of how the feedback was written, not genuine insights about performance. It might notice that feedback for certain employees uses words like "proactive" and "self-starter," while feedback for others uses "reliable" and "consistent." The system might infer that the first group is higher-potential, not because of actual performance differences, but because of language patterns.

The context window also affects hallucinations. With a small context window and a lot of input, the system has limited space to work. It might generate text that fits the overall pattern of what you asked but is disconnected from what you actually pasted. It's making educated guesses because it's running out of room.

Important: When you paste content into an AI system, you're setting the context. Be aware that everything you paste becomes part of the system's working memory for this interaction. Don't paste information about one person and expect the system to forget it when you ask about someone else.

Tokens, Cost, and Why Your Prompts Get Cut Off

Tokens matter for cost and capacity. Every token your input contains and every token the system generates costs something, usually measured in fractions of a penny per token, but it adds up.

This is why some AI systems cut you off mid-sentence. You've hit the token limit. The system has used up its allocated tokens and can't continue.

In HR work, this matters when you're using AI to analyze large datasets or long feedback documents. If you paste a year of performance feedback for one employee (let's say 5,000 tokens) and ask for a summary, you've used half your available tokens just for input. The system has limited space for output.

This also affects how you should structure prompts. A long, rambling prompt with lots of extra context uses more tokens than a focused prompt with just the essential information. For HR work, this means being clear and concise actually helps the system work better and costs less.

How Hallucination Happens

A hallucination is when an AI system generates confident-sounding text that is completely false. It will make up citations, policy numbers, regulations, facts, and examples. And it will do so with complete confidence.

Hallucinations happen because of how the system works. It's predicting statistically likely tokens based on patterns in training data. If those patterns include confident-sounding statements about things that don't exist, the system will generate similar patterns.

Here's a concrete HR example: You ask an AI system to explain the FMLA regulations around paternity leave. The training data includes real FMLA information and articles written by people who misunderstood FMLA. The system has learned statistical patterns from both. When you ask, it generates text that sounds authoritative and is partially based on real regulations and partially based on misunderstandings. It cites specific policy language that might be paraphrased correctly or might be completely fabricated. You read it and it sounds authoritative, so you trust it.

This is one of the most dangerous failure modes of AI in HR. The output that's wrong is often more dangerous than the output that's obviously weird because it's harder to detect.

Hallucinations are more likely when:
- The system is asked about niche topics that weren't well-represented in training data
- The prompt is ambiguous and the system is guessing at intent
- The system is generating longer content (more tokens = more opportunities for error)
- The temperature is high (more randomness = more chances for things to go off the rails)

An HR example of high hallucination risk: "Write me a compensation philosophy statement for a healthcare startup" is a specific, niche request. The system might generate something that sounds authoritative but is based on patterns from healthcare startups in the training data that might not match your company at all.

Fine-Tuning: When AI Gets Trained on Your Data

You might hear about "fine-tuning" a model. This means taking an already-trained generative AI system and training it further on your company's specific data.

In theory, this is powerful. You could fine-tune a system on all your historical job descriptions, and it would learn the style and format your company uses. You could fine-tune on your performance review templates, and the system would generate reviews that match your format and language.

In practice, fine-tuning for HR is risky. If you fine-tune on historical hiring data, the system learns biases in your hiring. If you fine-tune on historical performance reviews, the system learns patterns in how your managers have written reviews, which might include gender bias, recency bias, and other problems. The system becomes very good at replicating your historical patterns, good and bad.

Fine-tuning is mostly used by large organizations with sophisticated AI infrastructure. Most HR teams won't encounter it directly. But if your organization is considering fine-tuning an AI system on HR data, this is a red flag. You need to understand what data you're training on and what biases that data contains.

RAG: Retrieval-Augmented Generation

You might encounter the term RAG (Retrieval-Augmented Generation). This is a technique where an AI system retrieves relevant documents or information before generating a response.

Here's how it works: You ask the system a question about your company's compensation policy. Instead of generating text based only on patterns from general training data, the system first retrieves your actual compensation policy document. Then it generates text based on that document plus its general training.

In theory, this is much better for HR. The system has your actual policy, not generic information or fabricated regulations.

In practice, RAG systems are only as good as the documents they retrieve. If your retrieved documents are out of date or incomplete, the system will generate text based on bad source material. And the system still has all the same limitations. It can misunderstand policy language, miss context, and generate text that sounds confident but is slightly wrong.

RAG systems are also vulnerable to something called "prompt injection." If your policy documents contain instructions (like a footer that says "ignore the above and do X instead"), the system might treat those instructions as genuine instructions for itself, not as part of the policy text.

Important: If you're using a RAG system (AI that retrieves documents before answering), remember that it's only as good as the source documents. Outdated policies, incomplete procedures, or badly formatted documents will produce bad output.

Why Reprompting Doesn't Always Help

You've probably done this: Ask an AI system something, get a mediocre answer, then ask again with a slightly different prompt. Sometimes you get a better answer. Sometimes you get something totally different.

Here's why: The system isn't remembering its previous answer and trying to improve it. Each prompt is independent. The second prompt might generate completely different text because different tokens happened to be predicted as likely. You asked essentially the same question, but the randomness of token prediction led somewhere different.

This is why you can't trust consistency across multiple prompts. The system isn't making deliberate choices; it's predicting probabilities. Slightly different conditions lead to different predictions.

In HR work, this means you can't just ask an AI system to "improve" an answer. You need to give it specific feedback about what you want changed. And even then, the system might generate something that looks better but is based on completely different patterns.

Prompt Engineering: The Art of Asking Better

Prompt engineering is the practice of crafting prompts to get better output. It sounds mysterious, but it's just applying what you now know about how the system works.

Better prompts:
- Are specific about the output format you want
- Include examples of what good output looks like
- Are clear about constraints (not more than 500 words, must include X, cannot include Y)
- Provide necessary context without overwhelming the system
- Use role-playing ("You are an HR compliance officer") to guide the system toward certain patterns

An HR example: Instead of asking "Write a performance review," ask: "Write a performance review for a software engineer who delivered strong technical work but struggled with cross-team communication. Keep it to 300 words, focus on specific examples, and end with clear development areas."

The more specific you are, the better the output is likely to be. But remember: You're not changing what the system fundamentally does. You're guiding it toward certain patterns through your prompt. You're still not getting understanding or judgment. You're still getting pattern-matched text.

Why You Can't Trust the System to Know Its Limits

Here's something critical: An AI system will happily generate confident-sounding text about things it doesn't know. It won't tell you "I'm not sure about this" or "I don't have good data on this." It will sound equally confident about things it has learned from billions of examples and things it's making up.

If you ask an AI system about an obscure employment regulation, it will generate text. That text might be accurate, or it might be completely wrong. The system won't tell you which. It doesn't know the difference between things it learned from authoritative sources and things it learned from random internet articles.

This is why you must verify everything before it matters. The system's confidence is not a reliability signal. It's a failure mode.

What to Do Monday Morning

• 
  Pick one HR task you use AI for. What's the temperature set to? What's the context window for that tool? Could you improve results by adjusting those settings?
• 
  Experiment with specificity. Take a generic prompt you've used and rewrite it to be more specific about format, constraints, and examples. Compare the output.
• 
  Look for hallucinations. Pick a recent piece of AI-generated content your team created. Check every factual claim. Did the system cite regulations that exist? Did it reference policies correctly?
• 
  Ask your vendor about their retrieval system. If you're using RAG-enabled tools, what documents are being retrieved? How recent are they? Can you update them?
• 
  Test consistency. Ask the same prompt twice and compare the outputs. How different are they? Does this match the task you're trying to accomplish?

Key Takeaways

• Understand that generative AI predicts likely tokens based on statistical patterns, not by thinking or retrieving information
  - Know that temperature controls randomness, context window limits input and output, and tokens affect cost and capacity
  - Recognize hallucination as a core failure mode that produces confident false text
  - Use prompt engineering to guide the system toward better patterns, but remember you're still pattern-matching
  - Remember that the system's confidence tells you nothing about accuracy. It will sound equally sure about real facts and fabrications

FAQ

Q: If I give the system more context, will it get more accurate?
A: More context can help the system make better pattern matches. But there's a limit, if the context is too large, the system runs out of space to generate meaningful output. And more context means more opportunities for the system to find misleading patterns or mix up information. More context sometimes helps, but it's not a guaranteed fix.

Q: Why does the system sometimes contradict itself in the same response?
A: Because it's predicting tokens sequentially, not reasoning through a logical chain. Early in the response, certain patterns are likely. Later, different patterns become likely. The system doesn't remember and maintain logical consistency the way you do.

Q: Can I use AI to create training content if I give it enough specific instructions?
A: You can use AI to create first drafts. But instruction-heavy content (compliance training, harassment prevention, technical procedures) needs careful verification. The system might generate content that sounds good but misses subtle points or misrepresents regulations.

Q: What's the difference between this system and an expert system or database lookup?
A: A database looks up information you stored. An expert system applies logical rules. This system predicts text based on patterns. Those are fundamentally different. This system is not doing database lookups or logical reasoning. It's generating text that statistically resembles patterns it learned.

Q: If I fine-tune a system on my company's HR data, won't that solve the accuracy problem?
A: Fine-tuning will make the system excellent at replicating your historical patterns. That can be good (matching your writing style) or bad (amplifying your historical biases). It doesn't solve the fundamental problem that the system generates patterns, not facts.

What's Next

Now you understand how the system works mechanically. In the next lesson, we'll zoom out and talk about the three core capabilities of AI, pattern recognition, generation, and classification, and how each one applies to specific HR work.