A hospital pharmacist named Marcus asked an AI assistant a simple question during a night shift: what is the maximum daily dose of a particular antiepileptic for an adult with normal renal function? The assistant answered instantly, with a clean number and a confident one-line rationale. Marcus happened to know the real answer cold, and the number was wrong, low enough to undertreat a seizing patient. Curious, he opened a fresh window and asked the exact same question again. This time the assistant gave the correct number. Same model, same question, two different answers, both delivered with identical confidence and not a hint that one of them was wrong. Marcus did the right thing and verified against the reference, but the experience left him with a question that this lesson exists to answer: how can a tool be wrong and right about the same fact, and what is actually happening inside it when it drafts a clean prior authorization one moment and invents a diagnosis the next? You do not need to become an engineer to use these tools safely. But you do need a working mental model of the machinery, because the way generative AI works is the direct cause of every failure mode you will spend this program learning to catch.

The Core Idea: Prediction, Not Retrieval

Here is the single most important sentence in this lesson, and it is worth reading twice. A generative AI model does not look up answers; it predicts text. When you ask it a question, it is not searching a database of verified facts and returning the matching record. It is generating a response one small piece at a time, and at each step it is answering an entirely different question than the one you think you asked. You asked "what is the maximum dose?" The model is internally answering "given all of this text so far, what word or fragment most plausibly comes next?" It does this over and over, fragment by fragment, until it has produced a complete answer. The fluency is real. The grammar is perfect. The confidence is total. But underneath, the only thing happening is a very sophisticated, very fast version of guessing the next piece of text.

The pieces of text the model works with are called tokens. A token is roughly a word or part of a word; "metformin" might be one or two tokens, "renal" another, a space and punctuation their own. The model was trained by being shown an enormous quantity of text and learning, with extraordinary statistical precision, which tokens tend to follow which other tokens across billions of examples. That training produced a vast set of internal numerical weights that encode those patterns. When you prompt the model, it runs your text through those weights and produces, for every possible next token, a probability. Then it picks one (usually a high-probability one), adds it to the running text, and repeats. That loop, predict a token, append it, predict the next, is the entire engine. Everything the model can do, the brilliant PA draft and the fabricated dose alike, comes out of that same loop.

A generative model is a next-token prediction engine. It does not retrieve facts; it generates the most statistically plausible continuation of the text in front of it.

Why It Sounds So Right (and Why That Is the Trap)

If the model is just guessing the next token, why is it so good? Because the training data was vast and mostly coherent, the statistical patterns it learned are genuinely rich. Medical and pharmaceutical text has strong regularities: the sentence "the maximum recommended dose is" is very often followed by a number and a unit; "contraindicated in patients with" is followed by a condition; a prior-authorization justification has a predictable shape. The model learned those shapes deeply. So when it drafts a clinical justification, it produces something that has the correct structure, the correct vocabulary, and the correct rhythm of a real justification, because it has seen thousands of them. The output is fluent because fluency, the surface form of language, is exactly what next-token prediction is built to reproduce.

Here is the trap, stated plainly. Fluency and accuracy are two different things, and the model only guarantees the first. The same mechanism that makes "the maximum recommended dose is" reliably produce a number does not guarantee that the specific number is the correct one. The model will confidently complete the pattern; whether the value it fills in matches reality depends on whether that exact fact was strongly and consistently represented in its training, whether the surrounding context nudged it correctly, and a degree of genuine randomness in how it picks tokens. For a famous, frequently stated fact, the most probable token is usually the correct one. For an edge case, a less common drug, an unusual indication, a specific renal threshold, the most probable token may be a plausible-sounding wrong one. The output looks identical either way. That is why a pharmacist cannot use the surface quality of an AI answer as evidence of its correctness. The surface quality is precisely the thing the model is best at faking.

Why the Same Question Gives Two Answers

Marcus's puzzle, two different doses for the same question, has a precise explanation, and it reveals something important. When the model picks the next token, it does not always pick the single most probable one. To make output sound natural and not robotic, these systems introduce a controlled amount of randomness, often described with a setting called "temperature." A higher temperature means the model is more willing to pick a slightly-less-probable token; a lower temperature makes it more deterministic. With any randomness above zero, the same prompt can branch down different paths on different runs. If the correct dose and an incorrect-but-plausible dose were both reasonably probable continuations, you can get one on the first run and the other on the second. Neither run knows it diverged. Each is internally consistent and confidently stated.

For a pharmacist, this has a concrete and slightly unsettling implication: the model's output is not stable the way a calculator's is. Two plus two is four every time. The maximum dose of a drug, asked of a generative model, is not guaranteed to be the same value every time, let alone the correct value. This is not a defect you can configure away entirely, and it is not a sign the tool is broken. It is a property of how the machine generates language. It is also the strongest possible argument for the program's central discipline: you verify the specific fact against an authoritative source every time, because the tool itself cannot promise you consistency, only plausibility.

What a Hallucination Really Is, Mechanically

"Hallucination" is the word everyone uses for when a model states something false as if it were true. Now that you understand the engine, you can see that a hallucination is not a special malfunction. It is the same next-token prediction doing exactly what it always does, in a situation where the most plausible continuation happens not to be true. The model has no separate "fact mode" and "guess mode." It is always generating the most plausible-sounding continuation. When that continuation matches reality, we call it correct. When it does not, we call it a hallucination. From the inside, the two are identical processes producing identical-looking output.

This reframing matters because it kills a comforting but dangerous assumption: that hallucinations are rare glitches you will obviously notice. They are not glitches at all; they are the normal operation of the machine landing on a wrong-but-plausible answer, and they are designed, structurally, to be unnoticeable. The model fabricates a clinical trial that does not exist, a drug interaction that is not real, a coverage criterion the payer never published, all in flawless professional prose, because producing flawless professional prose is the one thing it is built to do. A common and especially seductive version in our field is the fabricated citation: the model produces a reference, complete with authors, journal, year, and a plausible title, for a paper that was never written. It looks like a verifiable source. It is generated text shaped like a source. The lesson on clinical hallucinations later in this chapter goes deep on the specific clinical forms; the point here is that you now understand the cause. A hallucination is plausibility untethered from truth, which is the model's default state, not its failure state.

Grounding: Improving the Odds From Inside the Machinery

If plausibility is the problem, the partial solution is to make the truth also the most plausible continuation, and you do that by controlling what text the model sees before it answers. This is the single most useful technical idea for a working pharmacist to hold, and it has a name: grounding, often implemented through a technique called retrieval-augmented generation, or RAG. The idea is simple. Instead of asking the model to answer from its diffuse training memory, you first place the authoritative source text, the actual formulary entry, the real payer criteria, the patient's actual chart, directly into the model's context, and then ask it to answer using that. Now the most plausible continuation is shaped by the real document sitting right there in front of it, not by a vague statistical memory of a million documents.

Grounding dramatically improves accuracy, and most well-built pharmacy AI tools use some form of it: the good prior-authorization assistants are grounded on the actual chart and the actual payer rules, which is why they can be genuinely useful. But, and this is the part that keeps a pharmacist employed, grounding reduces hallucination; it does not eliminate it. A grounded model can still misread the source, blend the right document with its own training assumptions, extract the wrong value, or quietly add a detail the source does not contain. Grounding moves the odds strongly in your favor. It does not move them to certainty. So even with a grounded tool, the final verification, does this specific claim actually appear in this specific source, stays with you. Understanding grounding tells you which AI tools to trust more (the grounded ones) and exactly where their remaining risk lives (the gap between the source and what the model says the source says).

What This Mental Model Buys You

You now have a working model of the machine, and it pays off immediately in how you read AI output. When the model gives you a fluent answer, you know that fluency was free and guaranteed, so it tells you nothing about accuracy. When it states a specific number, you know that number is a probable continuation, not a retrieved fact, so it must be checked against an authoritative source. When it gives you two different answers, you understand the randomness underneath and you trust neither without verification. When it produces a perfect citation, you know a citation-shaped string is exactly the kind of thing the engine excels at generating, so you confirm the paper exists. When you use a grounded tool, you trust it more but you still verify the claim against the source, because the source-to-claim gap is where grounded tools fail. None of this requires engineering knowledge. It requires exactly the mental model you now hold, and the habit of letting that model drive your skepticism.

That is the whole purpose of understanding the machinery: not to make you an AI expert, but to make your verification intelligent rather than ritualistic. A pharmacist who knows why the model hallucinates verifies the right things in the right places. A pharmacist who treats the model as a magic answer box either trusts it too much, which is dangerous, or distrusts it uselessly, which wastes the real value it offers. The skilled middle, lean on it for the fluent draft, verify every load-bearing fact against the source, is only available to someone who understands what is actually happening inside. Now you do.

A Tour of the Failure Modes You Now Understand

With the engine clear in your mind, it is worth walking through the specific ways it fails in a pharmacy, because each one is now explainable rather than mysterious, and an explainable failure is a catchable failure. Consider the most common patterns a working pharmacist will actually meet.

The drifted dose. You ask for a dose and get a number that is in the right ballpark but slightly wrong, say, a maximum that is real for a related drug or a different indication. Mechanistically, the model landed on a high-probability number from a nearby pattern rather than the exact fact for this drug and this indication. It is plausible because it is close, which is exactly what makes it dangerous: a wildly wrong number gets caught, a slightly wrong one slips through. The defense is that doses are load-bearing facts and load-bearing facts get checked against an authoritative reference, every time, no exceptions for numbers that "look about right."

The confident citation. You ask for evidence and get a beautifully formatted reference, authors, journal, year, a plausible title, that does not exist. The engine is doing what it does best, generating text in the exact shape of a real citation, because it has seen thousands of them. The shape is perfect; the existence is fictional. The defense is to treat a citation as a claim to verify, not a source to trust, and to actually confirm the paper or guideline exists and says what the model claims.

The blended chart. Using a grounded tool on a real record, the output quietly mixes a value from one section with a label from another, or carries forward a detail the chart does not contain. This is the residual risk of grounding: the source was present, but the model misread or supplemented it. The defense is the source-to-claim check, confirming that each stated value actually appears, as stated, in the actual record.

The smoothed-over warning. Asked to simplify a complex clinical point for a patient, the model produces friendly, readable language that has quietly dropped or softened a real warning. Nothing was fabricated; something true was lost. The defense is to fact-check the simplified version against the full, true content, confirming that plain language did not become incomplete language. Every one of these is the same engine doing the same thing, generating the most plausible continuation, in a spot where plausible and true diverged. You do not need a different theory for each. You need the one mental model and the discipline to verify the load-bearing facts wherever the engine's plausibility might have drifted from the truth.

Key Takeaways

• A generative model does not retrieve facts; it predicts text one token at a time, always producing the most statistically plausible continuation of the input it was given.
• Fluency and accuracy are different things, and the model only guarantees fluency; perfect grammar and confident tone are the surface form it is built to reproduce, regardless of whether the content is true.
• The same question can yield different answers across runs because these systems use controlled randomness (temperature); model output is plausible, not stable like a calculator, which is why every load-bearing fact must be verified each time.
• A hallucination is not a special malfunction; it is normal next-token prediction landing on a plausible-but-false continuation, and it is structurally designed to look identical to a correct answer.
• Fabricated citations, complete with authors, journal, and title, are a classic clinical hallucination because a source-shaped string is exactly what the engine is good at generating.
• Grounding (retrieval-augmented generation) places the authoritative source in front of the model before it answers, which strongly improves accuracy and is why well-built pharmacy tools are grounded on the chart and payer rules.
• Grounding reduces hallucination but does not eliminate it; the remaining risk lives in the gap between the source and what the model says the source says, so the final claim-to-source verification stays human.
• Understanding the machine makes verification intelligent rather than ritualistic: it tells you that fluency proves nothing, specific facts must be checked, and grounded tools earn more trust but still require confirmation.