A hospital's new AI governance committee holds its first meeting, and a vendor is presenting. He says the model is "well calibrated, grounded on your formulary, with strong specificity and a human in the loop." Around the table, a chief nurse, a hospitalist, and a quality director nod along, and not one of them is certain they mean the same thing by those words. That uncertainty is not a small thing. In a room where AI tools are approved or rejected for use on real patients, the vocabulary is the instrument of judgment, and a word half-understood is a decision half-made. This lesson is the glossary that closes that gap. These are not data-science trivia. They are the words that let you ask the right question, read the right red flag, and tell a safe tool from a dangerous one.

Why the Words Are the Skill

Every term below has a precise meaning that matters clinically and a fuzzy meaning that lives in marketing decks, and the distance between the two is exactly where patients get hurt and committees get fooled. We will define each plainly, then attach a one-sentence reason you care, tied to the only thing that ultimately matters: whether the tool is safe for a patient and defensible in a record. Read these not as flashcards but as the operating language of a clinician who now works alongside AI and sometimes has to judge it. When you can hold the precise meaning while the vendor uses the fuzzy one, you have the single most valuable skill in the room.

Consider a small worked example before the definitions arrive, because it shows what these words are for. A product sheet says a tool is "94 percent accurate, AI-powered, and clinically validated." Read loosely, that is a reassuring sentence. Read precisely, it is three claims you can test. "94 percent accurate" is one number that hides two very different ones, the catch-rate and the clear-rate, and it says nothing about what a positive alert means at your bedside. "AI-powered" is not a specification at all; the useful word is "model," which forces the question of what goes in and what comes out. "Clinically validated" invites the follow-up that decides everything: validated on which patients, and are they like yours. The same sentence, translated word by word into precise language, turns from a brochure into a short list of questions the vendor must answer. That translation is the entire skill, and the rest of this lesson gives you the vocabulary to perform it on demand.

Words About How the Machine Works

Model and Large Language Model (LLM)

A model is a trained mathematical system that maps an input to an output: an image to a label, a patient's data to a risk score, a prompt to text. A large language model is the specific kind that generates text by predicting the next piece over and over. Why you care: "AI" is a marketing word, but "model" forces the real question, what does this specific system take in, and what does it put out, which is where all safety analysis begins.

Training Data

Training data is the body of examples a model learned from. Everything the model can do well, and every population it quietly does poorly on, traces back to what was and was not in this data. Why you care: when you ask "was this validated on patients like mine," you are really asking about training data, and a model trained on a population unlike yours is a model you cannot trust by default.

Prompt and Grounding

A prompt is the instruction and context you give a generative model. Grounding is the practice of giving the model real source material to work from, your formulary, this guideline, this chart, so it answers from provided facts rather than its foggy memory. Why you care: grounding is the difference between a model reciting a plausible guess and a model reasoning over a document you can check, and it is the backbone of nearly every safe clinical AI workflow.

Hallucination

A hallucination is a confident, plausible output that is simply not true: an invented lab value, a fabricated citation, a drug interaction that does not exist. It is not a rare glitch; it is the same generation mechanism producing text that happens not to match reality. Why you care: because it arrives in the same calm, competent voice as the correct material, you cannot detect it by tone, only by verifying the specific claim.

Context Window and Model Drift

The context window is how much text a model can consider at once; beyond it, the model simply does not "see" the information. Model drift is the gradual degradation of a model's performance over time as the real world changes out from under it. Why you care: a summary can miss what fell outside the context window, and a tool that was accurate at go-live can quietly become unsafe months later, which is why monitoring never ends.

Black Box

A black box model is one whose internal reasoning cannot be readily inspected or explained; you see the input and the output but not a clear "why." Why you care: much of modern AI is a black box to some degree, so you cannot always demand an explanation of a single output, which makes the outside checks (validation, calibration, the source attributes, and your own verification) matter more, not less.

Overfitting

Overfitting is when a model learns the quirks of its training data so closely that it performs impressively in the lab and poorly on real, new patients. Why you care: a dazzling published accuracy number can be a warning sign rather than a reassurance if the tool was never shown to hold up outside the data it was built on, which is why external and local validation matter.

Words About Whether to Trust a Number

Sensitivity and Specificity

Sensitivity is how well a tool catches the patients who truly have the condition (few false negatives). Specificity is how well it clears the patients who truly do not (few false positives). Why you care: a tool can be excellent at one and poor at the other, and which failure you can tolerate depends entirely on the clinical stakes, so a single "accuracy" number tells you almost nothing until it is split into these two.

Positive and Negative Predictive Value

Positive predictive value (PPV) is the chance that a positive result is truly positive; negative predictive value (NPV) is the chance a negative is truly negative. Crucially, both depend on how common the condition is in your population, not just on the tool. Why you care: in a low-prevalence setting, a tool with impressive accuracy can still throw false alarms most of the time it fires, and PPV is the number that tells you what a positive alert actually means for your patients.

These four numbers travel together, and it helps to see them side by side. Sensitivity and specificity are properties of the test itself: they answer "given the truth, how does the tool behave." Predictive values flip the question to the one that matters at the bedside: "given the tool's result, what is the truth likely to be," and that answer changes with prevalence even when the tool does not.

Calibration

Calibration is whether a model's stated probability matches reality: when it says 30 percent, do 30 in 100 such patients actually have the outcome. Why you care: a miscalibrated model's numbers are lies with decimal points, confident and precise and wrong, and calibration in your own setting is what makes a risk score usable rather than merely impressive. A useful analogy is the weather forecast. A "30 percent chance of rain" is only trustworthy if, across all the days the forecaster said 30 percent, it actually rained on about 3 in 10 of them. If it rains on 6 in 10 of those days, the forecast is not slightly off, it is systematically wrong in a way that looks precise, and you would stop trusting the number even though it still comes with a confident decimal. A clinical risk score deserves exactly that scrutiny: a sepsis model that says "30 percent risk" while 60 percent of such patients decompensate is miscalibrated, and acting on its 30 as if it were real understates the danger for every one of those patients.

Confidence Score

A confidence score is a number a model attaches to its own output, and it is tempting to read it as "how likely this is correct." Why you care: a confidence score reflects the model's internal certainty, not verified truth, and a generative model can be confidently wrong, so a high score is a prompt to verify important claims, never a substitute for doing so. Confidence and calibration are easy to confuse and worth separating: calibration is measured against reality across many cases, while a confidence score is the model's self-report on one output, and a black-box model can report high confidence on a hallucination.

False Positive and False Negative

A false positive flags something that is not there; a false negative misses something that is. Why you care: these are not symmetric in medicine. A false negative on a cancer screen and a false positive that triggers an invasive workup carry very different harms, and you cannot evaluate any predictive tool without asking which error it makes and what that error costs the patient.

Words About the Human and the Failure

Algorithmic Bias

Algorithmic bias is systematic worse performance for a particular group, usually one underrepresented in the training data. Why you care: it means a tool can raise the average quality of care while lowering it for exactly the patients already underserved, and that is both a patient-safety failure and a question about whether the standard of care was met.

Automation Bias

Automation bias is the human tendency to over-trust an authoritative machine output and skip the checking we would otherwise do, especially under time pressure. Why you care: it is the failure mode that turns a model's error into a patient's harm, because the tool being usually right is exactly what trains a busy clinician to stop looking.

Human in the Loop

Human in the loop means a competent person reviews and is accountable for the AI's output before it acts on a patient. Why you care: it is the design pattern that keeps accountability where it belongs, and "the human in the loop" only counts if that human actually verifies rather than rubber-stamps, which is the whole discipline this program builds.

Audit Trail

An audit trail is the record of what the AI produced, what the human did with it, and why, such that a decision can be reconstructed later. Why you care: when a chart is reviewed, a claim is audited, or a case is litigated, the audit trail is what shows a human made the call, and a decision no one can reconstruct is a decision you cannot defend.

In clinical AI, a word you half understand is a risk you cannot see. The vocabulary is not the soft part of the skill. It is the instrument you judge the tool with.

Words About Privacy, Law, and Accountability

PHI and the Business Associate Agreement

Protected health information (PHI) is individually identifiable health information governed by HIPAA. A business associate agreement (BAA) is the contract that lets a vendor handle PHI under HIPAA's rules. Why you care: pasting a patient's story into a consumer AI tool with no BAA is a breach, full stop, and the presence or absence of a BAA is the first question before any real patient data touches any tool.

Minimum Necessary

Minimum necessary is the HIPAA principle of using or disclosing only the least PHI needed for the task. Why you care: it is a habit that shrinks your risk surface every time you use AI, because the data you never put into a tool is the data that can never leak from it.

Predictive DSI and the Source Attributes

A predictive decision support intervention (predictive DSI) is the regulatory name, under the federal HTI-1 rule, for AI that predicts or classifies to support a clinical decision. The source attributes are the standardized set of facts a certified tool must disclose about how it was developed, validated, and where it applies. Why you care: this is your right to look under the hood, and the source attributes are exactly the information you need to answer "does this apply to my patient."

Standard of Care

The standard of care is what a reasonably prudent clinician would do in the same situation, and it is the yardstick in a malpractice case. Why you care: in 2026 it is evolving to include using AI appropriately, which can mean liability both for following a wrong AI output and for ignoring an accurate one, so the safe path is documented judgment, not blind trust or blanket refusal.

The Words in Motion: From Marketing to Judgment

Watch the vocabulary turn into judgment. The committee from the opening is now evaluating a sepsis prediction tool, and a member who holds these words precisely runs the room. The vendor leads with the headline: "94 percent accurate." Instead of nodding, she asks him to split it. What is the sensitivity and what is the specificity, because a single accuracy number could hide a tool that catches almost every septic patient while flooding the unit with false alarms, or one that is quiet but misses real cases. He gives the numbers, and now the conversation is real.

Next she asks about the population. On whom was this calibrated and validated, and does it include patients like ours, because a tool tuned on an academic center's patients may be miscalibrated on her community hospital's older, sicker population. This is the training-data question wearing its practical clothes. Then she asks the one that changes everything for a sepsis tool: what is the positive predictive value at our sepsis prevalence, because if only a small fraction of screened patients are truly septic, even a strong tool may be wrong most of the times it fires, and a unit that learns the alert is usually a false alarm will start ignoring it, which is automation bias inverted into alarm fatigue.

She asks whether the tool was tested for bias across the subgroups her hospital serves, because a model that performs worse for a population already underserved is a safety and equity failure, not a footnote. She asks to see the source attributes the tool must disclose as a predictive decision support intervention, so she can read its limits rather than take the brochure's word. And she asks the human-factors question the vendor never volunteers: when the alert fires, exactly who verifies it, do they have the time to actually assess the patient, and how is that verification recorded so the decision can be reconstructed later. In fifteen minutes, using nothing but the words in this lesson, she has moved the committee from "it sounds impressive" to a clear-eyed judgment about whether this tool is safe for her patients. That is what the vocabulary is for. It is not knowledge to recite; it is leverage to protect people.

The same translation habit applies to the small, everyday moments, not just the committee room. Three short cases show the terms doing their work. First, a miscalibrated sepsis score: a busy resident sees "risk 30 percent," treats it as low, and moves on, but the tool has never been checked against her hospital's outcomes, and in fact 60 percent of patients it labels 30 percent go on to decompensate. The word "calibration" is what turns her passing glance into a question, and the question is what protects the patient. Second, a PHI paste with no BAA: a coordinator, trying to save time, drops a patient's full narrative into a consumer chatbot that the health system never signed a BAA with. There was no bad intent and no dramatic failure, yet the moment the identifiable data left the protected environment, it was a breach, and "minimum necessary" would have told her that the safest data is the data she never pasted at all. Third, a drifting model: a readmission predictor praised at go-live begins firing on the wrong patients eight months later, after the payer mix and documentation patterns shifted underneath it. Nothing broke visibly; the tool simply drifted, and only ongoing monitoring, not the original validation certificate, could have caught it. In each case the failure was invisible until the right word made it visible, and the person holding that word was the only safeguard that would have fired in time.

It is worth naming what these cases have in common, because it is the same lesson every term in this glossary teaches from a different angle. The dangerous AI failure in medicine is rarely the obvious crash. It is the plausible output, the impressive number, the convenient shortcut, each of which looks fine until you translate it into precise language and ask the question the loose language was hiding. The resident who says "calibration," the coordinator who remembers "minimum necessary" and "BAA," and the analyst who names "drift" are all doing the same thing: refusing to accept a comfortable word in place of a testable one. That refusal, practiced until it is automatic, is what this vocabulary exists to build.

Notice, stepping back, that none of these terms lives alone. Accuracy only means something once split into sensitivity and specificity, which only translate to a bedside decision through positive predictive value, which only makes sense against your prevalence, which is only trustworthy if the model is calibrated on a population reflected in its training data. Hallucination is tamed by grounding and caught by the human in the loop, whose verification is preserved in the audit trail, which is what the standard of care and a future chart review will look for. PHI, the BAA, and minimum necessary form the privacy floor under all of it. The vocabulary is a web, and pulling any single thread brings the others with it, which is exactly why learning them together, as one connected language, is worth the effort. Master the web and you can follow any AI claim back to the question that tests it.

There is a deeper reason to invest here. Vendors, headlines, and even well-meaning colleagues will keep offering you these words in their loose, flattering sense, where "accurate" means trustworthy and "AI" means magic. Every time you quietly translate the loose word back into its precise one, you convert marketing into a testable claim, and a testable claim is one a patient can be protected from. That translation habit, running silently as you read a study, sit in a demo, or open a new tool in the EHR, is the practical payoff of this entire lesson. The terms are not the finish line. They are the lens you will look through for the rest of the program, and for the rest of your career alongside these tools.

Key Takeaways

• In a room where AI tools are approved for real patients, the vocabulary is the instrument of judgment, and a word half understood is a decision half made.
• How the machine works: a model maps inputs to outputs, its training data determines who it serves well, grounding feeds it real sources, hallucination is confident output that is simply untrue, and a black box hides its reasoning so outside checks matter more.
• Whether to trust a number: sensitivity and specificity split a meaningless "accuracy" into catch-rate and clear-rate; PPV and NPV depend on your prevalence; calibration is whether the stated probability is real; a confidence score is self-report, not verified truth.
• False positives and false negatives are not symmetric in medicine; always ask which error a tool makes and what it costs the patient.
• The human and the failure: algorithmic bias harms underrepresented groups, automation bias makes us over-trust the machine, and a real human in the loop verifies rather than rubber-stamps.
• The audit trail is what lets a decision be reconstructed and defended, showing a human made the call.
• Privacy and law: PHI is protected, a BAA is required before real patient data touches a tool, minimum necessary shrinks your risk, and the source attributes of a predictive DSI are your right to look under the hood.
• The standard of care now includes using AI appropriately, which is why documented human judgment, not blind trust or blanket refusal, is the defensible path.