Catching Hallucinations and Misalignment in Learning Output
A reviewer has an AI-drafted screen open in one window and three documents open behind it. The screen reads cleanly: a definition, a threshold, a practice item with feedback. She checks the fact against the policy, and it matches. She checks the item against the objective, and it looks fine. She nearly signs it. Then she does the one thing the two single checks could not do alone: she lays the source, the objective, and the accessibility standard side by side at the same time, and the screen falls apart. The threshold is real, but it answers a different objective than the one the module promised. The item is well written, but it tests recall when the objective demands application. Each single check passed. The three-way check caught it. That gap, between a screen that survives any one review and a screen that survives all three at once, is the whole subject of this lesson.
Why Single Checks Pass Broken Work
By Level 3 you can already do a great deal. You can ground a model on a source of truth, draft a module, generate an item bank, and run an accessibility pass. The trap that catches experienced people is not that they skip checks. It is that they run the checks one at a time, in sequence, and a sequential check has a blind spot that a wrong screen will hide in.
Start with vocabulary, because the program defines its terms on first use. A hallucination is a confident, fluent statement the model produces that is simply not true: an invented policy threshold, a fabricated citation, a procedure step that was never in the source. Misalignment is different and quieter. A misaligned screen contains true content, but the content does not serve the objective it sits under, or the assessment does not measure the skill the objective names. Why you care: a hallucination ships a false fact to the workforce, and a misalignment certifies the wrong skill. Both put your name on a record an auditor can reopen, and the two failures hide from different reviewers.
Run the checks separately and watch the blind spots open. A fact check against the source catches the hallucination but says nothing about whether the true fact belongs under this objective. An alignment check against the objective catches the misaligned item but assumes the fact it rests on is true, so it waves a confidently wrong threshold straight through. An accessibility check against the standard catches the missing caption and the failing contrast but cannot see that the caption transcribes a hallucinated sentence. Each reviewer is honest, competent, and looking at exactly the wrong axis to catch the others' error. The screen that is false-but-aligned, or true-but-misaligned, or correct-but-inaccessible, sails through every individual gate.
A screen that passes three checks run one at a time is not a verified screen. It is a screen that found the seam between your reviewers.
The Three-Way Check, Defined
The discipline that closes the seam is to cross-reference three things at the same time, on the same screen, before you sign it: the source the claim comes from, the objective the screen serves, and the standard the experience must meet. Not three reviews on three days. One pass that holds all three in view, because the errors that matter live in the relationships between them, not inside any one of them.
Define each axis precisely, in L&D terms, so the check is repeatable rather than a vibe.
Axis One: The Source
The source is the approved document the claim must trace to: the SOP, the policy, the regulation as loaded, the SME-verified transcript. The question is not "does this sound right" but "can I put my finger on the exact line in the approved source that says this." A claim with no source line is a hallucination until proven otherwise, no matter how plausible it reads. This is provenance: every load-bearing claim points back to where it came from. Why you care: a compliance officer does not ask whether the sentence is well written. They ask which document it came from, and "the model said so" is not a document.
Axis Two: The Objective
The objective is the measurable learning goal the screen is supposed to advance, written at a specific Bloom's level (the cognitive demand, from remembering up through applying, analyzing, and creating). Constructive alignment is the principle that the content, the activity, and the assessment all serve the same objective at the same cognitive level. The question on this axis is "does this true fact, and this item, actually move the learner toward this objective at the level the job demands." A true fact under the wrong objective is clutter. An item that tests recall when the objective says "apply" is misaligned even when every word in it is correct.
Axis Three: The Standard
The standard is the conformance bar the experience must meet to ship, anchored by WCAG 2.2 AA, the Web Content Accessibility Guidelines, level AA, the accessibility conformance target the program holds as a gate. The question is "can every learner, including those using a screen reader, captions, or keyboard-only navigation, actually reach this content and this assessment." Accessibility is not a polish step applied after the content is correct. It is a third axis the screen passes or fails on its own terms, and a beautiful, true, aligned screen that a screen-reader user cannot operate does not ship.
The power of the method is in the word simultaneous. When you hold all three axes on one screen, you can finally see the three errors that single checks are structurally unable to catch: the false-but-aligned screen (a hallucinated fact that happens to fit the objective perfectly), the true-but-misaligned screen (a real fact serving the wrong objective or tested at the wrong level), and the correct-but-inaccessible screen (right content, locked away from part of the workforce). Each is invisible on two of the three axes and lethal on the third.
The Error Each Single Check Misses
Make the abstraction concrete with the three signature failures, each named by which single check waves it through.
| Error type | What it looks like | Which single check passes it | What the three-way check sees |
|---|---|---|---|
| False but aligned | A hallucinated threshold that fits the objective so neatly it looks designed | The alignment check (it serves the objective) and the accessibility check (it reads fine) | No source line exists for the threshold, so the fact is unproven |
| True but misaligned | A correct fact, or a clean item, sitting under the wrong objective or testing the wrong Bloom's level | The fact check (it matches the source) and the accessibility check (it is reachable) | The objective demands application; the item tests recall, so it measures the wrong skill |
| Correct but inaccessible | Right content, right objective, but a color-coded diagram with no text alternative or an uncaptioned narration | The fact check and the alignment check both pass on content | A screen-reader or keyboard-only learner cannot reach the content, so it fails the standard |
| The compound trap | A plausible fact that is false, fits the objective, and is fully accessible | Two of three checks pass; the screen feels finished | Only the side-by-side source line exposes that the well-formatted, accessible, on-objective claim is invented |
Read the third column down the table. Every row passes at least one single check, and the dangerous rows pass two. The two-pass rows are the ones that ship, because two green lights feel like enough and the reviewer moves on. The three-way check refuses to grade on two out of three.
The error you will ship is never the one all three checks catch. It is the one two checks bless and the third would have killed, if the third had ever been in the room at the same time.
Running the Check in Practice
The method only helps if it survives contact with a real build, so make it a procedure, not an aspiration. For each load-bearing screen, open three references at once and ask three questions in one sitting.
- Source line. For every claim, threshold, procedure step, and number on the screen, point to the exact line in the approved source. No line, no ship. A claim that traces to "general knowledge" or to the model is a hallucination candidate, full stop.
- Objective fit and level. Name the objective this screen serves and its Bloom's level. Confirm the content advances that objective and the item measures that skill at that level. A true fact under the wrong objective gets cut or moved; an item at the wrong cognitive level gets rewritten, not kept because it is well phrased.
- Standard pass. Confirm the screen meets WCAG 2.2 AA on its own: text alternatives, captions and transcripts, color not used alone to carry meaning, keyboard operability, sufficient contrast. A screen that fails here is not "mostly done," it is not done.
The order matters less than the simultaneity. The point is that the same human, looking at the same screen, holds the source, the objective, and the standard in view together, so the false-but-aligned claim cannot hide behind alignment and the true-but-misaligned item cannot hide behind being true. When you find an error, you also log which axis it lived on, because the pattern of your catches tells you where your AI workflow is weakest and where to tighten the prompt or the grounding.
Two failure-injection habits sharpen the check. First, distrust the screens that feel most finished; a fluent model produces its most dangerous hallucinations in its most confident prose, and the smoothest threshold is the one most worth tracing to a source line. Second, treat every assessment item as guilty of measuring recall until it proves it measures the named skill, because item drafting is where misalignment hides most comfortably behind clean writing.
A Worked Example: Before and After
A learning team is rebuilding a hazardous-materials handling module for 3,000 warehouse staff. The AI drafts a screen on spill-response thresholds with a matching practice item. Watch the same screen die under single checks and survive under the three-way check.
Before (sequential checks). The fact reviewer reads the screen on Monday. The screen states that a spill above a stated volume requires evacuation and a regulator notification, and the reviewer confirms a real regulation has an evacuation-and-notification rule, so the fact check passes. The objective reviewer reads it on Wednesday. The objective is "apply the spill-response decision rule to a scenario," and the practice item presents a scenario and asks the learner to choose the action, so the alignment check passes. The accessibility reviewer runs the screen on Friday: captions present, contrast fine, keyboard operable, so the standard check passes. Three green lights across three days. The module ships. Six weeks later a real spill happens, a worker follows the screen, and evacuates at a volume far below the real threshold, triggering an unnecessary plant-wide evacuation and a regulator inquiry into why staff were trained to a number that does not appear anywhere in the actual regulation. The threshold was hallucinated. The fact reviewer had confirmed that a rule exists, never that this number was the rule. The objective reviewer had assumed the number was true. The accessibility reviewer transcribed the wrong number into a perfectly compliant caption. Every check did its job. The seam between them shipped the error.
After (three-way check). The same screen, one reviewer, three references open. Source line: she demands the exact line in the loaded regulation that states the threshold, and there is none, the model produced the number from training data, so the claim fails on the source axis before alignment or accessibility ever matter. The screen goes back with the threshold flagged as unsourced. The corrected screen carries the real threshold, traced to the regulation line. Objective fit: she confirms the scenario item tests the decision rule at the application level the objective names, and it does. Standard: she confirms the corrected caption transcribes the real number and the contrast and keyboard paths hold. One sitting, three axes, and the hallucinated threshold never reaches a worker. When the safety lead later asks "where did this threshold come from and who can a screen-reader user reach it," the answer is one breath: here is the regulation line, here is the objective it serves at the application level, here is the conformance result, all checked together before sign-off.
The module did not get slower because the check got stricter. It got safer because the check stopped being three separate reviews with a seam between them and became one review with no seam to hide in.
Building the Check Into an AI Workflow
A reactive three-way check at sign-off is good. A three-way check that also shapes how you prompt the model is better, because the cheapest error to catch is the one the model never makes. The same three axes that catch errors at review can be pushed upstream into the build, so the draft arrives closer to defensible.
On the source axis, push provenance into the generation step. Ground the model on the approved source and instruct it to attach, for every load-bearing claim, the exact line it drew from, or to refuse and flag the claim when no source supports it. A model told to "cite the source line or say you cannot" produces fewer confident hallucinations than a model told only to "write the module," because you have made the absence of a source visible in the draft instead of invisible until audit. The reviewer still confirms every line, but now starts from a draft that already exposes its own gaps.
On the objective axis, lock the objective and its Bloom's level into the prompt before any item is drafted. When the model knows the screen serves "apply the spill-response rule to a scenario" at the application level, it is far less likely to return a recall item, and when it does, the mismatch is obvious against the stated objective. Specifying the objective up front turns alignment from a hunt into a comparison: the screen either matches the named objective and level or it does not.
On the standard axis, build accessibility constraints into the media and interaction generation, not the review. Instruct the tooling to produce captions, text alternatives, and keyboard-operable components from the start, and treat the WCAG 2.2 AA criteria as generation requirements rather than a final inspection. The conformance check then verifies a build that was constructed to pass, rather than discovering at the end that the AI-narrated video shipped without captions.
None of this replaces the simultaneous human check at sign-off. It changes what that check is looking at. Instead of a draft optimized only to look finished, the reviewer holds a draft that already carries source lines, matches a stated objective and level, and was built to the standard, so the three-way pass becomes a confirmation rather than a salvage operation. The errors that survive into that pass are fewer and easier to see, and the ones that matter most, the relational ones, still get caught because a human is still holding all three axes at once.
The Iron Rule and the Three-Way Check
This lesson is the iron rule of the program, made operational at the screen level. AI assists, the human verifies, the human owns the decision, and "the AI wrote it" is never a defense. The three-way check is what "the human verifies" actually means when you do it properly: not a glance, not a single-axis pass, but a simultaneous cross-reference of source, objective, and standard, owned by a named person who signs the result. The accessibility gate is bright-line and lives inside the third axis: an AI-generated screen that fails WCAG 2.2 AA does not ship, no matter how true or how aligned, full stop. And the assessment rule lives inside the second axis: AI may draft the item, but a human confirms it measures the objective at the right level, because AI does not certify a learner as competent.
The reason the three-way check beats three separate checks is not that three reviewers are careless. It is that the most dangerous errors in AI learning output are relational. They live in the space between a fact and its objective, between an objective and its assessment, between content and the learner who has to reach it. A check that looks at one axis cannot see a relationship that spans two. Hold all three at once, and the false-but-aligned claim, the true-but-misaligned item, and the correct-but-inaccessible screen all become visible at exactly the moment you can still stop them.
Key Takeaways
- Sequential single checks have a structural blind spot: a fact check, an alignment check, and an accessibility check each see one axis and miss errors that live in the relationships between axes.
- A hallucination is a confident false statement with no source line; misalignment is true content serving the wrong objective or an item testing the wrong skill, and the two failures hide from different reviewers.
- The three-way check cross-references the source, the objective, and the standard simultaneously on one screen, owned by one named human, before sign-off.
- The three signature errors are false-but-aligned, true-but-misaligned, and correct-but-inaccessible; each passes at least one single check and the dangerous ones pass two.
- The source axis demands an exact line in an approved document for every load-bearing claim; "the model said so" is not provenance and a sourceless claim is a hallucination candidate.
- The objective axis enforces constructive alignment at the right Bloom's level, so a true fact under the wrong objective gets cut and a recall item under an application objective gets rewritten.
- The standard axis holds WCAG 2.2 AA as a gate, not a polish step: a true, aligned screen that a screen-reader or keyboard-only learner cannot operate does not ship.
- The method makes the iron rule operational: the human verifies by holding all three axes at once, because the error you will otherwise ship is the one two checks bless and the third, never in the room, would have killed.
Skill.re