A large academic medical center signs a contract for a sepsis early-warning model. The vendor study is impressive: sensitivity of 0.87, specificity of 0.83, validated across three health systems, published in a respected journal, cleared for its intended use. The governance committee is ready to sign off. Then one nurse informaticist asks a single question that stops the meeting: "Has anyone run this on our patients yet?" No one has. The committee pauses the go-live and orders a ninety-day shadow trial. When the numbers come back, the model that scored 0.87 in the vendor study is firing on the wrong patients, generating three false alarms for every true catch, and lagging the bedside nurse's own judgment by hours. Nobody was harmed, because nobody was ever exposed. That pause, that one question, is the whole discipline of this lesson.

Why Elsewhere Is Not Here

The most dangerous assumption in clinical AI is that a number measured somewhere else describes what will happen to your patients. It does not. A performance metric is not a property of the model alone. It is a property of the model running against a specific population, documented by specific clinicians, in a specific workflow, on a specific day. Change any of those and the number can move, sometimes dramatically, and almost always in the direction you did not want.

Consider what actually differs between a vendor's development site and yours. Disease prevalence differs: the base rate of the condition the model predicts is rarely the same across two hospitals. Patient mix differs: age distribution, comorbidity burden, acuity, insurance status, the referral patterns that decide who even walks through your doors. Documentation habits differ: whether your clinicians chart a symptom in a structured field or bury it in free text changes what the model can see. Lab assays differ: two labs measuring the same analyte can produce values on different scales, so a threshold learned elsewhere lands in the wrong place. EHR configuration differs: how your build maps flowsheet rows, order sets, and result feeds determines whether the model receives clean inputs or garbage. And clinical workflow differs: where the output appears, who sees it, and whether they can act on it at that moment decides whether the model helps or becomes noise.

Each of these is enough on its own to break a borrowed number. Together they guarantee that a model validated elsewhere is, at best, a well-supported guess about your setting. That is not a reason to distrust every tool; it is a reason to insist on proof before exposure.

Two Mechanisms That Break Borrowed Numbers

There are two distinct reasons performance fails to carry over, and confusing them leads committees to accept the wrong evidence. Understanding both is the core of local validation.

The first mechanism is the dependence of predictive value on prevalence. Sensitivity and specificity are, in the idealized case, properties of the test itself: sensitivity is the probability the model fires when the patient truly has the condition, and specificity is the probability the model stays quiet when the patient truly does not. But those are not the numbers a clinician cares about at the bedside. The clinician cares about positive predictive value (PPV): when the model fires, what is the probability the patient actually has the condition? And negative predictive value (NPV): when the model is quiet, what is the probability the patient is actually clear? PPV and NPV are not properties of the test alone. They depend on prevalence. Hold sensitivity and specificity fixed, drop the prevalence, and PPV falls, sometimes off a cliff. Why you care: a triage tool that looked precise in a high-prevalence referral center can become an alarm-fatigue machine in a lower-prevalence community hospital, firing again and again on patients who do not have the disease, even though its "accuracy" on paper is unchanged.

Prevalence, In Plain Numbers

Make it concrete. Take a model with sensitivity 0.90 and specificity 0.90, held fixed. In a population where the true prevalence of the condition is 10 percent, out of 1,000 patients, 100 have the disease and 900 do not. The model correctly flags 90 of the 100 (sensitivity 0.90). It falsely flags 90 of the 900 (specificity 0.90 means a 10 percent false-positive rate). So 180 alerts fire, and 90 are correct: PPV is 50 percent. One in two alerts is real.

Now move the exact same model, unchanged, to a population where prevalence is 1 percent. Out of 1,000 patients, 10 have the disease and 990 do not. The model flags 9 of the 10 true cases and falsely flags 99 of the 990. That is 108 alerts, of which 9 are correct: PPV is roughly 8 percent. More than eleven alerts fire for every real one. The model did not get worse; its sensitivity and specificity are identical. The population changed, and the number the clinician lives with collapsed. This is the single most common reason a "proven" tool drowns a unit in false alarms after go-live.

A tool proven somewhere else is a hypothesis about your patients, not a conclusion. Local validation is how you test the hypothesis before a single patient depends on the answer.

The second mechanism is dataset shift. This is the broader and more insidious failure. A model learns patterns from its training data, and some of those patterns are real clinical signal while others are spurious correlations specific to the training site. A classic example: a model that appears to detect a disease on chest images but has actually learned to recognize the portable scanner marker that a particular hospital placed on films of its sickest, most immobile patients. At that hospital the shortcut worked, because sicker patients were imaged portably. At yours, where the same marker means something else or does not exist, the shortcut is worthless or misleading. Dataset shift also covers slower drift: your population changes over time, a lab switches assays, a documentation template is redesigned, and the inputs the model sees stop matching what it was trained on. Why you care: dataset shift means the vendor's number can be perfectly honest and still not describe your patients, because the model was leaning on something that is not true where you work.

Clearance Is Not a Promise About Your Setting

Regulatory authorization is frequently misread as a safety guarantee. It is not. By early 2026, the FDA had authorized more than 1,350 AI and machine-learning-enabled medical devices, with radiology dominating the list (roughly 74 percent of the 2024 authorizations were radiological). That volume tells you the technology is real and the pathway active. It does not tell you any single tool will work on your patients.

FDA clearance is a regulatory authorization for a defined intended use. It attests that, for the population and task described in the submission, the device met the agency's bar for that pathway. It is scoped to that intended use and to the data the manufacturer submitted. It is not a promise that the tool performs at that level in your workflow, on your population, integrated into your EHR, read by your clinicians. Clearance answers a question about the device as characterized by its maker. Local validation answers the only question your patients care about: does it work here, now, on us?

Read the distinction carefully, because it is where governance committees most often get lulled. A cleared device used slightly outside its intended use, on a population that differs from the submission cohort, or wired into a workflow the manufacturer never contemplated, is operating in territory the clearance never covered. The clearance did not disappear, but it stopped speaking to your situation. That gap is what local validation closes, and it is why the CHAI assurance framework and the Joint Commission responsible-use guidance both name validation on data representative of your own population as a foundational element, not an optional extra.

To be precise about the boundary: clearance does not say the pivotal-study sensitivity and specificity will reproduce on your prevalence and patient mix, that the model will not lean on a spurious correlation absent from your data, that the output will land where your clinicians can act on it, or that the tool will keep performing as your population, labs, and templates drift. Every one of those questions is yours to answer locally, and treating clearance as if it answered them is how a compliant purchase becomes an unsafe deployment.

The Silent Trial: Proving It Before It Touches Care

The remedy is local validation, and its sharpest instrument is the silent trial, also called shadow mode. In a silent trial you turn the model on against your live data, but you do not let it influence care. The model generates its outputs, every one of them is logged, and each is compared against two references: ground truth (what actually turned out to be true for that patient) and current clinician judgment (what your clinicians decided without the model). The clinicians never see the model's output during the trial. No order changes, no alert fires to the bedside, no triage decision shifts. The tool is running, watching, and being measured, but it is inert with respect to the patient. Why you care: shadow mode is the only way to measure real performance on your patients across your subgroups before one of them is exposed to the tool's mistakes.

The silent trial does what no vendor study can: it produces performance numbers computed on your own patients, in your own workflow, using your own ground truth. Those are the numbers you make the go-live decision on. The vendor's numbers become what they always were, a hypothesis; your shadow-mode numbers become the evidence.

Define Ground Truth Before You Measure Anything

Ground truth is the reference standard you score the model against: the definitive answer to "did this patient actually have the condition or outcome the model is predicting?" It sounds obvious, and it is the step teams botch most often. If your ground truth is sloppy, every downstream number is meaningless, because you cannot tell a true positive from a false one. Ground truth must be defined in writing before the trial begins: what counts as a true case, over what time window, adjudicated by whom, using what source of record. For a sepsis model, does ground truth mean a clinician diagnosis, a billing code, a lab-and-vitals rule, or expert chart review? Each yields a different denominator and a different performance number. Pick the definition that matches the decision the tool is meant to support, write it down, and hold it fixed. Why you care: two teams can validate the same model and reach opposite conclusions purely because they defined ground truth differently.

Choose a Representative Sample

The patients in your silent trial must look like the patients the tool will actually run on. If you validate only on the day-shift medical floor but deploy hospital-wide, your numbers do not cover the ICU, the emergency department, or the night shift where staffing and documentation differ. Representativeness has a time dimension too: a sample drawn from one calm month may miss the seasonal surge when the tool matters most. Size the sample so you have enough true cases to estimate PPV and sensitivity with meaningful confidence, not just enough total patients. In a low-prevalence setting that can mean running the trial longer to accumulate enough positive cases. Why you care: a validation that omits a subgroup is not evidence about that subgroup, and deploying anyway is deploying on faith.

Pick Metrics That Match the Clinical Decision

Different clinical decisions demand different metrics. A screening tool meant to miss almost no true cases lives and dies by sensitivity and NPV: you care most about not letting a real case slip through. A tool that triggers an expensive or invasive workup lives and dies by PPV and specificity: you care most about not sending healthy patients down a harmful path. A single headline "accuracy" figure hides both concerns and should never be the basis of a go-live decision. Choose the two or three metrics that map to the harm you are trying to prevent, and evaluate the model on those. Why you care: optimizing or accepting the wrong metric means you can pass validation on paper and still cause harm in the exact way that matters for this tool.

Set the Pass/Fail Threshold Before You Look

This is a discipline, not a calculation. Decide, in writing, before you see any results, what performance the tool must hit to go live. What is the minimum PPV your unit will tolerate before alarm fatigue makes the tool a net negative? What sensitivity is non-negotiable given the harm of a miss? Set those numbers up front. If you wait until the data is in front of you, human nature will negotiate the threshold down to whatever the tool happened to score, and you will have validated nothing. Why you care: pre-registering your pass/fail bar is the only thing that makes a "pass" mean something, because it removes the temptation to move the goalposts to fit the result.

Compare Against the Standard of Care

The right question is never "is the model good?" It is "is the model better than what my clinicians already do?" Your current practice, the standard of care, is the real baseline. If your experienced bedside nurses already identify deteriorating patients earlier and more accurately than the model, deploying it adds alerts, cognitive load, and risk while subtracting nothing. The silent trial gives you exactly this comparison, because you logged clinician judgment alongside the model's output. Score them head to head. A tool that merely matches current practice while adding alert burden is a net loss; a tool that meaningfully beats it on the metric that matters is worth the integration cost. Why you care: the standard of care is the baseline a tool must clear to justify its existence, and skipping this comparison is how hospitals deploy tools that make care worse while looking modern.

Test the Workflow, Not Just the Model

A model can be statistically excellent and still fail because the surrounding workflow is broken. Validation must test the sociotechnical system, not only the algorithm. Where does the output land: an inbox no one checks, a banner that blends into thirty others, a page that reaches the right person at the right moment? Can the clinician act on it when it arrives, or does it fire during a code when no one can respond? What is the alert volume, and does it push the unit past the point where clinicians start ignoring alerts wholesale (alert fatigue)? A model with a beautiful PPV that fires into a workflow no one can use is worth nothing. Why you care: patients are harmed by the whole system, not the model in isolation, so the whole system is what you validate.

Worked Example: The Sepsis Model Caught in Shadow Mode

Return to the sepsis early-warning model from the opening. The vendor study reported sensitivity 0.87 and specificity 0.83, developed and validated across three health systems. On paper it looked strong, and it was FDA-cleared for its intended use. The governance committee, having learned this discipline, refused to go live on the vendor's numbers and ordered a ninety-day silent trial on the medical-surgical floors and the step-down unit.

First they defined ground truth in writing: a sepsis case was adjudicated by expert chart review against a fixed clinical definition within a set time window, not by billing codes, which they knew would inflate and distort the denominator. They set their thresholds before looking: the tool had to reach a PPV of at least 0.25 on their floors (they knew low PPV meant unmanageable false alarms) and had to fire earlier than the bedside nurse's documented concern in a majority of true cases to justify its existence. And they logged clinician judgment alongside every model output, so they could compare the tool to the standard of care.

The shadow-mode results told a story the vendor study could not. The prevalence of sepsis on these floors was materially lower than in the vendor's development cohort, which was weighted toward higher-acuity sites. With sensitivity and specificity roughly holding, the lower prevalence drove PPV down to about 0.14: fewer than one in seven alerts corresponded to a true septic patient. Worse, when the team examined timing, they found the model was firing after the bedside nurse had already documented concern in a majority of true cases. The nurses, in this setting, were the earlier and more precise detector. The tool was not beating the standard of care. It was trailing it while threatening to bury the unit in false alarms.

A subgroup analysis sharpened the picture. The timing lag was worst on the step-down unit, where documentation habits differed from the floors the vendor had emphasized, a textbook signature of dataset shift: the model had learned patterns that did not transfer to how this unit charted. Had the committee deployed on the vendor's numbers, they would have exposed their most fragile patients to a tool both noisier and slower than existing practice, and alert fatigue would likely have eroded response to the real alerts that did fire.

Because the tool ran in shadow mode, none of this reached a patient. The committee did not simply reject the tool: they fed the findings back to the vendor, requested a recalibration to local prevalence and a retraining that accounted for their documentation patterns, and defined a second silent trial with the same pre-registered thresholds. The value of the exercise was not that the tool failed. It was that the failure was measured on logged data instead of on patients, and the go-live decision rested on evidence from this hospital rather than a hypothesis borrowed from three others.

The Same Lesson in Imaging Triage

The identical pattern shows up in an imaging triage tool designed to prioritize suspected intracranial hemorrhage on the worklist. In the vendor study it flagged bleeds with excellent sensitivity. In a local silent trial, running behind the radiologists without changing the reading order, the team found the tool's apparent skill leaned partly on a scanner and protocol signature specific to the development sites. On the local scanner fleet, sensitivity for subtle bleeds dropped, and the tool's reprioritization would have pushed some genuinely urgent studies down rather than up. Same mechanism, different modality: a borrowed number, a spurious correlation, and a silent trial that caught it before the worklist ever changed. Performance is a property of the model plus your population plus your workflow, and only local validation measures the combination you run.

Operationalizing Local Validation in Governance

None of this works as a one-off heroic effort. It works as a standing requirement your governance committee attaches to every consequential tool. Make local validation a gate, not a suggestion. A tool that influences diagnosis, triage, dosing, or discharge does not go live until it has passed a silent trial with pre-registered thresholds on a representative local sample, scored against a written ground truth definition, compared head to head with the standard of care, and evaluated as a workflow rather than an algorithm in isolation.

Tie the requirement to the frameworks your accreditors already name. The CHAI assurance standards and the Joint Commission responsible-use guidance both call for validation on data representative of the population the tool will serve. Local validation satisfies that element in practice, and shadow mode does it without gambling with patients. Document the whole chain: the ground truth definition, the sample and why it is representative, the metrics and why they match the decision, the pre-registered thresholds, the shadow-mode results, the standard-of-care comparison, and the go-live or hold decision with the name of who made it. When an auditor asks how you know the tool is safe for your patients, that documented chain is your answer. "It was FDA-cleared" is not.

Finally, validation is not a one-time stamp. Populations drift, labs change assays, templates get redesigned, and the model that passed last year can quietly decay. Local validation before deployment is the entry gate. Ongoing monitoring, which the next stage of governance owns, keeps it honest after go-live. The discipline is the same throughout: never let a borrowed or aging number stand in for a measured one on the patients in front of you.

Key Takeaways

• A performance metric is a property of the model plus your population plus your workflow, not of the model alone. A number measured elsewhere is a hypothesis about your patients, never a conclusion, and must be proven locally before anyone is exposed.
• Predictive value depends on prevalence. Holding sensitivity and specificity fixed, a lower disease prevalence drives PPV down sharply, which is why a tool that looked precise in a high-prevalence site can become an alarm-fatigue machine in a lower-prevalence one.
• Dataset shift means a vendor's honest number can still fail on you, because the model may have learned spurious correlations (a scanner marker, a documentation habit) that do not exist or mean something different where you work.
• FDA clearance authorizes a defined intended use on the manufacturer's submitted evidence. With over 1,350 devices cleared by early 2026, clearance proves a regulatory bar was met, not that the tool works on your population, in your EHR, in your workflow.
• The silent trial (shadow mode) runs the model on live data with every output logged and compared to ground truth and to clinician judgment, while it influences no care. It is the only way to measure real local performance across your subgroups before a patient is exposed.
• Define ground truth in writing, choose a representative sample, pick metrics that match the clinical decision, and set the pass/fail threshold before you look. A threshold set after seeing the data is a rationalization, not a bar.
• Always compare the tool against the standard of care. If your clinicians already detect the condition earlier and more accurately, the tool adds alert burden and risk while subtracting nothing, and it is a net loss no matter how modern it looks.
• Validate the workflow, not just the model. Where the output lands, whether clinicians can act on it, and the alert volume decide whether a statistically excellent model helps patients or drowns the unit in noise. Make local validation a standing governance gate, documented end to end for auditors.