It is the Tuesday morning production meeting and the plant manager has one slide up. A vision system that has been grading welds on Line 3 for six weeks rejected 1,400 parts overnight, the night shift could not tell whether the rejects were real, and the line ran at half rate until 4 a.m. while a green operator hand-checked a pallet by flashlight. The system vendor swears the model is "performing within spec." The customer who buys those welded assemblies has an audit scheduled for Thursday. Somebody on the leadership team finally asks the question that should have been asked before the camera was ever bolted to the fixture: "What exactly can go wrong with this thing, how bad does it get, and what did we put in place to stop it?" Nobody has a document that answers that. That missing document is a risk assessment, and this lesson is about how to build one that turns "it should be fine" into a decision you can defend to a customer, an auditor, and the operator on Line 3 at 2 a.m.

Why "It Should Be Fine" Is Not a Decision

Every piece of AI that touches your floor is a bet. The bet is that the value it creates, fewer escapes, less unplanned downtime, a green crew that can do the work of a retiring expert, is larger than the harm it can cause when it is wrong. The problem is that most plants place that bet without ever writing down the downside. They run a flashy proof of concept, the demo works, somebody signs a purchase order, and the only record of the risk is a feeling in the room that "it should be fine." When the vision system rejects 1,400 good parts at 2 a.m., or the predictive maintenance model misses the one bearing that actually fails, "it should be fine" is exposed for what it always was: not a decision, just an absence of one.

A risk assessment is the document that replaces that feeling with a structured argument. It does three jobs. First, it forces you to list every realistic way the AI can fail, not the way the vendor demo failed, the way a brownfield plant on a hot afternoon with a thin crew actually fails. Second, it scores each failure so you can tell the difference between a nuisance and a catastrophe. Third, it pins a specific safeguard and a specific owner to each failure that scores high enough to matter. When you are done, you do not have a guarantee that nothing will go wrong. You have something better: a defensible record that you understood what could go wrong, you decided which risks were tolerable, and you put controls in place for the ones that were not.

This matters more in manufacturing than in almost any other field because of the cardinal rule of this whole program: the customer audits you, not the vendor. When a defect escapes to a customer because a vision model missed it, the containment, the corrective action, and the 8D (the eight-disciplines structured problem-solving report your customer requires) all land on your desk. "The model flagged it as good" is not an answer an IATF 16949 auditor accepts, and IATF 16949 is the automotive quality management standard most of your customers hold you to. The risk assessment is the evidence that you, the plant, took ownership of the AI's failure modes before they became the customer's problem.

A risk assessment does not promise nothing will go wrong. It proves you decided, on purpose, which risks you would accept and which you would control.

The FMEA You Already Know, Pointed at AI

Here is the good news, and it is genuinely good news for a plant team that is tired of consultants inventing new frameworks: you already own the right tool. It is the FMEA, the Failure Modes and Effects Analysis, the same structured worksheet your quality team uses to analyze a new process or part. An FMEA walks through every step, asks how that step can fail, asks what happens when it does, and scores the result. You are going to point that exact discipline at the AI on the line. You are not learning a new method. You are applying a method you already trust to a new kind of component.

Think of the AI system as one more station in your process flow, except this station can fail in ways a press or a torque gun cannot. A torque gun that fails usually fails loud and obvious: it stops, it throws a fault, the operator knows. An AI model fails quiet. It keeps producing confident output that looks exactly like correct output, while being wrong. That quiet failure mode is the whole reason AI needs its own risk assessment rather than a line on the existing process FMEA. The classic FMEA scoring still applies, but the failure modes are different in character, so you walk them deliberately.

The FMEA scores three things for each failure mode, each on a scale of one to ten, and multiplies them into a single number called the RPN, the Risk Priority Number. Severity asks how bad the harm is when this failure reaches the end of the line: a one is a minor annoyance, a ten is a safety event or a defect escape that injures a customer or stops their plant. Occurrence asks how often this failure is likely to happen: a one is almost never, a ten is routine. Detection asks how likely you are to catch the failure before it does harm, and here is the trap that catches every team the first time: a low number is good and a high number is bad. A detection score of one means you will almost certainly catch it; a ten means it will sail straight through to the customer undetected. Multiply the three and you get an RPN between 1 and 1000. The higher the number, the louder the risk is shouting at you.

Worked example. Take the vision system on Line 3. One failure mode is "model passes a real weld defect as good," a classic escape. Severity is high, because a bad weld in a structural assembly can hurt someone downstream, call it a 9. Occurrence on a drifting brownfield camera is realistically a 4, it does not happen every shift but it happens. Detection is the killer: if the only check is the model itself, and the model is the thing that failed, your detection is terrible, call it an 8, because nobody downstream is re-inspecting welds the camera "passed." That is 9 times 4 times 8, an RPN of 288. Now add one safeguard, a daily audit where an operator pulls ten "passed" parts and re-inspects them against the standard. Detection drops from 8 to 3, because you now have an independent net under the model. The RPN falls from 288 to 108. You did not change how often the model fails. You changed how long a failure can hide, and that single move cut the risk by almost two thirds.

The Failure Modes That Actually Bite on the Floor

A risk assessment is only as good as the failure list it starts from. The vendor will give you a clean list that does not include the messy ways their model breaks in your plant. Here is the floor-honest catalog, the failures that actually bite, grouped by the kind of AI you are deploying. Walk all of them for any system, because the worst surprises come from the failure mode nobody put on the list.

Vision and quality models

The escape. The model passes a real defect as good. This is the highest-severity failure in quality AI because it ends at the customer. Score severity high and detection honestly: if nothing re-checks the passed parts, detection is poor.

The false reject. The model rejects good parts. Operators experience this as the system "crying wolf," and it carries a cost most teams never quantify. Consider a line running 2,000 parts a shift at a 3 percent false-reject rate. That is 60 good parts a shift thrown into scrap or sent to a rework loop that costs labor. At a fully loaded 25 dollars per part of scrap and handling, that is 1,500 dollars a shift, roughly 1.1 million dollars a year, spent rejecting parts that were fine. False rejects do not show up as a quality escape, so they hide in the scrap bucket, and they quietly cost more than the escapes the system was bought to prevent. Worse, the human cost is real: an operator who gets burned by false alarms will tape over the reject light, and now your expensive safeguard is off.

Drift. The lighting changes between day and night shift, the camera angle creeps after a fixture bump, the incoming material gets a new surface finish from a different supplier lot, and the model that was 98 percent accurate in the demo silently degrades. Drift is an occurrence-and-detection problem: it will happen, and without a monitoring plan you will not see it until escapes or false rejects spike.

Predictive and prescriptive maintenance models

The miss. The model fails to flag the bearing, motor, or pump that is actually trending to failure, and the line goes down on a hot afternoon anyway. PdM (Predictive Maintenance, using sensor and historian data to forecast equipment failure) is sold on its catches, but its misses are the failure mode that costs you, because the whole investment was justified by avoided downtime.

Alert fatigue. The model cries wolf so often that the maintenance crew, already short three techs, stops opening the alerts. A PdM system that generates 40 alerts a week where 35 are noise trains the crew to ignore all 40, including the five real ones. This is a detection failure dressed up as a working system.

The unsafe action. A prescriptive model recommends an action that would be wrong or dangerous if a tech executed it without thinking, like derating a machine in a way that violates the process window, or skipping a PM (Preventive Maintenance task) the model decided was unnecessary.

Generative and knowledge models

The hallucinated spec. An AI drafting a work instruction or troubleshooting a fault invents a torque value, a temperature, or a procedure that sounds authoritative and is simply wrong. This is the single most dangerous generative failure on a floor, because a green operator has no way to know the 35 newton-meter spec the AI confidently stated should have been 25, and the assembly fails in the field.

The confidently wrong root cause. An AI assembling a fishbone or 5-Whys names a plausible root cause that the data does not actually support, and the team chases it, closes the 8D, and the defect comes back next month because the real cause was never touched.

The enshrined myth. A knowledge-capture system records a retiring expert's habit as fact when it was actually superstition, and now the AI teaches every new hire to "warm the machine for twenty minutes" for a reason that was never real, wasting time forever.

Cross-cutting failures that apply to all of them

The OT exposure. The AI is connected across the OT/IT boundary (OT is Operational Technology, the control-system network running your PLCs and machines; IT is your business network), and because 78 percent of OT networks lack centralized monitoring, you cannot fully see what that connection does. An AI that can write to a PLC (Programmable Logic Controller, the industrial computer that directly controls a machine) is a controls and a security risk at once.

The audit gap. The AI touches a quality decision but does not log what it did, so when the customer audits, you cannot reconstruct why a part was passed or a disposition was made. The decision happened; the evidence did not.

Scoring Honestly: The Three Numbers That Decide

The hard part of an FMEA is not the worksheet, it is the honesty. Teams systematically score their own systems too kindly, and the AI vendor in the room will gently encourage every optimistic number. Three disciplines keep the scoring honest enough to be useful.

Score detection against your actual safeguards, not your intentions. The single most common mistake is giving a model a good detection score because "an operator is supposed to double-check." If the double-check is not a written, owned, audited step that happens every shift, it does not exist, and your detection score should reflect a world where the model is the only thing standing between a defect and the customer. Be brutal here. Detection is where most of the real risk lives, and it is also the one number you can most directly improve with a safeguard, which is why the discipline pays off.

Score occurrence for your plant, not the demo. Vendor accuracy numbers are benchmarks measured on clean data in good lighting with the camera freshly calibrated. Your plant has drift, a hot afternoon, a wet part, a new material lot, and a fixture that gets bumped. Treat every vendor performance figure as a claim to verify, never a guarantee, and score occurrence based on how the system behaves in a real holdout test on your line, not on the slide. A model that scored 99 percent in the demo and 94 percent on your holdout has a real-world occurrence of misses that is six times higher than the slide suggested.

Score severity from the customer's seat, not yours. A false reject feels minor from the floor because no bad part shipped. From the plant's P&L it can be a million-dollar-a-year leak. An escape feels like one bad part from the floor; from the customer's seat it can be a line-down event at their plant and a containment that costs more than your whole AI program. Score severity by walking the failure all the way to where it actually lands, including the false-reject economics that hide in the scrap bucket and never reach the quality report.

Once every failure mode is scored, sort by RPN and draw a line. A common practice is to treat any RPN above 100, or any failure with a severity of 9 or 10 regardless of RPN, as requiring a documented safeguard before go-live. The severity override matters: a failure that can hurt someone gets a control even if it is rare and you think you would catch it, because the cost of being wrong about a safety event is not symmetric. You can be wrong about a nuisance. You cannot be wrong about a person.

Safeguards That Bring the Number Down

A risk assessment that lists failures and scores them but stops there is a complaint, not a control plan. The payoff is the safeguard column, where each high-RPN failure gets a specific, owned, testable control that drops one of the three numbers. You cannot easily change severity, the harm is the harm, so safeguards almost always work by reducing occurrence (make the failure happen less) or, more often and more cheaply, improving detection (catch the failure faster). Here is the toolkit, mapped to the failures above.

The independent re-check. The highest-leverage safeguard for any quality escape. Pull a sample of parts the model passed and re-inspect them against the standard, on a schedule, by a named person, logged. This is the safeguard that took the Line 3 RPN from 288 to 108. It works because it puts a net under the model that does not depend on the model being right.

The human sign-off gate. For any AI output that touches a customer-facing decision, an 8D root cause, a disposition, a work instruction, require a named human to verify and sign before it is released. This is the cardinal rule made operational: the model drafts, the human owns. It drops detection on hallucinated specs and confidently wrong root causes from terrible to good, because a qualified person checks the number against the drawing before it ships.

The drift monitor. For vision and predictive models, a scheduled check of model performance against a known-answer set, plus tracking of false-reject and escape rates over time. When the false-reject rate moves from 2 percent to 5 percent, the monitor catches drift weeks before the scrap bucket or the customer does. Drift monitoring converts an invisible occurrence problem into a visible, owned detection step.

Alert tuning and tiering. For PdM, set thresholds so the crew gets five real alerts a week, not forty, and tier them so a "check at next PM" alert looks different from a "this bearing fails this week" alert. This is the safeguard against alert fatigue, and it protects the detection value of the whole system, because a system the crew ignores has a detection score of zero no matter how good the model is.

The advisory boundary. Keep AI out of direct control of anything that moves. The model recommends; a human or a properly governed control system acts. This is the hard constraint that keeps an unsafe prescriptive action from executing automatically, and it is the right default in a brownfield plant where you cannot fully see the OT network. AI goes advisory first; direct control is a separate, much higher bar that a safety-critical loop rarely clears.

The audit log. Every AI-touched quality decision writes a record: what the model saw, what it concluded, who reviewed it, what they decided. This safeguard does not prevent a failure; it makes every failure reconstructable, which is what closes the audit gap and lets you pass the Thursday audit with AI in the loop.

Notice the pattern. Almost every safeguard is a detection improvement, and almost every one is cheap relative to the failure it prevents. The independent re-check costs an operator twenty minutes a shift. The escape it prevents can cost a six-figure containment. That asymmetry is the entire economic argument for doing the risk assessment at all: a small, owned, recurring control standing in front of a rare but catastrophic loss.

The Living Document and Who Owns It

The last failure mode of risk assessment itself is treating it as a one-time gate. Teams build a beautiful FMEA to clear the go-live meeting, file it, and never open it again. Then the camera angle drifts, a new material lot arrives, the model gets retrained by the vendor, and the assessment that was true in June is fiction by September. AI risk is not static, because the AI is not static and neither is your plant. The assessment has to live.

Living means three things. First, a review trigger. The risk assessment gets revisited on a schedule, quarterly is common, and on every change event: a model retrain, a new product on the line, a process change, a supplier change, or any incident where the AI was involved in a near-miss or an escape. The change is what invalidates the old scores, so the change is what triggers the re-score. Second, real owners. Each safeguard names a person, not a department, who is accountable for it running and for the evidence that it ran. "Quality will handle it" is how the daily re-check quietly stops happening in week three. A name on the line, with the daily log as proof, is what keeps it alive. Third, a link to incident response. When an AI-related quality event happens, the runbook should feed straight back into the assessment: this failure mode occurred, here is what its real occurrence and detection turned out to be, here is the safeguard we are adding or strengthening. The incident is data, and the assessment is where that data updates the bet.

Who owns the whole document? In a mature plant it sits with the AI governance forum, the standing group where quality, maintenance, OT security, and EHS (Environment, Health, and Safety) all have a seat, because a single AI system can fail across all four domains at once. The vision model is a quality risk, an OT risk if it is networked to the line, and an EHS risk if a missed defect can hurt someone. No single function can see all of that, which is exactly why "it should be fine" survives so long in plants without a forum: nobody whose job it is to ask the hard question is in the room. The risk assessment is the artifact that forum produces and maintains, and it is the thing the plant manager should have had on the slide that Tuesday morning. With it, the answer to "what can go wrong, how bad, and what did we do about it" is a document you hand across the table. Without it, the answer is a feeling, and feelings do not survive a customer audit.

Key Takeaways

• A risk assessment turns "it should be fine" into a defensible decision: it lists every realistic failure mode, scores each one, and pins a specific safeguard and owner to the ones that matter. It does not promise nothing goes wrong; it proves you decided which risks to accept and which to control.
• Use the FMEA you already own. Score Severity, Occurrence, and Detection one to ten, multiply for an RPN between 1 and 1000, and remember the trap: for Detection, a low number is good and a high number means the failure sails through to the customer.
• AI fails quiet. Unlike a torque gun that throws a fault, an AI model keeps producing confident output that looks correct while being wrong, which is exactly why it needs its own assessment rather than a single line on the process FMEA.
• Score honestly: detection against your actual written safeguards not your intentions, occurrence against a real holdout on your line not the vendor demo, and severity from the customer's seat including the false-reject economics that hide in the scrap bucket (a 3 percent false-reject rate on a 2,000-part shift can leak over a million dollars a year).
• The Line 3 example: adding one daily independent re-check dropped detection from 8 to 3 and cut the escape RPN from 288 to 108, without changing how often the model fails. Most safeguards work by improving detection, and they are cheap relative to the catastrophic loss they stand in front of.
• Apply a clear line: any RPN above 100, and any failure with severity 9 or 10 regardless of RPN, needs a documented safeguard before go-live. The severity override exists because you can be wrong about a nuisance but not about a person.
• Keep AI advisory and out of direct control of anything that moves, especially in a brownfield plant where 78 percent of OT networks lack centralized monitoring, and log every AI-touched quality decision so the customer audit is something you pass, not something you fear.
• The assessment must live: re-score on a schedule and on every change event (retrain, new product, process or supplier change, incident), name a real person for every safeguard, and let the AI governance forum of quality, maintenance, OT, and EHS own it together, because one AI system can fail across all four domains at once.