A plant manager came back from a competitor's "smart factory" tour fired up, and three weeks later a vision system was bolted onto the final inspection station on line two. The vendor demo had been flawless. On the floor it was a disaster within a month. The part was wet from the prior wash step, so reflections fooled the camera. The lighting changed between day and night shift, so the model that worked at noon failed at midnight. The process itself was not stable; the upstream tooling was mid-qualification and the parts kept changing, so there was no fixed target to learn. And there were almost no labeled defect images, because the plant had never photographed its defects, it had Dave's eyes instead. The system threw false rejects all day, the operators learned to ignore the green light, and within six weeks somebody quietly disabled it. The money was spent, the trust was burned, and the plant was worse off than before, because now the operators distrusted the next AI project too. None of this was an AI failure. It was a deployment-decision failure. Every one of those problems was visible before a dollar was spent. This lesson is the written kill-criteria list: the red flags that mean the honest answer is not no forever, but not yet.

"Not Yet" Is a Real Answer, and Often the Right One

The pressure to deploy AI right now is intense and it is mostly social, not technical. A VP toured a competitor and wants an AI plan. A vendor has a slick demo. The board asked "what are we doing about AI" in a way that made saying "nothing yet" feel like a career risk. In that environment, the most valuable skill an AI-aware manufacturer can have is the discipline to look at a specific line and say, with reasons, "not on this line, not yet, and here is exactly what would have to be true first." That is not Luddism. It is the same judgment a good engineer applies to any capital decision. You would not install a machine on a foundation that cannot hold it. AI is no different; it has prerequisites, and deploying it where the prerequisites are absent does not get you the benefit early, it gets you the line-two disaster and a burned crew.

The cost of a bad deployment is not just the wasted spend. It is the trust. Recall from the quality lessons that an operator who has been burned by a false alarm will disable the green light, and once a crew learns to ignore an AI system, you have lost not only that system but the cultural runway for the next one. With 47 percent of manufacturers already using AI in quality, up from 33 percent the year before, the question on most floors is no longer whether to use AI but where and when, and the where-and-when is precisely where plants destroy value by deploying into conditions that guarantee failure. A clean no on the wrong line protects the budget, the schedule, and the trust you will need when you deploy on the right line.

"Not yet" with a written list of what must be true first is not resistance to AI. It is the judgment that separates a deployment that ships value from one that burns trust.

So treat the rest of this lesson as a pre-deployment checklist. Each section is a red flag. Any one of them, present and unaddressed, is reason enough to hold. Several of them together mean the line is not a candidate at all yet. The goal is not to find reasons to avoid AI. It is to spend your scarce AI budget where it will actually return, and to be able to defend that choice to a VP who wants to deploy everywhere at once.

Red Flag One: No Labeled Data, No Failure History

This is the most common killer and the easiest to miss in a demo. A supervised AI model, the kind that grades parts or predicts a failure, learns from labeled examples. A defect-detection vision model needs images of good parts and images of each defect type, labeled, in enough quantity and variety that the model can learn the difference. A predictive-maintenance model needs failure history, runs to failure with the sensor data leading up to them, so it can learn what trending toward failure looks like. If you do not have those labeled examples, the model has nothing to learn from, and no amount of vendor cleverness conjures them out of nothing.

The brutal irony is that the plants most desperate for AI quality help are often the ones with the least labeled data, because their quality has always run on human judgment that was never recorded. The line-two plant had never photographed a defect in its history; it had Dave. So when the vendor said "our model just needs your defect images," the honest answer was "we do not have any," and that should have stopped the project right there, or redirected it. A vision system for a defect type you have seen five times in two years cannot be trained well, because five examples is not a dataset.

What "not yet" looks like here is concrete. Before you can deploy, you have to build the labeled set: start photographing and labeling defects now, or mine and label your historical records, or run a structured capture of the expert's judgment so it becomes labels. That is real work and real time, often months. It is also exactly the knowledge-capture move that pays off across every AI project. The point is that the data work comes first. Deploying a model on a dataset that does not exist is deploying nothing; it is buying a confident guesser. Ask any vendor the blunt question: how many labeled examples of each defect or failure does this need, and do we have them? If the answer is no, the answer is not yet.

Red Flag Two: An Unstable or Changing Process

An AI model learns a snapshot of the world and assumes the world holds still. The technical name for the failure when it does not is drift: the process changes, and the model that was trained on the old process degrades silently because it is still being measured against a target that has moved. On line two, three sources of instability were all present at once. The part was wet, so its appearance varied shot to shot. The lighting changed between shifts. And the upstream tooling was still being qualified, so the parts themselves were a moving target. A model cannot learn a stable signal from an unstable process, because there is no stable signal to learn.

This is the difference between the vendor's demo and your floor that nobody warns you about. The demo runs on a fixed, clean, well-lit, dry, stable sample. Your floor has lighting that changes, cameras whose angle drifts when somebody bumps the fixture, materials that vary by lot, and processes mid-change. A vision model precise at noon under demo conditions can fall apart at midnight under real ones, and a predictive model trained before a process change can start crying wolf after it. If the process is not yet in statistical control, if you cannot run a stable, repeatable result through it, you do not yet have a foundation an AI model can stand on.

The discipline here is older than AI and every quality engineer already owns it: stabilize the process first. Get the upstream tooling qualified. Fix the lighting so it is consistent across shifts. Lock the camera fixture. Get the wash-and-dry step under control so the part presents consistently. In Lean and Six Sigma terms, get the process into control before you try to automate the judgment on top of it. A model deployed on an unstable process will generate false rejects, the operators will lose trust, and you will be back to the line-two outcome. Not yet means: stable process first, model second.

Red Flag Three: A Safety-Critical Control Loop

This red flag is different from the others. The first two are about whether the model can work. This one is about where the model is allowed to act even if it works perfectly. The rule is a hard constraint: AI does not go directly into a safety-critical control loop first, no matter how good it looks. A safety-critical control loop is any place where the system's action can hurt a person or cause a catastrophic failure if it is wrong: a press that closes, a robot that swings, a furnace temperature, a pressure relief, an interlock that protects an operator's hands. Those loops are governed by functional-safety engineering for a reason, and "the AI was confident" is not a substitute for a safety rating and a fail-safe design.

The governing principle from across this program is that AI stays advisory and out of direct control of anything that moves, unless it is properly governed to a far higher standard than a typical model deployment. Advisory means the AI surfaces a recommendation and a human or a properly rated safety system decides and acts. The reason this matters so much in manufacturing is the OT boundary. OT (Operational Technology, the controllers and networks that physically run the machines, distinct from IT, the business computing side) is where the moving, dangerous equipment lives, and 78 percent of OT networks lack centralized monitoring. You cannot safely give control authority to a model on a network you cannot even fully see. If you cannot monitor it, you cannot govern it, and if you cannot govern it, it does not get control of the press.

So the kill criterion is plain: if a proposed AI deployment would put the model in direct command of a safety-critical actuator, the answer is no, not "not yet," for the first deployment. The right path is advisory first, with the human and the existing safety systems firmly in the loop, and a long, governed, audited road before anything closer to control is even discussed. A model that advises an operator to inspect a part is in a completely different risk class from a model that decides whether the press is safe to cycle. Keep them separate, and keep AI out of the second class until the governance is unarguable.

The Other Red Flags: Trust, Visibility, Stakes, and the Manual Test

Beyond the big three, a working checklist needs a few more flags that sink real deployments.

An OT network you cannot see. Tied to the safety point but broader: if you cannot monitor the network and the data flows where the AI would live, you cannot detect when it drifts, fails, or is tampered with. The brownfield reality is a 1990s PLC and a historian nobody has queried in years. You cannot bolt AI onto a plant you cannot see and call it governed. Get enough visibility to monitor the system before you trust it with anything.

A crew that will not trust it, with no plan to earn trust. The line-two failure was as much social as technical. False rejects burned the operators, and a burned operator disables the green light. If you have no plan for the false-reject economics, no agreed false-reject budget, and no plan to bring the operators in as partners rather than have a system dropped on them, you are not ready. Trust is a design requirement, not a rollout afterthought. Deploying into a hostile or uninformed crew is a kill criterion even when the model is good.

The stakes are too high for the maturity. Match the risk of the application to the maturity of the deployment. A first AI deployment that grades the single most safety-critical or customer-critical characteristic, the one whose escape is a recall, is the wrong place to learn. Start where a mistake is recoverable and visible, then earn your way to higher stakes. If the only candidate is the highest-stakes characteristic and you have no track record, that is a not-yet.

The false-reject economics do not close. A vision system's false-reject rate can quietly cost more than the escapes it catches. Precision and recall are dollars, not abstractions. If the projected false-reject rate, multiplied by the cost of sorting, re-inspecting, and scrapping good parts, exceeds the value of the escapes prevented, the deployment loses money even when it "works." Run that math before you sign, not after.

The manual test, the cleanest filter of all. Ask one question: can a knowledgeable human do this task reliably with the data available? If a human expert cannot tell good from bad from the same images the camera will see, or cannot predict the failure from the same signals the model will get, then the information is not in the data, and the AI cannot conjure it either. AI is a judgment multiplier, not a magic information source. If the answer to the manual test is no, the data is the problem, and no model fixes a data problem. If the answer is yes but only your retiring expert can do it, then your real first project is capturing that expert, not deploying the camera.

How to Write the Kill List and Use It as Leverage

A kill-criteria list is only useful if it is written down before the demo, because the demo is engineered to make you forget every item on it. Put it on one page and run every proposed deployment through it in the same meeting where the VP is excited. The list, in plain floor language, is this. Do we have enough labeled examples of every defect or failure this must catch? Is the process stable and in control, with consistent lighting, fixturing, and material? Would this put AI in direct control of anything safety-critical? Can we actually monitor the OT environment it lives in? Do we have a plan to earn and keep operator trust and a real false-reject budget? Are the stakes matched to our maturity? Do the false-reject economics close? And can a knowledgeable human do this task with this data at all? Any unaddressed no on that list is a hold.

Used well, this list is not a brake, it is a redirect, and that is how you sell it upstairs. When line two fails the labeled-data test and the stable-process test, the honest answer to the VP is not "we cannot do AI." It is "this line is not ready, and here is what we do instead: we capture the expert's defect knowledge and start building the labeled set now, we stabilize the wash and lighting, and meanwhile we deploy AI where it is actually ready, on the knowledge-capture project that needs no clean sensor data and burns no trust." That turns a no into a sequenced yes. It protects the budget from a guaranteed failure and points it at a real return. The brownfield reality makes this discipline more important, not less, because greenfield plants deploy AI 40 to 60 percent faster precisely because they do not have these red flags; the brownfield plant has to clear them first, deliberately, one at a time.

The deepest reason to hold the line on this is the cardinal rule of the whole program: the customer audits you, not the vendor, and accountability for an AI-touched decision stays with the plant and the human who signs the record. If you deploy AI into a line that fails these criteria and it ships a defect or causes an incident, "the vendor demo looked great" is no defense, just as "the model flagged it" is no defense to an auditor. The professional who can look at a hot prospect and say "not yet, and here is the written list of why and what comes first" is not the one slowing the plant down. They are the one making sure that when the plant does deploy, it ships value instead of burning the crew, and that judgment is exactly what an AI-aware manufacturer is paid for.

Key Takeaways

• "Not yet" is a legitimate and often correct answer, and it should come with a written list of exactly what must be true before the deployment is reconsidered. A clean no on the wrong line protects the budget, the schedule, and the operator trust you will need on the right line.
• No labeled data, no deployment. A defect model needs labeled defect images and a predictive model needs failure history. The plants most desperate for AI quality help often have the least labeled data because their quality ran on uncaptured human judgment. Build the labeled set first; deploying a model on a dataset that does not exist is buying a confident guesser.
• An unstable or changing process has no stable signal for a model to learn. Wet parts, shifting lighting, drifting camera angles, and mid-qualification tooling all guarantee drift and false rejects. Stabilize and control the process first, then deploy the model on top of it.
• AI does not go into a safety-critical control loop first, period. Keep AI advisory and out of direct control of anything that moves unless it is governed to a far higher standard, because 78 percent of OT networks lack centralized monitoring and you cannot safely give control to a model on a network you cannot see.
• Trust is a design requirement, not a rollout afterthought. False rejects burn operators, and a burned operator disables the green light, killing the system and poisoning the next one. No false-reject budget and no plan to bring operators in as partners is a kill criterion even when the model is good.
• Match stakes to maturity and run the false-reject math. A first deployment on the single most safety- or customer-critical characteristic is the wrong place to learn. And if the false-reject cost, the sorting and scrapping of good parts, exceeds the value of escapes prevented, the deployment loses money even when it works.
• Apply the manual test as the cleanest filter: can a knowledgeable human do this task with the same data the model will get? If not, the information is not in the data and no model can conjure it. If only your retiring expert can, your real first project is capturing that expert.
• The kill list is a redirect, not a brake. It turns a guaranteed-failure "deploy everywhere now" into a sequenced yes: stabilize and capture first, deploy where you are ready, defend the choice to leadership. Accountability stays with the plant, so "the vendor demo looked great" is no defense for deploying into conditions that were red-flagged before a dollar was spent.