AI for Manufacturing
Capable · M21 · lesson 21 of 22 · queued
Preview — browse every lesson free. Enroll to mark lessons complete, open partner links and save your progress. Login & enroll →
Verifying Captured Knowledge
📖
now learning

Verifying Captured Knowledge

15 min

For nineteen years, everyone at the plant knew the rule about the number-three extruder: you let it warm up for forty-five minutes before you run tight-tolerance product, because if you do not, the first hour drifts and you scrap the lot. The rule came from Dave, the operator who had run that line longer than some of the supervisors had been alive, and nobody questioned it. So when Dave finally retired and the team ran a structured knowledge-capture interview with an AI assistant to get his know-how down before he walked out the door, the forty-five-minute warm-up went straight into the new digital knowledge base, stated as fact, ready to teach the next shift. Here is the uncomfortable part. When a process engineer finally pulled the historian data on that extruder, the barrel-temperature trace showed the machine reached stable operating temperature in about eighteen minutes, not forty-five. The drift Dave remembered was real once, years ago, before a heater-band replacement and a controller upgrade fixed it. The forty-five-minute rule had quietly become a myth that cost the plant roughly twenty-seven minutes of warm-up scrap-free production every single startup, and the knowledge-capture project had just enshrined it in software for the next decade. This lesson is about the step that separates a knowledge base that makes a plant smarter from one that makes it dumber faster: verifying captured knowledge before you teach it.

Why Capture Without Verification Is Dangerous

The talent cliff is the reason knowledge capture matters at all. Roughly two million manufacturing workers need reskilling by 2026 against about five hundred thousand unfilled roles, and 85 percent of manufacturers say staffing shortages are hurting product quality. When Dave retires, the plant does not just lose a pair of hands; it loses the only person who could hear a bearing going bad and the only memory of why the line is set the way it is. Capturing that knowledge before it walks out the door is one of the highest-return moves a thinning plant can make. But capture and verification are two different jobs, and skipping the second one is how a plant turns a retiring expert into a permanent source of confident errors.

Tribal knowledge, the undocumented know-how that lives in people's heads, has a specific failure mode: it ages. The expert learned the rule under conditions that were true at the time. Then the conditions changed. The heater band got replaced, the supplier changed the resin, the controller firmware was updated, the tolerance on the print was loosened in a revision nobody on the floor read. The expert keeps applying the original rule because it has never visibly hurt them, and because a rule that produces good parts gets no scrutiny. The knowledge is not a lie. It is a true statement about a plant that no longer exists.

AI makes this more dangerous, not less, for one reason: it launders uncertainty into confidence. When Dave told a new hire the forty-five-minute rule, he said it the way people say remembered things, with the hedges and the war stories attached. When that same rule comes out of a clean digital knowledge base, formatted, bulleted, and authoritative, it reads like a verified spec. The hedges are gone. A green operator who has never seen the line will trust the knowledge base completely, because trusting it is the whole point of building it. The format conveys a certainty the underlying knowledge never earned.

Captured knowledge is a hypothesis wearing the costume of a fact. Verify it against data and a second expert before you let it teach the next shift.

The Three Ways Captured Knowledge Goes Wrong

Not every error in a knowledge base is the same kind of error, and they do not all get caught the same way. Sorting them is the first step to verifying efficiently.

The stale truth. This is the extruder warm-up. The knowledge was correct when learned and became wrong when the plant changed underneath it. Stale truths are the most dangerous because the expert states them with total sincerity and they pass the smell test of everyone who learned from the same expert. They are caught almost exclusively by data, because the historian, the quality records, and the maintenance history remember the change the people forgot. If a captured rule rests on a physical condition, that condition is measurable, and the measurement is your check.

The superstition. This is the rule that never had a mechanism, only a coincidence. "We never run this die on a Monday because it always throws scrap on Mondays." Maybe Monday scrap was real, but the cause was the weekend humidity in an unconditioned warehouse, not the day of the week, and now the warehouse is climate-controlled. Superstitions are caught by asking for the mechanism: why would this be true, physically? If the expert cannot name a cause and the data does not show the effect, you have a superstition, and enshrining it teaches the next shift to fear a ghost.

The personal preference dressed as a spec. This is the most subtle. The expert sets a parameter a certain way because that is how they were taught, or because it feels right, not because the process requires it. "Run the oven at 340, not 350." The print allows a range of 330 to 360. Both temperatures make good parts. The expert's 340 is a preference, not a requirement, but captured as a rule it removes a degree of freedom the next engineer might need, and it implies a precision the process does not actually demand. These are caught by checking the captured rule against the actual specification: the drawing, the print, the PPAP. PPAP is the Production Part Approval Process, the customer-approved package that defines what the part and process must meet. If the spec is wider than the captured rule, the rule is a preference, and it should be recorded as one, not as a law.

The Verification Workflow: Data Plus a Second Expert

The cardinal rule of verifying captured knowledge has two anchors, and you need both. Anchor one is the data: the historian, the CMMS, the quality records, the process parameters the plant actually ran. CMMS is the Computerized Maintenance Management System, the database of work orders and maintenance history. Anchor two is a second human expert who was not the source of the knowledge. Data catches the stale truths and the superstitions. A second expert catches the things the data does not record and challenges the things the first expert took for granted. Neither anchor alone is enough.

Step one: separate claims from stories during capture. When you run the knowledge-capture interview, mark every statement as either a checkable claim ("warm up forty-five minutes or the first hour drifts") or context ("this line has always been finicky"). The claims are what you verify. The AI assistant is excellent at this sorting: feed it the interview transcript and have it extract a list of discrete, testable assertions, each with the condition it depends on. You are not asking the model whether the claims are true. You are asking it to organize them so a human can check each one.

Step two: test each claim against the data. For the warm-up rule, pull the historian's barrel-temperature trace across startups and measure the actual time to stable temperature. For a "this bearing fails every summer" claim, pull the CMMS failure history and see whether the failures actually cluster in summer. For a "scrap spikes on this material lot" claim, pull the quality records by lot. The model can help you assemble and summarize the data pull, but you read the result and you decide whether it confirms or contradicts the captured rule. The historian remembered eighteen minutes; the rule said forty-five; the data wins.

Step three: bring in the second expert for what data cannot settle. Some captured knowledge is not in any database. "When the motor makes this specific whine under load, the coupling is about to go." There may be no sensor on that motor and no historian tag for that sound. This is where a second experienced person matters. They can say "yes, I have heard that and that is right," or "no, that whine is the cooling fan and it means nothing," or "that used to be true before we added the vibration sensor that catches it earlier now." A second expert is not a rubber stamp. They are an independent witness who breaks the single-source dependency that made the knowledge fragile in the first place.

Be deliberate about who the second expert is. The worst choice is someone who learned everything from the first expert, because they will confirm the same stale truths and superstitions with the same sincerity. The best choice is someone with comparable hands-on experience but a different lineage: a maintenance tech who came up at a different plant, a process engineer who worked the same machine family for a different product, or a recently retired hand from a sister site. The whole value of the second anchor is independence. If your second expert is just an echo of the first, you have verified nothing and spent the effort to feel verified, which is worse than skipping the step because it manufactures false confidence.

Step four: record the verdict and the evidence, not just the rule. When the warm-up rule is corrected to eighteen minutes, the knowledge base should not silently overwrite forty-five with eighteen. It should record: original captured rule, the historian evidence that contradicted it, the corrected value, the date, and the human who verified it. This matters because the next person who reads "eighteen minutes" deserves to know it was verified against data, not just asserted, and because if the heater band fails again and the warm-up time genuinely changes, the history tells the story.

The Economics of a Myth Left in the Knowledge Base

It is tempting to treat verification as overhead, a nice-to-have that slows down the capture project. The extruder math says otherwise. The myth cost twenty-seven minutes of unnecessary warm-up per startup. On a line that starts up twice a day, that is fifty-four minutes a day. Across roughly five hundred operating days over two years, that is about twenty-seven thousand minutes, or four hundred fifty hours, of a tight-tolerance line sitting idle for no reason. If that line contributes four hundred dollars an hour of margin when it runs, the myth, left in the knowledge base, would have quietly cost about one hundred eighty thousand dollars over the life of that captured rule. Verifying it took the process engineer about ninety minutes of pulling and reading a historian trace.

That asymmetry is the whole argument. Verification is cheap and bounded; a myth in the knowledge base is expensive and compounding, because every green operator who learns from it inherits the error and applies it for years. And the failure mode runs both directions. A stale rule can be too conservative, like the warm-up, costing money in lost capacity. It can also be too aggressive: an expert's "you can push this press to 1,100 strokes an hour, it handles it fine" might have been true on the old die and dangerous on the new one, and enshrining that without verification puts a green operator and a machine at risk. The cost of an unverified rule is not always money. Sometimes it is a safety incident or a defect escape that becomes a customer containment.

Consider how the compounding works in practice, because it is the part that gets underestimated in the budget meeting. When the floor was staffed by veterans, a bad rule reached at most a handful of people and got corrected by the person at the next machine. Once it sits in the knowledge base, it reaches everyone, immediately, with the authority of the system. A plant that hires twelve green operators over two years to backfill retirements has just handed all twelve the same myth, stated as fact, with no veteran beside any of them to push back. The error did not just persist; it scaled. That is the mechanism that turns a single forgotten heater-band replacement into a six-figure loss: the knowledge base is an amplifier, and it amplifies whatever you put in it, true or false, with equal confidence. Verification is what decides whether you are amplifying a verified spec or a remembered ghost.

There is also a quieter cost that does not show up on any loss chart: trust. The first time a green operator catches the knowledge base in an obvious error, like a warm-up time they can see is wrong with their own eyes, they start discounting everything else it tells them, including the rules that are correct and important. An unverified base does not just teach individual errors; it spends the credibility that makes the whole capture investment worthwhile. A base the crew has learned to second-guess is a base nobody uses under pressure, which is exactly when they need it most. Verification protects the asset's credibility, and credibility is the only reason a knowledge base changes behavior on the floor at all.

Why the green crew makes this urgent

The reason this cannot wait is the same reason capture matters: the crew is thinning and greening at the same time. When the floor was full of twenty-year veterans, a bad rule in someone's head got corrected by the veteran next to them who knew better. That informal peer-check is exactly what the talent cliff is destroying. A green operator reading a confident knowledge base has no veteran beside them to say "actually, that warm-up rule changed after we swapped the heater band." The knowledge base has to carry the verification the missing veteran used to provide, because there is no longer a Dave on the next machine to catch the error. Verification is not bureaucracy. It is the substitute for the peer-review the floor used to do automatically and can no longer afford.

Keeping the Knowledge Base Honest Over Time

Verification is not a one-time gate at capture. A rule that was true and verified in 2026 can go stale in 2028 when the supplier changes or the machine is rebuilt. A knowledge base that is verified once and never revisited becomes a new source of stale truths, just with a cleaner font. Keeping it honest requires a few habits that a thinning crew can actually sustain.

Date and condition every rule. Every verified rule should carry the date it was verified and the condition it depends on. "Warm up eighteen minutes (verified against historian, June 2026, valid for current heater-band configuration)." The condition is what tells a future reader when to re-check: if the heater band is replaced, anyone can see the rule's assumption changed.

Tie re-verification to physical changes, not the calendar. You do not need to re-verify everything every quarter. You need to re-verify a rule when the thing it depends on changes: a new resin supplier, a controller upgrade, a print revision, a major rebuild. Link the knowledge base to the change-management process so that when a controller is upgraded, the rules that depend on that controller get flagged for re-verification. This is far more sustainable than a blanket annual audit nobody has time for.

Let the floor flag suspected myths. The operators running the line are the best myth detectors if you give them a path. When a green operator notices the line hits temperature in eighteen minutes but the knowledge base says forty-five, that observation should trigger a verification, not get ignored because "the system says forty-five." A knowledge base that punishes the people who notice it is wrong will rot. One that rewards them stays honest.

Keep the human accountable for what the base teaches. The same principle that governs every AI-touched decision on the floor applies here. The customer audits you, not the vendor, and a knowledge base that taught a green operator a wrong torque spec is the plant's accountability, not the software's. A named human owns each verified rule, the way a named human signs an inspection. That ownership is what turns a knowledge base from a pile of confident assertions into an asset a plant can stand behind in an audit and trust on the floor.

Key Takeaways

  • Capture and verification are two different jobs. Capturing a retiring expert's knowledge is high-return, but enshrining it unverified turns the expert into a permanent source of confident errors, like the forty-five-minute warm-up rule the historian showed was actually eighteen.
  • AI launders uncertainty into confidence: a hedged, war-story rule from Dave reads like a verified spec once it sits in a clean digital knowledge base, and a green operator will trust it completely.
  • Captured knowledge fails in three distinct ways: the stale truth (true when learned, the plant changed), the superstition (a coincidence with no mechanism), and the personal preference dressed as a spec (a choice recorded as a law).
  • Verification needs two anchors, and you need both: the data (historian, CMMS, quality records) catches stale truths and superstitions, and a second independent expert catches what the data does not record and breaks the single-source dependency.
  • The four-step workflow: separate checkable claims from stories during capture, test each claim against the data, bring in a second expert for what data cannot settle, and record the verdict and the evidence, not just the corrected rule.
  • The economics are lopsided: verifying the warm-up rule took about ninety minutes; leaving the myth in the base would have cost roughly one hundred eighty thousand dollars in lost capacity over the rule's life, and some unverified rules carry safety risk, not just cost.
  • The thinning, greening crew makes verification urgent: the informal peer-check a veteran used to provide is gone, so the knowledge base itself must carry the verification, because there is no longer a Dave on the next machine to catch the error.
  • Keep the base honest over time: date and condition every rule, tie re-verification to physical changes rather than the calendar, let the floor flag suspected myths, and keep a named human accountable for what the base teaches, because the customer audits you, not the vendor.