AI for Manufacturing
Capable · M11 · lesson 11 of 22 · queued
Preview — browse every lesson free. Enroll to mark lessons complete, open partner links and save your progress. Login & enroll →
Getting Accurate Output from AI on Technical Work
📖
now learning

Getting Accurate Output from AI on Technical Work

15 min

A reliability engineer named Sam is rebuilding a gearbox on a hot Thursday, two techs short, with the line down and a plant manager checking in every twenty minutes. He pulls up an AI assistant and asks it to write the bolt-torque sequence for the housing. The answer comes back fast and clean: "Torque the eight housing bolts to 95 ft-lb in a crisscross pattern." It reads like something off a service manual. Sam almost runs with it. Then he does the one thing that separates a draft from a record: he opens the actual OEM (Original Equipment Manufacturer, the company that built the gearbox) service sheet sitting in the cabinet, and the real spec is 110 Nm in three staged passes, not 95 ft-lb in one. The AI did not look up his gearbox. It produced the average torque value for a part that looked like the one he described, in whatever unit was most common in its training data, and it skipped the staged passes entirely because his question never demanded them. Ninety-five ft-lb is about 129 Nm. On a magnesium housing that overtorque cracks the casting. The model was not broken. It answered the question Sam actually asked, which was a vague one, with the only thing it had, which was the internet average. Getting accurate output from AI on technical work is a skill, and that skill is the difference between a tool that drafts and a tool that hands your crew a confident way to break a part.

Why Fluent Is Not the Same as Correct

The previous lesson established that a vague prompt produces a fabricated answer. This lesson goes one layer deeper, into the failure that bites you even when your prompt looks reasonable: the model is built to sound right, and sounding right is not the same as being right. A large language model predicts the most likely next words. It has no internal sense of whether a torque value is physically safe, whether a tolerance is achievable on your machine, or whether a root cause matches your historian. It optimizes for fluency, and fluency is the disguise that a wrong technical answer wears to get past you.

This matters more on technical work than anywhere else, because technical work has a single correct answer that the customer can audit, and the failure is silent. When AI drafts a marketing email and the tone is slightly off, you notice. When AI drafts a torque spec and the number is 15 percent high, nothing on the page looks wrong. The formatting is crisp. The units are present. The pattern is named. The only way to know it is wrong is to check it against the drawing, and the entire pressure of a short-staffed shift is pushing you to skip exactly that step. The playbook is direct about the stakes: the job shifted from "produce the draft" to "verify the draft against the drawing, the standard, and the historian." Accuracy is not something you hope the model has. It is something you engineer into the prompt and then confirm on the way out.

Put a number on the cost of getting this wrong. A single defect escape that becomes a customer containment can cost more than a month of a reskilling program, and unplanned downtime is the tallest bar on most plants' loss charts. If a hallucinated 95 ft-lb cracks a housing, you have bought yourself a scrapped casting, a longer outage on a line that was already down, and a root-cause investigation that will eventually trace back to an unverified AI answer. Against those numbers, the two minutes it takes to force the model to the source document and check the result is the cheapest quality control in the building.

The model optimizes for sounding right. Your job is to engineer for being right, then verify it. Fluency is the disguise a wrong answer wears.

Anchor the Model to the Drawing, the Standard, the Work Order

The single most powerful technique for accurate technical output is to stop asking the model what it knows and start forcing it to read what you have. There are three documents that govern almost every technical decision on the floor: the drawing (the engineering print with the dimensions, tolerances, and notes), the standard (the procedure, the customer spec, the IATF 16949 or AS9100 requirement, the OEM service sheet), and the work order (the specific job, part numbers, quantities, and the actual line). When you paste the relevant text from these and instruct the model to answer only from what you pasted, you convert it from a confident guesser into a reader of your reality.

This is grounding, the floor-level version of what the AI world calls RAG (Retrieval-Augmented Generation, where the model is required to base its answer on source documents you provide rather than on its memory). You do not need to build any infrastructure to use it. You paste the source and you demand a citation. The instruction that does the work is concrete: "Using only the text I pasted below, state the answer and quote the exact line you used. If the answer is not in the provided text, reply 'not found in provided document' and do not fill it in from your own knowledge."

Return to Sam and the gearbox. The accurate version of his prompt is not cleverer wording. It is pasting the OEM service sheet section and writing: "From the procedure text below, give the housing bolt torque, the unit, the tolerance, the number of passes, and the pattern. Quote the line for each value. If a value is not stated, say so." Now the model returns "110 Nm, three passes at 30/60/100 percent, crisscross pattern" and quotes the row it read. Sam verifies the quote against the sheet in one glance. The hallucinated 95 ft-lb cannot survive this prompt, because the prompt never let the model reach for its memory in the first place.

What "anchor to the standard" looks like in practice

A quality engineer drafting a containment for a customer does not want the model's idea of what IATF 16949 (the automotive quality management standard the customer audits you against) requires. She wants the model to work from the customer's actual specific requirement document. The accurate prompt pastes the requirement and says: "Draft the containment description so that it satisfies the requirement in the text below. For each element of the containment, name the specific clause it addresses. Do not assert any requirement that is not in the pasted text." The model that grounds on the real requirement produces an audit-ready draft. The model left to its training produces a plausible-sounding requirement that may not match this customer at all, and "the model said the standard requires it" is exactly the answer the playbook warns will not survive an audit: the customer audits you, not the vendor.

Show It One Good Example, Then Ask for the Next

When you need the model to produce something in your plant's exact format, with your plant's exact conventions, do not describe the format in words and hope. Show the model one real, correct example and ask it to produce the next one in the same shape. This technique is called few-shot prompting, and on technical work it is the fastest way to lock the output to your standard instead of the generic one.

The reason it works is the same reason the model fails on vague prompts. The model copies patterns. If the only pattern it has is the internet average of a maintenance log, it gives you the internet average. If you hand it one of your own properly written CMMS (Computerized Maintenance Management System, the software that holds equipment records and work orders) entries, complete with your asset-numbering scheme, your fault-code conventions, and your labor-estimate style, it copies that instead. You are no longer fighting the model's defaults. You are steering it with an example it can imitate.

Here is the worked move. A maintenance tech has a stack of rough notes to turn into clean work-order descriptions. Instead of asking "write a work order for this," he pastes one finished example: "ASSET PMP-204 | Symptom: seal weep at outboard end, ~3 drops/min | Suspected cause: worn mechanical seal | Action: replace seal kit P/N 44-119, inspect shaft sleeve | Est labor: 2.5 hr | Parts staged: yes." Then he writes: "Using the exact format, fields, and style of the example above, convert the rough notes below into a work-order description. Keep the same field labels and order. If a field has no information in the notes, write 'TBD' rather than inventing it." The model now produces entries that drop straight into the CMMS without rework, because it is matching his record, not a generic one. The "TBD rather than inventing" clause is what keeps it honest when the notes are thin.

Two or three examples for tougher patterns

For something with more variation, like classifying a defect note into a fixed set of categories, one example is not enough. Give the model two or three, deliberately chosen to cover the edge cases, before you ask for the new one. If your defect categories are cosmetic, dimensional, functional, and material, show one example of each before handing it the ambiguous note. The model learns the boundaries from your examples far better than from a paragraph describing them. This is the same logic a trainer uses with a green operator: do not just describe a good part, show them a good one and a bad one and let them see the line between.

Decompose the Question So Each Step Is Checkable

Complex technical questions fail when you ask for the whole answer at once, because the model takes one confident leap and you cannot see where it went wrong. The fix is to break the question into steps the model has to show, so that each step is something you can verify before the next one builds on it. This is the floor version of what the AI world calls chain-of-thought: making the model reason out loud instead of jumping to a conclusion.

Consider a process engineer trying to figure out why first-pass yield (FPY, the percentage of parts that pass all inspections the first time with no rework) dropped two points on a molding line. Asking "why did my yield drop" gets a generic essay about common molding defects. The accurate approach decomposes it: "Work through this in numbered steps and show each step. Step 1: list the data I would need to diagnose a 2-point FPY drop on an injection molding line. Step 2: from only the data I provide below, state what each data point suggests. Step 3: rank the candidate causes by how well the data supports them. Step 4: for the top cause, state what additional measurement would confirm or rule it out. Do not state a root cause as confirmed; this is a hypothesis list for a human to verify." Now you can check the logic at each step. If Step 2 misreads a data point, you catch it before it poisons the ranking in Step 3.

Decomposition also exposes the model's hidden assumptions, which is where technical errors hide. When the model has to show its work, you can see it assume the part is dry when yours is wet, or assume a tolerance that does not match your print. A single-shot answer buries those assumptions inside a confident paragraph. A stepped answer puts them on the table where you can correct them. The playbook frames the destination plainly: an AI-assisted root cause should assemble the evidence, but the human owns the conclusion. Decomposition is how you keep the conclusion in human hands instead of accepting the model's leap.

Make the model state its assumptions explicitly. Add one clause: "Before you answer, list every assumption you are making about my process, my material, and my equipment. I will correct any that are wrong before you proceed." This converts the assumptions from invisible to reviewable. On a molding line where the model assumed a barrel temperature you never gave it, that single clause is the difference between a useful hypothesis and a confident answer to the wrong question.

Force the Model to Flag Its Own Uncertainty

The most dangerous technical answer is the one delivered with full confidence on a point the model is actually unsure about. Models do not naturally signal doubt. They state a 60-percent-likely guess and a near-certain fact in the same even tone. You have to make uncertainty visible, because on the floor the difference between "this is the spec" and "this is probably the spec" is the difference between a record and a guess.

The technique is to require the model to label its confidence and separate what it knows from what it is inferring. A practical instruction: "For each value or claim, mark it as one of: STATED (directly from the document I provided), INFERRED (your reasoning from what I gave you), or GENERAL (from your training, not specific to my plant). I will treat anything marked GENERAL as unverified until I check it." This single move turns an undifferentiated wall of confidence into a triage map. The STATED items you can use after a quick source check. The INFERRED items you scrutinize. The GENERAL items you never put on a record without verifying against the drawing or the historian (the time-series database that stores tag values like temperature, pressure, and vibration over time).

Walk through the payoff. A green tech asks the AI to help diagnose a recurring bearing failure. Without confidence labels, he gets a paragraph that mixes "your vibration data shows a clear inner-race defect frequency" (which the AI cannot actually know unless he gave it the data) with "inner-race defects often come from contamination or misalignment" (a general truth). With confidence labels, the AI is forced to mark the first claim GENERAL because the tech never provided vibration data, exposing that the model was about to assert a measurement it never saw. That exposure is the save. The playbook's predictive-maintenance lesson is built on turning a real signal into a verified work order, not a dashboard guess; confidence labels are how a green tech keeps an unverified guess from becoming a work order.

Reward the refusal, every time

An AI that says "I cannot determine the tolerance from what you gave me" has done your job for you by catching a gap before it reached the floor. An AI that fills that gap with a plausible number has set a trap. Build every technical prompt so that refusal is the safe, expected response when the source does not contain the answer. The instruction is simple and you should use it constantly: "If you do not have enough information to answer accurately, say exactly what is missing instead of guessing." The refusal is not the model failing. It is the model finally telling you the truth, which is that the answer is not where you assumed it was.

The Verification Pass That Makes It a Record

No prompting technique removes the verification step. It is the spine of this entire program. Even a perfectly grounded, decomposed, confidence-labeled answer is a draft until a human traces it to the real source and signs it. The techniques in this lesson make the draft faster, cleaner, and more honest about its own gaps. They do not make it true. You make it true. The discipline is non-negotiable precisely because the failure is silent and the customer audits you, not the vendor.

Run a structured verification pass, the same one every time, so it survives a hot afternoon when you are short two people. First, trace every number to its source. Each value the model gave you gets matched to the drawing, the standard, or the work order it claims to come from. If the model cited a line, confirm the line says what the model claims. If it did not cite a line, the number is unverified and does not go on a record. Second, check units and tolerances. Confirm every value carries the right unit and a tolerance, and sanity-check any conversion against a known anchor (1 inch is 25.4 mm; 1 ft-lb is about 1.36 Nm). The 95 ft-lb versus 110 Nm gap that almost cracked Sam's housing is exactly the kind of error this catches. Third, check the procedure against the actual process. A procedure that is internally consistent can still be wrong for your line if it assumes a step your process does not have or skips one it requires.

For higher-stakes output, add a holdout check. A holdout is a known-answer case you keep in your pocket to test whether the model and the workflow are trustworthy on a given day. Before you trust the model on a tolerance you do not know, give it three the model has not seen but you do know cold, embedded in the same kind of document. If it nails the three you can verify, you have evidence it read the document correctly. If it misses one of the three, you have learned not to trust it on the fourth you cannot check. This is the same logic as the false-reject discipline the playbook teaches for vision systems: you do not trust the green light until you have measured how often it lies. The holdout is how you measure whether the AI is reading your documents or hallucinating around them.

Accountability never transfers to the tool. When you sign the work instruction, the traveler, the 8D (the eight-discipline structured problem-solving report customers require), or the CMMS entry, you own every number on it, including the ones the AI drafted and labeled STATED. "The model wrote it" is not a defense to an auditor, a customer, or a green operator who followed a fabricated spec into a bad part. The prompting techniques are your tools. The verification pass is your signature. Keep them clearly separate and you get the speed of AI on technical work without inheriting its confidence in things that are not true.

Key Takeaways

  • Fluent is not correct. The model optimizes for sounding right, and on technical work the failure is silent: a torque value 15 percent high looks exactly as crisp as the real one. Accuracy is engineered into the prompt and confirmed on the way out, never assumed.
  • Anchor the model to the drawing, the standard, and the work order by pasting the real text and demanding it answer only from what you provided, with the exact source line quoted. The hallucinated 95 ft-lb cannot survive a prompt that never lets the model reach for its memory.
  • Show the model one real, correct example in your plant's format and ask for the next one in the same shape. Few-shot prompting locks output to your CMMS conventions and your defect categories instead of the internet average; use "TBD rather than invent" to keep thin notes honest.
  • Decompose complex questions into numbered, checkable steps so the model shows its reasoning and its assumptions instead of taking one confident leap. Make it list its assumptions about your process and material before it answers, so you can correct the wrong ones first.
  • Force the model to label every claim as STATED, INFERRED, or GENERAL so confidence becomes visible. Anything GENERAL is unverified until checked against the drawing or the historian, and a green tech avoids turning an unseen-data guess into a work order.
  • Reward refusal. An AI that says exactly what is missing has caught a gap before the floor; an AI that fills the gap with a plausible number has set a trap. Build every prompt so refusal is the safe default when the source lacks the answer.
  • Run the same verification pass every time: trace every number to its source, check units and tolerances against a known anchor, and check the procedure against your actual process. For high-stakes output, use a holdout of known-answer cases to measure whether the model is reading your documents or hallucinating around them.
  • Accountability never transfers to the tool. When you sign the instruction, the traveler, the 8D, or the CMMS entry, you own every number on it, including the AI-drafted ones. The techniques are your tools; the verification pass is your signature, and the customer audits you, not the vendor.