A superintendent walks a fourteen-story building at 5:14am with a coffee in one hand and a phone in the other, takes forty-seven photos, and by 7am owes a daily report, a look-ahead update, two NCRs, and a toolbox talk. Computer vision promises to turn those forty-seven photos into all of that, automatically. Sometimes it delivers something close to magic. Sometimes it says a finished slab is "in progress" and lists three trades that demobilized on Friday. This lesson explains, without a single equation, what computer vision actually does on a jobsite, why it lands around ninety percent, and exactly where the other ten percent will bite you. Get this and you will know which of those forty-seven photos to trust the machine on and which ones still need your eyes.

What the Machine Is Actually Doing When It "Looks"

Computer vision is the branch of AI that takes an image and tells you what is in it. On a jobsite the images come from a few sources you already know: a 360 camera clipped to a hard hat walking the floor (OpenSpace, Buildots), a fixed pole-mounted camera watching the whole site all day (EarthCam), a drone flying the sitework (DroneDeploy), or a laser scanner building a point cloud of existing conditions (Leica, Faro, Matterport). Different cameras, same underlying engine: software that has been shown millions of labeled construction images and has learned to recognize the patterns that distinguish drywall from concrete, a worker from a column, a fixture that is installed from one that is not.

The honest way to picture it is a brand-new safety inspector who has somehow looked at a million jobsite photos and is now very fast and very tireless, but who learned everything from photos and nothing from actually building. This inspector can glance at your 360 walk and catalog what is there faster than any human, never gets bored on photo five hundred, and never skips the boring back corner of the basement. That tireless consistency is the real gift, because human attention is exactly what fails on the forty-seventh photo at 6am. But this inspector has a specific kind of blindness we will get to, and the entire skill of using vision well is knowing where that blindness lives.

There are three jobs this engine does, and naming them keeps you from expecting the wrong one. It detects (there is a worker here, a fire extinguisher there), it classifies (this surface is poured concrete, that wall is gypsum board), and it measures progress over time (this week versus last week, model versus as-built). Detection and classification are the foundation; progress tracking is detection and classification run repeatedly and compared. When a vendor says "AI progress tracking," they mean the machine detected and classified the same building twice and told you what changed.

Why It Lands Around Ninety Percent, and Why That Is Truly Useful

Across standard scopes, jobsite computer vision recognizes what it is looking at correctly roughly eighty-five to ninety-five percent of the time. That number deserves a fair hearing before we attack the gap, because ninety percent is not a failing grade; it is a workforce multiplier. If a tool correctly catalogs ninety percent of what is in a 360 walk, it has done in two minutes the tedious inventory that would take a project engineer an hour, and it did it without the attention drift that makes the human miss things at the end of a long walk. For high-volume, repetitive looking, the machine is not just faster than a person; on the boring tasks it is often more consistent.

Why ninety and not ninety-nine? Because the engine learned from examples, and the real world keeps showing it things slightly different from its examples. It is excellent at the conditions it has seen a thousand times, a clean stud wall in good light from a normal angle, and it degrades as conditions drift from that. The accuracy is not one fixed number across your whole project; it is high on the standard stuff and lower on the unusual stuff, and the average you hear quoted hides that spread. The skill is not memorizing "ninety percent." It is knowing which parts of your specific building are the standard stuff the machine nails and which are the unusual stuff where its real accuracy quietly drops.

Ninety percent accuracy is not a single grade spread evenly across your building. It is near-perfect on what the machine has seen a thousand times and much weaker on what it has not. The quoted average hides the spread, and the spread is where your risk lives.

The Two Ways It Fails: False Positives and False Negatives

Vision errors come in exactly two flavors, and they cost you in opposite ways, so you have to manage them differently. A false positive is when the machine sees something that is not there: it tags a scaffold leg as a column form, calls a shadow a crack, or flags a worker as missing fall protection when the harness is just hidden behind a beam. A false negative is when the machine misses something that is there: a sleeve cast into a CMU wall does not register because it does not look like the sleeves in its training data, or a truly unprotected edge goes unflagged because the lighting washed it out.

These two failures are not equally dangerous, and which one hurts more depends entirely on what you are using the tool for. For progress tracking and quantity counting, false positives are annoying but cheap; you glance, you dismiss the scaffold-as-column, you move on. For safety, false negatives are the nightmare, because a hazard the machine fails to flag is a hazard nobody was told about, and "the system did not catch it" is not a defense that helps anyone who got hurt. This asymmetry is the whole reason the responsible posture for safety vision is different from the posture for progress vision: on safety you assume the machine will miss things and you keep a human in the loop on the high-consequence calls, full stop.

The deeper point is that the failures are not scattered randomly across your project like bad luck. They concentrate. They cluster in low light, in occlusion (something blocking the view), in half-finished trades that do not yet look like the finished examples the machine trained on, and in unusual or proprietary details the model has rarely seen. If you can name those conditions on your project, you can predict where the machine's real accuracy drops below the quoted average, and you can point human attention exactly there. That is the entire move: let the machine carry the standard, well-lit, fully-formed ninety percent, and put your eyes on the dark, blocked, half-built, weird ten percent.

A Word on Point Clouds and Scan-to-BIM

One flavor of jobsite vision deserves its own mention because it is becoming everyday work: scanning existing conditions into a point cloud and turning that into a model, called scan-to-BIM. A terrestrial laser scanner or a Matterport capture produces a dense cloud of measured points, millions of them, describing the real geometry of a space. Vision software then tries to recognize walls, floors, ceilings, and equipment in that cloud and generate a starting Revit model at a basic level of development, typically LOD 200, so a coordinator does not have to model existing conditions from scratch.

This is a real time saver and it has the same shape of limitation as everything else in this lesson. The point cloud's raw measurements are generally accurate; the recognition layer that turns points into "this is a wall, that is a duct" is the part that hits the ninety percent and misses the rest. Where it misses is predictable: MEP above ceilings, structural framing furred into walls, and anything occluded or proprietary. So a scan-to-BIM first pass is truly useful as a starting model, and it is malpractice to deliver it as a finished existing-conditions model without a human verifying the recognized elements against the actual space. The cloud measured the building candidly; the AI's interpretation of the cloud still needs a builder to check it.

Where Vision Actually Pays, and Where It Is Theater

Before the failure modes scare you off, it is worth being concrete about where jobsite vision earns its subscription, because the wins are real and specific. The clearest payoff is documentation that would otherwise not happen. A weekly 360 walk creates a complete, time-stamped visual record of every space, including the back corners nobody photographs, and that record is gold when a dispute arises six months later about whether the waterproofing was installed before the slab covered it. The machine did not have to be perfect to be valuable here; it had to be thorough and consistent, which it is, and the record it builds is more complete than any human would patiently create.

The second clear payoff is the quick comparison that surfaces a question a human would have missed. When the tool flags that the east wing looks two weeks behind the model, it is not issuing a verdict; it is asking a question you can then go answer, and surfacing that question early, before it becomes a visible slip, is worth real money even if half the flags turn out to be nothing. The value is in the prompt to look, not in the machine being right, which is exactly the predictive-attention posture from the engines lesson applied to vision.

Where vision becomes theater is when a firm buys it to replace a person's judgment rather than to extend their reach, and then trusts the output unverified because the dashboard looks authoritative. A beautiful progress dashboard that nobody reconciles against reality is worse than no dashboard, because it manufactures false confidence that flows into pay apps and owner reports. The tool that quietly builds your visual record and surfaces questions for a human to answer is paying for itself. The tool that is trusted to generate the answer, unwatched, is theater that will eventually produce a confident, wrong record on the day it matters most. Same software, two outcomes, decided entirely by whether a human stayed in the loop.

The Quiet Problem: The Machine Cannot Tell You When It Is Unsure

Here is the limitation that ties the others together and that almost no vendor demo mentions: when the machine is in its weak ten percent, it usually does not know it is in its weak ten percent. A human inspector who walks into a dark mechanical room squints, slows down, and says "I cannot see that corner well, let me get a light." The machine does not do that. It produces a confident classification for the dark corner exactly as confidently as it does for the bright, standard wall, because confidence and correctness are separate things inside it. It will tell you the slab is "in progress" with the same certainty it correctly identifies the drywall, and nothing in the output flags which call you can bank on and which you cannot.

Some tools do surface a confidence score, a number saying how sure the machine is, and where that exists it is truly useful: a low-confidence flag is the machine raising its hand, and you route those to a human. But many jobsite tools present every call with the same flat authority, and even the confidence scores are imperfect, sometimes high on a wrong answer. So you cannot rely on the machine to tell you where it is weak. You have to know where it is weak from the outside, from understanding that low light, occlusion, half-built work, and weird details are the failure zones, and route your attention there yourself. The machine's silence about its own uncertainty is precisely why the project-specific verification sentence has to come from you, the builder who can see the conditions, rather than from the tool.

This connects directly to the daily-report disaster we keep returning to. The reason the tool confidently reported a finished slab as "in progress" and listed demobilized trades is not that it was broken; it is that it was operating in its weak zone, on an unusual or transitional condition, and had no mechanism to tell you so. It narrated its low-confidence guesses in the same authoritative voice as its high-confidence facts, and a human who trusted the voice instead of knowing the failure zones shipped the result. The fix was never a better tool. The fix was a human who knew that transitional slab conditions are exactly where vision wobbles, and who therefore reconciled that line against the pour record before the report went out.

The Applied Problem: Catch One False Positive and One False Negative

Here is the exercise that turns this from reading into reflex. Take a single flagged result from a 360 capture on a real project, the kind of automated callout an OpenSpace or Buildots walk produces, and do not just accept or reject the whole thing. Hunt deliberately for one of each error type, because finding them trains your eye for where the machine is weak.

Look first for a false positive: a place where the machine confidently flagged something that is not actually what it claims. The classic is a scaffold or shoring leg tagged as a column form, or a stack of stored material read as installed work, or a reflection read as a defect. When you find it, note the condition that fooled the machine, because that condition will fool it again. Then hunt for a false negative, which is harder and more important: a place where something real is present and the machine said nothing. Scan the occluded zones, the dim areas, the half-installed trades. The cast-in sleeve it ignored, the embed it did not see, the edge it failed to flag because a stack of drywall was in the way.

Then write the one sentence that makes you dangerous in the best way: "On this project, the vision tool is reliable for X and must be human-checked for Y," where X is your standard, well-lit, fully-formed scopes and Y is your dark, occluded, half-built, or proprietary conditions. That sentence is the verification posture for every vision tool you will ever use, and it is not generic; it is specific to the building in front of you. The superintendent who can write it gets the ninety percent of effortless cataloging and keeps their eyes on the ten percent that determines whether today goes sideways. The one who cannot write it either trusts the machine into a wrong daily report or distrusts it into doing the whole inventory by hand. Neither of those is the job. The sentence is the job.

Key Takeaways

• Computer vision takes jobsite images (360 walks, fixed cameras, drones, laser scans) and does three things: detects, classifies, and tracks progress over time. Progress tracking is just detection and classification run twice and compared.
• It lands around eighty-five to ninety-five percent on standard scopes, which is a genuine workforce multiplier because it never suffers the attention drift that makes humans miss things on photo forty-seven at 6am.
• That average hides a spread: near-perfect on conditions it has seen a thousand times, much weaker on the unusual. Your risk lives in the spread, not the average.
• Failures come in two flavors. False positives (sees what is not there, like a scaffold leg as a column form) are cheap for progress work. False negatives (misses what is there, like a cast-in sleeve or an unprotected edge) are the nightmare for safety, where "the system did not catch it" protects no one.
• Failures cluster predictably in low light, occlusion, half-finished trades, and unusual or proprietary details. Name those conditions on your project and you can point human attention exactly where the machine is weak.
• Scan-to-BIM measures the building candidly in the point cloud, but its recognition of walls, ducts, and framing hits the same ninety percent and misses MEP above ceilings and framing in furred walls. A first pass is a starting model, never a finished one, without human verification.
• The artifact: one sentence per project naming what vision is reliable for and what must be human-checked. It is the verification posture for every vision tool, and it is specific to the building in front of you.