The eligibility worker had forty-one redeterminations due by the end of the week, and the new AI eligibility-support tool was, by every measure she could see, a gift. It read the income documents, cross-walked the household against the program rules, and produced a clean determination summary in the time it used to take her to open the file. On the screen in front of her was a TANF (Temporary Assistance for Needy Families, the time-limited cash-assistance program) redetermination for a single mother of three. The tool's summary was confident and complete: income calculated, household composition confirmed, work-requirement compliance assessed, recommended action stated in a tidy box at the bottom. "Recommended determination: Deny. Household exceeds the earned-income limit for the assistance unit size." She had clicked "approve the recommendation" on the last six cases because the tool had been right on all six. Her cursor was over the button. Then she stopped, not because anything looked wrong, but because of a single question her supervisor had drilled into the unit during training: not "is the tool right?" but "if a fair hearing officer asked me to explain this denial in my own words, could I?" She could not. She did not yet know why this family was over the limit, only that a model had said so. So she opened the file. The earned-income figure the tool used included a one-time back-pay lump sum that, under the state's own policy, was excluded from the monthly earned-income calculation for the work-requirement test. The family was eligible. The button she had almost pushed would have cut a household with three children off cash assistance on the strength of a number the model had handled wrong and she had never checked.

The Button and the Boundary

That moment, the cursor hovering over "approve the recommendation," is the whole subject of this lesson compressed into a single second. It is the exact point where AI support either stays support or quietly becomes the decision. The tool did not make the determination. A human did, or was about to. But if she had clicked without knowing why, the human in the loop would have been a formality, a hand on a rubber stamp. The determination would have been the model's in every way that matters, with a person's name attached to give it legal cover.

This is the distinction the entire program is built on, stated in the cardinal rule: AI informs, humans decide. By this point in the curriculum you have learned that an AI tool can draft a case note, summarize a record, and apply a policy rule, and that each of those outputs must be verified because the model generates plausible text, not verified fact. This lesson is about the last and most important step, the one that the prior lessons on eligibility support and policy misapplication lead directly into: the determination itself. The decision to approve or deny a person's benefits is not a documentation task. It is an exercise of delegated government authority, bound by due process, and it has to stay human in a way that is real, not nominal.

Keeping the determination human means more than having a person present when the tool runs. It means the accountable worker forms an independent judgment, can articulate the basis for the decision without reference to the tool, and owns the outcome. The worker in the opening did not do the first part until she forced herself to. When she did, the boundary held and a family kept its benefits. The gap between those two outcomes was about eleven minutes of file review. The cost of skipping it would have been a wrongful denial, a fair-hearing reversal, and a household without cash assistance during the weeks the appeal took to resolve.

A human in the loop who cannot explain the decision in their own words is not a decision-maker. They are a signature the model borrowed.

Why the Determination Is Different From the Draft

It is worth being precise about why the eligibility determination sits in a different category than the case note or the summary, because the difference is what makes the human-decision rule non-negotiable here rather than merely advisable.

A case note documents what happened. A determination decides what happens next. When an eligibility worker approves or denies a benefit, that worker is acting as the government, exercising authority that the law has delegated to a human officer of an agency. SNAP (the Supplemental Nutrition Assistance Program, the federal food-assistance benefit), Medicaid (the joint federal-state health-coverage program for low-income people), TANF, and housing assistance are all administered under statutes and regulations that contemplate a responsible human making the eligibility decision and standing behind it. The right to a fair hearing, the constitutional due-process protection that lets a person challenge an adverse benefits action, presumes there is a decision-maker who can be questioned and a rationale that can be examined.

An algorithm cannot be cross-examined. It cannot take an oath, cannot be asked what it considered, cannot be held to account if it was wrong. When a determination is challenged at a fair hearing, the agency must produce a human who decided and a reason that holds up. "The model recommended denial" is not a reason a hearing officer can accept, because it answers the wrong question. The question is not what the tool output. The question is why the agency, through its worker, concluded this person does not qualify. If the only honest answer is "the tool said so," the agency has no defensible determination. It has an automated decision wearing a worker's badge, and that is precisely the arrangement due process exists to prevent.

Consider the scale of the consequence. A wrong case note can be corrected, and if caught it embarrasses no one but the worker. A wrong determination removes food, medicine, shelter, or cash from a household that needed it, often during a period of acute crisis, and the harm lands before any appeal can reverse it. A family wrongly denied SNAP goes without groceries in the weeks it takes to get a hearing. A person wrongly cut from Medicaid skips a prescription. The determination is the moment AI is closest to doing direct, immediate harm to a real person, which is exactly why it is the moment human judgment must be most fully present.

The Rubber Stamp and Automation Bias

The threat to the boundary is rarely a worker who decides to let the machine take over. It is a worker, competent and conscientious, who slides into it without noticing, under exactly the conditions this field operates in: too many cases, too little time, a tool that is right most of the time, and a button that is faster to click than to question.

The mechanism has a name: automation bias, the well-documented human tendency to over-trust the output of an automated system, especially when we are busy, tired, or under pressure, and especially when the system is usually correct. Automation bias is not a character flaw. It is how attention works under load. The first six TANF cases the worker approved were correct, and each correct approval trained her, a little, to trust the seventh. By the time she reached the family that was wrongly flagged for denial, her default had quietly shifted from "verify, then decide" to "approve unless something looks wrong." Nothing looked wrong. Hallucinated and miscalculated output rarely does, as the earlier lessons established: the tool's denial recommendation was professionally formatted, internally consistent, and confidently stated. The error was not visible on the surface. It was only findable in the file.

The danger compounds with the tool's accuracy. A tool that is wrong half the time keeps everyone alert. A tool that is right ninety-five percent of the time is the dangerous one, because it earns a trust that the remaining five percent does not deserve, and that five percent, spread across a caseload of forty determinations a week, is two wrongful actions a week, roughly a hundred a year for a single worker, each one a household. The better the tool, the stronger the pull toward the rubber stamp, and the more deliberate the discipline that resists it has to be.

Signs the Boundary Is Eroding

Erosion of the human-decision boundary shows specific, observable symptoms. A worker, or a supervisor watching a unit, can learn to spot them:

• Speed that outpaces understanding. Determinations are being finalized faster than a human could actually review the underlying facts. If a worker closes more cases per hour than the files could be read, the human is not deciding.
• The "approve the recommendation" reflex. The worker's interaction with the tool has collapsed to clicking the recommended action without independently reconstructing why it is correct.
• Explanations that cite the tool. When asked why a determination was made, the answer references the model's output rather than the policy and the facts. "The system flagged it" instead of "the household's countable income exceeds the limit because of X."
• Verification only when something looks off. The worker checks the file only when the recommendation seems surprising, which means the confident-but-wrong outputs, the most dangerous kind, sail through unexamined.

The worker in the opening caught herself at the edge of the third and fourth symptoms. The fix was not to distrust the tool entirely. The tool had been genuinely useful on six cases. The fix was to restore the order of operations: the human forms the judgment, the tool informs it, and the determination is never finalized on the tool's say-so alone.

What Keeping It Human Actually Requires

Holding the boundary is a practice, not an attitude. It comes down to a small number of concrete requirements that turn "humans decide" from a slogan into something a worker actually does and a supervisor can actually check.

An Independent Basis for the Decision

The accountable worker must be able to state the reason for the determination in their own words, grounded in the policy and the specific facts of the case, without reference to the tool. The test is the one from the opening: if a fair-hearing officer asked you to explain this denial, could you, from the policy and the file, independently? If the answer is no, the determination is not yet yours, and it is not yet ready to finalize.

This does not mean ignoring the tool's output. It means treating the output as one input to your own reasoning, the same way you would treat a colleague's quick read of the case: useful, worth considering, and not a substitute for working the determination yourself. The worker in the opening got there by opening the file and tracing the income calculation. Eleven minutes. That is the price of an independent basis, and on a wrongful-denial case it is the cheapest eleven minutes the agency will ever spend.

Verification Before the Decision, Not After

The verification disciplines from the prior eligibility lessons, checking the income figures, confirming household composition, applying the current policy rule rather than a superseded one, all happen before the determination is finalized, not as a post-hoc audit. Catching policy misapplication after a denial has gone out means the family already received an adverse action and now has to appeal it. The whole point of keeping the human in the decision is that the human catches the error before it reaches the person. The lump-sum back-pay exclusion in the opening was a verification catch, and it worked because it happened before the click, not after.

Documenting the Human Rationale

Because the determination is a legal action subject to challenge, the human reasoning must be recorded, not just the outcome. The case record should show that a worker reviewed the facts, applied the policy, and concluded, with the basis stated. It should not read as if a tool produced a number and a person accepted it. If the AI tool was used, that is worth noting for transparency, but the substance of the record is the human rationale: the countable income, the applicable rule, the exclusions considered, the conclusion. This is what lets the agency defend the determination at a hearing, and it is what distinguishes a documented human decision from a logged machine output. A determination whose entire rationale is "tool recommended approve/deny" is not defensible, and a supervisor reviewing the unit should treat its appearance as a red flag.

The Authority to Overrule, and the Duty to Use It

Keeping the determination human is meaningless if the worker cannot, in practice, override the tool. The worker must have both the authority to disagree with the recommendation and the genuine ability to act on that authority without penalty, without a burdensome justification process that makes overriding harder than complying, and without a production metric that punishes the time it takes to do the verification that produces a correct override. An agency that counts determinations-per-hour and ignores wrongful-denial rates is, whether it intends to or not, paying workers to rubber-stamp. The duty to overrule a wrong recommendation is real, and the agency's job is to make exercising it possible. In the opening, the worker overrode a denial recommendation and approved the family. That override, correctly exercised, is the boundary working exactly as designed.

Where Accountability Lands

When a determination is wrong, the question of who is responsible has a clear and uncomfortable answer, and workers need to understand it before they click, not after.

The worker who finalizes the determination is accountable for it. Not the vendor who sold the tool, not the model that produced the recommendation, not the supervisor who deployed it across the unit. The same principle that governs AI-assisted documentation governs determinations, and it is even sharper here because the determination is an official action: "the AI recommended it" is not a defense at a fair hearing, in an agency review, or before a licensing body. The authority to decide was delegated to a human, and the accountability travels with the authority. A worker who finalizes a denial they did not independently understand has not transferred the risk to the tool. They have absorbed it.

This is not meant to frighten workers away from the tools. It is meant to locate the responsibility accurately, because accurate location of responsibility is what keeps the boundary real. When the worker knows the determination is theirs, the incentive to actually decide it, rather than approve it, is restored. The accountability is not a punishment bolted onto the workflow. It is the thing that makes the workflow trustworthy. A family facing a denial deserves to know that a person, accountable and able to explain it, decided their case. That assurance is exactly what the human-decision rule provides, and it is exactly what the rubber stamp destroys.

Supervisors carry a parallel accountability at the unit level. A supervisor who notices determinations closing faster than files can be read, or rationales that cite the tool instead of the policy, is seeing the boundary erode and has a duty to intervene before a wrongful action goes out. The unit-level signs in the earlier section are the supervisor's dashboard. Watching for them is how an agency keeps a hundred individual decisions human at once, rather than discovering at a hearing that its workers have been signing the model's name to their own.

Key Takeaways

• The eligibility determination is the moment AI support is closest to becoming the decision. The boundary either holds or quietly fails at the instant a worker clicks "approve the recommendation," and it fails the moment a human in the loop cannot explain the decision in their own words.
• A determination is categorically different from a case note. It is an exercise of delegated government authority bound by due process, and the right to a fair hearing presumes a human decision-maker who can be questioned and a rationale that can be examined. An algorithm cannot be cross-examined, so it cannot be the decider.
• The real threat is automation bias: the documented tendency to over-trust an automated system, strongest when the worker is busy and the tool is usually right. A tool that is right 95 percent of the time is the dangerous one, because the trust it earns is not deserved by the wrong 5 percent, and across a caseload that 5 percent is a household every few days.
• Watch for the erosion signs: determinations finalized faster than files could be read, the reflexive "approve the recommendation" click, explanations that cite the tool instead of the policy and facts, and verification that happens only when output looks surprising.
• Keeping it human requires an independent basis for the decision (could you explain it to a hearing officer from the policy and the file?), verification before the determination rather than after, a documented human rationale rather than a logged tool output, and a real, usable authority to overrule the recommendation.
• Verification must come before the click. Catching a policy misapplication after the denial has gone out means the family already suffered an adverse action and must now appeal; catching it before is the entire purpose of the human in the decision.
• Accountability for a determination lands on the worker who finalizes it, not the vendor or the model. "The AI recommended it" is not a defense at a fair hearing or in an agency review, because the authority to decide was delegated to a human and the accountability travels with it.
• Supervisors hold a unit-level accountability: speed that outpaces understanding and rationales that cite the tool are the dashboard warning that the boundary is eroding across the unit, and intervening before a wrongful action goes out is part of the job.