It is 4:47 on a Tuesday afternoon and Maria, a child protective services (CPS) caseworker with eleven years on the job, is sitting in her supervisor's office trying to figure out what just happened. A vendor had come in that morning to demonstrate what the county was about to purchase, a platform the procurement memo called "an AI case decision support system." The demo was polished. The presenter showed a screen where a child's file opened, a risk score appeared in a colored bar (red, orange, yellow), and a recommended action populated below it. The presenter said: "The AI analyzes the case and tells you what to do next." Maria asked, "But who decides whether to remove the child?" The presenter smiled and said, "The AI gives you the recommendation. That dramatically reduces the burden on your workers." Maria's supervisor is now drafting a memo to procurement. Something in that room felt deeply wrong, and both women are trying to put it into language.

That feeling has a name. It has several names, actually, and this lesson is going to give you every one of them. By the time you finish reading, you will be able to walk back into that procurement meeting, into a team training, into a supervisory review of a new platform, or into a conversation with a vendor, and explain precisely what type of AI you are looking at, what that type can and cannot do, and why "the AI tells you what to do next" is not a feature description. It is a misunderstanding of the technology and a violation of one of the most fundamental rules in human services: the decisions in this field are made by people. Not algorithms.

This lesson is Chapter 1.1 of the Level 1 program, and it does one specific job: it separates the three very different things that vendors, news articles, policy memos, and even colleagues all call "AI," and it plants the cardinal rule so firmly in your professional practice that it becomes second nature. The rule is simple. AI informs, humans decide. Everything else in this program builds from there.

The Problem with One Word for Three Very Different Things

When someone says "the AI" in a human-services context in 2026, they might mean any of three completely different kinds of software. They might mean a summarization model, which reads existing text (a case note, a court report, an intake form, a set of prior records) and compresses it into a shorter, cleaner version. They might mean a predictive screening model, which takes a set of structured inputs (ages, prior contact history, reported incident types, household composition data) and produces a numeric score or probability estimate based on patterns learned from historical cases. Or they might mean a generative language model, also called a large language model (LLM, an AI system trained on massive amounts of text to predict and generate human-like written language), which takes a prompt (an instruction or question given to the model) and writes new text: a case note draft, a client letter, an eligibility summary, a court report outline.

These three types of AI are as different from each other as a calculator is from a medical imaging scanner is from a dictation system. They fail in different ways. They carry different risks. They have different appropriate uses. And conflating them, calling them all "the AI," produces exactly the confusion in Maria's procurement meeting. A vendor who demonstrates a generative drafting tool and calls it a "case decision system" has either confused the types themselves or, more worryingly, is counting on their audience to do so.

Let's pull them apart slowly, because the stakes in this field are too high to rush through the definitions.

Type One: Summarization Models

A summarization model reads text that already exists and produces a shorter version of it. In human-services contexts, this is the most immediately useful and the least dangerous type of AI, which is why it has become the most widely adopted. The pattern sometimes called "Magic Notes" or "AI-assisted documentation" is a summarization workflow: the caseworker visits a family, speaks notes aloud or types brief observations into a mobile device, and an AI model drafts a structured case note from those observations. The model is not inventing anything; it is reorganizing and condensing what the caseworker provided.

The benefit here is concrete and urgent. Caseworkers in the United States spend a large share of every working day, often half or more, doing documentation: writing case notes, drafting court reports, completing intake assessments, filling eligibility forms. That paperwork burden is the single largest driver of burnout and voluntary turnover in the field, and when experienced caseworkers leave, their caseloads are absorbed by the workers who remain, which increases the risk of harm slipping through. A summarization tool that returns two or three hours per day to direct work with families is not a minor productivity enhancement. It is a structural intervention in the field's defining crisis.

But summarization models have a specific and important failure mode: they can invent. A model that has read ten thousand home visit case notes will, under some conditions, fill gaps in a sparse input with language that sounds like what a case note would say. If a caseworker provides incomplete notes and the model adds a detail that was never observed ("the home was clean and organized," "the parent appeared calm and cooperative"), that invented detail becomes part of a legal record. A case note is not a casual document. It is submitted to courts. It is cited in removal decisions. It is reviewed by advocates and attorneys. A fabricated detail in a case note can influence a decision about whether a child stays with a family, and no one would know the observation was never made. This is the failure mode the program calls hallucination (the production of fluent, confident text that is factually incorrect or entirely invented), and it is structural to how these models work, not a bug that a software update will eliminate.

The rule for summarization AI is therefore: grounded generation, not free generation. The model must work only from what was actually documented, and a human must verify every substantive claim in the output before the document enters the case record. This is not a bureaucratic add-on. It is the practice that protects a family.

Type Two: Predictive Screening Models

A predictive screening model is a different kind of AI entirely. It does not read or write language. It takes structured data, numbers and categories, and produces a score or a probability estimate based on patterns learned from historical outcomes. In child welfare, these are often called risk assessment tools or AI risk-screening tools. They might take inputs like the age of children in a household, the number of prior CPS (child protective services) contacts, the type of maltreatment reported, income level, housing stability indicators, and parent age, and produce a number like 0.78, which the system might label as "high risk."

These tools exist and are deployed in child-welfare agencies across the country. The Allegheny Family Screening Tool, deployed in Allegheny County, Pennsylvania, became one of the most discussed examples in the field, generating both serious research interest and serious civil-rights criticism. The criticism is not that the tool performs badly in a narrow statistical sense. The criticism is that the patterns it learned from historical data may reflect, and therefore amplify, historical inequities: over-reporting in low-income communities, over-investigation of families of color, and the disproportionate representation of poverty in the features treated as "risk." A model trained on data that reflects those historical disparities will produce scores that encode those disparities, and if the score drives decisions, the tool amplifies the inequity rather than correcting it.

The lesson from history is not that predictive screening has no place in the field. It is that every risk score is one input under mandatory human review, never a verdict. The score does not decide anything. The caseworker, the supervisor, and where applicable the court decide. The score is additional information that a professional evaluates alongside everything else they know about the case. An agency that uses risk scores as gates, automatically routing cases above a threshold to one track and below to another without human review at the decision point, has quietly handed a life-altering decision to an algorithm, and the people most likely to be harmed by that are the people who were already most likely to be harmed by the system's historical biases.

The equity obligation is explicit and non-negotiable: every AI screening tool in human services requires an equity audit (a systematic review of whether the tool produces different outcomes across racial, ethnic, income, and other demographic groups) before deployment and as a continuous practice during operation. An equity audit is not a one-time vendor certification. It is an ongoing responsibility of the agency and the workers in it.

Type Three: Generative Language Models

A generative language model (the large language model, or LLM, at the heart of tools like the AI chat assistants that became mainstream in the early 2020s) takes a prompt and writes new text. It is not summarizing existing text (though it can do that). It is not scoring structured data (though it can do that too, in a limited way). It is composing: generating a first draft of a document, a letter, an explanation, a plan. In human services, generative AI might draft a court report from a set of documented observations, write a benefits denial explanation, produce a resource referral letter for a client, or outline a service plan.

The creative power of generative models is also their greatest hazard in this field. A model that can write a plausible-sounding court report can write a plausible-sounding court report whether or not the observations it is describing actually occurred. It can write confidently about a family's history, a child's condition, or a parent's compliance with services, drawing on its training in the patterns of how such reports are written. And unlike a summarization model that at least had real input to work from, a generative model given a sparse or vague prompt can elaborate extensively from its general knowledge of what such cases look like. The result is a document that sounds authoritative and professional but may contain observations that were never made, facts that are not in the record, and characterizations that reflect a pattern rather than this family.

The safeguard is called grounding, specifically retrieval-augmented generation (RAG, a technique in which a model is given the actual relevant documents, the case file, the visit notes, the policy text, before generating a response, so it draws on the real record rather than general patterns). A grounded generative model drafting a court report has the caseworker's actual visit notes, the family's actual prior record, and the applicable policy text in front of it before it writes a word. An ungrounded model drafts from general knowledge of what court reports look like. The difference between a grounded and an ungrounded output can be the difference between a defensible legal record and a fabricated one, and in this field, that distinction can determine whether a child stays with a family.

Even a grounded generative model requires human verification before its output enters the record. Grounding reduces the risk of hallucination; it does not eliminate it. The professional operating the model is still responsible for reading the output, checking every substantive claim against the actual record, and standing behind the document as if they wrote every word. Because in a court's view, they did.

What None of Them Do: The Cardinal Rule

Here is the most important thing this lesson will teach, and it applies equally to all three types of AI, to every vendor claim, and to every AI-adjacent decision a human-services professional will ever face.

AI informs, humans decide. Always.

This is not a cautious hedge or a legal disclaimer. It is a substantive, non-negotiable rule about the nature of the decisions in this field. Let's be precise about why.

A decision to remove a child from a home is among the most consequential a government can make about a family. It is bound by due process (the legal requirement that before a person is deprived of a protected interest, they must receive notice and an opportunity to be heard). It is bound by equity (the obligation to ensure that the decision is not influenced by race, income, or other protected characteristics). It is subject to judicial review, to appeals, to challenge by advocates and attorneys. The caseworker who makes that decision, and the supervisor who approves it, and the court that authorizes it, are all accountable for it. They can be questioned, challenged, and if necessary held responsible.

An algorithm cannot be. An algorithm cannot appear in court to explain its reasoning. It cannot be cross-examined. It cannot be held accountable when it is wrong. It cannot weigh the specific, irreducible complexity of this family, in this moment, with all the context a trained professional has observed and understood. A risk score is not an assessment of this child; it is a statistical inference from patterns in data about many other children. A model output is not a judgment; it is the result of a mathematical process applied to a set of inputs. The decision is a professional and legal act, and it belongs to people who can be held responsible for it.

"The model said so" is never, under any circumstances, a sufficient reason for a consequential decision in human services. It is not a defense in court. It is not an explanation to a family. And it is not a professional judgment. It is an abdication of the responsibility that the caseworker, the supervisor, and the agency bear for the people in the system.

This rule has a specific implication for technology procurement. When a vendor describes their tool as "telling you what to do," "making the decision for you," "automating the determination," or "removing the burden of judgment from your workers," those are descriptions of a product that violates the cardinal rule. They are not features. They are risks. A caseworker is not a bottleneck to be removed from the decision process. A caseworker is the decision process. AI is a tool that can make the caseworker better informed, better supported, and faster on the documentation so they have more time for the actual work of being present with families. It cannot replace the judgment that protects those families.

A Concrete Day: What Each Type of AI Looks Like in Practice

Abstract definitions are useful, but the real test is whether you can recognize each type of AI when you encounter it in your actual work. Let's follow a child-welfare caseworker through a realistic day in 2026 and identify each type of AI as it appears.

Morning: Intake Review with Summarization AI

It is 8:15 AM. Keondra is a caseworker in a large county child-welfare agency. She opens her case-management system, Casebook, and sees that five new intake referrals came in overnight. Each referral has been automatically summarized by the agency's AI documentation tool. The summaries compress the intake worker's notes into two-paragraph case backgrounds. Keondra reads the first summary and notices it says the caller "expressed concern about substance use in the home." She pulls up the original intake note and sees the intake worker wrote that the caller "reported seeing the parent appearing disoriented and slurring words." The summary is reasonable, but the original language is more specific and more relevant to safety planning.

What happened here? The summarization model compressed specific language into a more general characterization. That is a legitimate failure mode: compression loses specificity. In a case note, specificity matters, because a court may later ask whether the investigator had reason to believe substance use was present. The specific original language is a stronger evidentiary anchor than the summarized version. Keondra makes a note: use the original language in the investigation record. The summary was a useful starting point, and she verified it against the source. That is the correct workflow for summarization AI.

Midday: Risk Screening Score Review

At 11:30 AM, Keondra is reviewing a new intake for a family with two prior contacts. The agency's screening dashboard shows a risk score of 0.82, displayed with an orange border and a label of "elevated risk." Below the score is a list of the inputs that contributed most heavily: prior contact frequency, household instability flags, age of youngest child (13 months), and a flag for housing benefit disruption.

What does that 0.82 mean? It does not mean there is an 82 percent probability that this child will be harmed. It means that in the historical data used to train the model, cases with these characteristics had outcomes that the model was trained to treat as concerning at a rate that produced this score. The inputs feeding the score include housing benefit disruption, which is a measure of economic stress, not parental intent or capacity. The score is the result of patterns in historical data, and the historical data reflects the demographics of the families that prior workers investigated, which in many systems overrepresents families of color and families in poverty.

Keondra is trained to treat this score as one input. She reads it. She considers it. She asks: what do I know about this family from the actual record, from the caller's account, from the prior contact narratives, that the score cannot capture? Her supervisor is available to consult. The decision to open or close this case is Keondra's, reviewed by her supervisor. The score neither opens nor closes it. If it did, the agency would have delegated a consequential decision to an algorithm, and the families of color and the families in poverty who are overrepresented in the training data would bear the consequences disproportionately.

Afternoon: Court Report Drafting with Generative AI

At 2:00 PM, Keondra needs to complete a court report for a hearing scheduled for Thursday. She has thorough visit notes from four home visits over the past six weeks, service participation records, a school attendance report for the oldest child, and a compliance summary from the family's assigned service provider. She opens her agency's AI court report drafting tool, which uses retrieval-augmented generation (RAG) to pull her visit notes and the service records into a structured context window before generating a draft report.

The draft comes back in four minutes. It is structured correctly, uses the court report format the agency requires, and captures the major themes from the visits. Keondra reads it paragraph by paragraph, checking each factual claim against her visit notes. She finds two issues. First, the draft describes the parent as "consistently engaging with services," but her visit notes indicate one missed appointment in week three that the service provider had flagged as a concern. The word "consistently" is not accurate. Second, the draft attributes a statement to the parent ("the parent reported the home environment has improved") that appears in the records, but Keondra's actual note says the parent "indicated the situation was better than before." Those are not the same claim, and the court report version is more definitive than what was actually said.

Keondra corrects both. She changes "consistently engaging" to "generally engaging with one missed appointment in week three, documented by the service provider." She changes the parent statement to match her original note. She reads the full draft two more times, then signs and submits. The AI saved her approximately ninety minutes of drafting time. The court report reflects what actually happened, because she verified every claim. That is the correct workflow for generative AI in case documentation.

The Lesson from the Day

Notice what each AI type contributed and what remained the caseworker's responsibility. The summarization tool condensed overnight intakes, saving reading time, and the worker verified before acting on any specific claim. The screening tool provided a risk signal, and the worker treated it as one input among many in a human decision. The generative tool drafted a court report, and the worker verified every substantive claim before signing. In each case, the tool did work that a trained professional would otherwise have to do manually, and the professional remained the decision-maker, the verifier, and the person accountable for the output.

Remove the human verification from any of those three steps and you have a different situation entirely. The summarization output enters the record unchecked, and compressed or invented details shape an investigation. The screening score routes the case without review, and historical bias becomes the operative decision-maker. The court report goes to a judge with claims the caseworker cannot personally verify, and a family's fate is influenced by a language model's best approximation of what such reports usually say. The technology is the same in both versions of this story. The professional practice is entirely different.

The Paperwork Burden and Why the Goldmine Matters

Understanding the three types of AI is not only about risk management. It is also about understanding where a genuine, urgent, humane benefit lives. The documentation burden in human services is not a minor inefficiency. It is a structural crisis that has been building for decades and that AI, used correctly, is positioned to address in a meaningful way.

Consider what the research and field experience tell us. Caseworkers routinely report spending half their working day or more on documentation rather than with the families they serve. A caseworker with a caseload of twenty families, spending four hours per day on documentation, spends the equivalent of two full days per week typing instead of being present. That time comes from somewhere: it comes from the family visits that are shortened, the phone calls that do not happen, the relationship-building that creates the trust that makes safe families possible. Burnout and voluntary departure from the field are strongly correlated with documentation burden, and every experienced caseworker who leaves for a less exhausting job takes their judgment, their relationships, and their institutional knowledge with them. Their caseload goes to the worker who remains.

The AI transcription and summarization tools that began entering the field in earnest in the early 2020s, what some agencies call the "Magic Notes" pattern, offered something the field had never had: a way to reduce documentation time without reducing documentation quality, and in some cases, improving it. When a caseworker can dictate their observations during a home visit drive and receive a structured case note draft in minutes, and when they verify that draft against their memory and their audio or written notes, the result can be more complete and better structured than what they would have produced under time pressure. The two or three hours returned per day are not small. They are the difference between a caseworker who has time to make one more home visit, have one more difficult phone conversation, think carefully about one more safety plan, and a caseworker who is typing until 7 PM to meet documentation requirements.

This is the goldmine: AI-assisted documentation that returns time to human connection. It is the use case that is most universal across child welfare, benefits administration, case management for housing and disability and aging, and every corner of the field where documentation competes with presence for the same finite hours. And it is the use case where the risk, correctly managed, is most limited: the model is working from what the caseworker documented, the worker verifies before submitting, and the decision about what to document and how to characterize it remains entirely human.

The phrase "returns time to human connection" is not poetic inflation. It is precise. A caseworker who is not typing is available to be present with a child. A supervisor who is not reviewing documentation backlogs is available to support a worker navigating a difficult case. An agency that has reduced the documentation burden on its workforce is an agency that can attract and retain the people who do the hardest work in the most important jobs the government has to offer.

Asking the Right Questions of Any AI in Your Agency

Maria's discomfort in that procurement meeting was entirely warranted. The vendor's description of their system, "the AI tells you what to do next," was either a misunderstanding of the technology's appropriate role or an attempt to obscure it. Either way, Maria and her supervisor had every right to stop the meeting and ask hard questions. This section gives you those questions.

Before any AI tool is deployed in a human-services context, the professionals who will use it and the supervisors and administrators responsible for the work should be able to answer five questions. These questions are not technical. They are professional and ethical, and the answers determine whether a tool is appropriate for the field.

Question one: What type of AI is this, and what does it actually do? Is it summarizing existing text? Is it scoring structured inputs? Is it generating new text? A vendor who cannot give a clear, jargon-free answer to this question has not earned the agency's trust. The three types operate differently, fail differently, and carry different risks. You need to know which one you are evaluating.

Question two: Where does the human decision point live? What decision does the tool inform, and what happens at the moment of decision? Is there a human, with specific training and accountability, who reviews the tool's output and makes the call? Or does the system route, score, flag, or determine without a human review step built into the workflow? If the answer is that the system "helps" but the human still "technically" has to click approve, ask what happens in practice when workers are managing high caseloads and time pressure. A workflow where a human must affirmatively override an AI recommendation is different from a workflow where a human must actively engage with the case on its merits. The second is what the field requires.

Question three: Has this tool been tested for equity? What does the performance data look like disaggregated by race, ethnicity, income, and geography? Has the vendor conducted an equity audit (a systematic assessment of whether the tool produces different outcomes across demographic groups) and made the results available to the agency? If the tool touches decisions that affect families, it must be evaluated for disparate impact. The history of benefits fraud detection systems in the Netherlands (the Dutch childcare-benefits scandal, where an algorithmic fraud detection system wrongly flagged tens of thousands of families, disproportionately those with dual nationalities or low incomes, for investigation and repayment demands) and unemployment fraud detection in Michigan (the MiDAS system, which generated tens of thousands of erroneous fraud determinations based on automated logic without adequate human review) shows that these systems can cause catastrophic harm at scale when equity is not built into both the design and the ongoing monitoring.

Question four: What is the failure mode, and who is harmed when it fails? Every type of AI fails. Summarization models compress or invent. Screening models encode bias. Generative models hallucinate. The question is not whether the tool will fail but what happens when it does and who bears the consequences. In human services, the people who bear the consequences of tool failure are, by definition, the most vulnerable people in the system: children, families in crisis, people applying for benefits they need to eat and stay housed. The agency's responsibility is to the people it serves, not to the vendor's accuracy statistics.

Question five: How does the agency maintain oversight, transparency, and accountability? Can the agency explain to a court, to an advocate, to a client, and to the public how AI was used in a given decision? Is there an audit trail showing what AI output was produced, what the worker reviewed, and what the human decision was? Due process (the legal guarantee that consequential decisions are made with notice and an opportunity to respond) requires that the people affected by decisions can understand and challenge them. "The AI said so" is not an explanation that satisfies due process. The agency needs to be able to articulate the human judgment behind every consequential decision, and to demonstrate that AI informed rather than replaced it.

Maria and her supervisor deserve credit for their instinct in that procurement meeting. The discomfort they felt was professional clarity cutting through vendor language. The questions above are how that clarity becomes a conversation that either gets honest answers or reveals that the vendor cannot provide them. In either case, the agency is better informed than before.

Putting It Together: The Vocabulary of Professional Practice

This lesson has introduced a set of terms that will appear throughout this program and, if you carry them into your professional practice, throughout your career. Let's collect them into a working vocabulary that you can use right away.

Summarization model: An AI that reads existing text and produces a condensed version. In human services, used to draft case note summaries, compress intake records, and support documentation workflows. Failure mode: compression loses specificity; in some conditions, the model invents details not present in the source. Correct use: as a starting draft, verified by the worker before submission.

Predictive screening model: An AI that takes structured inputs and produces a score or probability estimate based on patterns in historical data. In human services, used in risk assessment tools and safety screening tools. Failure mode: encodes and amplifies historical inequities in the training data. Correct use: as one input under mandatory human review, subject to equity auditing, never as a decision-maker.

Large language model (LLM): An AI trained on massive text data to predict and generate human-like written language. In human services, used to draft court reports, case notes, client letters, and eligibility summaries. Failure mode: hallucination, the generation of fluent, confident text that is factually incorrect or invented. Correct use: as a grounded draft tool, verified claim by claim before the document enters the record.

Hallucination: The specific failure mode of generative AI, in which the model produces text that sounds authoritative and professional but is factually incorrect or entirely invented. Not a bug to be fixed; a structural property of how these models work. The only reliable control is human verification.

Grounding and retrieval-augmented generation (RAG): The practice of providing a generative model with the actual source documents before it generates output. A grounded model draws on the real case record, the real visit notes, the real policy text. An ungrounded model draws on its general knowledge of what such documents usually say. Grounding reduces hallucination risk; it does not eliminate it.

Equity audit: A systematic assessment of whether an AI tool produces different outcomes across demographic groups, including race, ethnicity, income, and geography. Required before deployment and as ongoing practice for any tool that influences decisions affecting families. The history of algorithmic harm in human services, from the Allegheny screening debates to the Dutch and Michigan fraud-detection scandals, makes this non-negotiable.

Due process: The legal and constitutional guarantee that before a consequential decision is made about a person, they receive notice of it and an opportunity to be heard. In human services, due process governs decisions to remove a child, substantiate a report, or deny benefits. AI that makes these decisions without a human accountable for them violates due process.

Decision-aid: The correct role of AI in human-services decisions. A decision-aid provides information, analysis, drafts, or signals to a human professional who then exercises judgment and makes the decision. A decision-maker is a system that makes the decision itself. AI in human services must be a decision-aid, never a decision-maker.

The cardinal rule: AI informs, humans decide. The caseworker, the supervisor, and the court own every consequential call. Always.

These terms are not vocabulary for their own sake. They are the language that lets you participate in professional conversations about AI with clarity, identify risks that others may have missed, and hold vendors and platforms to the standard the people in the system deserve. The professional who can use these terms precisely is the professional who can advocate for responsible practice when it is under pressure from convenience, cost savings, or vendor enthusiasm.

Key Takeaways

• There are three distinct types of AI in human-services contexts, not one. Summarization models compress existing text. Predictive screening models score structured data against historical patterns. Generative language models (LLMs) write new text from a prompt. They fail in different ways, carry different risks, and require different safeguards. Conflating them is a professional error with real consequences.
• Summarization models are the most widely adopted and most immediately humane use case in the field: AI-assisted documentation (the Magic Notes pattern) returns hours to direct work with families, addressing the documentation burden that is the single largest driver of caseworker burnout and turnover. The safeguard is grounded generation and human verification before the document enters the record.
• Predictive screening models produce scores based on patterns in historical data, not assessments of this child or this family. They can encode and amplify historical inequities in their training data, making equity auditing before deployment and as continuous practice non-negotiable. Every risk score is one input under mandatory human review; it is never a verdict.
• Generative AI models can write a court report that sounds authoritative and completely accurate while containing details that were never observed. This failure mode, hallucination, is structural to how LLMs work and is not eliminated by grounding, though grounding (especially RAG, retrieval-augmented generation, which gives the model the actual case record before generating) substantially reduces the risk. Human verification of every substantive claim is required before any AI-drafted document enters the case record.
• The cardinal rule is absolute: AI informs, humans decide. The decisions in this field, to remove a child, substantiate a report, or deny benefits, are among the most consequential a government makes about a person's life. They are bound by due process, equity, and professional accountability. An algorithm cannot appear in court, cannot be cross-examined, and cannot be held responsible when it is wrong. The caseworker, supervisor, and court bear that responsibility and must exercise it.
• Due process and equity are not aspirational values that apply when convenient. They are legal and ethical obligations that govern every decision in the field. AI that quietly becomes the decision-maker, rather than a decision-aid, violates both. Every AI procurement, deployment, and ongoing practice must be evaluated against the question: is a human accountable for every consequential decision, and is the process defensible to a court, an advocate, and the family affected?
• The five questions every human-services professional should ask of any AI tool are: What type is it and what does it actually do? Where does the human decision point live? Has it been tested for equity? What is the failure mode and who is harmed? How does the agency maintain oversight, transparency, and accountability? These questions are not technical; they are professional and ethical, and vendors who cannot answer them honestly have not earned the agency's trust.
• The feeling that Maria and her supervisor had in that procurement meeting, that something was deeply wrong with "the AI tells you what to do" framing, was professional clarity. Naming it, with the vocabulary this lesson provides, is how that clarity becomes defensible professional practice. The workers in this field came to help families, protect children, and get people housed and fed. AI that returns hours from documentation to that mission, operated with the discipline that keeps every decision human and every outcome equitable, is an instrument in that mission. AI that quietly displaces human judgment is a threat to it.