What Artificial Intelligence Actually Is

Let's start with a definition that moves beyond the hype:

Artificial Intelligence (AI) is a software system designed to learn patterns from data and use those patterns to perform tasks, make predictions, or generate content—without explicit step-by-step instructions for every possible case.

This definition contains three critical ideas, so let's unpack each one.

Learning from Data, Not Following Rules

Traditional software is deterministic. A financial system calculates interest based on explicit formulas. If the interest rate is 5%, every account at 5% receives the exact same calculation. You can read the code, line by line, and understand what it does. There are no surprises.

AI is different. Instead of a programmer writing rules (if X then do Y), an AI system is trained on examples. It observes patterns in those examples and uses those patterns to handle new cases it has never seen before.

Consider a fraud detection scenario. Rather than a compliance officer writing 100 rules for suspicious transactions—"if amount exceeds $500k, flag it; if vendor is new, flag it; if transaction occurs outside business hours, flag it"—an AI system is fed thousands of examples of normal and fraudulent transactions. The system learns the implicit patterns that distinguish legitimate from suspicious activity. It might discover that the combination of a new vendor, an unusual amount, and timing outside standard business hours is more suspicious than any single factor alone.

This capability is powerful precisely because humans cannot express the pattern as an explicit rule. It's also why AI differs fundamentally from automation.

Probabilistic Output, Not Deterministic Certainty

Here's where many governance professionals stumble: an AI system does not "know" the right answer. It calculates the likelihood of an answer based on patterns in its training data.

Run the same transaction through a fraud detection model twice, and you may get slightly different risk scores—not because the system is broken, but because this is how learning-based systems operate. They output probabilities, not certainties.

This probabilistic nature has profound governance implications. With traditional software, you can test it: run the same input 100 times, get the same output 100 times, verify consistency. With AI, you cannot guarantee consistency. What you can do is measure accuracy: over 1000 test cases, does the model correctly classify 85% of them? That becomes your confidence metric.

The Black Box Challenge: Opacity at Scale

Modern AI systems, especially large neural networks, operate as "black boxes." You can see the input ("this transaction has these characteristics") and the output ("risk score: 0.78"), but the reasoning path in between is largely opaque.

This is not a flaw that can be engineered away; it's a consequence of how large AI systems work. When a system has billions of parameters—each one a tiny weight in a mathematical model—no human can trace through that computation and explain, "at step 47, the model considered factor X and weighted it by Y." The complexity is simply beyond human comprehension in real time.

This raises a governance challenge: if you cannot explain why an AI system made a decision, how do you defend that decision to a regulator? To an auditor? The answer is not to abandon AI, but to build stronger human oversight structures around it.

Types of AI and Scope

Narrow AI: The Only AI That Exists Today

Every AI system in use today—whether it flags fraudulent transactions, summarizes regulatory guidance, or generates marketing copy—is narrow AI. These systems excel at specific, well-defined tasks but cannot generalize to unrelated problems.

A fraud detection AI trained on payment transactions cannot analyze organizational culture. A document classifier trained on policy documents cannot predict market trends. Each system is narrow.

This is important because vendors sometimes claim to be building "general" AI—systems that reason across domains the way humans do. These claims are either overselling or, frankly, misinformed. General AI does not exist. The distinction matters for governance: you are not facing an AI system that will suddenly "learn" to do things outside its training domain.

Machine Learning: The System That Improves Itself

Machine Learning (ML) is a category of AI where the system improves its own accuracy by analyzing data, rather than being programmed with explicit rules.

Here's the workflow: you provide training data (examples of what you want the system to recognize). The system detects patterns in those examples. When tested on new data, it uses those patterns to make predictions. If it's wrong on some predictions, you can adjust the training data or the system's parameters, and it tries again. Over iterations, the system's accuracy improves.

In audit contexts, this is powerful. Instead of writing 100 rules for suspicious transactions, you feed an ML system thousands of examples of normal and fraudulent transactions. The system learns. But here's the governance reality: the system can learn harmful patterns too. If your training data reflects historical bias (e.g., "transactions from Department A were fraudulent more often"), the AI will learn and replicate that bias in real time.

Large Language Models: Pattern Prediction at Scale

Large Language Models (LLMs) like ChatGPT, Claude, or Gemini are a specific type of AI trained on vast amounts of text—billions of words from the internet, books, and other sources.

Here's how they work, in simplified terms: they learn statistical patterns about language. "When people write 'Dear Sir or Madam,' the next word is usually not 'banana.'" When you give an LLM a prompt, it generates text by predicting the next most likely word, then the next, then the next—iteratively building a response, one word at a time.

This is remarkable technology. LLMs can write coherent sentences, summarize complex documents, draft policy language, and answer questions in natural language. But here's the critical governance limitation: LLMs do not "know" facts. They predict statistically likely text. They are excellent at generating plausible-sounding text. They are not excellent at distinguishing true from false information. This limitation—the tendency to generate convincing falsehoods, called "hallucination"—is perhaps the most important constraint for oversight professionals to understand.

Generative AI: Content Creation from Prompts

Generative AI refers to systems that produce new content—text, images, code, summaries—from a prompt or instruction, rather than just classifying existing data or making predictions.

This is the category you are most likely to encounter in your organization. Text generation for document summarization. Code generation for faster development. Image generation for marketing. All of these fall under generative AI.

Where AI Actually Gets Used in Risk and Governance Work

The abstract definition is necessary, but so is the practical grounding. Here are real scenarios where AI shows up in audit, compliance, risk, and governance contexts:

Analyzing Risk Assessments for Common Themes

Your organization collects risk assessments from business units annually. Each one is narrative—written text describing the risks the unit faces. Instead of a compliance officer manually reading 50 assessments to identify common themes, an ML system can analyze the text and surface patterns: "Eight units mention cybersecurity risk; three mention supply chain disruption." This alerts you to emerging risks across the organization.

Drafting Policy Language with Required Expert Review

A new regulatory requirement arrives. Your compliance officer is stretched. An LLM can generate a first draft of policy language in minutes—not a final policy, but a starting point. The compliance officer reviews the draft against the actual regulation, makes corrections, and then the organization has a baseline to work from.

Classifying Control Documentation at Scale

Business units submit hundreds of control descriptions annually. Each one needs to be categorized as a preventive control, detective control, corrective control, etc. An AI system trained on examples can classify these in hours. Auditors then validate the classifications, spot-checking for accuracy. The AI provided the speed; humans provided the judgment.

Summarizing Board Materials for Executive Briefings

Your board receives a 100-page risk register. An LLM can generate a one-page executive summary highlighting the top 5 risks and their business impact. The Chief Risk Officer reviews the summary for accuracy, adds context if needed, and presents it to the board.

Three Examples: AI in Action and What to Look For

Example 1: Fraud Detection in Payment Monitoring

The Scenario: Your organization processes thousands of vendor payments monthly. You need to detect suspicious transactions before they are paid.

How AI Approach Works:

• Historical transaction data (5 years) is used to train an ML system: 99% normal transactions, 1% confirmed fraud

• The system learns implicit patterns of "normal" (regular employees, expected vendors, typical monthly amounts) vs. "unusual" (high velocity, new vendors, amounts outside historical range)

• When a new transaction arrives, the system scores it against learned patterns and flags those above a threshold for manual review

Why This Is AI, Not Automation: You cannot write a simple rule ("if amount > $100k, flag it") because not all large transactions are fraud, and some fraud involves many small transactions. The AI detects subtle pattern combinations that explicit rules would miss. It is probabilistic: it scores risk, not certifies fraud.

Governance Implications: The flagged transactions are not confirmed fraud; they are signals for investigation. The threshold for flagging is a governance choice (higher threshold = fewer false alarms, more missed fraud). The system can learn bias: if fraud in training data came disproportionately from a certain department, the system may over-flag that department in the future.

Example 2: Compliance Summarization with Expert Validation

The Scenario: A new data privacy regulation is issued. Your compliance team must assess what obligations it creates and what changes are required.

How AI Approach Works:

• Full regulatory guidance (50+ pages) is fed into an LLM

• The LLM generates a draft summary: key obligations, implementation timelines, affected business processes

• Your compliance team reviews the summary sentence by sentence, checking it against the actual regulation

• Only after validation is the summary used for impact assessment

Why This Is AI: Regulation interpretation is not deterministic. You don't have an algorithm for "what is the essence of this regulation." The LLM infers emphasis and importance from language patterns. Different prompts might yield slightly different summaries. But the speed is valuable: what might take a compliance officer two days to outline, the LLM does in seconds.

Governance Implications: The AI summary is a draft, not the judgment. Your team must verify that the summary is accurate (the LLM might misinterpret or hallucinate obligations). You must document that human expert review occurred before the assessment is used in governance.

Example 3: What Is NOT AI (But Might Sound Like It)

Scenario 1: A Rules Engine for Payment Approval

A system flags all payments > $500k for approval. Why is this not AI? The logic is explicit: if amount > 500000, then flag. The system always behaves the same way for the same input. You can read the rule and understand the behavior completely. This is automation.

Scenario 2: Pivot Tables and Variance Analysis

A financial analyst creates a pivot table showing transaction volumes by vendor and month, then compares actual to budget to find variances. Why is this not AI? The analyst is using mathematical formulas (sum, average) to transform and compare data. No learning from data is occurring; fixed logic is being applied. This is analytics or traditional data analysis.

Scenario 3: A SQL Query for Compliance Checking

A database query returns "all payments to vendors created in the last 30 days." The logic is deterministic (created_date within last 30 days) and explicit (vendor created_date). The results are reproducible and auditable. This is data retrieval with rules, not AI.

The Governance Takeaway: The distinction matters because each category requires different controls. A rules engine is controlled through change management and testing. An AI system is controlled through data quality, bias monitoring, and performance tracking. Applying the wrong controls to the wrong category creates blind spots.

What AI Cannot Do Reliably (And Why This Matters)

Understanding AI's strengths is necessary. Understanding its limitations is essential.

Cannot Distinguish True from False Information

This is the single most important limitation for governance professionals to internalize.

LLMs are trained to predict statistically likely text. They are not trained to fact-check. An LLM will confidently generate:

• False definitions presented as authoritative

• Completely fabricated citations to regulations that don't exist

• Plausible-sounding but incorrect interpretations of policy

• Made-up examples that feel entirely real

This is called "hallucination," and it is not a bug. It is a fundamental feature of how LLMs work. They generate the next statistically likely word, regardless of whether the word is true.

A concrete example: you ask an LLM, "What does SOX Section 404 require?" The system generates a confident, well-formatted response. It sounds authoritative. But it may include false requirements or miss critical nuances. You cannot trust it without verification by a human who has actually read SOX.

Cannot Understand Context the Way Humans Do

AI systems operate on statistical patterns extracted from training data. They do not have semantic understanding or intuition. They fail in subtle, context-specific ways:

• Missing significance: The system misses the significance of a rare detail because it's uncommon in training data

• Over-generalization: The system over-applies a pattern that is actually context-specific

• Jargon misinterpretation: The system misinterprets professional terminology if it appears rarely in training data

Suppose you deploy an AI system to classify audit findings as "high," "medium," or "low" severity. It classifies a finding about a missing signature on a sensitive approval as "low." Why? In the training data, signatures were statistically not correlated with severity. The system missed the organizational context: in your company, that signature is a key control. Without it, a major risk exists.

Cannot Explain Its Decisions at Scale

Modern AI systems, especially large ones, provide no human-readable explanations for how they decided something. You see the input and output, but the intermediate reasoning is opaque.

Some systems offer "explanations" (e.g., "this transaction was flagged because it exceeded the typical amount threshold and involved a new vendor"). But these are post-hoc rationalizations, not actual explanations of how the neural network computed its decision. True explainability at scale remains an unsolved problem in AI.

Governance implication: if you need to defend a decision to a regulator or audit, "the AI decided it" is insufficient. You must have either a human explanation or measurable evidence supporting the decision.

Cannot Generalize Beyond Its Training Context

An AI system trained on one type of data often fails entirely when applied to different data.

A fraud detection model trained on US transactions (dollar denominations, standard business hours, familiar vendor ecosystems) may perform poorly on international transactions (different currencies, business norms, unfamiliar vendors). The model was not trained on that data, so it never learned those patterns. It cannot generalize.

This has direct implications for vendor solutions: a fraud detection model effective at another financial institution may not work for yours if your transaction patterns differ significantly.

Cannot Adapt to Change Without Retraining

Once an AI system is trained, it is frozen. If the environment changes significantly, the system's performance degrades without deliberate retraining.

A fraud detection model trained on pre-pandemic transaction patterns may misclassify pandemic-era transactions as anomalous, generating false positives. The model doesn't "learn" from new data in real time. It requires deliberate retraining—feeding it new data, adjusting parameters, validating results.

Human Judgment Checkpoints Before Accepting AI Systems

Before an AI system is deployed in your organization, governance should require these checkpoints:

1. Verify the Problem Actually Requires AI

Does this task actually need learning and pattern detection? Or would a rule work? If you can write the logic explicitly, you probably don't need AI. Save the complexity for cases where AI truly adds value.

2. Understand the Training Data

What data was this AI trained on? Does it represent your organization's context? A fraud model trained on US transactions may fail internationally. A document classifier trained on formal policy documents may fail on informal control descriptions. Ask the vendor or team: "What is in the training data, and how representative is it to our use case?"

3. Assess for Bias

What groups or categories appear in the training data? Are any overrepresented or underrepresented? If your training data reflects historical bias, the AI system will learn and replicate it. Ask: "Has bias testing been done? What was found and how was it addressed?"

4. Define Success Metrics

How will you measure whether the AI is working correctly? Don't rely on vague assessment ("it seems to give reasonable answers"). Define measurable success: accuracy on your data, false positive rate, false negative rate, performance by category or demographic group.

5. Require Explainability Threshold

For this specific use case, can someone explain why a decision was made? Not all AI systems can provide explanations, but more important for high-stakes decisions (regulatory determinations, fraud findings, vendor decisions) is the ability to explain the reasoning to stakeholders and regulators.

6. Establish Human Review Process

Who investigates when the AI is wrong? How frequent are errors? Is there a process to escalate and learn from mistakes? Human review is not a weakness; it's a structural control.

Traceability and Defensibility: Documenting AI in Governance

If you cannot tell an auditor or regulator how a conclusion was reached and who is accountable for it, the conclusion is not ready for use.

In Audit Work

If you use AI to generate a summary of a risk assessment, document which human reviewed and validated it, when, and on what basis. The AI output is input to the audit process, not the audit conclusion itself.

In Compliance Work

If AI assists in classifying policies or identifying control obligations, retain the training data, validation process, and documentation of human review. For regulatory submissions, AI-generated content must be reviewed and attested to by a qualified human.

In Risk Work

If AI detects risks, document the threshold for escalation, the false-positive rate, and the review process. Governance must understand that the AI system is a detection tool, not an arbiter of risk.

Key Takeaways: The Foundation You Need

• AI is learned behavior from data, not programmed logic. This is the fundamental difference from traditional automation. Governance implications are profound.

• Today's AI is narrow. It excels at specific tasks; it cannot generalize like humans. Each AI system is designed for a bounded scope.

• LLMs generate plausible text, including false text. They predict the next word; they don't verify facts. Hallucination is a feature, not a bug.

• AI is probabilistic, not deterministic. Identical inputs may yield slightly different outputs. This is not a malfunction; it's how learning-based systems work.

• Opacity is a governance challenge, not a reason to trust blindly. Strong oversight is required precisely because the decision logic is not human-readable.

• AI matters most for problems that are too complex for explicit rules. For simple, rule-based problems, automation or analytics is often sufficient.

• Governance discipline requires precision. Distinguish AI from automation. Distinguish capability from hype. Distinguish confidence from accuracy.