Understand how LLMs are trained, how they generate text, and what determines their capabilities and limitations.

You're Already Using the Output—Now Understand the Engine

If you've used an LLM to draft a document, debug code, summarize a report, or answer a question, you already have an intuition for what these systems can do. But there's a gap that shows up fast in professional AI work: people who only understand the output struggle to predict behavior, set realistic expectations, diagnose failures, or make sound decisions about where to deploy these tools—and where not to.

This lesson closes that gap. We're not going to wade through graduate-level mathematics. We're going to build the mental model that experienced AI practitioners actually use: the one that tells you why an LLM behaved the way it did, and what you can do about it.

Why This Matters for AI Practitioners

LLMs sit at the center of nearly every significant AI product being built right now. They power assistants, agents, code tools, search interfaces, and document workflows. If you're evaluating vendors, designing systems, writing prompts, governing deployments, or advising stakeholders—you need a working model of what's happening inside the box.

The practitioners who cause the most expensive problems are not usually the ones who know too little about AI in general. They're the ones who know just enough to be confident, but have a critical misconception about how LLMs generate text. A misunderstanding about determinism, about what "understanding" means, or about how training data shapes outputs can lead to real failures: trust in outputs that shouldn't be trusted, surprise at failures that were predictable, or architectural choices that don't hold up under load.

Understanding how LLMs work is not academic background. It's operational knowledge.

Core Concepts

Training: Learning Statistical Patterns From Enormous Text Corpora

LLMs are trained by exposing a neural network to massive quantities of text—web pages, books, code repositories, scientific papers, forum discussions—and teaching it to predict what comes next. Given a sequence of tokens (roughly, word fragments), the model learns to assign probabilities to what token is likely to follow.

This is called next-token prediction, and it is deceptively simple. The model is not given explicit rules about grammar, facts, or reasoning. It learns these implicitly by being penalized for bad predictions and rewarded for accurate ones, billions of times over. What emerges from that process is a network with hundreds of billions of parameters—numerical weights—that encode a compressed representation of patterns across an enormous swath of human-generated text.

The training process has two major phases most practitioners should know: pre-training, where the model learns from raw text at scale, and fine-tuning (often including techniques like RLHF—Reinforcement Learning from Human Feedback), where the model is further shaped to be helpful, safe, and responsive to instructions. The model you interact with has been through both.

The Transformer: What Makes Modern LLMs Possible

The architectural breakthrough that enabled modern LLMs is the transformer, introduced in the 2017 paper "Attention Is All You Need." The key innovation is the attention mechanism: a way for the model to weigh the relevance of every other token in the input when processing any given token.

In practical terms, this means the model can handle long-range dependencies in text—the word "it" at the end of a paragraph can be correctly linked back to the noun it refers to at the beginning, even across hundreds of tokens. Earlier architectures (recurrent networks) struggled with this. Transformers handle it naturally, and at scale.

You don't need to understand attention weights mathematically to use this knowledge. What matters is the implication: context length is not a trivial engineering detail. The transformer attends to everything in its context window simultaneously. More context means more computation. Context window limits are a fundamental architectural constraint, not just a product decision.

Inference: How Text Is Actually Generated

When you submit a prompt, the model does not "think" and then type an answer. It generates one token at a time, each time running the full forward pass of the network to produce a probability distribution over its vocabulary, then sampling from that distribution to select the next token. That token is appended to the sequence, and the process repeats.

This has several non-obvious implications:

• LLMs are not retrieving stored answers. They are constructing responses token-by-token based on learned patterns. There is no lookup table. Every generation is a fresh computation.
• Temperature controls randomness. At temperature 0, the model always picks the highest-probability token (deterministic). At higher temperatures, lower-probability tokens get selected more often—increasing creativity and variation, but also increasing the chance of incoherence. This is a dial, not a switch.
• The model has no persistent memory between sessions (absent explicit tooling). Each conversation starts from a blank context. What feels like "memory" in a chat interface is usually the full conversation history being re-submitted as context on every turn.

Knowledge Cutoffs, Grounding, and What the Model "Knows"

LLM knowledge comes from training data, which has a cutoff date. The model has no awareness of events after that date unless information is provided in the prompt or retrieved via tools. This is not a bug—it's a fundamental property of the training paradigm.

More subtly: the model doesn't "know" things the way a database does. It has internalized statistical associations. For well-represented topics with consistent, accurate training data, those associations are highly reliable. For niche, technical, recently-changed, or contested information, they are much less so. The model's confidence in its output is not calibrated to its actual accuracy—which is why hallucination (generating plausible-sounding but incorrect content) is a structural property of LLMs, not an occasional bug to be patched away.

Real-World Examples

Retrieval-Augmented Generation (RAG)

One of the most common practical architectures built on top of LLMs is RAG. The insight behind it follows directly from understanding how LLMs work: the model's internal knowledge is limited and potentially stale, but it is extremely good at reading, synthesizing, and reasoning about text provided in its context window. So instead of relying on trained knowledge, you retrieve relevant documents at query time and inject them into the prompt.

The LLM's job shifts from "recall the answer" to "read these sources and produce a response." This dramatically reduces hallucination for factual questions and allows the system to work with current, proprietary, or specialized information the model was never trained on. Understanding the generation mechanism is what makes RAG's design logic obvious—rather than seeming like an arbitrary engineering choice.

Why Prompt Wording Changes Outputs Significantly

Practitioners sometimes find it surprising—or frustrating—that small changes in prompt wording produce noticeably different outputs. This makes complete sense once you understand next-token prediction. The model is not parsing your intent and then generating the best possible response. It is generating a continuation that is statistically consistent with the sequence you provided. The framing, vocabulary, and structure of your prompt all shape what continuations are probable.

Asking "What are some issues with this approach?" primes a different distribution than "What are the fatal flaws in this approach?"—even though you might mean the same thing. This is why prompt engineering is a real skill, and why systematic prompt testing matters in production systems.

Where People Get This Wrong

Practical Takeaways

These are the things that should change how you work after internalizing this lesson:

• Design for hallucination, not against it. Assume the model will sometimes generate incorrect information confidently. Build workflows that include verification steps, source grounding, or human review for high-stakes outputs—not as an afterthought, but as a structural element.
• Use the context window deliberately. What you put in the prompt, and how you structure it, directly shapes what the model generates. Front-load important instructions. For long contexts, consider whether critical information might be getting deprioritized by position.
• Treat temperature and sampling settings as a lever, not a default. For factual, deterministic tasks (classification, extraction, structured output), lower temperatures reduce variance. For generative, creative tasks, higher temperatures produce more useful diversity. Know what you're tuning and why.
• Know the knowledge cutoff and plan around it. If your use case involves recent events, proprietary data, or frequently-updated information, the model's internal knowledge is not the right source. Use RAG, tool calling, or prompt injection—don't expect the model to know what it couldn't have learned.
• Evaluate empirically, not impressionistically. LLM behavior is probabilistic. One impressive output doesn't mean the system is reliable; one bad output doesn't mean it's useless. Build evaluation datasets and measure systematically before drawing conclusions about capability.

Before You Move On

Make sure you can answer these questions with confidence before proceeding to the next lesson:

• What is next-token prediction, and why does it explain both the fluency and the hallucination tendencies of LLMs?
• What is the role of the attention mechanism, and what does context window length actually constrain?
• Why does prompt wording affect output—and what does that imply about prompt engineering as a practice?
• What is a knowledge cutoff, and what architectural pattern is commonly used to work around it?
• What does temperature control, and when would you want it higher versus lower?