โ†
AI for Pharma & Life Sciences
Aware ยท M16 ยท lesson 16 of 17 ยท queued
Preview โ€” browse every lesson free. Enroll to mark lessons complete, open partner links and save your progress. Login & enroll โ†’
What Large Language Models Actually Do in a Submission Workflow
๐Ÿ“–
now learning

What Large Language Models Actually Do in a Submission Workflow

15 min

It is a Tuesday morning in a sponsor's regulatory writing suite, and a Module 2.5 lead has just pasted the synopsis and the integrated efficacy section of a Phase 3 oncology Clinical Study Report into the enterprise large language model, with a single instruction: draft the Module 2.5.4 efficacy section of the Clinical Overview. Eleven seconds later, a clean, confident, beautifully structured draft appears. It reads like something a senior writer produced on a good day. It also contains a citation to Table 14.2.1.4 for progression-free survival in the intent-to-treat population, and that table does not exist in the final TLF package. To know why that happened, and why it is the single most dangerous thing an AI can do to a dossier, you have to understand what the model actually did in those eleven seconds. Not the marketing version. The mechanical version. This lesson traces the path of the text from prompt to draft, shows you exactly where it can go wrong, and explains why that matters for ICH E3 Section 11 alignment and for the 21 CFR Part 11 audit trail that will be read on Day 74 of the review cycle.

What the Model Is Doing When It Drafts Your 2.5

The first thing to internalize is that a large language model does not retrieve facts and then phrase them. It generates text one token at a time. A token is a chunk of language, sometimes a whole word like "efficacy," sometimes a fragment like "hazard" plus "ratio," sometimes a piece of punctuation or a number. The model holds a probability distribution over its entire vocabulary, which in 2026 production systems runs from roughly fifty thousand to two hundred thousand tokens, and at each step it samples the next token from that distribution, conditioned on every token that came before, including your prompt and everything it has already written.

When you ask for a Module 2.5.4 efficacy summary, the model is not consulting a database of your trial. It is producing the most statistically plausible continuation of the text you gave it, shaped by the patterns it learned from an enormous training corpus of clinical literature, regulatory documents, journal articles, and general prose. The reason the draft reads like a competent medical writer is that the training corpus was full of competent medical writing. The reason it can cite a table that does not exist is that the training corpus was full of tables cited in exactly that format, and "Table 14.2.1.4" is a statistically plausible string in a document about progression-free survival, whether or not that specific table is in your specific package.

This is the central mental model for everyone in this program. The model is a fluent pattern completer, not a reasoning engine with access to ground truth. It is extraordinary at producing the shape of a correct answer, because the shape is what it learned. Whether the answer is true is a separate question that the architecture does not, by itself, answer. The named author of the Clinical Overview owns the gap between the shape and the truth.

The Context Window Is the Model's Entire World

Everything the model can attend to lives inside the context window, the maximum number of tokens it can hold at once. In 2026 commercial deployments this ranges from around thirty-two thousand tokens to over a million, depending on the model tier and the sponsor's enterprise configuration. Whatever you load into the window is what the model can see. Whatever you do not load is invisible to it, and the model will not tell you it is missing.

This is the most underappreciated mechanic in regulated writing. If you paste the CSR synopsis and the integrated efficacy section but not the final TLF package, the model cannot verify a single table number against the actual tables, because the actual tables are not in its world. It will still produce table citations, because the pattern of a 2.5.4 efficacy section includes table citations, and it has no way to know that Table 14.2.1.4 is real and Table 14.2.1.5 is not. It does not experience the absence of the tables as a gap. It experiences nothing. It completes the pattern.

The same constraint runs in the other direction. A 600-page CSR with all appendices, a full TLF package, the Statistical Analysis Plan, and the protocol may not fit a smaller context window at all, and a window stuffed past its limit silently drops the oldest content. The writer who believes they loaded the entire CSR, when in fact the appendix 16.1.9 subject-level listings were truncated, is operating on a false sense of completeness. Retrieval-augmented generation, which pulls only the relevant chunks from a vector store into the window, exists precisely to manage this, and it is the subject of an entire Level 3 chapter. For now, the rule is simple and absolute: the model reasons only over what is in the window, and a missing source is not a flag, it is a silence.

Where the Text Actually Comes From, Section by Section

Walk through the draft 2.5.4 the model produced, and you can locate the origin of every sentence on a spectrum from grounded to invented. This spectrum is the writer's map of risk.

The structural scaffolding, the headings, the standard transitions, the phrasing conventions of an efficacy summary, comes almost entirely from the training corpus. This is the safest material, because the conventions of ICH E3 and ICH M4E are stable and well represented. When the model writes "The primary efficacy endpoint was progression-free survival, assessed by blinded independent central review," it is reproducing a phrasing pattern that is correct for thousands of oncology trials, and it is very likely correct for yours, assuming you told it the endpoint.

The specific factual claims, the hazard ratio, the median PFS in months, the confidence interval, the p-value, the number of events, come from one of two places, and the difference is everything. If those numbers were in the text you loaded, the model is transcribing them, and transcription can still go wrong, because the model can misread a value, swap an arm, or carry a number from one subgroup to another. If those numbers were not in the text you loaded, the model is generating a plausible value, and a plausible hazard ratio of 0.68 is indistinguishable, on the page, from the true hazard ratio of 0.71. Both look right. Only one is.

The cross-references, the citations to specific TLF tables, to CSR sections, to Module 2.7.3 sub-summaries, are the highest-risk material of all, and the reason is subtle. A cross-reference is a claim about the structure of a document the model often cannot see. "As shown in Table 14.2.1.4" is a claim that a table with that number exists and contains that result. The model produces the citation because the pattern demands one, not because it checked. This is why a fabricated TLF cross-reference is more dangerous than a fabricated sentence of prose: it survives spell-check, it survives a hasty reviewer, and it is caught only at the reference-QC stage of the publishing cycle, by which point the false citation may have been propagated into Module 2.7.3 and the integrated summary four times over.

Why the Same Prompt Gives a Different Draft

Run the same prompt twice and you will usually get two different drafts. The reason is a setting called temperature, the knob that controls how the model samples from its probability distribution. At a temperature of zero, the model always picks the single most probable next token, and output is close to deterministic for a fixed prompt and model version. At the common default of around 0.7, the model samples from the distribution of plausible next tokens, which introduces variation, and at 1.0 or above, variation widens and the rate of confident invention climbs.

Most enterprise medical-writing deployments tune temperature into a low-to-moderate band to balance natural prose against factual stability, and the writer usually cannot move the knob, because the product team set it. But the consequence for the writer is profound and is often missed: because output varies run to run, your documentation has to capture the specific run that produced the text you kept. The prompt alone is not enough. The audit trail needs the prompt, the system prompt, the model and version, the temperature, the timestamp, the sources loaded, and the final human-verified artifact. "We used the company AI" is not a record that survives an Office of New Drugs Information Request. The specific run is the record.

There is a deeper point hiding in temperature. The variation is not the model being creative in a human sense. It is the model exploring a probability landscape that has many plausible continuations, only some of which are true. The fact that two runs disagree about the median PFS is itself a signal: at least one of them is wrong, and possibly both. Disagreement across runs is one of the cheapest hallucination detectors you have, and a Level 3 chapter turns it into a protocol.

The System Prompt You Never See

Before your instruction ever reaches the model, the enterprise deployment prepends a system prompt, a block of instructions written by the product or platform team that sets the model's role, constraints, tone, and output format. A well-built medical-writing system prompt tells the model to write in the sponsor's house style, to align section structure to ICH E3 and ICH M4E, to refuse to invent reference numbers, to flag missing data rather than fill it, and to cite only sources present in the provided documents. A poorly built one says "follow ICH guidance" without naming which guidance, which gives the model latitude to produce something that looks compliant and is not.

You generally cannot see the system prompt, and you should know that it exists and that it is doing a great deal of the work that determines whether the draft is safe. Two deployments of the same underlying model can behave very differently because one has a system prompt that forces citation discipline and one does not. When a tool consistently produces clean, source-grounded drafts, that is usually the system prompt and the retrieval layer earning their keep, not the raw model being more honest. This matters for vendor evaluation, which is a Level 4 subject, but it matters for the individual writer today, because it explains why the same task feels reliable in one tool and reckless in another.

The Fabricated Table, Traced to Day 74

Return to the draft 2.5.4 and the citation to Table 14.2.1.4, the table that does not exist. Trace what happens if no one catches it. The draft is good, so a busy co-author lifts the efficacy paragraph into the integrated Module 2.5 draft. The Module 2.7.3 writer, working from the same model and the same loaded sources, produces a parallel summary that cites the same nonexistent table, because the model is consistent in its inventions when the inputs are consistent. The reference manager, run before the table package was finalized, does not flag it because the cross-reference format is valid even though the target is not. The submission locks. The dossier ships.

On Day 74 of the review cycle, the FDA Office of New Drugs sends a routine Information Request asking the sponsor to reconcile a stated progression-free survival result in the Clinical Overview against the corresponding TLF table. The reviewer cannot find Table 14.2.1.4. The sponsor's team now has to determine, under a tight clock, whether the number in the 2.5.4 is correct and merely mis-cited, or whether the number itself was invented along with the table. If the latter, the credibility cost extends beyond one paragraph, because a reviewer who finds one fabricated citation begins to distrust every citation, and the whole submission inherits a presumption of carelessness. This is why this program treats a hallucinated cross-reference as a 21 CFR Part 11 audit-trail problem and an ICH E3 alignment problem, not as a typo. The mechanical fact from the first section, that the model completes patterns rather than retrieving truth, is the entire reason the verification step cannot be skipped.

Reading a Draft the Way a Reviewer Reads It

There is a way of reading an AI draft that turns the mechanics into a habit. A reviewer at the Office of New Drugs does not read your Clinical Overview as prose; they read it as a set of claims that must each reconcile to a source. You can adopt the same posture toward your own AI draft, and once you do, the failure modes become visible rather than hidden.

Take a single sentence from the draft: "In the intent-to-treat population, treatment with the investigational agent reduced the risk of disease progression or death by 32 percent compared with the control arm, as shown in Table 14.2.1.4." Read as prose, it is excellent. Read as claims, it decomposes into four separate assertions, each with its own risk. First, there is a population claim, that the analysis is on the intent-to-treat set and not the per-protocol or safety set; the model can quietly swap analysis populations because the phrasing is interchangeable in the corpus even though the results are not. Second, there is a magnitude claim, the 32 percent risk reduction implying a hazard ratio of roughly 0.68; if that number was not in the loaded text, it is a generated value wearing the costume of a measured one. Third, there is a directional and comparator claim, that the effect favors the investigational arm over control; an arm swap here is rare but catastrophic and has happened. Fourth, there is the structural claim, that Table 14.2.1.4 exists and contains this result, which is the cross-reference risk from earlier in this lesson.

One fluent sentence, four independent verification tasks. The discipline is to verify each claim against the actual source rather than against the sentence's own confidence, because the sentence's confidence is constant whether the claim is true or false. The model writes a true hazard ratio and an invented one in exactly the same tone. Calibrated language, where uncertainty shows up as hedged phrasing, is a human habit the model does not reliably reproduce; it tends to write everything with the same even assurance. That uniform assurance is precisely why your eyes slide over the errors, and precisely why claim-by-claim reconciliation, not rereading, is the control that works.

What This Means for the Writer on Monday

None of this argues against using the model. A model that drafts a structurally sound 2.5.4 in eleven seconds is a genuine accelerant, and the writers who refuse it will be outpaced by the writers who use it well. The argument is that using it well is a discipline with named steps, and the discipline follows directly from the mechanics.

Because the model reasons only over the window, you load every source the claim depends on, especially the final TLF package, before you ask for a draft. Because the model invents plausible cross-references, you verify every table citation against the actual table, every time, and you treat an uncheckable citation as wrong until proven right. Because output varies with temperature, you capture the specific run in the audit trail. Because the system prompt shapes safety and you cannot see it, you learn which of your tools are grounded and which are not, and you raise your verification effort accordingly. And because the named author owns the gap between the shape of a correct answer and the truth of it, you never sign a document you have not reconciled to source. The eleven-second draft is the start of the work, not the end of it. The rest of Level 1 builds the map of where the work lives, and the rest of the program builds the workflows that make it defensible.

Key Takeaways

  • A large language model generates text one token at a time by sampling the most plausible continuation, conditioned on the context window. It does not retrieve your trial's facts; it completes the pattern of a correct-looking answer. The named author owns the gap between a correct-looking answer and a true one.
  • The context window is the model's entire field of view, and a missing source is a silence, not a flag. If you do not load the final TLF package, the model cannot verify a table number against it, yet it will still produce table citations because the pattern of a 2.5.4 includes them.
  • A fabricated TLF cross-reference is more dangerous than a fabricated sentence. It survives spell-check and a hasty reviewer, propagates across Module 2.5 and 2.7.3, and is caught only at reference QC, sometimes not until an Office of New Drugs Information Request on Day 74.
  • Temperature makes the same prompt produce different drafts, so the audit trail must capture the specific run: prompt, system prompt, model and version, temperature, timestamp, sources loaded, and the final human-verified artifact. Disagreement across runs is itself a cheap hallucination signal.
  • The system prompt you never see does much of the work that determines whether a draft is safe. Grounded, citation-disciplined output usually reflects a well-built system prompt and retrieval layer, not a more honest raw model. Learn which of your tools are grounded and raise verification where they are not.