A health-economics and outcomes-research lead at a sponsor has a label-expansion supplement in flight and a payer-access deadline behind it, and the evidence both will rest on is real-world: a retrospective cohort drawn from a closed-claims dataset in Komodo Health, validated against a federated electronic-health-record network in TriNetX, with a comparative-effectiveness analysis to be run in Aetion's environment. The deliverables are a study report formatted to the conventions a journal and a regulator expect, a payer dossier in AMCP format, and, if the evidence is strong enough, a section in the regulatory supplement that the FDA will evaluate against its real-world evidence framework. A large language model can accelerate every writing-heavy step of this, and the temptation is to let it. But real-world data carries a property that trial data does not, and it changes the entire accountability structure: the data was not collected for research. It was generated by the machinery of billing, care delivery, and reimbursement, for purposes that have nothing to do with the question being asked of it, which means the validity of every inference drawn from it is a human judgment that no model can own. This lesson designs the end-to-end workflow from cohort generation through the study report to the payer dossier, with that human-owned validity assessment as the structural spine.

Why Real-World Data Changes the Accountability Structure

In a randomized controlled trial, the data is collected prospectively, under a protocol, with case report forms designed to capture exactly the variables the analysis needs, and randomization balances the confounders you did not measure. Real-world data has none of these protections. A claims dataset records what was billed, not what was clinically true; a diagnosis code may be present because a condition was suspected and ruled out, absent because it was managed without a billable encounter, or miscoded because the coder optimized for reimbursement. An electronic-health-record dataset captures what a clinician chose to document in a system never designed for research, with missingness that is not random but tied to how sick the patient was and how the site operated. The fitness of this data for any given research question is therefore not a property of the data; it is a judgment about whether this particular data can support this particular inference, and that judgment is the most consequential and least delegable act in the entire workflow.

This is why the FDA's real-world evidence framework, and the good-practice expectations from ISPOR and the AMCP format, place so much weight on the provenance and validity of the data and so little on the polish of the report. A regulator evaluating an RWE study in a supplement is not primarily asking whether the prose is clear; they are asking whether the data source was fit for purpose, whether the cohort definition was specified before the analysis rather than tuned to the result, whether the confounding was addressed by a credible method, and whether the limitations inherent in non-research data were honestly characterized. A workflow that uses AI to make the report fluent while leaving these validity judgments implicit has optimized the least important dimension and left the load-bearing one unaddressed. The design discipline here is to make the validity assessment an explicit, human-owned, documented artifact that the AI-accelerated writing serves rather than obscures.

Stage One: Cohort Generation and the Specification Trap

The workflow begins with cohort generation in the source platform, translating a clinical question into a computable phenotype: the codes, the inclusion and exclusion windows, the index-event definition, the washout period, and the outcome definition that together select the patients. The model is a genuine accelerant here, because turning a clinical concept into a draft code set and a structured cohort specification is exactly the kind of structured translation it does well, and it can propose the ICD, NDC, and procedure codes that operationalize a condition far faster than a human assembling them by hand. This is real value, and it shortens a step that historically took an analyst days of careful list-building.

The trap at this stage is the most dangerous in the entire RWE workflow, and it is methodological rather than computational. The cohort definition must be specified before the analysis is run, because a definition tuned after seeing the results, dropping a code that weakens the effect, narrowing a window that sharpens it, is no longer a test of a hypothesis but a search for a favorable answer, and a regulator or a careful reviewer can often detect the fingerprints of that tuning. A model that helpfully proposes alternative cohort definitions when the first one does not produce a clean result is not assisting; it is industrializing the exact specification-searching that destroys an RWE study's credibility. The workflow therefore locks the cohort specification as a pre-specified artifact, with the model assisting the drafting of that specification and explicitly not optimizing it against outcomes, and the named methodologist owns the lock. The discipline is to use AI to specify well, once, and then to treat the specification as fixed.

Stage Two: The Analytical Plan and the Confounding Problem

With the cohort locked, the workflow drafts the analytical plan, the document that specifies how the comparison will be made and, above all, how confounding will be addressed, because in non-randomized data confounding is not a nuisance to be mentioned but the central threat to validity. The plan specifies the comparator, the outcome measures, the statistical method, propensity-score matching or weighting, instrumental variables, or another credible approach, the covariates to be balanced, and the sensitivity analyses that will test how fragile the result is to unmeasured confounding. The model can draft this plan competently from the study question and the data structure, articulating a methodologically conventional approach, and it can populate the covariate list and the sensitivity-analysis menu from the standard repertoire.

What the model cannot do is decide whether the chosen method actually neutralizes the confounding that matters for this question in this data, and that is the judgment the analytical plan exists to record. A propensity-score model can balance the covariates that were measured while leaving the unmeasured confounder, the reason a clinician chose this drug for this patient, entirely unaddressed, and a fluent plan can describe a rigorous-sounding method that does not in fact answer the threat it claims to address. The named epidemiologist or biostatistician owns the determination that the method is adequate to the confounding structure, that the measured covariates are the ones that matter, and that the sensitivity analyses are honest stress tests rather than reassuring decoration. The model drafts the plan; the human certifies that the plan is a credible defense against confounding, because confounding adequacy is a scientific judgment that the architecture of a language model cannot make.

Stage Three: The Study Report in ISPOR and AMCP Form

The study report assembles the locked cohort, the executed analytical plan, and the results into a document formatted to the conventions a journal and a regulator expect, drawing on ISPOR good-practice reporting and, for the payer-facing version, the AMCP format for formulary submissions. This is high-volume, high-value generation: the model can produce the methods section from the locked specification and analytical plan, populate the results narrative from the analytical output, and structure the document to the relevant template, compressing a writing task that historically consumed weeks. Because the methods and cohort were pre-specified and locked, the report's description of them is a transcription task with a verifiable source, which is exactly where AI assistance is safest.

The verification burden concentrates on two things: that the reported results faithfully transcribe the analytical output rather than a plausible-sounding approximation of it, and that the limitations section honestly characterizes the constraints of non-research data rather than burying them. The first is the familiar reconciliation discipline: every effect estimate, confidence interval, and sample count in the report is reconciled against the actual analytical output, because a model generating a results narrative can produce a fluent and slightly wrong hazard ratio as easily as a correct one. The second is subtler and more important: an RWE report's credibility lives in its limitations section, where the residual confounding, the missingness, the coding uncertainty, and the generalizability constraints are stated plainly, and a model left to its own devices tends to write limitations sections that are present but anodyne, mentioning constraints without conveying their weight. The named author owns the limitations as a substantive scientific statement, not a formality, because that section is where an honest RWE study distinguishes itself from a misleading one.

Stage Four: The Payer Dossier and the FDA RWE Framework

The final outputs diverge by audience, and the workflow has to serve two evaluators with different standards from the same validated core. The payer dossier in AMCP format presents the evidence for a formulary decision, emphasizing the economic model, the budget impact, and the comparative effectiveness in a structure pharmacy-and-therapeutics committees expect, and the model accelerates the assembly of this dossier from the study report and the economic analysis. The regulatory supplement section, by contrast, is evaluated against the FDA's real-world evidence framework, which asks whether the real-world data and the study design are adequate to support the specific regulatory claim, a far higher and more design-focused bar than a payer dossier typically applies.

The critical design point is that both outputs inherit their credibility from the same upstream validity assessment, and neither can manufacture credibility the underlying study does not have. If the data was not fit for the regulatory question, no amount of fluent dossier prose makes it fit, and a payer dossier that overstates an effect the study cannot support is a commercial and compliance liability. The workflow keeps the validity assessment visible in both outputs, scaled to the audience: the regulatory section foregrounds the fit-for-purpose argument and the design rigor the FDA framework demands, and the payer dossier presents the evidence honestly within the AMCP structure without inflating it. The model assembles both efficiently from the shared core; the named author ensures that each output claims only what the validated study supports, and that the same effect estimate is not described as definitive to a payer and exploratory to a regulator.

Stage Five: Validating the RWE Workflow as a System

As a Level 3 design, the RWE workflow is validated, not merely operated, and the validation discipline lands with particular force here because the failure modes are scientific rather than mechanical. The intended-use statement pins the workflow's purpose: accelerating the production of RWE study reports and dossiers from a pre-specified, locked analysis, explicitly not generating or optimizing the study design against outcomes. The fitness-for-purpose grading then concentrates verification where an undetected error is most damaging: cohort specification is methodology-critical because post-hoc tuning destroys credibility, the analytical-plan confounding judgment is the highest-stakes human determination in the workflow, results transcription is high risk because a wrong estimate reaches a regulator or payer, and report formatting is comparatively low risk once the content is verified.

Performance qualification for this workflow means more than checking that the document reads well. It means confirming, on a representative run, that the cohort specification was locked before analysis and was not altered after results were seen, that every reported estimate reconciles to the analytical output, that the limitations section substantively reflects the data's constraints, and that no output claims more than the validity assessment supports. The qualification record, kept under change control, is what lets the HEOR function assert that the workflow produces evidence the FDA framework and a P&T committee can rely on, rather than merely a polished narrative. The model accelerated the cohort drafting, the plan drafting, and the report and dossier assembly; the named methodologist owns the locked specification, the named epidemiologist owns the confounding judgment, and the named author owns the limitations and the claims, because in real-world evidence the validity of the inference is human-owned by the nature of the data itself.

Where the RWE Workflow Fails Silently

The RWE workflow's failures are quieter than a fabricated table, which makes them more dangerous, because a credibility-destroying methodological error produces a clean, confident, fluent report that looks exactly like a sound one. Let the model re-propose cohort definitions until the effect looks clean, and you have a specification-searched study whose result is an artifact of the search, dressed in the language of a pre-specified analysis. Accept a fluent analytical plan whose propensity model balances the measured covariates while ignoring the clinical reason the drug was chosen, and you have a rigorous-sounding defense against confounding that does not actually defend against it. Let the model write an anodyne limitations section, and you have hidden the very uncertainty that an honest RWE study is obligated to surface.

The deepest failure is forgetting that the data was not collected for research, and letting the fluency of the AI-assembled report paper over that origin. A regulator evaluating the supplement against the FDA RWE framework and a P&T committee evaluating the dossier are both, in their different ways, testing whether the inference is warranted by data that was generated for billing and care rather than for the question being asked. The workflow's entire architecture exists to keep that question answerable: the locked specification shows the analysis was not tuned, the documented confounding judgment shows the threat was confronted, the substantive limitations section shows the constraints were honored, and the audit trail ties it all to named human owners. The model can make every writing step faster, but it cannot make non-research data fit for a research question, and the workflow that pretends otherwise produces its most convincing output at the moment it is least trustworthy. The corollary for the HEOR lead is that the speed AI delivers must be spent on the validity work, not against it: the hours saved on cohort drafting, plan drafting, and report assembly are precisely the hours that should be reinvested in the pre-specification lock, the confounding judgment, and the limitations section, because those are the dimensions a regulator and a P&T committee actually weigh. A team that pockets the time savings as throughput and skips the validity work has used a powerful accelerant to ship indefensible evidence faster, which is the worst possible outcome of an AI-integrated RWE workflow.

Key Takeaways

• Real-world data was not collected for research, and that changes the entire accountability structure. Claims and electronic-health-record data are generated by billing and care delivery, with non-random missingness and reimbursement-driven coding, so the fitness of the data for any given inference is a human judgment that no model can own, and it is the most consequential and least delegable act in the workflow.
• The cohort specification must be locked before analysis, and the model must not optimize it against outcomes. A definition tuned after seeing results is a search for a favorable answer rather than a test of a hypothesis, and a model that helpfully re-proposes cohorts until the effect looks clean is industrializing the specification-searching that destroys an RWE study's credibility; the named methodologist owns the lock.
• Confounding adequacy is the highest-stakes human judgment, and a fluent plan can describe a method that does not actually defend against it. A propensity model can balance measured covariates while leaving the clinical reason a drug was chosen entirely unaddressed, so the named epidemiologist certifies that the method neutralizes the confounding that matters and that the sensitivity analyses are honest stress tests, not reassuring decoration.
• An RWE report's credibility lives in its limitations section, which the model tends to write anodyne. Beyond reconciling every estimate to the analytical output, the named author owns the limitations as a substantive scientific statement that conveys the weight of residual confounding, missingness, coding uncertainty, and generalizability constraints, because that section is where an honest RWE study distinguishes itself from a misleading one.
• The payer dossier and the regulatory supplement inherit credibility from one shared validity assessment and cannot manufacture more. The AMCP-format dossier and the FDA-RWE-framework section are evaluated against different standards but rest on the same locked, validated core, so the named author ensures each output claims only what the study supports and that the same estimate is never described as definitive to a payer and exploratory to a regulator.