Overview

Lesson 1.3: Building Living Literature Reviews with AI

Traditional systematic reviews represent a point-in-time synthesis: they capture the state of evidence at the moment of literature search completion, then become progressively less current as new studies are published. In rapidly moving fields, clinical medicine, vaccine efficacy, environmental policy, machine learning applications, a systematic review can be substantially outdated within a year of publication, and the evidence base it synthesizes may be materially different from what decision-makers actually need.

The living systematic review (LSR) model addresses this problem by designing reviews as continuously maintained knowledge systems rather than static publications. First formalized by Elliot and colleagues in 2014 and subsequently adopted by Cochrane and other major systematic review organizations, living reviews incorporate new evidence on an ongoing basis, monthly, quarterly, or upon significant publication, without requiring complete re-review cycles. The result is a synthesis that remains actionable over time.

AI has transformed the practical feasibility of living reviews. The ongoing monitoring, alert management, and incremental paper ingestion that make living reviews operationally sustainable at small team scale have become tractable through intelligent automation. This lesson covers the architecture of an AI-supported living review workflow: how to set up monitoring pipelines, configure relevance filters, design incremental update protocols, and maintain methodological consistency across update cycles.

Title

Lesson 1.3: Building Living Literature Reviews with AI

Purpose

This lesson teaches researchers how to design and maintain living systematic reviews, continuously updated research syntheses that incorporate new evidence as it's published, leveraging AI for automated monitoring, alert systems, and paper ingestion. You'll learn to transition from static reviews to dynamic knowledge systems that remain current without requiring complete re-review cycles.

What is a Living Systematic Review?

A living systematic review (LSR) is a systematic review that is continually updated, incorporating relevant new evidence as it becomes available. The defining feature is ongoing maintenance rather than a single completed update cycle: new literature is monitored, screened, and, when relevant, integrated into the synthesis on a regular schedule.

Living reviews exist on a spectrum. At one end, rapid-cycle living reviews (common in clinical guidelines contexts) update monthly and are designed for near-real-time evidence tracking. At the other end, annual update reviews follow a yearly cycle that approximates traditional systematic review updates but with pre-established infrastructure that makes each cycle faster. The appropriate cycle frequency depends on how quickly evidence in the field is accumulating and how consequential lagged synthesis would be for the review's audience.

Living reviews differ from 'updated systematic reviews' in an important structural sense: an updated systematic review is a new review that happens to cover the same question as a previous one. A living review is a single, continuously maintained synthesis with version history. This distinction matters because living reviews must maintain methodological consistency across update cycles, the synthesis framework, data extraction template, and quality assessment criteria should be stable, with changes documented in version logs.

The Cochrane Collaboration has been a leader in formalizing living review standards, and its guidance documents provide useful reference points for structuring living review protocols. PROSPERO now has specific registration procedures for living reviews, which include committing to an update frequency and disclosing the monitoring and screening infrastructure. Registering as a living review signals to the research community that the synthesis will remain current and helps prevent duplication of effort.

Setting Up AI-Powered Monitoring Infrastructure

The infrastructure for a living review has three core components: a literature monitoring system that continuously scans publication databases and preprint servers; a relevance filter that identifies incoming records likely to meet inclusion criteria; and an ingestion workflow that processes confirmed relevant records into the synthesis.

For literature monitoring, the standard approach combines database alert services (PubMed alerts, Embase saved searches, Web of Science journal watch) with preprint server monitoring (bioRxiv, medRxiv, SSRN, arXiv as appropriate to the field) and citation alert tools (Google Scholar alerts for forward-citations to key included studies). Configure each alert using your systematic search strategy from the original review, the same Boolean query, subject headings, and filters. This ensures monitoring coverage aligns with the original search scope.

AI tools augment this infrastructure in two key ways. First, they can aggregate and deduplicate across multiple alert sources, which individually surface overlapping records. A paper posted to bioRxiv in January may generate a preprint alert, then a journal publication alert in March, then citation alerts when other papers reference it. Without deduplication, this creates triple processing load. Reference management APIs (Zotero, EndNote) with deduplication plugins, or dedicated tools like Rayyan's alert import feature, handle this automatically.

Second, AI-powered relevance filters act as a pre-screening layer for incoming alerts. Rather than having a human reviewer assess every alert, a configured AI classifier (using the same criteria specification from the original screening stage) assigns a relevance score to each new record. Records above a threshold score are queued for human triage; records below the threshold are archived for potential spot-checking but not actively reviewed. This filter is analogous to the AI screening tier from the initial review, but operating on a continuous incoming stream rather than a static corpus.

The relevance filter must be recalibrated periodically, ideally at each update cycle, because the AI's performance can drift as the field evolves and terminology changes. A filter calibrated in 2024 may underperform on 2026 publications that use different terminology for the same concepts.

Designing the Update Cycle

An effective living review update cycle has four stages: monitoring period, batch screening, data extraction and quality assessment, and synthesis update.

During the monitoring period, the alert and filter infrastructure operates passively. Incoming records are scored by the relevance filter, and high-scoring records accumulate in a review queue. The length of the monitoring period defines the update cycle frequency, monthly monitoring periods yield monthly updates, quarterly periods yield quarterly updates.

At the end of each monitoring period, the accumulated queue undergoes batch screening by human reviewers. This is the critical quality control step: human reviewers assess each record in the queue against the inclusion criteria, applying the same operationalized criteria as the original review. New records that pass screening proceed to data extraction. The batch screening stage also serves as an ongoing calibration check for the relevance filter, when human reviewers consistently disagree with filter scores (including or excluding records that the filter incorrectly classified), the filter specification should be adjusted.

Data extraction for new records should use the same data extraction template as the original review. Template consistency is essential for pooling data across update cycles. If new records report outcomes using instruments or at time points not anticipated in the original template, the template must be formally extended with version documentation rather than ad hoc modifications.

Synthesis update involves incorporating newly extracted data into the existing synthesis. For quantitative synthesis (meta-analysis), this means adding new data points to pooled estimates and recalculating effect sizes, confidence intervals, and heterogeneity statistics. For narrative synthesis, it means reassessing whether new evidence changes, confirms, or introduces nuance to previous conclusions. Every update should produce a versioned synthesis document with a clear update log: what new records were incorporated, what their contributions were, and whether and how synthesis conclusions changed.

Version control is non-negotiable in living reviews. Use a structured versioning system (e.g., v1.0 for original publication, v1.1 for first update, v2.0 for a major revision that changes core conclusions) and archive each version in a publicly accessible format. Cochrane living reviews use a permanent DOI for each version with metadata linking each to the review's full version history.

Using AI for Incremental Data Extraction

Data extraction is the most labor-intensive per-paper stage in living review maintenance, and AI assistance at this stage significantly affects the sustainability of update cycles.

LLM-based extraction tools can process full-text papers and populate structured data extraction forms with high accuracy for well-defined fields: study design, participant characteristics, intervention details, outcome measures, follow-up duration, and statistical results. The key implementation requirement is a precisely structured extraction prompt that maps to your specific data extraction template.

An effective extraction prompt should specify: the field being extracted, its data type and format (e.g., 'extract the sample size as a single integer'), the location in the paper where the information is typically found, and how to handle common reporting variations (e.g., 'if sample size is reported separately for each group, report the total'). This level of specificity reduces hallucination risk, the tendency of LLMs to generate plausible-sounding but incorrect data when the information is absent or ambiguous in the source text.

Always require the LLM to report a confidence level for each extracted field and to flag fields where the information was absent, ambiguous, or required inference rather than direct extraction. Human reviewers should verify all low-confidence extractions and all fields that were flagged. This tiered approach, AI extracts, human verifies flagged items, maintains accuracy while substantially reducing the per-paper human time investment.

One important caution: LLMs can extract values that appear in papers but represent different constructs than your template specifies. For example, a paper might report both an 'intention-to-treat' sample size and a 'per-protocol' sample size. An LLM instructed to 'extract sample size' without further specification may extract either, and different papers may yield different choices. Precise prompt specification eliminates most such ambiguities, but human verification of a systematic sample of extractions is still recommended at each update cycle.

For statistical results specifically, maintain additional caution. LLMs can correctly transcribe numbers from results sections but may select the wrong statistic (e.g., reporting the unadjusted effect estimate when your synthesis requires the adjusted estimate, or the wrong confidence interval). A verification rule requiring human check on all statistical extractions, even for high-confidence AI extractions, is a reasonable safeguard in quantitative living reviews.

Maintaining Methodological Consistency Across Update Cycles

One of the most significant challenges in living review maintenance is ensuring that the methodological standards applied in each update cycle are consistent with those of the original review and all previous updates. Inconsistency, particularly inconsistency in inclusion criteria application, quality assessment, or data extraction, can introduce systematic biases that masquerade as genuine temporal trends in the evidence.

Formal documentation is the foundation of consistency. The living review protocol should be archived in a version-controlled repository (GitHub, OSF), with all changes logged and dated. Any modification to inclusion criteria, data extraction templates, quality assessment instruments, or synthesis methods should be treated as a protocol amendment: described, justified, applied prospectively (not retroactively unless the original data are re-extracted), and documented in the published review's version log.

Reviewer consistency is a parallel concern. Living reviews conducted by small teams face the risk that reviewer turnover, learning effects, or interpretive drift cause criteria to be applied differently at different update cycles. Periodic inter-rater reliability checks, comparing two reviewers' independent screening or extraction decisions on the same batch of records, help detect and correct drift. The kappa statistics from these checks should be reported in the review's ongoing methods documentation.

When the evidence base evolves in ways that make the original synthesis framework insufficient, new outcome measures become standard, new subgroup analyses become relevant, the comparator landscape changes, the protocol must be formally revised rather than silently adapted. A well-documented major revision (reflected in a version number increment, e.g., v1.x to v2.0) is methodologically defensible. Undocumented adaptations that appear in the synthesis without explanation undermine the review's credibility.

The AI tools used in monitoring, screening, and extraction should also be version-controlled. Document which model version was used for screening or extraction at each update cycle. Model updates by AI providers can change classification behavior in ways that affect the systematic review's results if unmonitored.

Sustainability: Team Structure and Resource Planning

The primary reason living reviews fail is not methodological. It is operational. Review teams underestimate the ongoing resource commitment, and living reviews become dormant after one or two update cycles, defeating their purpose.

Successful living review teams design for sustainability from the outset. This means accurately estimating per-cycle workload before committing to a specific update frequency. A useful estimation approach: run the batch screening and extraction workflow for a single simulated update cycle using the previous three months of monitoring data, measure the actual time required, and use this measurement to set a realistic update frequency.

For most small research teams (two to four people), a quarterly update cycle is the most sustainable baseline for a literature base generating moderate alert volume. Monthly updates require either very low alert volume, substantial AI automation of screening and extraction, or dedicated team capacity that most academic groups cannot sustain. Setting an overly ambitious update frequency and failing to maintain it is worse for the review's credibility than committing to a more modest but reliably maintained schedule.

Clear role assignments are essential. At minimum, a living review needs: a lead reviewer responsible for overall protocol maintenance and synthesis updates; a monitoring coordinator responsible for managing alert streams and filter maintenance; and one or more screening reviewers who conduct batch triage at each update cycle. In small teams, these roles may overlap, but explicit responsibility assignment prevents the diffusion of accountability that leads to missed update cycles.

Funding models for living reviews deserve explicit planning. Traditional grant structures fund a single research product (the initial publication), not ongoing maintenance. Living review teams should seek supplemental funding for maintenance phases, institutional repository support for version archiving, or embed living review maintenance into programmatic research activities. Some journals and evidence synthesis organizations (including Cochrane) now provide infrastructure support for registered living reviews, which can offset some maintenance costs.

Publication and Dissemination Strategies for Living Reviews

Living reviews present unique publication and dissemination challenges because they do not fit the standard model of a single publication with a fixed date.

Several publication models have emerged. The continuous publication model, used by Cochrane living reviews, publishes each update as a new version of the same review, with a persistent DOI and version history accessible through the publisher's platform. This model is ideal for reviews hosted by systematic review organizations but requires institutional affiliation with such an organization.

The periodic update publication model publishes formal update articles at predetermined intervals (typically annually or every two years) in standard journals. Between publications, updated results are available in a public repository (OSF, institutional repository) with clear version dating. This model fits more naturally into standard academic publication incentive structures and allows credit attribution for each update cycle.

Preprint servers increasingly support living document versions, and bioRxiv has explicit provisions for updated preprint versions with version history. This model is particularly useful for fields where preprint citation is standard and for maintaining currency between formal publication cycles.

Regardless of publication model, each version of the living review should clearly state: the date of the most recent literature search, the number of update cycles completed, and a summary of how synthesis conclusions have changed across update cycles. Readers need this information to assess the review's currency and to understand the evidence trajectory over time.

Dissemination channels should be actively managed, not passive. When a living review update changes conclusions or adds substantial new evidence, proactive dissemination to the review's target audience, clinical guidelines developers, policy makers, field researchers, ensures the updated synthesis actually influences practice. An updated review that reaches the same audience as the original publication is far more valuable than one that accumulates silent version history in a repository.

Summary

Living systematic reviews represent the evolution from static evidence synthesis to dynamic knowledge infrastructure. Their value is highest in rapidly evolving fields where point-in-time reviews become outdated quickly and where synthesis currency directly affects decision quality.

AI makes living reviews operationally sustainable at small team scale by automating the most repetitive components: monitoring deduplication, relevance filtering, and structured data extraction. The human investment in living reviews concentrates on the tasks that require judgment: batch screening triage, extraction verification, and synthesis interpretation.

The critical success factors for living reviews are: infrastructure design that matches alert volume and team capacity; methodological documentation that ensures consistency across update cycles; realistic update frequency commitments; clear role assignments; and proactive dissemination that ensures updated evidence reaches its intended audience.

Building a living review requires more upfront infrastructure investment than a traditional static review, but that investment pays dividends over multiple years of maintenance as the evidence base continues to evolve around a research question that remains important.