Key takeaways:
- The system met the 90% quality benchmark for appropriate and cost-effective screening in two health systems.
- Errors were most common at 3-year, 3- to 5-year and 6-month surveillance intervals.
In over 90% of cases, an AI-based clinical decision support system delivered colonoscopy surveillance intervals aligned with established guidelines, drawing on patients’ colonoscopy and pathology data.
“Importantly, our system did this across two very different institutions, with different endoscopy reporting software, different providers and different documentation styles, suggesting the approach can travel rather than be effective at only one site,” lead author Ashwin Rao, MD, gastroenterology fellow at Baylor College of Medicine, told Healio.
Rao and colleagues developed an AI system that uses large language models (LLMs) to gather data and compute optimal colonoscopy surveillance intervals based on 2020 U.S. Multi-Society Task Force (USMSTF) guidelines.
USMSTF guidelines are a national quality benchmark for surveillance intervals, designed to help evaluate procedure- and pathology-based risk factors to determine when a patient should have their next colonoscopy.
The optimal target to achieve appropriate and cost-effective screening is 90% compliance with USMSTF. However, according to study background, fewer than half of recommendations comply with the guidelines in practice. Oversurveillance and undersurveillance are common, contributing to unnecessary procedural risk, inefficient resource use and delayed diagnoses.
“Earlier decision-support tools for colonoscopy surveillance recommendations either relied on manual data entry or couldn’t interpret free-text clinical notes, where most of the information needed to assign these intervals actually resides,” Rao said. “We hope that an AI decision-support system capable of reading and processing that data effectively will assist clinicians and help more patients receive guideline-based care.”
Rao and colleagues’ clinical decision support system (CDS) extracted information from diagnosis-bearing sections of pathology reports and data from the electronic health record, including adenoma presence and count, large polyp presence, bowel preparation adequacy and other factors.
The researchers developed the CDS in a retrospective study and then evaluated its performance in two independent validation cohorts: one internal cohort from the same health system and one external cohort from a second health system.
The development cohort included 256 adults (mean age, 61 years; 52.3% women) undergoing CRC screening, diagnostic colonoscopy or postpolypectomy surveillance between April and May 2025.
The internal validation cohort included 450 adults (mean age, 61 years; 53.1% women) undergoing colonoscopy between June and July 2025. In this cohort, the CDS delivered guideline-concordant surveillance intervals in 94% of cases (95% CI, 91.8%-96%).
The external validation cohort was drawn from a second health care system and included 415 adults (mean age, 58 years; 57.6% women) undergoing screening colonoscopy. In this cohort, surveillance intervals aligned with USMSTF guidelines in 92.8% of cases (95% CI, 90.1%-95.2%).
With data from this cohort, CDS delivered guideline-concordant surveillance intervals 94% (95% CI, 91.8%-96%) of the time. Guideline concordance was lower for the 3- to 5-year (71%) and 6-month (53.8%) intervals.
The external validation cohort was drawn from a second health care system, consisting of 415 adults (mean age, 58 years; 57.6% women) undergoing screening.
In this cohort, surveillance intervals aligned with USMSTF in 92.8% (95% CI, 90.1%-95.2%) of cases. CDS accuracy was below the 90% threshold for 3- to 5-year intervals (62.2%) and 6-month intervals (60%).
Across cohorts, misclassifications were concentrated in cases involving high polyp burden or complex lesion management requiring 3-year, 3- to 5-year and 6-month surveillance intervals.
“When misclassifications occurred, they most often resulted in undersurveillance. The most common error source was ambiguity in the underlying clinical documentation, particularly pathology wording that made adenoma counts difficult to determine,” Rao said.
“Other discordant cases involved complex documentation scenarios, such as diagnostic biopsies of large polyps being interpreted as complete resections.”
Hallucination-related errors were rare, observed in fewer than 1% of cases.
To develop on findings, Rao and colleagues are planning a prospective study to evaluate the CDS performance under real-world conditions.
“We are building our system as an EHR-embedded application, so it can be added in the same plug-and-play way that hospitals and clinics now adopt ambient AI scribes,” Rao said. “We are also studying — directly with endoscopists — when and how the recommendation should appear, so it lands at the right moment and is genuinely useful rather than another alert that gets dismissed.
“Ultimately, we envision a system that improves care by promoting appropriate surveillance while reducing endoscopists’ mental workload, rather than adding to it,” he added.
For more information:
Ashwin Rao, MD, can be reached at ashwin.rao@bcm.edu.
Perspective
The concept of extracting surveillance-relevant information from colonoscopy and pathology reports is not new.
More than a decade ago, natural language processing (NLP)-based systems were developed to identify colonoscopy quality metrics and generate surveillance recommendations from free-text reports. However, conventional NLP approaches largely relied on handcrafted rules, predefined dictionaries and site-specific algorithms to identify key variables such as adenoma number, size, histology, bowel preparation quality and completeness of examination.
Because colonoscopy and pathology reports often contain heterogeneous terminology, variable reporting styles, and complex relationships between endoscopic and histologic findings, these systems typically required substantial customization and maintenance when deployed across institutions.
LLMs represent an important advance because they can interpret free-text reports in a more flexible and contextual manner, allowing integration of information across colonoscopy and pathology reports with less dependence on manually engineered rules. In the present study, the combination of LLM-based variable extraction with a deterministic guideline-based rules engine preserves transparency and auditability while improving scalability and portability across different practice settings.
Clinically, this approach may help standardize surveillance recommendations and reduce unwarranted variation in practice. Importantly, discrepancies in surveillance interval recommendations are not confined to non-gastroenterologists; substantial variation persists even among gastroenterologists, despite the availability of established guidelines.
Such variation contributes to both oversurveillance and undersurveillance. By reducing unnecessarily short surveillance intervals in patients with low-risk findings, an AI-assisted system could decrease avoidable colonoscopies, procedural risks, health care costs and pressure on limited endoscopy capacity.
Conversely, by ensuring that patients with advanced adenomas, multiple adenomas or other high-risk findings receive appropriately timed follow-up, it may reduce delayed surveillance and ultimately lower the risk for postcolonoscopy colorectal cancer.
Future work should focus on prospective EHR integration, improved handling of ambiguous pathology reports, incorporation of prior colonoscopy history, calibrated uncertainty assessment and selective human review for complex cases. Ultimately, the value of such systems lies not only in automating recommendations but also in promoting consistent, evidence-based surveillance practices that improve resource allocation and colorectal cancer prevention outcomes.
Han-Mo Chiu, MD, PhD
National Taiwan University Hospital
Disclosures: Chiu reports lecture honoraria from Boston Scientific, Fujifilm and Olympus; research funding from the National Science and Technology Council of Taiwanese Government; and serving as principal investigator of the Taiwan Colorectal Cancer Screening Program.
Ask a clinical question and tap into Healio AI’s knowledge base.
- PubMed, enrolling/recruiting trials, guidelines
- Clinical Guidance, Healio CME, FDA news
- Healio’s exclusive daily news coverage of clinical data
