bioRxiv · 10.64898/2026.10.04.756483
Version-aware question answering over living oncology guidelines
Abstract
Living clinical guidelines revise recommendations as evidence emerges, so an AI answer can be faithful to a real guideline and still be out of date. Here we examine this failure using ASCOBench, 288 unique questions in 96 three-turn conversations grounded in versioned American Society of Clinical Oncology (ASCO) breast and prostate cancer guidelines, with oncologist-reviewed reference answers. Oncologists corrected one in four version-sensitive answers drafted by a frontier model, against fewer than one in ten factual ones. Across the guideline corpus, 19 recommendations changed between versions, seven of them reversals, yet only 1 of 14 superseded documents states that it has been replaced. With a strong model, retrieval over several guideline versions made stale answers four times more frequent than answering without retrieval. SentryLine, a verification-first system, lowered incorrect answers to 4.2-5.2% from 9.7-22.9% for baselines across three models. Recognizing change before a guideline declares it remains an open problem.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Ravi Kumar, T., Kaniyar Narayana Iyengar, A. I., Khan, M. A., Khakwani, K. Z. R., Kathuria, K., Sonbol, M. B., Riaz, I., Gupta, V.. 2026-10-11. Version-aware question answering over living oncology guidelines. https://doi.org/10.64898/2026.10.04.756483
Cite the original work for its findings. Save a collection to share your selection of sources.