BioResearchBench: 50 Long Horizon Biological Research Tasks

Abstract

AI models are used in biology research to study disease mechanisms, discover potential treatments, analyze experimental data, troubleshoot problems, and plan next steps. Useful assistance depends on working with data in various scientific formats, interpreting results, and deciding what to test next. We present BioResearchBench, a benchmark of 50 research questions developed by PhD biologists drawing on their deep research expertise to reflect realistic problems encountered in their fields. Questions were refined through expert review and iterative model testing. The collection spans 10 diverse biology subdomains and includes sequencing data, proteomic measurements, microscopy images, assay readouts, and clinical datasets. We compared recorded outcomes and examined model responses in depth across the 50 tasks on OpenAI GPT-6 Astra, Claude Opus 5.5, Gemini 4 Argon, Meta Muse Wonder, and SpaceXSI Grok 4.7. Success across our benchmark was 18% for Claude Opus 5.5, 13% for Gemini 4 Argon, 11% for Meta Muse Wonder, 4% for GPT-6 Astra, and 4% for Grok 4.7. These results describe a deliberately challenging collection selected through expert review and model testing. Despite low whole task success, the selected responses met most rubric requirements on average: the average fraction of criteria missed ranged from 18% to 32% across models. Across models, the proportion of eligible tasks with at least one category failure was highest in computation and data handling (66.9% failure), followed by uncertainty and identification (55.3% failure), reporting and completeness (54.7% failure), biological interpretation (39.0% failure), controls and causal claims (37.2% failure), and choosing the next experiment or decision (31.0% failure). Gemini 4 Argon had the highest proportion of reviewed tasks without a recorded category failure in computation and data handling (45% success) and biological interpretation (69% success). Claude Opus 5.5 led in controls and causal claims (72% success) and choosing the next experiment or decision (75% success). GPT-6 Astra led in uncertainty and identification (56% success) which included judging the precision of estimates and recognizing when measurements could not distinguish between biological explanations. GPT-6 Astra and Meta Muse Wonder tied for the highest proportion of answers with no recorded omissions or reporting failures (51% success). Five case studies presented here show how top AI models can perform many analytical steps correctly yet misinterpret what results signify. For example, some answers described cells as fully recovered when oxygen consumption improved even though mitochondrial membrane potential and cell number remained below control levels. These model answers therefore overstated the extent of cell recovery. Other answers incorrectly called a protein a direct microRNA target, although the experiment had not tested whether the microRNA acted directly on its mRNA or affected it through another process. In a protein turnover task, some model answers misestimated protein half-lives because they did not account for the gradual labeling of amino acids used to make new proteins. These model errors can lead researchers to pursue the wrong explanation even when individual analytical steps are completed correctly. This matters for major scientific goals such as drug discovery, where progress depends on choosing useful experiments and tools, interpreting unexpected results, troubleshooting failed approaches, and deciding which direction is most worth pursuing. Evaluating AI research assistance therefore requires checking whether its explanations, conclusions, and proposed experiments follow from the data and experimental design. These cases highlight continuing challenges in scientific reasoning for frontier AI models: bringing together different types of biological data, analyzing them with appropriate tools and databases, and using the results to draw appropriate conclusions and guide the next steps in research. The observed errors suggest training priorities in data analysis, biological interpretation, and experimental judgment and motivate evaluations that follow researchers and models through iterative scientific work.