On the Comparability of Studies in Predictive Ecotoxicology

Abstract

Comparability across in silico predictive ecotoxicology studies remains a significant challenge, particularly when assessing model performance. In this work, we identify key criteria necessary for meaningful comparison between independent studies: (i) the use of identical datasets that represent the same chemical and/or taxonomic space; (ii) consistent data cleaning procedures; (iii) identical train/test splits; (iv) clearly defined evaluation metrics, as subtle differences--such as alternative formulations of R2--can lead to misleading discrepancies; and (v) transparent reporting through code and dataset sharing. Our review of recent literature on fish acute toxicity prediction reveals a critical gap: no two studies fully meet these criteria, rendering cross-study comparisons unreliable. This lack of comparability hampers scientific progress in the field. To address this, we advocate for the adoption of benchmark datasets with standardized cleaning protocols, version control, and defined data splits. We further emphasize the importance of precise metric definitions and transparent reporting practices, including code availability and the use of structured reporting or data sheets, to foster reproducibility and advance the discipline.