Benchmarking Uncertainty and Improving Risk Ranking in Multi-task Bioactivity Prediction

Abstract

Multi-task bioactivity prediction transfers information across assays, but experimental prioritization also requires uncertainty estimates that identify unreliable predictions. We benchmarked prediction accuracy and uncertainty calibration for five multi-task predictors on 100 ChEMBL27 test assays and 100 CHEMBL37 assays under the random split and the clustering split. Calibration used native, Deep Ensemble and MC Dropout uncertainty; risk ranking was compared for two Gaussian process (GP) backbones. Adaptive deep kernel fitting (ADKF) achieved the best mean calibration results, while deep kernel transfer (DKT) ranked errors more effectively than ADKF with native uncertainty. We also introduce Influence Calibrated Support Reconstruction (ICSR), a risk score for kernel-based predictors. ICSR combines measured support reconstruction errors with query-specific influence and stabilizes their weighted average toward the full-support mean to adjust Gaussian process uncertainty while preserving predicted activities. With DKT and ADKF, ICSR improved all three mean risk-ranking metrics over native uncertainty and an adapted neighborhood comparator in every panel and split. ICSR achieved a mean half-query MAE reduction (R50) of 14.2-19.6%, where R50 measures the percentage decrease in mean absolute error (MAE) after retaining the lowest-risk half of the queries. A retrospectively selected SARS-CoV-2 main protease case further illustrated its use for selecting more reliable predictions. The benchmark supports joint assessment of calibration and error ranking, while ICSR improves selective use of kernel-based bioactivity predictions.