Calibrating Classifiers Across Covariates: Hierarchical Vocal Density Estimation for Acoustic Monitoring at Scale

Abstract

Passive acoustic sensors and machine learning classifiers offer a scalable approach to monitoring wildlife populations. These classifiers typically detect whether a species is present in a short time-window of audio; ecologists, however, want ecologically meaningful measures such as indices of site-level abundance. Vocal density, the proportion of time-windows containing a vocalization, is a useful abundance proxy. Recovering vocal density from classifiers requires calibrating their outputs against expert-labeled data. Because classifier performance varies between sites, a single global calibration yields overconfident site-level estimates, yet calibrating every site independently requires prohibitive labeling effort. Estimating site-level vocal density at the scale of modern sensor networks demands a label-efficient alternative. We present a Bayesian hierarchical extension of Platt scaling, a commonly used calibration method, that calibrates classifiers at the site level while borrowing strength across sites. Where site-level covariates are available, a Gaussian-process prior lets the model learn how calibration varies across covariate space. Averaging calibrated per-clip probabilities across a site's recordings yields a posterior distribution over vocal density, and we show how to carry that uncertainty into downstream regressions of vocal density on environmental covariates. Across simulations and two fully annotated field datasets from Hawai'i and the Pacific Northwest, our model attained desired credible-interval coverage of vocal density estimates with narrower intervals than independent site-level calibration at equal labeling effort, and lower mean squared error. Global calibration, by contrast, failed to attain coverage except when simulated sites were genuinely homogeneous. Unlike site-level calibration, our model enables calibration at sites with zero labeled data, and incorporating covariates further improved precision when site heterogeneity was covariate-driven. Applied to a 283-site dataset in Pennsylvania with only two labeled clips per site, we recovered known habitat associations for the declining Wood Thrush (Hylocichla mustelina). Our method provides a principled, label-efficient route from bioacoustic classifier scores to site-level abundance indices with well-quantified uncertainty, making rigorous ecological inference feasible at the scale at which acoustic sensor networks are now deployed.