Signal Versus Noise: Evaluating iNaturalist Photos as a Source of Quantitative Phenotypic Data in Plethodon Salamanders using Autoresearch and Agentic AI

Abstract

Community-science platforms such as iNaturalist now contain tens of millions of georeferenced, photographically vouchered biodiversity records, yet extracting reliable quantitative measurements from opportunistic photographs remains methodologically challenging. Here, I evaluate the signal-to-noise ratio of iNaturalist photos for phenotyping Plethodon salamanders across two trait classes: continuous dorsal brightness (a proxy for ecogeographic clines predicted by Glogers rule and the thermal melanism hypothesis) and discrete color morph frequency in P. cinereus. I optimized a color-extraction pipeline using an agent-guided parameter search adapted from the autoresearch framework (Karpathy 2026; Schmidgall et al. 2025), exploring crop fraction, color space, normalization, and quality-control thresholds across 50 bounded micro-experiments. Applying the production HSV pipeline to 103,653 observations of 34 species, I found negligible geographic structure in dorsal brightness (R2 = 0.001), even within P. cinereus alone (n = 71,627). Variance decomposition showed that photographer identity explains 23.3% of brightness variance, geography 5.1%, species 1.6%, and time of day 0.3%, with 69.7% residual. In contrast, a hue-threshold morph classifier recovered a significant geographic signal in red-back frequency (R2 = 0.008, p < 0.001), 7x stronger than the brightness result, though still weaker than the supervised CNN of Hantak et al. (2022; pseudo-R2 {approx} 0.04). These results indicate that citizen-science photographs are poorly suited to continuous quantitative phenotyping under current collection conditions, whereas discrete categorical traits remain recoverable with appropriate classifiers. The autoresearch loop clarified the failure mode: no tested parameter configuration recovered a meaningful brightness signal from a dataset dominated by observer effects.