bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/2025.08.05.668631

Are our 95% CIs only worth 45% confidence?

Knief, U.·Forstmeier, W.
View original source ↗Export metadata ↓

Abstract

O_LIWhen multiple studies on the same research question, or multiple analyses of the same dataset are summarized in a meta-analysis, our confidence intervals (CIs) are put to a relentless test of reliability. While we might hope that 95% of the CIs would contain the meta-analytic mean value (and therefore presumably also the true value), a recent meta-analysis of 512 meta-analyses in ecology and evolution suggests that only a sobering 45% of them do. C_LIO_LIAs this paradox of overconfidence continues to confuse researchers, we attempt to explain where most of the heterogeneity in findings might be coming from. Much of it is unsurprising, because conventional 95% CIs reflect only sampling noise and ignore other sources of error such as arbitrary analysis decisions ("model uncertainty"). Being aware of multiple sources of error beyond sampling noise, the replication crisis logically follows from an anticonservative statistical practice that allows for overinterpretation beyond the legitimate inference space, which in fact is narrower than commonly acknowledged. C_LIO_LIWe explain how to calculate extended confidence intervals (CIext) that also cover other sources of biological and analytical heterogeneity, and we clarify which CIext is valid for which extended inference space. We further show how multiple versions of analysis of the same dataset can be merged into a many-analyses meta-analysis (MAMA) which yields a CI that accounts for two sources of error. C_LIO_LIAt the same time, we recognize that high heterogeneity estimates can be artefactual, arising from "comparing apples and oranges" (either in terms of statistical metric or biological interpretation). The estimate of only 45% deserved confidence therefore appears far too pessimistic. Overall, we argue for greater caution in interpreting both confidence intervals and heterogeneity estimates, and for clearer separation of comparable from non-comparable evidence. C_LI

Explore related subjects

ecologyPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Knief, U., Forstmeier, W.. 2025-08-07. Are our 95% CIs only worth 45% confidence?. https://doi.org/10.1101/2025.08.05.668631

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Phenotypic diversity acts as a higher-order functional trait of populations

Functional traits are usually assigned to individuals, yet some properties exist only as differences among population members. Whether such higher-order properties predict collective performance remains unclear. Here we used the aquatic plant Spirodela polyrhiza to test whether phenotypic and genetic differences among genotypes predict non-additive population growth. We grew twelve Japanese strains alone and in all pairwise mixtures and quantified their growth dynamics. Mixtures sometimes outperformed the mean of their component monocultures, and some exceeded the better-performing monoculture, although diversity effects were negative on average. Positive effects were most strongly associated with between-strain differences in frond colour and morphology, identifying specific dimensions of phenotypic diversity as population-level predictors of synergistic growth. Genome-wide higher-level association mapping identified loci where between-strain diversity was associated with growth effects, implicating stress responses, secondary metabolism and frond morphology. Loci associated with reduced performance implicated chemical perception and self/non-self recognition, whereas those associated with positive effects reflected functional differentiation in stress responses and physiological niches. Our findings suggest that some traits generate new functions at the population level through variation among individuals, making diversity itself a higher-order functional property.

ecology↗
bioRxiv2026

Developmental cold exposure increases the ability to maintain body temperature during future cold challenges in a free-living altricial bird

Developing in suboptimal temperatures can have widespread negative effects in organisms, but early exposure to thermal challenges can also trigger increased investment in thermoregulatory capacity. Within birds, the effects of cold exposure during incubation may differ between precocial and altricial species due to differences in the ontogeny of thermoregulation. To date, most of the evidence showing that embryonic cold exposure shapes future thermoregulatory capacity comes from a few precocial species in laboratory settings. Less is known about sensitivity to low-temperature incubation in altricial species which only become fully endothermic after hatching. In this study, we tested the effects of cold temperatures during embryonic development by temporarily cooling the nest boxes of free-living tree swallows (Tachycineta bicolor) during late incubation and subsequently exposing nestlings to an acute cold challenge. Cold exposure during early development made nestlings better at maintaining their body temperatures during the acute challenge. Developmentally cold-exposed birds also had higher thyroxine (T4) levels when sampled at lower ambient temperatures, whereas control birds showed the opposite relationship. Among nestlings that failed to maintain their body temperature during the acute cold challenge, developmentally cold-exposed nestlings secreted more corticosterone during the acute cold challenge than control nestlings. Despite these changes, cold-exposed birds did not differ from controls in cold-induced metabolic rates, body mass, or bacteria killing ability. These results are, to our knowledge, the first to show that developmental cold exposure increases the capacity to maintain body temperature during future cold challenges in a songbird. These findings suggest that experiencing even a mild decline in ambient temperatures during embryonic development could prime birds to cope more effectively with harsh or variable environmental conditions later in life.

ecology↗
bioRxiv2026

Winter-run Chinook salmon juvenile recruitment and early life history response to flow in a heavily altered tailwater

Diverse life history strategies allow species to spread risk across a mosaic of habitat conditions. In Chinook Salmon (Oncorhynchus tshawytscha), particularly in the Central Valley of California, this is expressed through varied adult run timings and juvenile outmigration strategies. Anthropogenic impacts, particularly dams, disrupt these fundamental life stage transitions, often forcing managed populations to adapt to homogenized, less stochastic hydrologic regimes. This is exemplified in the Sacramento River, where the conservation of endangered Sacramento River winter run Chinook salmon requires balancing complex water operations with the ecological needs of a population confined to a short stretch of suitable habitat downstream of Shasta Reservoir. Using a 23 year dataset (2002 to 2024), we evaluated the combined effects of flow, temperature, and spawner abundance on juvenile production and life history expression in this habitat. We found that flow and spawner abundance best predicted juvenile abundance at Red Bluff Diversion Dam while temperature had considerably less support. The proportion of juveniles migrating as smolts exhibited significant density dependence that was negatively correlated to both female spawner abundance and peak flows. These results suggest focusing on temperature management alone may be insufficient. Effective management strategies should consider flow variability and habitat restoration to facilitate varying migration strategies and expand upstream rearing capacity. By addressing these physical and hydrologic constraints, managers can better support the full suite of life history strategies necessary for the resilience of winter run Chinook salmon.

ecology↗