bioRxiv Science⌕ Search

Biology subjects

Tejavibulya, L.

Publications and source records attributed to Tejavibulya, L..

8 recordsLinked to original sources

The Hidden Landscape of Missed Effects in Human Functional Neuroimaging

Functional neuroimaging aims to uncover brain processes underlying behavior and disease, yet studies are often underpowered to detect these effects. How this literature has shaped our understanding of brain function remains unknown, and little guidance exists for planning better powered studies. An underappreciated barrier is that commonly reported effect sizes across the brain are inflated, biasing study planning. Here, we introduce a correction for this inflation bias and show how more accurate studies can be planned using corrected effect size benchmarks from a mega-analysis of 63 typical studies across seven large datasets (52,979 participants). We find that common methods of planning studies based on uncorrected effects lead to roughly half the expected detections at typical sample sizes, with limited spatial overlap with original findings. These missed effects collectively explain meaningful additional variance in the desired outcome. We show how to recover missed effects by planning not only for power but also for a target number of detections via corrected benchmarks, or by taking a whole-brain approach with multivariate effects that individual research groups can detect (n < 50 compared to n > 1,000 for a typical univariate effect). These findings lay the groundwork for more informed study planning and a richer understanding of the widespread nature of brain effects, with implications for shared challenges (and solutions) across biomedicine.

neuroscience↗

Overlooked features lead to divergent neurobiological interpretations of brain-based machine learning biomarkers

A central objective in human neuroimaging is to understand the neurobiology underlying cognition and mental health. Machine learning models trained on brain connectivity data are increasingly used as tools for predicting behavioral phenotypes 1,2, enhancing precision medicine 3,4, and improving generalizability compared to traditional MRI studies 5. However, the high dimensionality of brain connectivity data makes model interpretation challenging 6. Prevailing practices within the field rely on sparsely selected brain connectivity features, implicitly interpreting identified feature networks as uniquely representative of a given phenotype while overlooking others. Here, we show that commonly overlooked brain connectivity features can achieve similar prediction accuracies while yielding markedly different neurobiological interpretations. Using four large-scale neuroimaging datasets spanning over 12,000 participants and 13 outcomes, we demonstrate that this phenomenon is widespread across cognitive, developmental, and psychiatric phenotypes. It extends to both functional connectivity (fMRI) and structural (DTI) connectomes and remains evident even in external validation. These findings suggest that common practices may lead to feature overinterpretation and a misrepresentation of the neurobiological bases of brain-behavior associations. Such interpretations present only the "tip of the iceberg" when certain disregarded features may be just as meaningful, potentially contributing to ongoing issues surrounding reproducibility within the field. More broadly, our results point to the possibility that multiple neurobiologically distinct models may exist for the same phenotype, with implications for identifying meaningful subtypes within clinical and research populations.

neuroscience↗

Variation in moment-to-moment brain state engagement changes across development and contributes to individual differences in executive function

Neural variability, or variation in brain signals, facilitates dynamic brain responses to ongoing demands. This flexibility is important during development from childhood to young adulthood, a period characterized by rapid changes in experience. However, little is known about how variability in the engagement of recurring brain states changes during development. Such investigations would require the continuous assessment of multiple brain states concurrently. Here, we leverage a new computational framework to study state engagement variability (SEV) during development. A consistent pattern of SEV changing with age was identified across cross-sectional and longitudinal datasets (N>3000). SEV developmental trajectories stabilize around mid-adolescence, with timing varying by sex and brain state. SEV successfully predicts executive function (EF) in youths from an independent dataset. Worse EF is further linked to alterations in SEV development. These converging findings suggest SEV changes over development, allowing individuals to flexibly recruit various brain states to meet evolving needs.

neuroscience↗

Brain-handedness associations depend on how and when handedness is measured

Hand preference is ubiquitous, intuitive, and often simplified to right- or left-handed. Accordingly, differences between right- and left-handed individuals in the brain have been established. Nevertheless, considering handedness as a binarized construct fails to capture the variability of brain-handedness associations across different domains or activities. Further, many cultures, environments, and generations impose right-handed norms, and handedness preferences can change over the lifespan. As a result, brain-handedness associations may depend on how and when handedness is measured. We used two large datasets, the Human Connectome Project-Development (HCP-D; n=465; age=5-21 years) and Human Connectome Project-Aging (HCP-A; n=368; age=36-100 years), to explore handedness preferences and brain-handedness associations. Nine items from the Edinburgh Handedness Inventory were associated with resting-state functional connectomes. We show that brain-handedness associations differed across the two cohorts. Moreover, these differences depended on the way handedness was measured. Given that brain-handedness associations differ across handedness measures and datasets, we caution against a one-size-fits-all approach to neuroimaging studies of this complex trait.

neuroscience↗

Brain-phenotype predictions can survive across diverse real-world data

Recent work suggests that machine learning models predicting psychiatric treatment outcomes based on clinical data may fail when applied to unharmonized samples. Neuroimaging predictive models offer the opportunity to incorporate neurobiological information, which may be more robust to dataset shifts. Yet, among the minority of neuroimaging studies that undertake any form of external validation, there is a notable lack of attention to generalization across dataset-specific idiosyncrasies. Research settings, by design, remove the between-site variations that real-world and, eventually, clinical applications demand. Here, we rigorously test the ability of a range of predictive models to generalize across three diverse, unharmonized samples: the Philadelphia Neurodevelopmental Cohort (n=1291), the Healthy Brain Network (n=1110), and the Human Connectome Project in Development (n=428). These datasets have high inter-dataset heterogeneity, encompassing substantial variations in age distribution, sex, racial and ethnic minority representation, recruitment geography, clinical symptom burdens, fMRI tasks, sequences, and behavioral measures. We demonstrate that reproducible and generalizable brain-behavior associations can be realized across diverse dataset features with sample sizes in the hundreds. Results indicate the potential of functional connectivity-based predictive models to be robust despite substantial inter-dataset variability. Notably, for the HCPD and HBN datasets, the best predictions were not from training and testing in the same dataset (i.e., cross-validation) but across datasets. This result suggests that training on diverse data may improve prediction in specific cases. Overall, this work provides a critical foundation for future work evaluating the generalizability of neuroimaging predictive models in real-world scenarios and clinical settings.

neuroscience↗

Power and reproducibility in the external validation of brain-phenotype predictions

Identifying reproducible and generalizable brain-phenotype associations is a central goal of neuroimaging. Consistent with this goal, prediction frameworks evaluate brain-phenotype models in unseen data. Most prediction studies train and evaluate a model in the same dataset. However, external validation, or the evaluation of a model in an external dataset, provides a better assessment of robustness and generalizability. Despite the promise of external validation and calls for its usage, the statistical power of such studies has yet to be investigated. In this work, we ran over 60 million simulations across several datasets, phenotypes, and sample sizes to better understand how the sizes of the training and external datasets affect statistical power. We found that prior external validation studies used sample sizes prone to low power, which may lead to false negatives and effect size inflation. Furthermore, increases in the external sample size led to increased simulated power directly following theoretical power curves, whereas changes in the training dataset size offset the simulated power curves. Finally, we compared the performance of a model within a dataset to the external performance. The within-dataset performance was typically within r=0.2 of the cross-dataset performance, which could help decide how to power future external validation studies. Overall, our results illustrate the importance of considering the sample sizes of both the training and external datasets when performing external validation.

neuroscience↗

The effects of data leakage on neuroimaging predictive models

Predictive modeling has now become a central technique in neuroimaging to identify complex brain-behavior relationships and test their generalizability to unseen data. However, data leakage, which unintentionally breaches the separation between data used to train and test the model, undermines the validity of predictive models. Previous literature suggests that leakage is generally pervasive in machine learning, but few studies have empirically evaluated the effects of leakage in neuroimaging data. Although leakage is always an incorrect practice, understanding the effects of leakage on neuroimaging predictive models provides insight into the extent to which leakage may affect the literature. Here, we investigated the effects of leakage on machine learning models in two common neuroimaging modalities, functional and structural connectomes. Using over 400 different pipelines spanning four large datasets and three phenotypes, we evaluated five forms of leakage fitting into three broad categories: feature selection, covariate correction, and lack of independence between subjects. As expected, leakage via feature selection and repeated subjects drastically inflated prediction performance. Notably, other forms of leakage had only minor effects (e.g., leaky site correction) or even decreased prediction performance (e.g., leaky covariate regression). In some cases, leakage affected not only prediction performance, but also model coefficients, and thus neurobiological interpretations. Finally, we found that predictive models using small datasets were more sensitive to leakage. Overall, our results illustrate the variable effects of leakage on prediction pipelines and underscore the importance of avoiding data leakage to improve the validity and reproducibility of predictive modeling.

neuroscience↗

Large-scale differences in functional organization of left- and right-handed individuals using whole-brain, data-driven analysis of connectivity

Handedness influences differences in lateralization of language areas as well as dominance of motor and somatosensory cortices. However, differences in whole brain functional organization due to handedness have been relatively understudied beyond pre-specified networks of interest. Functional connectivity offers the ability to unravel differences in the functional organization of the whole brain. Here, we compared connectivity profiles of left- and right-handed individuals using data-driven parcellations of the whole brain. We explored differences in connectivity profiles of previously established regions of interest, and showed functional organization differences between primarily left- and primarily right-handed individuals in the motor, somatosensory, and language areas using functional connectivity. We then proceeded to investigate these differences in the whole brain and found that the functional organization of left- and right-handed individuals are not specific to regions of interest. In particular, we found that connections between and within-hemispheres and the cerebellum show distinct patterns of connectivity. Together these results shed light on regions of the brain beyond those traditionally explored that contribute to differences in the functional organization of left- and right-handed individuals.

neuroscience↗