bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/2020.01.07.897991

A validation framework for neuroimagingsoftware: the case of populationreceptive fields

Lerma-Usabiaga, G.·Benson, N.·Winawer, J.·Wandell, B. A.
View original source ↗Export metadata ↓

Abstract

Neuroimaging software methods are complex, making it a near certainty that some implementations will contain errors. Modern computational techniques (i.e., public code and data repositories, continuous integration, containerization) enable the reproducibility of the analyses and reduce coding errors, but they do not guarantee the scientific validity of the results. It is difficult, nay impossible, for researchers to check the accuracy of software by reading the source code; ground truth test datasets are needed. Computational reproducibility means providing software so that for the same input anyone obtains the same result, right or wrong. Computational validity means obtaining the right result for the ground-truth test data. We describe a framework for validating and sharing software implementations. We apply the framework to an application: population receptive field (pRF) methods for functional MRI data. The framework is composed of three main components implemented with containerization methods to guarantee computational reproducibility: (1) synthesis of fMRI time series from ground-truth pRF parameters, (2) implementation of four public pRF analysis tools and standardization of inputs and outputs, and (3) report creation to compare the results with the ground truth parameters. We identified realistic conditions that lead to imperfect parameter recovery in all four implementations, and we provide means to reduce this problem. The computational validity framework supports scientific rigor and creativity, as opposed to the oft-repeated suggestion that investigators rely upon a few agreed upon packages. The framework and methods can be extended to other critical neuroimaging algorithms. Having validation frameworks help (1) developers to build new software, (2) research scientists to verify the softwares accuracy, and (3) reviewers to evaluate the methods used in publications and grants. Author SummaryComputer science provides powerful tools and techniques for implementing and deploying software. These techniques support software collaboration, reduce coding errors and enable reproducibility of the analyses. A further question is whether the software estimates are correct (valid). We describe a framework for validating and sharing software implementations based on ground-truth testing. We applied the framework to four separate applications that implemented population receptive field (pRF) estimates for functional MRI data. We quantified the validity, and we also documented limitations with these applications. Finally, we provide ways to mitigate these limitations. Implementing a software validation framework along with sharing and reproducibility is an important step for the complex methods used in neuroscience. Validation will help developers to build new software, researchers verify that the results are valid, and reviewers to evaluate the precision of methods in publications and grants.

Source connections

Explore related subjects

neurosciencePreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Lerma-Usabiaga, G., Benson, N., Winawer, J., Wandell, B. A.. 2020-01-08. A validation framework for neuroimagingsoftware: the case of populationreceptive fields. https://doi.org/10.1101/2020.01.07.897991

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Novel Tau Interaction Partners as Potential Biomarkers and Therapeutic Targets in Neurodegenerative Diseases

Alzheimers disease is a progressive neurodegenerative disorder leading to cognitive decline and ultimately death. One of the main hallmarks of this disease is the accumulation of pathological tau protein into neurofibrillary tangles. To better understand these pathological mechanisms, it is essential to characterize the tau protein interactome in health and disease. In this study, we performed a yeast two-hybrid screening using a human brain cDNA library, from which we identified 15 novel tau-interacting proteins that bind to full-length human tau, or its fragments: the N-terminus, microtubule-binding domain, or C-terminus. We subsequently validated these novel interactions using in vitro co-immunoprecipitation and colocalization assays with both physiological and pathological tau isoforms. Furthermore, their interactions with the human tau protein were analyzed using surface plasmon resonance. Finally, an in silico interactome analysis of these newly discovered tau-binding partners revealed several additional interactors that can link them to downstream processes critical to neurodegenerative pathways Our findings offer fresh insights into tau biology and highlight novel targets for therapeutic intervention and biomarker development.

neuroscience↗
bioRxiv2026

Early visual cortex is not functionally necessary for temporal attention: A TMS study

Temporal attention enhances the selective processing of information at a specific point in time and improves perception. Previous neuroimaging studies investigating its neural mechanisms have either conflated temporal attention with temporal expectation or provided correlational evidence. Thus, whether early visual cortex is functionally necessary for temporal attention remains unknown. Here, we combined psychophysics with transcranial magnetic stimulation (TMS) to examine whether V1/V2 are functionally necessary and play a causal role in temporal attention. Observers performed an orientation-discrimination task in which two consecutive stimuli, T1 and T2, were presented in the periphery, preceded by a precue providing either valid or neutral temporal information. Double-pulse TMS was delivered under four conditions: at T1 (T1-TMS), at T2 (T2-TMS), at both T1 and T2 (both-TMS), or not at all (none-TMS), to assess its effects on temporal attention and the efficacy of the stimulation was verified by pupil size analysis. Behavioral results revealed robust temporal attention benefits on d', reaction time, and the Balanced Integration Score (BIS) across all four TMS conditions, indicating that the benefits of temporal attention were unaffected by occipital TMS. Furthermore, pupil dilation was largest in the both-TMS condition, followed by the T1-TMS and T2-TMS conditions, and smallest in the none-TMS condition, providing physiological verification of the stimulation protocol. Together, these findings reveal the first causal evidence that the early visual cortex is not functionally necessary for endogenous temporal attention.

neuroscience↗
bioRxiv2026

Disentanglement of persistent and dynamic neural substrates of working memory

Competing theories of working memory have alternatively emphasized the importance of persistent and dynamic substrates of this cognitive function. Here, we directly tested these theories by computationally isolating the contributions of persistent and dynamic activity to working memory. We analyzed neuronal population activity recordings in macaques doing match/non-match working memory tasks. We first extracted signatures of persistent and dynamic activity through cue-specific temporal means, temporal principal components, and correlations of activity patterns across task epochs. We then generated synthetic data constrained by these signatures, and decoded cue stimuli from these data, separately from persistent and non-persistent neurons. We found that high decoding performance arose from a nuanced interplay of persistent and dynamic activity. Specifically, we could decode stimuli at empirical or above-chance levels across all tasks only from persistent neurons, but only after constraining correlations of activity patterns across epochs. In this way, we showed distinct roles for persistent and dynamic neuronal activity that help to unify two competing theories of working memory. More broadly, we also advanced computational controlled experiments as a principled framework for adjudicating between competing models of neuronal population activity.

neuroscience↗