Timing Is Everything: Temporal Dynamics of BOLD Responses to Naturalistic Features in Movie Watching
Naturalistic stimuli, such as movies and narratives, are increasingly used in cognitive neuroscience to map cognitive and affective processes onto brain activity measured with functional MRI (fMRI). Features extracted from movies span multiple levels, from computational visual and auditory inputs to physiological signals and subjective ratings. However, the temporal alignment between these features and the blood-oxygen-level-dependent (BOLD) response varies considerably, and the commonly used canonical hemodynamic response function (HRF) with temporal derivatives may not adequately capture these delays. In this study, we analyzed three movie-watching datasets using cross-correlation and finite impulse response (FIR) deconvolution to map the unconstrained temporal dynamics of visual, auditory, pupillary, and Theory of Mind (ToM) features across the brain. Our results demonstrate that while the canonical HRF effectively captures basic sensory features, it introduces systematic misalignments for inherently delayed signals. Because physiological markers (pupil size) and subject reports (ToM) intrinsically lag the underlying neural events, standard HRF convolution overcompensates for their biological latency, introducing a redundant phase mismatch or "double-delay." Furthermore, our unconstrained FIR models revealed distinct inter-regional temporal hierarchies across the cortex. Recognizing the inherent collinearity of real-world stimuli, these estimated profiles capture the bundled, multi-dimensional dynamics of naturalistic processing rather than perfectly isolated feature effects. Ultimately, these findings highlight the necessity of flexible, reliability-tested temporal modeling to accurately map the complex processing timescales engaged during naturalistic viewing.