Tracking neural representations of attended and unattended features in multisensory working memory over time

Abstract

Working memory supports goal-directed behavior by maintaining task-relevant information. However, a growing number of studies shows that even task-irrelevant features are automatically encoded and can interfere with the recall of task-relevant information. While this phenomenon is well documented in unisensory contexts, it remains unclear whether and how task-irrelevant information persists in multisensory working memory. We examine the role of cross-modal binding by tracking the dynamic neural representation of audio-visual objects under varying selective attention conditions, using an EEG-based audio-visual delayed-match-to-sample task. Participants attended to either auditory or visual features (selective attention), or to both features (conjunction) of two sequential audio-visual items and subsequently compared those task-relevant features to an audio-visual probe. Further, to investigate the influence of bottom-up factors on cross-modal binding, we manipulated spatial congruency by presenting both features from either the same (i.e., in the center) or from disparate positions. Behaviorally, task-irrelevant features interfered with performance even under selective attention, consistent with automatic cross-modal binding and encoding into working memory. This interference was even stronger with spatially disparate presentation, suggesting that bottom-up attentional dynamics strengthen cross-modal bindings and their subsequent storage. Condition-level representational similarity analysis (RSA) showed that EEG activity patterns under selective attention more closely resembled those of conjunction trials than unisensory trials, indicating that task-irrelevant features were incorporated into multisensory object-level representations. This conjunction-similarity persisted in attend-visual trials, but declined over time in attend-auditory trials, reflecting partial filtering of task-irrelevant orientations. Crucially, activity patterns never shifted fully towards an auditory-only profile, indicating that irrelevant visual features were not fully excluded. In addition, feature-level RSA corroborated that both sound frequency-specific and orientation-specific information were present in EEG activity patterns, irrespective of the attended modality. Overall, these results demonstrate the persistence of task-irrelevant information in multisensory working memory and offer critical insights into how attentional processes shape its representational architecture.