Biology subjects

Norman, K. A.

Publications and source records attributed to Norman, K. A..

At least 37 records · Page 2Linked to original sources

bioRxiv2021

Brain kernel: a new spatial covariance function for fMRI data

A key problem in functional magnetic resonance imaging (fMRI) is to estimate spatial activity patterns from noisy high-dimensional signals. Spatial smoothing provides one approach to regularizing such estimates. However, standard smoothing methods ignore the fact that correlations in neural activity may fall off at different rates in different brain areas, or exhibit discontinuities across anatomical or functional boundaries. Moreover, such methods do not exploit the fact that widely separated brain regions may exhibit strong correlations due to bilateral symmetry or the network organization of brain regions. To capture this non-stationary spatial correlation structure, we introduce the brain kernel, a continuous covariance function for whole-brain activity patterns. We define the brain kernel in terms of a continuous nonlinear mapping from 3D brain coordinates to a latent embedding space, parametrized with a Gaussian process (GP). The brain kernel specifies the prior covariance between voxels as a function of the distance between their locations in embedding space. The GP mapping warps the brain nonlinearly so that highly correlated voxels are close together in latent space, and uncorrelated voxels are far apart. We estimate the brain kernel using resting-state fMRI data, and we develop an exact, scalable inference method based on block coordinate descent to overcome the challenges of high dimensionality (10-100K voxels). Finally, we illustrate the brain kernels usefulness with applications to brain decoding and factor analysis with multiple task-based fMRI datasets.

neuroscience↗

bioRxiv2021

Increasing stimulus similarity drives nonmonotonic representational change in hippocampus

Studies of hippocampal learning have obtained seemingly contradictory results, with manipulations that increase coactivation of memories sometimes leading to differentiation of these memories, but sometimes not. These results could potentially be reconciled using the nonmonotonic plasticity hypothesis, which posits that representational change (memories moving apart or together) is a U-shaped function of the coactivation of these memories during learning. Testing this hypothesis requires manipulating coactivation over a wide enough range to reveal the full U-shape. To accomplish this, we used a novel neural network image synthesis procedure to create pairs of stimuli that varied parametrically in their similarity in high-level visual regions that provide input to the hippocampus. Sequences of these pairs were shown to human participants during high-resolution fMRI. As predicted, learning changed the representations of paired images in the dentate gyrus as a U-shaped function of image similarity, with neural differentiation occurring only for moderately similar images.

neuroscience↗

bioRxiv2021

High-order areas and auditory cortex both represent the high-level event structure of music

Recent fMRI studies of event segmentation have found that default mode regions represent high-level event structure during movie watching. In these regions, neural patterns are relatively stable during events and shift at event boundaries. Music, like narratives, contains hierarchical event structure (e.g., sections are composed of phrases). Here, we tested the hypothesis that brain activity patterns in default mode regions reflect the high-level event structure of music. We used fMRI to record brain activity from 25 participants (male and female) as they listened to a continuous playlist of 16 musical excerpts, and additionally collected annotations for these excerpts by asking a separate group of participants to mark when meaningful changes occurred in each one. We then identified temporal boundaries between stable patterns of brain activity using a hidden Markov model and compared the location of the model boundaries to the location of the human annotations. We identified multiple brain regions with significant matches to the observer-identified boundaries, including auditory cortex, medial prefrontal cortex, parietal cortex, and angular gyrus. From these results, we conclude that both higher-order and sensory areas contain information relating to the high-level event structure of music. Moreover, the higher-order areas in this study overlap with areas found in previous studies of event perception in movies and audio narratives, including regions in the default mode network. Significance StatementListening to music requires the brain to track dynamics at multiple hierarchical timescales. In our study, we had fMRI participants listen to real-world music (classical and jazz pieces) and then used an unsupervised learning algorithm (a hidden Markov model) to model the high-level event structure of music within participants brain data. This approach revealed that default mode brain regions involved in representing the high-level event structure of narratives are also involved in representing the high-level event structure of music. These findings provide converging support for the hypothesis that these regions play a domain-general role in processing events occurring over long timescales.

neuroscience↗

bioRxiv2020

Narratives: fMRI data for evaluating models of naturalistic language comprehension

The "Narratives" collection aggregates a variety of functional MRI datasets collected while human subjects listened to naturalistic spoken stories. The current release includes 345 subjects, 891 functional scans, and 27 diverse stories of varying duration totaling ~4.6 hours of unique stimuli (~43,000 words). This data collection is well-suited for naturalistic neuroimaging analysis, and is intended to serve as a benchmark for models of language and narrative comprehension. We provide standardized MRI data accompanied by rich metadata, preprocessed versions of the data ready for immediate use, and the spoken story stimuli with time-stamped phoneme- and word-level transcripts. All code and data are publicly available with full provenance in keeping with current best practices in transparent and reproducible neuroimaging.

neuroscience↗

bioRxiv2020

Learning to use episodic memory for event prediction

Recent human behavioral and neuroimaging results suggest that people are selective in when they encode and retrieve episodic memories. To explain these findings, we trained a memory-augmented neural network to use its episodic memory to support prediction of upcoming states in an environment where past situations sometimes reoccur. We found that the network learned to retrieve selectively as a function of several factors, including its uncertainty about the upcoming state. Additionally, we found that selectively encoding episodic memories at the end of an event (but not mid-event) led to better subsequent prediction performance. In all of these cases, the benefits of selective retrieval and encoding can be explained in terms of reducing the risk of retrieving irrelevant memories. Overall, these modeling results provide a resource-rational account of why episodic retrieval and encoding should be selective and lead to several testable predictions.

neuroscience↗

bioRxiv2020

Moment-by-moment tracking of naturalistic learning and its underlying hippocampo-cortical interactions

Every day our memory system achieves a remarkable feat: We form lasting memories of stimuli that were only encountered once. Here we investigate such learning as it naturally occurs during story listening, with the goal of uncovering when and how memories are stored and retrieved during processing of continuous, naturalistic stimuli. In behavioral experiments we confirm that, after a single exposure to a naturalistic story, participants can learn about its structure and are able to recall upcoming words in the story. In patients undergoing electrocorticographic recordings, we then track mnemonic information in high frequency activity (70 - 200Hz) as patients listen to a story twice. In auditory processing regions we demonstrate the rapid reinstatement of upcoming information after a single exposure; this neural measure of predictive recall correlates with behavioral measures of event segmentation and learning. Connectivity analyses on the neural data reveal information-flow from cortex to hippocampus at the end of events. On the second time of listening information-flow from hippocampus to cortex precedes moments of successful reinstatement.

neuroscience↗

bioRxiv2020

Thinking ahead: prediction in context as a keystone of language in humans and machines

Departing from traditional linguistic models, advances in deep learning have resulted in a new type of predictive (autoregressive) deep language models (DLMs). Using a self-supervised next-word prediction task, these models are trained to generate appropriate linguistic responses in a given context. We provide empirical evidence that the human brain and autoregressive DLMs share three fundamental computational principles as they process natural language: 1) both are engaged in continuous next-word prediction before word-onset; 2) both match their pre-onset predictions to the incoming word to calculate post-onset surprise (i.e., prediction error signals); 3) both represent words as a function of the previous context. In support of these three principles, our findings indicate that: a) the neural activity before word-onset contains context-dependent predictive information about forthcoming words, even hundreds of milliseconds before the words are perceived; b) the neural activity after word-onset reflects the surprise level and prediction error; and c) autoregressive DLM contextual embeddings capture the neural representation of context-specific word meaning better than arbitrary or static semantic embeddings. Together, our findings suggest that autoregressive DLMs provide a novel and biologically feasible computational framework for studying the neural basis of language.

neuroscience↗

bioRxiv2020

Sculpting New Visual Concepts into the Human Brain

Learning requires changing the brain. This typically occurs through experience, study, or instruction. We report a proof-of-concept for a new way for humans to acquire visual knowledge by directly sculpting activity patterns in the human brain that mirror those expected to arise through learning. We used a non-invasive technique (closed-loop real-time functional magnetic resonance imaging neurofeedback) to create new categories of visual objects in the brain, without the participants explicit awareness. After neural sculpting, participants exhibited behavioral and neural biases for the sculpted, but not for the control categories. The ability to sculpt new perceptual distinctions in the human brain offers a new paradigm for human fMRI research that allows for non-invasive, causal testing of the link between neural representations and behavior. As such, beyond its current application to perception, our work potentially has broad relevance to other domains of cognition such as decision-making, memory, and motor control. Significance StatementObjects that belong to the same category tend to elicit similar patterns of brain activity. Here we reverse this mapping and ask whether neural similarity is sufficient to induce increased perceptual discrimination and categorical perception. We do this by using real-time fMRI to modify neural representations of objects in high-level visual cortex. Participants viewed an object and received closed-loop neurofeedback that pushed them to represent the object more similarly to a brain activity pattern we chose for that category. After successfully self-modulating their brain activity, participants began to perceive objects assigned to the same brain pattern as more categorically distinct from those assigned to a different brain pattern. These findings open a new avenue for understanding and accelerating human learning.

neuroscience↗

bioRxiv2020

Closed-loop fMRI neurofeedback to reduce negative attentional bias in depression

Depressed individuals show an attentional bias toward negatively valenced stimuli and thoughts. Here we present a novel closed-loop neurofeedback procedure that seeks to remediate this bias. Internal attentional states were detected by applying machine learning techniques to fMRI data in real-time, and externalized using a visually presented stimulus that the participant could learn to control. We trained 15 depressed and 12 healthy control participants over three fMRI sessions, preceded and followed by behavioral and clinical assessments. Initially, depressed participants were more likely than non-depressed participants to get "stuck" in negative attentional states, but this diminished with neurofeedback training relative to controls. Depression severity also decreased from pre- to post-training. These results demonstrate that our method is sensitive to the negative attentional bias in depressed individuals, and its reduction after training showcases the potential of this method as a treatment in the future.

neuroscience↗

bioRxiv2020

Think Like an Expert: Neural Alignment Predicts Understanding in Students Taking an Introduction to Computer Science Course

How do students understand and remember new information? Despite major advances in measuring human brain activity during and after educational experiences, it is unclear how learners internalize new content, especially in real-life and online settings. In this work, we introduce a neural measure for predicting and assessing learning outcomes. Our approach hinges on the idea that successful learning involves forming the "right" set of neural representations, which are captured in "canonical" activity patterns shared across individuals. Specifically, we hypothesized that understanding is mirrored in "neural alignment": the degree to which an individual learners neural representations match those of experts, as well as those of other learners. We tested this hypothesis in a longitudinal functional MRI study that regularly scanned college students enrolled in an introduction to computer science course. We additionally scanned graduate student "experts" in computer science. We found that alignment among students successfully predicted overall performance in a final exam. Furthermore, within individual students, concepts that evoked better alignment with the experts and with their fellow students were better understood, revealing neural patterns associated with understanding specific concepts. These results provide support for a novel neural measure of concept understanding that can be used to assess and predict learning outcomes in real-life contexts.

neuroscience↗

bioRxiv2020

Behavioral, physiological, and neural signatures of surprise during naturalistic sports viewing

Surprise signals a discrepancy between past and current beliefs. It is theorized to be linked to affective experiences, the creation of particularly resilient memories, and segmentation of the flow of experience into discrete perceived events. However, the ability to precisely measure naturalistic surprise has remained elusive. We used advanced basketball analytics to derive a quantitative measure of surprise and characterized its behavioral, physiological, and neural correlates in human subjects observing basketball games. We found that surprise was associated with segmentation of ongoing experiences, as reflected by subjectively perceived event boundaries and shifts in neocortical patterns underlying belief states. Interestingly, these effects differed by whether surprising moments contradicted or bolstered current predominant beliefs. Surprise also positively correlated with pupil dilation, activation in subcortical regions associated with dopamine, game enjoyment, and long-term memory. These investigations support key predictions from event segmentation theory and extend theoretical conceptualizations of surprise to real-world contexts.

neuroscience↗

bioRxiv2020

Relating the past with the present: Information integration and segregation during ongoing narrative processing

This study examined how the brain dynamically updates event representations by integrating new information over multiple minutes while segregating irrelevant input. A professional writer custom-designed a narrative with two independent storylines, interleaving across minute-long segments (ABAB). In the last (C) part, characters from the two storylines meet and their shared history is revealed. Part C is designed to induce the spontaneous recall of past events, upon the recurrence of narrative motifs from A/B, and to shed new light on them. Our fMRI results showed storyline-specific neural patterns, which were reinstated (i.e. became more active) during storyline transitions. This effect increased along the processing timescale hierarchy, peaking in the default mode network. Similarly, the neural reinstatement of motifs was found during part C. Furthermore, participants showing stronger motif reinstatement performed better in integrating A/B and C events, demonstrating the role of memory reactivation in information integration over intervening irrelevant events.

neuroscience↗

bioRxiv2019

Leveraging shared connectivity to aggregate heterogeneous datasets into a common response space

Connectivity hyperalignment can be used to estimate a single shared response space across disjoint datasets. We develop a connectivity-based shared response model that factorizes aggregated fMRI datasets into a single reduced-dimension shared connectivity space and subject-specific topographic transformations. These transformations resolve idiosyncratic functional topographies and can be used to project response time series into shared space. We evaluate this algorithm on a large collection of heterogeneous, naturalistic fMRI datasets acquired while subjects listened to spoken stories. Projecting subject data into shared space dramatically improves between-subject story time-segment classification and increases the dimensionality of shared information across subjects. This improvement generalizes to subjects and stories excluded when estimating the shared space. We demonstrate that estimating a simple semantic encoding model in shared space improves between-subject forward encoding and inverted encoding model performance. The shared space estimated across all datasets is distinct from the shared space derived from any particular constituent dataset; the algorithm leverages shared connectivity to yield a consensus shared space conjoining diverse story stimuli.\n\nHighlightsO_LIConnectivity SRM estimates a single shared space across subjects and stimuli\nC_LIO_LITopographic transformations resolve idiosyncrasies across individuals\nC_LIO_LIShared connectivity space enhances spatiotemporal intersubject correlations\nC_LIO_LISemantic model-based encoding and decoding improves across subjects\nC_LIO_LITransformations project into a consensus space conjoining diverse stimuli\nC_LI

neuroscience↗

bioRxiv2019

Joy division: Reward prediction errors create event boundaries in memory

We remember when things change. Particularly salient are experiences where there is a change in rewards, eliciting reward prediction errors (RPEs). This feature of memory may be useful because it can help us find greater rewards and avoid lesser ones in the future. How do RPEs influence our memory of those experiences? One idea is that this signal directly enhances the encoding of memory. Another, not mutually exclusive, idea is that the RPE signals a deeper change in the environment, and leads to the mnemonic separation of subsequent experiences from what came before, thereby creating a new latent context and a more separate memory trace. We tested this in four experiments in which participants learned to predict rewards associated with a series of images within visually-distinct \"rooms.\" High magnitude RPEs indicated a change in the underlying distribution of rewards. To test whether these large RPEs created a new latent context, we first assessed recognition priming for sequential pairs that contained or did not contain a high-RPE event, as well as out-of-sequence pairs (Exp. 1: n=27 & Exp. 2: n=83). We found evidence of recognition priming for both sequential pair types, including the pair with the high-RPE event, indicating that the high-RPE event is bound to its predecessor in memory. Given that high-RPE events are themselves preferentially remembered (Rouhani et al, 2018), we next tested recognition priming for pairs that had one item in between them (i.e. the pairs were either across a high-RPE event or not), where none of the tested items were high-RPE items (Exp. 3: n=85). Here, sequential pairs across a high-RPE no longer showed recognition priming whereas pairs within the same latent reward state did, providing initial evidence for an RPE-modulated event boundary. We then investigated whether RPE event boundaries disrupt temporal memory of those events (Exp. 4). After reward learning, we asked participants to order and estimate the distance between two events that had either included a high-RPE event between them, or not. We found (n=49) and replicated (n=77) worse sequence memory for events across a high-RPE event. Altogether, these findings demonstrate that high-RPE events are both more strongly encoded and act as event boundaries that interrupt the sequential integration of events. We captured these effects in a variant of the Context Maintenance and Retrieval model (CMR; Polyn, Norman & Kahana, 2009), modified to incorporate RPEs into the encoding process.

neuroscience↗

bioRxiv2019

Temporal integration of narrative information in a hippocampal amnesic patient

Default network regions appear to integrate information over time windows of 30 seconds or more during narrative listening. Does this long-timescale capability require the hippocampus? Amnesic behavior suggests that the hippocampus may not be needed for online processing when input is continuous and semantically rich: amnesics can participate in conversations and tell stories spanning minutes, and when tested immediately on recently heard prose their performance is relatively preserved. We hypothesized that default network regions can integrate the semantically coherent information of a narrative across long time windows, even in the absence of the hippocampus. To test this prediction, we measured BOLD activity in the brain of a hippocampal amnesic patient (D. A.) and healthy control participants while they listened to a seven-minute narrative. The narrative was played either in its intact form, or as a paragraph-scrambled version, which has been previously shown to interfere with the long-range temporal dependencies in default network activity. In the intact story condition, D. A.s moment-by-moment BOLD activity spatial patterns were similar to those of controls in low-level auditory cortex as well as in some high-level default network regions (including lateral and medial posterior parietal cortex). Moreover, as in controls, D. A.s response patterns in medial and lateral posterior parietal cortex were disrupted when paragraphs of the story were presented in a shuffled order, suggesting that activity in these areas did depend on information from 30 seconds or more in the past. Together, these results suggest that some default network cortical areas can integrate information across long timescales, even in the absence of the hippocampus.

neuroscience↗

bioRxiv2019

Multiple memories can be simultaneously reactivated during sleep as effectively as a single memory

Memory consolidation involves the reactivation of memory traces during sleep. If many memories are reactivated each night, how much do they interfere with one another? To explore this question, we examined whether reactivating multiple memories incurs a cost to sleep-related benefits by contrasting reactivation of multiple memories versus single memories during sleep. First, participants learned the on-screen location of different images. Each image was part of a semantically interconnected group (e.g., images of different cats). Groups were comprised of either one, two, or six images. During sleep, group-related sounds (e.g., "meow") were unobtrusively presented to reactivate memories for half of the groups. The benefit in location recall for cued versus non-cued items was independent of the number of items in the group, suggesting that reactivation occurs in a simultaneous, promiscuous manner. Intriguingly, sleep spindles and delta-theta power modulations were sensitive to group size and reflected the extent of previous learning. Our results demonstrate that multiple memories may be consolidated in parallel without compromising each memorys sleep-related benefit, suggesting that the brains capacity for reactivation is not strictly limited by separate resources needed for individual memories. These findings highlight alternative models for parallel consolidation that should be considered in future studies.

neuroscience↗

bioRxiv2019

Rational use of Episodic and Working Memory: A Normative Account of Prospective Memory

AO_SCPLOWBSTRACTC_SCPLOWHumans often simultaneously pursue multiple plans at different time scales. The successful realization of non-immediate plans (e.g., post package after work) requires keeping track of a future plan while accomplishing other intermediate tasks (e.g., write a paper), a capacity known as prospective memory. This capacity requires the integration of noisy evidence from perceptual input with evidence from short-term working memory (WM) and longer-term or episodic memory (LTM/EM). Here we formulate a set of dual-task problems in empirical studies of prospective memory as problems of computational rationality, and ask how a rational model should exploit noisy perception and memory to maximize payoffs. The model combines reinforcement learning (optimal action selection) with evidence accumulation (optimal inference) in order to derive good decision parameters for optimal task performance (i.e., performing an ongoing task while monitoring for a cue that triggers executing a second prospective task). We compare model behavior to key accuracy and reaction time phenomena in human performance. Thus, we offer a normative approach to theorizing and modeling these phenomena without assumptions about mechanisms of attention or retrieval. This approach can be extended to study meta-parameters governing the boundedly rational use of memory in planned action in health, as well as compensatory mnemonic strategies that may be rational responses to disturbances of these mechanisms in neuropsychiatric disorders.

animal behavior and cognition↗

bioRxiv2019

Structured event memory: a neuro-symbolic model of event cognition

Humans spontaneously organize a continuous experience into discrete events and use the learned structure of these events to generalize and organize memory. We introduce the Structured Event Memory (SEM) model of event cognition, which accounts for human abilities in event segmentation, memory, and generalization. SEM is derived from a probabilistic generative model of event dynamics defined over structured symbolic scenes. By embedding symbolic scene representations in a vector space and parametrizing the scene dynamics in this continuous space, SEM combines the advantages of structured and neural network approaches to high-level cognition. Using probabilistic reasoning over this generative model, SEM can infer event boundaries, learn event schemata, and use event knowledge to reconstruct past experience. We show that SEM can scale up to high-dimensional input spaces, producing human-like event segmentation for naturalistic video data, and accounts for a wide array of memory phenomena.

neuroscience↗