Biology subjects

Master, S. L.

Publications and source records attributed to Master, S. L..

6 recordsLinked to original sources

bioRxiv2023

Trying harder: how cognitive effort sculpts neural representations during working memory

The neural mechanisms by which motivational factors influence cognition remain unknown. Using fMRI, we tested how cognitive effort impacts working memory (WM). Participants were precued whether WM difficulty would be hard or easy. Hard trials demanded more effort as a later decision required finer mnemonic precision. Behaviorally, pupil size was larger and response times were slower on hard trials suggesting our manipulation of effort succeeded. Neurally, we observed robust persistent activity in prefrontal cortex, especially during hard trials. We found strong decoding of location in visual cortex, where accuracy was higher on hard trials. Connecting these across-region effects, we found that the amplitude of delay period activity in frontal cortex predicted decoded accuracy in visual cortex on a trial-wise basis. We conclude that the gain of persistent activity in frontal cortex may be the source of effort-related feedback signals that improve the quality of WM representations stored in visual cortex.

neuroscience↗

bioRxiv2023

Wagers for work: Decomposing the costs of cognitive effort

Some aspects of cognition are more taxing than others. Accordingly, many people will avoid cognitively demanding tasks in favor of simpler alternatives. Which components of these tasks are costly, and how much, remains unknown. Here, we use a novel task design in which subjects request wages for completing cognitive tasks and a computational modeling procedure that decomposes their wages into the costs driving them. Using working memory as a test case, our approach revealed that gating new information into memory and protecting against interference are costly. Critically, other factors, like memory load, appeared less costly. Other key factors which may drive effort costs, such as error avoidance, had minimal influence on wage requests. Our approach is sensitive to individual differences, and could be used in psychiatric populations to understand the true underlying nature of apparent cognitive deficits. Author SummaryAnyone who has tried to mentally calculate how much to tip at a restaurant knows that cognitive effort can feel aversive. Doing math in your head, like most high-level cognitive abilities, depends critically on working memory (WM). We know that WM is sometimes effortful to use, but we dont know which aspects of WM use drive these effort costs. To address this question, we had participants request wages in exchange for performing various tasks that differed in their specific WM demands. Using computational models of their wage demands, we demonstrated that some aspects of WM are costly, such as bringing new information into memory and preventing interference. Other factors, like the amount of information in memory and attempts to avoid mistakes, were less costly. Our approach identified which specific subcomponents of WM are aversive. Future research could use these methods to test theories about how motivational problems might be masquerading as cognitive deficits in psychiatric populations.

neuroscience↗

bioRxiv2021

Learning Rates Are Not All the Same: The Interpretation of Computational Model Parameters Depends on the Context

Reinforcement Learning (RL) models have revolutionized the cognitive and brain sciences, promising to explain behavior from simple conditioning to complex problem solving, to shed light on developmental and individual differences, and to anchor cognitive processes in specific brain mechanisms. However, the RL literature increasingly reveals contradictory results, which might cast doubt on these claims. We hypothesized that many contradictions arise from two commonly-held assumptions about computational model parameters that are actually often invalid: That parameters generalize between contexts (e.g., tasks, models) and that they capture interpretable (i.e., unique, distinctive) neurocognitive processes. To test this, we asked 291 participants aged 8-30 years to complete three learning tasks in one experimental session, and fitted RL models to each. We found that some parameters (exploration / decision noise) showed significant generalization: they followed similar developmental trajectories, and were reciprocally predictive between tasks. However, generalization was significantly below ceiling. Furthermore, other parameters (learning rates, forgetting) did not show evidence of generalization, and sometimes even opposite developmental trajectories. Interpretability was low for all parameters. We conclude that the systematic study of context factors (e.g., reward stochasticity; task volatility) will be necessary to enhance the generalizability and interpretability of computational cognitive models. HighlightsO_LIComputational cognitive modeling often makes two implicit assumptions: 1) Model parameters generalize between studies and models. 2) Parameters are interpretable, i.e., neurally, cognitively, and/or mechanistically specific. C_LIO_LIWe empirically investigate these assumptions in Reinforcement Learning (RL), using a large developmental dataset featuring three different learning tasks in a within-participant design. C_LIO_LIWe find that no parameters generalize between tasks in terms of absolute parameter values. However, RL decision noise/exploration parameters generalize in terms of between-participant variation, showing similar age trajectories across tasks. Interpretability was low for all parameters, especially those related to learning and memory. C_LIO_LIEven for parameters that showed generalization, the degree of generalization was significantly lower than in an artificial sample with perfect generalization for most parameters, revealing a relative lack of generalization. C_LIO_LIThese findings are in accordance with previous claims about the developmental trajectory of decision noise/exploration parameters, but suggest that claims about the development of learning rates or Forgetting might not be generalizable: these parameters are unlikely to capture the same neuro-cognitive processes across different tasks. C_LIO_LIWe suggest ways forward to improve the generalizability and interpretability of model parameters, in particular by focusing more on task context. C_LI

neuroscience↗

bioRxiv2020

Modeling Changes in Probabilistic Reinforcement Learning during Adolescence

In the real world, many relationships between events are uncertain and probabilistic. Uncertainty is also likely to be a more common feature of daily experience for youth because they have less experience to draw from than adults. Some studies suggests probabilistic learning may be inefficient in youth compared to adults [1], while others suggest it may be more efficient in youth that are in mid adolescence [2, 3]. Here we used a probabilistic reinforcement learning task to test how youth age 8-17 (N = 187) and adults age 18-30 (N = 110) learn about stable probabilistic contingencies. Performance increased with age through early-twenties, then stabilized. Using hierarchical Bayesian methods to fit computational reinforcement learning models, we show that all participants performance was better explained by models in which negative outcomes had minimal to no impact on learning. The performance increase over age was driven by 1) an increase in learning rate (i.e. decrease in integration time horizon); 2) a decrease in noisy/exploratory choices. In mid-adolescence age 13-15, salivary testosterone and learning rate were positively related. We discuss our findings in the context of other studies and hypotheses about adolescent brain development. Author summaryAdolescence is a time of great uncertainty. It is also a critical time for brain development, learning, and decision making in social and educational domains. There are currently contradictory findings about learning in adolescence. We sought to better isolate how learning from stable probabilistic contingencies changes during adolescence with a task that previously showed interesting results in adolescents. We collected a relatively large sample size (297 participants) across a wide age range (8-30), to trace the adolescent developmental trajectory of learning under stable but uncertain conditions. We found that age in our sample was positively associated with higher learning rates and lower choice exploration. Within narrow age bins, we found that higher saliva testosterone levels were associated with higher learning rates in participants age 13-15 years. These findings can help us better isolate the trajectory of maturation of core learning and decision making processes during adolescence.

neuroscience↗

bioRxiv2020

Understanding the Unique Advantage of Adolescents in Stochastic, Volatile Environments: Combining Reinforcement Learning and Bayesian Inference

During adolescence, youth venture out, explore the wider world, and are challenged to learn how to navigate novel and uncertain environments. We investigated whether adolescents are uniquely adapted to this transition, compared to younger children and adults. In a stochastic, volatile reversal-learning task with a sample of 291 participants aged 8-30, we found that adolescents outperformed both younger and older participants. We developed two independent cognitive models, based on Reinforcement learning (RL) and Bayesian inference (BI). The RL parameter for learning from negative outcomes and the BI parameters specifying participants mental models peaked closest to optimal in adolescents, suggesting a central role in adolescent cognitive processing. By contrast, persistence and noise parameters improved monotonously with age. We distilled the insights of RL and BI using principal component analysis and found that three shared components interacted to form the adolescent performance peak: adult-like behavioral quality, child-like time scales, and developmentally-unique processing of positive feedback. This research highlights adolescence as a neurodevelopmental window that may be specifically adapted for volatile and uncertain environments. It also shows how detailed insights can be gleaned by using cognitive models in new ways.

neuroscience↗

bioRxiv2019

Distentangling the systems contributing to changes in learning during adolescence

Multiple neurocognitive systems contribute simultaneously to learning. For example, dopamine and basal ganglia (BG) systems are thought to support reinforcement learning (RL) by incrementally updating the value of choices, while the prefrontal cortex (PFC) contributes different computations, such as actively maintaining precise information in working memory (WM). It is commonly thought that WM and PFC show more protracted development than RL and BG systems, yet their contributions are rarely assessed in tandem. Here, we used a simple learning task to test how RL and WM contribute to changes in learning across adolescence. We tested 187 subjects ages 8 to 17 and 53 adults (25-30). Participants learned stimulus-action associations from feedback; the learning load was varied to be within or exceed WM capacity. Participants age 8-12 learned slower than participants age 13-17, and were more sensitive to load. We used computational modeling to estimate subjects use of WM and RL processes. Surprisingly, we found more robust changes in RL than WM during development. RL learning rate increased significantly with age across adolescence and WM parameters showed more subtle changes, many of them early in adolescence. These results underscore the importance of changes in RL processes for the developmental science of learning.\n\nHighlights- Subjects combine reinforcement learning (RL) and working memory (WM) to learn\n- Computational modeling shows RL learning rates grew with age during adolescence\n- When load was beyond WM capacity, weaker RL compensated less in younger adolescents\n- WM parameters showed subtler and more puberty-related changes\n- WM reliance, maintenance, and capacity had separable developmental trajectories\n- Underscores importance of RL processes in developmental changes in learning

neuroscience↗