bioRxiv Science⌕ Search

Biology subjects

Soltani, A.

Publications and source records attributed to Soltani, A..

At least 19 recordsLinked to original sources

Value-guided attention links what we learn to how much we learn

Learning in uncertain environments requires identifying the relevant associations between stimuli, actions, and outcomes and determining how strongly to update these associations. Although often treated separately, these components likely interact in the brain. We hypothesized that this interaction shapes individual learning rates according to cue-choice alignment and reward outcome, thereby improving discrimination between competing cues. We tested this hypothesis using a probabilistic learning task in which human participants predicted outcomes based on multiple cues and reward feedback. We measured gaze and manipulated cue saliency to assess and influence which cues were preferentially processed during choice and feedback. Computational modeling revealed that learning rates were selectively enhanced for cues supporting the chosen option after reward and for cues opposing it after no reward. This learning-rate asymmetry based on cue-choice alignment sharpened discrimination among predictive cues, increased robustness to noise, and improved performance. Moreover, differential gaze toward supporting and opposing cues predicted this asymmetry, which was causally altered by manipulating cue saliency. Together, our results suggest that attention provides a unifying mechanism for coordinating what we learn from with how much we learn, helping preserve distinctions among competing cues and bringing several learning asymmetries within a common framework.

neuroscience↗

Mecp2 deficiency impairs microscale cortical network topology and dynamics in a Rett syndrome mouse model

Rett syndrome is a debilitating neurodevelopmental disorder with cerebral processing impairments caused by MECP2 loss-of-function mutations. Mecp2-deficient mouse models reveal disruptions of microscale cortical circuits. Yet how cellular-scale information processing is altered in Mecp2-deficient microscale functional networks is unknown. We investigated the development of functional connectivity, network topology, and dynamics in microelectrode array (MEA) recordings of primary cortical cultures from Mecp2-deficient and wild-type mice. Mecp2-deficient cortical networks developed more slowly and showed decreased functional connectivity compared to wild-type, leading to smaller network size, density, and strength of connectivity. Altered network topological features in Mecp2-deficient microscale circuits predicted decreased efficiency and information-sharing capacity. This reveals developmental deficits in microscale functional networks, which may in turn underlie the cortical decline and severe cognitive disability in Rett syndrome. These findings also offer circuit-level targets and an in-vitro approach for evaluating new therapeutic products for restoring microscale network function.

neuroscience↗

Sex differences in task engagement and lapse rate during reward learning plateaus

Our understanding of sex differences in reward learning has been limited due to the predominant study of males. Here, we evaluated sex differences in flexible learning in two domains: the learning of stimulus-and action-based associations and their reversals. During action-based learning, rats selected between two identical visual stimuli presented on a touchscreen, where the spatial location predicted a higher probability of reward. For stimulus-based learning, rats chose between two distinct visual stimuli presented in pseudorandom spatial locations, one of which was associated with a higher probability of reward. Reversal phases involved switching reward contingency between the two actions or stimuli. We found that females did not differ in their discrimination or reversal learning accuracy compared to males, yet they collected fewer rewards and were more likely to omit trials than males in both domains. To gain a detailed understanding of differences across conditions, we modeled animals trial-by-trial choices using reinforcement learning (RL) models and examined their steady-state behavior to capture transitions between distinct behavioral states. Although the estimated parameters of the best-fitting RL model revealed some sex differences, the model that incorporated transitions between different behavioral states provided a better overall fit to the data. This model also revealed that across all reversal phases, females exhibited a higher transition-specific lapse rate than males, indicating greater task disengagement once there was no need for further learning. Together, our fine-grained analysis of behavior adds to a growing literature on sex differences in flexible reward learning.

animal behavior and cognition↗

Neural mechanisms underlying robust target selection in response to microstimulation of the oculomotor system

Despite its prevalence in studying the causal roles of different brain circuits in cognitive processes, electrical microstimulation often results in inconsistent behavioral effects. These inconsistencies are assumed to be due to multiple mechanisms, including habituation, compensation by other brain circuits, and contralateral suppression. Considering the presence of reinforcement in most experimental paradigms, we hypothesized that interactions between reward feedback and microstimulation could contribute to inconsistencies in behavioral effects of microstimulation. To test this, we analyzed data from electrical microstimulation of the frontal eye field of male macaques during a value-based decision-making task and constructed network models to capture choice behavior. We found evidence for microstimulation-dependent adaptation in saccadic choice, such that in stimulated trials, monkeys choices were biased toward the target in the response field of the microstimulated site (Tin). In contrast, monkeys showed a bias away from Tin in non-stimulated trials following microstimulation. Critically, this bias slowly decreased as a function of the time since the last stimulation. Moreover, microstimulation-dependent adaptation was influenced by reward outcomes in preceding trials. Despite these local effects, we found no evidence for the global effects of microstimulation on learning and sensitivity to the reward schedule. By simulating choice behavior across various network models, we found a model in which microstimulation and reward-value signals interact competitively through reward-dependent plasticity can best account for our observations. Our findings indicate a reward-dependent compensatory mechanism that enhances robustness to perturbations within the oculomotor system and could explain the inconsistent outcomes observed in previous microstimulation studies.

neuroscience↗

Contribution of amygdala to dynamic model arbitration under uncertainty

Intrinsic uncertainty in the reward environment requires the brain to run multiple models simultaneously to predict outcomes based on preceding cues or actions, commonly referred to as stimulus- and action-based learning. Ultimately, the brain also must adopt appropriate choice behavior using reliability of these models. Here, we combined multiple experimental and computational approaches to quantify concurrent learning in monkeys performing tasks with different levels of uncertainty about the model of the environment. By comparing behavior in control monkeys and monkeys with bilateral lesions to the amygdala or ventral striatum, we found evidence for dynamic, competitive interaction between stimulus-based and action-based learning, and for a distinct role of the amygdala. Specifically, we demonstrate that the amygdala adjusts the initial balance between the two learning systems, thereby altering the interaction between arbitration and learning that shapes the time course of both learning and choice behaviors. This novel role of the amygdala can account for existing contradictory observations and provides testable predictions for future studies into circuit-level mechanisms of flexible learning and choice under uncertainty.

neuroscience↗

Contribution of statistical learning to mitigating the curse of dimensionality in reinforcement learning

Natural environments are rich with patterns and regularities. Thus, it is not surprising that most animals have evolved neural mechanisms to detect and adapt to these regularities. Such detections and adaptations can, in turn, influence several cognitive functions including attention, perception, and memory, thereby enhancing survival. Here, we investigated whether detecting environmental regularities that are irrelevant to obtaining rewards can influence learning about multi-dimensional choice options--a process often constrained by the scarcity of reward feedback. To explore this, we trained human participants to perform a multidimensional reward- learning task alongside an orthogonal sequence-prediction task. We found that although feature- specific regularities in the sequence-prediction task were not predictive of reward, they incidentally biased participants behavior toward the feature with regularity during the reward- learning task. Fitting choice behavior with various computational models revealed that this effect was more consistent with modulations in learning rather than decision making, as evidenced by higher learning rates for this feature. This was particularly apparent for learning from chosen, rewarded options and unchosen, unrewarded options, demonstrating that environmental regularities can amplify confirmation bias in reward learning. Our results thus extend the notion of confirmation bias in learning about options and actions to their features. Furthermore, they suggest that temporal regularities can influence reward learning by biasing the association of reward with specific features and by enhancing confirmation bias. These effects help reduce dimensionality of the learning task to mitigate the curse of dimensionality in reward learning.

neuroscience↗

Training-dependent gradients of timescales of neural dynamics in the primate prefrontal cortex and their contributions to working memory

Cortical neurons exhibit multiple timescales related to dynamics of spontaneous fluctuations (intrinsic timescales) and response to task events (seasonal timescales) in addition to selectivity to task-relevant signals. These timescales increase systematically across the cortical hierarchy, e.g., from parietal to prefrontal and cingulate cortex, pointing to their role in cortical computations. It is currently unknown whether these timescales depend on training in a specific task and/or are an inherent property of neurons, and whether more fine-grained hierarchies of timescales exist within specific cortical regions. To address these questions, we analyzed single-cell recordings within five subregions of the prefrontal cortex (PFC) of male macaques before and after training on a working-memory task. We found fine-grained but opposite gradients of intrinsic and seasonal timescales that mainly appeared after training. Intrinsic timescales decreased whereas seasonal timescales increased from posterior to anterior subregions within both dorsal and ventral PFC. Moreover, training was accompanied by increases in proportions of neurons that exhibited intrinsic and seasonal timescales. These effects were comparable to the emergence of response selectivity due to training. Finally, task selectivity accompanied opposite neural dynamics such that neurons with task-relevant selectivity exhibited longer intrinsic and shorter seasonal timescales. Notably, neurons with longer intrinsic and shorter seasonal timescales exhibited superior population-level coding, but these advantages extended to the delay period mainly after training. Together, our results provide evidence for plastic, fine-grained gradients of timescales within PFC that can influence both single-cell and population coding, pointing to the importance of these timescales in understanding cognition. Significance statementRecent studies have demonstrated that neural responses exhibit dynamics with different timescales that follow a certain order or hierarchy across cortical areas. While the hierarchy of timescales is consistent across different tasks, it is unknown if these timescales emerge only after training or if they represent inherent properties of neurons. To answer this question, we estimated multiple timescales in neural response across five subregions of the monkeys lateral prefrontal cortex before and after training on a working-memory task. Our results provide evidence for fine-grained gradients related to certain neural dynamics. Moreover, we show that these timescales depend on and can be modulated by training in a cognitive task, and contribute to encoding of task-relevant information at single-cell and population levels.

neuroscience↗

Hierarchical gradient of timescales in the mammalian forebrain

Many anatomical and physiological features of cortical circuits, ranging from the biophysical properties of synapses to the connectivity patterns among different neuron types, exhibit consistent variation along the hierarchical axis from sensory to association areas. Notably, the scale of temporal correlation of neural activity at rest, known as the intrinsic timescale, increases systematically along this hierarchy in both primates and rodents, analogous to the growing scale and complexity of spatial receptive fields. However, how the timescales for task-related activity vary across brain regions and whether their hierarchical organization appears consistently across different mammalian species remain unexplored. Here, we show that both the intrinsic timescale and the timescales of task-related activity follow a similar hierarchical gradient in the cortices of monkeys, rats, and mice. We also found that these timescales covary similarly in both the cortex and basal ganglia, whereas the timescales of thalamic activity are shorter than cortical timescales and do not conform to the hierarchical order predicted by their cortical projections. These results suggest that the hierarchical gradient of cortical timescales might be a universal feature of intra-cortical circuits in the mammalian brain. Significance StatementA gradual increase in the intrinsic timescales of cortical activity along the anatomical hierarchy reflects the functional specialization of cortical circuits. However, it is unknown whether this gradient of timescales is a common feature across distinct mammalian species in both intrinsic and task-related timescales and whether it is also observed in subcortical areas. This study reveals that the hierarchical gradient of multiple cortical timescales is conserved across multiple mammalian species. By contrast, thalamic timescales were shorter than cortical timescales and did not follow the hierarchical order inferred from their cortical projections. These findings imply a crucial role of intra-cortical connections in structuring distinct temporal dynamics observed across the cortex.

neuroscience↗

Contributions of attention to learning in multi-dimensional reward environments

Real-world choice options have many features or attributes, whereas the reward outcome from those options only depends on a few features/attributes. It has been shown that humans learn and combine feature-based with more complex conjunction-based learning to tackle challenges of learning in complex reward environments. However, it is unclear how different learning strategies interact to determine what features should be attended and control choice behavior, and how ensuing attention modulates future learning and/or choice. To address these questions, we examined human behavior during a three-dimensional learning task in which reward outcomes for different stimuli could be predicted based on a combination of an informative feature and conjunction. Using multiple approaches, we first confirmed that choice behavior and reward probabilities estimated by participants were best described by a model that learned the predictive values of both the informative feature and the informative conjunction. In this model, attention was controlled by the difference in these values in a cooperative manner such that attention depended on the integrated feature and conjunction values, and the resulting attention weights modulated learning by increasing the learning rate on attended features and conjunctions. However, there was little effect of attention on decision making. These results suggest that in multidimensional environments, humans direct their attention not only to selectively process reward-predictive attributes, but also to find parsimonious representations of the reward contingencies for more efficient learning. Significance StatementFrom trying exotic recipes to befriending new social groups, outcomes of real-life actions depend on many factors, but how do we learn the predictive values of those factors based on feedback we receive? It has been shown that humans simplify this problem by focusing on individual factors that are most predictive of the outcomes but can extend their learning strategy to include combinations of factors when necessary. Here, we examined interaction between attention and learning in a multidimensional reward environment that requires learning about individual features and their conjunctions. Using multiple approaches, we found that learning about features and conjunctions control attention in a cooperative manner and that the ensuing attention mainly modulates future learning and not decision making.

neuroscience↗

Sex-dependent contributions of ventrolateral orbitofrontal cortex and basolateral amygdala to learning under uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach and particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory DREADDs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-post reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of vlOFC, but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment. Significance StatementInflexible learning is a feature of several neuropsychiatric disorders. We investigated how the ventrolateral orbitofrontal cortex (vlOFC) and basolateral amygdala (BLA) are involved in learning of stimuli or actions under reinforcement uncertainty. Following chemogenetic inhibition of these regions in both male and females, we measured learning and adjustments to deterministic and probabilistic reversals. For action learning, BLA, but not vlOFC, is needed for probabilistic reversal learning. However, BLA is not necessary for initial probabilistic learning or retention, indicating a critical role for learning of unexpected changes. For stimulus learning, vlOFC, but not BLA, is required for adjustments to reversals, particularly in females. These findings provide insight into the complementary cortico-amygdalar substrates of learning under different forms of uncertainty.

neuroscience↗

Mechanisms of adjustments to different types of uncertainty in the reward environment across mice and monkeys

Despite being unpredictable and uncertain, reward environments often exhibit certain regularities, and animals navigating these environments try to detect and utilize such regularities to adapt their behavior. However, successful learning requires that animals also adjust to uncertainty associated with those regularities. Here, we analyzed choice data from two comparable dynamic foraging tasks in mice and monkeys to investigate mechanisms underlying adjustments to different types of uncertainty. In these tasks, animals selected between two choice options that delivered reward probabilistically, while baseline reward probabilities changed after a variable number (block) of trials without any cues to the animals. To measure adjustments in behavior, we applied multiple metrics based on information theory that quantify consistency in behavior, and fit choice data using reinforcement learning models. We found that in both species, learning and choice were affected by uncertainty about reward outcomes (in terms of determining the better option) and by expectation about when the environment may change. However, these effects were mediated through different mechanisms. First, more uncertainty about the better option resulted in slower learning and forgetting in mice, whereas it had no significant effect in monkeys. Second, expectation of block switches accompanied slower learning, faster forgetting, and increased stochasticity in choice in mice, whereas it only reduced learning rates in monkeys. Overall, while demonstrating the usefulness of entropy-based metrics in studying adaptive behavior, our study provides evidence for multiple types of adjustments in learning and choice behavior according to uncertainty in the reward environment.

neuroscience↗

Novel entropy-based metrics for predicting choice behavior based on local response to reward

For decades, behavioral scientists have used the matching law to quantify how animals distribute their choices between multiple options in response to reinforcement they receive. More recently, many reinforcement learning (RL) models have been developed to explain choice by integrating reward feedback over time. Despite reasonable success of RL models in capturing choice on a trial-by-trial basis, these models cannot capture variability in matching. To address this, we developed novel metrics based on information theory and applied them to choice data from dynamic learning tasks in mice and monkeys. We found that a single entropy-based metric can explain 50% and 41% of variance in matching in mice and monkeys, respectively. We then used limitations of existing RL models in capturing entropy-based metrics to construct a more accurate model of choice. Together, our novel entropy-based metrics provide a powerful, model-free tool to predict adaptive choice behavior and reveal underlying neural mechanisms.

neuroscience↗

Neural mechanisms of distributed value representations and learning strategies

Learning appropriate representations of the reward environment is extremely challenging in the real world where there are many options to learn about and these options have many attributes or features. Despite existence of alternative solutions for this challenge, neural mechanisms underlying emergence and adoption of value representations and learning strategies remain unknown. To address this, we measured learning and choice during a novel multi-dimensional probabilistic learning task in humans and trained recurrent neural networks (RNNs) to capture our experimental observations. We found that participants estimate stimulus-outcome associations by learning and combining estimates of reward probabilities associated with the informative feature followed by those of informative conjunctions. Through analyzing representations, connectivity, and lesioning of the RNNs, we demonstrate this mixed learning strategy relies on a distributed neural code and distinct contributions of inhibitory and excitatory neurons. Together, our results reveal neural mechanisms underlying emergence of complex learning strategies in naturalistic settings.

neuroscience↗

Unique features of stimulus-based probabilistic reversal learning

Reversal learning paradigms are widely-used assays of behavioral flexibility with their probabilistic versions being more amenable to studying integration of reward outcomes over time. Prior research suggests differences between initial and reversal learning, including higher learning rates, a greater need for inhibitory control, and more perseveration after reversals. However, it is not well-understood what aspects of stimulus-based reversal learning are unique to reversals, and whether and how observed differences depend on reward probability. Here, we used a visual probabilistic discrimination and reversal learning paradigm where male and female rats selected between a pair of stimuli associated with different reward probabilities. We compared accuracy, rewards collected, omissions, latencies, win-stay/lose-shift strategies, and indices of perseveration across two different reward probability schedules. We found that discrimination and reversal learning are behaviorally more unique than similar: latencies to select the better option, win-stay strategies, and perseveration were of a different pattern in reversal learning compared to discrimination learning. Additionally, fit of choice behavior using reinforcement learning models revealed a lower sensitivity to the difference in subjective reward values (greater exploration) and higher learning rates for the reversal phase. Interestingly, a consistent reward probability group difference emerged with a richer environment associated with longer reward collection latencies than a leaner environment. We also replicated previous reports on sex differences in reversal learning. Future studies should systematically compare the neural correlates of fine-grained behavioral measures to reveal possible dissociations in how the circuitry is recruited in each phase.

animal behavior and cognition↗

Separable influences of reward value on visual processing and choice

Primate vision is characterized by constant, sequential processing and selection of visual targets to fixate. Although expected reward is known to influence both processing and selection of visual targets, similarities and differences between these effects remains unclear mainly because they have been measured in separate tasks. Using a novel paradigm, we simultaneously measured the effects of reward outcomes and expected reward on target selection and sensitivity to visual motion in monkeys. Monkeys freely chose between two visual targets and received a juice reward with varying probability for eye movements made to either of them. Targets were stationary apertures of drifting gratings, causing the endpoints of eye movements to these targets to be systematically biased in the direction of motion. We used this motion-induced bias as a measure of sensitivity to visual motion on each trial. We then performed different analyses to explore effects of objective and subjective reward values on choice and sensitivity to visual motion in order to find similarities and differences between reward effects on these two processes. Specifically, we used different reinforcement learning models to fit choice behavior and estimate subjective reward values based on the integration of reward outcomes over multiple trials. Moreover, to compare the effects of subjective reward value on choice and sensitivity to motion directly, we considered correlations between each of these variables and integrated reward outcomes on a wide range of timescales. We found that in addition to choice, sensitivity to visual motion was also influenced by subjective reward value, even though the motion was irrelevant for receiving reward. Unlike choice, however, sensitivity to visual motion was not affected by objective measures of reward value. Moreover, choice was determined by the difference in subjective reward values of the two options whereas sensitivity to motion was influenced by the sum of values. Finally, models that best predicted visual processing and choice used sets of estimated reward values based on different types of reward integration and timescales. Together, our results demonstrate separable influences of reward on visual processing and choice, and point to the presence of multiple brain circuits for integration of reward outcomes.

neuroscience↗

Multiple timescales of neural dynamics and integration of task-relevant signals across cortex

Recent studies have proposed the orderly progression in the time constants of neural dynamics as an organizational principle of cortical computations. However, relationships between these timescales and their dependence on response properties of individual neurons are unknown. We developed a comprehensive method to simultaneously estimate multiple timescales in neuronal dynamics and integration of task-relevant signals along with selectivity to those signals. We found that most neurons exhibited multiple timescales in their response, which consistently increased from parietal to prefrontal to cingulate cortex. While predicting rates of behavioral adjustments, these timescales were not correlated across individual neurons in any cortical area, resulting in independent parallel hierarchies of timescales. Additionally, none of these timescales depended on selectivity to task-relevant signals. Our results not only suggest multiple canonical mechanisms for increasing timescales of neural dynamics across cortex but also point to additional mechanisms that allow decorrelation of these timescales to enable more flexibility.

neuroscience↗

Dissociable roles for Anterior Cingulate Cortex and Basolateral Amygdala in Decision Confidence and Learning under Uncertainty

It has been suggested the subjective sense of certainty, or confidence, in ambiguous sensory cues can alter the interpretation of reward feedback and facilitate learning. We trained rats to report the orientation of ambiguous visual stimuli according to a spatial stimulus-response rule. Following choice, rats could wait a self-timed delay for reward or initiate a new trial. Waiting times increased with discrimination accuracy, demonstrating that this measure could be used as a proxy for confidence. Chemogenetic silencing of BLA shortened waiting times overall whereas ACC inhibition rendered waiting times insensitive to confidence-modulating attributes of visual stimuli, suggesting contribution of ACC but not BLA to confidence computations. Subsequent reversal learning was enhanced by confidence. Both ACC and BLA inhibition blocked this enhancement but via differential modulation of learning strategies and consistency in using learned rules. Altogether, we demonstrate dissociable roles for ACC and BLA in transmitting confidence and learning under uncertainty.

neuroscience↗

Combinations of low-level and high-level neural processes can account for distinct patterns of context-dependent choice

Context effects have been explained by either high-level cognitive processes or low-level neural adjustments but not their combination. It is currently unclear how these processes interact to shape individuals responses to context. Here, we used a large cohort of human subjects in experiments involving choice between two or three gambles in order to study the dependence of context effects on neural adaptation and individuals risk attitudes. We found no evidence that neural adaptation on long timescales (~100 trials) contributes to context effects. However, we identified two groups of subjects with distinct patterns of responses to decoys, both of which depended on individuals risk aversion. Subjects in the first group exhibited strong, consistent decoy effects and became more risk averse due to decoy presentation. In contrast, subjects in the second group did not show consistent decoy effects and became more risk seeking. The degree of change in risk aversion due to decoy presentation was positively correlated with the initial degrees of risk aversion. To explain these results and reveal underlying neural mechanisms, we developed a new model that incorporates both low- and high-level processes to fit individuals choice behavior. We found that observed decoy effects can be explained by a combination of adjustments in neural representations and competitive weighting of reward attributes, both of which depend on risk aversion but in opposite directions. Altogether, our results demonstrate how a combination of low- and high-level processes shapes multi-attribute choice, modulates overall risk preference, and explains distinct behavioral phenotypes. Significance statementA large body of experimental work has illustrated that the introduction of a new, and often irrelevant, option can influence preference among the existing options, a phenomenon referred to as context or decoy effects. Although context effects have been explained by high-level cognitive processes--such as comparisons and competitions between attributes--or low-level adjustments of neural representations, it is unclear how these processes interact to shape individuals responses to context. Here, we show that both high-level cognitive processes and low-level neural adjustments shift risk preference during choice between multiple options but in opposite directions. Moreover, we demonstrate that a combination of these processes can account for distinct patterns of context effects in human subjects.

neuroscience↗