Biology subjects

Shourkeshti, A.

Publications and source records attributed to Shourkeshti, A..

3 recordsLinked to original sources

bioRxiv2024

Humans forage for reward in reinforcement learning tasks

How do we make good decisions in uncertain environments? In psychology and neuroscience, the classic view is that we calculate the value of each option, compare them, and choose the most rewarding modulo exploratory noise. An ethologist, conversely, would argue that we commit to one option until its value drops below a threshold and then explore alternatives. Because the fields use incompatible methods, it remains unclear which view better describes human decision-making. Here, we found that humans use compare-to-threshold computations in classic compare-alternative tasks. Because compare-alternative computations are central to the reinforcement-learning (RL) models typically used in the cognitive and brain sciences, we developed a novel compare-to-threshold model ("foraging"). Compared to previous RL models, the foraging model better fit participant behavior, better predicted the tendency to repeat choices, and predicted held-out participants that were almost impossible under comparealternative models. These results suggest that humans use compare-to-threshold computations in sequential decision-making.

neuroscience↗

bioRxiv2024

Persistent Decision-Making in Mice, Monkeys, and Humans

Humans have the capacity to persist in behavioural policies, even in challenging environments that lack immediate reward. Persistence is the scaffold on which many higher executive functions are built. However, it remains unclear whether humans are uniquely persistent or, instead, if this capacity is widely conserved across species. To address this question, we compared humans with mice and monkeys in harmonised versions of a dynamic decision-making task. The task encouraged all species to strike a balance between persistently exploiting one policy and exploring alternative policies that could become better at any moment. Although all three species had similar strategies, we found that both primate species--humans and monkeys--were able to persist in exploitation for longer than the mice. The similarities in persistence patterns in humans and monkeys, as opposed to mice, may be related to the various ecological, neurobiological, or cognitive factors that differ systematically between these species.

animal behavior and cognition↗

bioRxiv2023

Pupil size predicts the onset of exploration in brain and behavior.

In uncertain environments, intelligent decision-makers exploit actions that have been rewarding in the past, but also explore actions that could be even better. Several neuromodulatory systems are implicated in exploration, based, in part, on work linking exploration to pupil size-a peripheral correlate of neuromodulatory tone and index of arousal. However, pupil size could instead track variables that make exploration more likely, like volatility or reward, without directly predicting either exploration or its neural bases. Here, we simultaneously measured pupil size, exploration, and neural population activity in the prefrontal cortex while two rhesus macaques explored and exploited in a dynamic environment. We found that pupil size under constant luminance specifically predicted the onset of exploration, the first exploratory trial in a sequence, beyond what could be explained by reward history. Pupil size also predicted disorganized patterns of prefrontal neural activity at both the single neuron and population levels, even within periods of exploitation. Ultimately, our results support a model in which pupil-linked mechanisms promote the onset of exploration via driving the prefrontal cortex through a critical tipping point where prefrontal control dynamics become disorganized and exploratory decisions are possible. Significance StatementHumans and other animals learn about the world through exploration: through making decisions that offer the opportunity to learn and discover, even when these decisions are not the best option in the moment. Neuroscience research has historically focused on understanding good choices, delivering many key insights into the neural mechanisms involved in these calculations. However, much less is known about how the brain generates exploratory decisions. This study identifies certain "early warning signs" of exploratory decisions in the brain and body, including certain signals in size of the pupil and the speed of neural activity in the prefrontal cortex. These early warning signs suggest that exploration may be the result of a critical tipping point in prefrontal brain states.

neuroscience↗