bioRxiv Science⌕ Search

Biology subjects

Shenhav, A.

Publications and source records attributed to Shenhav, A..

4 recordsLinked to original sources
bioRxiv2018

The Eighty Five Percent Rule for Optimal Learning

Researchers and educators have long wrestled with the question of how best to teach their clients be they human, animal or machine. Here we focus on the role of a single variable, the difficulty of training, and examine its effect on the rate of learning. In many situations we find that there is a sweet spot in which training is neither too easy nor too hard, and where learning progresses most quickly. We derive conditions for this sweet spot for a broad class of learning algorithms in the context of binary classification tasks, in which ambiguous stimuli must be sorted into one of two classes. For all of these gradient-descent based learning algorithms we find that the optimal error rate for training is around 15.87% or, conversely, that the optimal training accuracy is about 85%. We demonstrate the efficacy of this Eighty Five Percent Rule for artificial neural networks used in AI and biologically plausible neural networks thought to describe human and animal learning.

animal behavior and cognition↗
bioRxiv2017

A control theoretic model of adaptive behavior in dynamic environments

To behave adaptively in environments that are noisy and non-stationary, humans and other animals must monitor feedback from their environment and adjust their predictions and actions accordingly. An under-studied approach for modeling these adaptive processes comes from the engineering field of control theory, which provides general principles for regulating dynamical systems, often without requiring a generative model. The proportional-integral-derivative (PID) controller is one of the most popular models of industrial process control. The proportional term is analogous to the \"delta rule\" in psychology, adjusting estimates in proportion to each successive error in prediction. The integral and derivative terms augment this update to simultaneously improve accuracy and stability. Here, we tested whether the PID algorithm can describe how people sequentially adjust their predictions in response to new information. Across three experiments, we found that the PID controller was an effective model of participants decisions in noisy, changing environments. In Experiment 1, we re-analyzed a change-point detection experiment, and showed that participants behavior incorporated elements of PID updating. In Experiments 2-3 we developed a task with gradual transitions that we optimized to detect PID-like adjustments. In both experiments, the PID model offered better descriptions of behavioral adjustments than both the classical delta-rule model and its more sophisticated variant, the Kalman filter. We further examined how participants weighted different PID terms in response to salient environmental events, finding that these control terms were modulated by reward, surprise, and outcome entropy. These experiments provide preliminary evidence that adaptive behavior in dynamic environments resembles PID control.

neuroscience↗
bioRxiv2017

Dissociable components of the reward circuit are involved in appraisal versus choice

People can evaluate a set of options as a whole, or they can approach those same options with the purpose of making a choice between them. A common network has been implicated across these two types of evaluations, including regions of ventromedial prefrontal cortex and the posterior midline. We test the hypothesis that sub-components of this reward circuit are differentially involved in triggering more automatic appraisal of ones options (Dorsal Value Network) versus explicitly comparing between those options (Ventral Value Network). Participants undergoing fMRI were instructed to appraise how much they liked a set of products (Like) or to choose the product they most preferred (Choose). Activity in the Dorsal Value Network consistently tracked set liking, across both task-relevant (Like) and task-irrelevant (Choose) trials. In contrast, the Ventral Value Network was sensitive to evaluation condition (more active during Choose than Like trials). Within vmPFC, anatomically distinct regions were dissociated in their sensitivity to choice (ventrally, in medial OFC) versus appraisal (dorsally, in pregenual ACC). Dorsal regions additionally tracked decision certainty across both types of evaluation. These findings suggest that separable mechanisms drive decisions about how good ones options are versus decisions about which option is best.

neuroscience↗
bioRxiv2017

Dissociable neural mechanisms track evidence accumulation for selection of attention versus action

Decision-making is typically studied as a sequential process from the selection of what to attend (e.g., between possible tasks, stimuli, or stimulus attributes) to the selection of which actions to take based on the attended information. However, people often gather information across these levels in parallel. For instance, even as they choose their actions, they may continue to evaluate how much to attend other tasks or dimensions of information within a task. We scanned participants while they made such parallel evaluations, simultaneously weighing how much to attend two dynamic stimulus attributes and which response to give based on the attended information. Regions of prefrontal cortex tracked information about the stimulus attributes in dissociable ways, related to either the predicted reward (ventromedial prefrontal cortex) or the degree to which that attribute was being attended (dorsal anterior cingulate, dACC). Within dACC, adjacent regions tracked uncertainty at different levels of the decision, regarding what to attend versus how to respond. These findings bridge research on perceptual and value-based decision-making, demonstrating that people dynamically integrate information in parallel across different levels of decision making.\n\nNaturalistic decisions allow an individual to weigh their options within a particular task (e.g., how best to word the introduction to a paper) while also weighing how much to attend other tasks (e.g., responding to e-mails). These different types of decision-making have a hierarchical but reciprocal relationship: Decisions at higher levels inform the focus of attention at lower levels (e.g., whether to select between citations or email addresses) while, at the same time, information at lower levels (e.g., the salience of an incoming email) informs decisions regarding which task to attend. Critically, recent studies suggest that decisions across these levels may occur in parallel, continuously informed by information that is integrated from the environment and from ones internal milieu1,2.\n\nResearch on cognitive control and perceptual decision-making has examined how responses are selected when attentional targets are clearly defined (e.g., based on instruction to attend a stimulus dimension), including cases in which responding requires accumulating information regarding a noisy percept (e.g., evidence favoring a left or right response)3-7. Separate research on value-based decision-making has examined how individuals select which stimulus dimension(s) to attend in order to maximize their expected rewards8-11. However, it remains unclear how the accumulation of evidence to select high-level goals and/or attentional targets interacts with the simultaneous accumulation of evidence to select responses according to those goals (e.g., based on the perceptual properties of the stimuli). Recent work has highlighted the importance of such interactions to understanding task selection12-15, multi-attribute decision-making16-18, foraging behavior19-21, cognitive effort22,23, and self-control24-27.\n\nWhile these interactions remain poorly understood, previous research has identified candidate neural mechanisms associated with multi-attribute value-based decision-making11,28,29 and with selecting a response based on noisy information from an instructed attentional target3-5. These research areas have implicated the ventromedial prefrontal cortex (vmPFC) in tracking the value of potential targets of attention (e.g., stimulus attributes)8,11 and the dorsal anterior cingulate cortex (dACC) in tracking an individuals uncertainty regarding which response to select30-32. It has been further proposed that dACC may differentiate between uncertainty at each of these parallel levels of decision-making (e.g., at the level of task goals or strategies vs. specific motor actions), and that these may be separately encoded at different locations along the dACCs rostrocaudal axis32,33. However, neural activity within and across these prefrontal regions has not yet been examined in a setting in which information is weighed at both levels within and across trials.\n\nHere we use a value-based perceptual decision-making task to examine how people integrate different dynamic sources of information to decide (a) which perceptual attribute to attend and (b) how to respond based on the evidence for that attribute. Participants performed a task in which they regularly faced a conflict between attending the stimulus attribute that offered the greater reward or the attribute that was more perceptually salient (akin to persevering in writing ones paper when an enticing email awaits). We demonstrate that dACC and vmPFC track evidence for the two attributes in dissociable ways. Across these regions, vmPFC weighs attribute evidence by the reward it predicts and dACC weighs it by its attentional priority (i.e., the degree to which that attribute drives choice). Within dACC, adjacent regions differentiated between uncertainty at the two levels of the decision, regarding what to attend (rostral dACC) versus how to respond (caudal dACC).

neuroscience↗