Biology subjects

Ballard, I. C.

Publications and source records attributed to Ballard, I. C..

3 recordsLinked to original sources

bioRxiv2018

Joint Modeling of Reaction Times and Choice Improves Parameter Identifiability in Reinforcement Learning Models

BackgroundReinforcement learning models provide excellent descriptions of learning in multiple species across a variety of tasks. Many researchers are interested in relating parameters of reinforcement learning models to neural measures, psychological variables or experimental manipulations. We demonstrate that parameter identification is difficult because a range of parameter values provide approximately equal quality fits to data. This identification problem has a large impact on power: we show that a researcher who wants to detect a medium sized correlation (r = .3) with 80% power between a variable and learning rate must collect 60% more subjects than specified by a typical power analysis in order to account for the noise introduced by model fitting.\n\nNew MethodWe derive a Bayesian optimal model fitting technique that takes advantage of information contained in choices and reaction times to constrain parameter estimates.\n\nResultsWe show using simulation and empirical data that this method substantially improves the ability to recover learning rates.\n\nComparison with Existing MethodsWe compare this method against the use of Bayesian priors. We show in simulations that the combined use of Bayesian priors and reaction times confers the highest parameter identifiability. However, in real data where the priors may have been misspecified, the use of Bayesian priors interferes with the ability of reaction time data to improve parameter identifiability.\n\nConclusionsWe present a simple technique that takes advantage of readily available data to substantially improve the quality of inferences that can be drawn from parameters of reinforcement learning models.\n\nHighlights- Parameters of reinforcement learning models are particularly difficult to estimate\n- Incorporating reaction times into model fitting improves parameter identifiability\n- Bayesian weighting of choice and reaction times improves the power of analyses assessing learning rate

neuroscience↗

bioRxiv2018

Causal Evidence for the Dependence of the Magnitude Effect on Dorsolateral Prefrontal Cortex

Impulsivity refers to the tendency to insufficiently consider alternatives or to overvalue rewards that are available immediately. Impulsivity is a hallmark of human decision making with well documented health and financial ramifications. Numerous contextual changes and framing manipulations powerfully influence impulsivity. One of the most robust such phenomenon is the finding that people are more patient as the values of choice options are increased. This magnitude effect has been related to cognitive control mechanisms in the dorsal lateral prefrontal cortex (dlPFC). We used repetitive transcranial magnetic stimulation (rTMS) to transiently disrupt dlPFC neural activity. This manipulation dramatically reduced the magnitude effect, establishing causal evidence that the magnitude effect depends on dlPFC.

neuroscience↗

bioRxiv2018

Hippocampal Pattern Separation Supports Reinforcement Learning

1Animals rely on learned associations to make decisions. Associations can be based on relationships between object features (e.g., the three-leaflets of poison ivy leaves) and outcomes (e.g., rash). More often, outcomes are linked to multidimensional states (e.g., poison ivy is green in summer but red in spring). Feature-based reinforcement learning fails when the values of individual features depend on the other features present. One solution is to assign value to multifeatural conjunctive representations. We tested if the hippocampus formed separable conjunctive representations that enabled learning of response contingencies for stimuli of the form: AB+, B-, AC-, C+. Pattern analyses on functional MRI data showed the hippocampus formed conjunctive representations that were dissociable from feature components and that these representations influenced striatal PEs. Our results establish a novel role for hippocampal pattern separation and conjunctive representation in reinforcement learning.

neuroscience↗