Dunovan, K. | bioRxiv Science

bioRxiv2018

Reward-driven changes in striatal pathway competition shape evidence evaluation in decision-making

Cortico-basal-ganglia-thalamic (CBGT) networks are critical for adaptive decision-making, yet how changes to circuit-level properties impact cognitive algorithms remains unclear. Here we explore how dopaminergic plasticity at corticostriatal synapses alters competition between striatal pathways, impacting the evidence accumulation process during decision-making. Spike-timing dependent plasticity simulations showed that dopaminergic feedback based on rewards modified the ratio of direct and indirect corticostriatal weights within opposing action channels. Using the learned weight ratios in a full spiking CBGT network model, we simulated neural dynamics and decision outcomes in a reward-driven decision task and fit them with a drift diffusion model. Fits revealed that the rate of evidence accumulation varied with inter-channel differences in direct pathway activity while boundary height varied with overall indirect pathway activity. This multi-level modeling approach demonstrates how complementary learning and decision computations can emerge from corticostriatal plasticity. Author summaryCognitive process models such as reinforcement learning (RL) and the drift diffusion model (DDM) have helped to elucidate the basic algorithms underlying error-corrective learning and the evaluation of accumulating decision evidence leading up to a choice. While these relatively abstract models help to guide experimental and theoretical probes into associated phenomena, they remain uninformative about the actual physical mechanics by which learning and decision algorithms are carried out in a neurobiological substrate during adaptive choice behavior. Here we present an "upwards mapping" approach to bridging neural and cognitive models of value-based decision-making, showing how dopaminergic feedback alters the network-level dynamics of cortico-basal-ganglia-thalamic (CBGT) pathways during learning to bias behavioral choice towards more rewarding actions. By mapping "up" the levels of analysis, this approach yields specific predictions about aspects of neuronal activity that map to the quantities appearing in the cognitive decision-making framework.

neuroscience↗

bioRxiv2017

Dual learning mechanisms drive adaptive inhibitory control

Goal-directed behavior requires integrating action selection processes with learning systems that adapt control using environmental feedback. These functions intersect in the basal ganglia (BG), which has at least two targets of plasticity: a dopaminergic modulation of striatal pathways and cortical modulation of the subthalamic nucleus (STN). Dual learning mechanisms suggests that feedback signals have a multifaceted impact on BG-dependent decisions. Using a hybrid of accumulation-to-bound decision models and reinforcement learning, we modeled the performance of humans in a stop-signal task where participants (N=75) learned the prior distribution of the timing of a stop signal through trial-and-error feedback. Changes in the drift-rate of the action execution process were driven by errors in action timing, whereas adaptation in the boundary height served to increase caution following failed stops. These findings highlight two interactive learning mechanisms for adapting the control of goal-directed actions based on dissociable dimensions of feedback error.\n\nAuthor SummaryMany complex behavioral goals rely on ones ability to regulate the timing of action execution while also maintaining enough control to cancel actions in response to \"Stop\" cues in the environment. Here we examined how these two fundamental components of behavior become tuned to the control demands of the environment by combining principles of reinforcement learning with accumulator models of decision making. The synthesis of these two theoretical frameworks is motivated by previous work showing that reinforcement learning and control rely on overlapping circuitry in the basal ganglia. Leveraging knowledge about the interaction of learning and control signals in this network, we formulated a computational model in which performance feedback is used to modulate key mechanisms of the decision process to facilitate goal acquisition. Model-based analysis of behavioral data collected on an adaptive stop-signal task revealed two critical learning mechanisms: one that adjusts the accumulation rate of the \"Go\" signal to errors in action timing and another that exercises caution by raising the height of the execution boundary after a failed Stop trial. We show how these independent learning mechanisms interact over the course of learning, shedding light on the behavioral effects plasticity in different pathways of the basal ganglia.

neuroscience↗