Biology subjects

Bogacz, R.

Publications and source records attributed to Bogacz, R..

3 recordsLinked to original sources

bioRxiv2018

Predicting the effects of deep brain stimulation using a reduced coupled oscillator model

Deep brain stimulation (DBS) is known to be an effective treatment for a variety of neurological disorders, including Parkinsons disease and essential tremor (ET). At present, it involves administering a train of pulses with constant frequency via electrodes implanted into the brain. New closed-loop approaches involve delivering stimulation according to the ongoing symptoms or brain activity and have the potential to provide improvements in terms of efficiency, efficacy and reduction of side effects. The success of closed-loop DBS depends on being able to devise a stimulation strategy that minimizes oscillations in neural activity associated with symptoms of motor disorders. A useful stepping stone towards this is to construct a mathematical model, which can describe how the brain oscillations should change when stimulation is applied at a particular state of the system. Our work focuses on the use of coupled oscillators to represent neurons in areas generating pathological oscillations. Using a reduced form of the Kuramoto model, we analyse how a patient should respond to stimulation when neural oscillations have a given phase and amplitude. We predict that, provided certain conditions are satisfied, the best stimulation strategy should be phase specific but also that stimulation should have a greater effect if applied when the amplitude of brain oscillations is lower. We compare this surprising prediction with data obtained from ET patients. In light of our predictions, we also propose a new hybrid strategy which effectively combines two of the strategies found in the literature, namely phase-locked and adaptive DBS.\n\nAuthor summaryDeep brain stimulation (DBS) involves delivering electrical impulses to target sites within the brain and is a proven therapy for a variety of neurological disorders. Closed loop DBS is a promising new approach where stimulation is applied according to the state of a patient. Crucial to the success of this approach is being able to predict how a patient should respond to stimulation. Our work focusses on DBS as applied to patients with essential tremor (ET). On the basis of a theoretical model, which describes neurons as oscillators that respond to stimulation and have a certain tendency to synchronize, we provide predictions for how a patient should respond when stimulation is applied at a particular phase and amplitude of the ongoing tremor oscillations. Previous experimental studies of closed loop DBS provided stimulation either on the basis of ongoing phase or amplitude of pathological oscillations. Our study suggests how both of these measurements can be used to control stimulation. As part of this work, we also look for evidence for our theories in experimental data and find our predictions to be satisfied in one patient. The insights obtained from this work should lead to a better understanding of how to optimise closed loop DBS strategies.

neuroscience↗

bioRxiv2018

Learning the payoffs and costs of actions

A set of sub-cortical nuclei called basal ganglia is critical for learning the values of actions. The basal ganglia include two pathways, which have been associated with approach and avoid behavior respectively, and are differentially modulated by dopamine projections from the midbrain. According to the influential opponent actor learning model, these pathways represent learned estimates of the positive and negative consequences (payoffs and costs) of actions. The level of dopamine release controls to what extent payoffs and costs enter the overall evaluation of actions. How the knowledge about payoff and cost is acquired is still an open question, even though many theories describe learning from feedback in the basal ganglia. We examine whether a set of plasticity rules proposed to model reinforcement learning in the pathways of the basal ganglia is suitable to extract payoffs and costs from a reward prediction error signal. First, we determine the result of such learning, both analytically and via simulations, for different reward schedules that feature payoffs and costs. Then, we combine the plasticity rules with a decision rule to examine the emerging effect of dopaminergic modulation on the willingness to work for reward. We find that the plasticity rules are suitable to infer the mean payoffs and costs of actions, if those occur at different moments in time. Successful learning requires differential effects of positive and negative reward prediction errors on the two pathways, and a weak decay of synaptic weights over trials. We also confirm that dopaminergic modulation produces effects on the willingness to work for reward similar to those observed in classical experiments.\n\nAuthor summaryThe basal ganglia are structures underneath the surface of the vertebrate brain, associated with error driven learning. Much is known about the anatomical and biological features of the basal ganglia; scientists now try to understand the algorithms implemented by these structures. Numerous models aspire to capture the learning functionality, but many of them only cover some specific aspect of the algorithm. Instead of further adding to that pool of partial models, we unify two existing ones - one which captures what the basal ganglia learns, and one that describes the learning mechanism itself. The first model suggests that the basal ganglia keeps track of both positive and negative consequences of frequent opportunities, and weighs these by the motivational state in decisions. It explains how payoff and cost are represented, but not how those representations arise. The other model consists of biologically plausible plasticity rules, which describe how learning takes place, but not how the brain makes use of what is learned. We show that the two theories are compatible. Together, they form a model of learning and decision making that integrates the motivational state as well as the learned payoffs and costs of opportunities.

neuroscience↗

bioRxiv2017

Theory of reinforcement learning and motivation in the basal ganglia

This paper proposes how the neural circuits in vertebrates select actions on the basis of past experience and the current motivational state. According to the presented theory, the basal ganglia evaluate the utility of considered actions by combining the positive consequences (e.g. nutrition) scaled by the motivational state (e.g. hunger) with the negative consequences (e.g. effort). The theory suggests how the basal ganglia compute utility by combining the positive and negative consequences encoded in the synaptic weights of striatal Go and No-Go neurons, and the motivational state carried by neuromodulators including dopamine. Furthermore, the theory suggests how the striatal neurons to learn separately about consequences of actions, and how the dopaminergic neurons themselves learn what level of activity they need to produce to optimize behaviour. The theory accounts for the effects of dopaminergic modulation on behaviour, patterns of synaptic plasticity in striatum, and responses of dopaminergic neurons in diverse situations.

bioinformatics↗