bioRxiv Science⌕ Search

Biology subjects

Perez, O. D.

Publications and source records attributed to Perez, O. D..

10 recordsLinked to original sources

What group averages conceal: functional heterogeneity in human eyeblink habituation

In animal learning research, learning is often represented by plotting a behavioral measure as a function of training trials. A particularly clear case is habituation, a basic form of learning in which repeated presentation of a stimulus produces a decrement in responding. Although retention tests provide the strongest basis for evaluating durable habituation once short-lived performance effects have dissipated, the pattern of response change across stimulus repetitions, or habituation curve, remains theoretically and empirically relevant because it is used to characterize determinants of habituation, individual and clinical profiles, and functional forms, including linear, curvilinear, asymptotic, and mixed incremental-decremental patterns of responding. However, group averaged curves may conceal substantial individual heterogeneity. Here, we analyzed archived human eyeblink habituation data from 157 participants to ask whether the curve shape selected for the group average reflects the curve shapes observed at the individual level. Five candidate functions were fitted separately to each participant and to the corresponding group average. No single function characterized most individuals. More importantly, the model selected for the group average differed from the most frequent individual model in all four groups. When data were pooled across groups, the average favored a dual-process form, a shape that matched the individual plurality in none of them. Simulation analyses showed that averaging heterogeneous individual trajectories can itself produce a group curve that favors a more complex model. Our findings show that group averaged habituation curves should not be treated as direct descriptions of the typical individual trajectory.

animal behavior and cognition↗

Intentional control of instrumental punishment

Punishment raises a basic question for human cognition: do agents suppress actions because they represent the negative consequences those actions produce? Everyday deterrence, from parental reprimands to legal sanctions, assumes that people regulate behavior by learning what their actions cause and by evaluating those consequences negatively. However, in spite of this widely-held assumption, experimental evidence that punished instrumental behavior is controlled by the current value of the punisher is currently lacking. To test this idea, we applied an outcome revaluation procedure to instrumental punishment in rats. Rats first learned to press two levers for food. One lever was then additionally punished by response-contingent footshock, whereas the other remained unpunished. The aversiveness of the shock was subsequently reduced by pairing it with food in the absence of the levers. To test if the punishers value was part of the animals representation of the consequences of their actions, responding was tested in extinction, with no shock delivered. We found that revaluation of the shock selectively abolished the suppression of the punished response. Thus, punishment was not a fixed behavior controlled by the prior occurrence of shock, but depended on the shocks current value. Our results show that instrumental punishment can be controlled by a value-sensitive representation of the aversive consequence produced by the action and is, in this sense, structured in the same manner as canonical goal-directed reward seeking.

animal behavior and cognition↗

Distributed activity in the human posterior putamen distinguishes goal-directed from habitual control in humans

How do some individuals rapidly form habits while others maintain flexible, goal-directed control? Using multivariate fMRI decoding in 199 participants, we show that distributed neural activity patterns in the left posterior putamen during initial learning predict individual behavioral strategy on a subsequent outcome devaluation test. This prediction generalized across two independent cohorts of healthy adults and psychiatric patients with heterogeneous diagnoses and was anatomically specific to the posterior putamen. Critically, predictive neural signatures were present during training, before strategy expression after devaluation, enabling prospective classification of habitual versus goal-directed behavior. These findings demonstrate that stable individual differences in behavioral control are reflected in circumscribed brain activity during learning, highlighting the posterior putamen as a candidate neural marker of habit propensity with potential clinical relevance.

neuroscience↗

The status of human goal-directed actions after manipulations of the action-outcome contingency

Goal-directed actions are those performed with the expectation of producing a specific outcome and are therefore sensitive to changes in the outcomes current value without re-experiencing the action-outcome contingency. Disrupting the action-outcome association by a number of interventions can reduce responding, but whether such interventions also affect the goal-directedness of human responses remains unknown. We trained participants to perform different actions for two distinct outcomes in two groups. One group experienced extinction (i.e., complete suspension of outcome delivery for a specific previously rewarded action) whereas the other was given non-contingent training (i.e., outcomes were presented independently of responding). Goal-directed control was tested by outcome devaluation, making one of the two outcomes less desirable, and responding was then tested in the absence of outcome presentations. Despite having been significantly reduced in both groups by extinction and non-contingent training, responding remained sensitive to outcome value, suggesting that human goal-directed behavior is resistant to action-outcome contingency interventions. We discuss these results in light of recent theories of goal-directed behavior.

animal behavior and cognition↗

Dual-system avoidance: extension of a theory

Our theory of positively reinforced free-operant behavior (Perez & Dickinson, 2020) assumes that responding is controlled by 2 systems. One system is sensitive to the correlation between response and reinforcement rates and controls goal-directed behavior, whereas a habitual system learns by reward prediction error. We present an extension of this theory to the aversive domain that explains why free-operant avoidance responding increases with both the experienced rate of negative reinforcement and the difference between this rate and that programmed by the avoidance schedule. The theory also assumes that the habitual component is reinforced by the acquisition of aversive inhibitory properties by the feedback stimuli generated by responding, which then act as safety signals that reinforce habit performance. Our analysis suggests that the distinction between habitual and goal-directed control of rewarded behavior can also be applied to the aversive domain.

animal behavior and cognition↗

Confirmation bias optimizes reward learning

Confirmation bias--the tendency to overweight information that matches prior beliefs or choices--has been shown to manifest even in simple reinforcement learning. In line with recent work, we find that participants learned significantly more from choice-confirming outcomes in a reward-learning task. What is less clear is whether asymmetric learning rates somehow benefit the learner. Here, we combine data from human participants and artificial agents to examine how confirmation-biased learning might improve performance by counteracting decisional and environmental noise. We evaluate one potential mechanism for such noise reduction: visual attention--a demonstrated driver of both value-based choice and predictive learning. Surprisingly, visual attention showed the opposite pattern to confirmation bias, as participants were most likely to fixate on "missed opportunities", slightly dampening the effects of the confirmation bias we observed. Several million simulated experiments with artificial agents showed this bias to be a reward-maximizing strategy compared to several alternatives, but only if disconfirming feedback is not completely ignored--a condition that visual attention may help to enforce.

animal behavior and cognition↗

A theory of actions and habits in free-operant behavior: The interaction of rate correlation and contiguity systems.

Contemporary theories of instrumental performance assume that responding can be controlled by two behavioral systems, one goal-directed that encodes the outcome of an action, and one habitual that reinforces the response strength of the same action. Here we present a model of free-operant behavior in which goal-directed control is determined by the correlation between the rates of the action and the outcome whereas the total prediction error generated by contiguous reinforcement by the outcome controls habitual response strength. The outputs of these two systems summate to generate a total response strength. This cooperative model addresses the difference in the behavioral impact of ratio and interval schedules, the transition from goal-directed to habitual control with extended training, the persistence of goal-directed control under choice procedures and following extinction, among other phenomena. In these respects, this dual-system model is unique in its account of free-operant behavior.

animal behavior and cognition↗

Visual search mimics configural processing in human causal learning

Theories of learning distinguish between elemental and configural stimulus processing depending on whether stimuli are processed independently or as whole configurations. Evidence for elemental processing comes from findings of summation in animals where a compound of two dissimilar stimuli is deemed to be more predictive than each stimulus alone, whereas configural processing is supported by experiments employing similar stimuli in which summation is not found. However, in humans the summation effect is robust and impervious to similarity manipulations. In three experiments in human predictive learning, we show that summation can be obliterated when partially reinforced cues are added to the summands in training and test. This lack of summation only holds when the partially reinforced cues are similar to the reinforced cues (Experiment 1) and seems to depend on participants sampling only the most salient cue in each trial (Experiments 2a and 2b) in a sequential visual search process. Instead of attributing our and others instances of lack of summation to the customary idea of configural processing, we offer a formal sub-sampling rule that might be applied to situations in which the stimuli are hard to parse from each other.

animal behavior and cognition↗

A re-examination of responding on ratio and regulated-probability interval schedules

The higher response rates observed on ratio than on matched interval reward schedules has been attributed to the differential reinforcement of longer inter-response times (IRTs) on the interval contingency. Some data, however, seem to contradict this hypothesis, showing that the difference is still observed when the role of IRT reinforcement is neutralized by using a regulated-probability interval schedule (RPI). Given the mixed evidence for these predictions, we re-examined this hypothesis by training three groups of rats to lever press under ratio, interval and RPI schedules across two phases while matching reward rates within triads. At the end of the first phase, the master ratio and RPI groups responded at similar rates. In the second phase, an interval group yoked to the same master ratio group of the first phase responded at a lower rate than the RPI group. Post-hoc analysis showed comparable reward rates for master and yoked schedules. The experienced response-outcome rate correlations were likewise similar, and approached zero as training progressed. We discuss these results in terms of dual-system theories of instrumental conditioning.

animal behavior and cognition↗

The role of stimulus similarity on the summation effect in human causal learning

Several contemporary models of associative learning anticipate that the higher responding to a compound of two cues separately trained with a common outcome than to each of the cues alone -a summation effect-is modulated by the similarity between the cues forming the compound. Here, we explored this hypothesis in a series of causal learning experiments with humans. Participants were presented with two visual cues that separately predicted a common outcome and later asked for the outcome predicted by the compound of the two cues. Importantly, the cues similarity was varied between groups through changes in shape, spatial position, color, configuration and rotation. In variance with the predictions of these models, we observed similar and strong levels of summation in both groups across all manipulations of similarity (Experiments 1-5). The summation effect was significantly reduced by manipulations intended to impact assumptions about the causal independence of the cues forming the compound, but this reduction was independent of stimulus similarity (Experiment 6). These results are problematic for similarity-based models and can be more readily explained by rational approaches to causal learning.

animal behavior and cognition↗