bioRxiv Science⌕ Search

Biology subjects

Dimigen, O.

Publications and source records attributed to Dimigen, O..

17 recordsLinked to original sources

Hierarchical Predictive Processing during Natural Reading

Key neural mechanisms underlying reading, especially in the context of sentences and texts, remain elusive. An influential theory states that the brain rapidly generates a hierarchy of probabilistic predictions about upcoming words at multiple levels of linguistic representation. When encountering a word, a reader integrates sensory input with the linguistic predictions, and rely on the resulting prediction error signals to achieve efficient comprehension. Here, we tested this hierarchical predictive account with a natural reading task while EEG and eye movements were simultaneously recorded. We find distinct prediction error signals attributable to word contextual, lexical, and orthographic levels of predictions. These prediction error signals exhibit a temporal order. Our results also indicate that context-based predictions rapidly constrain orthographic-level predictions. We thus provide evidence for a hierarchical predictive process central to efficient natural reading.

neuroscience↗

Visual consequences of saccades explain early cortical response dynamics during natural vision

During natural vision, each saccadic eye movement evokes a large cortical potential known as the lambda response. Traditionally, lambda responses have been considered feedforward responses to new visual input arriving at fixation. In contrast, here we show that they are best explained by the retinal shifts induced by the saccade itself. We first recorded EEG data for thousands of saccades during free viewing of natural scenes and then replayed the same retinal image shifts on a high-speed display during fixation. Temporal response dynamics were virtually identical for real saccades and replayed motion, and most strongly aligned to saccadic peak velocity. A computational model based only on saccade trajectories, 1/f natural-scene statistics, and known human spatiotemporal sensitivity parsimoniously reproduced these dynamics. Our results show that saccade intervals should not be viewed as a gap in visual processing, but as a structured sensory event that shapes human cortical activity during natural viewing.

neuroscience↗

Pupil Constriction Causes Activity in the Human Retina and Visual System

Pupil responses shape the earliest stages of visual perception by regulating the amount of light that enters the eye. How this affects visual processing is still poorly understood. Here we present evidence that pupil constriction causes activity in the human retina and visual system, independently of visual stimulation. Healthy human participants (N=119) viewed brief visual stimuli (light increments or decrements) while pupil size, retinal activity (electroretinogram), and brain activity (electroencephalogram) were recorded. As expected, visual stimulation triggered an initial burst of retinal activity followed by pupil constriction (the pupil light response). Importantly, we used trial-to-trial variability in constriction latency to reveal a previously unknown retinal response that is locked to pupil constriction, rather than to visual stimulation. Presumably, and in line with similar findings in mice, this constriction-locked retinal activity is a response to the sudden decrease in retinal light exposure that accompanies pupil constriction (although contribution of iris muscle activity is not conclusively ruled out). A similar constriction-locked response emerged later over visual cortex. Given these findings, an important open question is how the visual system maintains brightness constancy despite pupil-induced retinal and cortical activity.

neuroscience↗

Rapid Invisible Frequency Tagging (RIFT) with a consumer monitor: A proof-of-concept

Rapid Invisible Frequency Tagging (RIFT) enables neural frequency tagging at rates above the flicker fusion threshold, eliciting steady-state responses to flicker that is almost imperceptible. While RIFT has proven valuable for studying visuospatial attention, it has so far relied on costly projector systems, typically in combination with magnetoencephalography (MEG). The recent emergence of high-speed organic light-emitting diode (OLED) monitors for consumers suggests that RIFT may also be feasible with much more accessible hardware. Here, we provide a proof-of-concept demonstrating successful RIFT using a consumer-grade 480 Hz OLED monitor in combination with electroencephalography (EEG). We also share practical recommendations for achieving precise stimulus timing at 480 Hz with minimal frame drops. In a central fixation task, participants viewed a tapered disc stimulus flickering either centrally or peripherally. Luminance was modulated sinusoidally at 60 Hz or 64 Hz, frequencies at which the flicker was barely visible. Photodiode recordings confirmed that the monitor delivered accurate frame timing with few dropped frames. Cross-coherence analysis between occipital EEG channels and a photodiode revealed robust, frequency-specific neural tagging responses for central stimuli at both frequencies. In comparison, weaker coherence was observed for 60 Hz peripheral flicker. Our findings demonstrate that RIFT can be reliably implemented using affordable stimulation hardware, a low-density EEG montage, and a minimal processing pipeline. We hope that this lowers barriers to entry, facilitating broader use of RIFT in basic research and in applied settings where cost and portability matter.

neuroscience↗

A high-speed OLED monitor for precise stimulation in vision, eye-tracking, and EEG research

The recent introduction of organic light-emitting diode (OLED) monitors with refresh rates of 240 Hz or more opens new possibilities for their use as precise stimulation devices in vision research, experimental psychology, and electrophysiology. These affordable high-speed monitors, targeted at video gamers, promise several advantages over the cathode ray tube (CRT) and liquid crystal display (LCD) monitors commonly used in these fields. Unlike LCDs, OLED displays have self-emitting pixels that can show true black, resulting in superior contrast ratios, a broad color gamut, and good viewing angles. More importantly, the latest gaming OLEDs promise excellent timing properties with minimal input lags and rapid transition times. However, OLED technology also has potential drawbacks, notably Auto-Brightness Limiting (ABL) behavior, where the local luminance of a stimulus can change with the number of currently illuminated pixels. This study characterized a 240 Hz OLED monitor, the ASUS PG27AQDM, in terms of its timing properties, spatial uniformity, viewing angles, warm-up times, and ABL behavior. We also compared its responses to those of CRTs and LCDs. Results confirm the monitors excellent temporal properties with CRT-like transition times (around 0.3 ms), wide viewing angles, and decent spatial uniformity. Additionally, we found that ABL could be prevented with appropriate settings. We illustrate the monitors benefits in two time-critical paradigms: Rapid "invisible" flicker stimulation and the gaze-contingent presentation of stimuli during eye movements. Ourfindings suggest that the newest gaming OLEDs are precise and cost-effective stimulation devices for visual experiments that have several key advantages over CRTs and LCDs.

neuroscience↗

Spontaneous Fluctuations in Pupil Size Shape Retinal Responses to Visual Stimuli

Visual perception is shaped at the earliest stage by the size of the eyes pupil, which determines how much light enters the eye and how well this light is focused. But the exact role of pupil size in visual perception is still poorly understood. We recorded pupil size and electrical activity from the retina and brain while healthy human participants viewed full-screen flashes. We found that early retinal responses, which peaked {+/-}25 ms after stimulus onset and predicted subsequent activity over visual cortex, were strongly affected by stimulus intensity. Importantly, pupil size, at least within the range of naturally occurring fluctuations, did not affect the amplitude of these early retinal responses, despite resulting in substantial changes in retinal light exposure. However, the direction of pupil-size change at the moment of stimulus presentation did modulate the amplitude of early retinal responses, which were enhanced during phases of dilation as compared to constriction. Based on these results, we suggest that fast-acting adaptation processes may normalize early retinal responses with respect to changes in retinal light exposure that result from spontaneous changes in pupil size: an initial form of brightness constancy. These results shed new light on, and raise important and previously unasked questions about, the role of pupil size in visual perception.

neuroscience↗

Neural mechanism underlying preview effects and masked priming effects in visual word processing

Two classic experimental paradigms - masked repetition priming and the boundary paradigm - have played a pivotal role in understanding the process of visual word recognition. Traditionally, these paradigms have often been employed by different communities of researchers, with their own long-standing research traditions. Nevertheless, a review of the literature suggests that the brain-electric correlates of word processing established with both paradigms may show interesting similarities, in particular with regard to the location, timing, and direction of N1 and N250 effects. However, as of yet, no direct comparison has been undertaken between both paradigms. In the current study, we used combined eye-tracking/EEG to perform such a within-subject comparison using the same materials (single Chinese characters) as stimuli. Our results show the typical early repetition effects of N1 and N250 for both paradigms. However, repetition effects in N250 (i.e., a reduced negativity following identical-word primes/previews as compared to different-word primes/previews) were larger in the boundary paradigm than with masked priming. For N1 effects, repetition effects were similar across the two paradigms showing a larger N1 after repetitions as compared to alternations. Therefore, the results indicate that at the neural level, a briefly presented and masked foveal prime produces qualitatively similar facilitatory effects on visual word recognition as a parafoveal preview before a saccade, although such effects appear to be stronger in the latter case.

neuroscience↗

(Micro)saccade-related potentials during face recognition: A study combining EEG, eye-tracking, and deconvolution modeling

Under natural viewing conditions, complex stimuli such as human faces are typically looked at several times in succession, implying that their recognition may unfold across multiple eye fixations. Although electrophysiological (EEG) experiments on face recognition typically prohibit eye movements, participants still execute frequent (micro)saccades on the face, each of which generates its own visuocortical response. This finding raises the question of whether the fixation-related potentials (FRPs) evoked by these tiny gaze shifts also contain psychologically valuable information about face processing. Here we investigated this question by co-recording EEG and eye movements in an experiment with emotional faces (happy, angry, neutral). Deconvolution modeling was used to separate the stimulus-ERPs to face onset from the FRPs generated by subsequent microsaccades-induced refixations on the face. As expected, stimulus-ERPs exhibited typical emotion effects, with a larger early posterior negativity (EPN) for happy/angry compared to neutral faces. Eye-tracking confirmed that participants made small saccades within the face in 98% of the trials. However, while each saccade produced a strong response over visual areas, this response was unaffected by the faces emotional expression, both for the first and for subsequent (micro)saccades. This finding suggests that the faces affective content is rapidly evaluated after stimulus onset, leading to only a short-lived sensory enhancement by arousing stimuli that does not repeat itself during immediate refixations. Methodologically, our work demonstrates how eye-tracking and deconvolution modeling can be used to extract several brain responses from each EEG trial, providing insights into neural processing at different latencies after stimulus onset.

neuroscience↗

How the dominant reading direction changes parafoveal processing: A combined EEG/eye-tracking study

Reading directions vary across writing systems. Through long-term experience readers adjust their visual systems to the dominant reading direction in their writing systems. However, little is known about the neural correlates underlying these adjustments because different writing systems do not just differ in reading direction, but also regarding visual and linguistic properties. Here, we took advantage that Chinese is read to different degrees in left-right or top-down directions in different regions. We investigated visual word processing in participants from Taiwan (both top-down and left-right directions) and from mainland China (only left-right direction). Combined EEG/eye tracking was used together with a saccade-contingent parafoveal preview manipulation to investigate neural correlates, while participants read 5-word lists. Fixation-related potentials (FRPs) showed a reduced late N1 effect (preview positivity), but this effect was modulated by the prior experience with a specific reading direction. Results replicate previous findings that valid previews facilitate visual word processing, as indicated by reduced FRP activation. Critically, the results indicate that this facilitation effect depends on experience with a given reading direction, suggesting a specific mechanism how cultural experience shapes the way people process visual information.

neuroscience↗

Parafoveal and foveal N400 effects in natural sentence reading: Evidence from overlap-corrected fixation-related potentials

The depth at which parafoveal words are processed during reading is an ongoing topic of debate. Recent studies using RSVP-with-flanker paradigms have shown that implausible words within sentences elicit N400 components while they are still in parafoveal vision, suggesting that the semantics of parafoveal words can be accessed to rapidly update the sentence representation. To study this effect in natural reading, we combined the co-registration of eye movements and EEG with the deconvolution modeling of fixation-related potentials (FRPs) to test whether semantic plausibility is processed parafoveally during Chinese sentence reading. For one target word per sentence, both its parafoveal and foveal plausibility were orthogonally manipulated using the boundary paradigm. Consistent with previous eye movement studies, we observed a delayed effect of parafoveal plausibility on fixation durations that only emerged on the foveal word. Crucially, in FRPs aligned to the pre-target fixation, a clear N400 effect emerged already based on parafoveal plausibility, with more negative voltages for implausible previews. Once participants fixated the target, we again observed an N400 effect of foveal plausibility. Interestingly, this foveal N400 was absent whenever the preview had been implausible, indicating that when a words (im)plausibility is already processed in parafoveal vision, this information is not revised anymore upon direct fixation. Implausible words also elicited a late positive complex (LPC), but exclusively in foveal vision. Our results provide convergent neural and behavioral evidence for the parafoveal uptake of semantic information, but also indicate different contributions of parafoveal versus foveal information towards higher-level sentence processing.

neuroscience↗

Auditory Language Comprehension During Saccadic Eye Movements: An Investigation Using the N400 Event-Related Brain Potential

In many situations of everyday life, it is important to quickly understand a spoken message despite distraction by an already ongoing activity. Previous dual-task studies recording the N400 component of the event-related potential (ERP) have shown that auditory language comprehension can be strongly delayed by temporally overlapping additional tasks. In the current study, we investigated whether this interference is aggravated by the need for saccadic eye movements and visuospatial attention shifts in the overlapping task. In two dual-task experiments, a synonymy judgment task for spoken words was combined with a visual discrimination task at different stimulus onset asynchronies. In half of the trials, the primary visual task required a 10{degrees} exogenous saccade towards a peripheral stimulus. The timing of semantic processing in the secondary task was assessed by recording the N400. We replicated that with increasing temporal overlap between the tasks, the N400 component was strongly delayed. However, additional saccade-related processes in the primary task had no detrimental effects on concurrent spoken language processing on their own. Unexpectedly, we also observed that a preceding saccade consistently facilitated subsequent motoric processing in the visual task (as indexed by manual reaction times and the lateralized readiness potential), suggesting that saccade execution as such can significantly enhance subsequent processing.

neuroscience↗

Is there magnocellular facilitation of early neural processes underlying visual word recognition? Evidence from masked repetition priming with ERPs

An influential theory in the field of visual object recognition proposes that fast magnocellular (M) information facilitates neural processing of spatially more fine-grained but slower parvocellular (P) information. While written words can be considered as a special type of visual objects, it is unknown whether magnocellular facilitation also plays a role in reading. We used a masked priming paradigm that has been shown to result in neural facilitation in visual word processing and tested whether these facilitating effects are mediated by the magnocellular system. In two experiments, we manipulated the influence of magnocellular and parvocellular systems on visual processing of a contextually predictable target character by contrasting high versus low spatial frequency and luminance versus color contrast, respectively. In addition, unchanged (normal) primes were included in both experiments as a manipulation check. As expected, unchanged primes elicited typical repetition effects in the N1, N250 and P3 components of the ERP in both experiments. In the experiment manipulating spatial contrast, we obtained repetition effects only for the N1 component for both M- and P-biased primes. In the luminance versus color contrast experiment, repetition effects were found in N1 and N250 for both M- and P- biased primes. Furthermore, no interactions were found between M- vs. P-biased prime types and repetition. Together these results indicate that M- and P- information contributes jointly to early neural processes underlying visual word recognition.

neuroscience↗

Parafoveal words can modulate sentence meaning: Electrophysiological evidence from an RSVP-with-flanker task

During natural reading, readers also take up some visual information from not-yet-fixated words to the right of the current fixation and it is well-established that this parafoveal preview facilitates the subsequent foveal processing of the word. However, the extraction and integration of word meaning from the parafoveal word and its possible influence on the semantic sentence context are controversial. In the current study, we recorded event-related potentials (ERPs) in the RSVP-with-flankers paradigm to test whether and how updates of sentential meaning that are based only on parafoveal information influence the subsequent foveal processing. Using Chinese sentences, the sentence congruency of parafoveal and foveal target words were orthogonally manipulated. In contrast to previous research, we also controlled for potentially confounding effects of parafoveal-to-foveal repetition priming (identity preview effects) on the N400. Crucially, we found that the classic effect of foveal congruency on the N400 component only appeared when the word in preview had been congruent with sentence meaning; in contrast, there was no N400 when the preview word had been incongruent. These results indicate that sentence meaning rapidly adapts to parafoveal preview, which already changes the context for the then fixated word. We also show that a correct parafoveal preview generally attenuates the N400 once a word is fixated, regardless of congruency. Taken together, our findings underline the highly generative and adaptive framework of language comprehension.

neuroscience↗

Eye contact in active and passive viewing: event-related brain potential evidence from a combined eye tracking and EEG study

Eye contact is a salient social cue which is assumed to influence early brain processes involved in face perception. The N170 component in the event-related potential (ERP) has frequently been reported to be larger to faces with an averted rather than direct gaze towards the observer. In most studies, however, this effect has been investigated in comparatively artificial, passive settings where participants were instructed to fixate their gaze while observing occasional gaze changes in stimulus faces. Yet, it is unclear whether similar mechanisms are in place during naturalistic gaze interactions involving the continuous interplay of directed and averted gaze between the communication partners. To fill this gap, we compared passive viewing of gaze change sequences with an active condition where participants own gaze continuously interacted with the gaze of a stimulus face; while recording ERPs and monitoring gaze with eye tracking. In addition, we investigated the relevance of emotional facial expressions for gaze processing. For both passive viewing and active interaction, N170 amplitudes were larger when the gaze of stimulus faces was averted rather than directed at the participants. Furthermore, eye contact decreased P300 amplitudes in both conditions. Emotional facial expression influenced N170 amplitudes but did not elicit an early posterior negativity nor did it interact with gaze direction. We conclude that comparable mechanisms of gaze perception are in place in gaze interaction as compared to passive viewing, encouraging the further study of the eye contact effect in naturalistic settings.

neuroscience↗

Analyzing combined eye-tracking/EEG experiments with (non)linear deconvolution models

Fixation-related potentials (FRPs), neural responses aligned to saccade offsets, are a promising tool to study the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/EEG experiments: (i) the synchronization of data streams, (ii) the removal of ocular artifacts, (iii) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, (iv) and the fact that numerous low-level stimulus and saccade properties also influence the post-saccadic neural responses. While effective solutions exist for the first two problems, the latter ones are only beginning to be addressed. In the current paper, we present and review a unified framework for FRP analysis that allows us to deconvolve overlapping potentials and control for linear and nonlinear confounds on the FRPs. An open software implementation is provided for all procedures. We then demonstrate the advantages of this approach for data from three commonly studied paradigms: face perception, scene viewing, and natural sentence reading. First, for a traditional ERP face recognition experiment, we show how deconvolution can separate stimulus-ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in scene viewing, we isolate multiple non-linear influences of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable fixation-related brain potentials during natural vision.

neuroscience↗

Optimized ICA-based removal of ocular EEG artifacts from free viewing experiments

Combining EEG with eye-tracking is a promising approach to study neural correlates of natural vision, but the resulting recordings are also heavily contaminated by activity of the eye balls, eye lids, and extraocular muscles. While Independent Component Analysis (ICA) is commonly used to suppress these ocular artifacts, its performance under free viewing conditions has not been systematically evaluated and many published findings display residual artifacts. Here I evaluated and optimized ICA-based correction for two tasks with unconstrained eye movements: visual search in images and sentence reading. In a first step, four parameters of the ICA pipeline were systematically varied: the (1) high-pass and (2) low-pass filter applied to the training data, (3) the proportion of training data containing myogenic saccadic spike potentials (SP), and (4) the threshold for eye tracker-based component rejection. In a second step, the eye-tracker was used to objectively quantify correction quality of each ICA solution, both in terms of undercorrection (residual artifacts) and overcorrection (removal of neurogenic activity). As a benchmark, results were compared to those obtained with an alternative spatial filter, Multiple Source Eye Correction (MSEC). With commonly used settings, Infomax ICA not only left artifacts in the data of both tasks, but also distorted neurogenic activity during eye movement-free intervals. However, correction could be drastically improved by training the ICA on optimally filtered data in which SPs were massively overweighted. With optimized procedures, ICA removed virtually all artifacts, including the SP and its associated spectral broadband artifact, with little distortion of neural activity. It also outperformed MSEC in terms of SP correction. Matlab code is provided.\n\nAuthor NoteI would like to acknowledge Maarten De Schuymer, who conducted early explorations of the effects of high-pass filtering that helped to initiate the current work. I am also grateful to Lisa Spiering for assisting with the MSEC correction and Werner Sommer for providing an excellent working environment. Collection of one of the datasets was supported by a grant from DFG (FG-868-A2). Comments or corrections are highly appreciated.

neuroscience↗

Unfold: An integrated toolbox for overlap correction, non-linear modeling, and regression-based EEG analysis

Electrophysiological research with event-related brain potentials (ERPs) is increasingly moving from simple, strictly orthogonal stimulation paradigms towards more complex, quasi-experimental designs and naturalistic situations that involve fast, multisensory stimulation and complex motor behavior. As a result, electrophysiological responses from subsequent events often overlap with each other. In addition, the recorded neural activity is typically modulated by numerous covariates, which influence the measured responses in a linear or nonlinear fashion. Examples of paradigms where systematic temporal overlap variations and low-level confounds between conditions cannot be avoided include combined EEG/eye-tracking experiments during natural vision, fast multisensory stimulation experiments, and mobile brain/body imaging studies. However, even \"traditional\", highly controlled ERP datasets often contain a hidden mix of overlapping activity (e.g. from stimulus onsets, involuntary microsaccades, or button presses) and it is helpful or even necessary to disentangle these components for a correct interpretation of the results. In this paper, we introduce unfold, a powerful, yet easy-to-use MATLAB toolbox for regression-based EEG analyses that combines existing concepts of massive univariate modeling (\"regression ERPs\"), linear deconvolution modeling, and non-linear modeling with the generalized additive model (GAM) into one coherent and flexible analysis framework. The toolbox is modular, compatible with EEGLAB and can handle even large datasets efficiently. It also includes advanced options for regularization and the use of temporal basis functions (e.g. Fourier sets). We illustrate the advantages of this approach for simulated data as well as data from a standard face recognition experiment. In addition to traditional and non-conventional EEG/ERP designs, unfold can also be applied to other overlapping physiological signals, such as pupillary or electrodermal responses. It is available as open-source software at http://www.unfoldtoolbox.org.

neuroscience↗