Biology subjects

Keshava, A.

Publications and source records attributed to Keshava, A..

6 recordsLinked to original sources

bioRxiv2024

Combining EEG and Eye-Tracking in Virtual Reality - Obtaining Fixation-Onset ERPs and ERSPs

Extensive research conducted in controlled laboratory settings has prompted an inquiry into how results can be generalized to real-world situations influenced by the subjects actions. Virtual reality lends itself ideally to investigating complex situations but requires accurate classification of eye movements, especially when combining it with time-sensitive data such as EEG. We recorded eye-tracking data in virtual reality and classified it into gazes and saccades using a velocity-based classification algorithm, and we cut the continuous data into smaller segments to deal with varying noise levels, as introduced in the REMoDNav algorithm. Furthermore, we corrected for participants translational movement in virtual reality. Various measures, including visual inspection, event durations, and the velocity and dispersion distributions before and after gaze onset, indicate that we can accurately classify the continuous, free-exploration data. Combining the classified eye-tracking with the EEG data, we generated fixation-onset ERPs and ERSPs, providing further evidence for the quality of the eye movement classification and timing of the onset of events. Finally, investigating the correlation between single trials and the average ERP and ERSP identified that fixation-onset ERSPs are less time-sensitive, require fewer repetitions of the same behavior, and are potentially better suited to study EEG signatures in naturalistic settings. We modified, designed, and tested an algorithm that allows the combination of EEG and eye-tracking data recorded in virtual reality.

neuroscience↗

bioRxiv2024

Low-Dimensional Representations of Visuomotor Coordination for Natural Behavior

Understanding how the eyes, head, and hands coordinate in natural contexts is a critical challenge in visuomotor coordination research, often constrained by sedentary tasks, cued actions, or restricted settings. To address this gap, we conducted an experiment where participants could self-generate pick-and-place actions on a life-size shelf in a virtual environment, recording concurrent gaze and body movements. Subjects exhibited intricate translation and rotation movements of the eyes, head, and hands during the task. We employed a time-dependent principal component analysis to study the relationship between the movements of the eyes, head, and hands relative to the onset of the action. We reduced the overall dimensionality into 2D representations, capturing up to 65% of the movement variance just in time with the actions. Our analysis revealed a synergistic coupling of the eye-head and eye-hand systems, as well as a strong coupling within the head-hand system. The eyes synchronized with the head and hands close to the action onset, with variations in coupling observed in horizontal and vertical planes, indicating distinct mechanisms for coordination in the brain. Crucially, the head and hands remained tightly coupled throughout the observation period, suggesting a shared neural code driving these effectors. Notably, the low-dimensional representations of the eye-head-hand movement vectors showed maximum predictive accuracy of the actions location ~200ms before the action onset, highlighting just-in-time coordination among the three effectors. Furthermore, the predictive accuracy was significantly influenced by the location of the upcoming action. Our study emphasizes the differential visuomotor control subject to the task structure, providing insights into the dynamic interplay of eye, head, and hand movements during natural behavior. NEW & NOTEWORTHYStudying natural, self-initiated, complex visuomotor coordination, we observe low-dimensional dynamics with distinct patterns along horizontal and vertical axes. The eyes horizontal movement showed notable independence, aligning with head and hand movements just in time for action. Notably, around critical events, the dimensionality of the complex movements is further reduced, indicating dynamic correspondence of eye-head-hand coordination.

neuroscience↗

bioRxiv2021

Realism Matters. Eye movements in VR reveal prior tool knowledge, intended interaction, and end-state comfort planning

AO_SCPLOWBSTRACTC_SCPLOWSeminal studies on human cognitive behavior have been conducted in controlled laboratory settings, demonstrating that visual attention is mainly goal-directed and allocated based on the action performed. However, it is unclear how far these results generalize to cognition in more naturalistic settings. The present study investigates active inference processes revealed by eye movements during interaction with familiar and novel tools with two levels of realism of the action affordance. We presented participants with 3D tool models that were either familiar or unfamiliar, oriented congruent or incongruent to their handedness, and asked participants to interact with them by lifting or using. Importantly, we used the same experimental design in two setups. In the first experiment, participants interacted with a VR controller in a low realism environment; in the second, they performed the task with an interaction setup that allowed differentiated hand and finger movements in a high realism environment. We investigated the differences in odds of fixations and their eccentricity towards the tool parts before action initiation. The results show that participants fixate more on the tools effector part before action initiation for the use task for unfamiliar tools. Furthermore, with more realistic action affordances, subjects fixate more on the tools handle as a function of the tools orientation, well before the action was executed. Secondly, the spatial viewing bias on the tool reveals early fixations are influenced by the task and the familiarity of the tools. In contrast, later fixations are associated with the manual planning of the interaction. In sum, the findings from the experiments suggest that fixations are made in a task-oriented way to plan the intended action well before action initiation. Further, with more realistic action affordances, fixations are made towards the proximal goal of optimally planning the grasp even though the perceived action on the tools is identical for both experimental setups. Taken together, proximal and distal goal-oriented planning is contextualized to the realism of action/interaction afforded by an environment.

neuroscience↗

bioRxiv2021

Coordinating With a Robot Partner Affects Action Monitoring Related Neural Processing

Robots start to play a role in our social landscape, and they are progressively becoming responsive, both physically and socially. It begs the question of how humans react to and interact with robots in a coordinated manner and what the neural underpinnings of such behavior are. This exploratory study aims to understand the differences in human-human and human-robot interactions at a behavioral level and from a neurophysiological perspective. For this purpose, we adapted a collaborative dynamical paradigm from Hwang et al. (1). All 16 participants held two corners of a tablet while collaboratively guiding a ball around a circular track either with another participant or a robot. In irregular intervals, the ball was perturbed outward creating an artificial error in the behavior, which required corrective measures to return to the circular track again. Concurrently, we recorded electroencephalography (EEG). In the behavioral data, we found an increased velocity and positional error of the ball from the track in the human-human condition vs. human-robot condition. For the EEG data, we computed event-related potentials. To explore the temporal and spatial differences in the two conditions, we used time-regression with overlap-control and corrected for multiple-comparisons using Threshold-Free-Cluster Enhancement. We found a significant difference between human and robot partners driven by significant clusters at fronto-central electrodes. The amplitudes were stronger with a robot partner, suggesting a different neural processing. All in all, our exploratory study suggests that coordinating with robots affects action monitoring related processing. In the investigated paradigm, human participants treat errors during human-robot interaction differently from those made during interactions with other humans.

neuroscience↗

bioRxiv2021

Just-in-time: gaze guidance behavior while action planning and execution in VR

Natural eye movements have primarily been studied for over-learned activities such as tea-making, sandwich-making, and hand-washing, which have a fixed sequence of associated actions. These studies indicate a sequential activation of low-level cognitive schemas facilitating task completion. However, it is unclear if these action schemas are activated in the same pattern when a task is novel and a sequence of actions must be planned in the moment. Here, we recorded gaze and body movements in a naturalistic task to study action-oriented gaze behavior. In a virtual environment, subjects moved objects on a life-size shelf to achieve a given order. In order to compel cognitive planning, we added complexity to the sorting tasks. Fixations aligned with the action onset showed gaze as tightly coupled with the action sequence, and task complexity moderately affected the proportion of fixations on the task-relevant regions. Our analysis further revealed that the gaze was allocated to action-relevant targets just in time. Planning behavior predominantly corresponded to a greater visual search for task-relevant objects before the action onset. The results support the idea that natural behavior relies very little on working memory, and humans refrain from encoding objects in the environment to plan long-term actions. Instead, they prefer just-in-time planning by searching for action-relevant items in the moment, directing their body and hand to it, monitoring the action until it is terminated, and moving on to the next action. Author SummaryEye movements in the natural environment have primarily been studied for over-learned and habitual everyday activities (tea-making, sandwich-making, hand-washing) with a fixed sequence of associated actions. These studies show eye movements correspond to a specific order of actions learned over time. In this study, we were interested in how humans plan and execute actions for tasks that do not have an inherent action sequence. To that end, we asked subjects to sort objects based on object features on a life-size shelf in a virtual environment as we recorded their eye and body movements. We investigated the general characteristics of gaze behavior while acting under natural conditions. Our paper provides a comprehensive approach to preprocess naturalistic gaze data in virtual reality. Furthermore, we provide a data-driven method of analyzing the different action-oriented functions of gaze. The results show that bereft of a predefined action sequence, humans prefer to plan only their immediate actions where eye movements are used to search for the target object to immediately act on, then to guide the hand towards it, and to monitor the action until it is terminated. Such a simplistic approach ensures that humans choose sub-optimal behavior over planning under sustained cognitive load.

neuroscience↗

bioRxiv2020

Embodied Spatial Knowledge Acquisition in Immersive Virtual Reality: Comparison of Direct Experience and Map Exploration

Investigating spatial learning in virtual environments allows studying different sources of information under controlled conditions. We built a virtual environment in the style of a European village and investigated spatial knowledge acquisition by experience in the virtual environment and by the use of an interactive map. We tested knowledge of cardinal directions, building-to-building orientation, and judgment of direction between buildings. We find that judgment of directions was more accurate after virtual reality exploration than after map exploration, and the opposite results were observed for knowledge of cardinal directions and relative orientation between buildings. Further, the alignment effect was confined to the map exploration condition. Taken together, our results suggest that the source of spatial exploration differentially influenced spatial knowledge acquisition.

neuroscience↗