Adaptive Gaze and Hand Coordination while Manipulating and Monitoring the Environment in Parallel

Abstract

Research on visually guided object manipulation has shown that participants fixate goal locations-such as objects to be grasped and locations where they are placed-prior to hand arrival, with gaze serving two primary functions: directing the hand (or object in hand) to the vicinity of the goal using peripheral vision and gaze related signals, and guiding the hand using central vision as it approaches the goal. However, in real world scenarios, manipulation tasks are often performed while concurrent monitoring of the environment, resulting in competition for gaze. Here we examined gaze-hand coordination under such conditions. Participants performed a manipulation task, that involved grasping balls and placing them at target locations, while concurrently monitoring a display to detect probabilistically occurring visual events, which required central vision. Participants managed gaze competition in two main ways. First, fixations allocated to the action task were brief and prioritized directing the hand towards the goal (object or target location); participants then relied on tactile feedback to complete the action (grasping or placing the object). When tactile feedback was reduced-by using a tool instead of the fingertips to perform the task-gaze additionally served the guiding function. Second, participant reduced gaze competition by exploiting temporal regularities of events in the monitoring task. Specifically, they adjusted both gaze allocation and hand movement timing to reduce the likelihood that action task fixations would coincide with visual events. These findings demonstrate how individuals flexibly integrate sensorimotor control with analysis of environmental statistics to manage competing visual demands. Significance StatementIn everyday behaviour, we often perform manual tasks while simultaneously monitoring the environment, creating competition for gaze. How the brain resolves this competition remains poorly understood. Using a novel paradigm combining an object manipulation task with visual event monitoring, we show that participants integrate knowledge of sensorimotor demands and temporal regularities in the monitoring task to manage gaze. Specifically, we found that participants preferentially allocated gaze to the action task when it is most critical for sensorimotor control and when the likelihood of a visual event was low. Additionally, participants adjusted their hand movement timing based on event statistics to reduce gaze competition. These findings reveal how the brain dynamically allocates gaze resources across competing sensorimotor and visual task demands.