Analyzing Gaze and Hand Movement Patterns in Leader-Follower Interactions During a Time-Continuous Cooperative Manipulation Task

Abstract

In daily life, people often interact by taking on leader and follower roles. Unlike laboratory experiments, these interactions unfold naturally and continuously. Although it is well established that gaze typically precedes object manipulation, much less is known about how gaze-hand patterns evolve in interactive settings where one person must take the others actions into account. Here we examine predictive, planning-related behavior in a two-player tabletop game called "do-undo." Participants alternated as Leader and Follower. The Leader performed simple pick-and-place actions to alter the arrangement of objects, while the Follower used other objects to restore the previous configuration. We recorded eye and hand movements, along with object trajectories, using a system that combined eye tracking with multi-camera motion capture. Touch sensors on the players hands provided precise timing of contacts, allowing us to segment cooperative action into well-defined temporal intervals. As expected, eye fixations consistently preceded manipulation, but clear role differences emerged. Leaders looked more often and earlier at target objects. In many trials, their gaze anticipated not only their own actions but also those required of the Follower. Leaders also more frequently checked the outcome of the do-undo sequence. Both roles showed gaze patterns consistent with memorization, but alternating gazes between objects and destinations were much more common in Leaders. Some patterns suggested longer-term planning beyond the immediate action. These findings reveal distinct decision-making and planning strategies in Leaders and Followers. Leaders consider not only their own next moves but also the potential actions of their partners, shedding light on the complex cognitive processes that underly everyday human interaction.