Fajen, B. R. | bioRxiv Science

bioRxiv2023

Coordination of gaze and action during high-speed steering and obstacle avoidance

When humans navigate through complex environments, they coordinate gaze and steering to efficiently sample the visual information needed to guide movement. Gaze and steering behavior during high-speed self-motion has been extensively studied in the context of automobile driving along a winding road. Theoretical accounts that have emerged from this work capture behavior during movement along explicit, well-defined paths over flat, obstacle-free ground surfaces. However, humans are also capable of visually guiding self-motion over uneven terrain that is cluttered with obstacles and may lack an explicit path. An extreme example of such behavior occurs during first-person view drone racing, in which pilots maneuver at high speeds through a dense forest. In this study, we explored the gaze and steering behavior of skilled drone pilots. Subjects guided a simulated quadcopter along a racecourse embedded within a forest-like virtual environment built in Unity. The environment was viewed through a head-mounted display while gaze behavior was recorded using an eye tracker. In two experiments, subjects performed the task in multiple conditions that varied in terms of the presence of obstacles (trees), waypoints (hoops to fly through), and a path to follow. We found that subjects often looked in the general direction of things that they wanted to steer toward, but gaze fell on nearby objects and surfaces more often than on the actual path or hoops. Nevertheless, subjects were able to perform the task successfully, steering at high speeds while remaining on the path, passing through hoops, and avoiding collisions. Furthermore, in conditions that contained hoops, subjects adapted how they approached the most immediate hoop in anticipation of the position (but not the orientation) of the subsequent hoop. Taken together, these findings challenge existing models of steering that assume that steering is tightly coupled to where actors look.

neuroscience↗

bioRxiv2021

Decoding Estimates of Curvilinear Self-Motion from Neural Signals in a Model of Primate MSTd

Self-motion produces characteristic patterns of optic flow on the eye of the mobile observer. Movement along linear, straight paths without eye movements yields motion that radiates from the direction of travel (heading). The observer experiences more complex motion patterns while moving along more general curvilinear (e.g. circular) paths, the appearance of which depends on the radius of the curved path (path curvature) and the direction of gaze. Neurons in brain area MSTd of primate visual cortex exhibit tuning to radial motion patterns and have been linked with linear heading perception. MSTd also contains neurons that exhibit tuning to spirals, but their function is not well understood. We investigated in a computational model whether MSTd, through its diverse pattern tuning, could support estimation of a broader range of self-motion parameters from optic flow than has been previously demonstrated. We used deep learning to decode these parameters from signals produced by neurons tuned to radial expansion, spiral, ground flow, and other patterns in a mechanistic neural model of MSTd. Specifically, we found that we could accurately decode the clockwise/counterclockwise sign of curvilinear path and the gaze direction relative to the path tangent from spiral cells; heading from radial cells; and the curvature (radius) of the curvilinear path from activation produced by both radial and spiral populations. We demonstrate accurate decoding of these linear and curvilinear self-motion parameters in both synthetic and naturalistic videos of simulated self-motion. Estimates remained stable over time, while also rapidly adapting to dynamic changes in the observers curvilinear self-motion. Our findings suggest that specific populations of neurons in MSTd could effectively signal important aspects of the observers linear and curvilinear self-motion. Author SummaryHow do we perceive our self-motion as we move through the world? Substantial evidence indicates that brain area MSTd contains neurons that signal the direction of travel during movement along straight paths. We wondered whether MSTd neurons could also estimate more general self-motion along curved paths. We tested this idea by using deep learning to decode signals produced by a neural model of MSTd. The system accurately decoded parameters that specify the observers self-motion along straight and curved paths in videos of synthetic and naturalistic scenes rendered in the Unreal game engine. Our findings suggest that MSTd could jointly signal self-motion along straight and curved paths.

neuroscience↗