From haptic exploration to object perception in 3D

Abstract

Active touch determines which parts of an object are sampled and therefore which sensory information becomes available for perception. However, little is known about how the three-dimensional geometry of an object guides exploratory hand movements and how the information acquired during exploration contributes to perceptual decisions. Here, we investigated these questions in two experiments in which blindfolded participants explored 3D-printed replicas of bell-peppers while their hand movements were recorded using motion tracking. The task of participants' was to identify the reference object among two comparison objects (target & competitor). In the first experiment, objects were presented in a fixed orientation, whereas in the second, object orientation varied across presentations to disrupt the correspondence between hand configuration and object geometry. We first trained a neural network to predict the spatial distribution of touch directly from three-dimensional surface structure. The model successfully predicted touch locations on previously unseen objects, outperforming a generic spatial exploration prior in both experiments. Geometric analyses revealed that haptic saliency was associated with specific combinations of local surface curvatures rather than simply with the overall magnitude of curvature. We then examined whether the information acquired during exploration could predict subsequent recognition choices, considering global and local surface geometry and potential memory decay of acquired object representations. Models integrating global and local information across the complete exploration reliably identified the objectively correct object and correct participants' choices. In contrast, restricting the comparison to temporally proximal portions of successive explorations reduced objective recognition accuracy but increased correspondence with participants' errors. Overall, our findings suggest that haptic exploration is at least to some extent driven by object-specific geometry, but that the information acquired is not necessarily integrated optimally into perceptual decisions. Instead, memory limitations and potentially incomplete integration may constrain how sampled three-dimensional shape information contributes to recognition.