EEG decoding reveals neural predictions for naturalistic material behaviors

Abstract

Material properties like softness or stickiness determine how an object can be used. Based on our real-life experience, we form strong expectations about how objects should behave under force, given their typical material properties. Such expectations have been shown to modulate perceptual processes, but we currently do not know how expectation influences the temporal dynamics of the cortical visual analysis for objects and their materials. Here, we tracked the neural representations of expected and unexpected material behaviors using time-resolved EEG decoding in a violation-of-expectation paradigm, where objects fell to the ground and deformed in expected or unexpected ways. Participants were 25 men and women. Our study yielded three key results: First, both objects and materials were represented rapidly and in a temporally sustained fashion. Second, objects exhibiting unexpected material behaviors were more successfully decoded than objects exhibiting expected behaviors within 190ms after the impact, which might indicate additional processing demands when expectations are unmet. Third, general signals of expectation fulfillment that generalize across specific objects and materials were found within the first 150ms after the impact. Together, our results provide new insights into the temporal neural processing cascade that underlies the analysis of real-world material behaviors. They reveal a sequence of predictions, with cortical signals progressing from a general signature of expectation fulfillment towards increased processing of unexpected material behaviors. Significance StatementIn the real world, we can make accurate predictions about how an objects material shapes its behavior: For instance, we know that cups are typically made of porcelain and shatter when we accidentally drop them. Here, we use EEG to experimentally test how expectations about material behaviors impact neural processing. We showed our participants videos of objects that exhibited expected material behaviors (such as a glass shattering when falling to the ground) or unexpected material behaviors (such as a glass melting upon impact). Our results reveal a hierarchy of predictions in cortex: The visual system rapidly generates signals that index whether expectations about material behaviors are met. These signals are followed by increased processing of objects displaying unexpected material behaviors.