AffectRoute: Role-Structured EEG and Peripheral Physiology for Subject-Independent Affect Regression

Abstract

Subject-independent affect regression from physiological signals remains challenging because emotional responses vary substantially across individuals, widely used datasets provide only coarse trial-level annotations, and heterogeneous physiological modalities may not contribute reliably when treated as if they were interchangeable predictors. We have developed AffectRoute, a protocol-conditioned subject-independent affect regression that is conditioned on the protocol and assigns separate predictive functions to information from the population, electroencephalography (EEG), and peripheral physiological signals (PPS). First, a source-population prior establishes a trial-level affective anchor using only the data from source participants. TrajBridge then combines an EEG representation that is supervised by REFED for participant-specific adjustments with temporal structure obtained from the continuous REFED annotations in order to create a weakly supervised segment-resolved pseudo-trajectory and to establish a frozen trial-level baseline. PhysioRoute next reduces the remaining error by breaking down the residual correction into a source-derived direction, which is estimated from the out-of-fold residuals within the source group, and a channel-specific magnitude derived from the PPS. When evaluated on DEAP and DREAMER using a leave-one-subject-out approach at the participant level, AffectRoute showed consistent step-by-step improvements in both the mean absolute error and the concordance correlation coefficient. A method that relied solely on the source data was clearly worse than PhysioRoute, showing that the final improvement cannot be accounted for by transferable source residual regularity alone. Conventional alternatives to fusing the PPS were also found to be consistently less effective, although analyses at the channel level and with a leave-one-channel-out design showed that the peripheral contributions are axis-dependent yet distributed across channels. These results indicate that structured residual inference is an effective alternative to unrestricted multimodal fusion for subject-independent affect regression.