Adaptive Multimodal Temporal Transformer for Bio Signal Driven Stroke Rehabilitation Prognosis

Abstract

Stroke rehabilitation requires continuous, individualized assessment of recovery progress to optimize treatment planning. Existing prognosis models often rely on static, single-timepoint predictions and lack adaptive fusion of multi-modal bio-signals, limiting their clinical interpretability and utility. To address these challenges, we propose AMTT-Net, an adaptive multimodal temporal transformer for dynamic stroke rehabilitation prognosis. AMTT-Net integrates wearable biosensor data, video keypoints, and clinical records through an attention-based Adaptive Fusion Module and Trajectory-Aware Prediction Heads to jointly predict continuous recovery trajectories and responder likelihoods under different rehabilitation modalities. Evaluations on the StrokeBalance-Sim dataset demonstrate that AMTT-Net achieves superior performance in both trajectory regression and responder classification tasks, while providing interpretable, patient-specific insights to support personalized rehabilitation strategies.