Mobius: Mixture-Of-Experts Transformer Model in Epigenetics of ME/CFS and Long COVID

Abstract

Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS) and Long COVID are chronic debilitating post-infectious illnesses that collectively affect up to 470 million individuals. Unlike illnesses of comparable scale, there are no validated blood or imaging tests for the clinical diagnosis of these conditions. Currently, these conditions are diagnosed through clinical exclusion, resulting in approximately 90% of ME/CFS patients being incorrectly diagnosed as Long COVID patients. This misdiagnosis contributes to delayed care and millions of dollars in healthcare burdens. We present Mobius, a transformer-based model that uses autoencoder-derived features from blood DNA methylation to distinguish ME/CFS, Long COVID, and healthy controls. Using 852 samples from 14 distinct datasets, our method employs three innovations: (i) self-supervised masked pretraining to learn epigenetic patterns, (ii) a sparsely-gated mixture-of-experts architecture to handle heterogeneous data, and (iii) an adaptive computation time mechanism for dynamic inference. Mobius achieved 97.06% accuracy (macro-F1 0.95, AUROC 0.96), outperforming current symptom-based diagnostics (58%) and baseline models such as XGBoost (82%). Ablation experiments showed that pretraining added 6% accuracy and that the gating and adaptive depth contributed an additional 7%. Our open-source pipeline could enable a much-needed objective blood test for these conditions and guide targeted precision medicine therapies.