BenchMHC: a reproducible framework for peptide-MHC presentation prediction
Predicting which peptides an MHC molecule presents is the first step in neoantigen and epitope discovery pipelines. Most published predictors are released for inference only, and each is trained on a different dataset, so the effect of a new method cannot be separated from the effect of its training data. We present BenchMHC, an open framework that reproduces, retrains and evaluates pMHC presentation models on public data only, and that records every implementation decision made in a reproduction. Our NetMHCpan-4.1 reproduction reaches 0.794 per-allele Mean Top-K against 0.790 for the official webserver, and 0.0025 Median FRANK against 0.0022. It scores 280,000 peptide/allele pairs in 27 s, about 350 times faster than the original binary. We then examine the training data. In the public NetMHCpan-4.1 eluted-ligand sets, 86 to 99% of hits are found in the human proteome, against about 12% of their decoys, so a model can separate hits from decoys using this difference rather than features related to MHC binding. Redrawing the decoys within the source protein of each hit and matching their cysteine status, with no other change, raises per-allele Mean Top-K on the regenerated v2.0.0 evaluation set from 0.734 to 0.755, and increases Median FRANK on CD8 epitopes from 0.0025 to 0.0053. The same change improves one evaluation set and degrades the other. Code (Apache-2.0) and model checkpoints (CC BY-NC-4.0): https://github.com/instadeepai/BenchMHC-public.