Probing Physical Intuition in Large Language Models through Partner-Dependent Mutation Prediction

Abstract

Structure-guided mutation design requires relating a proposed substitution to its molecular environment to anticipate its effects. Existing evaluations of large language models (LLMs) in protein engineering emphasize sequence-based prediction or tool-integrated workflows, leaving unclear whether LLMs themselves can infer mutation effects from structural evidence without dedicated predictors. We evaluate physical intuition-qualitative reasoning that connects structural observations with molecular interaction principles-using a benchmark that tests the same substitution in the same protein across different binding partners, with predictions based on local atomic coordinates and no dedicated mutation-effect predictors. In the tested comparisons, the best-performing LLM achieved group accuracy comparable to the evaluated energy predictors and Biomni, an LLM-driven tool workflow, while the methods differed in their prediction patterns. Most evaluated LLMs achieved higher group accuracy for larger affinity improvements. Structural interventions support selective dependence on mutation-site and partner information, while assessable structural claims largely agree with the supplied coordinates. These findings support direct LLM structural reasoning as a basis for developing rational mutation-design tools that connect molecular observations with partner-specific effects.