Learning Free Energy Pathways through Reinforcement Learning of Adaptive Steered Molecular Dynamics

Ho, N.; Cava, J. K.; Vant, J.; Shukla, A.; Miratsky, J.; Turaga, P.; Maciejewski, R.; Singharoy, A.

doi:10.1101/2022.10.04.510845

Home/Archive/bioRxiv

bioRxiv · 10.1101/2022.10.04.510845

Learning Free Energy Pathways through Reinforcement Learning of Adaptive Steered Molecular Dynamics

Ho, N.·Cava, J. K.·Vant, J.·Shukla, A.·Miratsky, J.·Turaga, P.·Maciejewski, R.·Singharoy, A.

View original source ↗Export metadata ↓

Abstract

In this paper, we develop a formulation to utilize reinforcement learning and sampling-based robotics planning to derive low free energy transition pathways between two known states. Our formulation uses Jarzynskis equality and the stiffspring approximation to obtain point estimates of energy, and construct an informed path search with atomistic resolution. At the core of this framework, is our first ever attempt we use a policy driven adaptive steered molecular dynamics (SMD) to control our molecular dynamics simulations. We show that both the reinforcement learning and robotics planning realization of the RL-guided framework can solve for pathways on toy analytical surfaces and alanine dipeptide.

Explore related subjects

biophysics Preprint

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Ho, N., Cava, J. K., Vant, J., Shukla, A., Miratsky, J., Turaga, P., Maciejewski, R., Singharoy, A.. 2022-10-05. Learning Free Energy Pathways through Reinforcement Learning of Adaptive Steered Molecular Dynamics. https://doi.org/10.1101/2022.10.04.510845

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Design of colipids with controlled lateral distance of head group moieties for targeting strategies in the field of lipid-based nanoparticles

Lipid nanoparticles and liposomes are nanoparticle drug formulations which are firmly established in clinics. In order to expand their application and improve efficacy, surface modifications can be a key strategy, especially for cell specific targeting. Targeting can be achieved by integrating ligand functionalized lipids or lipidoids on the nanoparticle surface. In this case, not solely the targeting ligand itself, but also the structure of the whole targeting lipid/lipidoid can affect the interaction of the targeting ligand with the binding motive on cells. Especially, distal spacer structures between lipid anchor and ligand unit, often composed of polyethylene glycol chains, provide a certain distance of the ligand from the nanoparticle surface. A concept which is currently neglected is the control of the lateral distance between ligand structures on the nanoparticle. Herein we investigated the possibility to design lipidoids with a lateral spacer composed of polyethylene glycol units. We synthesized and characterized different lipidoids which provide a primary amine in the head group for possible ligand coupling. In our model lipidoids the head groups were functionalized with a paramagnetic probe to apply electron paramagnetic resonance (EPR) methods for the characterization of the lipidoid interaction with model nanoparticles, in our case liposomes. Continuous wave EPR was used to study the efficiency of the novel lipidoids to integrate into lipid bilayers of model liposomes, which was shown to depend on lipidoid structure and liposome composition. Furthermore, using double electron-electron resonance (DEER), we observed lipidoid conformational changes occurring upon integration into liposomes.

biophysics↗

bioRxiv2026

Beyond Sequence and Structure: Pretraining PLMs on Continuous Biophysical Fields

We introduce the Biophysical Transformer, a protein language model that adds continuous biophysical fields as a third pretraining signal beside sequence and structure. A Poisson-Boltzmann solver computes the electrostatic potential around each three-residue token, and the model learns to predict it from sequence alone. A rotationally invariant spherical-harmonic descriptor serves as the loss target, eliminating the need for frame alignment during training and for coordinates at inference. Against a matched, one-flag control on the FLIP Meltome thermostability benchmark, the field objective raises Spearman correlation on every trained head of both human splits.The gain on human survives residualizing dipeptide composition under two architecturally different readouts and survives removing the test proteins that overlap the pretraining corpus. No effect survives on the organism-mixed split. At 1.1M parameters, both the Biophysical Transformer and its sequence-only control exceed FLIP's one-hot CNN and untrained-ESM baselines on every split. All results rest on a single pretraining run per arm,and the current design does not separate field content from generic regularization.

biophysics↗

bioRxiv2026

An auditable harness for biomolecular dynamics simulations by AI agents

AI agents now generate biological hypotheses faster than they can be tested. Hypotheses stated at the atomic level can be examined with molecular dynamics (MD) simulations. For new questions without a reference, trust in an agent-run simulation rests on a tool-written process record, not the agent's account. Existing MD agents leave this record to the language model. Here we present MDClaw, a harness whose validating tools and directed acyclic graph (DAG) check stages before execution and record every simulation stage. On MDDataBench, which grades outputs against deposited simulations, four open-weight models with MDClaw passed all three runs on 81-100% of 31 test systems. The DAG complements this outcome check. An ablation without the DAG revealed 11 successful runs that had skipped minimization or equilibration, or simulated a different structure. MDClaw carried questions from human and AI scientists through simulation to trajectories traceable to their inputs and settings.

biophysics↗