Reinforcement Learning for Efficient Locomotion of Bio-Inspired Microswimmers in Low Reynolds Number Fluids

Abstract

Microswimmers, microscopic self-propelled agents, play vital roles in biological processes such as cellular motility, chemotaxis, and biofilm formation, and inspire the design of artificial microrobots for targeted drug delivery, microsurgery, and environmental sensing. Operating at low Reynolds numbers, where viscous forces overwhelmingly dominate and inertial effects are negligible, microswimmers face unique hydrodynamic constraints. According to Purcells Scallop Theorem, time-reversible or reciprocal motions cannot produce net locomotion, necessitating complex, non-reciprocal deformation cycles for effective propulsion. In this study, we couple low Reynolds number hydrodynamics, modeled via the Stokes equation and Oseen tensor, with reinforcement learning (RL) algorithms to autonomously discover optimal swimming gaits across a range of biologically inspired microswimmer models. Beginning with the classical Najafi-Golestanian three-sphere swimmer, whose cyclical deformation mimics flagellar or ciliary beating, we validate our framework by reproducing established propulsion patterns. We then enhance biological realism by replacing rigid linkers with oscillatory elastic springs and incorporating electrostatic interactions among charged spheres, modeling chemotactic responses and intercellular electrostatic effects observed in microorganisms. Extending beyond linear architectures, we investigate more complex geometries inspired by microbial shapes and locomotion mechanisms, including triangular, plus-like, and cage-like swimmers that emulate bacterial flagella bundling, amoeboid deformation, and cargo encapsulation strategies. Our reinforcement learning approach reveals emergent locomotion strategies reminiscent of natural traveling-wave patterns, phase synchronization, and directional taxis. Remarkably, the cage-like swimmer exhibits nearly triple the displacement of the classical three-sphere model, emphasizing the influence of geometric constraints and structural complexity on swimming efficiency. This interdisciplinary work bridges biophysical modeling and machine learning, providing insights into microscale motility mechanisms and advancing the engineering of biomimetic microrobots capable of navigating complex fluidic environments in biomedical and ecological contexts.