From diminishing returns to entrenchment: a unifying theory of epistasis along adaptive walks revealed by Fisher's geometric model

Abstract

Epistasis makes the fitness effect of a mutation depend on the genetic background in which it occurs, thereby shaping the accessibility and reversibility of evolutionary trajectories. Along adaptive walks, this path-dependent epistasis can take distinct forms: contingency, when a mutation requires prior substitutions to be beneficial; entrenchment, when later substitutions make its reversion increasingly deleterious; and diminishing returns, when successive beneficial mutations reduce one anothers effects. Although these regimes have been documented experimentally, the conditions under which each predominates remain poorly understood. Here we use Fishers geometric model to derive a general framework for path-dependent epistasis under stabilizing selection on a multidimensional phenotype. We show that epistasis between substitutions has a simple geometric interpretation: contingency and entrenchment arise when the collateral effects of mutations, orthogonal to the direction of the optimum, are compensated by the preceding or subsequent adaptive path, whereas diminishing returns arise when successive substitutions remain strongly aligned with the same direction of selection. Analytical results and simulations reveal a transition controlled by a single composite parameter combining phenotypic complexity, mutation size, and distance to the optimum. Far from the optimum, adaptive walks are dominated by diminishing returns epistasis. As populations approach the optimum, or as phenotypic complexity increases, antagonistic pleiotropy generates systematic contingency and entrenchment. At mutation-selection-drift equilibrium, these effects become strong, rapidly established, and increase with phenotypic complexity. These results show that contingency and entrenchment do not require specific molecular interactions between residues: they emerge generically from nonspecific epistasis produced by stabilizing selection on pleiotropic traits. Fishers geometric model thus unifies diminishing returns, contingency, and entrenchment as distinct regimes of the same underlying geometry of adaptation.