Stochastic Decision Policy Is Optimal Under Costly Control

Abstract

Human and animal decisions are often noisy. This variability is usually attributed to noise that corrupts the decision evidence, a defect that an ideal decision maker would eliminate. Here we ask whether stochastic choice can instead be optimal once the cost of implementing prescribed policies is counted. We treat the decision maker as an ideal observer that maximizes expected gain but pays costs to sharpen its policy away from a random default. Using sequential sampling, a setting in which the optimal policy is precisely defined, we show that this trade-off makes a stochastic policy optimal, replacing the hard decision boundaries of classical theory with soft stopping zones whose softness is set by the price of control. This soft policy aligns with the strategy used in cutting-edge artificial learning agents for robustness. Noise that corrupts the agent's belief, an important confound of noisy behavior, produces superficially similar yet distinctive signatures in behavioral policies. Leveraging this, we recovered the policies of two macaques with low-dimensional recurrent neural networks and found that their behavior carries both a stochastic policy and belief noise. By separating variability in constructing a decision belief from stochasticity in selecting actions, these results establish that stochastic decision policies can themselves be optimal, offering a common principle for understanding behavioral variability in biological and artificial decision makers.