bioRxiv2017
One of the triumphs of evolutionary biology is the discovery of robust mechanisms that promote the evolution of cooperative behaviors even when those behaviors reduce the fertility or survival of cooperators. Though these mechanisms, kin selection, reciprocity, and nonlinear payoffs to cooperation, have been extensively studied separately, investigating their joint effect on the evolution of cooperation has been more difficult. Moreover, how these mechanisms shape variation in cooperation is not well known. Such variation is crucial for understanding the evolution of behavioral syndromes and animal personality. Here, I use the tools of kin selection theory and evolutionary game theory to build a framework that integrates these mechanisms for pairwise social interactions. Using relatedness as a measure of the strength of kin selection, responsiveness as a measure of reciprocity, and synergy as a measure of payoff nonlinearity, I show how different combinations of these three parameters produce directional selection for or against cooperation or variation in levels of cooperation via balancing or diversifying selection. Moreover, each of these outcomes maps uniquely to one of four classic games from evolutionary game theory, which means that modulating relatedness, responsiveness, and synergy effectively transforms the payoff matrix from one the evolutionary game to another. Assuming that cooperation exacts a fertility cost on cooperators and provides a fertility benefit to social partners, a prisoners dilemma game and directional selection against cooperation occur when relatedness and responsiveness are low and synergy is not too positive. Enough positive synergy in these conditions generates a stag-hunt game and diversifying selection. High levels of relatedness or responsiveness turn cooperation from a fitness cost into a fitness benefit, which produces a mutualism game and directional selection for cooperation when synergy is not too negative. Sufficiently negative synergy in this case creates a hawk-dove game and balancing selection for cooperation. I extend the results with relatedness and synergy to larger social groups and show that how group size changes the effect of relatedness and synergy on selection for cooperation depends on how the per capita benefit of cooperation changes with group size. Together, these results provide a general framework with which to generate comparative predictions that can be tested using quantitative genetic techniques and experimental techniques that manipulate investment in cooperation. These predictions will help us understand both interspecific variation in cooperation as well as within-population and within-group variation in cooperation related to behavioral syndromes.