Extending the IICR to complex non-stationary structured models

Abstract

Population genetic studies use genetic data to understand aspects of the evolutionary history of species, particularly to infer their demography. Here, we extend the non-stationary structured coalescent framework to complex models where the parameters of the model, and thus the state space, change between stationary periods. To do this, we introduce glue matrices that allow us to map one state space to another, hence enabling the computation of the Inverse Instantaneous Coalescence Rate (IICR) using Q-matrices of different dimensions. This approach allows us to study models where the number of demes changes due to extinction or foundation. Our analysis confirms that interpreting IICR curves as indicators of population size changes can be misleading. However, we also show that there are cases where new deme foundations can generate an increase in the IICR (forward in time). We explore the impact of changes in migration rates, timing of deme creation or deletion, and sampling strategies on the IICR, identifying several counter-intuitive results. For instance, we find that the IICR starts to decrease several generations before any change in the number of demes, as if the population "knew" something was about to happen. It is particularly unexpected when a deme is founded and the population is only increasing. Our results underscore the need for careful interpretation of PSMC and similar curves, which are often used and interpreted as population size changes. By presenting examples of IICRs for various transitions and sampling scenarios, we emphasize the importance of nuanced approaches in population genetic studies.