Melfi, A. | bioRxiv Science

bioRxiv2018

The Site Frequency Spectrum under Finite and Time-Varying Mutation Rates

The diversity in genomes is due to the accumulation of mutations and the site frequency spectrum (SFS) is a popular statistic for summarizing genomic data. The current coalescent algorithm for calculating the SFS for a given demography assumes the [->] 0 limit, where is the mutation probability (or rate) per base pair per generation. The algorithm is applicable when N, N being the haploid population size, is negligible. We derive a coalescent based algorithm for calculating the SFS that allows the mutation rate (t) as well as the population size N(t) to vary arbitrarily as a function of time. That algorithm shows that the probability of two mutations in the genealogy becomes noticeable already for = 10-8 for samples of n = 105 haploid human genomes and increases rapidly with . Our algorithm calculates the SFS under the assumption of a single mutation in the genealogy, and the part of the SFS due to a single mutation depends only mildly on the finiteness of . However, the dependence of the SFS on variation in can be substantial for even n = 100 samples. In addition, increasing and decreasing mutation rates alter the SFS in different ways and to different extents.

genetics↗

bioRxiv2017

The validity of the coalescent approximation for large samples

The Kingman coalescent, widely used in genetics, is known to be a good approximation when the sample size is small relative to the population size. In this article, we investigate how large the sample size can get without violating the coalescent approximation. If the haploid population size is 2N, we prove that for samples of size N1/3-{epsilon},{epsilon} > 0, coalescence under the Wright-Fisher (WF) model converges in probability to the Kingman coalescent in the limit of large N. For samples of size N2/5-{epsilon} or smaller, the WF coalescent converges to a mixture of the Kingman coalescent and what we call the mod-2 coalescent. For samples of size N1/2 or larger, triple collisions in the WF genealogy of the sample become important. The sample size for which the probability of conformance with the Kingman coalescent is 95% is found to be 1.47 x N0.31 for N [isin] [103, 105], showing the pertinence of the asymptotic theory. The probability of no triple collisions is found to be 95% for sample sizes equal to 0.92 x N0.49, which too is in accord with the asymptotic theory.\n\nVarying population sizes are handled using algorithms that calculate the probability of WF coalescence agreeing with the Kingman model or taking place without triple collisions. For a sample of size 100, the probabilities of coalescence according to the Kingman model are 2%, 0%, 1%, and 0% in four models of human population with constant N, constant N except for two bottlenecks, recent exponential growth, and increasing recent exponential growth, respectively. For the same four demographic models and the same sample size, the probabilities of coalescence with no triple collision are 92%, 73%, 88%, and 87%, respectively. Visualizations of the algorithm show that even distant bottlenecks can impede agreement between the coalescent and the WF model.\n\nFinally, we prove that the WF sample frequency spectrum for samples of size N1/3-{epsilon} or smaller converges to the classical answer for the coalescent.

genetics↗