In Eurasia, humans once had Neanderthals for neighbors. That time of co-existence seems to have left its marks in our genome; non-Africans today share more genetic variants with Neanderthals than Africans do. But does this really mean there was interbreeding between humans and the Hominids next door?

Some have previously proposed an alternative explanation for that pattern of greater relatedness between Neanderthals and non-Africans. Maybe the ancestral population in Africa was structured such that the ancestors of the Neanderthals came from the same sub-population as the ancestors of humans. Though considered unlikely by some, the ancestral structure hypothesis has remained a formal hole in the argument that we once interbred with Neanderthals.

But now, in the latest issue of GENETICS, Lohse and Frantz reject the alternative hypothesis, confirming not only that interbreeding is the most likely explanation, but that all the information needed to distinguish between the two possibilities is contained within a single genome of each type: Neanderthal, European/Asian, African and a chimpanzee outgroup. That means the same method can be used to detect ancestral admixture in other extinct or rare species, for which minimal samples of individuals are available.

In fact, the pair were originally motivated to develop the method for studies of species for which samples and genomic resources are severely limited. Lohse, at the University of Edinburgh, works on speciation and population history in a range of insects. Frantz and his colleagues at Wageningen University are looking at island radiation of pig species, many of which are critically endangered.

So how does the new approach work?

Since the Neanderthal genome was first sequenced in 2010, the best genomic evidence for interbreeding had been a simple summary statistic that compared the number of mutations shared by Eurasians and Neanderthals with the number shared by Africans and Neanderthals (known as the “D-statistic”). The D-statistic has also been used as evidence of interbreeding in studies of other animals.

Handy as it is, the D-statistic can’t tell the difference between the interbreeding and ancestral structure scenarios. The evidence that has swayed opinion in favor of interbreeding has largely come from studies of modern human population diversity. Patterns of linkage disequilibrium and allele frequency spectra in people today seem more consistent with interbreeding than ancestral structure.

In the new approach, the authors perform an explicit statistical comparison between the two scenarios using maximum likelihood analysis of triplets of genomes (Neanderthal, African and either Asian or European).

To squeeze more information out of the limited samples, they considered all types of mutations, not just variants shared by Eurasians and Neandertals and those shared by Africans and Neanderthals. They adopted a computational technique (published in GENETICS in 2012) to calculate the probability of mutational patterns in short blocks of sequence under both models. Dividing the genomes into blocks that were short enough to ignore recombination within them allowed the authors to fit these scenarios numerically, without relying on simulations.

Their analysis rejects the ancestral structure model and supports Neanderthal admixture in Eurasia with parameters consistent with previous studies. They did estimate a higher percentage of admixture than other approaches, but Lohse says this parameter is highly sensitive to some of the model assumptions. For example, this study (and previous studies) assumed the population size was equal between groups and constant throughout time. But new evidence from a higher quality Neanderthal genome suggests the Neanderthal population was smaller than the human population. This means the model used by Lohse and Frantz will slightly overestimate the admixture rate.

But Lohse thinks the new method is an exciting development for people studying ancient or precious samples. It also demonstrates how much subtle population history can be extracted from just a few genomes.

Lohse K. (2014). Neandertal Admixture in Eurasia Confirmed by Maximum-Likelihood Analysis of Three Genomes, Genetics, 196 (4) 1241-1251. DOI:

Lohse K. & N. H. Barton (2011). A General Method for Calculating Likelihoods Under the Coalescent Process, Genetics, 189 (3) 977-987. DOI:

Cristy Gelling is a science writer, lapsed yeast geneticist, and former Communications Director at the GSA.

View all posts by Cristy Gelling »