Mean and variance of FST in a finite number of incompletely isolated populations

Masatoshi Nei, Aravinda Chakravarti, Yoshio Tateno

Research output: Contribution to journalArticlepeer-review


In the presence of migration FST in a finite number of incompletely isolated populations first increases, but after reaching a certain maximum value, it starts to decline and eventually becomes 0. The mean and variance of FST in this process are studied by using the recurrence formulas for the moments of gene frequencies in the island model of finite size as well as by using Monte Carlo simulation. The mean and variance in the early generations can be predicted by the approximate formulas developed. On the other hand, if we exclude the cases of an allele being fixed in all subpopulations, the mean of FST eventually reaches a steady-state value. This value is given by 1 - 2NT(1 - λ) approximately, where NT is the total population size and λ is the rate of decay of heterozygosity at steady state. It is shown that the mean and variance of FST depend on the initial gene frequency and when this is close to 0 or 1, Lewontin and Krakauer's test of the neutrality of polymorphic genes is not valid.

Original languageEnglish (US)
Pages (from-to)291-306
Number of pages16
JournalTheoretical Population Biology
Issue number3
StatePublished - Jun 1977

ASJC Scopus subject areas

  • Ecology, Evolution, Behavior and Systematics

Fingerprint Dive into the research topics of 'Mean and variance of F<sub>ST</sub> in a finite number of incompletely isolated populations'. Together they form a unique fingerprint.

Cite this