23
Copyright 0 1983 by the Genetics Society of America MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION MASATOSHI NEI,* TAKE0 MARUYAMA** AND CHUNG-I WU* *Center for Demographic and Population Genetics, The University of Texas at Houston, Houston, Texas 77025, and ** National Institute of Genetics, Mishima, Japan Manuscript received March 29, 1982 Revised copy accepted November 26,1982 ABSTRACT Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating isolation it is assumed that hybrid sterility or inviability is caused by incompatibility of alleles at one or two loci, and evolution of reproductive isolation occurs by random fixation of different incompatibility alleles in different populations. Mutations are assumed to occur following either the stepwise mutation model or the infinite-allele model. Computer simulations by using 1~6’~ stochastic differential equations have shown that in the model used the reproductive isolation mech- anism evolves faster in small populations than in large populations when the mutation rate remains the same. In populations of a given size it evolves faster when the number of loci involved is large than when this is small. In general, however, evolution of isolation mechanisms is a very slow process, and it would take thousands to millions of generations if the mutation rate is of the order of per generation. Since gene substitution occurs as a stochastic process, the time required for the establishment of reproductive isolation has a large vari- ance. Although the average time of evolution of isolation mechanisms is very long, substitution of incompatibility genes in a population occurs rather quickly once it starts. The intrapopulational fertility or viability is always very high. In the model of premating isolation it is assumed that mating preference or compatibility is determined by male- and female-limited characters, each of which is controlled by a single locus with multiple alleles, and mating occurs only when the male and female characters are compatible with each other. Computer simulations have shown that the dynamics of evolution of premating isolation mechanism is very similar to that of postmating isolation mechanism, and the mean and variance of the time required for establishment of premating isolation are very large. Theoretical predictions obtained from the present study about the speed of evolution of reproductive isolation are consistent with empirical data available from vertebrate organisms. HE crux of speciation is the development of reproductive isolation between T populations. Reproductive isolation may be attained by either premating or postmating isolation mechanism. There are many different ways of developing premating or postmating isolation mechanisms (DOBZHANSKY 1970; MAYR 1970), but the actual process of development is not well understood. The classical view of the development of postmating isolation such as hybrid sterility and inviability is that it occurs by fixation of a special set of interacting genes or incompatibility genes in different populations (DOBZHANSKY 1937; Genetics 103 557-579 March, 1983.

MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

Copyright 0 1983 by the Genetics Society of America

MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION

MASATOSHI NEI,* TAKE0 MARUYAMA** AND CHUNG-I WU*

*Center for Demographic and Population Genetics, The University of Texas at Houston, Houston, Texas 77025, and * * National Institute of Genetics, Mishima, Japan

Manuscript received March 29, 1982 Revised copy accepted November 26,1982

ABSTRACT

Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating isolation it is assumed that hybrid sterility or inviability is caused by incompatibility of alleles at one or two loci, and evolution of reproductive isolation occurs by random fixation of different incompatibility alleles in different populations. Mutations are assumed to occur following either the stepwise mutation model or the infinite-allele model. Computer simulations by using 1 ~ 6 ’ ~ stochastic differential equations have shown that in the model used the reproductive isolation mech- anism evolves faster in small populations than in large populations when the mutation rate remains the same. In populations of a given size it evolves faster when the number of loci involved is large than when this is small. In general, however, evolution of isolation mechanisms is a very slow process, and it would take thousands to millions of generations if the mutation rate is of the order of

per generation. Since gene substitution occurs as a stochastic process, the time required for the establishment of reproductive isolation has a large vari- ance. Although the average time of evolution of isolation mechanisms is very long, substitution of incompatibility genes in a population occurs rather quickly once it starts. The intrapopulational fertility or viability is always very high. In the model of premating isolation it is assumed that mating preference or compatibility is determined by male- and female-limited characters, each of which is controlled by a single locus with multiple alleles, and mating occurs only when the male and female characters are compatible with each other. Computer simulations have shown that the dynamics of evolution of premating isolation mechanism is very similar to that of postmating isolation mechanism, and the mean and variance of the time required for establishment of premating isolation are very large. Theoretical predictions obtained from the present study about the speed of evolution of reproductive isolation are consistent with empirical data available from vertebrate organisms.

HE crux of speciation is the development of reproductive isolation between T populations. Reproductive isolation may be attained by either premating or postmating isolation mechanism. There are many different ways of developing premating or postmating isolation mechanisms (DOBZHANSKY 1970; MAYR 1970), but the actual process of development is not well understood.

The classical view of the development of postmating isolation such as hybrid sterility and inviability is that it occurs by fixation of a special set of interacting genes or incompatibility genes in different populations (DOBZHANSKY 1937;

Genetics 103 557-579 March, 1983.

Page 2: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

558 M. NEI, T. MARUYAMA AND C.-I. WU

MULLER 1942). Because of their special gene interactions, these mutations are not supposed to reduce the fertility of the individuals within populations. DOBZHANSKY (1937) and MULLER (1942) suggested that fixation of such incom- patibility genes is possible if they affect some other characters that increase the fitness of their carrier. In practice, however, there is little evidence that a sterility barrier develops through this pleiotropic effect (see, however, NEVO 1969). Recently, NEI (1975, 1976) suggested that such incompatibility genes can be fixed in relatively small populations by random genetic drift without any pleiotropic effect and presented a mathematical model. He also indicated the possibility that ethological isolation-one form of premating isolation-evolves by random fixation of genes controlling ethological isolation mechanisms.

In recent years a number of authors (e.g.,WILsoN, SARICH and MAXSON 1974; BUSH et al. 1977; WHITE 1978) have suggested that chromosomal mutations play a major role in the development of postmating isolation. In this view hybrid sterility is caused mainly by the unbalanced gene content in gametes produced at meiosis. In some plants such as Oenothera the sterility barrier between species is apparently originated by fixation of translocation chromosomes, each of which reduces fertility by 50%. It should be noted, however, that in random mating populations translocation chromosomes cannot be fixed easily unless the population size is extremely small (WRIGHT 1941). Some other chromosomal mutations such as inversions seem to reduce the fertility of heterozygotes to a small extent, and these mutations may be fixed with a higher probability (e.g., BENCTSSON and BODMER 1976; BUSH et al. 1977; HEDRICK 1981). In this case, however, many chromosomal mutations must be fixed before a sufficient sterility barrier is established. Furthermore, there are several other problems in the theory of evolution of reproductive isolation by chromosomal mutations, as will be discussed later. It is, therefore, unlikely that chromosomal mutations are as important as gene mutations in the development of reproductive isolation.

The purpose of this paper is to present mathematical models of the evolution of reproductive isolation and to examine the factors that determine the speed of the evolution. Our basic idea is similar to NEI’S (1976), but unlike his paper, we consider both mutation and fixation of genes that determine reproductive isolation. We shall investigate both postmating and premating isolation mech- anisms.

THE MODELS

Postmating isolation There is much experimental support for the classical view of evolution of

postmating reproductive isolation. For example, OKA and his associates (e.g., OKA 1957, 1974; CHU and OKA 1970; SANO, CHU and OKA 1979) have conducted an extensive study about the genetic basis of hybrid inviability and sterility in rice and its related species and discovered many complementary genes that control pollen fertility and zygote viability in hybrids (see OKA 1978 for a brief summary of his work). In the case of Oryza perennis (wild rice) the F1 inviability between subspecies barthii and its relatives is controlled by a set of two

Page 3: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

REPRODUCTIVE ISOLATION 559

complementary dominant lethals, D1 and Dz, which interrupt cell differentiation in the early stage of development. Similar genetic systems have been discovered for F1 weakness in crosses between 0. sativa (rice) subspecies (OKA 1957), between Triticum (wheat) species (TSUNEWAKI and KIHARA 1962) and between barley varieties (TAKAHASHI, HAYASHI and MORIYA 1976), although the number of loci involved is not always two. In the hybrids between Gossypium hirsutum and G. barbadense (cottons) the female sterility in corky plants is controlled by disharmonious interaction (incompatibility) of alleles at a locus (STEPHENS 1946). OKA (1978) lists many other examples of both F1 hybrid inviability and infertility in plants. Similar examples from animals are also available. PRAKASH (1972) has shown that the male sterility of the hybrids between United States strains and Bogota strains of Drosophila pseudoobscura can be explained by incompatibility genes at four loci. TAUBER, TAUBER and NICHOLS (1977) have also reported that the temporal reproductive isolation between two species of lacewings is con- trolled by alleles at two independent loci. WATANABE (1979) recently discovered a gene that rescues the hybrid inviability between D. melanogaster and D. simulans. In some Drosophila species the number of incompatibility genes involved in hybrid inviability or sterility seems to be fairly large (DOBZHANSKY 1970; KILLAS and ALAHIOTIS 1982).

These examples suggest that hybrid sterility or inviability is controlled by some special genes which are dysfunctional (incompatible) for the development of hybrids but perfectly functional in individuals within species. NEI (1975) has discussed one possible scheme of this incompatible gene function in hybrids at the enzyme level. Our problem in this paper is to develop mathematical models that explain how different sets of incompatibility genes are fixed in two different populations when they are isolated. Although a number of authors have recently discussed the possibility of sympatric speciation (e.g., BUSH 1975; WHITE 1978), we shall restrict ourselves to allopatric speciation in this paper.

At this point we note that, although two or more loci are involved in most of the examples mentioned, this is not essential for hybrid sterility (or inviability) to be developed. Namely, even with a single locus, hybrid sterility may be developed if the alleles in the two populations are sufficiently different in function, as in the case of the corky gene in cotton. Mathematically, the one- locus model is much simpler to work with than the multilocus model. Therefore, we first study this model and then examine the two-locus model.

One-locus model: In this model we assume that there is an underlying character related to hybrid sterility or inviability, and a series of alleles control this character. Namely, allele A, has a phenotypic effect (ia) proportional to i in the scale of this character (Figure 1). We also assume that homozygote A A and heterozygotes A,A, + 1 and A, - IA, are completely fertile, but the heterozygotes for alleles two or more steps apart ( A A + 2 , A A + 3, etc.) are infertile (Table 1). One may criticize this assumption as too simple and suggest that the fitness of A,A, should be given as (1 - s)lz - J I - l , where 0 < s < 1 and I i - j I > 1. However, the latter assumption does not seem to give any deeper insight into our problem, so we use the simpler mode. (We note that, although we are considering postmating isolation here, the present model is applicable also to such characters

Page 4: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

560 M. NEI, T. MARUYAMA AND C.-I. WU

Allele

-2a -a 0 a 2a Phenotypic effect

FIGURE 1.-Stepwise mutation model for hybrid sterility (or inviability) genes. Allele A, is assumed to have phenotypic effect io on the character that determines hybrid sterility.

TABLE 1

Fertilities or viabilities for various genotypes in the stepwise mutation model. Here only five alleles are shown

A--2 A- I Ao A, A2 . A-2 . 1 1 0 0 0 A-, . 1 1 1 0 0 Ao . 0 1 1 1 0 . A, . 0 0 1 1 1 . A2 0 0 0 1 1

as flowering time in plants and developmental time in animals, which generate important premating isolation mechanisms; PATERNIANI 1969; TAUBER, TAUBER and NICHOLS 1977). There are several ways to incorporate mutation in this model. We assume that A, mutates to A, - 1 and A, + each with probability v/2. This is analogous to OHTA and KIMURA’S (1973) stepwise mutation model for electrophoretic variation of proteins. As will be shown later, however, speciation proceeds rather slowly with this model. We have, therefore, introduced another model called the infinite allele model, which is again similar to KIMURA and CROW’S (1964) model for protein variation. In this model we assume that the genic effect is no longer one-dimensional as in the case of the stepwise mutation model but mutation occurs in many (theoretically infinite) directions without backward mutation and that all homozygotes ( A A ) and heterozygotes for an allele (A,) and its immediate parental (A, - 1) or descendant (A, + 1 ) allele are completely fertile, but other genotypes are all infertile.

In both the stepwise mutation and infinite allele models turnover of alleles would not occur in a large population, since new mutations that are two or more steps apart produce inviable or infertile heterozygotes, and thus the same allele, or the same pair of alleles, that originally existed in the population tends to stay forever. In a relatively small population, however, any allele may become predominant, and a new mutant allele which is derived from the predominant allele may be fixed in the population by random genetic drift without reducing the intraspecific viability or fertility. If this process continues independently in a pair of populations that have originated from the same stock, the two populations tend to accumulate different alleles and eventually will develop a sterility barrier. Namely, in our model, reproductive isolation occurs only in relatively small populations.

Two-locus model: To see the effect of the number of loci involved in a given incompatibility system, we also studied the two-locus model. In this case we

Page 5: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

REPRODIJCTIVE ISOLATION 561

used the stepwise mutation model for each of the two loci A and B and assumed that hybrid sterility occurs in the same way as in the case of the one-locus model for each of the two loci (Figure 2). Mutation was also assumed to occur in the same way as that for the one-locus model.

Premating isolation There are three different hypotheses about the evolution of premating (eth-

ological) isolation, if we exclude the sympatric speciation scheme (adaptive model; BUSH 1982). FISHER (1930) and DOBZHANSKY (1940) proposed that pre- mating isolation is developed by natural selection when two populations that have already developed postmating reproductive isolation become sympatric, whereas MULLER (1942) postulated that premating isolation is a by-product of genetic divergence that occurs when two allopatric populations adapt to differ- ent environments. The first hypothesis seems to have been originally suggested by ALFRED WALLACE. Extending MULLER’S hypothesis, NEI (1976) recently pro- posed that premating isolation such as ethological isolation occurs mainly by random fixation of genes that control mating preference or compatibility be- tween males and females. A large number of experiments have been done for testing WALLACE’S hypothesis, but the results obtained are conflicting (see SVED 1981a). Recently, SAWYER and HARTL (1981) and SVED (1981a,b) studied this problem mathematically, showing that under certain circumstances premating isolation may evolve in this fashion.

There is a large amount of data indicating that premating isolation evolved in allopatric populations without having postmating isolation mechanisms (MAYR 1963; KANESHIRO 1980). In these cases, however, it is not clear whether this isolation evolved through MULLER’S scheme (pleiotropic effect) or NEI’S scheme (mutation-drift model), but there are a number of examples in which mating incompatibility is involved. For example, in Drosophila the compatibility of male and female genitalia seems to be an important isolation mechanism, because genitalia are species specific and there is great interspecific variation in the morphology in both males and females. Of course, this mechanism of isolation would not be as rigid as that assumed in the old lock and key hypothesis. It is also known that females in some Drosophila species accept only males showing a certain range of wing-beat speeds at the time of mating.

AI-2BJ-2 A,-lB,-2 * I Bl-2 Al+l8,-2 A,+2BJ-2 FIGURE 2.-Chromosome types that are compatible with A,B,. Diploid zygotes that consist of

compatible chromosomes are assumed to be viable or fertile, whereas all others are inviable or infertile.

Page 6: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

562 M. NEI, T. MARUYAMA AND C.-I. WU

In Drosophila the male and female mating behaviors seem to be controlled by separate sets of genes (TAN 1946; EWING 1969; ZOUROS 1981). In this paper we consider an extended form of NEI'S model of premating isolation, which closely parallels our scheme of postmating isolation.

We assume that loci A and B control male-limited and female-limited mor- phological, physiological or behavioral characters, respectively, and mutations at these loci change the characters stepwise as in the case of postmating isolation in Figure 1. In the present paper we consider a haploid model and assume that males having allele A, at locus A (A,BI, A,Bz, A,B3, etc.) mate only with females having alleles B, - 1, B,aand B, + 1 at locus B (AIS, ~ 1, A2B, - 1, . - - , AIB,, AZB,, e , AlB, + 1, AZB, + 1, etc.), whereas females having allele B, mate only with males having alleles A, - ], A, and A, + (Table 2). Individuals that partici- pate in incompatible matings are assumed to produce no offspring. We also assume that loci A and B are unlinked, so that when A,B, mates with AkB, +

four different genotypes A,BJ, A,B, + I , AkB, and AkB, + are produced each with probability 1/4. In this case we consider 2N haploid individuals with equal numbers of males and females.

In diploid organisms the actual phenotypic values of the mating-controlling character will be determined by the joint effect of the two alleles at the A or B locus. Our haploid model can be applied to diploid organisms, if we assume that the phenotypic value is given by the sum of the phenotypic effects of the two alleles. Thus, for example, male genotype AIAIB,B, will mate with female genotype AkAlBlBz with probability 1, whereas male AIAIB,B, will mate with female AkAlBZB3 with probability 1/2. When the number of alleles is large, however, our assumption is not always realistic. For example, according to our assumption, male AlAIB,B, will mate with female A ~ A I B ~ B ~ with probability 0 but will mate with female AkAlBlBs with probability 1/2, although the pheno- typic value in the A scale (male character) of AIAIB,BJ is closer to the phenotypic value in the B scale (female character) of AkAlB3B3 than to that of AkAIBIBB. In practice, however, the number of segregating alleles in a population is generally

TABLE 2

Mating abilities for some examples of the combinations of male and female genotypes Allele A, controls a male-limited chararter, whereas B, controls a

female-limited character

Frmale grnotypc Male

genotype A, ,Bt I A, IB , A, ,B,+, A$, I A,B, A,B,+, . A, IB, I 1 1 0 * * 1 1 0 * A, IB, 1 1 0 * * 1 1 0 *

At ~ B z t i 1 1 0 * * 1 1 O *

A,B, i 1 1 1 * * 1 1 3 - AB, 1 1 1 * . 1 1 1 *

A,B,+I 1 1 1 * * 1 3 1 *

. . . . . . . . . . . .

Page 7: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

REPRODUCTIVE ISOLATION 563

very small, as will be shown later, so that this situation will occur very rarely. Therefore, we believe that our haploid model is adequate for getting a rough idea about the dynamics of mating preference genes in diploid organisms. Theoretically, of course, it is possible to develop a diploid model in which the aforementioned deficiency is removed. However, in the presence of multiple alleles with new mutations the number of possible mating types in a population often becomes prohibitively large for a mathematical treatment. Therefore, we have not been able to get satisfactory results with the diploid model even using a fairly efficient computer.

Our genetic model of determination of mating preference has some similarity with SVED’S (1981a). He considered the male and female characters that are controlled by polygenes, but the gene action assumed is considerably different from ours. Furthermore, his model is for studying WALLACE’S scheme of evolu- tion of premating isolation.

MATHEMATICAL METHOD AND SIMULATION

As mentioned earlier, we assume that one population splits into two at an evolutionary time, and thereafter no migration occurs between them. We also assume that reproductive isolation occurs through random fixation of incom- patibility alleles in each population. We study the gene frequency changes in each descendant population by using the diffusion method. In practice, it is difficult to solve the diffusion equations analytically in the present case, so that we simulate the allele frequency changes using the method of ITB’s stochastic differential equation as adapted to population genetics problems by MARUYAMA (MARUYAMA and NEI 1981). Since the two populations accumulate different alleles by mutation and genetic drift, reproductive isolation gradually develops.

One-locus model: Let N and v be the effective population size and the mutation rate per locus per generation, respectively. In the case of the one-locus model of postmating isolation we denote by x , (T) the frequency of allele A, at time T . We renumber all alleles using only nonnegative integers to simplify the mathematical presentation. We also measure T in units of 2N generations, i.e., T = t / 2 N , where t stands for generation. Consider the case where n + 1 alleles Ao, AI , . . . , An are present in the population, and let W, be the viability or fertility of genotype AAJ. In our model W, = 1 if I i - j I i: 1; otherwise W , = 0. Under random mating the mean fitness is

n

fi = 2 x ~ ( T ) x J ( T ) w , J , v

whereas the mean fitness for allele A, is n

w,= 2 xJ(T)w, . J = o

According to the diffusion theory, the change in x~(T) in a short time interval d7 is given by

n

dxi(7) = 2 eijdBj + 2 N M ( x i ( T ) ) d T (1) j = 1

Page 8: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

564 M. NEI, T. MARUYAMA AND C.-I. W U

for i = 1, 2, . . . , n, where the B,’s are independent Brownian motion variables and e,’s are the elements of the positive definite square root of the drift matrix [x,(r)(& - x , ( T ) ) ] , in which S,, = 1 and S , = 0 if i # j (see MARUYAMA and NEI 1981). M(x,(T)) is the mean gene frequency change per generation. For the infinite allele model we use

M(x,(T)) = x ~ ( T ) ( W , - W)/W - VX,(T), (2)

following MARUYAMA and NEI (1981), whereas for the stepwise mutation model we use

M(x,(r)) = x,(T)(W, - W)/W. (3)

In both (2) and (3) no consideration has been made about new mutant alleles, and these will be introduced by the method that will be described.

Equation (1) is a commonly used form of representing the diffusion process, but there are many other ways of representation. In this study we use ITOH’S (1979) form, in which the square root matrix [e , ] is not involved. Furthermore, in actual computation we approximate (1) by the corresponding difference equation considering time interval of Ar rather than dr. The difference equation in ITOH’S (1979) form is

n

/ = 1 J # C

Ax,(T) = a(i, j ) ~ B , ( A T ) + 2NM(xl(r))AT, (4)

where a(i, j ) = 1 if i c j , a(i, j) = -1 if i > j , and Bll(Ar) for i c j is an independent random variable following the normal distribution with mean 0 and variance Ar, whereas B,,(AT) = B,(AT).

New mutations were introduced after selection and genetic drift operated. In the case of infinite allele model they were introduced in the same way as that of MARUYAMA and NEI (1981). Namely, in every time interval of AT the number of mutations to be introduced was determined by using the Poisson distribution with mean ZNVAT/E, where E is the initial frequency of a mutant allele to be introduced. e is a small quantity but larger than 1/2N or Ar. In the present case we used E = 0.01, 0.005 or 0.002, depending on the value of 2Nv. All new mutations were assumed to be different from the extant alleles. In the case of stepwise mutation model we first noted that the number of A, alleles existing in the population is 2Ny,(~) , where y , ( ~ ) = x,(T) + Ax,(r). This pool of alleles was assumed to produce 2Ny,(T)vAr/E mutations, on the average, during Ar. We again used the Poisson distribution to determine the number of mutations to be introduced for each AT. Once a mutation occurred, A, was changed to either A, - 1 or A, + 1 with probability 1/2. xl(T + AT) was then obtained by Y,(T) + Ay,(.), where A Y ~ ( T ) is the change in the frequency of A, due to mutation. (In the infinite allele model xl(T + Ar) was obtained by x , ( T ) + Ax,(r) for preexisting alleles; see MARUYAMA and NEI 1981.) Although the initial frequency of a new mutant allele was set to be E, its frequency in the subsequent generations was followed even if they were smaller than E. Allele A, was considered to be lost from the population only when x l ( T ) became 0 or negative. In the formulation

Page 9: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

REPRODUCTIVE ISOLATION 565

we have negle.cted X O ( T ) , but this can be obtained by 1 - ~ ~ ( 7 ) - ~ ~ ( 7 ) . . . - xn + k(r), where k is the number of new mutant alleles introduced.

It is clear that the change of x , ( T ) in a population (sample path) can be followed by applying (4) repeatedly and introducing new mutations in the scheme previously mentioned. Since B,(AT) can easily be generated by com- puter, this method greatly facilitates the computation of x,(T). The accuracy of the simulation largely depends on the values of E and AT. We used A r = 0.0005 - 0.005 in the present case.

In our model we must specify two parameters, i.e., N and v. We used various values of N from lo2 to lo6 and various values of mutation rates ranging from

to low3. In most computations v = lou5 was used. Starting from two identical populations, which were monomorphic initially, we followed the types and frequencies of alleles in each population for a specified number of generations. The interpopulational and intrapopulational fertilities (viabilities) were then examined periodically. This was repeated 100 times or more for each parameter set of N and v. The interpopulational fertility was measured by the average fertility of hybrids between two populations. The intrapopulational fertility was measured by the average fertility of individuals within the same populations.

One might wonder whether our diffusion approximations of allele frequency changes are sufficiently accurate or not, because W, = 0 for I i - j I > 1 and theoretically the selection terms x,(W, - W)/W can be much larger than the drift terms in (1). In practice, however, the frequencies of incompatible alleles are always kept in low frequency within a population, so that is almost always very close to 1 (see Table 3). Therefore, the selection terms never become excessively large compared with the drift terms. This situation is analogous to the case of recessive lethal genes, where the diffusion approximation and the discrete generation treatment give essentially the same result about the dynam-

TABLE 3

Representative examples of allele frequencies within populations for the case of one-locus stepwise mutation model

Allele frequencies

Example A-2 A-I A0 A1 A2

1 2 3 4 5 6 7 8 9 10

0 0 0 0 0 0 0 0 0 0

0.0026 0.0033 0 0 0.1430 0.0039 0.0419 0 0.1820 0.0014

0.9947 0.9967 0.9106 0.9896 0.8570 0.9807 0.9581 0.6933 0.8180 0.9766

0.0027 0 0.0894 0.0104 0 0.0154 0 0.3067 0 0.020

0 0 0 0 0 0 0 0 0 0

~~

The most frequent allele is denoted by Ao. These results were obtained by printing the allele frequencies every 4N generations after the population reached the equilibrium status. 2Nv = 0.5 and N = 5000 were used.

Page 10: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

566 M. NEI, T. MARUYAMA AND C.-I. W U

ics of gene frequency changes (e.g., ROBERTSON and NARAIN 1971). This justifies the use of the diffusion method in the present case. Furthermore, our results of Monte Carlo simulations for the case of N = 100 and 2Nv = 1 have indicated that our computations in terms of (4) are sufficiently accurate for our purpose. (The results are not shown here.)

Two-locus model: For the complete treatment of a two-locus problem, we must consider gamete frequency changes rather than gene frequency changes, since in the presence of linkage disequilibrium the genetic change of populations cannot be completely specified by gene frequencies. However, the mathematical treatment of gamete frequency changes is much more cumbersome than that of gene frequencies, pa2ticularly when multiple alleles exist at each locus and mutation is allowed to occur. This is because the number of gamete types rapidly increases with increasing number of alleles, and in the presence of mutation we must specify the initial linkage disequilibria for new mutant alleles. We have, therefore, decided to assume free recombination and linkage equilib- rium between the loci concerned and compute gamete frequencies by the products of gene frequencies. In the present case, of course, linkage equilibrium does not hold even for the case of free recombination, but the effect of this assumption on the final result seems to be relatively small, since we are concerned with the process of gene substitution rather than the equilibrium gamete frequencies under certain types of balancing selection.

Let x, and y, be the frequencies of allele A, at locus A and allele B, at locus B, respectively. Here we have dropped argument T for simplicity. In the case of postmating isolation, we denote the fertility or viability of genotype A,B3/AkBr by WCIklr which takes a value of 1 or 0 according to the definition given in Figure 2. The frequency of this genotype is given by X$&kyl. Therefore, the mean fitness (Wc) of allele A, and the mean fitness (I&') of the entire population are given by

w l = yjxkylWyk1, (5) 3 h J

w = x,w,. (6)

The mean of gene frequency change for allele A, is then given by (3). Similarly, the mean of gene frequency change for allele B, can be obtained by an equivalent formula. New mutations can also be incorporated by the same method as that described earlier. The mutation rate and population size used were the same as those for the one-locus model.

In the case of premating isolation, we used a two-locus model, as mentioned earlier. In this case, however, we considered a haploid scheme and assumed that mating preference is determined by locus A in males and locus B in females. Therefore, selection at locus A occurs only in males, whereas selection at locus B occurs only in females. We also consider the mating-type fitness rather than the usual fitness. Thus, for example, the fitness (W13kl) for mating ALB,(d) x A&(?) is 1 and that (Wl,kl+2) for A1BJ(d) X AkB,+2(?) is O by the definition given earlier. W, and W can, therefore, be written as in (5) and (6). However, the first term in (2) or (3) is given by xc(WL - W)/(ZW), assuming that the sex ratio is 1:1.

1

Page 11: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

REPRODUCTIVE ISOLATION 567

All other computations are the same as those for the two-locus model of postmating isolation, including the assumption of linkage equilibrium between the two loci. We note that the selection scheme in the present model is somewhat different from that of NEI (1976), where all females were assumed to be fertile. We have not used the latter scheme, because this requires a larger amount of computer time in the presence of multiple alleles and mutations.

RESULTS

Postmating isolation One-locus model: In our model mutation occurs in every generation with a

certain probability, so that even in the same population sterile or inviable individuals may appear. In practice, however, the frequency of such individuals is very low because of strong selection against incompatible alleles within populations. This can be seen from Table 3, which shows ten examples of allele frequencies in a population at various evolutionary times. In this table the most frequent allele is denoted by A,. It is clear that most of the times there is a predominant allele in the population, and its neighboring alleles exist in very low frequency. Furthermore, the event where more than three alleles exist is extremely rare. Therefore, the proportion of sterile individuals is generally very low. For example, the proportion of sterile individuals in example 1 of Table 3 is only 2 X 0.0026 X 0.0027 = 7 X Of course, occasionally two alleles have a high frequency in the process of gene substitution. In this case, however, a third allele is virtually nonexistent as in the case of example 8, so that the intra- populational fertility or mean fitness is almost 1.

The distributions of interpopulational fertility (average hybrid fertility or viability) for the stepwise mutation model with v = and N = 5000 (2Nv = 0.1) are given in Figure 3. As expected, it takes a long time for any pair of populations to develop hybrid sterility or inviability. Even when vt = 1, i.e., at t = IO5, there are only 5% of the cases where interpopulational fertility ( f ) is 0. In this case the distribution of f is nearly inverse L-shaped, and most pairs of populations still show a fertility higher than 98%. As generations proceed, the frequency of f = 0 gradually increases, and when vt = 5, i.e., t = 5 x IO5, this frequency is about 20%. When vt = 10 (t = lo6), the distribution is nearly U- shaped, and in about 52% of the cases f = 0. With further increase of vt the frequency of f = 0 still increases, but the increase is very slow, and when vt = 20 ( t = 2 x lo6), the frequency is about 70%. It is noted that in most cases the interpopulational fertility is either 1 or 0, but there are a few percentage of the cases where f is intermediate. These of course represent the cases in which different hybrid sterility alleles are being fixed in the two populations con- cerned. The small proportion of intermediate fertilities indicates that, once incompatibility alleles start to be fixed, the fixation occurs relatively rapidly. The pattern of the temporal change of the distribution of interpopulational fertility is nearly the same for all values of 2Nv and N as long as gene substitution takes place, but the change is slower when 2Nv and N are large than when these are small.

Figure 4 shows the relationship between the average fertility over all repli-

Page 12: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

568 M. NEI, T. MARUYAMA AND C.-I. WU

1 - - v t = 1

.5 -

v t = 5

J U 0 1 b

Y

% U

.5

-

0

O F -

I 4 - I 4

-

F e r t i l i t y

FIGURE 3.-Distributions of interpopulational fertilities at various evolutionary times. N = 5000 and v = are used. The number of replications used is 100.

v t : 10 v t = 20

1 I

5 10 15 20 vt

FIGURE 4.-Relationships between the average fertility over all replications (fi and evolutionary time (vt) for various values of ZNv, where v = is used. The stepwise mutation model is used. The number of replications used is 100. The symbols U, A, 0,. and W represent the cases of N = 5 X lo", 2.5 X lo4, 5 X lo3 and 500, respectively. The solid line shows the expected lowest fertility.

- - I 4 1

cations and evolutionary time for various values of N when v = is fixed. Since the interpopulational fertility for each pair of populations is 0 or 1 in most cases, the average fertility ( f ) in this figure can be regarded approximately as the proportion of pairs of populations in which the sterility barrier has not been established. Figure 4 shows that the speed of evolution of hybrid sterility depends heavily on the value of N. When N is greater than 5 x lo4, virtually no

Page 13: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

REPRODUCTIVE ISOLATION 569

differentiation of alleles occurs between populations, so that the populations remain interfertile even when vt is as large as 20 (t = 2 x lo6). The probability of developing a sterility barrier increases as N decreases. When N = 2.5 x lo4, this probability is appreciable but the evolution is still slow. A rapid evolution of a sterility barrier occurs when 2Nv is smaller than 0.1. There is, however, an upper limit for the speed of evolution of a sterility barrier. It is given by the solid line in Figure 4. This line is for neutral alleles which are subject to no selection. Of course, neutral alleles do not produce a sterility barrier, but the hypothetical fertility can be computed by assuming that the heterozygotes for alleles which are two or more steps apart in phenotypic scale in Figure 1 are sterile. This hypothetical fertility is given by

(MARUYAMA 1977). Figure 4 shows that when N = 500 (Nv = 0.01) the dynamics of hybrid sterility genes is virtually the same as that of neutral alleles. This is because when 2Nv is small the number of alleles segregating in a population is generally two, so that the gene frequency change is determined almost exclu- sively by genetic drift.

It is noted that when 2Nv is as small as 0.01, the probability of establishment of hybrid sterility increases rather rapidly in the early generation, but the rate of increase of the probability gradually declines as vt increases. When vt is as large as 10, the increase of the probability is very slow. The reason for this is that in the stepwise mutation model backward mutation occurs and this slows down the speed of evolution of a sterility barrier. It should also be noted that in the present model the sterility barrier developed between a pair of populations may later disappear by backward mutation with a small probability. This probability is nil in the case of the infinite allele model.

When the infinite allele model is used, the dynamics of evolution of hybrid sterility when 2Nv is large are virtually the same as that for the stepwise mutation model (Figure 5). Thus, when 2Nv > 0.5, little differentiation of

O O 0

0 5 10 15 20 vt

FIGURE 5.-Replications between the average fertility over all replications (fj and evolutionary time (vt) for the infinite allele model when v = is used. The number of replications is 100. The symbols 0, A, 0, 0 and represent the cases'of N = 5 x IO4, 2.5 X lo4, IO4, 5 X lo3 and 500, respectively. The solid line shows the expected lowest fertility.

Page 14: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

570 M. NEI, T. MARUYAMA AND C.-I. WU

incompatibility alleles occurs between isolated populations, and the probability of establishment of a sterility barrier increases as 2Nv decreases. However, the upper limit of the probability of developing hybrid sterility for a given evolu- tionary time is substantially higher in this model than in the stepwise mutation model. This upper limit can again be computed from the hypothetical fertility for neutral alleles, which is given by

f = (I + 2vt)e-2"t. (8)

This expression can be obtained from LI'S (1977) theory of genetic differentiation of alleles between two populations. Compared with the case of the stepwise mutation model, the fertility given by this formula declines rather rapidly as vt increases, and it becomes almost 0 even with vt = 4. This is, of course, due to the fact that there is no backward mutation in this model. Clearly, the speed of evolution of the sterility barrier depends on the gene action of the incompati- bility genes involved.

In these studies we fixed the mutation rate and varied the population size to see the effect of N. As mentioned earlier, the speed of evolution of hybrid sterility also depends on the mutation rate. If v increases for a given value of N, the speed of evolution of hybrid sterility also increases but not in proportion to v. For example, in the case of N = IO4 and v = IOp5, it takes about 1.5 X lo6 generations for the average interpopulational fertility to become about 50% (see Figure 4). This is 5.8 times slower than that for the case of neutral genes (2.6 x lo5 generations). (This value is obtainable from (7).) If we increase the mutation rate ten times (v = decreases ten times for neutral genes (2.6 X lo4 generations). However, our computer simulations have shown that the t0.5 value for incompatibility genes is about 6 x lo5 generations. This is about 20 times slower than that for the case of neutral genes. In the case of N = lo4 and v = (2Nv = 20) we have obtained to5 = 5 X lo5 generations, whereas the t0.5 for neutral genes is 2.6 X lo3 generations, the ratio of the former to the latter being 192.3. This indicates that if 2Nv becomes larger than 2 the effect of increase of mutation rate on is small.

In the case of the infinite allele model, we have not studied the effect of mutation rate, but this is expected to be qualitatively similar to that for the stepwise mutation model.

Two-locus model: The relationship between f and vt for the two-locus model for the case of v = is given in Figure 6. Comparison of this figure with figure 4 shows that the speed of development of hybrid sterility is considerably faster in the two-locus model (two character model) than in the one-locus model. In the case of N = 5000 the average fertility for vt = 5 is about 0.4 for the two-locus model but 0.6 for the one-locus model. Namely, the former is approx- imately the square of the latter fertility. This property approximately holds for all values of N and v examined.

These results suggest that, if there are n independent loci that control various characters related to hybrid sterility (or inviability), the average interpopula- tional fertility is approximately given by f " , where f is the average fertility for

2Nv = 2), the time required for 50% fertility

Page 15: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

REPRODUCTIVE ISOLATION 571

LL ;E I 8 8 : ; : : : : i o o 0 0 0 0 0 , 0 0 0 , a a a

a * o a a a a .

OO 5 10 15 20 v t

FIGURE 6.-Relationships between the average fertility over all replications ( f i and evolutionary time (vt) for the two-locus stepwise mutation model when v = is used. The number of replications is 100. The symbols 0, A, 0, and represent the cases of N = 5 x IO4, 2.5 x IO4, IO4, 5 X lo3 and 500, respectively.

the one-locus model. Namely, the larger the number of loci involved, the faster the development of hybrid sterility.

Premating isolation The extent of premating isolation can be measured by the proportion (P) of

random pairs of individuals that can mate with each other between two populations. Mathematically, this is identical with the interpopulational fertility. In the present case each of the two loci shows its phenotypic effect only in one sex, but otherwise the gene action is the same as that for the two-locus model considered before. Because of the sex-limited gene expression, the effect of selection for the genes is expected to be smaller in this case than in the latter. Our computer simulations have shown that the temporal change of the distri- bution of P is similar to that of the case of postmating isolation, although the speed of the change depends on v and N. The number of segregating alleles within populations was also generally very small. Figure 7 shows the relation- ship between the average proportion of compatible mating and evolutionary time (vt) for the case of v = lod5. It is clear that the general pattern of the relationship is somewhat similar to that for the one-locus model of postmating isolation in Figure 4. In this case, however, even with N = 50,000 the probability of development of premating isolation is appreciable. This is probably due to the fact that selection for a locus occurs only in one sex, and, thus, there is more room for genetic drift to operate. On the other hand, when N = 500, the speed of evolution of premating isolation is somewhat slower than that of hybrid sterility.

In the case of premating isolation we did not study the infinite allele model, but from the study of postmating isolation we would expect that this model gives a faster evolution of premating isolation. We would also expect that the speed of evolution of premating isolation increases as the number of loci involved increases. Namely, the evolution of premating isolation is qualitatively similar to that of postmating isolation in our models.

Effects of migration In these studies we assumed that the two populations in question are geo-

Page 16: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

572 M. NEI, T. MARUYAMA AND C.-I. WU

I * I

5 10 15 20 vt

FIGURE 7.-Relationships between the average proportion of compatible matings over all repli- cations and evolutionary time (vt) when v = is used. The number of replications is 100. The symbols U, A, 0, 0 and represent the cases of N = 5 x IO4, 2.5 x lo4, lo4, 5 x lo3 and 500, respectively.

graphically or ecologically isolated and there is no migration between them. In nature, however, this may not necessarily be the case. Therefore, we have studied the effect of migration by using the one-locus stepwise mutation model of postmating isolation. In this case we used v = and N = 5000, considering various values of 2Nm, where m denotes the rate of gene exchange between the two populations. The results obtained are given in Figure 8. It is interesting to see that even a small amount of gene migration retards the evolution of reproductive isolation considerably. Namely, when one gamete is exchanged every generation between two populations, i.e., 2Nm = 1, virtually no differen- tiation of incompatibility alleles occurs between the two populations. Only when 2" is smaller than 0.25 does appreciable gene differentiation occur.

Of course, this conclusion will change if there are more than one loci contributing to hybrid sterility. As shown earlier, hybrid sterility is established more easily when many loci are involved. In this case the effect of migration is expected to be smaller, for the fertility for n loci would again be given by T" approximately. Nevertheless, in the presence of migration with 2Nm >> 1, evolution of hybrid sterility would not occur easily at least in the present model.

DISCUSSION

In the literature the word speciation has been used with many different meanings (see MAYR 1963; GRANT 1971; PATERSON 1981), but the most common usage is for the process of splitting of one species into two or more species with the development of isolation mechanisms. Some authors (e.g., MAYR 1963; CARSON 1971) have hypothesized that speciation occurs as a consequence of adaptation to new environments either through preadaptation of genes or through postadaptation, and for this to occur a drastic change in the genome called genetic revolution is necessary. As mentioned earlier, however, the number of loci involved in the determination of reproductive isolation is not necessarily large, and the loci may not be directly related to adaptation (NEI 1975, 1976). The models of reproductive isolation considered here are based on this idea. Namely, in our models speciation or reproductive isolation occurs as

Page 17: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

REPRODUCTIVE ISOLATION 573

c

L I .. i

OO 5 10 vt

FIGURE 8.-Relationships between the average fertility over all replications (fi and evohtionary time in the presence of migration. N = 5000 and v = were used. The number of repiications is 100. The symbols 0, A, 0,. and represent the cases of 2Nm = 0.5,0.2,0.1,0.05 and 0, respectively.

a result of random fixation of incompatibility alleles in geographically isolated populations, and this process has nothing to do with adaptation. If two descend- ant species are different in adaptability, the difference is assumed to have occurred owing to genes different from the incompatibility genes. The incom- patibility genes we have considered are primarily concerned with the develop- ment of certain organs or characters that determine viability or mating ability. Of course, under certain circumstances, new mutations in these genes may increase or decrease the adaptability of the population to a certain environment. In this case the probability of fixation of the mutant genes may increase or decrease.

In the study of speciation such mystic words as genetic revolution and quantum speciation are still commonly used. In MAYR'S (1963) definition, how- ever, genetic revolution is simply a large genetic change of population during a short period of time that often occurs at the time of population bottleneck. It is concerned primarily with adaptation to a new environment and does not explain how reproductive isolation really occurs. CARSON'S (1971) flush-founder theory and TEMPLETON'S (1980) theory of genetic transilience are also essentially the same as MAYR'S genetic revolution, although they emphasize the importance of epistatic gene interaction and a smaller number of genes involved. As MAYR (1963, 1970) stressed, the essence of speciation is acquisition of isolation mech- anism, and to understand the speciation process we must study the genetic basis of isolation mechanism directly.

Recently, a number of authors (SAWYER and HARTL 1981; FELSENSTEIN 1981; SVED 1981a, b) have studied the evolution of reproductive isolation mathemati- cally. None of them, however, considered the entire process of completion of reproductive isolation. Our model is different from these in the sense that not only the fixation of incompatibility alleles in populations but also the effect of mutation can be studied. Furthermore, our model enables us to study the speed of evolution of reproductive isolation when mutation rate and population size are known. It should also be noted that our model is stochastic, so that the establishment of reproductive isolation during a given period of evolutionary time occurs only with a certain probability.

Page 18: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

574 M. NEI, T. MARUYAMA AND C.-I. W U

As mentioned earlier, our model is only a first approximation to nature, and it may be unrealistic in some respects. For example, we have not considered the effects of genes on the X chromosome, although in many bisexual organisms hybrid sterility or inviability often occurs only in one sex (HALDANE 1922), and in some species ethological isolation is controlled by genes on the X chromosome (TAN 1946; EWING 1969; KAWANISHI and WATANABE 1981). Furthermore, when many loci are involved, the actual scheme of gene interaction is likely to be more complicated than our model. It should also be noted that speciation is a complex process, and there are many possible ways of speciation other than the model considered here (see MAYR 1963; BUSH 1975).

Nevertheless, our model gives certain predictions about the evolution of reproductive isolation, which can be compared with available data. The follow- ing are some of these predictions.

1. The evolution of reproductive isolation occurs more easily in small popu- lations than in large populations. This prediction is in agreement with actual observations. MAYR (1970) examined the relationship between speciation and population size in many different groups of organisms and concluded that speciation occurs faster in small populations than in large populations. CARSON (1970) also argued that many species of Drosophila in Hawaii emerged through a small bottleneck or even from a single gravid female. MAYR and CARSON regarded these observations as supports for their theories, but it is possible that the primary cause is random fixation of incompatibility genes such as those we have considered.

2. Speciation is generally a very slow process. This is obviously true, since we cannot observe the speciation event in our lifetime except in very unusual circumstances. Examining many extant and extinct vertebrates, RENSCH (1959) has concluded that the species life of vertebrates is on the average 100,000- 2,000,000 years. MAYR (1963) also presented many examples of slow rates of evolution of reproductive isolation. Furthermore, WILSON, MAXSON and SAR- ICH’S (1974) and PRACER and WILSON’S (1975) studies on immunological distances between two species that produce viable hybrids suggest that the average time of divergence for these species is about 2 million years in mammals and about 20 million years in frogs and birds. In addition, WILSON et al.’s (1974a) Figure 2 indicates that the divergence time has a wide variation as expected from our Figures 4-7.

3. Speciation occurs faster when the number of loci involved is large than when this is small. We have no experimental data to support this principle directly, but WILSON et al.’s (1974a) and PRACER and WILSON’S (1975) data on the evolutionary times of hybridizable species in mammals, birds, and frogs can be explained by this principle. As mentioned before, they have estimated that the average time since divergence between two hybridizable species in mam- mals is about 2 million years, whereas in birds and frogs it is about 20 million years. It is possible that this difference is caused by the difference in the number of loci that are involved in the postmating reproductive process. This postmating reproductive process is much more complex in mammals than in frogs and birds, because fertilization occurs internally and fetuses develop inside the

Page 19: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

REPRODUCTIVE ISOLATION 575

uterus. It is, therefore, likely that the number of loci involved in the postmating reproductive system of mammals is larger than that of birds and frogs. We would than expect that the time of divergence required for the evolution of hybrid inviability is shorter in the former than in the latter. It is also possible that the difference between mammals and birds/frogs is due to the difference in effective population size between the two groups of organisms, since the effective size of mammalian species is apparently generally smaller than that of birds/frogs (BUSH et al. 1977). This possibility is also consistent with our model.

WILSON, MAXSON and SARICH (1974) attempted to explain the difference in evolutionary time of hybridizable species between mammals and birds/frogs by the possible difference in the rate of regulatory evolution. Later, finding that the average rate of change in chromosome number in mammals is about 20 times higher than that in frogs, WILSON, SARICH and MAXSON (1974) suggested that gene arrangement may be responsible for regulatory evolution, which in turn has made speciation in mammals faster than that in frogs. It is, however, possible that the correlation between the rate of speciation and the rate of chromosomal change is coincidental. This is because both chromosomal change (BUSH et al. 1977) and evolution of reproductive isolation (NEI 1976; present paper) are expected to be faster in small populations than in large populations and the effective size of mammalian species appears to be generally smaller than that in other vertebrates.

In our view it is unlikely that the change in chromosome number is as important as gene mutation for speciation. If the change in regulatory systems of genes is responsible for interspecific hybrid sterility or inviability, it should occur through mutational changes of regulatory genes themselves. At the molecular level the regulation of gene function occurs either through RNAs or through proteins, and the change in RNAs and proteins must first occur in the genes coding for them. Of course, there is a possibility that the change in gene regulation occurs through a position effect that is generated by chromosomal changes (BUSH 1982). In this case, however, it is not clear why only heterozygotes are adversely affected, so that hybrid sterility is generated. It should also be noted that although frog species have undergone a relatively small number of chromosome number changes other types of changes such as deletion and insertion seem to have occurred quite frequently (WHITE 1978). This weakens the position effect hypothesis. For many years CARSON has investigated the relationship between chromosomal changes and speciation in Hawaiian Dro- sophila. His recent conclusion (CARSON 1981) that “there is no evidence that the chromosomal variability plays an active role in the dynamics of the speciation process” is in agreement with our view (see also FUTUYMA and MAYER 1980).

Some authors (e.g., WHITE 1978; THALER, BONHOMME and BRITTON-DAVIDIAN 1981) have advocated speciation by chromosomal change on the basis that when there are chromosome number races within a species the hybrids between them sometimes show a reduced fertility (e.g., YOSIDA 1980). This reduced fertility may certainly be due to meiotic irregularities in the hybrids, as they assume. However, it is also possible that the primary factor for the reduced hybrid fertility is the mutational change of incompatibility genes and the chromosomal

Page 20: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

576 M. NEI, T. MARUYAMA AND C.-I. W U

differentiation is coincidental, as mentioned earlier. It will be interesting to distinguish between these two hypotheses by conducting proper segregation analyses.

We have seen that when 4Nv is small, the population dynamics of incompat- ibility genes is essentially the same as that for neutral genes. Therefore, the dynamics can be predicted by the stochastic theory of neutral alleles. For example, in the case of infinite allele model, the probability distribution of the frequency of new mutant genes at time t is given by formula (7) in NEI (1976), and the average time for the original allele or alleles to disappear from the population is 4N +- l/v generations approximately. The average time for repro- ductive isolation to be completed between two descendant populations is expected to be of the same order of magnitude when 4Nv << 1. Therefore, it is inversely proportional to mutation rate. This itself shows that speciation is a very slow process. If 4Nv is large, it would take more time as shown in this paper. Of course, here we are concerned with the average time for development of reproductive isolation. In some cases the ancestral stock from which two descendant populations are derived may be polymorphic for incompatibility genes, as in the cases of examples 5, 8 and 9 in Table 3. In these cases the evolution of reproductive isolation would occur much faster particularly in the presence of bottleneck effects.

A number of authors (e.g., POWELL 1978; KILLAS and ALAHIOTIS 1982) have reported that reproductive isolation can be enhanced in laboratory conditions. This type of enhancement of reproductive isolation often occurs extremely rapidly in terms of geological time. However, this is not surprising if we consider the possibility that some loci are subject to weaker incompatibility selection than those considered in this paper and, thus, tend to be more polymorphic. If incompatibility loci are highly polymorphic with many alleles, it is easy to change the intensity of reproductive isolation by selection or genetic drift. However, if the effects of incompatibility genes are small, complete reproductive isolation would not be attained quickly because in this case many gene substi- tutions are required before completing the isolation. In other words, experimen- tal demonstration of rapid enhancement of reproductive isolation does not necessarily mean that the establishment of reproductive isolation occurs very rapidly.

We thank G. L. BUSH, S. R. DIEHL, D. J. FIJTUYMA, T. GO~OBORI and 0. MAYO for their comments on an earlier version of this paper. This study was supported by grants from the National Science Foundation and the National Institutes of Health.

LITERATURE CITED

BENGTSSON, B. 0. and W. F. BODMER, 1976 OR the increase of chromosome mutations under

BUSH, G. L., 1975

BUSH, G. L., 1982

BUSH, G. L., S. M. CASE, A. C. WILSON and J. L. PATTON, 1977

random mating. Theor. Pop. Biol. 9: 260-281.

Modes of animal speciation. Annu. Rev. Ecol. Syst. 6: 339-364.

W h a t do we really know about speciation? pp. 119-128. In: Perspectives

Rapid speciation and chromosomal

on Evolution, Edited by R. MILKMAN. Sinauer, Sunderland, MA.

evolution in mammals. Proc. Natl. Acad. Sci. USA 74: 3942-3946.

Page 21: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

REPRODUCTIVE ISOLATION 577

CARSON, H. L., 1970

CARSON, H. L., 1971 Speciation and the founder principle. Stadler Genet. Symp. 3 51-70.

CARSON, H. L., 1981

CHU, Y.-E. and H.-I. OKA, 1970 The genetic basis of crossing barriers between Oryza perennis

DOBZHANSKY, TH., 1937 Genetics and the Origin of Species. Columbia University Press, New York.

DOBZHANSKY, TH., 1940 Speciation as a stage in evolutionary divergence. Am. Nat. 74: 312-321.

DOBZHANSKY, TH., 1970 Genetics of the Evolutionary Process. Columbia University Press, New

EWING, A. W., 1969 The genetic basis of sound production in Drosophila pseudoobscura and

FELSENSTEIN, J., 1981 Skepticism towards Santa Rosalia, or why are there so few kinds of animals?

FISHER, R. A., 1930

FUTUYAMA, D. J. and G. C. MAYER, 1980 Non-allopatric speciation in animals. Syst. Zool. 2 9 254-

GRANT, V., 1971

HALDANE, J. B. S., 1922

HEDRICK, P. W., 1981 The establishment of chromosomal variants. Evolution 35: 322-332.

ITOH, Y., 1979 Random collision process of oriented graph. Institute of Statistical Mathematics (Japan), Research Memorandum no. 154 1-20.

KANESHIRO, K. V., 1980 Sexual isolation, speciation and the direction of evolution. Evolution 34 437-444.

KAWANISHI, M. and T. K. WATANABE, 1981 Genes affecting courtship song and mating preference in Drosophilo melanogaster, Drosophila simulans, and their hybrids. Evolution 35: 1128-1133.

KILLAS, G. and S. N. ALAHIOTIS, 1982 Genetic studies on sexual isolation and hybrid sterility in long-term cage populations of Drosophila melanogaster. Evolution 36 121-131.

KIMIJRA, M. and J. F. CROW, 1964 The number of alleles that can be maintained in a finite population. Genetics 4 9 725-738.

LI, W.-H., 1977 Distribution of nucleotide differences between two randomly chosen cistrons in a finite population. Genetics 85: 331-337.

Chromosome tracers of the origin of species. Science 168 1414-1418.

Chromosomes and speciation. Chromosomes Today 7: 150-164.

subsp. Barthii and its related taxa. Evolution 24: 135-144.

York.

Drosophila persimilis. Anim. Behav. 1 7 556-560.

Evolution 35: 124-138.

The Genetical Theory of Natural Selection. Clarendon Press, Oxford.

271.

Plant Speciation. Columbia University Press, New York.

Sex ratio and unisexual sterility in hybrid animals. J . Genet. 12: 101-109.

MARUYAMA, T., 1977

MARUYAMA, T. and M. NEI, 1981

MAYR, E., 1963 Animal Species and Evolution. Harvard University Press, Cambridge, MA.

MAYR, E., 1970 Populations, Species, and Evolution. Harvard University Press, Cambridge, MA.

MIJLLER, H. J., 1942

Stochastic Problems in Population Genetics. Springer-Verlag, Berlin.

Genetic variability maintained by mutation and overdominant selection in finite populations. Genetics 98: 441-459.

Isolating mechanisms, evolution and temperature. pp. 71-125. In: Biological Symposia 6, Edited by TH. DOBZHANSKY. Jaques Cattell Press, Lancaster, PA.

NEI, M., 1975

NEI, M., 1976

Molecular Population Genetics and Evolution. North Holland, New York.

Mathematical models of speciation and genetic distance. pp. 723-765. In: Population Genetics and Ecology, Edited by S. KARLIN and E. NEVO. Academic Press, New York.

Discussion on Littlejohn's paper. pp. 485-489. In: Proceedings of the lnternationol Conference on Systematic Biology. National Academy of Science, Washington, D.C.

A model of mutation appropriate to estimate the number of electrophoretically detectable alleles in a finite population. Genet. Res. 22: 201-204.

NEVO, E., 1969

OHTA, T. and M. KIMURA, 1973

Page 22: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

578 M. NEI, T. MARUYAMA AND C.-I. WU

OKA, H.-I., 1957 Genic analysis for the sterility of hybrids between distantly related varieties of

Analysis of genes controlling F, sterility in rice by the use of isogenic lines.

Reproductive barriers distributed in plant species and their genetic control (abstr.). pp. 56-61. In: The Dynamics of Speciation in Plants and Animals, Edited by H.-I. OKA and 0. KITAGAWA. National Science Foundation and Japan Society for Promotion of Science.

Selection for reproductive isolation between two populations of maize, Zea mays. Evolution 23: 534-547.

PATERSON, H. E. H., 1981 The continuing search for the unknown and unknowable: a critique of contemporary ideas on speciation. S. Afr. J. Sci. 77: 113-119.

POWELL, J . R., 1978 The founder-flush speciation theory: an experimental approach. Evolution 32:

PRAGER, E. M. and A. C. WILSON, 1975 Slow evolutionary loss of the potential far interspecific hybridization in birds: a manifestation of slow regulatory evolution. Proc. Natl. Acad. Sci. USA

Origin of reproductive isolation in the absence of apparent genic differentiation

cultivated rice. J. Genet. 55: 397-409.

Genetics 77: 521-534. OKA, H.-I., 1974

OKA, H.-I., 1978

PATERNIANI, E., 1969

465-474.

72: 200-204.

PRAKASH, S., 1972

RENSCH, B., 1959

ROBERTSON, A. and P. NARAIN, 1971

in a geographic isolate of Drosophila pseudoobscura. Genetics 72 143-155.

Evolution Above the Species Level. Columbia University Press, New York.

The survival of recessive lethals in finite populations. Theor. Pop. Biol. 2: 24-50.

Genetic studies of speciation in cultivated rice. I. Genic analysis for the F, sterility between 0. sativa L. and 0. glaberrima Steud. Jpn. J. Genet. 54:

On the evolution of behavioral reproductive isolation: the Wallace

STEPHENS, S. G., 1964 The genetics of 'corky'. I. The New World alleles and their possible role as

A two-sex polygenic model for the evolution of premating isolation. I. Deter-

SVED, J . A., 1981b A two-sex polygenic model for the evolution of premating isolation. 11. Computer

TAKAHASHI, R., J. HAYASHI and I. MORIYA, 1976 Inheritance and linkage studies of the chlorotic

TAN, C. C., 1946 Genetics of sexual isolation between Drosophila pseudoobscura and Drosophila

TAUBER, C. A., M. J. TAUBER and J. R. NICHOLS, 1977 Two genes control seasonal isolation in sibling

TEMPLETON, A .R., 1980 The theory of speciation via the foundor principle. Genetics 94: 1011-

Processes of speciation and semi-

Comparative gene analysis of common wheat and its

A gene that rescues the lethal hybrids between Drosophila melanogaster

SANO, Y., Y.-E. CHU and H.-I. OKA, 1979

121-132.

SAWYER, S. and D. HARTL, 1981 effect. Theor. Pop. Biol. 19: 261-273.

an interspecific isolating mechanism. J. Genet. 47 150-161.

ministic theory for natural populations. Genetics 97 197-215. SVED, J. A., 19810

simulation of experimental selection procedures. Genetics 97: 217-235.

hybrids due to complementary dominant genes. Barley Genet. Newsl. 6: 71-73.

persimilis. Genetics 31: 558-573.

species. Science 197: 592-593.

1038.

speciation in the house mouse. Symp. 2001. Soc. Lond. no. 4 7 27-41.

ancestral species. I. Necrosis. Ipn. J. Genet. 37 474-484.

and D. simulans. Jpn. J. Genet. 5 4 325-331.

THALER, L., F. BONHOMME and J. BRITTON-DAVIDIAN, 1981

TSUNEWAKI, K. and H. KIHARA, 1962

WATANABE, T. K., 1979

WHITE, M. J. D., 1978 Modes of Speciation. W. H. Freeman and Co., San Francisco

Page 23: MODELS OF EVOLUTION OF REPRODUCTIVE ISOLATION · Mathematical models are presented for the evolution of postmating and premating reproductive isolation. In the case of postmating

REPRODUCTIVE ISOLATION 579

WILSON, A. C., L. R. MAXSON and V. M. SARICH, 1974 Two types of molecular evolution: evidence from studies of interspecific hybridization. Proc. Natl. Acad. Sci. USA 71: 2843-2847.

The importance of gene rearrangement in evolution: evidence from studies on rates of chromosomal, protein, and anatomical evolution. Proc. Natl. Acad. Sci. USA 71: 3028-3030.

On the probability of fixation of reciprocal translocations. Am. Nat. 75: 513- 522.

WILSON, A. C., V. M. SARICH and L. R. MAXSON, 1974

WRIGHT, S., 1941

YOSIDA, T. H., 1980 Cytogenetics of the Block Rat. University of Tokyo Press, Tokyo.

ZOUROS, E., 1981 The chromosomal basis of sexual isolation in two sibling species of Drosophila: D. orizonensis and D. mojavensis. Genetics 97 703-718.

Corresponding editor: B. S. WEIR