Introduction

The genetic basis of the correlation of two characters may be the result of genes having effects on both characters, pleiotropy, or of gametic (linkage) disequilibrium between different loci that influence the two different traits. Of particular interest in life history evolution, and potentially for the maintenance of genetic variation, is when different components of fitness are negatively correlated, e.g. individuals with high reproduction also have low viability, or vice versa. It is generally assumed that such negative correlations may occur when there is a metabolic or ecological cost to having a high value for one fitness component that subsequently results in a low value for another fitness component.

Rose (1985) suggested that the genetic basis of these negative correlations may be loci with antagonistic pleiotropy, i.e. alleles at these loci result, for example, in both high reproduction and low viability, and other alleles result in both low reproduction and high viability. In addition, the amount of gametic disequilibrium in outbred populations, except for closely linked loci, generally appears to be low, making gametic disequilibrium a less appealing hypothesis in large, stable, outbreeding populations. As a result, antagonistic pleiotropy has become a central part of life history theory, evolution of senescence and other topics in evolutionary ecology (e.g. Rose, 1991; Roff, 1992; Stearns, 1992; Bulmer, 1994; Charlesworth, 1994). Furthermore, it has been suggested that such negative correlations may result in the maintenance of high amounts of genetic variation for fitness traits (Prout, 1980; Falconer, 1981; Rose, 1982; Charlesworth & Hughes, 1996). This has occurred in spite of the fact that, initially, Rose (1982) and, recently, Curtsinger et al. (1994) have shown that it is difficult in theory to meet the conditions for the maintenance of a genetic polymorphism for a locus with antagonistic polymorphism.

In this paper, some of the findings of Curtsinger et al. (1994) are re-examined and extended and the maintenance of polymorphism via antagonistic pleiotropy is compared with that from other mechanisms of maintenance. In particular, the potential influence of finite population size, sex-dependent selection and partial inbreeding on the maintenance of polymorphism will be examined. Overall, it appears quite difficult in theory to maintain genetic variation with antagonistic pleiotropy, even more so than previously suggested. This, of course, does not mean that there is not a contribution from antagonistic pleiotropy to traits affecting fitness, but empirical evidence suggesting that it is the mechanism for the maintenance of polymorphism should be carefully scrutinized.

Models and results of differential selection for reproduction and viability

General model

With antagonistic pleiotropy, it is assumed, for example, that high reproductive effort, such as high fecundity in females or high mating success in males, may result in lowered subsequent viability. Or vice versa, high effort expended in survival may result in subsequent lower reproductive effort or success. To examine this, if we let the reproductive values and viability of genotype AiAj be fij and vij, respectively, then the conditions for a stable polymorphism are

assuming that the fitness components are multiplicative (Rose, 1982; Hedrick, 1985). (The conditions for a polymorphism when fitness components are additive are even more restrictive and therefore will not be discussed here.) These conditions for a polymorphism from antagonistic pleiotropy are the same as when fitnesses vary over time in different generations, i.e. the geometric means of the homozygotes must be smaller than the geometric mean of the heterozygote (Haldane & Jayakar, 1963; Hedrick, 1986).

Let us parameterize these values based on the level of selection against the homozygote (s) and dominance (h), so that the relative reproductive values are 1−s1, 1−h1s1, and 1, and the relative viability values are 1, 1−h2s2 and 1−s2 for genotypes A1A1, A1A2 and A2A2 respectively. For a stable polymorphism, then

If there is the same level of dominance for both traits, assuming h=h1=1−h2 [an example of what Curtsinger et al. (1994) term parallel dominance], then

The conditions for the maintenance of polymorphism can also be given as

Figure 1 gives the conditions for a polymorphism when h=0.5 as a function of the overall fitness of the two homozygotes (between the solid lines). A polymorphism may be maintained in this case, but the conditions are somewhat restrictive, particularly when the amount of selection is low (1−s1 and 1−s2 approach 1). For example, if h=0.5 and s1=0.1, then s2 must be between 0.0952 and 0.1053, a range of only 0.0101. Only when there are large selective differences, which if s1 approaches 1 imply sterility or if s2 approaches 1 imply lethality, are the conditions broadened significantly.

Fig. 1
figure 1

When h=0.5, the region of a polymorphism (between solid lines) and the region of an equilibrium between 0.2 and 0.8 (between dashed lines).

Full size image

In finite populations, Robertson (1962) elegantly demonstrated that the impact of selection for heterozygotes in a finite population was a function of the equilibrium allele frequency in an infinite population. He summarized his findings in the following way, ‘If the equilibrium gene frequency lies outside the range of 0.2 and 0.8, selection for the heterozygote may over a large range of population sizes in fact magnify the effect of reduced population size in leading to fixation.’ Robertson (1962) came to this conclusion from examining N(s1+s2) values up to 60, where N is the population size, a quite large value to occur in nature. In other words, for more maintenance of genetic variation than for neutrality, balancing selection must result in an equilibrium between approximately 0.2 and 0.8. Outside these equilibrium values, the overall impact of balancing selection is to drive the allele frequency towards low or high frequencies, thereby causing genetic drift to become more effective. As a result, the rate of loss of genetic variation is greater than if there were no selection at all.

In order to understand the general impact of finite population size on the maintenance of genetic variation with antagonistic pleiotropy, I determined the range of fitnesses that gives an equilibrium between 0.2 and 0.8 for the A2 allele (p2.e). These limits are given in Fig. 1 between the dashed lines, a region approximately 64.2% of that for a polymorphism. In other words, in a finite population, even though the conditions for a polymorphism in an infinite population are met, approximately 35.8% of the time, the genetic variation is lost as fast or faster than with neutrality.

As h deviates from 0.5, these conditions gradually become more restrictive. For example, the conditions for h=0.1 are given in Fig. 2 (between the solid lines), a region slightly smaller, 14.7% less, than when h=0.5. However, the region in which the equilibrium is between 0.2 and 0.8 (between the dashed lines) is much smaller, 50.5% less, than the comparable region for h=0.5. In other words, even though the conditions for a polymorphism may be met with extreme h-values, many of the equilibrium frequencies for A2 are very near 1.0, so that it is unlikely that these alleles would be maintained in a finite population.

Fig. 2
figure 2

When h=0.1, the region of a polymorphism (between solid lines) and the region of an equilibrium between 0.2 and 0.8 (between dashed lines).

Full size image

When h closely approaches 0 or 1, the region of polymorphism declines dramatically, and polymorphism can no longer be maintained. If h is equal to 0 or 1, the situation is similar to that when one of the homozygotes has the same value as the heterozygote for both traits and the other homozygote varies, the absolute dominance model Prout, 1968). In this case, there is no possibility of a stable polymorphism for differences in reproduction and viability, and these conditions become more restrictive than for the temporal variation model. To illustrate, let the fitnesses of A1A1, A1A2 and A2A2 be 1, 1 and 1−t1 for reproduction and 1, 1 and 1+t2 for viability, or let the two fitness arrays occur in different generations. For temporal variation, there is a stable polymorphism if (1−t1)(1+t2)<1 and (1−t1)+(1+t2)≲2 (Haldane & Jayakar, 1963), i.e. the geometric mean of the fitness of the variable homozygote is less than unity and the arithmetic mean is greater than unity. On the other hand, with selection for different fitness components, there is either directional selection for one of the alleles or, at best, a neutral equilibrium when (1−t1)(1+t2)=1 (Prout, 1967).

Sex-limited selection

In many instances, selection for reproduction at a given locus is likely to be in only one sex because female and male reproductive success is probably determined by different genes (Service, 1993; Curtsinger et al., 1994; Charlesworth & Hughes, 1996 and references therein). In addition, recent evidence suggests that genes influencing viability may have strong sex-dependent effects as well (e.g. Nuzhdin et al., 1997). To examine the influence of such sex-limited effects, the situation in which selection for both reproduction and viability are limited to one sex will be examined first, followed by the situation in which selection for only one fitness component, reproduction, is limited to one sex.

First, in the situation in which selection for both reproduction and viability occur in only one sex, the conditions for a polymorphism are

where f̄ij and \(\overline{v}\)ij are the average reproductive and viability values over both sexes. Obviously, if the genotypes only differ in reproductive fitness in one sex, then the conditions are more restrictive (the selective differences are smaller, so there is less possibility for stable polymorphism). For example, if we assume that s1=s2=0 in one sex, so that s1/2 and s2/2 are the mean selective values over both sexes, then the conditions for polymorphism are

If we assume the same dominance for both traits as above, then for a polymorphism

Figure 3 gives the region for a polymorphism for this situation when h=0.5 and h=0.1. The region of polymorphism is approximately 16% that when there is selection in both sexes. Furthermore, when s1=0.1 in the sex with selection, so that s1/2=0.05 (and h=0.5), then s2 must be between 0.0976 and 0.1026, a range of 0.0050, which is slightly less than half that when selection occurs in both sexes (s2/2 must be between 0.0488 and 0.0513).

Fig. 3
figure 3

When there is selection in only one sex, the region of a polymorphism for h=0.5 (between solid lines) and h=0.1 (between dashed lines).

Full size image

Secondly, if a given genotype has high preadult survival, it may have a lowered reproductive value that occurs in only one sex, i.e. the sex limitation occurs for only one fitness component. For example, organisms may expend a great deal of energy on juvenile survival, but there may be cost only to one sex in reproductive effort. This can be modelled as in Table 1 where reproductive selection operates only in females. Here, the nine mating types have different contributions to the progeny pool based on the relative female fecundity values. The progeny (the next generation) then have differential survival based on the relative viabilities that occur in both sexes. In this case, the genotypical frequencies in the progeny after viability selection (summing the columns in Table 1 and multiplying by the appropriate viability) are

Table 1 The mating types and the frequency of progeny produced when there is selection in females for reproduction and viability selection for progeny of both sexes (see text)
Full size table

where P11, P12 and P22 are the frequencies of genotypes A1A1, A1A2 and A2A2, respectively, p1 and p2 are the frequencies of alleles A1 and A2 and ¯ v is the sum of the numerators on the right side of the equations. These equations can then be iterated to determine the conditions for a polymorphism.

For this situation, Fig. 4 gives the equilibrium conditions (between solid lines) and the values that give equilibria of 0.2 and 0.8 (between broken lines). For a polymorphism, it is necessary for selection to be stronger for reproduction (1−s1 values lower) to offset the amount of viability selection that acts in both sexes. As expected, the region of stable polymorphism includes the line s1=2s2, in which selection is exactly twice as large for reproduction (although in one sex) as viability selection. The region of an equilibrium of A2 between 0.2 and 0.8 is only 56.6% of the region for a stable polymorphism, suggesting that maintenance of genetic variation in a finite population would be difficult for many combinations of fitness values that satisfy these conditions for a polymorphism.

Fig. 4
figure 4

When there is selection for reproduction in only one sex and for viability in both sexes, the region of a polymorphism (between solid lines) and the region of an equilibrium between 0.2 and 0.8 (between dashed lines) for h=0.5.

Full size image

Partial inbreeding

For organisms with partial inbreeding, the conditions for a stable polymorphism are more restrictive than when there is random mating (Hayman, 1953; for a review, see Hedrick, 1990a). Intuitively, this occurs because the frequency of heterozygotes is reduced with inbreeding, countering the effect of balancing selection that increases the frequency of heterozygotes. To examine the conditions for a polymorphism with partial inbreeding as well as reproduction and viability selection, let us examine the case of partial selfing (S), where the remainder of the population is randomly mated (1−S=T). The genotypical recursion equations can be written as (e.g. Hedrick, 1985)

where w̄ is the sum of the numerators on the right side of the equations. These equations can then be iterated to give the conditions for a polymorphism.

Figure 5 gives the conditions for a polymorphism when S=0, 0.5 and 0.95. Notice that, when S=0.95, the region of polymorphism is very narrow for nearly the whole range of selection. For example, with s2=0.5, then s1 must be between 0.492 and 0.507, a range of only 0.015. The regions of polymorphism for S equal to 0.5 or 0.95 are 63.8% and 19.0%, respectively, of that for random mating. Obviously, for a population with a high amount of self-fertilization, it is very difficult to maintain a polymorphism with antagonistic pleiotropy.

Fig. 5
figure 5

When there is partial selfing of the amount S, the region of a polymorphism when S=0 (between solid lines), S=0.5 (between dashed lines) and S=0.95 (between dotted lines) for h=0.5.

Full size image

Discussion

From the situations examined here, it is obvious that it is difficult to maintain a genetic polymorphism based on antagonistic pleiotropy. This, of course, was the primary message of Curtsinger et al. (1994) and was also stated by Rose et al. (1987). Two situations in which it appears very unlikely that antagonistic pleiotropy would maintain a polymorphism are when there is selection limited to only one sex and when there is a high amount of inbreeding. In addition, even when the conditions for a polymorphism are met, the equilibrium may be less than 0.2 or greater than 0.8, and the maintenance of genetic variation in a finite population may be less than that expected for a neutral variant. In general, only when the selective differences are large and fairly similar in size is the likelihood of a stable polymorphism from antagonistic pleiotropy significant. In these cases, it may be possible to measure the selective effects of the locus on different fitness components and to observe significant deviations from Hardy–Weinberg proportions.

There have been few examples in natural populations that appear to support antagonistic pleiotropy. The studies showing the strongest reproductive and viability effects on single genes are those in red deer, Cervas elaphus (Pemberton et al., 1988, 1991). They estimated the amount of selection related to allozyme loci and suggested that their results support ‘the idea that countervailing selection in different fitness components (antagonistic pleiotropy) is a common and powerful force maintaining polymorphism in natural populations’ (Pemberton et al., 1991). However, it would be useful to integrate their data from these two studies carefully to estimate the overall fitnesses for each genotype. In this manner, it could be determined whether these fitnesses would actually predict a balanced polymorphism and whether it is the result of antagonistic pleiotropy.

Rose et al. (1987) and Curtsinger et al. (1994) suggest that antagonistic pleiotropy should generate substantial dominance variance. However, as they state, there is little empirical evidence for high levels of dominance variance, consistent with low amounts of antagonistic pleiotropy. On the other hand, Charlesworth & Hughes (1996) demonstrate, by taking into account age structure, that antagonistic pleiotropy may not necessarily generate high levels of dominance variance. The implication from their discussion is that the level of dominance variance may not be a critical parameter indicating the presence or absence of significant amounts of antagonistic pleiotropy.

Reversal of dominance

The only major exception to these conclusions appears to be when there is a reversal of dominance such that the heterozygote is similar in fitness to the favourable homozygote for both traits (see Curtsinger et al., 1994 for an extensive discussion; Hoekstra et al., 1985 for a discussion of the phenomena when fitnesses vary in space). Both suggest that the likelihood of a reversal of dominance is low. The conclusions of Curtsinger et al. (1994) are based partly on the detailed analysis by Keightley & Kacser (1987) of a model of a branched multienzyme pathway with a single input and two outputs, i.e. the pleiotropic characters. Keightley & Kacser (1987) conclude that ‘four conditions must be simultaneously satisfied if substantial differences in dominance of pleiotropically related characters are to be observed.’

To illustrate the impact that this phenomenon could have, let us assume that h1=h2=h, making the conditions for a polymorphism with the fitnesses as given above

For the reversal of dominance to result in the heterozygote being similar to the favourable homozygote, then h should be small. At the extreme, if h=0, then there is a polymorphism for any combination of selective values. Even for h=0.2 and s1=0.1, then the range of s2 for a polymorphism is large, i.e. from 0.025 to 0.41.

Gametic disequilibrium

Both Rose (1982) and Curtsinger et al. (1994) considered the conditions for polymorphism when two loci contribute to the components of fitness, i.e. there are two loci that each have antagonistic pleiotropic effects. Curtsinger et al. (1994) have demonstrated that the two-locus conditions are significantly more restrictive than the one-locus conditions. Another explanation for the negative correlation of fitness components is that different loci may influence different traits and, if they are in gametic disequilibrium, a negative association may result.

To understand this mechanism, let us assume that locus A influences reproduction and locus B influences viability with the same fitness values as given above. When the fitness components at the two loci are multiplicative, then the two-locus fitnesses can be given as in Table 2. Notice that there is no possibility for a stable polymorphism when all genotypes are present because genotype A1A1B2B2 has the highest fitness and should eventually become fixed. This result is independent of the level of dominance for the two traits (assuming the dominance parameter is bounded by 0 and 1). On the other hand, if linkage disequilibrium is generated by small population size, mixing of populations or some other factor, then a negative correlation in the fitness components can occur. For example, if only gametes A1B1 and A2B2 are present (complete coupling linkage disequilibrium), so that only the genotypes on the diagonal from the top left to the bottom right in Table 2 are present, then there is a negative correlation, and there also appears to be a stable polymorphism. However, if there is any recombination at all, the other two gametes and their genotypes would be produced, the negative correlation would decay and selection would result in the eventual fixation of genotype A1A1B2B2.

Table 2 The two-locus multiplicative fitness values when reproduction is influenced by locus A and viability is influenced by locus B
Full size table

Comparison with other mechanisms for the maintenance of polymorphism

How does antagonistic pleiotropy compare with other mechanisms for the maintenance of polymorphism (see also Prout, 1998)? For the maintenance of a polymorphism with heterozygote advantage (overdominance), the fitness of the heterozygote needs to be larger than either of the homozygotes, and generally the fitnesses are given as 1−s1, 1 and 1−s2 for genotypes A1A1, A1A2 and A2A2 respectively. Note that this is only equivalent to the conditions for antagonistic pleiotropy in the extreme condition of h1=h2=0, i.e. reversal of dominance with complete recessivity.

For the maintenance of a polymorphism with variable selection in space (Levene, 1953), the harmonic mean of the heterozygote needs to be larger than the harmonic mean of both homozygotes, a broader condition than the geometric mean condition for antagonistic pleiotropy. Even though Maynard Smith & Hoekstra (1980) stated that the conditions for a polymorphism for variable selection in space were not very robust, the conditions are broader than for antagonistic pleiotropy in general, and several factors, e.g. limited gene flow (Christiansen, 1974), habitat selection (Hoekstra et al., 1985; Hedrick, 1990b) and inbreeding (Hedrick, 1998), can make the conditions for variable selection in space even broader. The conditions for a polymorphism with differential selection between the sexes (Kidwell et al., 1977) also have the harmonic mean conditions, again broader than the conditions for antagonistic pleiotropy.

For maintenance of a polymorphism with variable selection over generations (Haldane & Jayakar, 1963), the geometric mean of the heterozygote needs to be larger than the geometric means of both homozygotes, a condition equivalent to the geometric mean condition for antagonistic pleiotropy. However, with absolute dominance, variable selection over generations can maintain a polymorphism, whereas antagonistic pleiotropy cannot. Furthermore, for some patterns of diapause or dormancy, variable selection over generations can have broader conditions for a polymorphism (Hedrick, 1996).