The cost of complexity in adaptation

Biological complexity is a vague notion, but we seem to have an intuitive understanding of it. The human eye is more complex than the nautilus eye, multi-cellular organisms are generally more complex than unicellular organisms, the brain is a complex organ, the human brain is more complex than the C. elegans brain etc. Moreover, we usually think of complexity as a “good” thing. For example, complexity may be useful for adaptability: more complex organisms may adapt better to specialized environments. In statistical terms, if you have more free parameters, (i) you can fit a larger class of datasets, (ii) you can fit a particular dataset better than a simpler model with less free parameters (ignoring over-fitting, which is, I think, an interesting issue in the biological context).

Is complexity an unreservedly “good” thing? The answer to this is clearly no. Otherwise, we wouldn’t have any simple organisms populating the earth. There are obvious costs to being complex: more complex organisms tend to consume more energy, they tend to develop more slowly etc. But let’s ignore these costs and just think about adaptability. Is complexity always good for adaptability? In The Genetical Theory of Natural Selection (1930), the great geneticist and statistician Ronald Fisher (inventor of ANOVA and maximum likelihood estimation among other things and one of the founding fathers of what is known as the modern synthesis in evolutionary biology) identified a quite severe cost to complexity in adaptation: complexity slows down adaptation. The intuitive reason for this is simple: random mutations of a fixed size are less likely to be favorable in more complex organisms (more on this in a moment). That complexity slows down adaptation should also not be surprising to anybody who has tried to train a statistical model on some data: more complex models in general take longer to train. In the biological context, this simple observation turns out to have important, and rather troubling, implications, but it is important to first have a simple quantitative estimate of how severe the cost of complexity is.

Fisher studied adaptive evolution (or Darwinian evolution) using a very simple geometric model in which an organism (or more properly an “average” organism in a population) is represented as a point in a high-dimensional space where each dimension corresponds to a trait or character. Random mutations move the population around in this high-dimensional space. Complexity is defined as the number of independent traits (dimensionality of the space). There is assumed to be a single fitness maximum at the origin. Fitness decreases as a monotonic function (usually Gaussian) of the distance from the origin. Mutations that move the population toward the origin are “favorable” mutations, those that move the population away from the origin are unfavorable. Natural selection is thus hill-climbing on a smooth fitness landscape in this model, but quite an inefficient one from an engineering perspective, because mutations are completely random. An engineer would, for example, make use of the gradient information to propose intelligent moves, rather than completely random ones.

Unfortunately, Fisher’s results based on this model were flawed, because in his analyses he ignored an important population genetic factor: the probability of fixation (fixation means the mutated variant of a gene spreading over the whole population) when a mutation is favorable. Small mutations are more likely to be favorable (you can immediately see this from the geometric model). But they have a lower probability of fixation, when they are favorable (this is because larger mutations, when they are favorable, tend to increase the fitness more). This flaw also affects Fisher’s result on the cost of complexity in adaptation.

Orr (2000) re-does Fisher’s analysis of the cost of complexity taking into account the probability of fixation. He shows that the cost of complexity is in fact even larger than Fisher calculated.

Here’s Orr’s analysis in summary. Let’s denote the location of the population in the n-dimensional space by \mathbf{z}=[z_1, z_2, \ldots, z_n]. The fitness is a Gaussian function of distance from the origin: \bar{w} = \exp(-z^2/2) where z = ||\mathbf{z}|| (Euclidean norm). To focus on the effects of complexity (n) only on the rate of adaptation, Orr considers a scenario where populations with different n start at a fixed distance z from the origin and produce mutations of a fixed size r. Thus both of these factors are complexity-independent.

The rate of adaptation in the geometric model is given by: d\bar{w}/dt = (d\bar{w}/dz) (dz/dt). The first term is independent of n. It turns out that the second term is approximately given by (the derivation is quite straightforward and requires minimal knowledge of population genetics):

\frac{dz}{dt} = -2 N \mu P_a z \bar{\Delta}z_{fav} \bar{\Delta}z_{fix}

where N is the population size, \mu is the total mutation rate, P_a is the proportion of random mutations that are favorable, \bar{\Delta}z_{fav} is the mean distance traveled to the origin for favorable mutations and \bar{\Delta}z_{fix} is the mean distance traveled to the origin for mutations that get fixed in the population (\bar{\Delta}z_{fix} > \bar{\Delta}z_{fav} because mutations that travel a larger distance to the origin have a higher probability of fixation).

Complex organisms pay a triple price in this expression: (1) \bar{\Delta}z_{fav} scales roughly as \sqrt{n}; (2) \bar{\Delta}z_{fix} scales roughly as \sqrt{n}; (3) P_a decrease with n (this is the factor Fisher identified). I tried to picture all of these factors in the figure below:


In the two-dimensional case (top), favorable mutations (green arrows) move the population to a point that lies somewhere on the red arc ABC. The mean distance traveled toward the origin is clearly smaller than r (this is achieved only when the mutation is pointing straight toward the origin). In the one-dimensional case (bottom) \bar{\Delta}z_{fav} = r. You can convince yourself that this geometric intuition will hold in general and thus \bar{\Delta}z_{fav} will decrease with n. Similarly among favorable mutations, those that travel farther toward the origin will have a higher probability of fixation. This is represented by the green distribution over the arc ABC. Now clearly the distance traveled by these mutations toward the origin will be larger than \bar{\Delta}z_{fav}, but it’s still smaller than r which is equal to \bar{\Delta}z_{fix} in the one-dimensional case. The third factor P_a (proportion of random mutations that are favorable) is given by the ratio of the area ABCZA to the area of the entire red circle in the two-dimensional case. This ratio is smaller than a half. It is exactly equal to a half in the one-dimensional case. Again you can convince yourself geometrically that P_a will in fact decrease with n.

Orr shows that for small n the rate of adaptation declines approximately as n^{-1}, whereas for large n, it declines much faster than n^{-1}. This is a troubling result! The ability to evolve rapidly is crucial for organisms in cases where there is a threat of extinction. If complexity reduces the pace of adaptation in such a radical way, one would expect a pretty strong selective advantage for simplicity. Then the question would be: how did complex organisms evolve (as they surely did) despite a selective force against complexity? I’ll write more about the implications of this result and possible ways of resolving this puzzle in a separate post.

Fisher’s geometric model is obviously a cartoon model of adaptive evolution, so you may wonder if the cost of complexity still applies if we make the model more realistic. For example, Fisher’s model assumes universal pleiotropy (a mutation potentially influences all traits), which is a very strong assumption. Does the cost of complexity still apply if we introduce modular pleiotropy where a mutation can only affect a subset of the traits? Or what if the magnitude of average mutational change depends on complexity, with more complex organisms having larger magnitude mutations? It turns out that the cost of complexity is fairly robust to these and some other manipulations. Again, I will write more about these in a later post.

So, here’s the power of theory and why I absolutely love theory! With such a simple model, using just a pen, a paper and your imagination (maybe also a little bit of numerical computation, but nothing fancy), you make a very strong prediction about the entire history of the natural world (that’s billions of years!), a prediction about every single species of organism that has ever lived! The rate of adaptive evolution should slow down for more complex organisms! Whether it is feasible to test this prediction or whether this prediction turns out to be true or not, just being able to make the prediction should give us great intellectual pleasure.