Neutral biodiversity (ecology)

What part of the lives of trees in a forest is determined by chance and what part is biology? The chance is described by neutral biodiversity theory. Subtracting the predictions of this theory from decades of forest census data reveals the biology. Includes a short film.

A forest observed, a forest deciphered — the story of this page told in seven minutes: thirty years of tree censuses on Barro Colorado Island, Panama, built by Smithsonian field crews and made public, test neutral theory exactly and show that measured life histories, seed arrival from the surrounding forest and shared good and bad years predict which species gain and lose trees, which are common, and how much the whole forest fluctuates.

The island and the census

On Barro Colorado Island, in the Panama Canal, fifty hectaresa hectare is 10,000 square meters, about two and a half acres; fifty hectares is half a square kilometer of tropical forest were censused in 1982, again in 1985, and every five years from then through 2015: eight complete censuses over 33 years. In each one, field crews tag, identify and measure every free-standing woody stem at least one centimeter thick, and record which stems have died, grown or newly appeared since the census before. The plot belongs to ForestGEO, the Smithsonian Tropical Research Institute’s worldwide network of forest census plots, and every census is public. The first census found 235,360 living stems of 320 species. The counts are used here in two ways. The first is an exact test of neutral theory, in which species differ only by luck; it uses only trees with trunks at least ten centimeters across, about 21,000 in each census, of 230-odd species (255 over all eight censuses). The second is the smallest model that can predict which species will gain trees and which will lose them. Details of both are in the papers listed below.

Most species in the plot are rare: the commonest has about two thousand trees, but in the 2005 census 167 of the 230 species had 63 or fewer, and 22 had a single tree. The censuses also show each species gaining or losing trees from one census to the next. Part of that change is chance, which trees happen to die and which leave offspring; ecologists call that part driftchange in a forest’s mix of species produced by chance deaths and births alone, with no species favored. The rest reflects real differences among the species. Telling the two apart requires knowing exactly what chance alone would do.

Neutral theory

The lopsided pattern of a few common species and many rare ones had a statistical description long before it had an explanation. In a 1943 paper the statistician R. A. Fisher and two colleagues proposed that, in a large sample, the number of species represented by exactly $n$ individuals is

$$S(n) \;=\; \alpha\,\frac{x^{n}}{n},$$

where $x$ is a number a little below one, set by the size of the sample, and $\alpha$, now called Fisher’s alpha, measures diversity. In this log-series, species seen once outnumber those seen twice, and so on down a long tail.

Stephen Hubbell, an ecologist who developed his ideas largely on the Barro Colorado data, proposed a mechanism in 1997 and set it out fully in a 2001 book, The Unified Neutral Theory of Biodiversity and Biogeography. It is neutral in the sense that species do not matter: every tree, whatever its species, has the same chances of dying, of leaving offspring and of having arrived as a seed from outside. The plot holds a fixed number of trees, $J$. When one dies, its place goes with probability $m$ to a seed from the surrounding region, drawn in proportion to how common each species is out there. Otherwise it goes to the offspring of a tree already in the plot, picked at random. In the region, new species arise rarely and steadily, and regional abundances then follow Fisher’s log-series exactly, with Hubbell’s “fundamental biodiversity number” $\theta$ in the role of $\alpha$. Two numbers, $\theta$ and $m$, govern everything.

Hubbell wrote in the book itself that “the assumption of complete neutrality is patently false.” Ecologists adopted the theory anyway. It fits the abundance curves of many kinds of community with two or three numbers, whereas richer models fit no better and, when they miss, one cannot tell which ingredient is to blame. It can also be solved: the same mathematics had been worked out in population genetics, where Warren Ewens’s formula of 1972 gives the exact odds of any sample of gene variants drifting without natural selection. Early fits had shown that the theory matches a single Barro Colorado census well, and by 2005 Rampal Etienne had extended Ewens’s formula to the probability of an entire census of Hubbell’s plot. Neutral theory thus became community ecology’s working null modela deliberately simple theory used as a baseline: where real data depart from it, something the theory leaves out is at work: the prediction of chance alone, kept because its failures point to real biology.

Solving neutral theory exactly

Neutral theory comes in two forms. In the spatially implicit form just described, the plot is a well-mixed collection of $J$ trees taking in seeds from a large, featureless region. In the spatially explicit form every tree stands somewhere on a landscape and drops its seeds within a few tens of meters. How many seeds cross the plot’s edge then follows from the rules, with no immigration number left to adjust. Both can be solved exactly. The implicit form yields exact formulas for what repeated censuses measure, above all how much each species’ count should change between censuses by drift alone. The explicit form, whose predictions for a plot’s species counts had come only from simulation or a 2018 approximation, reduces to one equation.

The reduction to one equation works because species in the explicit form never interact, so each species is a family tree descended from one founder. Let $\phi(x,s)$ be the probability that a tree standing at position $x$ a time $s$ ago has at least one living descendant inside the plot today. The expected number of species in a plot of area $A$ is then

$$S(A) \;=\; \nu\rho\int_0^{\infty}\! ds\!\int\! d^{2}x\;\phi(x,s), \qquad\qquad \frac{\partial\phi}{\partial s} \;=\; D\,\nabla^{2}\phi \;-\; \nu\,\phi \;-\; b\,\phi^{2}.$$

Here $\rho$ is the density of trees and $b$ a tree’s birth rate. New species arise at the small rate $\nu$, matched by a slight excess of deaths over births, and $D = b\sigma^2$ sets how fast a line of descent wanders when seeds land a typical distance $\sigma$ from their parent. The first relation adds up, over every place and past moment at which a founder could have appeared, the chance that its line survives into the plot now. The second says how that chance changes as one looks further back: its three terms spread the lineage by seed dispersal, thin it by the excess deaths, and account for the branching of the family tree. A variant of the equation gives the expected number of species at each abundance, which is solved numerically to a stated accuracy.

The tree density comes from the census; the explicit form’s two remaining inputs, the seed distance $\sigma$ and the speciation rate $\nu$, had been fitted earlier, in 2002 and 2018, to a different kind of data: how quickly the species makeup of Panama’s forests changes with the distance between plots. The fitted seed spread is about 40 meters along each axis ($\sigma \approx 28$ m in the equation above). With those values, the 2005 tree count, and nothing adjusted to this plot, the full solution expects 82 species among trees at least ten centimeters thick, give or take 9, and 1.4 species present as a single tree. The census found 230 and 22. At the same inputs the 2018 approximation gives 47 and 0.7, so the exact answer roughly doubles the expected counts without coming near the census, and no inputs consistent with the Panama data do much better. The shortage of rare species is a property of the theory itself: a neutral forest whose seeds fall near their parents cannot hold nearly as many rare species as this one.

The 2005 census against neutral theory solved exactly. Bars: the number of species that have a given number of trees (trunks at least ten centimeters across; 230 species, 20,850 trees in all), in classes that double in width from single-tree species at the left, with every species of 64 or more trees pooled in the last bar. Squares: the number of species neutral theory expects in each class, from the full solution of its spatially explicit form, with seed distance and speciation rate set by earlier fits across Panama and nothing adjusted to this plot; the error bars are the theory’s own chance spread. The theory comes close only for the pooled common species, 63 counted against $57 \pm 8$ expected. In every other class it falls short, and worst among the rarest: 22 species have a single tree where the theory expects 1.4, and the plot holds 230 species where it expects 82. (Redrawn for this page from the data of Figure 3 of the first paper.)

The second comparison with the censuses uses change over time. Under drift a species’ count takes a random walk, one tree at a time, and the solved theory says how far it should wander. A species with $n_t$ trees at one census should show by the next a squared change that averages about

$$\mathbb{E}\big[(n_{t+1}-n_t)^{2}\big] \;\simeq\; \frac{2K\,n_t}{J}\Big(1-\frac{n_t}{J}\Big),$$

where $K$ is the number of trees that died and were replaced between the two censuses. The expected change grows in proportion to abundance because drift acts on one tree at a time; a cause acting on a whole species at once, a run of dry years, say, would give the square of abundance instead. Across the eight censuses the observed changes exceed the neutral expectation at every abundance, by two to six times for most species. The commonest may overshoot the most, though that hint rests largely on two species with long steady trends. Overall, the mix of species changes about as fast as drift would change it if trees were replaced four times as often as they are. It was already known that abundances at Barro Colorado change faster than drift allows; the exact solution shows how large the excess is at every abundance but assigns it no cause. The search for one follows below.

How much a species’ tree count changes from one census to the next, against how many trees it started the interval with (eight censuses, 1982–2015, trunks at least ten centimeters across). Orange: the observed squared change, averaged over the species in doubling bins of starting abundance $n_t$, with a 95% range from resampling the 255 species. Violet: the exact neutral expectation at the deaths and replacements actually recorded (line), and the same averaged over each bin’s species (squares). Both rise roughly in proportion to abundance, as drift acting one tree at a time should. The observed changes, though, run two to six times above chance in every bin up to a few hundred trees. In the top bin they run about thirteen times above it, a rise that rests largely on the steady trends of two common species. (Redrawn for this page from the data of Figure 4a of the first paper.)

After neutral theory

A theory that treats species alike can never predict which species will gain trees and which will lose them. The next question is how little must be added to the neutral lottery before a model can. The answer tried here is to let species differ in the one way the censuses measure directly: their life histories, meaning how fast their trees grow, how long they survive and how many young they leave. Kenneth Jops, James Dalling and James O’Dwyer tabulated those rates from these censuses in 2025, for each species in eight size classesgroups of stems by trunk diameter, here a seedling class and seven diameter classes above it; a stem moves up a class as it grows, inferring seed output from the size at which a species first reproduces. The model follows 84 species, each moving up through the size classes at its own rates. The censused stems of those species are capped, all together, at their average count, about 200,000; when one dies, a waiting seedling of any species, drawn by lot, takes its place. Seeds also arrive from outside in proportion to each species’ share of plots elsewhere in Panama. The last ingredient comes from the sapling records: measured sapling growth ran fast for nearly every species at once in the 1980s, slow in the 1990s and near average since. Each census interval therefore gets one random factor, shared by every species, that speeds or slows the growth of small trees; its strength is measured from those records, not tuned to any abundance.

The model’s first test is the forecast one could have made in 1995. The rates are re-measured from the first four censuses alone and applied to each species’ standing mix of small and large stems in 1995, 2000 and 2005 to predict the change in its large trees (trunks about eight centimeters and up) five and ten years on. Predicted and observed changes have a rank correlationa correlation computed on the ordering of species rather than on their values, asking only whether two quantities rank the species the same way; 1 is a perfect match, 0 none of 0.71 one census ahead and 0.68 two censuses ahead, well above the 0.54 that comes from simply extending each species’ past trend. The forecast works because a species with many saplings just below large-tree size is about to gain large trees, whatever it did before. Under neutral theory nothing measured early could rank the later changes, yet the 1980s size structures rank the species’ changes from 1990 to 2015 with a correlation that none of 600,000 simulated neutral forests and reshufflings reaches.

The forecast one could have made in 1995. Each dot is one of the model’s 84 species starting from one of three censuses (1995, 2000 and 2005). Across: the change actually counted in its large trees by the next census, five years on. Up: the change predicted by applying the species’ rates of growth and survival, measured from the 1982–1995 censuses alone, to its standing mix of stem sizes. Larger dots mark species with more large trees at the start; the axes are logarithmic away from zero, and a dot on the diagonal is a perfect forecast. The forecasts rank the species’ gains and losses with a correlation of 0.71 and get the direction right in 74% of the forecasts where the count changed, though they run high in total, with more dots above the line than below. (Redrawn for this page from the second paper’s forecast data; its Figure 3 shows the same test two censuses ahead.)

The model’s second test is the long run. Run for thousands of model years, the simulated forest settles into a steady mix of species, in which each species’ number of large trees is its seed arrivals times its local gain, meaning the large trees its life history yields per arriving seed. With seeds arriving by regional abundance alone, the species come out in roughly the right order (rank correlation 0.67): the locally common species are the regionally common ones. The amounts are wrong, though, with far too many trees for species whose seedlings most often reach large size. An alternative is to weight each species’ arrivals down by its local gain, so that strong competitors are weak colonizers, an old idea in ecology called the competition–colonization trade-off. That brings the amounts close as well (rank correlation 0.73), and the abundance curve nearer the census than a fitted neutral curve. This is a phenomenologicalchosen because it reproduces what is observed, without a measured mechanism behind it repair, since nobody has yet measured which seeds actually arrive.

The shared good and bad years matter most for the forest as a whole. Without them the model forest is twelve times too steady from census to census; with them the flow of trees into the large size classes varies about as much as it really does. One number per interval produces different swings in different species because each has a different share of its stems at sapling size at the time. Even so, species of middling abundance change two to three times too little in the model, and among seedlings and small saplings the model ranks the species poorly.

The papers

• Neutral biodiversity theory solved and tested against three decades of Barro Colorado censuses — Matthew D. Schwartz (Harvard) and James P. O’Dwyer (Illinois); in preparation.
• A minimal model of a forest after neutral theory (PDF) — Matthew D. Schwartz (Harvard) and James P. O’Dwyer (Illinois). A preliminary version of the paper.
• Predicting the life history of forests — a third paper, in preparation.

Supplementary material

Every dataset the papers analyze is public. The tree censuses both papers rest on are listed below.

• doi:10.15146/5xcp-0d46 — the complete 1982–2015 censuses of the Barro Colorado 50-hectare plot (R. Condit, R. Pérez, S. Aguilar, S. Lao, R. Foster and S. Hubbell, 2019), archived on Dryad; the tree counts both papers analyze.

References

← back to the web summaries