MR 9 – Empirical Properties of Correlation

The evidence comes from Gunter Meissner’s study of the Dow Jones Industrial Average (the Dow) and its 30 member stocks, using daily closing prices from January 1972 to July 2017. That window produced 11,214 daily observations per stock, so 11,214 × 30 = 336,420 closing prices in total. The study always used whichever 30 companies made up the Dow at each point in time, because the index membership changes as weaker names drop out and stronger ones are added.

The returns were grouped into monthly bins. Inside each month a Pearson correlation was computed for every pair of stocks, giving a 30 by 30 matrix of 900 values. With 534 months in the sample, that adds up to 534 × 900 = 480,600 correlation values. To get one number per month, the unit correlations on the diagonal (each stock with itself) were removed and the rest were averaged. The study also built a second series, the volatility of these monthly averaged correlations, which measures how much the correlation level itself moves around.

Classifying the state of the economy

Each month was tagged with one of three economic states based on GDP (gross domestic product) growth:

• an expansionary period, where GDP grew by 3.5% or higher;
• a normal economic period, where growth ran between 0% and 3.49%;
• a recession, defined as two consecutive quarters of negative growth.

Plotted month by month, both the correlation level and its volatility look noisy, with sharp spikes and collapses. Once the months are sorted by economic state, though, a clear pattern appears.

The two difference columns are derived from the printed figures.

Correlations are lowest when the economy is growing strongly. A plausible reason is that in a boom, share prices respond mostly to company specific (idiosyncratic) news, so stocks drift independently. In a recession, macroeconomic forces dominate, and a common downturn drags many stocks lower together, which lifts correlation. The great recession of 2007 to 2009 showed this vividly, with the average Dow correlation climbing to 96.97% in February 2009.

Correlation volatility is also lowest in an expansion. The study expected recession volatility to exceed normal-period volatility, but the numbers show the opposite (80.48% against 83.06%). The interpretation offered is that once correlations are high in a recession, they tend to stay high without much extra movement. For a trader or risk manager the practical message is that correlation risk, meaning both a higher level and more instability, rises in bad economic times, which is exactly when diversification is most needed.

Table 1 already hints that states with higher correlation tend to show higher correlation volatility. The study checks this directly by plotting each month’s correlation volatility against that month’s correlation level for the full 1972 to 2017 sample and fitting a polynomial trendline of order 2.

The cloud of points slopes upward, and the fitted curve steepens at higher correlation levels. Months with an average correlation near 0.2 cluster at low volatility, while months with average correlation of 0.8 or more sit near the top of the volatility range. So, the study finds a positive link between how high equity correlations are and how volatile they are: when stocks are moving together strongly, the strength of that co-movement is itself less stable.

This matters when building stress scenarios. A model that raises correlations in a downturn but holds their volatility fixed understates how uncertain the dependence structure becomes, and positions whose value depends on correlation feel both effects at once.

A variable is mean reverting when it tends to be pulled back toward a long-term average. Finance assumes this for many quantities, including interest rates, volatilities and credit spreads. A fixed coupon bond that does not default is an extreme case: issued at par of, say, USD 100, it returns to exactly USD 100 at maturity. Interest rates follow a softer cycle, rising in a boom as demand for capital grows, cooling the economy, then falling below their long-run level in a slowdown before recovering as activity picks up again.

The condition for mean reversion

Write S_t for the value at time t and S_t−1 for the value one period earlier. Mean reversion is present when the change in the variable is negatively related to its previous level:

If the previous value was unusually low, the next change tends to be positive, pulling the variable up; if it was unusually high, the next change tends to be negative. The strength of this pull goes by several names: the mean reversion rate, the speed of mean reversion, or simply gravity.

From the Vasicek process to a regression

The discrete Vasicek process, which builds on the Ornstein-Uhlenbeck process, makes the pull explicit:

To isolate the pull, drop the random term and set Δt = 1. With a = 1 the gap to the mean closes fully in one step: if the previous value is 80 and the long-term mean is 100, the change is 1 × (100 − 80) = 20, so the variable lands on 100. With a = 0.5 half the gap closes each period, and with a = 0 there is no reversion at all. Without the random term the process becomes

The second line has the form of a simple linear regression, Y = α + βX, with the change as Y and the lagged level as X:

So, the regression slope β equals minus the mean reversion rate, and the intercept equals a times the long-term mean. Running this regression on the 534 monthly averaged Dow correlations gives y = −0.7903x + 0.256 with R² = 0.3948. The mean reversion rate is therefore 79.03% per month: on average, about four fifths of any gap between the current correlation and its long-term mean of 32.38% is closed in the following month. As a consistency check, a times the mean is 0.7903 × 0.3238 = 0.2559, which matches the fitted intercept of 0.256. The monthly correlation chart shows the same behaviour, with an upward spike usually followed by a sharp fall the next month and vice versa.

Once the rate and the long-term mean are known, the deterministic part of the process gives a one-step forecast. Rearranging the regression form for the new level gives

The forecast leaves out the stochastic term σ_Sε√Δt, which a full simulation of correlation paths would include. The next example uses original numbers to practise the whole chain, from regression output to forecast to autocorrelation.

Autocorrelation measures how strongly a series is correlated with its own earlier values. Positive autocorrelation is often called persistence; a fund analyst, for example, wants to know whether a fund’s above-market performance tends to carry on into later periods. Autocorrelation can be modelled through Robert Engle’s ARCH (Autoregressive Conditional Heteroscedasticity) model, which was recognised with a Nobel prize, or through Tim Bollerslev’s GARCH (Generalized Autoregressive Conditional Heteroscedasticity) extension. The simpler route used here is to correlate the series with a lagged copy of itself.

For a lag of one period the autocorrelation AC is the ordinary Pearson formula applied to the series and its one-period-lagged version:

Why the two properties add up to one

Mean reversion and autocorrelation pull in opposite directions. A strong pull back to the mean erases yesterday’s deviation quickly, so today’s value carries little memory of yesterday’s. Algebraically, regressing S_t on S_t−1 gives a slope of 1 − a, and for a long, stable series that slope is close to the lag-one autocorrelation. For the Dow sample the one-month autocorrelation is 20.97%, and 20.97% plus the mean reversion rate of 79.03% is exactly 100%. Strictly, the identity holds for the regression slope; the Pearson autocorrelation matches it when the variances of the current and lagged series are equal, which is very nearly true over 534 months.

How autocorrelation changes with the lag

Repeating the calculation for time lags of 1 to 10 months shows the usual decay: more distant months carry less information about the current correlation. The decay is not perfectly smooth. In the study, the two-month lag produced the highest autocorrelation, slightly above the one-month value, and later lags drift down with small bumps.

For the distribution tests the study switched from monthly averages to the individual pairwise correlations among the 30 Dow stocks over the 1972 to 2017 window, built from daily data, giving 464,580 correlation values. The histogram is strongly skewed toward positive values: 77.23% of all 464,580 correlations were above zero, and the bars rise steadily toward the region between 0.6 and 0.9 before dropping off sharply near 1.

The study tried 61 candidate distributions against this histogram, using three standard goodness of fit tests: Kolmogorov-Smirnov, Anderson-Darling and Chi-Squared. The Johnson SB distribution gave the best fit. It is a flexible family with four parameters: two shape parameters (γ and δ), a location parameter μ and a scale parameter. Its key feature is that it lives on a bounded range, which suits a correlation that can never leave the interval from −1 to 1, and its shape parameters let it bend into a heavily skewed form.

Standard choices such as the normal, lognormal and beta distributions fitted poorly. This answers a common modelling question: although stock prices, bond prices, exchange rates, commodities and volatilities are routinely modelled with normal or lognormal assumptions, equity correlations should not be. A normal distribution places mass outside the feasible range and cannot capture the heavy lean toward positive values. The study also ran the fit separately for each economic state and found only slight, statistically insignificant differences.

Six recessions fell inside the 1972 to 2017 window: a severe one in 1973-74 after the first oil price shock, a short one in 1980, another severe one in 1981-82 after the second oil price shock, a mild recession in 1990-91, a mild 2001 recession when the Internet bubble burst, and the great recession of 2007-09 that followed the global financial crisis. For each, the study measured the change in correlation volatility before the recession began, defined as the 6-month change in the 6-month moving average of correlation volatility. Severity is the total decline in GDP during the recession.

Rows are ordered from most to least severe; the last column is derived from the sign of the volatility change.

Ordering by severity makes the size of the 2007-09 downturn plain: a 14.75% fall in GDP, deeper than either oil-shock recession. More interesting is the pattern in the volatility column. Before five of the six recessions, correlation volatility declined; the only exception is 1990-91, where it edged up by 0.06%. This fits Table 1, since correlation volatility is lowest in expansions and expansions often come just before a downturn.

However, the size of the volatility decline tells you nothing useful about how bad the recession will be. The largest drop (−10.12% before 1980) preceded a moderate recession, while the deepest recession was preceded by one of the smallest drops (−2.64%). A regression of severity on the volatility change is almost flat with an R² close to zero, so the relationship is statistically non-significant. Work with longer data reaching back to 1920 was under way when the study was published.

Preliminary work extended the analysis to bonds (7,645 correlations) and to default probabilities (4,655 correlations). Both broadly share the properties of equity correlations, with differences in degree.

The equity figures here come from the comparison in the bond and default study and differ slightly from the 1972 to 2017 monthly figures used earlier (for example 77.51% against 79.03% mean reversion).

Bonds show a higher correlation level but less correlation volatility than equities. Default probabilities sit slightly below equities on level and slightly above them on volatility. Mean reversion is present in both, though at roughly a quarter to a third per period it is much weaker than the very strong equity reversion. By the add-to-one relation, that means bond and default probability correlations are considerably more persistent.

On distribution, the default probability correlations look like the equity ones and are again best replicated by the Johnson SB distribution. Bond correlations have a more normal shape: the generalised extreme value distribution fits best, and the normal distribution fits quite well. The study also reports that bond correlation levels and volatilities tend to be higher in bad economic times, as with equities. These bond and default results were still being checked on a larger sample.

Implications for modelling

Taken together, the findings give concrete guidance. Correlation inputs should be state dependent, higher and more uncertain in recessions. Correlation models should include mean reversion, very strong for equities and weaker for bonds and default probabilities. And simulations of equity or default probability correlation should draw from a bounded, skewed distribution such as Johnson SB rather than a normal.