QTA 11: Non-Stationary Time Series
A covariance-stationary series holds its mean, its variance and its autocovariances steady wherever in the sample you look, and failing one of those conditions makes it non-stationary. Three causes account for almost all of it in economic and financial data: a time trend, seasonality tied to the calendar, and a unit root, more often called a random walk.
A time trend says only that the series tends to grow, and that the growth is a function of the clock. Seasonality makes the mean depend on the month or the quarter, and it is handled either by letting the mean shift with the period of the year or by working with the year-over-year change, which cancels the pattern. Random walks are the third source and the most pervasive, since virtually all asset prices behave like one.
Deterministic trends and stochastic trends need different treatment
Every non-stationary series carries a trend that is either deterministic or stochastic. A deterministic trend, such as a time trend or a fixed seasonal pattern, is pinned down once you know the period. A random walk is the leading stochastic trend, depending equally on every past shock, and no calendar knowledge says where it goes next.
Treatment follows from diagnosis. Where the effects are deterministic, subtracting the fitted trend leaves a covariance-stationary series. Where a stochastic trend is present, only a difference of the form Y in period t minus Y in period t minus j delivers stationarity, and differencing removes time trends and, at the seasonal frequency, fixed seasonality too.
A time trend moves the mean of a series along in a way that depends only on the date. The plainest version is the linear time trend.
Because the expectation moves with t, the process is non-stationary, and the slope reads as the average change from one observation to the next. Adding higher powers of time bends the line.
Ordinary least squares estimates these parameters consistently and the estimators are asymptotically normal, so standard errors and t-statistics can be used provided the residuals really are white noise. When they are not, the t-statistics and the model R-squared are misleading.
Why a linear trend is often the wrong shape
A positive linear trend adds the same quantity each period, so the growth rate it implies keeps shrinking as the series gets larger, and a negative slope eventually drives the fitted series below zero. Constant growth rates fit economic behaviour better, and relating the natural log of the variable to a linear trend delivers that.
Taking expectations of the change in the log leaves a constant, so the expected proportional change does not depend on where in the sample you are. A log-quadratic trend lets the growth rate itself move.
A negative estimate of the second coefficient describes growth that is slowly fading. The log-linear form is usually enough, and the quadratic term earns its place only when the fade is pronounced.
Do not judge a trend model by its R-squared
Fitting all four specifications to a long run of quarterly real GDP produces very high R-squared values, higher still for the quadratic versions. Trending series deliver that automatically, and the statistic drifts towards one as the sample lengthens. Residual diagnostics carry the information instead, and here the residuals from all four models are visibly persistent rather than white noise.
A seasonal series shifts in a way that repeats through the year. New housing runs fastest over the summer and slows once winter arrives, while natural gas consumption does the reverse. Both patterns follow the weather, which is predictable.
Markets produce their own versions. Under the January effect, equity returns in January run above those of every other month, with tax optimisation as one explanation: investors sell losing positions in late December to realise the losses, then buy back in January. Calendar effects extend the idea to shorter cycles, since Monday returns are statistically significantly lower than Friday returns in United States equity data.
Building the dummy variable regression
Suppose the mean differs by period and the pattern repeats every s periods, which is four in quarterly data and twelve in monthly data.
Each indicator takes the value one when t modulo s equals j and zero otherwise, the modulo function returning the remainder after division, so that 10 (mod 6) = 4. One dummy has to go, because keeping all s of them alongside an intercept makes the regressors perfectly multicollinear, which is the dummy variable trap.
With the final dummy omitted, the intercept is the average of the series in period s and each remaining coefficient measures the gap from that base. Dropping the intercept and keeping all s dummies is equally valid, and then each coefficient is the average value in its own season.
Monthly gasoline consumption as an illustration
Monthly growth in United States gasoline consumption from 1992 until 2018 makes a clean example, with growth defined as 100 times the change in the natural log of consumption. March is the strongest growth month and September is one of four months showing large declines. Eleven monthly dummies leave residuals that look close to white noise, but the Ljung-Box Q-statistic with ten lags comes out at 33. Against a chi-squared distribution with ten degrees of freedom the p-value is below 0.1%, so no serial correlation is rejected.
Residuals from a model built only from trends and seasonal dummies tend to carry structure of their own. Where they are serially correlated but still covariance-stationary, add an autoregressive or moving average component, so that the model addresses trend, seasonality and cycle together.
A process is trend-stationary when removing a deterministic time trend leaves a covariance-stationary remainder. If the residuals from a trend regression behave like an AR(1), adding a lagged level should push them towards white noise, and seasonal dummies can sit in the same equation.
The trend coefficient carries long-run growth, the dummies carry seasonal shifts away from it, and the lagged level carries the cycle.
What the gasoline consumption models show
| Specification | Trend | Trend squared | AR term | R-squared | Ljung-Box Q, ten lags |
|---|---|---|---|---|---|
| Dummies and linear trend | 0.00428 (40.662) | 0.851 | 2193.3 (0.000) | ||
| Dummies and quadratic trend | 0.00901 (25.120) | -1.4e-05 (-13.549) | 0.908 | 1752.6 (0.000) | |
| Dummies, quadratic trend and AR(1) | 0.000437 (2.567) | -8.25e-07 (-2.343) | 0.959 (61.307) | 0.993 | 36.4 (0.000) |
Source: gasoline sales results reported in the chapter.
The quadratic term arrives with a negative coefficient that looks statistically significant, and the R-squared climbs from 0.851 to 0.908 and then to 0.993. That last figure flatters the model, since a trend makes R-squared far less informative, and the residuals from the quadratic specification are strongly persistent.
Adding the autoregressive term helps without settling matters. The estimated coefficient of 0.959 sits close to one, saying the detrended series is probably not covariance-stationary. The Ljung-Box statistic falls from 2193.3 to 1752.6 and then to 36.4, yet the p-value stays at 0.000 in every row.
This is where trend-stationary modelling reaches its limit. Once a random walk component is present, no amount of detrending removes the non-stationarity.
The simplest random walk sets today equal to yesterday plus a fresh shock.
Substituting the same relation in for the lagged value brings in the previous two shocks and the level two periods back. Repeat to the start of the sample and the lagged level disappears.
Every shock between period 1 and period t enters with the same weight, and so does the starting value. In a stationary first-order autoregression, repeated substitution puts a coefficient raised to a rising power in front of each older shock, and stationarity requires it to be below one in absolute value, which makes the distant past fade. The random walk has no such decay, so a shock alters every future value permanently and the series spreads out instead of reverting.
Unit roots generalise the random walk
A unit root process keeps the random walk at the long horizon while allowing stationary behaviour in the short run. Lag polynomials write this compactly.
Take the second-order autoregression with coefficients 1.8 and -0.8 on the two lags. The polynomial 1 – 1.8L + 0.8L squared acting on Y factors into (1 – L) times (1 – 0.8L), and the characteristic equation returns two roots, z = 1 and z = 1/0.8 = 1.25. The family takes its name from the first root being exactly unity, while the second exceeds one in absolute value and is stationary. When both types are present the non-stationary root governs. Adding a constant gives a random walk with drift.
Three difficulties separate a unit root series from a covariance-stationary one.
Inference stops working
Fit an ARMA model to a series with a unit root and its parameter estimators lose normality. What they follow instead is the Dickey-Fuller distribution, asymmetric, dependent on sample size, and sensitive to which deterministic terms the model holds. Its tails are fatter than the normal, so critical values are larger in absolute value and inference becomes unreliable.
Spurious relationships are hard to rule out
Two series have a spurious relationship when nothing fundamental connects them, yet regressing one on the other returns a coefficient that looks convincingly different from zero under conventional critical values. Unit root series attract these false findings, so impressive coefficients can be misleading and produce large out-of-sample forecasting errors.
There is no level to come back to
A stationary autoregression mean reverts, so its long-run mean can be estimated and its distant forecasts settle there, while a random walk forecasts the current value at any horizon. Simulate two AR(2) processes from the same initial value and shocks, the stationary one with 1.8 and -0.9 and the unit root one with 1.8 and -0.8, and the paths track closely for roughly the first 20 observations before separating.
Differencing solves all three at once
Take the first difference of a series with a unit root and the result has none. In a random walk with drift the lagged levels cancel, leaving the drift constant plus a white noise shock. The general case follows the same route.
Since the remaining polynomial belongs to a stationary process, the difference must itself be stationary, which is why it is the default transformation.
Differencing is only right when a unit root is genuinely there, and plenty of economic series are persistent while still stationary. Differencing a stationary series is over-differencing. A constant plus white noise, an AR(0), has a first difference equal to the current shock minus the previous one, a moving average that is not invertible, and the extra parameters cost accuracy.
The Dickey-Fuller regression
When the series really is a random walk, subtracting the lagged level from both sides leaves the difference equal to the shock alone, so the coefficient on the lagged level is zero. That is the null hypothesis, and the alternative is that it is negative, which corresponds to a covariance-stationary process. The alternative is one-sided, because a positive coefficient implies an explosive process.
The word augmented refers to those lagged differences, whose job is to absorb short-run dynamics so that the residual is white noise, a precondition for the statistic to follow the Dickey-Fuller distribution. Lag length comes from minimising the AIC, which leans towards larger models than the BIC, and that leaning is welcome: too many lags costs a little power while too few invalidates the distribution.
Deterministic terms change the critical value
The choice of deterministic terms matters more than the lag length. Each extra term skews the Dickey-Fuller distribution further left, so the critical value becomes more negative and rejection gets harder. The opposite risk is worse. Leave the constant out when the series is trend-stationary and the null is asymptotically never rejected, leaving the test with no power at all.
The recipe is to test each deterministic regressor with its own t-statistic at a size of 10%, dropping the trend first and then the constant. Most applications keep the constant.
When the null cannot be rejected, difference the series and test again, and a difference that still fails calls for a second difference. Most economic and financial series contain exactly one unit root, so they are integrated of order one, written I(1), and their differences are I(0). Consumer price series such as the CPI or the RPI are the one class with mixed evidence of I(2) behaviour.
A short series of ten observations is tested for a random walk using the plain Dickey-Fuller regression, with no intercept and no lagged differences. Starting from an initial value of 6, the observations are 6, 3, 4, 2, 3, 1, 2, 0, 1 and -1.
| Lagged level | Difference | Product | Lagged level squared |
|---|---|---|---|
| 6 | -3 | -18 | 36 |
| 3 | 1 | 3 | 9 |
| 4 | -2 | -8 | 16 |
| 2 | 1 | 2 | 4 |
| 3 | -2 | -6 | 9 |
| 1 | 1 | 1 | 1 |
| 2 | -2 | -4 | 4 |
| 0 | 1 | 0 | 0 |
| 1 | -2 | -2 | 1 |
| Sum | -32 | 80 |
Source: original illustration built for this lesson.
For a regression through the origin the slope is -32 divided by 80, or -0.40. The residuals are -0.6, 2.2, -0.4, 1.8, -0.8, 1.4, -1.2, 1.0 and -1.6, and their squares add to 16.20. With nine observations and one parameter the residual variance is 16.20 divided by 8, or 2.025, giving a residual standard deviation of 1.4230. The standard error is 1.4230 divided by the square root of 80, or 0.1591, so the statistic is -2.51. Against critical values of -1.942 at 5% and -2.571 at 1%, the null is rejected at 5% only.
Published ADF results arrive as a table with one row per configuration of deterministic terms. Settle which row is valid, then compare its statistic with its own critical values. The default premium, which subtracts the interest rate on a portfolio of Aaa-rated corporate bonds from the rate on a Baa-rated portfolio, is tested on 40 years of monthly data between 1979 and 2018. Lag length minimises the AIC across models with up to 24 lags.
| Deterministic terms | Lagged level | Constant | Trend | Lags | 5% critical value | 1% critical value |
|---|---|---|---|---|---|---|
| None | -5.34e-03 (-1.220) | 16 | -1.942 | -2.571 | ||
| Constant | -0.042 (-3.251) | 0.045 (3.037) | 10 | -2.868 | -3.444 | |
| Constant and trend | -0.048 (-3.473) | 0.063 (3.009) | -5.00e-05 (-1.217) | 10 | -3.420 | -3.978 |
Source: default premium results reported in the chapter.
Start at the bottom. The trend carries a t-statistic of -1.217, nowhere near significant at the 10% level, so it goes. One row up, the constant has a t-statistic of 3.037 and is rejected as zero at that level, so the middle row is the one to judge. Its ADF statistic is -3.251 in the table and -3.23 in the surrounding discussion. Either figure falls below the 5% critical value of -2.868 and above the 1% value of -3.444, so the null is rejected at 5% but not at 1%. The critical values themselves move from -1.942 with no deterministic terms to -2.868 with a constant and -3.420 with a trend.
ADF tests are run on the natural log of the ten-year United States government bond interest rate using data from 1988 until the end of 2017, with lag lengths from the AIC and critical values appropriate to the sample size and the terms included.
| Deterministic terms | Lagged level | Constant | Trend | Lags | 5% critical value | 1% critical value |
|---|---|---|---|---|---|---|
| None | -0.003 (-1.666) | 7 | -1.942 | -2.572 | ||
| Constant | -0.009 (-1.425) | 0.010 (1.027) | 4 | -2.870 | -3.449 | |
| Constant and trend | -0.085 (-4.378) | 0.188 (4.260) | -0.000 (-4.109) | 3 | -3.423 | -3.984 |
Source: practice question data reported in the chapter.
Take two independent unit root processes with Gaussian white noise shocks and regress one on the other using 100 observations. Nothing connects them, so a 5% test should reject 5% of the time. In simulation the slope is statistically different from zero in over 75% of the runs, and the rejection probability climbs towards one as the sample grows. The problem appears whenever the dependent variable and one or more regressors are non-stationary, so differencing whatever is not stationary removes the danger.
The Russell 1000 and the yen exchange rate
Put the weekly log level of the Russell 1000 index on the left and, on the right, the previous week’s closing log of the Japanese yen against the dollar. The output invites a story about currency moves leading equity prices.
| Specification | Intercept | Slope | R-squared |
|---|---|---|---|
| Log levels | 22.855 (95.823) | -3.232 (-65.932) | 67.6% |
| Returns | 0.000 (4.489) | -0.055 (-2.414) | 0.3% |
Source: results reported in the chapter, t-statistics in parentheses.
In log levels the slope of -3.232 arrives with a t-statistic of -65.932 and the regression explains 67.6% of the variation. It is spurious, because both series carry unit roots, the index with an obvious time trend and the exchange rate with a less pronounced one. The residuals are highly persistent, and an ADF test on them fails to reject the unit root null.
Differencing both sides repairs the analysis. Regressing the weekly return of the index on the previous week’s exchange rate return gives a slope of -0.055 with a t-statistic of -2.414 and an R-squared of only 0.3%. Both differenced series are stationary, which is what ordinary least squares needs in large samples, and the currency carries almost no information about next week’s index return.
Differencing is required only when a unit root is present, and many series are highly persistent without containing one. When a series resists easy classification, build the model both ways and compare what they imply.
The default premium illustrates the point, since its ADF test rejects the unit root null yet its persistence is high. An AR(3) is estimated on the level and an AR(2) on the difference, the order dropping by one because modelling the difference already assumes a unit root.
| Specification | Constant | First lag | Second lag | Third lag |
|---|---|---|---|---|
| Levels, AR(3) | 0.044 (0.362) | 1.330 (29.539) | -0.524 (-7.307) | 0.152 (3.385) |
| Differences, AR(2) | -0.000 (-0.014) | 0.360 (8.028) | -0.182 (-4.057) | |
| Differences model rewritten in levels | -0.000 (-0.014) | 1.360 (30.305) | -0.542 (-7.481) | 0.182 (4.057) |
Source: default premium estimates reported in the chapter.
An AR(2) fitted to the difference of the default premium returns coefficients of 0.360 on the first lagged difference and -0.182 on the second.
Substituting gives 1 + 0.360 = 1.360, then -0.182 – 0.360 = -0.542, then -(-0.182) = 0.182, against 1.330, -0.524 and 0.152 from the AR(3) estimated on the level. The two sets look interchangeable and are not. The levels model mean reverts to 1.06%, while the differenced model contains a unit root by construction. Forecasting forward 24 months from January 2018, the levels model returns towards its long-run mean while the differenced model predicts almost no change.
Seasonal differencing
Seasonal differencing offers a different route for a seasonal series that also contains a unit root. Subtracting the observation from the same period a year earlier clears out deterministic seasonality, time trends and unit roots together. In a quarterly series with fixed seasonal effects and a growth rate, the trend terms leave four times the trend coefficient, and the dummies cancel because the indicator takes the same value at t and at t minus 4.
What remains is covariance-stationary and reads naturally as the year-over-year change, or the year-over-year growth rate when the variable has been logged. Taking 100 times the difference between the log of gasoline consumption and its value twelve months earlier produces exactly that, with the seasonal swings gone.
Forecasting a model with trends, seasonal dummies and cyclical terms works like forecasting a stationary ARMA. The forecast is the expected value of the future observation conditional on information at time T, and in a linear trend model the shock has a mean of zero, so the forecast is the trend line at the future date.
Forecasting a series with seasonality
Seasonal dummy models add one piece of bookkeeping, tracking the period the forecast date falls in. The one-step forecast is the intercept plus the coefficient for period (T+1) modulo s, the omitted period carrying zero. If a quarterly model omits the fourth-quarter dummy and T is period 125, then (125 + 1) modulo 4 is 2, so the forecast is the intercept plus the second-quarter coefficient.
Once a cyclical component joins the dummies, forecasts are built recursively, with the deterministic part of each step depending on the period T plus h falls in. The influence of the most recent observation fades as the horizon lengthens, so these models mean revert to the period-specific means carried by the dummies rather than to one unconditional mean.
Quarterly growth in housing starts, measured to the end of 2018, is fitted with seasonal dummies and the fourth-quarter dummy left out. The intercept comes to -15.79 and the three quarterly coefficients to 6.23, 56.77 and 10.61.
Interval forecasts
An interval depends on the variance of the forecast error, the gap between the realisation at T plus h and the forecast made for it. In a linear trend model that error is the shock at T plus h, so with Gaussian errors the 95% interval is the point forecast plus or minus 1.96 times the shock standard deviation, estimated from the residual variance. With autoregressive or moving average components that variance grows with the horizon.
Annual euro-area real GDP, in billions of 2010 euros, is fitted with a linear trend over 1995 to 2018. The result is a constant of -234178.8 plus 121.3 times t, with t the calendar year, so the last observation carries t = 2018. Residual standard deviation comes to 262.8 and the errors are Gaussian white noise.
Forecasting levels from a log model
The log is a nonlinear and concave transformation, so by Jensen’s inequality the expected level is not the exponential of the expected log. Assuming Gaussian shocks gives a way through, because the exponential of a normal variable is log-normal and the mean of a log-normal adds half the variance inside the exponential.
A log-linear trend on the same data gives a constant of -18.15 and a slope of 0.0136, with a shock standard deviation of 0.0322. Half the variance is 0.0005, small here but wrong to omit. The expected logs are -18.15 + 0.0136 times 2019 = 9.308, then 9.322 and 9.336, and exponentiating each after adding 0.0005 gives level forecasts of 11,031.4, 11,186.9 and 11,344.6. The bounds apply to the log, so they act multiplicatively. Exponentiating plus and minus 1.96 times 0.0322, which is 0.0631, gives multipliers of 0.939 and 1.065, so the 2019 interval runs from 10,358.5 to 11,748.4.