QM 5 – Time-Series Analysis | 01 Quantitative Methods

Worked examples are fully visible. Check-yourself items are study aids you can reveal one at a time.

A time series is simply a variable observed repeatedly through time: quarterly revenue for a company over five years, the monthly average of a daily exchange rate, the annualised inflation rate month by month. Two questions drive everything in this reading. What model describes how the series has behaved? And what does that model imply about the next observation, or the one after that?

The tool we reach for is linear regression, and that is exactly where the trouble starts. Ordinary least squares carries a set of assumptions, and time-ordered data violate them routinely. Two violations matter above all others.

Correlated residuals

The first problem is serial correlation in the error term. Consider the workhorse of this reading, an autoregressive model in which the explanatory variable is the previous observation of the dependent variable itself:

A first-order autoregression. The regressor is a lagged value of the regressand.

In an ordinary cross-sectional regression, where the dependent and independent variables are genuinely distinct series, serially correlated errors leave the coefficient estimates consistent even though the standard errors are wrong. That comfortable result disappears here. When the regressor is a past value of the dependent variable, serial correlation in makes the estimates of both the intercept and the slope inconsistent. The numbers themselves become untrustworthy, not merely their standard errors.

A mean or variance that drifts

The second problem is that the statistical properties of the series change as time passes. If the average level of the series in the first half of the sample differs from its average level in the second half, or if the series is placid for a decade and then violent for two years, then a single regression fitted across the whole span is describing something that does not exist as a stable object. Estimates from such a regression carry no economic meaning.

Neither problem is fatal, because in most cases the series can be transformed until the assumptions hold. Taking natural logarithms, taking first differences, splitting the sample at a policy break, adding a lag one year back: these are the standard repairs, and most of this reading is about knowing which repair a particular series needs. The order in which observations arrive is part of the data. Unlike a cross-section, a time series cannot be shuffled, because the growth and persistence patterns that scrambling would destroy are precisely what damages the statistical properties of the estimated coefficients.

The sequence of this lesson follows the sequence an analyst would actually use. We start with trend models, move to autoregressive models and the stationarity condition they require, then handle random walks, unit roots, moving averages, seasonality, changing error variance, and finally regressions that involve more than one time series at once.

The simplest forecast of all is a straight line through the data. A linear trend model assumes the series climbs or falls by a constant amount each period:

The independent variable is time itself, coded 1 in the first sample period and rising by 1 thereafter.

Here is the intercept, is the trend coefficient (the slope of the fitted line), and is the error. Both coefficients come from ordinary least squares, which chooses the line minimising the sum of squared residuals. The fitted value in any period is obtained by substituting that period number:

Forecasting is nothing more than extending the time counter past the end of the sample.

A quick illustration of the mechanics: if equals 5.1 and equals 2, the fitted value at is 15.1 and at it is 17.1. Every step forward adds 2, regardless of where the series happens to be. That constant increment is the defining property of a linear trend, and it is also its main limitation.

Example 1 · Worked

It is January 2020. Lisette Miller, a fixed-income analyst in a bank trust department, wants a forecast of inflation. She fits a linear trend to monthly US Consumer Price Index inflation expressed as an annual percentage rate, where 1% is entered as 1.0. The sample runs from January 1995 through June 2019, giving 294 monthly observations, so the model is for to 294. With 294 observations and two parameters, the regression has 292 degrees of freedom, and the critical value for a t-statistic at the 0.05 level is 1.97.

Linear trend in monthly CPI inflation, January 1995 to June 2019

Item	Coefficient	Standard error	t-Statistic
Intercept	2.7845	0.3732	7.4611
t (Trend)	−0.0037	0.0022	−1.68
R²	0.0099
Standard error of regression	3.1912
Observations	294
Durbin–Watson	1.2145

Source data: US Bureau of Labor Statistics.

1. What is the fitted equation, and which coefficients are statistically significant?

Solution. The fitted equation is y_t = 2.7845 − 0.0037t. The intercept t-statistic of 7.4611 is far above the critical value of 1.97, so the intercept is significant. The trend t-statistic is −1.68, and its absolute value falls below 1.97, so the downward drift in inflation over this sample is not statistically significant.

2. Compute the fitted value for the first month of the sample and for the last month of the sample.

Solution. For January 1995, :
2.7845 − 0.0037(1) = 2.7808%.

For June 2019, the 294th and final month, :
2.7845 − 0.0037(294) = 2.7845 − 1.0878 = 1.697%.

Both are in-sample fitted values. Comparing them with the actual figures tells Miller how well the line describes the past, but it is not what the model is for.

3. Forecast inflation for June 2020, twelve months beyond the end of the sample.

Solution. June 2020 sits 12 periods past the last observation, so :
2.7845 − 0.0037(306) = 2.7845 − 1.1322 = 1.6523%.

This is an out-of-sample forecast, and it is the reason the model was estimated in the first place.

4. How much of the variation in monthly inflation does the model explain?

Solution. The R² is 0.0099, so the trend line accounts for 0.99% of the variation in monthly inflation. Whatever the point forecast says, the uncertainty around it is enormous.

Reading the trend coefficient in the right units

Inflation here is entered as an annual percentage rate with 1% coded as 1.0. A trend coefficient of −0.0037 therefore means the fitted annualised inflation rate falls by 0.0037 percentage points, that is by roughly a third of a basis point, in each successive month. Over the full 294-month sample that accumulates to a decline of about 1.09 percentage points, which is exactly the gap between the fitted 2.7808% at the start and 1.697% at the end. Units are the single most common source of error in trend questions, so always check what a coefficient of one would mean before interpreting the coefficient you actually have.

Where linear trends fail

Inflation happens to suit a linear trend reasonably well: the series does not sit above or below the fitted line for long stretches, and the residuals look unpredictable. Most economic and financial series are not so obliging. Their slopes and intercepts shift as the underlying economy shifts, and a single line fitted to the whole history splits the difference between regimes. The damage is often worst at the end of the sample, which is precisely where an analyst needs the model to be accurate.

Many financial series grow by a constant percentage rather than a constant amount. Company revenue compounds; a straight line cannot describe compounding. Fitting a linear trend to such data produces residuals that are persistently positive in one stretch and persistently negative in another, which is the visual signature of a misspecified model. The alternative is a log-linear trend, which fits exponential growth.

Start from the exponential form:

Exponential growth: the level of the series is an exponential function of time.

Take the ratio of consecutive observations and the intercept cancels, leaving . The proportional change from one period to the next is therefore , and it is the same in every period. That is what constant growth means. If on annual data, the series grows at 4% per year.

Taking natural logarithms of both sides turns the exponential into something ordinary least squares can handle, because the equation becomes linear in the coefficients:

The log-linear trend model. The dependent variable is the natural log of the series, not the series.

Two consequences follow, and both are examinable. First, the predicted trend value of the series itself is not but its exponential:

Always exponentiate before quoting a level forecast from a log-linear model.

Second, the slope is a continuously compounded growth rate. A value of implies a per-period growth rate of , or 5.13%, not 5%.

Figure 1: Constant amount added versus constant rate of growth

A linear trend adds the same number of units each period, so it plots as a straight line. A log-linear trend adds the same percentage each period, so it bends upward. Curvature in a plot of the raw data is the first hint that logs are needed.

Example 2 · Worked

In September 2019, technology analyst Ray Benedict models quarterly sales for Starbucks Corporation. He uses 74 observations running from the second quarter of fiscal 2001 (beginning April 2001) to the third quarter of fiscal 2019 (ending June 2019). He first fits the linear trend .

Linear trend in Starbucks quarterly sales, 74 quarterly observations

Item	Coefficient	Standard error	t-Statistic
Intercept	137.4213	82.99	1.6559
t (Trend)	80.2060	1.9231	41.7066
R²	0.9603
Standard error of regression	353.36
Observations	74
Durbin–Watson	0.40

Sales are in USD millions. Source data: Bloomberg.

On the numbers alone the fit looks excellent. Why should Benedict reject this model?

Solution. The trend coefficient carries a t-statistic of 41.7066 and the R² is 0.9603, so a candidate reading only the summary statistics would accept the model. Plotting the data against the fitted line tells a different story. Before 2008 the line sits persistently below actual sales; from then until 2015 it sits persistently above them; afterwards it wanders. The residuals are consistently positive from 2001 to 2008, consistently negative from 2008 to 2015, and positive again from 2017 to 2019.

That pattern is serial correlation in the errors, and the Durbin–Watson statistic of 0.40 confirms it. Because the model is a trend fit, the assumption of uncorrelated residuals has been violated and the reported statistics cannot be relied upon. A high R² does not rescue a misspecified model.

Example 3 · Worked

The curvature in the sales plot suggests exponential growth, so Benedict re-estimates using the natural log of sales over the same 74 quarters: .

Linear trend in the natural log of Starbucks quarterly sales

Item	Coefficient	Standard error	t-Statistic
Intercept	6.7617	0.0327	206.80
t (Trend)	0.0295	0.0008	36.875
R²	0.9771
Standard error of regression	0.1393
Observations	74
Durbin–Watson	0.26

Source data: Compustat.

1. Forecast Starbucks sales for the fourth quarter of 2019.

Solution. The fourth quarter of 2019 is period 75. Work in logs first:
ln ŷ₇₅ = 6.7617 + 0.0295(75) = 6.7617 + 2.2125 = 8.9742.

Then exponentiate to recover the level:
ŷ₇₅ = e^8.9742 = USD 7,896.7 million.

2. What quarterly growth rate does the slope imply?

Solution. The slope is a continuously compounded rate, so the implied simple growth rate per quarter is
e^0.0295 − 1 = 0.0299394, or 2.99394% per quarter.

3. What would the linear trend model of Example 2 have forecast for the same quarter, and what does the gap show?

Solution. Using the linear estimates with :
137.4213 + 80.2060(75) = 137.4213 + 6,015.45 = USD 6,152.87 million.

The linear model forecasts roughly USD 1.7 billion less than the log-linear model for the same quarter. Two models with high R² values on the same data can disagree violently out of sample, because they are extrapolating different shapes. The log of sales tracks its trend closely and does not stray above or below it for long, so the log-linear specification is the better description of this series.

The choice between the two trend specifications is not a matter of taste. It follows from how the series grows:

A series that tends to increase by a roughly constant amount each period belongs in a linear trend model.
A series that tends to increase by a roughly constant rate each period belongs in a log-linear trend model.

A plot usually settles the question in seconds. Curvature that steepens as the level rises points to logs; a broadly straight path points to the linear form.

The formal test: Durbin–Watson on the residuals

Visual inspection of a residual plot is suggestive, not conclusive. Because both trend models are single-variable regressions in which the explanatory variable is time and not a lagged dependent variable, the Durbin–Watson statistic is valid and is the correct test. If a trend model has genuinely captured the behaviour of the series, its Durbin–Watson statistic should not differ significantly from 2.0. A significantly low value indicates positive serial correlation; a significantly high value indicates negative serial correlation. Either way, the correlation left in the errors is information the model has failed to use, which means a better model exists.

Durbin–Watson evidence from the two trend models

Model	Durbin–Watson	Lower critical value at 0.05	Conclusion
Linear trend in CPI inflation (Example 1)	1.2145	above 1.77	Reject no positive serial correlation
Log-linear trend in Starbucks sales (Example 3)	0.26	above 1.60	Reject no positive serial correlation

In both cases the test statistic falls below the lower critical value, so the null hypothesis of no positive serial correlation in the errors is rejected.

Both trend models fail. For inflation, the linear trend leaves positive serial correlation in the errors, so a regression of inflation on time alone breaks the least squares assumption of uncorrelated errors. For Starbucks, the log-linear fit looked far better than the linear one, and the log of sales does hug its trend line, yet a Durbin–Watson statistic of 0.26 against a lower critical value above 1.60 is not close. The errors remain positively serially correlated.

The general lesson is the one that motivates the rest of the reading. Trend models are cheap and sometimes adequate, but they frequently leave predictable structure in the residuals. Where they do, a model that uses the past values of the series itself will do better.

If the residuals from a trend model are correlated with their own past, then the past of the series contains information the trend has not extracted. An autoregressive model puts that information to work directly by regressing the series on its own lagged values. Because the dependent and independent variables are now the same series, the usual and notation is dropped and everything is written as .

AR(1): only the most recent observation is used to predict the current one.

AR(p): the p most recent observations are used. The order p is chosen by testing, not assumed.

The stationarity requirement

The regressor in an autoregression is itself a random variable, and that is not a technicality. Estimating such a model by ordinary least squares delivers valid inference only if the series being modelled is covariance stationary, sometimes called weakly stationary. The intuition is that the statistical character of the series does not change as time passes. Formally, three conditions must hold.

Constant, finite mean. The expected value of the series is the same in every period and is finite: with . The absolute value rules out a mean that runs off to minus infinity as well as one that runs off to plus infinity.
Constant, finite variance. The variance is the same in every period and is finite.
Constant, finite autocovariance. The covariance of the series with itself a fixed number of periods away is the same in every period and is finite.

The second and third conditions can be written together:

Setting s equal to zero recovers the variance condition, because the covariance of a variable with itself is its variance.

What happens if the condition fails and the model is estimated anyway? The output is spurious. The estimate of is biased and every hypothesis test built on it is invalid. There is no salvaging the regression by adjusting standard errors; the object being estimated does not exist.

Recognising a stationary series

A plot answers the question most of the time. If the series wanders around a roughly constant level with a roughly constant spread and shows no regular within-year pattern, treating it as covariance stationary is reasonable. Monthly CPI inflation, plotted over 1995 to 2019, looks like that.

Many series encountered in investment work do not. Consider the failures on display in this reading:

Trending series. Starbucks quarterly sales rise steadily, so the mean is not constant. As a general rule, any series that a linear or log-linear trend model describes accurately is not covariance stationary, although a transformation of it may be. Macroeconomic aggregates such as income and consumption behave the same way.
Seasonal series. Monthly US retail sales that are not seasonally adjusted peak every December and collapse every January, and they also grow over time. Regular within-year movement means the mean varies by month, so the mean is not constant.
Random walks. These have a variance that grows without limit, as the next sections show.

One warning applies to every stationary series, original or transformed: stationarity in the past is not a promise about the future. A model that fitted beautifully can fail when the regime that generated the data changes.

An autoregressive model can be estimated by ordinary least squares provided the series is covariance stationary and the errors are uncorrelated. The second condition still has to be verified, but the Durbin–Watson statistic can no longer do the job. Once the explanatory variables include lagged values of the dependent variable, the Durbin–Watson test is invalid. Statistical packages still print it; ignore it.

The correct test examines the autocorrelations of the residuals. An autocorrelation is the correlation of a series with its own past, and the order gives the number of periods of lag:

The kth-order autocorrelation of a time series. Its absolute value cannot exceed 1.

Population autocorrelations are unobservable, so the sample mean replaces the population mean and the estimated autocorrelation is computed from the data:

The sample autocorrelation at lag k.

The same definition applied to the error term, whose expected value is assumed to be zero, gives the error autocorrelations. These are what we care about: if the model is correctly specified, every error autocorrelation should be zero, because a non-zero one would mean predictable structure has been left in the residuals.

The three-step procedure

Testing whether an error autocorrelation at a given lag equals zero uses a t-test built on the residual autocorrelation and its standard error. The standard error of a residual autocorrelation is remarkably simple:

T is the number of observations in the series. With 100 observations the standard error is 0.1.

The procedure is then:

Estimate a candidate autoregressive model, usually starting with AR(1).
Compute the autocorrelations of the residuals from that model.
Divide each residual autocorrelation by and compare the resulting t-statistic with the critical value. If any autocorrelation is significantly different from zero, the model is misspecified.

For seasonally unadjusted data, analysts often compute as many autocorrelations as there are periods in a year, so four for quarterly data and twelve for monthly data.

Example 4 · Worked

Analyst Melissa Jones models the gross margin of Intel Corporation, defined as (Sales minus Cost of goods sold) divided by Sales. Her sample is quarterly and spans 1Q 2003 to 2Q 2019. The correct specification is not known in advance, yet she expects this quarter figure to depend on last quarter figure, so an AR(1) is the natural starting point: Gross margin_t = b₀ + b₁(Gross margin_t−1) + ε_t.

AR(1) model of Intel gross margin, 65 quarterly observations

Item	Coefficient	Standard error	t-Statistic
Intercept	0.1513	0.0480	3.15
Gross margin_t−1	0.7462	0.0809	9.2236
R²	0.5746
Standard error of regression	0.03002
Observations	65
Durbin–Watson	1.743

Autocorrelations of the residual from the Intel AR(1) model

Lag	Autocorrelation	Standard error	t-Statistic
1	0.1308	0.1240	1.0545
2	−0.2086	0.1240	−1.6818
3	0.0382	0.1240	0.3080
4	0.0608	0.1240	0.4903

Source data: Bloomberg.

1. Verify the standard error of the residual autocorrelations and state whether the model is correctly specified.

Solution. The sample has 65 observations, so the standard error of each estimated autocorrelation is
1 ÷ √65 = 0.1240, matching the table.

With 65 observations and two parameters the model has 63 degrees of freedom, and the critical t-value at the 0.05 level is about 2.0. The largest residual autocorrelation t-statistic in absolute value is 1.6818 at lag 2, which is below 2.0. None of the first four autocorrelations differs significantly from zero, so the residuals are not serially correlated and the AR(1) specification is acceptable. The Durbin–Watson figure of 1.743 in the table is reported but must not be used, because the regressor is a lagged dependent variable.

2. Both coefficients look significant. Confirm this and write the forecasting equation.

Solution. The intercept t-statistic is about 3.2 and the slope t-statistic is 9.2236, both above the critical value of roughly 2.0, so each null hypothesis of a zero coefficient is rejected. Since the expected value of the error is zero, the forecasting equation drops the error term:
Gross margin_t+1 = 0.1513 + 0.7462(Gross margin_t).

3. Forecast next quarter gross margin if the current quarter figure is 55%, and again if it is 65%.

Solution. Enter the margin as a decimal.
At 0.55: 0.1513 + 0.7462(0.55) = 0.1513 + 0.41041 = 0.5617, or 56.17%. The margin is forecast to rise.

At 0.65: 0.1513 + 0.7462(0.65) = 0.1513 + 0.48503 = 0.6363, or 63.63%. The margin is forecast to fall.

The model pulls the series toward a particular level from either side. The next section identifies that level.

Analysts rarely want only the next period. A quarterly sales model is usually asked for the next four quarters. Producing a forecast more than one period ahead requires a small piece of machinery called the chain rule of forecasting.

The one-period-ahead forecast is direct, because the current value is known:

One period ahead, made in period t.

Two periods ahead would require , which is not yet observable. The chain rule substitutes the forecast of that value in place of the unknown actual value:

The one-period-ahead forecast becomes the input to the two-period-ahead forecast.

The same substitution can be repeated indefinitely, and the same logic extends to AR(p) models: each unknown future value in the equation is replaced by the forecast already produced for it.

Uncertainty compounds along the chain. The two-period forecast inherits the error in forecasting from and then adds the error in forecasting from the forecast of . Provided the model is well specified, its prediction errors are not serially correlated, and in that case the variance of a multiperiod forecast exceeds the variance of a single-period forecast. The further out the horizon, the wider the interval that should surround the point estimate.

Example 6 · Worked

At the beginning of 2020, an analyst wants Intel gross margin two quarters ahead using the AR(1) model from Example 4. The current quarter gross margin is 63%.

Produce the one-period-ahead and two-period-ahead forecasts.

Solution. First step, using the observed current value of 0.63:
0.1513 + 0.7462(0.63) = 0.1513 + 0.470106 = 0.6214.

Second step, substituting that forecast back into the same equation:
0.1513 + 0.7462(0.6214) = 0.1513 + 0.46369 = 0.6150.

With a current gross margin of 63%, the model predicts 61.50% two quarters out. Notice the direction: 63% is above the mean-reverting level of 59.61%, so each successive forecast steps down toward that level without ever quite arriving.

When two models both pass their specification tests, the tie is broken on forecast accuracy. The model with the smaller variance of forecast errors is the more accurate one, and it will also report the smaller standard error of the regression, a figure printed directly in standard output.

The distinction that matters is where the errors come from.

In-sample forecast errors are the residuals of a fitted model, measured over the same period used to estimate it.
Out-of-sample forecast errors are the differences between actual values and forecasts for periods outside the estimation window.

Out-of-sample accuracy is the one that matters commercially, because the future is always out of sample. Much published analysis reports only in-sample fit, which flatters a model that has been tuned to its own history.

The standard out-of-sample criterion is the root mean squared error:

The square root of the average squared forecast error. Smaller is better.

Example 7 · Worked

Analyst Lisette Miller returns to monthly US inflation, now modelled directly rather than by a trend line. Her data are annualised monthly percentage changes in the CPI over a 24-year span beginning January 1995. She first estimates an AR(1) on data to December 2018, with denoting February 1995.

AR(1) model of monthly CPI inflation at an annual rate, February 1995 to December 2018

Item	Coefficient	Standard error	t-Statistic
Intercept	0.13346	0.2134	0.6254
Inflation_t−1	0.3984	0.0544	7.3235
R²	0.1586
Standard error of regression	2.9687
Observations	287
Durbin–Watson	1.8442

The curriculum text quotes the intercept of this regression as 1.3346 and uses that value in every forecast it computes from the model, so 1.3346 is used below.

Autocorrelations of the residual from the inflation AR(1) model

Lag	Autocorrelation	Standard error	t-Statistic
1	0.0777	0.0590	1.3175
2	−0.1653	0.0590	−2.8013
3	−0.1024	0.0590	−1.7362
4	−0.0845	0.0590	1.4324

1. Is the AR(1) model correctly specified?

Solution. With 287 observations the standard error of each residual autocorrelation is 1 ÷ √287 = 0.0590. With 287 observations and two parameters the model has 285 degrees of freedom, and the critical t-value at the 0.05 level is about 1.97. The second-lag autocorrelation carries a t-statistic of −2.8013, whose absolute value exceeds 1.97. That autocorrelation is significantly different from zero, so the residuals are serially correlated and the AR(1) model is misspecified.

2. The remedy is to add a lag. Evaluate the AR(2) model below.

Solution. The AR(2) specification is Inflation_t = b₀ + b₁(Inflation_t−1) + b₂(Inflation_t−2) + ε_t, estimated over March 1995 to December 2018.

AR(2) model of monthly CPI inflation at an annual rate

Item	Coefficient	Standard error	t-Statistic
Intercept	1.5996	0.2245	7.1252
Inflation_t−1	0.4759	0.0583	8.1636
Inflation_t−2	−0.1964	0.0583	−3.368
R²	0.1907
Standard error of regression	2.9208
Observations	286
Durbin–Watson	1.9934

Autocorrelations of the residual from the inflation AR(2) model

Lag	Autocorrelation	Standard error	t-Statistic
1	0.0032	0.0591	0.0536
2	0.0042	0.0591	0.0707
3	−0.0338	0.0591	−0.5696
4	0.0155	0.0591	0.2623

With 286 observations and three parameters the model has 283 degrees of freedom and the critical value is again about 1.97. All three coefficients are significant, and none of the first four residual autocorrelations has a t-statistic anywhere near the critical value. Miller fails to reject the hypothesis that each is zero, so the AR(2) model is correctly specified.

3. Inflation last month was 4% annualised and the month before that 3%. How much does the forecast differ between the two models?

Solution. AR(1), using the intercept of 1.3346:
1.3346 + 0.3984(4) = 1.3346 + 1.5936 = 2.93% approximately.

AR(2):
1.5996 + 0.4759(4) − 0.1964(3) = 1.5996 + 1.9036 − 0.5892 = 2.91% approximately.

The misspecified AR(1) model forecasts inflation 2 basis points higher. The gap is small here, but an incorrect model applied to a larger portfolio decision would not always be so forgiving.

4. Compare the two models on in-sample forecast error.

Solution. Compare the standard errors of the regressions: 2.9687 for AR(1) against 2.9208 for AR(2). The AR(2) figure is smaller, so it has the lower in-sample forecast error variance. As a ratio,
2.9208 ÷ 2.9687 = 98.39%.

The AR(2) in-sample forecast error is a little over 98% of that from the AR(1) model, which agrees with the specification test.

Example 8 · Worked

The same two models, estimated over 1995 to 2018, are now judged on data they never saw: annualised US CPI inflation for the nine months from January 2019 to September 2019.

Out-of-sample forecast errors, January 2019 to September 2019

Month	Infl(t)	Infl(t−1)	Infl(t−2)	AR(1) error	Squared	AR(2) error	Squared
January	0.0000	0.0000	0.0000	0.1335	0.0178	−1.6000	2.5599
February	2.4266	0.0000	0.0000	−2.2931	5.2585	0.8266	0.6833
March	4.9070	2.4266	0.0000	−3.8068	14.4916	2.1522	4.6320
April	3.6600	4.9070	2.4266	−1.5716	2.4699	0.2014	0.0406
May	1.2066	3.6600	4.9070	0.3850	0.1482	−1.1714	1.3722
June	1.2066	1.2066	3.6600	−0.5924	0.3510	−0.2488	0.0619
July	3.6600	1.2066	1.2066	−3.0458	9.2770	1.7228	2.9680
August	1.2066	3.6600	1.2066	0.3850	0.1482	−1.8982	3.6030
September	0.0000	1.2066	3.6600	0.6142	0.3772	−1.4554	2.1181
Average squared error					3.6155		2.0043
RMSE					1.9014		1.4157

Small mismatches between an error column and its squared-error column reflect rounding. Source data: US Bureau of Labor Statistics.

1. Verify the AR(1) error for March 2019.

Solution. February inflation was 2.4266, so the AR(1) forecast for March is
0.13346 + 0.3984(2.4266) = 0.13346 + 0.96676 = 1.10022.

Actual March inflation was 4.9070. The tabulated AR(1) column measures forecast minus actual:
1.10022 − 4.9070 = −3.8068, matching the table. Squaring gives 14.4916. Note that the AR(2) column is signed the other way, as actual minus forecast, which leaves the squared errors unaffected.

2. Verify both RMSE figures and state which model forecasts better out of sample.

Solution. Sum the nine AR(1) squared errors and divide by nine:
32.5394 ÷ 9 = 3.6155, and √3.6155 = 1.9014.

For AR(2):
18.0390 ÷ 9 = 2.0043, and √2.0043 = 1.4157.

As a ratio, 1.4157 ÷ 1.9014 = 74.46%. The AR(2) root mean squared error is under three-quarters of the AR(1) figure, so the AR(2) model was more accurate out of sample as well as in sample. Nine observations is a thin basis for the comparison, and coefficient stability, covered next, is a further consideration.

The estimates of a time-series model are not properties of the world; they are properties of the sample. Move the window and the coefficients move with it. An earlier sample can produce coefficients quite unlike those from a later sample, and a longer sample can produce coefficients unlike those from a shorter one. The effect goes beyond the numbers to the specification itself: an AR(1) model may serve for one window while an AR(2) model is required for another.

Economic theory offers little guidance on how long a window to use. What it does offer is the stationarity requirement. Because the model is valid only for a covariance-stationary series, the sample should not span a change in the data-generating regime. Two classic examples make the point. Data from a fixed exchange rate era should not be pooled with data from a floating era, since exchange rate volatility is structurally higher when rates float. Similarly, many analysts regard US inflation and interest rates since the 1960s as spanning several distinct Federal Reserve policy regimes, and therefore refuse to model the whole span as one sample.

Practical procedure: plot the data and look for a level shift or a change in volatility before estimating anything. Where a policy change has a known date, test whether the relationship differs on either side of it.

Example 9 · Worked

A colleague challenges Miller’s decision to model US inflation as a single AR(2) series from 1995 to 2018, arguing that the aggressive Federal Reserve response to the crisis that began in 2007 split the sample into two regimes: one to 2007 and another from 2008. Miller re-estimates an AR(1) model on data from January 2008 to December 2018.

AR(1) model of monthly CPI inflation at an annual rate, January 2008 to December 2018

Item	Coefficient	Standard error	t-Statistic
Intercept	0.8431	0.2969	2.8397
Inflation_t−1	0.5036	0.0758	6.6438
R²	0.2536
Standard error of regression	3.0742
Observations	132
Durbin–Watson	1.8164

Autocorrelations of the residual, 2008 to 2018 sample

Lag	Autocorrelation	Standard error	t-Statistic
1	0.0999	0.087	1.1479
2	−0.1045	0.087	−1.2015
3	−0.1568	0.087	−1.8051
4	0.0500	0.087	0.5750

Source data: US Bureau of Labor Statistics.

1. Is an AR(2) model needed for the shorter sample?

Solution. No. The standard error of the residual autocorrelations is 1 ÷ √132 = 0.087, matching the table. The critical t-value for significance here is 1.99, and the largest absolute t-statistic among the first four autocorrelations is 1.8051. Miller cannot reject the null hypothesis that the residuals are serially uncorrelated, so the AR(1) model is adequate for 2008 to 2018. That is the opposite of the conclusion reached on the full 1995 to 2018 sample, where AR(1) residuals were serially correlated and an AR(2) model appeared necessary.

2. Last month inflation was 4% annualised and the month before 3%. Compare the forecast from the shorter-sample model with the AR(2) forecast from the longer sample.

Solution. Shorter sample AR(1):
0.8431 + 0.5036(4) = 0.8431 + 2.0144 ≈ 2.86%.

Longer sample AR(2), from Example 7: 2.91%.

The correctly specified model for the shorter sample forecasts 0.05 percentage points below the correctly specified model for the longer sample. Both models pass their diagnostics, yet they disagree, and the disagreement is large enough to matter for an investment decision.

3. Which sample should be used?

Solution. The shorter one. Monthly US inflation was far more volatile in the middle of the 1995 to 2018 period than at either end, which means the series is probably not covariance stationary across the full span. The evidence is direct: the standard deviation of annualised monthly inflation was 2.86% over 1995 to 2007 but 3.54% over 2008 to 2018, largely because of the volatility around the 2008 crisis. More than one regime is present, so a separate model for the later period is the right choice. Residual autocorrelations alone could never have revealed this; knowledge of policy history and judgement did.

Not every series reverts to a level. In a random walk, each observation equals the previous one plus a fresh, unforecastable shock:

A simple random walk, that is, a random walk without drift.

Two observations follow at once. First, this is the AR(1) model with the intercept set to zero and the slope set to one. Second, because the shock has an expected value of zero, the best available forecast of the next value is today value, and indeed today value remains the best forecast for every period beyond that. Currency exchange rates are the classic example. A long research literature finds that elaborate forecasting models fail to beat the current spot rate as a predictor of the future spot rate.

Why an AR model cannot be estimated on a random walk

A random walk fails covariance stationarity on two counts.

No mean-reverting level. The AR(1) mean-reverting level is . With and this becomes 0 divided by 0, which is undefined.

Variance without an upper bound. Suppose the series starts at . Then , whose variance is . Next, , and because the shocks are mutually uncorrelated the variance is . Continuing this argument, the variance in period is , which grows without limit as time passes. A covariance-stationary series must have finite variance, so a random walk is not covariance stationary.

Figure 3: Constant spread versus a spread that widens forever

A stationary series stays inside a band whose width does not depend on how long you watch. A random walk has no such band: the dispersion around its starting point widens in proportion to elapsed time, so no finite variance exists.

First differencing

Standard regression cannot be used on a random walk, but the series can often be converted into one that is well behaved. Define a new series equal to the change from one period to the next:

First differencing subtracts the immediately preceding observation from the current one.

Applying this to a random walk leaves . That is an AR(1) model with both the intercept and the slope equal to zero, so its mean-reverting level is 0 divided by 1, that is zero, and its variance is in every period. Both the mean and the variance are constant and finite, so the differenced series is covariance stationary and can be modelled by linear regression.

The reward is diagnostic rather than predictive. Since both coefficients are zero, the differenced model forecasts no change, which is what a random walk implies in the first place. What the exercise establishes is that the original series is a random walk, and that is worth knowing.

Random walk with drift

Adding a non-zero intercept produces a random walk with drift, a series that tends to rise or fall by a constant amount each period on top of the random shock:

A random walk with drift is the AR(1) model with a slope of one and a non-zero intercept.

Because the slope is still one, the mean-reverting level is still undefined and the series is still nonstationary. First differencing gives with a non-zero intercept, so the differenced series fluctuates around the drift term.

Example 10 · Worked

An AR(1) model is fitted to the Japanese yen per US dollar exchange rate using month-end observations from October 1980 through August 2019, and then to the first difference of that series.

JPY/USD exchange rate: levels AR(1) and first-differenced AR(1)

Item	Levels model	First-differenced model
Intercept (coefficient)	0.8409	−0.2185
Intercept (standard error)	0.6503	0.2142
Intercept (t-statistic)	1.2931	−1.0200
Lagged term (coefficient)	0.9919	0.0287
Lagged term (standard error)	0.0047	0.0464
Lagged term (t-statistic)	211.0426	0.6185
R²	0.9897	0.0008
Standard error of regression	4.5999	4.6177
Observations	467	466
Durbin–Watson	1.9391	2.0075

Residual autocorrelations at the first four lags are 0.0302, 0.0741, 0.0427 and −0.0034 for the levels model, and −0.0023, 0.0724, 0.0387 and −0.0062 for the differenced model. Source data: US Federal Reserve Board of Governors.

1. What do the levels estimates suggest, and can the reported t-statistics be trusted?

Solution. The intercept of 0.8409 has a t-statistic of only 1.2931 and is not significantly different from zero, while the slope of 0.9919 is very close to one. That is the signature of a random walk. The t-statistics themselves, however, cannot be used to test the hypothesis, because if the series really is a random walk it is not covariance stationary and the standard errors from the AR estimation are invalid. Testing this properly requires the unit root test of the next section.

2. What do the differenced estimates show?

Solution. Neither the intercept (t-statistic −1.0200) nor the coefficient on the lagged change (t-statistic 0.6185) differs significantly from zero, and no residual autocorrelation is significant. Both coefficients being zero, with uncorrelated errors, is exactly the pattern a random walk implies. The evidence is consistent with the JPY/USD rate being a random walk.

Why the R-squared comparison points the wrong way

The levels model has an R² of 0.9897 and the differenced model an R² of 0.0008, yet the differenced model is the one to use. There is no contradiction. The levels regression measures how well this month exchange rate predicts next month exchange rate, and for a random walk that is almost perfect by construction, since the level barely moves relative to its own size. The differenced regression measures whether last month change predicts this month change, and for a random walk it should predict nothing at all, so a near-zero R² is the expected and correct result. The general rule is that R² cannot be used to choose between a model in levels and a model in differences. It also follows that no trading strategy can profit from forecasting changes in a genuine random walk.

The traditional way to judge stationarity is to inspect the autocorrelations of the series at successive lags. For a stationary series, the autocorrelations are either statistically indistinguishable from zero at all lags or they fall away quickly as the lag lengthens. A nonstationary series behaves otherwise. This inspection is informal, and a sharper tool exists.

Frame the question through the AR(1) model. For the series to be covariance stationary, the absolute value of the lag coefficient must be below 1.0. If that coefficient equals exactly 1.0, the series carries a unit root, which makes it a random walk and rules out covariance stationarity. If the absolute value exceeds 1.0 the series has an explosive root. Every random walk, with or without drift, has a unit root, and any series with a unit root fails covariance stationarity.

The tempting approach is to estimate the AR(1) model and run a t-test of the hypothesis that the slope equals one. That approach is invalid, and for a reason worth memorising: if the slope really is one, the series is not covariance stationary and the estimated coefficient does not follow a t-distribution, so the t-test has no basis.

The transformation that makes the test work

Dickey and Fuller (1979) solved the problem by rearranging the model. Subtract the lagged value from both sides of the AR(1) equation:

Both sides of the AR(1) model, less the lagged value.

The Dickey–Fuller regression: the first difference on the left, the lagged level on the right.

Now a slope of one in the original model corresponds to in the transformed one, so testing whether the transformed coefficient is zero is the same as testing for a unit root.

The Dickey–Fuller test at a glance

Element	Detail
Regression	First difference of the series on a constant and the first lag of the level
Null hypothesis	g₁ = 0, the series has a unit root and is nonstationary
Alternative hypothesis	g₁ < 0, the series has no unit root and is stationary
Test statistic	The conventional t-statistic on the estimated g₁
Critical values	Revised values from Dickey and Fuller, larger in absolute value than conventional t critical values

Using conventional critical values would reject the unit root far too often. Most econometric packages implement the corrected values.

Example 11 · Worked

In January 2012, equity analyst Aron Berglin models the quarterly sales of AstraZeneca, a British and Swedish biopharmaceutical company headquartered in London, using sales in US dollars from January 2000 to December 2011 plus earlier observations where lags require them. A log-linear trend fits better than a linear trend, but the Durbin–Watson statistic from the log-linear regression is only 0.7064, so he rejects the hypothesis of serially uncorrelated errors and abandons the trend model. He proposes an AR(1) in the log of sales.

1. What must Berglin do before estimating that autoregression?

Solution. He must apply the Dickey–Fuller test for a unit root in the log of quarterly sales. If the test fails to reject the null hypothesis, the log of sales has a unit root, it is not covariance stationary, and an AR model estimated on it would be invalid. On this series the test statistic would indeed leave him unable to reject the null hypothesis of a unit root.

2. What is the remedy?

Solution. Model the first difference as an autoregression instead of the level. Differencing the log of sales removes the strong upward trend that dominates the level series, leaving something that can plausibly be treated as covariance stationary.

Example 12 · Worked

Berglin therefore estimates an AR(1) on the log difference of AstraZeneca quarterly sales, that is on ln(Sales_t) − ln(Sales_t−1) regressed on ln(Sales_t−1) − ln(Sales_t−2), over January 2000 to December 2011.

AR(1) model of log differenced AstraZeneca quarterly sales

Item	Coefficient	Standard error	t-Statistic
Intercept	0.0222	0.0071	3.1268
ln Sales_t−1 − ln Sales_t−2	−0.5493	0.1236	−4.4442
R²	0.3005
Standard error of regression	0.0475
Observations	48
Durbin–Watson	1.6874

Autocorrelations of the residual, AstraZeneca log differenced model

Lag	Autocorrelation	Standard error	t-Statistic
1	0.2809	0.1443	1.9466
2	−0.0466	0.1443	−0.3229
3	0.0081	0.1443	0.0561
4	0.2647	0.1443	1.8344

Source data: Compustat.

1. Is the model well specified?

Solution. With 48 observations the standard error of each residual autocorrelation is 1 ÷ √48 = 0.1443, matching the table. With 48 observations and two parameters there are 46 degrees of freedom, and the critical t-value at the 0.05 level is above 2.0. The largest absolute t-statistic among the first four autocorrelations is 1.9466, below 2.0, so no significant residual autocorrelation is present and the model can be used. Both coefficients are individually significant.

2. Interpret the two coefficients.

Solution. The intercept of 0.0222 says that if sales were unchanged this quarter, so that the differenced variable is zero, sales are predicted to grow by 2.22% next quarter. The slope of −0.5493 says the model subtracts 0.5493 times this quarter growth from that baseline. Growth this quarter therefore reduces predicted growth next quarter, which is a mean-reverting pattern in the growth rate. All the growth rates here are exponential, that is continuously compounded, growth rates.

3. AstraZeneca sales were USD 8,405 million in the third quarter of 2011 and USD 8,872 million in the fourth. Forecast sales for the first quarter of 2012.

Solution. Let be the fourth quarter of 2011, so is the third quarter of 2011 and is the first quarter of 2012.

Step 1, build the current differenced value:
ln 8,405 = 9.0366 and ln 8,872 = 9.0907, so y_t = 9.0907 − 9.0366 = 0.0541.

Step 2, forecast the next differenced value:
ŷ_t+1 = 0.0222 − 0.5493(0.0541) = 0.0222 − 0.029717 = −0.0075.

Step 3, convert the log change back into a level. Since ŷ_t+1 equals ln(Sales_t+1 ÷ Sales_t), exponentiate:
e^−0.0075 = 0.9925.

Step 4, multiply by the latest actual sales figure:
Sales_t+1 = USD 8,872 million × 0.9925 = USD 8,805 million.

The model predicts a slight decline in AstraZeneca sales in the first quarter of 2012.

Two quite different things share the name moving average. One is a smoothing device applied to observed data. The other is a forecasting model built on unobserved errors. Confusing them is a standard trap.

Smoothing with an n-period moving average

To see a long-term pattern through short-term noise, replace each observation with the average of itself and the previous observations:

An n-period moving average of a time series.

Example 13 · Worked

AstraZeneca quarterly sales in 2011 were USD 8,490 million in the first quarter, USD 8,601 million in the second, USD 8,405 million in the third and USD 8,872 million in the fourth.

Compute the four-quarter moving average of sales as at the beginning of the first quarter of 2012.

Solution.
(8,490 + 8,601 + 8,405 + 8,872) ÷ 4 = 34,368 ÷ 4 = USD 8,592 million.

Applied to monthly US retail sales, where December spikes and January collapses every year, a twelve-month moving average strips out the seasonal swing and reveals the underlying path: steady growth through the second half of 2008, roughly a year of decline, then steady growth again. Applied to monthly Brent crude oil spot prices, which have no comparable seasonality, it still smooths the month-to-month noise into a longer movement.

The weakness is equally visible in the oil price series. A simple moving average always lags a large move, because it gives equal weight to every period in the window. When prices climbed sharply through late 2007 and the first half of 2008 the average rose only gradually, and when they fell late in 2008 the average again trailed behind. Useful for seeing the past; poor for predicting the future.

Moving-average models for forecasting

The forecasting version is a different object entirely. A moving-average model of order 1 writes the series as a weighted combination of the current and previous random shocks:

An MA(1) model. Theta is the model parameter; the errors are uncorrelated with zero mean and constant variance.

Unlike the simple average, this places unequal weights on the two terms: 1 on the current shock and on the previous one. Note also that the errors are never observed directly, which is what makes these models awkward to fit.

The general form uses lagged errors with individual weights:

An MA(q) model.

Telling AR and MA processes apart

The autocorrelations of the series settle the question, and the distinction is sharp.

For an MA(q) process, the first autocorrelations are significantly different from zero and every autocorrelation beyond lag is zero. The process has a memory of exactly periods. In an MA(1) series, any observation is correlated with the observations immediately before and after it and with nothing else.
For an autoregressive process, the autocorrelations start large and decay gradually rather than stopping abruptly.

This cut-off against gradual-decay contrast is the practical rule for choosing between the two families, and it also fixes the order when the moving-average family wins. In practice most financial series are better described by autoregressive models.

Figure 4: Autocorrelation signatures of the two model families

Read the autocorrelations before choosing a family. Spikes that fade slowly point to an autoregressive model; spikes that stop dead after lag q point to a moving-average model of order q.

Example 14 · Worked

The S&P BSE 100 Index tracks the largest 100 companies listed on BSE Ltd., formerly the Bombay Stock Exchange. The first six autocorrelations of monthly returns, computed from January 2000 to December 2013 on 168 observations, are shown below.

Autocorrelations of annualised monthly returns, S&P BSE 100

Lag	Autocorrelation	Standard error	t-Statistic
1	0.1103	0.0772	1.4288
2	−0.0045	0.0772	−0.0583
3	0.0327	0.0772	0.4236
4	0.0370	0.0772	0.4793
5	−0.0218	0.0772	−0.2824
6	0.0191	0.0772	0.2474

Source data: BSE Ltd.

What model describes these returns?

Solution. With 168 observations the standard error of each autocorrelation is 1 ÷ √168 = 0.0772, as reported. The critical t-value is about 1.98 at the 0.05 level, and the largest absolute t-statistic is 1.4288 at lag 1. None of the six autocorrelations differs significantly from zero.

Since an MA(q) process would show significant autocorrelations, and none is significant, the returns behave as MA(0). Allowing a non-zero mean, that model is x_t = μ + ε_t, which is not predictable at all.

The same evidence rules out the autoregressive family. An AR(1) process would show a significant first autocorrelation followed by a gradual decay, and not even the first autocorrelation is significant. Higher-order AR models are excluded for the same reason. The conclusion, that short-horizon index returns resist prediction, is what most of the research literature reports.

Seasonality means a regular pattern of movement within the year: a retailer whose fourth quarter always dwarfs its first, or a furniture chain whose sales spike in the same month every year. At first sight seasonality looks incompatible with autoregressive modelling, because the autocorrelations then differ from season to season. It is not. The fix is a seasonal lag.

Detecting it

Seasonality announces itself in the residual autocorrelations of a model fitted without it. Look at the autocorrelation at the seasonal lag: the fourth for quarterly data, the twelfth for monthly data. If that autocorrelation is significantly different from zero while the model contains no seasonal term, the model is misspecified and must not be used for forecasting.

Correcting it

Add the value from one year earlier as an extra explanatory variable. For quarterly data on an AR(1) base:

An AR(1) model with a seasonal lag, for quarterly observations.

The forecast made in period for period from this specification is:

Note the third lag, not the fourth: from the standpoint of period t+1, the observation one year earlier is x at t minus 3.

For monthly data the seasonal term is the twelfth lag. Re-estimate, then check the residual autocorrelations again, including the seasonal one, to confirm the correction worked.

Example 15 · Worked

Return to Starbucks. The log-linear trend failed on its Durbin–Watson statistic, so the natural next step is to remove the exponential trend by first-differencing the log of sales and fitting an AR(1) to the result, using quarterly data from the last quarter of 2001 to the second quarter of 2019.

AR(1) on log differenced Starbucks sales, without a seasonal lag

Item	Coefficient	Standard error	t-Statistic
Intercept	0.0469	0.0080	5.8625
ln Sales_t−1 − ln Sales_t−2	−0.4533	0.1069	−4.2404
R²	0.2044
Standard error of regression	0.0611
Observations	72
Durbin–Watson	1.9904

Residual autocorrelations without the seasonal lag

Lag	Autocorrelation	Standard error	t-Statistic
1	0.0051	0.1179	−0.0433
2	−0.1676	0.1179	−1.4218
3	−0.0130	0.1179	−0.1099
4	0.7630	0.1179	6.4720

Source data: Bloomberg.

1. Why must this model be rejected?

Solution. With 72 observations the standard error of each residual autocorrelation is 1 ÷ √72 = 0.1179, as reported. With 72 observations and two parameters there are 70 degrees of freedom and the critical t-value is about 1.99 at the 0.05 level.

The fourth autocorrelation, which is the seasonal one for quarterly data, is 0.7630 with a t-statistic of 6.4720. That is far above 1.99, so the null hypothesis of a zero fourth autocorrelation is rejected. Strong seasonality remains in the residuals, the model is misspecified, and it must not be used to forecast.

2. Add the seasonal lag and reassess.

Solution. The revised model regresses the log difference on its own first lag and on the log difference four quarters earlier, over 2002 to 2019.

AR(1) with seasonal lag on log differenced Starbucks sales

Item	Coefficient	Standard error	t-Statistic
Intercept	0.0107	0.0059	1.8136
ln Sales_t−1 − ln Sales_t−2	−0.1540	0.0729	−2.1125
ln Sales_t−4 − ln Sales_t−5	0.7549	0.0720	10.4847
R²	0.7032
Standard error of regression	0.0373
Observations	69
Durbin–Watson	2.0392

Residual autocorrelations with the seasonal lag

Lag	Autocorrelation	Standard error	t-Statistic
1	0.0135	0.1204	0.1121
2	−0.0171	0.1204	−0.1420
3	0.1589	0.1204	1.3198
4	−0.1498	0.1204	−1.2442

With 69 observations the autocorrelation standard error is 1 ÷ √69 = 0.1204, as reported. No t-statistic among the first four autocorrelations is significant, and the overall regression is significant at the 0.01 level, so the model is a reasonable working specification. The R² rises from 0.2044 to 0.7032, more than three times higher, so the seasonal lag explains a great deal of what the first model missed. Note also that the sample shrinks from 72 to 69 observations because five lags are now required rather than two.

3. Sales grew by 1% last quarter and by 2% four quarters ago. Forecast this quarter growth.

Solution. Substitute both growth rates as decimals:
0.0107 − 0.1540(0.01) + 0.7549(0.02) = 0.0107 − 0.00154 + 0.015098 = 0.0243, or 2.43%.

Had both prior growth rates been zero, the forecast would simply be the intercept, 0.0107 or 1.07%. All of these are exponential growth rates.

Example 16 · Worked

An analyst wants to forecast monthly sales growth for Canadian retailers of furniture and home furnishings, working with data that have not been seasonally adjusted, in order to decide whether to recommend the shares. An AR(1) is fitted to annualised monthly sales growth from January 1995 to December 2012.

AR(1) and AR(1) with a twelfth seasonal lag, Canadian furniture retail sales growth

Item	Without seasonal lag	With seasonal lag
Intercept (coefficient)	1.0518	0.2371
Intercept (standard error)	0.1365	0.0900
Intercept (t-statistic)	7.7055	2.6344
Sales growth_t−1 (coefficient)	−0.2252	−0.0792
Sales growth_t−1 (standard error)	0.0665	0.0398
Sales growth_t−1 (t-statistic)	−3.3865	−1.9899
Sales growth_t−12 (coefficient)		0.7798
Sales growth_t−12 (standard error)		0.0388
Sales growth_t−12 (t-statistic)		20.0979
R²	0.0509	0.6724
Standard error of regression	1.8198	1.0717
Observations	216	216
Durbin–Watson	2.0956	2.1784
Twelfth residual autocorrelation	0.7620	−0.1168
t-Statistic on that autocorrelation	11.2059	−1.7176

The residual autocorrelation standard error is 0.0680 in both models. Prior observations are used for the lags, so the observation count does not fall. Source data: Statistics Canada (Government of Canada).

1. Diagnose the model without the seasonal lag.

Solution. With 216 observations the autocorrelation standard error is 1 ÷ √216 = 0.0680, as reported, and with two parameters the model has 214 degrees of freedom, giving a critical t-value of about 1.97. The twelfth residual autocorrelation, the seasonal one for monthly data, is 0.7620 with a t-statistic of 11.2059. The null hypothesis of a zero twelfth autocorrelation is emphatically rejected, and several other lags are significant too. The model is misspecified and cannot be used.

2. Diagnose the model with the seasonal lag.

Solution. Adding the twelfth lag pulls the seasonal residual autocorrelation down to −0.1168, with a t-statistic of −1.7176 whose absolute value is below the critical 1.97. None of the first twelve residual autocorrelations is significant, so no meaningful serial correlation remains. The R² rises from 0.0509 to 0.6724. The model can now be used to forecast.

3. Sales grew at an annual rate of 10% last month and 5% twelve months ago. Forecast the current month growth rate.

Solution.
0.2371 − 0.0792(0.10) + 0.7798(0.05) = 0.2371 − 0.00792 + 0.03899 = 0.2682, or 26.8% at an annual rate.

Had both inputs been zero, the model would forecast growth of about 23.7% at an annual rate, which is simply the intercept.

Two further topics complete the single-series toolkit: a hybrid specification that combines the autoregressive and moving-average families, and a treatment of error variance that changes through time.

Autoregressive moving-average models

An ARMA(p, q) model includes lags of the series and lagged error terms:

The b terms are autoregressive parameters and the theta terms are moving-average parameters.

Advocates argue that the extra flexibility improves fit and forecasts. The curriculum is sceptical, and the reasons are worth knowing because they are examinable.

Unstable parameters. A small change in the data sample, or even in the starting values supplied to the estimation routine, can produce very different final estimates.
Order selection is a matter of judgement. The criteria for choosing and are imperfect, and a chosen model may still forecast poorly.
Data hunger. Even sympathetic authors regard fewer than 80 observations as too few for an ARMA model, and they advise against applying one to company quarterly sales or gross margins even where fifteen years of quarterly data exist.

In most cases a plain autoregressive model, with a seasonal lag where needed, forecasts about as well with far less complexity. Treat claims that a particular ARMA specification beats all others with suspicion.

Autoregressive conditional heteroskedasticity

Everything so far has assumed homoskedasticity, meaning that the variance of the error term does not depend on anything else. Heteroskedasticity is the opposite case, where the error variance depends on the explanatory variable. In a time-series setting a particular form of this shows up repeatedly: the variance of the current error depends on how large recent errors were. Robert F. Engle first proposed a test for it in 1982, work recognised in part by his shared Nobel Prize in Economics in 2003, and he named the phenomenon autoregressive conditional heteroskedasticity.

The consequence is serious. If the error variance is not constant, any AR, MA or ARMA model built on the series reports coefficient standard errors that are wrong, so hypothesis tests built on them are invalid. Lagged terms can appear statistically significant when they are not, and a model built on those terms leads to poor decisions.

The first-order version conditions the distribution of the error on the previous error:

ARCH(1). If the coefficient on the lagged squared error is zero, the variance is constant at a-nought.

When that coefficient is positive, a large shock in one period raises the expected size of the shock in the next, which is the volatility clustering familiar from financial markets.

Testing for ARCH(1) and using the result

Engle showed that the test is a simple auxiliary regression. Take the squared residuals from a previously estimated time-series model and regress them on a constant and their own first lag:

The ARCH(1) test regression. If the estimate of a-one is significantly different from zero, the series has ARCH(1) errors.

The payoff is that the error variance itself becomes forecastable:

A one-period-ahead variance forecast from an estimated ARCH(1) model.

Example 17 · Worked

Miller tests whether the residuals of her AR(1) inflation model display ARCH(1). She runs the test twice, once on the full 1995 to 2018 sample and once on the shorter 2008 to 2018 sample. Because the test uses the first lag of the residuals, each test has one observation fewer than the model it examines.

ARCH(1) test regressions on residuals from AR(1) models of monthly CPI inflation

Item	March 1995 to December 2018	February 2008 to December 2018
Intercept (coefficient)	6.3626	6.2082
Intercept (standard error)	1.4928	2.2873
Intercept (t-statistic)	4.2622	2.7142
Lagged squared residual (coefficient)	0.2754	0.3336
Lagged squared residual (standard error)	0.0570	0.0830
Lagged squared residual (t-statistic)	4.8316	4.0229
R²	0.0759	0.1113
Standard error of regression	23.7841	24.64
Observations	286	131
Durbin–Watson	2.0569	2.0385

Source data: US Bureau of Labor Statistics.

1. What do the two tests conclude?

Solution. On the full sample the coefficient on the previous period squared residual has a t-statistic of 4.8316, comfortably above any conventional critical value, so the null hypothesis that the error variance is independent of past errors is rejected. On the shorter sample the corresponding t-statistic is 4.0229, and the same conclusion follows. In both windows the errors are heteroskedastic, so the t-statistics from the underlying inflation models are not reliable and should not drive an investment strategy.

2. Using the full-sample estimates, forecast the error variance next period if the current error is 0% and if it is 1%.

Solution. The estimated variance equation is .

With a current error of 0%:
6.3626 + 0.2754(0) = 6.3626.

With a current error of 1%, the squared error is 1:
6.3626 + 0.2754(1) = 6.6380.

A single one-percentage-point error raises the forecast error variance by about 4.3%.

Beyond ARCH(1)

Two consequences and two extensions round out the topic. If ARCH is present, the regression standard errors must be repaired, typically by generalized least squares or another heteroskedasticity correction. And if the ARCH structure has been modelled, the variance can be predicted, which is directly useful for risk work.

The extensions generalise the same idea. An ARCH(p) model lets the current error variance depend on the squared errors of the previous periods:

ARCH of order p.

Generalized autoregressive conditional heteroskedasticity models, or GARCH, stand in the same relation to the error variance that ARMA models stand in to the series itself. They inherit the same fragility: results can depend heavily on the sample window and on the starting values chosen for estimation. Anyone applying GARCH should check that the estimates survive changes to both.

Earlier regression work routinely put one time series on the left of an equation and another on the right, and ignored unit roots entirely. That is a mistake with consequences. A series containing a unit root is not covariance stationary, and if any series in a linear regression contains a unit root, the ordinary least squares test statistics may be meaningless. Before running such a regression, each series must be analyzed for nonstationarity, and where nonstationarity is present, for cointegration.

Start with the two-series case: one dependent variable and one independent variable. Apply a unit root test, typically Dickey–Fuller, separately to each. Five scenarios can arise.

The five scenarios for a regression of one time series on another

Scenario	Unit root in dependent?	Unit root in independent?	Can linear regression be used?
1	No	No	Yes, safely
2	Yes	No	No
3	No	Yes	No
4	Yes	Yes, but not cointegrated	No
5	Yes	Yes, and cointegrated	Yes, with care

Scenarios 2, 3 and 4 all fail for the same reason: the regression error term is not covariance stationary.

In the mixed cases, scenarios 2 and 3, the error term of the regression cannot be covariance stationary. At least one of three regression assumptions is then violated: that the expected value of the error is zero, that its variance is the same for every observation, and that errors are uncorrelated across observations. Coefficients and standard errors become inconsistent, and a coefficient may look highly significant while meaning nothing. Granger and Newbold (1974) named this the spurious regression problem.

Cointegration

Scenario 5 is the interesting one. Two series are cointegrated when a long-run economic or financial relationship ties them together so that they do not drift arbitrarily far apart over time. A shared common trend is the usual mechanism. When two series with unit roots are cointegrated, the error term of a regression of one on the other is covariance stationary, so the coefficients and standard errors are consistent and hypothesis tests are meaningful.

One qualification matters. A cointegrating regression estimates the long-run relationship between the series. It is not necessarily a good description of their short-run dynamics, for which error correction models exist as a specialist topic.

Figure 5: Two nonstationary series that share a trend

Cointegration is not about the individual series, which may wander without bound. It is about the combination: if a linear combination of them stays inside a stable band, a regression of one on the other has a covariance-stationary error and can be interpreted.

The Engle–Granger test

Where both series have a unit root, cointegration is tested as follows.

Estimate the regression .
Apply a Dickey–Fuller test for a unit root to the residuals of that regression. Because the residuals depend on estimated coefficients, the ordinary Dickey–Fuller critical values are not appropriate. Use the values computed by Engle and Granger, which allow for the uncertainty in the regression parameters.
If the test does not reject the null hypothesis of a unit root in the error term, the error is not covariance stationary, the two series are not cointegrated, and any apparent relationship between them is spurious.
If the test rejects that null hypothesis, treat the error as covariance stationary and the series as cointegrated. The parameters and standard errors are then consistent and support hypothesis tests about the long-run relationship.

Example 18 · Worked

Three short cases, each testing whether the analyst may proceed to a regression.

1. Researchers examine the Fisher effect by regressing US Treasury bill returns on expected inflation, using 181 quarterly observations from the fourth quarter of 1968 through the fourth quarter of 2013. What must they establish first?

Solution. They must run a Dickey–Fuller test on each series separately. If they can reject the null hypothesis of a unit root for the T-bill return series and reject it for the expected inflation series, then both are covariance stationary, this is scenario 1, and the linear regression results are valid. Rejecting for one series only would place them in scenario 2 or 3, where the regression must not be used.

2. Johann de Vries regresses the monthly percentage change in the Johannesburg Stock Exchange All Share Index on the lagged price-to-earnings ratio of the index, using data from January 1994 to December 2013. The coefficient is negative and significant and the R² is high. What should he do before accepting the result?

Solution. Run unit root tests on both series. A high R² and a significant coefficient are exactly what a spurious regression produces. If either series has a unit root, and there is published evidence that the price-to-earnings series is nonstationary, the regression cannot support a conclusion that returns are predictable from the P/E ratio.

3. An analyst wants to know whether the natural log of Intel sales and the natural log of nominal GDP are cointegrated, using quarterly data from the first quarter of 1995 through the fourth quarter of 2019. Set out the steps.

Solution. First, test each series for a unit root. If the null hypothesis of a unit root cannot be rejected for either, both are nonstationary and the cointegration question arises.

Second, estimate ln Intel sales_t = b₀ + b₁(ln GDP_t) + ε_t and apply the Engle–Granger version of the Dickey–Fuller test to the residuals. Rejecting the null hypothesis of a unit root in the residuals means rejecting the null hypothesis of no cointegration, so the two series are cointegrated and the regression can be used to estimate the long-run relationship between Intel sales and GDP.

More than one independent variable

The logic extends directly to multiple regression on three or more series.

No series has a unit root. Multiple regression is safe. For instance, regressing returns on the Fidelity Select Technology Portfolio on returns to the S&P 500 Growth Index and the S&P 500 Value Index, monthly from October 2015 to August 2019, is valid provided a Dickey–Fuller test rejects a unit root in all three series.
Some but not all have unit roots. The regression error cannot be covariance stationary, so multiple regression must not be used.
All have unit roots. Test for cointegration by estimating the multiple regression and applying the Engle–Granger Dickey–Fuller test to its residuals. Failing to reject a unit root in the residuals means no cointegration and no usable regression.
All have unit roots and are cointegrated. Modelling three or more cointegrated series is genuinely difficult and lies beyond this curriculum. An analyst who has not mastered the technique should avoid forecasting models built on several series with unit roots, because the coefficients may be inconsistent and the forecasts wrong.

Everything in this reading assembles into a single sequence. Follow it in order and the specification largely chooses itself.

Frame the investment problem and make an initial choice. One option is a regression model that predicts a variable from hypothesised causal relationships with other variables. The other is a time-series model that predicts a variable from its own history. This lesson is about the second.
Compile the series and plot it. Look for a linear trend, an exponential trend, seasonality, or a shift in the mean or variance partway through the sample. Each of these is a departure from covariance stationarity.
If there is no seasonality and no shift, try a trend model. Decide between linear and log-linear from the plot, estimate it, compute the residuals, and apply the Durbin–Watson test. If the residuals show no significant serial correlation, the trend model is enough and can be used for forecasting.
If the trend residuals are serially correlated, move to an autoregressive model, but first make the series stationary. A linear trend calls for first differencing. An exponential trend calls for taking natural logs and then first differencing. A significant shift in the sample calls for separate models on either side of the break. Significant seasonality calls for seasonal lags.
Fit a short autoregression to the transformed series. Estimate an AR(1) and test the residual autocorrelations. If none is significant, use it.
If residual autocorrelation remains, add lags. Move to AR(2), retest, and keep raising the order until the residual serial correlation is no longer significant.
Check for seasonality. Either inspect the plot for regular within-year patterns or look at the seasonal autocorrelation of the residuals, the fourth lag for quarterly data and the twelfth for monthly data, along with the lags on either side. Where seasonality is present, add the seasonal lag to the autoregressive model.
Test for ARCH. Regress the squared residuals on their own first lag. If the coefficient is not significantly different from zero, the standard errors from the model can be trusted. If it is significant, correct for the heteroskedasticity using generalized least squares or a comparable method.
Evaluate out-of-sample performance. Compare it with the in-sample performance to see whether the model generalises.

Figure 6: The model selection sequence

Each stage produces a testable result, and a failure at any stage sends you to the next transformation rather than to a more elaborate model.

What this reading deliberately leaves out

Two limitations deserve mention. First, the reading shows how to produce point forecasts but not how to measure the uncertainty around them, which can be very large. Doing that properly requires accounting for both the error term and the uncertainty in the estimated parameters, and it becomes involved once there is more than one independent variable. Second, the inflation example split the sample using outside knowledge of Federal Reserve policy. Where an analyst suspects multiple regimes but has no external information identifying them, switching regression models can identify regimes from the series itself.

Check yourself

1. A fitted log-linear trend model has an intercept of 4.2 and a trend coefficient of 0.018 on quarterly data. What is the forecasted level of the series in period 40, and what quarterly growth rate does the model imply?

Work in logs first: 4.2 + 0.018(40) = 4.92. Then exponentiate, because the predicted level in a log-linear model is e raised to the fitted log value: e^4.92 = 136.99 approximately. The implied growth rate is e^0.018 − 1 = 0.018163, or about 1.82% per quarter, not 1.8%. Forgetting to exponentiate, and quoting the slope itself as the growth rate, are the two standard errors on this calculation.

2. An analyst fits an AR(1) model and reports a Durbin–Watson statistic of 1.98, concluding that the residuals are free of serial correlation. What is wrong with the conclusion, and what should have been done?

The Durbin–Watson statistic is invalid whenever the explanatory variables include lagged values of the dependent variable, which is exactly the case in any autoregression. The correct procedure is to compute the autocorrelations of the residuals, divide each by its standard error of one over the square root of the number of observations, and compare the resulting t-statistics with the critical value. The Durbin–Watson statistic remains valid for trend models, where the regressor is time rather than a lagged dependent variable.

3. An AR(1) model of a quarterly series has an intercept of 0.42 and a slope of 0.65. The current value is 1.80. Calculate the mean-reverting level and the forecast two quarters ahead.

The mean-reverting level is 0.42 ÷ (1 − 0.65) = 0.42 ÷ 0.35 = 1.20. Since the current value of 1.80 sits above that level, the forecasts should step downward. One period ahead: 0.42 + 0.65(1.80) = 1.59. Two periods ahead, applying the chain rule by substituting the first forecast: 0.42 + 0.65(1.59) = 1.4535. The series moves toward 1.20 without reaching it.

4. Why does a random walk have no finite mean-reverting level, and why does that prevent an AR(1) model from being estimated on it?

A random walk is an AR(1) process with an intercept of zero and a slope of one. The mean-reverting level is the intercept divided by one minus the slope, which becomes zero divided by zero and is therefore undefined. The variance also fails: it equals the number of elapsed periods less one, multiplied by the shock variance, so it grows without bound. Failing both the constant-mean and finite-variance conditions, the series is not covariance stationary, and estimating an AR model on it produces biased coefficients and invalid tests.

5. In a Dickey–Fuller test, what exactly is regressed on what, what is the null hypothesis, and why can conventional t critical values not be applied?

The first difference of the series is regressed on a constant and the first lag of the level of the series, which comes from subtracting the lagged value from both sides of the AR(1) equation. The coefficient on that lagged level equals the AR(1) slope minus one. The null hypothesis is that this coefficient is zero, meaning the slope is one, the series has a unit root and is nonstationary; the alternative is that it is negative. Conventional critical values do not apply because under the null the series is not covariance stationary, so the statistic does not follow a t-distribution. The Dickey and Fuller critical values are larger in absolute value.

6. A quarterly AR(1) model of sales growth leaves a fourth residual autocorrelation of 0.68 with a t-statistic of 5.4 against a critical value of 1.99. What is the diagnosis, what is the correction, and what happens to the sample size?

The fourth lag is the seasonal lag for quarterly data, and a significant seasonal autocorrelation means significant seasonality remains in the residuals. The model is misspecified and must not be used to forecast. The correction is to add the value from four periods earlier as an extra explanatory variable, giving x_t = b₀ + b₁x_t−1 + b₂x_t−4 + ε_t, then retest the residual autocorrelations. If the estimation does not draw on observations from before the stated sample start, the usable sample shrinks, because five lags are now required rather than two.

7. An ARCH(1) test on the residuals of a time-series model gives a coefficient of 0.31 on the lagged squared residual with a t-statistic of 3.8. State the two practical consequences and compute the variance forecast if the estimated intercept is 5.20 and the current residual is 2.

First, the standard errors reported by the original time-series model are incorrect, so its t-statistics cannot support hypothesis tests; generalized least squares or another heteroskedasticity correction is needed. Second, the conditional variance is now forecastable. Applying the variance equation: 5.20 + 0.31(2²) = 5.20 + 1.24 = 6.44 for the next period.

8. A dependent series and an independent series are each found to have a unit root. Under what condition may they still be used in a linear regression, and how is that condition tested?

Only if they are cointegrated, meaning a long-run relationship holds them together so they do not diverge without bound. The test is the Engle–Granger procedure: estimate the regression of one series on the other, then apply a Dickey–Fuller test for a unit root to the residuals, using the Engle and Granger critical values rather than the standard ones because the residuals depend on estimated coefficients. Rejecting a unit root in the residuals means the error term is covariance stationary, the series are cointegrated, and the coefficients and standard errors are consistent for the long-run relationship. Failing to reject means the regression is spurious.

QM 5 – Time-Series Analysis

Overview

Why time series break the standard regression assumptions

Correlated residuals

A mean or variance that drifts

Linear trend models

Where linear trends fail

Log-linear trend models and exponential growth

Choosing a trend model and testing for correlated errors

The formal test: Durbin–Watson on the residuals

Autoregressive models and covariance stationarity

The stationarity requirement

Recognising a stationary series

Testing an autoregressive model with residual autocorrelations

The three-step procedure

Mean reversion and the mean-reverting level

Multiperiod forecasts and the chain rule of forecasting

Comparing forecast accuracy in sample and out of sample

Instability of coefficients and the choice of sample period

Random walks and first differencing

Why an AR model cannot be estimated on a random walk

First differencing

Random walk with drift

Unit roots and the Dickey-Fuller test

The transformation that makes the test work

Moving averages and moving-average models

Smoothing with an n-period moving average

Moving-average models for forecasting

Telling AR and MA processes apart

Seasonality and the seasonal lag

Detecting it

Correcting it

ARMA models and autoregressive conditional heteroskedasticity

Autoregressive moving-average models

Autoregressive conditional heteroskedasticity

Testing for ARCH(1) and using the result

Beyond ARCH(1)

Regressions with more than one time series: unit roots and cointegration

Cointegration

The Engle–Granger test

More than one independent variable

Determining an appropriate model for an investment problem

What this reading deliberately leaves out