QM 3 – Model Misspecification

Every regression result you will ever read rests on two design choices that were made before a single coefficient was estimated: which variables were allowed into the equation, and what algebraic shape the relationship was forced to take. Together those two choices are the model specification. Get them right and the standard machinery of t-statistics, F-statistics and confidence intervals means what it claims to mean. Get them wrong and the same machinery produces confident-looking output that is simply not trustworthy.

This reading is about the ways specification goes wrong, the statistical damage each way does, the tests that expose it, and the repairs available. The four topics ahead are omitted variables and the related functional-form failures, heteroskedastic errors, serially correlated errors, and multicollinear regressors. It is worth noticing at the outset that the first topic causes the other three as often as not. A model that leaves out something important, or that squeezes a curved relationship into a straight line, frequently announces the fact through non-constant error variance or correlated residuals. Diagnosis therefore runs in both directions: a failed assumption test is evidence of a specification problem, and a specification problem is the first place to look when a test fails.

Five principles for building the model in the first place

Before turning to failures, it helps to have a short standard for what good looks like. The curriculum sets out five principles.

The last principle is the bridge to the rest of this reading. The diagnostics that follow are how you find out whether it has been met.

Specification choices that are easy to overlook

Two further choices belong under the specification heading even though they are treated in detail elsewhere. The first is how to formulate a regressor that is qualitative rather than numeric. Industry membership, investment-grade status or whether a quarter fell in a recession cannot enter an equation as a raw category; they enter as indicator variables coded one or zero. The second is what to do when the dependent variable itself is qualitative, for example a company either defaults or does not. Ordinary least squares is the wrong tool for a binary outcome, and a logistic regression, which models the log odds of the event, is the correct specification instead.

A third strand of diagnostic work sits alongside the four covered here. Instead of asking whether the equation is correct, it asks whether a small number of observations are doing most of the work. That analysis of influence looks for high-leverage points, where an independent variable takes an extreme value, and for outliers, where the dependent variable does. Detecting such influential data points matters because a single observation can drag a fitted line a long way. Influence analysis, qualitative regressors and logistic regression are the subject of the next learning module; this one stays with the four classical assumption failures.

Functional form can fail in four recognised ways, and the first of them is the most damaging. Omitted variable bias is the distortion that follows when a genuinely important independent variable is left out of the equation.

Suppose the relationship in the world is described by two regressors:

and suppose the analyst estimates only:

What happens next depends entirely on one thing: whether the missing regressor is correlated with the one that was kept.

Case one: the omitted variable is uncorrelated with the included one

If the two regressors are unrelated, the part of the true equation attributable to the dropped variable has nowhere to go except into the error term. The residual in the estimated model effectively becomes the following:

That composite residual will not have an expected value of zero, and depending on how the dropped variable behaves it will not be independent and identically distributed either. The consequence is a biased estimate of the intercept. The slope on the retained variable, however, is still estimated correctly, because the contaminating term carries no information about it. This is the milder of the two cases.

Case two: the omitted variable is correlated with the included one

Now the damage spreads. Because the missing variable has been pushed into the error term, and because it moves with the retained regressor, the error term is correlated with a regressor. That correlation breaks a foundational condition of least squares. The estimated slope, the estimated intercept and the residuals are all wrong, and the coefficient estimates are not merely biased but inconsistent, meaning that collecting more data does not cure them. Worse for the analyst reading the output, the estimated standard errors are inconsistent too, so the t-statistics and P-values printed beside the coefficients cannot be used to test anything at all.

The practical signature of an omitted variable is that the residuals from the fitted model retain a visible relationship with something outside the model. Residuals from a correctly specified regression should look like noise when plotted against any candidate variable. When they instead line up along a slope, the candidate belongs in the equation.

Omitting a variable is only the first of four functional-form failures. The remaining three are more subtle because the variables involved are all present; it is the way they have been expressed or assembled that is wrong. The table below sets out all four together with the assumption violation each tends to produce.

Note how often the consequence column names one of the three assumption violations covered later in this lesson. Specification errors and assumption failures are two views of the same problem.

Using the wrong form of a variable

The second failure is fitting a straight line through a relationship that is not straight. Before specifying an equation, ask whether economic theory suggests curvature, and confirm by plotting the data. A useful diagnostic is to check whether the relation straightens out once one of the variables is expressed as a proportional change rather than a level. When it does, taking the natural logarithm of that variable is the correction: the logarithm converts a proportional relationship into a linear one, which is exactly the shape least squares is built to estimate.

Using unscaled data where scaled data belong

The third failure arises when raw figures are used and comparability demands normalised ones. Analysts run into this constantly when comparing companies of different sizes. A regression using absolute currency amounts across firms whose revenue differs by two orders of magnitude is measuring size as much as anything else. The familiar answer in financial analysis is the common-size statement, which restates each line item as a proportion of a base such as total revenue or total assets. Common-size figures make profitability, leverage and efficiency directly comparable across a peer group, and the same logic applies inside a regression.

Pooling samples that do not belong together

The fourth failure is combining data from periods or populations that follow different processes. The usual culprit is a structural break inside the sample window: a change in government regulation, or a regime shift from a quiet volatility environment to a turbulent one. Data of this kind betray themselves in a scatterplot as two or more separate clusters, widely spaced and each showing little internal correlation, because the cluster means sit far apart. A line fitted through the whole cloud passes through neither group.

The remedy is not to fit a compromise. When the available observations clearly fall into distinguishable subsamples, the analyst should estimate the model on whichever subsample best matches the conditions expected over the forecast period, and accept the smaller sample size as the price of relevance.

Linear regression assumes that the variance of the errors is the same for every observation. Errors with that property are homoskedastic. When the variance instead differs from observation to observation, the errors are heteroskedastic, and the assumption is violated.

Heteroskedasticity is common in financial data and its sources are largely the specification failures of the previous two sections: variables left out, a functional form that does not match the relationship, and transformations that were needed and not applied. Extreme values of an independent variable can also produce it.

Unconditional heteroskedasticity

The first variety is unconditional. Here the error variance changes across observations but is not correlated with any of the independent variables in the regression. This still violates the assumption, but it is the benign case: it creates no major problem for statistical inference, and an analyst who finds only unconditional heteroskedasticity can proceed with the usual tests.

Conditional heteroskedasticity

The second variety is the one that matters. Conditional heteroskedasticity means the error variance is correlated with, that is conditional on, the values taken by the independent variables. Panel B of Figure 2 is exactly this case. It damages inference in two places at once.

• The overall F-test becomes unreliable. Under conditional heteroskedasticity the mean squared error is a biased estimator of the true population variance, and the F-statistic for the significance of the regression as a whole is built on it.
• The individual t-tests become unreliable. Heteroskedasticity introduces bias into the estimators of the coefficient standard errors, so the denominators of the t-statistics are wrong.

The direction of the error is what makes this dangerous rather than merely inconvenient. With financial data the standard errors are typically underestimated. A t-statistic is a coefficient divided by its standard error, so a standard error that is too small produces a t-statistic that is too large. The analyst then finds significant relationships where none exists and commits Type I errors, rejecting a null hypothesis that is in fact true, more often than the stated significance level implies. In other words, conditional heteroskedasticity does not make a model look weak. It makes a weak model look strong.

Plotting residuals against each regressor is a reasonable first look, and it is what most analysts do. But eyes disagree, and a formal test does not. The Breusch–Pagan test is the standard diagnostic in financial analysis for conditional heteroskedasticity, and its logic is easy to hold in your head once you see the three steps laid out.

Step 2 is where the intuition lives. The squared residual of an observation is a crude estimate of the error variance at that observation. Regress those squared residuals on the original regressors:

If the errors are conditionally heteroskedastic, each squared residual carries information about the regressors, because the regressors are what drive the variance. The auxiliary regression will then explain a meaningful share of the variation in the squared residuals and its R-squared will be large. If the errors are homoskedastic, the auxiliary regression explains nothing and its R-squared is near zero.

The statistic is approximately chi-square distributed with k degrees of freedom, k being the number of independent variables in the original Step 1 regression. The null hypothesis is that all the slope coefficients in the auxiliary regression are zero, which is to say that the squared residuals are unrelated to the regressors and there is no conditional heteroskedasticity. The alternative is that at least one of them is non-zero. Because only a large statistic counts as evidence, this is a one-tailed test on the right side of the distribution.

Once conditional heteroskedasticity is confirmed, the simplest and most widely used repair is to recompute the coefficient standard errors using a formula that does not assume constant error variance. These are robust standard errors, also called heteroskedasticity-consistent standard errors or White-corrected standard errors. Every serious statistical package produces them on request, and using them should be routine.

The key point about the correction is what it leaves alone. The estimated coefficients do not move at all. Only the standard errors are replaced, and with them the t-statistics and P-values that depend on them. The correction is a repair to the inference, not to the fit.

The third assumption failure concerns a different property of the errors. Multiple regression assumes the errors of different observations are uncorrelated with one another. When they are correlated across observations, the errors are serially correlated, and the phenomenon is equally often called autocorrelation.

Serial correlation is above all a time-series problem, because time-ordered observations are the natural setting for one period to leave a trace in the next. It can also appear in panel data, which combines a cross-section with a time dimension. It does not usually trouble a pure cross-section, where there is no meaningful ordering of the observations.

Positive and negative, and why order matters

Serial correlation comes in two signs. Under positive serial correlation, a positive residual in one period makes a positive residual in the following period more likely, and equally a negative residual makes another negative residual more likely. The visual result is a residual series that drifts in long, stable runs on one side of zero before crossing over. Under negative serial correlation the opposite holds: a positive residual raises the odds of a negative residual next, so the series flips sign repeatedly and looks jagged.

Positive serial correlation is much the more common of the two in practice, and the analysis usually assumes first-order serial correlation, meaning correlation between adjacent observations only. In a time series that amounts to saying the sign of the residual tends to persist from one period into the next.

Consequences, and the one case that is genuinely serious

The main damage serial correlation does is to the estimated standard errors of the coefficients. Whether it also damages the coefficients themselves depends on a single structural feature of the model: whether any regressor is a lagged value of the dependent variable.

Without a lagged dependent variable the parameter estimates stay consistent and need no adjustment; only the standard errors do. With one, serial correlation in the error term makes every parameter estimate inconsistent, which is to say an invalid estimate of the true parameter.

Taking the common case of positive serial correlation without a lagged dependent variable, two things go wrong with the statistics. First, the F-statistic tends to be inflated, because the mean squared error understates the population error variance. Second, the coefficient standard errors are typically underestimated, so the t-statistics are inflated. The consequence is identical to the heteroskedastic case: more Type I errors, and a model that looks better than it is. This symmetry is worth memorising, because examiners test it. Conditional heteroskedasticity and positive serial correlation both deflate standard errors and both inflate t-statistics.

There is also an economic reading of the finding. A serially correlated time series is by construction partly predictable. If asset prices exhibited such a pattern, investors would identify it and trade on it until the pattern disappeared, which is the direct implication of the efficient market hypothesis. Under even weak-form efficiency, then, serial correlation should not be present in financial market data, and its appearance is worth investigating as information rather than merely patching.

Two tests dominate. The Durbin–Watson test measures autocorrelation by comparing the squared differences between successive residuals with the sum of the squared residuals. It is familiar and easy to compute, but it carries a real limitation: it tests only for first-order serial correlation, so a model whose errors are correlated at a lag of four quarters or twelve months can pass it comfortably.

The Breusch–Godfrey test is the more robust alternative because it can detect autocorrelation up to any pre-designated order p, that is, correlation between the error in period t and the error in period t minus p. Its structure closely mirrors the Breusch–Pagan test: estimate the model you care about, save the residuals, then run an auxiliary regression on them.

Notice the difference from the heteroskedasticity test. There, the dependent variable of the auxiliary regression was the squared residual and the regressors were unchanged. Here the dependent variable is the residual itself, and the original regressors are joined by one or more lagged residuals. The question being asked is whether last period residual helps predict this period residual once the regressors have had their say.

The hypotheses follow directly. The null is that the coefficients on the lagged residuals are zero, so there is no serial correlation in the residuals up to lag p. The alternative is that the correlation at a minimum of one lag is different from zero, meaning autocorrelation is present at or below the order tested. Under the null, the statistic follows an approximate F distribution with n minus p minus k minus 1 and p degrees of freedom, p being the count of lags included and k the count of regressors in the original equation. Statistical packages report this F-statistic and its P-value directly, so in practice you compare the P-value with your chosen significance level, or the statistic with the critical F-value.

Correcting for serial correlation

As with heteroskedasticity, the standard fix adjusts the standard errors rather than the coefficients. The adjusted quantities travel under several names: serial-correlation consistent standard errors, serial correlation and heteroskedasticity adjusted standard errors, Newey–West standard errors, and simply robust standard errors. A practical advantage is that these methods repair conditional heteroskedasticity at the same time, which is why one correction can be applied when both problems are suspected. Under the hood they replace the estimated matrix of variances and covariances with a version whose consistency does not depend on either constant error variance or independent errors.

Two cautions are worth carrying away. The correction does not remove the serial correlation from the residuals; it only makes the standard errors account for it. And a model that needs the correction may be telling you something more useful than the correction itself, namely that a variable is missing or that the functional form is wrong.