Eduzan

Learning Hub

Eduzan

Eduzan / 01 Quantitative Methods

QM 2 – Evaluating Regression Model Fit and Interpreting Model Results

Worked examples are fully visible. Check-yourself items are study aids you can reveal one at a time.

Simple linear regression asks a narrow question: how much of the movement in one variable can be traced to the movement in a second variable. That is a useful question, but it is almost never the whole question in investment work. Returns, credit outcomes and valuation multiples respond to several forces at once, and those forces are usually correlated with one another. Attributing everything to a single driver therefore produces a coefficient that quietly absorbs the influence of every driver you left out.

Multiple linear regression is the natural extension. It models the linear relation between one dependent variable and two or more independent variables at the same time, so each independent variable is assessed while the others are held fixed. That single change in framing is what makes the technique useful, and it is also what makes the interpretation of the output more demanding.

Three representative investment problems

The curriculum motivates the tool with three situations that are worth carrying in your head, because they show the three distinct purposes a regression can serve.

Attribution of portfolio returns. A portfolio manager wants to know which underlying factors drive a stock excess return: the size effect, the value effect, profitability, and investment aggressiveness. Estimating a Fama–French five-factor model gives the manager a factor-by-factor decomposition rather than a single beta.
Prediction of a corporate outcome. A financial adviser asks whether financial leverage, profitability, revenue growth and changes in market share, taken together, indicate that a company is heading towards financial distress. Here the point is forecasting a future state, not explaining a past one.
Testing an economic proposition. An analyst studies whether different dimensions of country risk, namely political stability, economic conditions, and environmental, social and governance (ESG) considerations, affect equity returns within that country. The regression is being used to test an existing theory.

Those three purposes, identifying relations between variables, testing theories, and forecasting, are the only reasons to run a regression at all. It is worth naming the purpose before specifying the model, because the purpose determines which diagnostics matter most. A model built for forecasting is judged on out-of-sample performance. A model built to test a theory is judged on whether the coefficient of interest is reliably estimated.

One warning belongs here rather than later. Adding explanatory variables is cheap, and modern software will happily estimate a model with a dozen of them. The curriculum is explicit that models with several explanatory variables are acceptable only when they are subject to fundamental justification and to a range of statistical tests. Economic reasoning comes first; the software comes second.

The tools you will meet

Regression estimation is a solved computational problem, and every environment an analyst is likely to use provides it. Knowing the function names is not examinable, but recognising them makes the output tables in this reading much less mysterious.

Software commonly used to estimate regressions

Environment	Typical function or command
Excel	Regression, within the Data Analysis toolpak
R	lm
Python	statsmodels.lm; sklearn.linear_model.LinearRegression; scipy.stats.linregress
STATA	regress
SAS	PROC REG; PROC GLM

All of these produce the same estimates from the same data. The differences lie in the default diagnostics each one prints alongside the coefficients.

Estimating a regression is one step inside a longer loop. The loop matters because most of the decisions that determine whether a model is any good are taken before the estimation and after it, not during it. The process below is the map you should be able to reconstruct from memory.

Figure 1: The regression process as a decision loop

Every “No” branch returns to estimation rather than to publication. A model reaches the bottom box only after the assumptions hold, the overall fit is significant, and no competing specification does better.

Step one: what kind of dependent variable do you have

The first fork is about the variable being explained. If it is continuous, for example a monthly return, the traditional linear regression model is the usual starting point. If it is discrete, for example an indicator that equals one when a company is a takeover target and zero when it is not, a linear model is the wrong shape and the relation should be estimated as a logistic regression instead. This distinction is decided by the nature of the data, not by preference.

Step two: which independent variables, and of what kind

Independent variables may themselves be continuous, such as accounting or market characteristics of a company, or discrete. Discrete explanatory variables are called dummy variables and record membership of a class, such as an industry sector. A model can mix both kinds freely; the fork above concerns only the dependent variable.

Steps three to five: estimate, test, adjust

Once specified, the model is estimated and then interrogated. Two questions are asked in sequence. Do the underlying assumptions of linear regression hold, judged largely from the residuals? And does the model meet the goodness-of-fit standard the analyst has set? A failure on either question sends you back to adjust the specification and re-estimate, not forward. Only when the assumptions hold, the overall fit is significant, and the specification is the best among the candidates considered, does the analyst then ask whether out-of-sample performance is acceptable. A model that clears all of that can be used to identify relations, to test theories, or to forecast.

Example 1 · Worked

You are a junior analyst helping to develop several regression models for your industry sector. For each issue below, state the action you should take.

Issues raised during model development

Issue
The variable being explained equals 1 when a company is a merger target and 0 in every other case.
A model has been estimated with five independent variables and not one of them is a significant explanatory variable.
The residuals do not look homoskedastic, so a regression assumption is being violated.
The regression assumptions hold, the overall fit is significant, and the specification is the best of those considered.

1. Identify the action called for by each of the four issues.

Solution. Work down the decision loop in Figure 1 and each answer falls out of the branch you are standing on.

Issue 1. The dependent variable is discrete rather than continuous, so the first fork sends you left: use logistic regression.

Issue 2. The overall fit is not significant, so the model fails the fit test: adjust the model and re-estimate.

Issue 3. A regression assumption is violated, so the model fails the assumptions test: adjust the model and re-estimate. Note that issues 2 and 3 arrive at the same action from different branches.

Issue 4. Every test has been cleared, so the model can be taken forward: use the model for analysis and prediction.

Start from what a regression is trying to do. The dependent variable moves about its own mean, and the total amount of that movement across the sample is a fixed quantity known as the variation of Y, or equivalently the sum of squares total.

The sum of squares total: the raw material the regression tries to explain.

Simple regression attempts to account for that variation using the variation in a single independent variable X. Multiple regression attempts exactly the same thing, but draws on the variations in a set of independent variables X₁, X₂, up to X_k. The goal has not changed; only the number of tools available to reach it has changed.

The simple linear regression equation, with one explanatory variable.

The multiple linear regression equation, estimated over n observations with k independent variables.

Two halves of the same equation

Split the right-hand side in two. Everything involving the k independent variables is the deterministic part of the model: given the values of the explanatory variables and the parameters, it is fully determined. The error term is the stochastic part, the random component that no explanatory variable in the model accounts for. Nearly every assumption you are about to meet is a statement about that error term, which is why the residuals do so much of the diagnostic work later in this reading.

Notice also the notation. The subscript i indexes the observation and runs from 1 to n. On an independent variable there are two subscripts: X_2i is the value of the second explanatory variable for observation i. The requirement that n must be larger than k is not decorative. Each independent variable consumes a degree of freedom, so a model estimated on too few observations relative to its size cannot separate the influence of one variable from another.

Parameters and their estimates

The equation above is written in terms of population parameters, which are unknown. What software returns are estimates, conventionally marked with a caret. Writing the estimated relation for a three-variable model:

The estimated regression equation. The caret marks an estimated, not a true, coefficient.

Note that the error term has disappeared from the estimated equation. That is deliberate: the fitted equation gives the predicted value of the dependent variable, and the residual is what remains when the actual observation is compared with that prediction. The residuals are the empirical stand-in for the unobservable errors, and every diagnostic plot in this reading is built from them.

Two terms complete the vocabulary. The coefficient b₀ is the intercept, the expected value of the dependent variable when every independent variable equals zero. The remaining coefficients b₁ through b_k are the slope coefficients, also called partial regression coefficients or partial slope coefficients. The word “partial” carries the entire meaning, and the next section unpacks it.

A caution about the intercept

The intercept is defined at the point where every explanatory variable equals zero, and that point frequently sits far outside the range of the observed data or describes a state of the world that cannot occur. In a Fama–French regression it happens to be interpretable, because a month in which every factor return is zero is perfectly conceivable and the intercept then measures unexplained excess return. In a regression of company value on revenue and headcount it is not, because a company with zero revenue and zero staff is not in the sample. Report the intercept because the equation needs it, but resist reading economic content into it unless the zero point is genuinely meaningful.

A slope coefficient in a multiple regression must be read with more care than its counterpart in a simple regression. It answers a conditional question: if this one explanatory variable moves by one unit and every other explanatory variable in the model stays exactly where it is, by how much does the dependent variable move? In the model

A three-variable model used to fix the interpretation of a partial slope.

the coefficient b₂ records the change in Y produced by a one-unit change in X₂ while X₁ and X₃ are held constant. Drop X₁ and X₃ from the model and b₂ will generally change, because the surviving variable is then left carrying whatever influence the removed variables used to carry. The number therefore belongs to the model, not to the variable.

Figure 2: What holding the other variables constant means

The common slope of the two lines is the partial coefficient on X1; the vertical distance between them is the effect of X2. Reading either one on its own from a plot of Y against X1 alone would confuse the two.

Figure 2 also makes clear what “holding constant” does and does not claim. Nothing has been held constant experimentally. The regression is a statistical device that separates the joint movement of the explanatory variables arithmetically. When two independent variables move closely together in the sample, that separation becomes fragile, which is exactly why the independence of the explanatory variables appears as an assumption later.

A worked interpretation of a bond index model

Consider an estimated regression in which the monthly excess returns of a bond index (RET) are explained by the change in monthly government bond yields (BY) and the change in investment-grade credit spreads (CS). Using 60 monthly observations, the estimated model is:

Estimated bond index model, 60 monthly observations.

Example 2 · Worked

Use the estimated bond index model above to answer the following.

1. Interpret the intercept.

Solution. Set both explanatory variables to zero. In a month in which government bond yields do not change and investment-grade credit spreads do not change, the bond index is expected to return 0.0023% per month, which compounds to approximately 0.028% per year.

2. Interpret the coefficient on the change in government bond yields.

Solution. A one-unit rise in the monthly government bond yield, with the credit spread change held constant, is associated with a change in the bond index return of −5.0585%. Expressed the way a fixed income analyst would express it, the bond index has an empirical duration of 5.0585. The sign is what economic reasoning demands: yields up, prices down.

3. Interpret the coefficient on the change in credit spreads.

Solution. If investment-grade credit spreads widen by one unit while the change in government bond yields is held constant, the bond index return changes by −2.1901%. Again the sign is intuitive: spread widening is a repricing of credit risk and hurts the index.

4. Calculate the expected excess return for a month in which the change in credit spreads is 0.001 and the change in government bond yields is 0.005.

Solution. Substitute both values into the estimated equation and keep every digit of the coefficients.

RET = 0.0023 − 5.0585(0.005) − 2.1901(0.001)
RET = 0.0023 − 0.0252925 − 0.0021901
RET = −0.0252, or −2.52%.

Notice the arithmetic of contribution. The yield move contributes roughly ten times as much to the answer as the spread move does, not because its coefficient is only about twice as large, but because the yield change itself is five times as large. Coefficient magnitude and variable magnitude both matter.

Once the coefficients are estimated, generating a predicted value is mechanical. Choose a set of assumed values for the independent variables, substitute them into the fitted equation, and read off the result. The care required lies not in the arithmetic but in three habits: keep the signs of the coefficients straight, use the full precision that was reported rather than a rounded version, and check that the assumed values are inside the range over which the model was estimated.

The predicted value of the dependent variable given assumed values for the independent variables.

The most common error in an exam setting is a sign error on a negative coefficient multiplied by a negative assumed value. The second most common is silently rounding the intercept. The worked example below is built around a case where the source curriculum itself illustrates the second point.

Example 3 · Worked

An institutional salesperson has read a research report in which you regressed the monthly excess returns of a portfolio (RETRF) on three Fama–French factors. MKTRF is the excess return on the market. SMB captures the return gap that separates small-capitalisation shares from large-capitalisation ones. HML captures the equivalent gap separating value shares from growth shares. Every return here is expressed in whole percentage points, so an input of 1 stands for 1%. The estimated equation is:

Estimated three-factor model for the portfolio excess return.

1. Interpret the intercept.

Solution. Set all three factor returns to zero. With MKTRF, SMB and HML each at zero, the portfolio is expected to return 1.5324%. Because the factors are excess returns, this figure is the part of the portfolio performance that the three factors do not account for.

2. Interpret each slope coefficient.

Solution. Every slope is read with the other two factors held constant.

MKTRF: if the market excess return rises by 1%, the portfolio return is expected to rise by 0.5892%. The portfolio carries roughly six-tenths of market risk.

SMB: if the size factor return rises by 1%, the portfolio return is expected to fall by 0.8719%. A negative loading on SMB is the signature of a large-capitalisation tilt.

HML: if the value factor return rises by 1%, the portfolio return is expected to fall by 0.056%. The loading is small, so the portfolio is close to style-neutral on the value dimension.

3. Calculate the predicted value of the portfolio return if MKTRF = 1, SMB = 4 and HML = −2.

Solution. Substitute the three assumed values, taking care with the negative HML input multiplied by the negative HML coefficient, which produces a positive contribution.

R = 1.534 + 0.5892(1) − 0.8719(4) − 0.0560(−2)
R = 1.534 + 0.5892 − 3.4876 + 0.112
R = −1.2524.

The portfolio is expected to underperform by about 1.25% in that month. The large negative contribution comes from SMB: a loading of −0.8719 applied to a size factor return of 4 costs 3.4876%, which swamps everything else.

A note on precision. The estimated equation reports the intercept as 1.5324, while the solution line above substitutes 1.534, as the source curriculum does, and reaches −1.2524. Carrying the reported intercept of 1.5324 through the same arithmetic gives −1.2540 instead. The difference of 0.0016 is entirely down to the intercept used, and it is a useful reminder that the last digit of an intercept survives all the way into the forecast.

What comes after interpretation

Interpreting coefficients one at a time answers only part of the analyst question. Three further questions follow immediately, and they are the subject of the next learning module rather than this one, but knowing that they are coming shapes how you read the output table in the next section.

How well does the model account for the dependent variable overall? The question of how much a fitted equation explains is answered by measures of goodness of fit, principally adjusted R-squared, and by analyzing the results assembled in the ANOVA table.
Are several coefficients jointly significant? A joint hypothesis test asks about two or more coefficients together, rather than testing each one in isolation. You formulate the hypotheses over a group of slope coefficients and evaluate them with an F-test.
How reliable is a predicted value? A point forecast is of limited use without an interval around it.

For this reading, the ANOVA and goodness-of-fit numbers are treated as descriptive output to be read correctly, not as tests to be performed.

Ordinary least squares (OLS) will return coefficients from any data set you feed it. Whether those coefficients support valid statistical inference is a separate matter, and it depends on a set of conditions being met. Simple regression rested on four assumptions. Multiple regression restates them with small modifications to account for the presence of several explanatory variables, and adds a condition about the relations among those explanatory variables.

Linearity. The relation connecting the dependent variable to the independent variables is linear in form.
Homoskedasticity. The spread of the regression residuals is the same at every observation.
Independence of errors. Observations do not depend on one another, which implies the residuals show no correlation from one observation to the next.
Normality. The regression residuals follow a normal distribution.
Independence of independent variables. This condition has two parts. Part 5a requires that the independent variables are not random. Part 5b requires that no exact linear relation exists between two or more of the independent variables, or between combinations of them.

The five assumptions and what each one protects

Assumption	What it requires	Why it matters
Linearity	A straight-line form connects the dependent and independent variables	A curved relation forced into a straight line produces biased coefficients and patterned residuals
Homoskedasticity	Residual spread is constant across all observations	Uneven spread leaves the coefficients usable but corrupts the standard errors and therefore the t-statistics
Independence of errors	Residuals are uncorrelated across observations	Correlated residuals understate the true uncertainty, so significance is overstated
Normality	Residuals are drawn from a normal distribution	The distributions used for hypothesis tests and confidence intervals depend on it
Independence of independent variables	Explanatory variables are not random and are not exactly linearly related	Without it the coefficients cannot be separated from one another, and in the exact case cannot be computed at all

The independence condition in practice

Part 5b deserves separate attention because it behaves differently from the others. The independence condition is what makes estimation possible in the first place. If one explanatory variable is an exact linear function of another, or of a combination of others, the model simply cannot be estimated: there is no unique set of coefficients that fits the data, because the offending variables carry identical information.

The far more common case is an approximate rather than exact linear relation. Large pairwise correlations between explanatory variables are the usual warning sign. Here the model can be estimated and the software will report coefficients without complaint, but the interpretation becomes problematic. The estimates become unstable, standard errors inflate, and a variable that is genuinely important can appear insignificant. Detecting this condition formally, and the remedies for it, are covered later in the curriculum; at this stage the requirement is to recognise the symptom.

Why an exact linear relation makes the model unestimable

Suppose you include both a total expense figure and its two components, fixed cost and variable cost, as explanatory variables. The three are linked by an identity, so an infinite family of coefficient combinations reproduces exactly the same fitted values: add one unit to the coefficient on total expense and subtract one from each component coefficient, and nothing changes. Least squares has no basis for choosing among them, so no unique solution exists. The practical lesson is to check for definitional identities among your explanatory variables before estimating anything, because this failure is one of specification rather than of data.

Homoskedasticity made visual

Of the five, homoskedasticity is the one most readily judged by eye, and Figure 3 shows the contrast you are looking for.

Figure 3: Constant residual spread against widening residual spread

Panel A is what a well-behaved model looks like: a horizontal band of roughly constant width centred on zero. Panel B violates homoskedasticity, since the residual spread depends on the predicted value.

Two cautions apply when reading such plots. First, absence of an obvious pattern is evidence, not proof; formal tests exist and come later. Second, in empirical work the assumptions frequently fail to hold, particularly with financial time series. Detecting failures and mitigating them is a large part of applied regression work, and the reason this reading spends so much effort on diagnostics.

To make the diagnostics concrete, the curriculum estimates one model and then examines it from every angle. The data are 10 years of monthly total excess returns for a stock referred to as ABC, giving 120 monthly observations, and the explanatory variables are the three Fama–French factors already introduced: market excess return (MKTRF), size (SMB) and value (HML).

The three-factor model of ABC excess returns, indexed by time t rather than by observation i.

Look at the data before estimating anything

The sensible first move is a scatterplot matrix, also called a pairs plot. It arranges every variable against every other variable in a grid, with the distribution of each variable shown along the diagonal. Figure 4 shows the layout schematically.

Figure 4: How a scatterplot matrix is laid out

Diagonal cells carry the distribution of a single variable. Off-diagonal cells carry a pairwise scatter with a fitted line, so the bottom row shows the dependent variable against each explanatory variable and the interior cells show the explanatory variables against one another.

Read the matrix in two passes. The first pass runs along the row or column containing the dependent variable and asks whether the relation with each explanatory variable looks linear and which way it points. In the ABC data, ABC excess returns show a positive relation with the market factor MKTRF, no apparent relation with the size factor SMB, and a negative relation with the value factor HML. The second pass looks at the interior cells, where explanatory variables meet one another. SMB and HML show little or no correlation, indicated by a relatively flat fitted line for that pair, and that is a desirable characteristic between explanatory variables because it means assumption 5b is comfortably satisfied.

A third benefit is incidental but valuable: because every data point is plotted, extreme values and outliers are visible before any model is estimated.

Why SMB looks irrelevant in the scatterplot yet matters in the regression

A pairwise scatterplot compares two variables in isolation and therefore mixes together the direct influence of SMB on ABC returns with the influence of everything else that moves alongside SMB. The regression does something different: it measures the partial relation, the part of the movement in ABC returns tied to SMB once MKTRF and HML have been accounted for. That partial relation can be clearly significant, as it is in the estimated model below, while the raw two-variable picture shows nothing at all. This is the single most important reason not to select explanatory variables by scanning a correlation matrix.

The estimated model

Estimating the model in any of the packages listed earlier produces the standard three-block output: a summary of fit, an ANOVA table, and the coefficient block.

ABC returns explained by the three-factor model: regression statistics

Statistic	Value
Multiple R	0.6238
R-Squared	0.3891
Adjusted R-Squared	0.3733
Standard Error	0.0628
Observations	120

ABC returns explained by the three-factor model: ANOVA

Source	df	SS	MS	F	Significance F
Regression	3	0.2914	0.0971	24.6278	0.0000
Residual	116	0.4575	0.0039
Total	119	0.7489

ABC returns explained by the three-factor model: coefficient estimates

Term	Coefficient	Standard error	t-Stat.	P-value	Lower 95%	Upper 95%
Intercept	0.0052	0.0061	0.8435	0.4007	−0.0070	0.0173
MKTRF	1.2889	0.1538	8.3791	0.0000	0.9842	1.5935
SMB	−0.5841	0.2664	−2.1922	0.0304	−1.1118	−0.0564
HML	−0.6810	0.2231	−3.0523	0.0028	−1.1229	−0.2391

Note how the coefficient block confirms the point made in the reveal above: SMB carries a t-statistic of −2.1922 and a P-value of 0.0304, so it is significant at the 5% level despite showing no visible pairwise relation with ABC returns.

Reading the three blocks together

The fit block says the three factors account for 38.91% of the variation in ABC excess returns, and 37.33% once the count of explanatory variables is penalised. Multiple R is simply the square root of R-squared for a model of this form. The standard error of 0.0628 is expressed in the units of the dependent variable, so a typical miss is a little over 6 percentage points of monthly excess return, which is large and a useful corrective to the impression that a 38.91% R-squared conveys.

The ANOVA table splits the sum of squares total into the part the regression accounts for and the part left in the residuals. Its degrees of freedom follow directly from the model size: three explanatory variables give 3 regression degrees of freedom, and 120 observations less 3 explanatory variables less 1 intercept leaves 116 residual degrees of freedom, with 119 in total.

The coefficient block is where interpretation happens. ABC carries a market loading of 1.2889, so it is more volatile than the market with respect to the market factor. It loads negatively on both SMB and HML, which describes a large-capitalisation growth profile. The intercept of 0.0052 is not distinguishable from zero, since its P-value is 0.4007 and its 95% interval spans zero from −0.0070 to 0.0173.

Example 4 · Worked

Use only the figures printed in the three output tables above to verify that the output is internally consistent. This is a habit worth building, because a transcription error in an exam vignette is far easier to catch this way than by re-reading the numbers.

1. Confirm the sum of squares figures and the degrees of freedom.

Solution. The regression and residual sums of squares must add to the total: 0.2914 + 0.4575 = 0.7489, which matches the total row. Degrees of freedom behave the same way: 3 + 116 = 119, and 119 equals 120 observations less one.

2. Confirm R-squared and Multiple R from the ANOVA table.

Solution. R-squared is the share of the sum of squares total accounted for by the regression: 0.2914 ÷ 0.7489 = 0.3891, matching the fit block. Multiple R is its square root: the square root of 0.3891 is 0.6238.

3. Confirm the mean squares, the F-statistic and the standard error.

Solution. Each mean square is the corresponding sum of squares divided by its degrees of freedom.

Regression: 0.2914 ÷ 3 = 0.0971.
Residual: 0.4575 ÷ 116 = 0.0039.
F is the ratio of the two mean squares, carried at full precision rather than at the rounded values printed: 0.0971333 ÷ 0.0039440 = 24.628, which reproduces the printed 24.6278 to three decimal places. Working instead from the rounded 0.0971 and 0.0039 gives about 24.9, so precision matters here.
The standard error of the regression is the square root of the residual mean square: the square root of 0.0039440 is 0.0628.

4. Confirm the adjusted R-squared figure.

Solution. Adjusted R-squared penalises R-squared for the number of explanatory variables by using the degrees of freedom rather than the raw observation count.

1 − (1 − 0.3891) × (119 ÷ 116) = 1 − 0.6109 × 1.02586 = 1 − 0.62670 = 0.3733.

The penalty is worth 0.0158, which is small here because 120 observations comfortably support three explanatory variables. In a model with 20 explanatory variables and 60 observations the gap would be far wider, and that is precisely the situation adjusted R-squared exists to expose.

5. Confirm the 95% confidence interval reported for HML.

Solution. The interval is the coefficient plus or minus the critical t-value for 116 degrees of freedom multiplied by the standard error. That critical value is 1.9806.

Half-width: 1.9806 × 0.2231 = 0.4419.
Lower bound: −0.6810 − 0.4419 = −1.1229.
Upper bound: −0.6810 + 0.4419 = −0.2391.

Both match the printed values. The interval lies entirely below zero, which is the same conclusion the P-value of 0.0028 delivers.

With the model estimated, attention shifts to the residuals. Regression software produces a standard set of diagnostic plots, and each one is aimed at a particular assumption. Knowing which plot answers which question is the examinable skill in this part of the reading.

Which diagnostic plot addresses which question

Plot	What it is used to detect
Pairwise scatterplot of the dependent variable against an independent variable	Whether the relation is linear, and where outliers lie
Pairwise scatterplot of one independent variable against another	Whether the independence of the independent variables is at risk
Residuals against the predicted value	Clustering or fanning that indicates the residuals are not homoskedastic, and potential outliers
Residuals against an independent variable	Extreme values of that variable that may distort the estimated parameters, and any residual pattern signalling misspecification
Normal Q-Q plot of residuals	Whether the residuals are normally distributed

Residuals against the predicted value

Start with a scatterplot of the residuals against the predicted values of the dependent variable. For the ABC model, visual inspection shows no directional relation between the two: the cloud sits about a line centred near a residual value of 0.00. That outcome is what you want, because residuals should behave independently of what the model predicts. It suggests the errors have a constant variance and are uncorrelated with one another, so several of the underlying assumptions survive the test.

The same plot flags three residuals as potential outliers, marked with square markers: Months 7, 25 and 95. Outliers are not automatically errors. The right response is to investigate whether some shock outside the model occurred at those points in time, rather than to delete the observations.

Residuals against each independent variable

Next come three plots, one for each factor, showing the residuals against MKTRF, against SMB and against HML in turn. Again, visual inspection reveals no directional relation between the residuals and any explanatory variable, which suggests no assumption of multiple linear regression is being violated. Importantly, the same three potential outliers identified in the residuals-versus-predicted plot reappear in these plots, which corroborates the earlier finding rather than adding a new one.

The normal Q-Q plot

The final diagnostic addresses normality. A normal quantile-quantile plot, or Q-Q plot for short, visualises the distribution of a variable by setting it against a normal distribution. Applied to a regression, it compares the standardised residuals of the model with a theoretical standard normal distribution. If the residuals really are normally distributed, the plotted points should line up along the diagonal.

The diagonal is a benchmark you can check arithmetically. For a normal distribution, 5% of observations fall below −1.65 standard deviations, so the residual sitting at the 5th percentile of the sample should appear at roughly −1.65 on the theoretical axis. Departures tell you where the sample distribution is heavier or lighter than the normal benchmark.

Figure 5: Reading a normal Q-Q plot

Points that track the dashed diagonal through the centre and separate from it at both ends describe a fat-tailed residual distribution: extreme outcomes occur more often than the normal benchmark allows.

For the ABC model, the picture is largely reassuring with three exceptions. Beyond −2 standard deviations, observations 25 and 95 fall well below the theoretical standard normal range, and observation 7 sits above the diagonal at around +2.5 standard deviations, somewhat above the theoretical range. These are the same three points the residual plots flagged. Setting them aside, the Q-Q plot provides ample evidence that the residuals as a whole are distributed consistently with the normal distribution, so the conclusion is that the model error term is close to normally distributed.

One practical note carries beyond this reading. Fat-tailed residual distributions are a commonly observed feature of financial time series, so a Q-Q plot that bends away at both ends is a familiar sight rather than a rare pathology. What it tells you is that inference based on normal-theory standard errors will understate the frequency of extreme outcomes.

Example 5 · Worked

Your model explains the monthly price change of a cryptocurrency (CRYPTO) with two explanatory variables: the price change in gold (GOLD) and the price change in a technology stock index (TECH). The sample runs over five years of monthly data, and you have produced several diagnostic charts alongside the estimated regression. Your research director asks you the following.

1. If the correlation between GOLD and TECH turns out to be significant, which assumption may be violated?

Solution. A significant pairwise correlation between two explanatory variables points to an approximate linear relation between GOLD and TECH. That would violate the independence of independent variables assumption, specifically part 5b, and it should be explored further before the coefficients are relied upon. Note that the model would still estimate; the problem is that the two partial slopes become difficult to separate and to interpret.

2. What is the purpose of a plot of the regression residuals against the predicted value of CRYPTO?

Solution. This plot is used to look for clustering or any pattern suggesting the residuals are not homoskedastic, and to identify potential outliers. A horizontal band of even width centred on zero indicates that the constant-variance assumption is holding.

3. What is the purpose of a plot of the regression residuals against GOLD?

Solution. Two purposes. It reveals extreme values of that independent variable, which can exert heavy influence on the estimated regression parameters. It also reveals any relation between the residuals and that explanatory variable, which would indicate the model is misspecified, because a well-specified model leaves nothing systematic in the residuals for an included variable to explain.

4. What is the purpose of a normal Q-Q plot of the residuals?

Solution. It is used to explore whether the residuals are normally distributed, which is a key assumption of linear regression and the basis for the standard errors, t-statistics and confidence intervals the output reports.

Example 6 · Worked

Still working on the CRYPTO model, answer the following.

1. Which of the following does a pairwise scatterplot detect? (A) Linearity in the relation tying the explained variable to the explanatory ones. (B) Heteroskedasticity in the residual terms. (C) Normality in the residual terms.

Solution. A is correct. The pairwise scatterplot visualises the relations between the dependent variable and the explanatory variables, so it is the tool for judging linearity. Heteroskedasticity is judged from a residual plot, and normality from a Q-Q plot, so B and C describe different diagnostics.

2. A scatterplot of CRYPTO price changes against TECH price changes shows an upward-sloping cloud with several points far from the mass. Interpret it.

Solution. There appears to be a positive relation between CRYPTO and TECH, and it may well be significant. Several potential outliers are also apparent. Both observations are useful before estimation: the first supports including TECH as an explanatory variable, and the second warns that a handful of observations may drive the fitted line.

3. Which of the following does a normal Q-Q plot detect? (A) Linearity in the relation tying the explained variable to the explanatory ones. (B) Heteroskedasticity in the regression residual terms. (C) Normality in the regression residual terms.

Solution. C is correct. The normal Q-Q plot exists to explore whether the residuals are normally distributed. It says nothing directly about linearity or about the constancy of the residual variance.

4. The Q-Q plot from the CRYPTO regression shows points tracking the diagonal in the centre but deviating sharply beyond plus and minus 2 standard deviations. Interpret it.

Solution. The residuals are not normally distributed. The deviation from the diagonal past ±2 standard deviations, together with several apparent outliers, indicates that the distribution of residuals is fat-tailed. Extreme residuals occur more often than a normal distribution would produce. This pattern is common in financial time series, so it should be expected rather than treated as a data error, but it does mean that inference resting on normality needs to be treated with caution.

Pulling the reading together

Three things should stay with you from this module. First, the partial slope coefficient is a conditional statement, and it belongs to the model in which it was estimated rather than to the variable itself. Second, five assumptions stand between an estimated equation and valid inference, and four of the five concern the error term. Third, the diagnostic plots are not decoration: each one is matched to a specific assumption, and being able to say which plot answers which question is worth more in an exam than being able to recite the assumptions in order.

Check yourself

1. An analyst wants to model whether a company will breach a debt covenant in the coming year, coding the outcome as 1 for a breach and 0 otherwise, using four continuous financial ratios. Which model should be estimated, and why?

A logistic regression. The first fork in the regression process turns on the dependent variable, not the independent variables. Here the variable being explained is discrete rather than continuous, so a linear model is the wrong shape and the relation should be estimated as a logistic regression. The fact that all four explanatory variables are continuous is irrelevant to this choice.

2. In the estimated model RET = 0.0023 − 5.0585BY − 2.1901CS, an analyst states that the bond index return falls by 5.0585% whenever government bond yields rise by one unit. What is missing from that statement?

The condition that the change in credit spreads is held constant. Every slope in a multiple regression is a partial coefficient, so the statement is only correct when the other explanatory variable is fixed. Without that qualification the analyst is describing a simple regression coefficient, which would in general be a different number because it would also absorb whatever influence credit spreads carry.

3. Using RETRF = 1.5324 + 0.5892MKTRF − 0.8719SMB − 0.0560HML, calculate the predicted portfolio excess return when MKTRF = 2, SMB = −1 and HML = 3.

Substitute each assumed value and watch the signs. 1.5324 + 0.5892(2) = 1.5324 + 1.1784. The SMB term is −0.8719 × (−1) = +0.8719, a positive contribution because both the coefficient and the assumed value are negative. The HML term is −0.0560 × 3 = −0.168. Adding: 1.5324 + 1.1784 + 0.8719 − 0.168 = 3.4147, or about 3.41%.

4. A regression output reports R-squared of 0.3891 and adjusted R-squared of 0.3733 on 120 observations with three explanatory variables. A colleague proposes adding six more variables and expects R-squared to improve. Is that expectation reasonable, and what should be watched instead?

R-squared will improve, or at worst stay the same, because it never falls when a variable is added. That is exactly why it is the wrong measure for comparing models of different sizes. The figure to watch is adjusted R-squared, which uses the degrees of freedom rather than the raw observation count and therefore falls when a variable adds less explanatory power than it costs. With 120 observations the penalty for three variables is only 0.0158, but it grows quickly as variables are added.

5. Two explanatory variables in a proposed model are total operating expense and its two components, staff cost and non-staff cost, all three included together. What happens when the model is estimated, and how does this differ from the case of two highly correlated but distinct variables?

The three variables are joined by an exact identity, so the model cannot be estimated at all: no unique set of coefficients exists, since adjustments to one coefficient can be exactly offset by adjustments to the others. That is the strict form of the assumption on independence of independent variables. Two merely correlated variables produce a different and milder problem: the model estimates, but standard errors inflate and the individual coefficients become unstable and hard to interpret.

6. You plot the residuals of a monthly returns regression against the predicted values and find a narrow band of points on the left of the chart widening steadily to a broad band on the right. Which assumption is in question, and what happens to your inference if you ignore it?

Homoskedasticity, the requirement that the residual spread is the same at every observation. The fanning pattern means the spread depends on the predicted value. The coefficient estimates themselves remain usable, but the standard errors are wrong, so the t-statistics, P-values and confidence intervals built on them are unreliable. A variable can appear significant when it is not, which is the more dangerous direction of error.

7. A Q-Q plot of standardised residuals shows the points sitting neatly on the diagonal from about −2 to +2 standard deviations, then dropping well below the line at the far left and rising well above it at the far right. What is the diagnosis, and is it unusual for financial data?

The residual distribution is fat-tailed: extreme values occur more frequently than a normal distribution would produce, even though the centre of the distribution is well behaved. It is not unusual at all, since fat-tailed residual distributions are a commonly observed feature of financial time series. The practical implication is that normality-based intervals will be too narrow in the tails, so risk measures derived from this model will understate the chance of extreme outcomes.

End of lesson.

Back to 01 Quantitative Methods

QM 2 – Evaluating Regression Model Fit and Interpreting Model Results

Overview

Why one explanatory variable is rarely enough