QTA 14: Machine-Learning Methods

Machine-learning methodologies fall into three families, separated by what the algorithm is handed and what it must produce.

Unsupervised learning

Here the algorithm hunts for patterns with no explicit target. It may group observations into clusters, or condense them into a handful of factors accounting for most of what is going on. Since nothing is being predicted, the technique can look unrewarding, yet it describes a dataset and exposes how it is put together. Anomaly detection illustrates the point. A bank hunting suspicious transactions cannot say in advance which variables separate them from the rest, so isolating what sets some apart is itself valuable, and the bank can use what it finds to train a fraud model.

Supervised learning

Supervised learning is about prediction, and it comes in two shapes. In one, the value of a variable is to be predicted, for example the price of a house. In the other, an observation is sorted into a category, for example a loan classified as one that will repay or one that will default. Either way, some labelled data must exist to learn from. For house valuation that data holds features such as lot size and square feet of living space, alongside selling prices, which are the labels. For loans it holds the income of borrowers and their credit scores, with labels recording which loans defaulted.

The prediction may be a time-series one, such as gross national product, or where the S&P 500 index will stand a year out. It may equally be cross-sectional: what the neighbours would get for their apartment, which is not in the sample. Credit decisions are a well-established classification success, since a lender must sort borrowers by whether they are acceptable credit risks.

Reinforcement learning

Reinforcement learning deals with a series of decisions taken in a shifting environment, and it proceeds by trial and error rather than by fitting labelled examples. Risk management applies it to unwinding a large block of shares, to portfolio management, and to hedging derivatives portfolios. A fourth category, semi-supervised learning, sits between the first two: the objective is prediction, but only part of the data is labelled, and the remainder is used to find patterns among the explanatory variables.

Data preparation begins with scale. Many machine-learning approaches insist that every variable be measured on a common one, since otherwise a feature recorded in hundreds swamps one recorded in tenths for no better reason than the units chosen. Two methods do the rescaling: standardization and normalization.

Standardization subtracts the sample mean of a variable from every observation on it and divides by the sample standard deviation. Writing x_ij for the jth observation on the ith variable:

What emerges is a scale on which the variable has zero mean and unit variance. Normalization, sometimes called the min-max transformation, works from the extremes of the sample instead, producing a variable bounded between zero and one that will not usually have zero mean or unit variance.

Every input goes through whichever route is taken, while the variables being predicted do not need rescaling. Standardization is preferred where the data span a wide scope and include outliers, since normalization would compress most observations into a band that misrepresents the original spread.

Cleaning is unglamorous and swallows an enormous share of the effort, up to 80% of the time a data analyst spends. Large datasets nearly always arrive with defects, and whether a project succeeds often turns on how well they were dealt with before any model was fitted. Five problems recur.

Inconsistent recording

Everything must be recorded the same way. Dates in two formats, currencies mixed inside one column, or a category spelled differently in different rows will all be misread by an algorithm that cannot know they refer to the same thing.

Unwanted and duplicate observations

Observations with no bearing on the task should be taken out. Duplicate observations should go too, since a record counted twice pulls the fitted model toward whatever it says and introduces bias.

Outliers

An observation sitting several standard deviations from the mean deserves a careful look before it is left in. Points like that move results a long way, so it is worth establishing whether the value is genuine or a recording error.

Missing data

Gaps are the most common problem of all. Where only a handful of observations have them, those observations can simply be dropped. Where there are too many for that, one option is to substitute the mean or the median of the observations on that feature which are present, and another is to estimate each missing entry from other features. A model trained on unrepaired gaps, duplicated rows and mixed units learns those artefacts as though they were signal.

Principal components analysis, usually shortened to PCA, is a main tool of unsupervised learning and a long-established method for reducing dimensionality. It manufactures a small number of new variables, the components, that carry nearly all the information held in a large set of correlated variables, and the components that come out are uncorrelated with one another.

Datasets often carry many features telling much the same story, which leaves the model built on them hard to interpret. PCA trims that number, cutting the underlying sources of uncertainty. Each component is only a linear combination of the original features, so the technique is easy to implement however large the dataset.

The yield-curve application

A classic use is reducing yield curve movements to a few explanatory variables. Imagine ten years of daily interest rate movements at maturities of one-month, three-months and six-months, then one-year, three-years, five-years, ten-years and 30-years. PCA looks for a small set of uncorrelated variables such that the observed movements are close to a linear combination of them. The dominant one is a parallel shift, where every rate moves the same way by roughly the same amount. Next comes a twist, where short rates travel in one direction and long rates in the other.

Source: the chapter’s principal components table, transposed so each row is one component, for maturities of one-month to 20-years.

Those loadings come from seven Treasury rates observed monthly from January 2012 through to December 2021, which gives 120 data points. Reproducing the movements in full needs all seven components, but the description becomes adequate long before that: the first, close to a parallel shift, explains 73.3% of the variation on its own, and the first three together explain more than 99%. The first row shows why, since all seven loadings are positive and of similar size. The second row changes sign between short maturities and long ones, which is the twist.

Any clustering method needs a rule for how far apart two observations are, and two are in common use. Take two features, x₁ and x₂, and two points P and Q in that plane. Squaring the gap in each dimension, adding the squares and taking the root gives the L₂-norm, the distance travelled as the crow flies. With m features it extends in the obvious way.

The Manhattan measure, or L₁-norm, adds the absolute differences instead of squaring them, approximating the distance a car covers between two buildings on a grid of streets.

The K-means algorithm is an uncomplicated unsupervised method for splitting observations into clusters, and a useful way to expose how a dataset is arranged. The number wanted, K, is fixed by the analyst in advance, which is why several values are usually tried.

The four steps

First, pick starting positions for the K centroids at random, a centroid being the centre of a cluster. Second, assign every data point to whichever centroid is nearest. Third, move each centroid to the centre of the points assigned to it. Fourth, repeat those two steps until the centroids stop moving. The middle steps need a distance measure, Euclidean or Manhattan. Because everything hangs on distance, the features must be scaled first; skipping that lets whichever feature carries the largest units decide the whole allocation.

An application to returns and yields

Applying the algorithm to annual value-weighted stock index returns, covering all stocks on the NYSE, Amex and NASDAQ, together with Treasury bill yields from 1927 to 2021, and setting K = 2, produces fitted centroids at [-7.70,3.51] and at [25.42,3.16]. Two regimes have emerged, separating stock returns into boom years and bust years. The Treasury bill coordinates, 3.51 and 3.16, sit close together, so the equity return is doing the separating.

K-means is easy to understand and to implement. Set against that, the number of clusters has to be specified a priori, and because allocation rests on distance from a centroid the method tends to carve out spherical clusters. Real data sometimes forms clusters of quite different shape, and those it handles badly.

The distance formulas describe the gap between one point and another. K-means, though, aims not at the gap between pairs of points but at the gap between each point and the centroid it belongs to. Write d_j for the distance between data point j and its own centroid, with j running from 1 to n. Inertia is the sum of those distances squared.

Lower inertia means a better fit. Since the opening centroids are random, the algorithm is usually run several times from different starting positions, which sometimes yields different clusters. For a given K, the clustering to keep is the one with the smallest inertia.

Choosing K

Inertia behaves rather like R² in a regression, which never falls when another explanatory variable is added: inertia never rises as centroids are added. Push K to n, its largest possible value, and every point becomes its own cluster with inertia falling to zero. That model fits perfectly and says nothing, which is why choosing K well is a real practical problem. One approach computes inertia across a range of K and plots the results. The chart is a scree plot, and the same device serves for deciding how many components to retain in PCA. What you look for is an elbow, a point past which inertia only creeps down as K rises.

Doing this for the stock return and Treasury bill data, plotting K from 1 to 10, produces a slight elbow at K = 3, which suggests three clusters might be better there. A second route is the silhouette coefficient, which for each observation weighs the distance to other points in its own cluster against the distance to points in the closest other cluster. The K delivering the highest silhouette score is the one to prefer.

Overfitting describes a model made too large, or handed more parameters than the problem justifies, as when a high-dimensional polynomial is fitted to data whose real shape is roughly quadratic. Such a model absorbs the random noise in the training data rather than only the signal beneath it.

The clearest symptom is a model that performs markedly worse the moment it meets data it has not seen. Model building uses a training set and a validation set: parameters are estimated on the first, and the second gives an independent look at how the fitted model behaves. An overfitted model flatters itself, since its training error can be very low and possibly close to zero. Applied outside that set its performance is likely to be poor, and it will not generalize.

The problem bites harder in machine learning than in conventional econometrics because of the parameter count. A standard linear regression carries few. Neural networks, taken up in the next chapter, commonly carry several thousand, and every one is another opportunity to fit noise.

Remedies

The most direct response is to shrink the model, dropping features or lowering the order of the specification until training and validation errors converge. Gathering more observations helps too, since noise averages out over a larger sample while genuine signal does not. An untouched test sample, and cross-validation where data are scarce, both guard against the illusion of a good fit.

Underfitting is the mirror image of overfitting, and it happens when patterns genuinely present in the data go uncaptured. Suppose the link between how a hedge fund performs and how large it is, with size taken as assets under management, is expected to be quadratic. Very small funds lack resources and spread costs too thinly, while very large ones may struggle to execute quickly without pushing prices against themselves. A linear model cannot represent that shape; it would report performance moving monotonically with size, and would be underfitted, where a nonlinear specification would do the job.

Leaving out relevant interaction terms is a second instance of the same failure. Underfitting is more likely in conventional models than in machine-learning approaches that impose no assumption about the structure of the relationship. Machine-learning models can underfit too, when the inputs are too few or of insufficient quality, or when the measures taken against overfitting have been applied too aggressively.

Model size sits between the two errors

Settling the size of a machine-learning model, which decides whether the data end up over-fitted, under-fitted or fitted about right, is an instance of the bias-variance tradeoff. An underfitted model omits relevant factors or interactions, so its predictions are biased but their variance is low. An overfitted model has little bias and a great deal of variance. Large models carrying many features sit at the low-bias, high-variance end and perform worse on the test sample than on the training set, while smaller models carry more bias and less variance, and can do better on the test sample.

Three fits to one scatter of points make the tradeoff concrete. A linear regression is not rich enough to describe the series and gives heavily biased predictions. A twentieth order polynomial contours the training points almost exactly and is an overfit, with a high variance of errors. A quadratic sits between the two and strikes the right balance.

In conventional econometrics it is common, though not universal, to hold back part of a sample so that a fitted model can be tried on observations of the dependent variable it has never seen. That creates the familiar split between the in-sample portion used for estimation and the out-of-sample portion, sometimes called a hold-out sample. Machine learning takes hold-out testing more seriously, since overfitting is a bigger danger and no specification arrives handed down by theory. Rather than two parts, the sample is cut into three.

What each of the three does

The training set is where model parameters are estimated, the intercept and slopes of a regression or the weights of something larger. This is the slice of data the computer genuinely learns from.

The validation set is where competing models are compared, to see which generalizes best outside the estimation sample. Once that comparison is made, the validation set has been contaminated by the act of choosing, and it can no longer deliver an independent verdict on the winner.

The test set exists for that reason. It is kept aside until the final model has been settled, and it then measures how effective that model really is. A model that generalizes fits the test sample almost as well as the training sample, because the machine has picked up the relationships between features and outputs without also fitting the noise, which does not repeat.

How much data goes where

There is no definitive answer here, and researchers land in different places. One rule of thumb gives roughly two-thirds of the sample to training and splits the remaining third equally between validation and testing. The more data points in total, the less the exact division matters. A training sample that is too small biases the parameter estimates, while a validation sample that is too small makes evaluation unreliable.

Ordering matters too. Where the output data have no natural sequence, as with cross-sectional data, all three samples are drawn at random from the whole dataset. Where the data form a time series, the convention takes training data from the beginning, then validation data, and places test data at the end, which tests the model on the most recent observations.

A three-way split assumes there is enough data to make all three pieces a reasonable size. When there is not, cross-validation squeezes more from what is available. Training and validation data are merged into one pool, with only the test data held back, and that pool is divided into equally sized sub-samples, the estimation running repeatedly with a different sub-sample left out each time. This is k-fold cross-validation, and values of k = 5 or 10 are the usual choices for n pooled observations.

Five folds in practice

Take k = 5. The pool is partitioned into five equally sized, randomly selected sub-samples, each holding 20% of it, labelled k₁ through to k₅. The first estimation runs on k₁ to k₄ and leaves out k₅. The second runs on k₁ to k₃ with k₅, leaving out k₄. Continuing produces k validation results, and averaging them measures performance without depending on which slice was held out.

Raising k enlarges the training sample used in each run, which is worth having when the overall number of observations is low. Pushing it to the limit gives k = n, so the count of folds equals the count of pooled observations. Every run then estimates on all but one observation and validates on that single point, which is leave-one-out cross-validation.

The price is computation, since the model is fitted k times rather than once. Cross-validation is a response to scarce data rather than a routine step, and where the sample supports three healthy sub-samples the simple split is preferred.

Reinforcement learning develops a policy for a sequence of decisions so as to maximise a reward. Algorithms of this kind now beat the strongest human players at chess and at Go, having learned by playing against themselves through systematic trial and error. In finance they are applied to technical trading rules, to splitting a large-volume trade so it sells quickly without moving the price against the seller, and to deciding how much of a position to hedge. One drawback is appetite for data: these algorithms need considerably more training data than other machine-learning approaches, and a machine built this way starts out making many errors and improves only with practice.

States, actions and rewards

States describe the environment the algorithm finds itself in, an action is the decision it takes, and a reward is what that decision earns. The aim is to pick the decision maximising the value of total subsequent rewards, and a discount rate may be applied to that total. After a number of trials the algorithm holds a running estimate of what action A is worth when the environment is in state S, denoted Q and called the Q-value. The worth of occupying state S is the largest Q-value on offer there, and the best action is whichever A attains it.

Exploration and exploitation

An algorithm that always takes the best action identified so far may settle on a suboptimal policy, because it never tries anything new. The way round this is a split between exploitation, taking the best action known, and exploration, trying a new one. A probability p is attached to exploitation and 1-p to exploration, and p is raised as trials accumulate and the algorithm learns which strategy works.

Updating a Q-value

Suppose action A is taken in state S and the total subsequent rewards, possibly discounted, turn out to be R. Under the Monte Carlo method the Q-value is revised as follows.

Temporal difference learning is the alternative. It looks only one decision ahead and assumes the best strategy identified so far will be followed from that point onward, so the reward realised over one step is combined with the current estimated worth of the state reached.

Source: the chapter’s Q values for four states and three actions, transposed so that each row is one state.

Real problems carry far more states and actions, and the state-action table then fills very slowly. Neural networks are used in that situation to estimate the whole table from the observations available, an approach known as deep reinforcement learning.

Natural language processing, also called text mining, is the branch of machine learning that reads and interprets human language, written and spoken. An early use in finance came when the US Securities and Exchange Commission deployed it to detect accounting fraud. Recognising particular words to establish the purpose of a message is another, letting a financial institution ask helpline callers why they are calling and route them without anyone triaging. Newswire statements can be sorted into categories such as corporate, government, human interest, environmental, social or education, or by country. Judging sentiment is a third use, and corporations read from social media how the market has taken a new product. Against a human reader the machine is vastly faster, cannot skip an aspect built into its design, and treats every document identically.

The three stages

The process has three stages: capturing the language, as a transcript or a written document, then pre-processing the text, then analysing it for the purpose at hand. Assuming the material is already electronic, pre-processing breaks into five operations.

To tokenize the passage is to separate it into words, normally discarding punctuation, spacing and special symbols, and converting capitals to lower case. Stop word removal strips out words carrying no informational value, present only to make a sentence flow, such as has, a, the and also. Stemming replaces words with their stems, so disappointing and disappointed both become disappoint. Lemmatization replaces words with their lemmas, so good, better and best all become good. Finally the n-grams are identified, groups of words carrying a specific meaning together that must be treated as a unit, such as red herring or San Diego.

Stemming and lemmatization exist so related words are handled as one, which simplifies what follows. Most straightforward tasks then treat what remains as a bag of words, meaning word order and the links between words are ignored apart from the n-grams.

Scoring a piece of text for sentiment

To classify a newsfeed announcement as positive, neutral or negative, a dictionary of sentiment words already sorted under those headings is applied, the words in each category are counted, and the proportions of positive and negative words calculated. The larger proportion settles the sentiment. Consider a short newswire item in which pre-tax earnings rose 0.1% on the year, disappointing investors even though total sales grew in double digits, with earlier safety worries now resolved and an analyst voicing relief about concerns that accidents would cost the firm market share.

Source: the classification of the sample newswire report given in the chapter.

Five positive words stand against only four negatives, so this feed reads as slightly positive. The example also exposes the weakness of the approach, since several negative words sit inside counterfactual clauses saying matters turned out better than feared. Careful research design is needed where sentence structure is formal or complex. Note too that the procedure involves no learning at all, since a fixed dictionary is applied under fixed rules. The alternative takes announcements human readers have already classified and lets an algorithm learn from them, which makes the exercise supervised learning.