QTA 13: Simulation and Bootstrapping

Every simulation begins by generating random values from an assumed distribution, and here the language runs ahead of the reality. Numbers produced by a computer are not random. They come out of deterministic functions, built so the output is hard to predict and looks random to most statistical tests. Values of that kind are pseudo-random numbers, and the machinery behind them is a pseudo-random number generator, or PRNG.

A PRNG has to start somewhere, and its starting point is the seed value, which fixes the internal state of the generator. Run the same generator twice from one seed and the identical sequence comes back, without exception. That sounds like a defect and is in fact among the most useful properties the machinery has.

Two uses for reproducible output

The first is replication across experiments. Because a sequence can be recreated on demand, rival models or competing estimators can be compared on identical simulated data, which strips out the noise that would otherwise pollute the comparison. The same feature lets a result be reproduced years later, which matters for regulatory compliance.

The second is coordination across machines. Large portfolios holding thousands of instruments are driven by a common set of fundamental factors, and a distributed risk calculation must feed every machine the same simulated factor values, otherwise the joint behaviour of the instruments falls apart. A common seed across a cluster keeps those factors aligned.

One practical point matters before any distribution can be simulated. Generators return standard uniforms, meaning draws from Uniform(0,1), so those have to be converted into draws from whatever distribution the DGP names.

Suppose an experiment calls for data from a heavy-tailed distribution, for instance a generalized Student’s t with six degrees of freedom. The textbook definition of that variable is a ratio.

The definition cannot be applied as it stands, because the generator supplies uniform values and nothing else. A general result rescues the situation. Any variable X with cumulative distribution function F_X has the property that F_X(X) is uniform. Read backwards, that gives the tool used in practice: push a uniform through the inverse cumulative distribution function and a draw from the target comes out.

Four independent uniforms therefore become four independent draws from the generalized Student’s t. Each simulated value sits on the horizontal axis at the point whose cumulative probability equals the uniform behind it, so reading across to the curve and dropping down performs the whole transformation.

The same construction handles a discrete random variable, which also has a cumulative distribution function. That function climbs in steps, and a uniform landing inside one step returns the outcome attached to it.

Once values can be produced from the assumed distribution, an experiment follows five steps whatever is being approximated.

1. Draw a data set according to the assumed DGP.
2. Evaluate the function or statistic of interest on that data set.
3. Repeat the first two steps until b replications exist.
4. Estimate the quantity wanted from the b stored values.
5. Measure accuracy with the standard error, and raise b if it is not small enough.

Replications and sample size are different counts

Two counts turn up in every experiment and they are easy to confuse. The letter b counts replications, meaning how many times the whole exercise repeats, while n counts the observations inside one simulated sample. In the simplest case each replication yields a single data point. Elsewhere it generates a series of n observations, and the statistic is computed on that series. Either way the b stored values are independent by construction.

Estimating a mean then amounts to averaging them.

Since that average runs over independent and identically distributed variables, the Law of Large Numbers applies and the approximation settles on the true value as replications grow.

The Central Limit Theorem covers the same average, which is what makes the accuracy of a simulation measurable rather than a matter of judgement.

Plenty of moments are analytically tractable: the mean and variance of a normal variable can simply be written down. Tractability runs out quickly though, mainly for variables built out of complex dynamic models and for expectations of complicated nonlinear functions.

Accuracy is governed by the variance of the simulated expectation, being the variance of the transformed variable over the replication count. That numerator is unknown and gets estimated from the simulated values.

Quantiles as well as means

The same output supports quantiles. Rank the b draws in ascending order and take position b multiplied by alpha: that empirical quantile estimates the true one. Value-at-risk asks for the lower quantiles of a return distribution, with alpha typically 0.1%, 1%, 5% or 10%.

Finite sample properties of an estimator

Most inference here rests on asymptotic arguments from the Law of Large Numbers and the Central Limit Theorem, which describe behaviour as a sample grows without bound. Real samples are finite, and the approximation can be poor when n is small. An experiment tabulates the finite sample distribution of an estimator instead: draw a sample, estimate the parameter, repeat b times, and bias, variance and quantiles all follow from the b estimates.

Source: each row holds b = 1,000 replications of a sample mean, drawn from a normal variable whose mean is 1.

The standard error of a simulated expectation is the standard deviation of the simulated values over the square root of b, so it shrinks in proportion to one over the square root of the replication count. That rate holds for any moment approximated this way, and it survives a wildly nonlinear function, because the draws stay independent.

Pricing a European call option shows the arithmetic in a setting worth caring about. The option pays the greater of zero and the terminal stock price less the strike, so the payoff bends at the strike and its expectation cannot be recovered from the expected stock price. A model for the terminal price is needed, and the standard choice makes the log stock price normally distributed.

Run that experiment on a two-year European call struck at the level of the underlying index, letting the replication count climb in multiples of four, and every fourfold rise in b roughly halves the error: 71.23 falls to 30.19, then to 15.39, and by 12,800 replications it stands at 3.64. Precision bought this way is expensive. At 12,800 replications the price comes out at 275.62, so a two-sided 95% confidence interval reaching 1.96 standard errors either side still spans about USD 14, roughly 5% of the option value.

Raising b is the obvious route and often the wrong one, since a complex simulation can be slow. Two alternatives sharpen accuracy at a fixed replication count: antithetic variates and control variates.

The variance of a total formed by adding random variables is the summed variances plus twice every distinct pairwise covariance. Independent replications zero those covariances out, while negative ones make the total vary less than independence allows. That is the opening antithetic variates exploit.

They are a second set of values built to correlate negatively with the values already drawn, arriving in pairs from a single uniform: if the first is U, its partner is 1 minus U. Sending both through the inverse CDF carries that negative correlation into the target distribution, and only b divided by 2 uniforms fill b simulated values.

One condition governs whether the technique delivers. Negative correlation has to survive the function being applied, which is guaranteed only when that function is monotonic in the draw. A payoff that turns around converts negatively correlated inputs into positively correlated outputs, and the standard error then rises. The method fills the probability distribution more evenly, at a cost well below running more replications.

Return to the two-year call. Its value is a nonlinear function of the simulated normal draws, so pairing a draw with its negative gives a correlation between the resulting option prices of only -43 percent. That puts the reduction in the standard error at approximately 25 percent, worth about 78 percent more independent replications.

Control variates attack the same problem from another angle. Any simulated value splits into the quantity being approximated plus a mean zero disturbance. A control variate is a second variable, computed on the same draw, whose own expected value is zero but which correlates with that disturbance. Adding it cannot bias the approximation, and the correlation lets part of the error cancel.

A good control variate has two properties. It should be cheap to build from the draw already in hand, since a slow one wastes computation that would have bought more replications, and it should correlate strongly with the function whose expectation is wanted, because that correlation determines how much error cancels. The weight comes from a regression of the simulated values on the control variate.

Valuing a call option with a put option as the control

The call option experiment supplies a natural candidate. A European put on the same underlying, with matching strike and maturity, is driven by the same simulated terminal prices, so valuing it alongside the call costs nothing extra. It pays the greater of zero and the strike less the terminal price and has a closed-form value, which makes a mean zero variable easy to build: simulate the put payoff and subtract the analytical Black-Scholes-Merton price.

One experiment runs on the variable under study and on a second whose properties are already known, using the same draws for both. Sampling error then contaminates the known problem much as it contaminates the unknown one, so comparing the simulated answer against the analytical one calibrates away part of the error. Here the simulated call and put prices correlate at -44 percent.

Source: the price column uses independent draws. The remaining columns give the standard error under each of the four schemes.

Reading across any row shows both effects. At 12,800 replications the error falls from 3.64 under independent draws to 2.75 with antithetic variables and to 3.21 with the control variate, while the two together reach 2.43. Used jointly they cut the standard error by about 33 percent against independent sampling, equivalent to roughly 125 percent more independent replications. The two are complementary.

Both approaches use observed data, but at different points. A simulation uses it only to calibrate, meaning to pin down key model parameters such as the mean and standard deviation of the S&P 500, which then need an assumed distribution for the standardised returns before the DGP is complete. A bootstrap goes at the data directly, modelling nothing and assuming no distribution, and the only thing it simulates is which observations get chosen.

One fact makes the method work. Whatever distribution a bootstrap wants to sample from is the one that generated the data already in hand, and although nobody knows what it is, the observations themselves resemble it in a probabilistic sense. That is the advantage over Monte Carlo: no DGP has to be specified, so no DGP can be misspecified.

How the iid bootstrap builds a sample

Two classes of bootstrap are used in risk management, and the simpler is the iid bootstrap. For a sample of size m from a data set holding n observations, generate m integers at random from 1 through n, drawing with replacement, and use them as indices to pick which observed values go in. The result is a bootstrap sample.

Take five observations drawn from a set of twenty. The first bootstrap sample might use observations 3, 6, 2, 5 and 10, and the second 6, 3, 2, 18 and 15, so the two partly overlap. A later sample might use 8, 12, 8, 2 and 5, repeating observation 8. Repeats are not a flaw in the procedure. They are the procedure.

The iid bootstrap suits observations that are independent through time, which is defensible in places. Take the S&P 500 during 2005 or during 2017, both calm years with low volatility: the distribution of daily returns looks much the same in January as in December, so picking days at random does no damage. Financial data are more often dependent through time, and resampling single days destroys the structure that matters.

The circular block bootstrap, usually shortened to CBB, is the simplest repair. Rather than drawing one observation at a time, it draws blocks of q consecutive observations with replacement, and whatever dependence lives inside a block travels into the bootstrap sample untouched.

The word circular describes what happens at the end of the data. With 100 observations and a block size of 10 there are 100 blocks, not 91. The first 91 hold ten consecutive observations each, and the last nine wrap around, as though observation one followed observation 100, so the block starting at observation 92 finishes with observations 1 through 9.

Blocks are drawn with replacement until the sample is long enough, and if they overshoot the last observations are discarded: with a block size of ten and a target sample of 75, eight blocks are selected and the final five observations dropped. The block length q has to be long enough to capture the dependence and short enough to leave a decent number of distinct blocks, and the usual rule of thumb sets it at the square root of the sample size.

Bootstrapping a value-at-risk estimate

In the leading application, market risk, the p-value-at-risk is the loss level the portfolio exceeds with probability 1 less p over a stated horizon.