QTA 3: Common Univariate Random Variables

A Bernoulli random variable produces one of two values, 0 or 1, and nothing else. It fits any binary situation: a bull or bear market, a corporation that defaults or does not, a transaction flagged as fraudulent. Convention calls 1 a success and 0 a failure, which reads oddly in risk work, since 1 usually marks the unwelcome state.

One parameter, p, controls everything. It is the probability of observing a 1.

Its CDF is a step function with levels zero below 0, then 1 minus p from 0 up to but not including 1, and one from 1 onward. Writing Y ~ Bernoulli(p), where the tilde reads as “is distributed as”, the moments follow from the definition of an expectation.

With p close to 0 or close to 1, one outcome dominates the sample and dispersion is small. Uncertainty peaks in the middle: the variance is maximised at V[Y] = 25% when p = 50%, where failure and success are equally likely.

Where the Bernoulli appears in risk

Credit modelling uses it constantly. Set the observed value to 1 on default and 0 otherwise, and the problem reduces to estimating p. That probability of default is built from borrower-level inputs such as a credit score together with wider drivers such as the GDP growth rate.

The same device tracks extreme losses. Assign a 1 to any month whose portfolio loss exceeds Value-at-Risk and a 0 otherwise, and the sequence becomes raw material for backtesting a VaR model.

Run the same Bernoulli experiment n times, keep the trials independent, and count the successes. That count is a binomial random variable with two parameters: n, the number of experiments, and p, the probability any one succeeds. If X1 through Xn are independent Bernoulli variables sharing p, their sum is written B(n, p).

The leading term, read aloud as “n choose y”, tallies the orderings of the n experiments that deliver exactly y successes.

A two-coin illustration makes the counting visible. Call heads a success, so that Y ~ B(2, 0.5). Two flips produce four sequences, each carrying probability (1/2) squared, or 1/4. One yields no heads, two yield a single head, one yields two, so the masses at 0, 1 and 2 are 1/4, 1/2 and 1/4. The CDF is the running total, defined using the floor function so a non-integer argument drops to the integer below.

A binomial variable is never negative, is always an integer, and cannot exceed n. A small p pushes mass toward zero and gives a right-skewed shape.

The binomial counts successes across a fixed number of experiments. The Poisson counts events across a fixed span of time, with no upper limit. Values are non-negative integers, and one parameter governs it: the hazard rate, written lambda, the average number of events per interval.

Shape follows lambda. A small lambda concentrates mass near zero and leaves a long right tail, so the distribution is right-skewed. By lambda of 15 the shape is nearly symmetric, the signal that a normal approximation is becoming usable.

Infinite divisibility, and why it matters for time series

The Poisson has a property few discrete distributions share: it is infinitely divisible. Add two independent Poisson variables and the result is Poisson with the two rates summed. Aggregating counts from weeks into months therefore leaves the family unchanged and only rescales the parameter, so anyone modelling default counts or operational loss events can pick the sampling frequency freely.

In survival modelling lambda is read as a hazard rate, the intensity at which events arrive, which connects the Poisson to the exponential.

The uniform is the simplest continuous random variable. Every value between a and b is equally likely and nothing outside can occur, so the density does not depend on the argument at all.

The CDF returns zero below a, rises in a straight line with slope one divided by (b minus a), and stays at one above b. Setting a to 0 and b to 1 gives the standard uniform, with mean 1/2 and variance 1/12. Every other uniform is a stretched and shifted standard uniform.

That is why simulation engines generate standard uniforms first. Feed one into the inverse of any CDF and the output follows that distribution, so Monte Carlo work for pricing and economic capital rests on it.

Take Y ~ U(-2, 4). The mean is -2 plus 4, all over 2, which equals 1. The variance is the squared width, 36, divided by 12, which equals 3. For the chance that Y sits between -1 and 2, the numerator is 2 minus (-1), which is 3, and the denominator is 4 minus (-2), which is 6, so the probability is 0.5.

For a standard uniform, probability between two points depends only on the gap. Landing between 0.2 and 0.4 carries probability 0.4 minus 0.2, or 20%.

No distribution is used more heavily in risk management than the normal, also called the Gaussian distribution after Carl Friedrich Gauss, or the bell curve after its shape. Two parameters describe it: mu, the mean, and sigma squared, the variance.

Several properties explain the popularity. Many continuous quantities are approximately normal already, and many discrete ones can be approximated by a normal. The Central Limit Theorem places it at the heart of hypothesis testing, the process by which data are used to judge a statistical statement. Normal variables are infinitely divisible, which suits models treating prices as evolving continuously. The family is closed under linear operations, and estimators derived under normality often have simple closed forms. The Student’s t, the chi-squared and the F all descend from it.

Symmetry gives the normal zero skewness, and its kurtosis is 3. That value is the yardstick against which other distributions are called heavy-tailed or fat-tailed, meaning they put more probability on extreme deviations.

The special case of the standard normal

Setting mu to 0 and sigma squared to 1 gives the standard normal, denoted Z, with lower-case phi for its density and upper-case phi for its CDF. Two families of quantile are quoted constantly: interval coverage measured in standard deviations, and the critical values leaving a chosen tail probability outside.

Source: Table 3.1, left panel, of the GARP chapter on common univariate random variables.

Source: Table 3.1, right panel, same chapter. Each tail holds half the stated tail probability.

Practitioners often substitute 2 for 1.96, which is close enough for conversation and wrong for a report. A standard normal puts 2.5% above 1.96 and 2.5% below -1.96, leaving 95% between. Observations beyond 3 sigma turn up about once in 370 draws.

Sums behave conveniently too. Adding two independent normal variables gives a normal with the means added and the variances added. Independent normal daily log returns therefore imply normal weekly and monthly log returns, which lets a single daily volatility estimate be scaled across horizons.

Every normal variable is a shifted and stretched standard normal. Start with Z ~ N(0, 1) and set X as mu plus sigma times Z, so E[X] equals mu and V[X] equals sigma squared. Running that backwards is called standardising.

Symmetry halves the work. The chance that Z falls below a negative value equals one minus the chance it falls below the matching positive value, which is why published tables carry only the positive side.

The same mechanics generate the return quantiles behind a daily VaR figure. Suppose a stock return is normal with a daily mean of 8% divided by 252 and a daily variance of 20% squared divided by 252, which puts the daily mean at 0.031% and the daily standard deviation at 1.26%. Multiplying the standard normal quantiles by that standard deviation and adding the mean gives -3.86% using -3.09, then -2.89% using -2.32, and -2.04% using -1.645.

When a normal can stand in for a discrete distribution

A binomial with plenty of experiments, or with a middling probability, has a nearly symmetric mass function that a normal reproduces well. The working rule is that np and n(1 minus p) should both exceed 10, in which case B(n, p) is approximated by N(np, np(1-p)). Taking n as 25 and p as 0.4 gives np equal to 10 and n(1 minus p) equal to 15, so the approximating normal is N(10, 6). This is an application of the Central Limit Theorem.

A Poisson can be approximated in the same spirit, by a normal with mean and variance both set to lambda, and the fit improves as lambda grows. The standard recommendation is far more demanding: lambda should exceed 1,000 before the approximation is treated as accurate.

A variable Y is lognormal when the natural logarithm of Y is normal. Equivalently, take a normal variable and exponentiate it.

The consequence that matters commercially is a hard floor at zero. X can be as negative as it likes, but the exponential of any number is positive, so Y never turns negative. Model a stock price as normal and you concede a positive probability, however tiny, that the price falls below zero. The lognormal removes that defect, and it underpins the Black-Scholes-Merton option pricing model.

Note that mu and sigma squared belong to the underlying normal variable, not to Y, and the mean picks up an extra term.

Convexity puts the expectation of the transform above the transform of the expectation, so E[Y] always sits above exp(mu). Figures typical of annual equity returns show the size of the effect. With mu at 8% and sigma at 20%, the expected value is 1.105. Raising sigma to 40% lifts it to 1.174, and instead moving mu to 16% at sigma of 20% gives 1.197.

Extra variance raising the mean looks strange until the asymmetry of returns is taken into account. The worst possible return is -100%, since a position cannot lose more than its value, while nothing caps the upside. A nonlinear transform of a symmetric normal gives an asymmetric result, so Y is right-skewed even though X is not.

Square a standard normal variable, then add several such squares from independent draws. The result is a chi-squared variable.

The single parameter v, the Greek letter nu, is the degrees of freedom and must be a positive integer. The name reflects how much data remains for testing: fitting consumes observations, so n observations used to estimate k parameters commonly leave n minus k degrees of freedom.

A sum of squares cannot be negative, so the density exists only for positive values. At v equal to 1 the mode sits at 0. Raising v pushes the density right and spreads it out, as the moments predict. It is right-skewed for every v, though the skewness fades as v rises, and once v passes 25 a chi-squared is well approximated by a normal with mean v and variance 2v.

Where a chi-squared shows up in risk work

Hypothesis testing about model parameters is the main application. Any statistic built from squared standardised deviations lands here: goodness-of-fit tests, tests of restrictions on regression coefficients, and the backtesting statistics that judge whether a VaR model throws too many exceptions. It also models quantities that are always positive, such as the VIX Index, where a symmetric distribution would be plainly wrong.

The Student’s t is a close relative of the normal with visibly heavier tails. It was developed for testing hypotheses on small samples, where the extra tail weight compensates for uncertainty in an estimated variance. One parameter controls it, again written v, and its main effect is on the tails.

Dividing a standard normal by another random variable rather than a constant is what thickens the tails, because the denominator occasionally comes out small and inflates the ratio. As v climbs the denominator becomes more predictable, and the Student’s t converges to a standard normal.

More generally the moment of order m exists only when v exceeds m, so a low parameter can leave a distribution without a finite variance at all. Kurtosis is always above 3 wherever it exists, which is the formal statement that the Student’s t is heavier-tailed than the normal.

Standardised and generalised versions

Fixing the variance by the degrees of freedom is awkward when tail shape and scale need setting separately. Multiplying Y by the square root of (v minus 2) divided by v gives mean 0 and variance 1 for any v above 2, which is the standardised Student’s t. Recentring and rescaling to a chosen mean and variance produces the generalised Student’s t, written Gen. t with subscript v and arguments mu and sigma squared, so location, scale and tail weight become independent.

The gap in tail behaviour is large enough to change a risk number. For a normal variable an observation more than 4 sigma from the mean arrives about 1 in 15,000 times. For a standardised Student’s t with v set to 8, the same event occurs about 1 in 580 times, more than twenty times as often.

The F is the other distribution that appears whenever hypotheses about model parameters are tested. It carries two parameters written as subscripts: v1 is the numerator degrees of freedom and v2 the denominator degrees of freedom.

Values are always positive, since both chi-squared components are. The PDF and the CDF involve integrals with no closed form, so software or tables do the evaluation. The shape resembles a chi-squared density.

Reading the two parameters

The hypothesis under test fixes v1, typically a small number such as 1, 2 or 3, because it counts the restrictions being tested jointly. Sample size fixes v2, which is therefore relatively large. Once v2 is large its chi-squared component has mean 1 and a variance close to zero, so the denominator behaves like a constant and the distribution collapses toward the numerator, with mean 1 and variance 2 divided by v1.

Comparing configurations makes the roles clear. An F with v1 equal to 3 and one with v1 equal to 4, both at v2 equal to 10, share a mean of 1.25, since the mean depends only on the denominator parameter. The extra numerator degree of freedom shifts the distribution right and pulls probability toward that mean, so the version with v1 equal to 3 has the larger variance. Raising the denominator parameter instead moves it left and thins the right tail: with v1 equal to 3 and v2 taken to infinity, the mean is 1 and the variance is 2/3.

Two connections tie the F back to what came before. With v1 equal to 1 and v2 large, the F is close to a chi-squared with one degree of freedom. More usefully, squaring a Student’s t with v degrees of freedom gives an F with 1 and v degrees of freedom, which is why a two-sided t test and the matching F test always agree.

Where the Poisson counts events, the exponential measures the wait until one happens. A single parameter, beta, sets both moments: E[Y] equals beta and V[Y] equals beta squared.

The link with the Poisson is direct. If loan defaults per quarter follow a Poisson process, the gap between one default and the next is exponential, with the two parameters tied to one another. A count model and a waiting-time model are two views of the same arrival process.

Memorylessness and what it does not say

Exponential variables are memoryless: the distribution of remaining waiting time does not depend on how long the wait has already lasted. Suppose time to default for a company is exponential with a known and constant beta. The probability of default within the next year, days 0 to 365, is then identical to the probability between a year and two years from now, days 365 to 730, provided the company survived the first year.

One caution attaches. Memorylessness conditions on survival to the start of the window and holds the window length fixed. It does not claim that defaulting within the first year is as likely as within the first two.

Some quantities in risk cannot leave the interval from 0 to 1. A probability of default is one, and a recovery rate is another. The Beta distribution is built for that support, and it commonly represents uncertainty about a probability of success rather than the outcome itself. Two parameters, alpha and beta, jointly set the mean and the variance.

Shape responds in a way that is easy to remember. Raising alpha drags the distribution toward 1 and raising beta drags it toward 0, while raising both in proportion holds the mean and squeezes the variance. When both fall below 1 the density piles up against the boundaries, giving a U shape that suits outcomes tending to be near-total or near-nothing.

Two special cases connect the Beta to distributions already covered. Setting alpha and beta both to 1 flattens the density, so the Beta collapses to a standard uniform. When the two are similar and both reasonably large, above 10 as a rough guide, the Beta is well approximated by a normal.

Take alpha equal to 7 and beta equal to 3, leaning toward the upper bound. The mean is 7 divided by 10, or 0.7, and the variance is 21 divided by 1,100, giving 0.0191 and a standard deviation of about 0.138. As a model for recovery on a senior secured claim, that puts the central estimate at 70% with real uncertainty around it and never permits a value outside the range from 0 to 1.

Every distribution so far has a fixed shape once its parameters are chosen. Mixtures relax that. A mixture is assembled from two or more component distributions, and although the treatment here uses two, any finite number can be combined.

Construction runs in two stages. Draw from a Bernoulli variable W with parameter p. If the draw is a 1, take the observation from the first component; if it is a 0, take it from the second. Over many draws, 100 times p percent of the sample comes from the first component.

Moments of a mixture

Central moments of a mixture are awkward to compute directly. Non-central moments are not, since they are weighted averages of the component moments, and the central moments follow from them.

The payoff is a single fact: a mixture can be skewed and heavy-tailed even when every component is a normal variable with no skewness and a kurtosis of exactly 3. That is what makes mixtures useful for financial data.

Three mixtures that recur in practice

A bimodal mixture puts equal weight on two normal components with distinct means and small variances, and the density shows two separate peaks. A contaminated normal keeps both means at zero but pairs a common low-variance component with a rare wide one, leaving a density more peaked in the middle and heavier in the tails, much like a generalised Student’s t. Mixing components that differ in both mean and variance delivers skewness too: drawing 90% of the time from one normal component and otherwise from a normal with mean -1.5 and variance 2 gives a mean of 0 and a variance of 1 by construction, with a skewness of -0.96 and a kurtosis of 5.50, so the result is strongly left-skewed and mildly heavy-tailed.

The idea extends past two components. A Student’s t is itself an infinite mixture of normal components, each drawing its variance from a chi-squared distribution, which is another route to why its tails run so much heavier.