QTA 2: Random Variables

Univariate random variables are sorted by the kind of number set they produce. A discrete random variable produces distinct, separated values, while a continuous random variable produces values from an uncountable set, such as any point on the real line or inside a stated interval of it.

The set of values a variable may take is called the support of the function. For a variable flagging default the support is the pair zero and one; for the face of a die it is the six integers from one to six.

Finite, countably infinite and uncountable supports

Discrete does not have to mean small. The support may be finite, or infinite provided it is countable, which means its elements can be lined up against the integers one for one. A variable counting the trading days until a firm is downgraded has no upper limit, yet every value it produces is a whole number, so it stays discrete. What separates it from a continuous variable is the presence of gaps: between neighbouring support values lies a stretch of the line the variable never visits.

The split matters because the two classes are described by different objects. A discrete random variable is described by a probability mass function, a continuous one by a probability density function. In practice the difference is of little consequence, since the concepts introduced for discrete variables carry over with sums replaced by integrals, and the CDF is common to both. One boundary case needs a name: a degenerate random variable puts all of its probability on a single point, so it always returns the same value.

The probability mass function, written PMF, answers the narrowest question available: given a value in the support, how likely is the variable to return exactly that value. Because its outputs are probabilities, only two conditions are required. Every value returned must be non-negative, and the values returned across the whole support must add to one. The first rules out negative chances; the second forces the variable to produce something.

Writing the PMF as f with a subscript X and an argument x keeps the underlying variable visible and separate from the realization being tested.

The Bernoulli mass function

The leading example of a discrete variable with finitely many values is the Bernoulli random variable, which returns either zero or one and nothing else. It appears constantly in risk work, since binary events such as the default of a loan translate into it directly. Write p for the probability of returning one.

Substituting the two admissible inputs shows what the compact form does. At zero it collapses to 1 – p and at one to p, so the probability of observing a zero is 1 – p and of observing a one is p. The same function can be written out as a short list instead.

Both forms satisfy the requirements: each output lies between zero and one, and the outputs add to p plus 1 – p. A mass function on a larger support is checked the same way.

The counterpart of the mass function is the cumulative distribution function, written CDF and denoted with a capital F. Where the mass function isolates one value, the CDF accumulates: it returns the total probability of observing a value less than or equal to its input, written Pr(X ≤ x).

Two consequences follow at once. The CDF is monotonic and increasing in x, because moving the input to the right can only add probability. And it is defined for every real number, not merely for support values, since the question can be asked anywhere on the line. A Bernoulli variable therefore has a mass function defined at two points and a cumulative distribution function defined everywhere, in the shape of a step function.

Neither function holds information the other lacks. Going from mass to cumulative means adding every mass at or below the point of interest, and going back means differencing, so for a variable supported on consecutive integers the mass at a point is what the cumulative function gained on arriving there.

The expected value of a random variable, written E[X], is a weighted average of everything the variable can produce, with the mass function supplying the weights. For a discrete variable the calculation is nothing more than multiplying each value by its probability and adding the column.

Applied to a Bernoulli variable the sum has two terms and the result is memorably simple, since the mean of an indicator is the probability it flags.

Feed a random variable into a function and what emerges is another random variable, so it has an expectation of its own. The recipe does not change: weight each output of the function by the probability of the input that produced it, then add.

Applied to the exponential of a Bernoulli variable, the two outputs exp(0) and exp(1) carry probabilities 1 – p and p, so the expectation is (1 – p) + p exp(1).

Valuing a payoff is an expectation of a function

A call option is a derivative contract whose payoff at expiry is a nonlinear function of the price of the underlying asset. Let S stand for whatever the underlying asset is worth on the expiry date and K for the strike price. The payoff is c(S) = max(S – K, 0). Finish above K and the holder collects S – K; finish at or below K and the call option expires worthless. Valuing it means computing the expectation of a function of a random variable.

The expectation operator takes a random variable and returns a weighted average of its possible values. Its most useful property is that it is a linear operator, so constants and sums pass straight through it. In the rule below, a, b and c are constants while X and Y are random variables.

Two results drop out at once. The expectation of a constant is that constant, so E[a] = a, and an expectation is itself a constant, so E[E[X]] = E[X].

Linearity is a property of the operator, not a licence to move it through anything. For a nonlinear function the operator does not pass through, so in general E[g(X)] is not equal to g(E[X]). The reciprocal makes the point sharply, since E[1/X] differs from 1/E[X] for any non-degenerate random variable.

Jensen’s inequality gives the direction of the error

When the function has a definite curvature, the gap between the two orderings has a known sign. A function h is convex if the chord joining any two points on its graph lies at or above the graph between them, and g is concave if the chord lies at or below it. Convexity is the case that keeps coming up, since a great many derivative positions bend upward as the underlying asset gets more expensive.

A moment is what comes back when the expectation operator is applied to a function chosen to isolate one feature of a distribution. Four are used routinely in finance and risk management, describing location, spread, asymmetry and the weight of the tails. The first is the expectation itself, and every higher moment shares one template, differing only in the power applied.

Moments built on deviations from the mean are called central moments, since they are centred on the first moment. Non-central moments skip the subtraction. They are less informative alone, because shifting the mean shifts every one of them, but they are easier to compute, so the usual route is to compute them and convert.

The four named moments

The mean is the first moment, denoted by the Greek letter mu, and it reports the average value of X, also described as the location of the distribution. The variance is the second central moment, denoted by sigma squared, and it measures dispersion by averaging the squared deviation from the mean, so it is indifferent to the direction of a deviation.

The standard deviation is the positive square root of the variance and is usually the more convenient of the two, because its units match those of the mean. If X is a return measured in percent, the mean and the standard deviation are both percentages while the variance is in squared percentages.

The final two named moments are standardized rather than merely central. Skewness divides the third central moment by the cube of the standard deviation, and kurtosis divides the fourth by its fourth power.

An odd power keeps the sign of the deviation, which is why skewness detects asymmetry, and a negative value says a large negative outcome is more likely than an equally large positive one. Measured asset returns usually come out negatively skewed: gains arrive more often, while losses, when they arrive, are larger.

An even power discards the sign, so kurtosis responds to large deviations either way and says nothing about which side they favour. Its benchmark is the normally distributed random variable, whose kurtosis is 3. Anything above 3 is heavy-tailed or fat-tailed, as financial return distributions commonly are, and subtracting the benchmark gives excess kurtosis.

Standardizing a random variable

Both standardized moments rest on one construction. Subtract the mean and divide by the standard deviation, and the result has mean zero and variance one, since mu and sigma are constants that pass through the expectation operator, adding a constant leaves a variance unchanged, and multiplying by b multiplies a variance by b squared. Because skewness and kurtosis are defined on that quantity, both are unit-free and comparable across variables with quite different means and variances.

Rescaling data is routine. Returns get quoted as percentages, series get demeaned so the average is zero, and positions get levered or shorted. All of these are linear transformations. Write the transformed variable as Y = a + bX, with a and b constant. The constant a is a location shift and b is a scale, names that describe exactly which measures each one touches.

The mean result follows from linearity of the expectation operator. The variance result reflects the fact that variance measures deviations around the mean, so sliding the distribution sideways by a leaves it untouched while stretching by b multiplies every squared deviation by b squared. The absolute value appears because a standard deviation is never negative.

Both higher named moments are defined on the standardized quantity, and standardizing strips out a location shift and a positive scale before either power is applied. So if b is positive, making Y an increasing transformation of X, both measures come through untouched and Y reports the same skewness and kurtosis as X. A negative b splits the two. Skewness uses the third power, so reversing the direction keeps the magnitude and flips the sign, while kurtosis uses the fourth power and is blind to direction. Shorting a position therefore turns a negatively skewed exposure into a positively skewed one while leaving its tail weight alone.

Source: the properties of linear transformations set out in the chapter, with the quantile-based measures listed alongside the four moments.

A continuous random variable can take any of an infinite number of values, or more precisely it is a random variable with continuous support. Some have support across the whole real number line, as yields and returns do, and others are confined to an interval, in the way a chance of rain tomorrow has to sit inside the range 0 to 1.

Continuous variables use a probability density function, written PDF, in place of the mass function. The density returns a non-negative value at every point in the support, but that value is not a probability, because a single value on its own carries probability 0.

Why a single point carries no probability

The reason is a counting argument. There are infinitely many values in the support, so if each carried a probability above zero the total would exceed one and break an axiom of probability. Watching the total get spread thinner makes that less startling. Take a 10-sided die showing the numbers 0.05, 0.15 and so on up to 0.95, where each face has a 10% probability. Give the die 1,000 sides showing 0.0005 up through 0.9995, and each face carries a 0.1% chance. As the number of sides grows the chance of any specific value shrinks towards zero. Carry that process to its limit, to a die whose sides never run out, and what remains is a continuous distribution, which is why probability is measured over ranges rather than at points.

The requirement that a mass function sums to one survives the move, with the sum replaced by an integral. Areas under the density carry the meaning: the area beneath the curve between two points is the probability of landing between them, and the height indicates which regions are more likely.

The cumulative distribution function needs no translation

The CDF of a continuous random variable means what it meant for a discrete one: the total probability that X is at or below the input. With the point masses gone, the density and the cumulative function are linked by calculus.

One consequence catches people out. Because Pr(X = a) and Pr(X = b) are both zero for a continuous random variable, the four probabilities Pr(a ≤ X ≤ b), Pr(a < X ≤ b), Pr(a ≤ X < b) and Pr(a < X < b) all come to the same number. For a discrete random variable they need not be, since an endpoint may carry mass of its own.

Expectations follow the same substitution of an integral for a sum. That looks like a different object from the discrete expectation, but an integral can be approximated by a sum, so the two are versions of one weighted average. A discrete variable placing mass at seven points is a coarse approximation to a continuous density, and adding points improves it.

The most widely used continuous distribution in finance and economics is the normal, also called the Gaussian distribution, whose density is governed by two parameters fixing the mean and the variance. Its cumulative distribution function has no closed form expression, so it is evaluated numerically. Working directly with a density is rare in any case, since common distributions have documented properties and their moments can be looked up.

Moments are the usual way to describe and compare random variables. Quantiles offer a second route, built on cutoffs rather than averages: they split the outcomes of a variable into buckets of roughly equal probability. For a random variable X, discrete or continuous, the a-quantile is the smallest number q whose cumulative probability reaches a.

Selecting a smallest value is only needed for distributions containing regions that carry no probability, where several points could satisfy the condition. In most finance and risk management applications the assumed distributions are continuous, so exactly one number q answers for each a. The quantile function is then well defined, and it is the inverse of the cumulative distribution function.

The cumulative distribution function converts values in the support into cumulative probabilities, which always lie between 0 and 1, and the quantile function runs that mapping backwards. Plotting the two side by side shows one graph as the reflection of the other through the 45 degree diagonal.

The median and the interquartile range

Two quantile-based measures are used constantly. The median is the 50% quantile, the point dividing the probability in half, and it measures location. In a symmetric distribution the mean and the median coincide; where a distribution is skewed they generally differ, and under negative skew the mean sits below the median.

The interquartile range, abbreviated to IQR, is the distance between the 75% quantile and the 25% quantile. It measures dispersion much as the standard deviation does, and for a normal distribution it is commonly set beside an interval two standard deviations wide.

Both depend on relatively central parts of the distribution and never on the extreme tail, which makes them less responsive than the mean and the standard deviation to what happens far out. Quantiles carry a stronger advantage too: they are always well defined, while moments including the mean may fail to be finite when a variable is very heavy-tailed.

The mode

One further measure of central tendency, the mode, records common tendency rather than average or midpoint: it marks where observations pile up most thickly, and for a continuous random variable it is the highest point of the density function. A variable may have more than one mode, and the convention is to treat every local maximum of the density as one. Two modes make a distribution bimodal, and more than two make it multimodal. Bimodal loss distributions are familiar in credit portfolios, where one hump gathers the ordinary years and a second, far to the right, gathers the years in which correlated exposures fail together.