QTA 1: Fundamentals of Probability
Statistics, econometrics and risk management all sit on top of probability. Many events a risk manager cares about carry a number, such as the loss on a portfolio or the count of defaults in a mortgage pool. Others do not: the type of a financial institution and the rating on a corporate bond are categorical variables, which probability handles too.
The sample space collects every possible outcome
An experiment produces one outcome, and the set of every outcome it could have produced is the sample space, written as capital omega. For returns on the S&P 500 it is the set of all real numbers, truncated below because a return cannot fall past -100%. For the direction of that return it is {Positive, Negative}, and for corporate defaults it is {Default, No Default}.
Dice are the standard illustration. If two identical six-sided dice cannot be told apart, only the pair of values is observed, so the sample space contains 21 elements. Colour one of them and {1,2} differs from {2,1}, giving 36 values. Ask for the sum instead and it becomes {2, … , 12}.
An event is a subset, and the event space collects the subsets
An event is any subset of the sample space, denoted by the lower case omega, and it may hold one outcome, several, or none. One holding exactly one outcome is an elementary event, and one holding nothing is the empty set. Events are sets, so set notation is how they are written. Take the sum of two dice: the odd sums form one event, and the sums of at least nine form another.
The event space, written as a script F, collects every combination of outcomes that can be assigned a probability, and it is abstract rather than tied to an application. Suppose an experiment has the two outcomes {A} and {B}. Four events follow: A occurs and B does not, B occurs and A does not, both occur, or neither. The event space is {A, B, {A, B}, and the empty set}, and holding finitely many outcomes makes it a discrete probability space.
Corporate default gives {Default, No Default, {Default, No Default}, and the empty set}. Two members look impossible, yet a probability of 0 can be assigned to each, and the event space holds every set that takes one.
Because events are sets, the operators of set algebra carry over unchanged. The intersection of A and B is the set of outcomes in A and also in B. The union is the set in A, or in B, or in both, counted once each. The complement of A is every outcome not in A. One further case is named separately: two events are mutually exclusive when either can happen but never both, so their intersection is empty.
Set operations on the two dice example
Return to the sum of two dice and take the event that the sum is at least nine alongside the event that it is odd. Their intersection holds the outcomes meeting both conditions, which happens when the sum is nine or 11, and their union holds every outcome in either set without duplication.
The complement of the odd sum event is the even sum event. Intersect those two and nothing survives, so the intersection is the null set and the pair is mutually exclusive.
Probability measures the likelihood of an event, attaching to every member of the event space a number that lies between 0 and 1, endpoints included. An event carrying probability 0 never occurs and one carrying probability 1 always occurs. Read one way, that number is the frequency with which the event would appear across a long run of independent repetitions, which is the frequentist interpretation. Finance rarely offers repeatable experiments, so the subjective interpretation is also in use, reflecting an individual set of beliefs. An executive fixing a probability for a recession, an interest rate increase or a rating downgrade in a scenario analysis or a stress test works in that mode.
The three fundamental principles
Three principles, known together as the axioms of probability, are stated using events, event spaces and sample spaces, with Pr the function returning the probability of an event.
Two properties follow at once, the complement rule and the addition rule.
When the sample space is discrete, each outcome carries its own probability and the probability of an event is the sum over the outcomes inside it.
Two fair six-sided dice are rolled and can be told apart, so the sample space holds 36 equally likely ordered pairs, each with probability 1/36. Event A is that the sum is at least nine, event B that it is odd.
Probability is often wanted inside a slice of the event space rather than across all of it, once something else is known. That is a conditional probability, computed on the condition that another event occurs.
Rearranging gives the multiplication rule, which is how a joint probability is built from a conditional one.
Conditioning behaves as though B were the new event space and A an event inside it. One fair die makes the point. The probability of rolling a three is 1/6. Told the number rolled is odd, the possibilities collapse to {1, 3, 5} and the probability of a three rises to 1/3.
Conditional and unconditional probabilities answer different questions
An unconditional probability carries no side information, while a conditional probability is quoted once some has arrived, and the two can differ enormously. The probability that a large financial institution fails is normally low, but once another one has already failed, as in 2008, a further failure becomes far more likely.
Conditioning is also how new information updates a prior view. Two students face an exam passed by 50% of test takers, so each looks like a 50% prospect. The average study time needed to pass is 200 hours. Student X studied less than 100 hours and Student Y more than 400 hours, and a survey finds pass rates of 10% and 80% for those groups, which become the two conditional probabilities.
Event A carries probability 16%, event B carries 48%, and the region in both carries 12%.
A conditional probability from bank supervision
Systemically Important Financial Institutions, or SIFIs, are designated as deeply connected to wide portions of the economy, which brings extra regulation and supervision. Suppose that in any given year the probability of one or more SIFI failures is 1%, that once at least one has failed there is a 20% chance the number failing is exactly 1, 2, 3, 4 or 5, and that more than five cannot fail. Each count carries an unconditional probability of 0.2%.
Let E1 be the event that one or more fail, so Pr(E1) = 1%, and let E2 be the event that two or more fail. E2 sits inside E1, so the intersection is E2 itself, with probability equal to the four counts from two to five, or 0.8%. Dividing gives Pr(E2 given E1) = 0.8%/1% = 80%. A failure in any year would be a surprise, yet a second then becomes likely.
One major application of conditional probability runs the logic backwards. Rather than narrowing an unconditional probability, the law of total probability rebuilds one from conditional pieces, and it works whenever the conditioning sets carry probabilities that sum to one.
The rule has to hold, because every outcome inside A falls in exactly one conditioning event, so slicing A up and reassembling it can only recover A.
Four mutually exclusive events B1, B2, B3 and B4 cover the sample space, and event A overlaps the first three. The probability inside A and B1 is 14%, inside A and B2 is 9%, and inside A and B3 is 12%, while A and B4 never overlap. Within B1, the part outside A carries 16%.
Independence runs through statistics and econometrics, and the definition is short. Two events are independent when the probability that one occurs does not depend on whether the other does.
Substituting that product into the definition of conditional probability shows what independence means in practice: conditioning changes nothing.
Why mutually exclusive events cannot be independent
The two ideas are often confused and they are close to opposites. If A and B are mutually exclusive, B cannot occur once A has occurred, so the outcome of A says a great deal about B. In numbers, with both probabilities strictly above 0, independence forces the joint probability above 0 as well, so the pair must sometimes turn up together. Mutually exclusive events have a joint probability of 0 by definition and fail that test. Genuine independence looks different, as with two events each having a 40% chance whose intersection carries 40% X 40% = 16%.
An economy contains two companies. Company A defaults with probability 10% during the year, Company B with probability 20%, and the two defaults occur independently.
Independence, like probability itself, can be restated to hold inside a slice of the event space. Two events are conditionally independent given a third event when the product rule holds among the conditional probabilities.
Neither version implies the other. A pair of events can be unconditionally dependent while remaining conditionally independent, and another pair can be unconditionally independent while turning dependent once a third event is imposed.
A pair that is dependent overall and independent within C
Take two events with Pr(A) = 40% and Pr(B) = 40% whose intersection carries 25% rather than the 16% independence would demand, so across the full sample space they are dependent. Now condition on a third event C. Inside the restricted space the figures become Pr(A given C) = 50% and Pr(B given C) = 50%, whose product is 25% and matches the conditional joint probability. So A and B are conditionally independent given C without being independent overall, and two consequences follow: Pr(A given B and C) = 50%, and Pr(B given A and C) = 50%. A common driver C generates the apparent link, and holding it fixed makes it disappear.
Credit risk is full of this structure. Two borrowers in one industry default together more often than their standalone default probabilities suggest, yet conditional on the state of that industry their defaults may be close to independent. Portfolio models rest on that split between a common factor and the idiosyncratic part.
| Relationship | Condition that defines it | Effect of learning that B occurred |
|---|---|---|
| Mutually exclusive | Joint probability is 0 | A is ruled out |
| Independent | Joint probability equals the product | Pr(A) does not move |
| Conditionally independent given C | Product rule holds once C is imposed | Nothing changes if C is known |
Source: the definitions in the chapter. The third column is an interpretation for study purposes.
Bayes rule builds one conditional probability out of another conditional probability and two unconditional ones. It is the definition of conditional probability rearranged, and among the most used results in finance.
The derivation takes one line. The definition of conditional probability makes the joint probability Pr(A given B) times Pr(B), and reversing the roles makes it Pr(B given A) times Pr(A). Setting the two equal gives the rule. Pr(B) is the prior probability held before the evidence, and Pr(B given A) is the posterior held after it.
Suppose 10% of fund managers are superstars. In any year a superstar beats the benchmark by more than 5% with probability 20%, while a normal fund manager manages it with probability 5%. One manager has just beaten her benchmark by 7%.
When Bayes rule tells you nothing new
If A and B are independent, then Pr(A given B) = Pr(A) and updating leaves the probability where it started. If B is a subset of A, then Pr(B given A) = Pr(B)/Pr(A), which depends only on unconditional probabilities and never moves. Useful updating needs evidence that is informative without being conclusive.