OR 4 – Risk Measurement and Assessment

Operational risk is the risk of a loss that arises because people, systems or internal processes fall short or fail, or because something happens outside the firm. Because the definition is built around things that go wrong, a firm cannot manage or measure this risk well without a careful record of what has already gone wrong inside its own walls. That is why many institutions treat the incident database as the base layer on which everything else in their operational risk management (ORM) framework is built.

A neat way to see this is the framework that ING drew in 2000. It placed four activities in concentric rings. The loss database sat in the middle. Around it came risk and control self-assessments (RCSAs), which draw on the loss history to judge risks and controls. The third ring held key risk indicators (KRIs), which monitor the risks that the RCSAs identify. The outer ring was about lessons learned, triggered whenever a large incident occurs or a high exposure is found. The first half of this lesson follows the same path, moving outward ring by ring.

Uses of loss data beyond capital

Supervisors have looked hard at loss data quality since the Basel Committee published its standardized approach (SA) reform in December 2017, because under that approach a bank’s own loss history feeds its capital charge. Yet capital is only one use. Loss records show where failures cluster and which controls broke, so management can fix weaknesses before they repeat and can improve performance and profitability. Internal incidents also help analysts identify and size scenarios and calibrate capital models. External loss data, meaning incidents at peer firms, show how exposures have played out elsewhere.

Loss data also matters under Pillar 2. A supervisor that sees a firm collecting, understanding and analysing its incidents well has evidence of sound governance, and good data collection tends to reduce both how often Pillar 2 capital add-ons are imposed and how large they are.

Under the standardized approach the Basel Committee on Banking Supervision (BCBS) sets eight criteria for the quality of loss data and the way it is gathered, set out in its 2017 criteria on how loss data is identified, collected and treated. Four of them carry most of the weight:

• a history of 10 years of loss data;
• a €20,000 minimum threshold for collecting losses;
• mapping of internal losses to the Basel event-type categories when they are reported;
• reporting of the date of occurrence and of any recovery, together with a process for independent review of whether the loss data is complete and accurate.

The remaining criteria deal with the governance of loss collection and with keeping credit risk and market risk events out of the dataset, among other points.

Documented procedures

The Basel text also expects a bank to have written procedures for identifying, collecting and treating internal losses (point b). The guidance should say what must be reported (dates, thresholds, event types, loss amounts, impact types and so on), because a loose definition produces inconsistent data. This is harder for operational risk than for its neighbours. A market loss on a derivatives book or a default on a corporate loan is triggered by a clear external event, and even though credit default can be defined in more than one way, each definition is legally precise. An operational event is often fuzzier.

Picture a software fault in a mobile banking app that holds up customer payments. Customers complain and ask for compensation, posts spread on social media, review sites repeat the criticism, senior managers spend days on the problem, and the IT team has to fix and upgrade the app. Deciding which of those effects to record, and putting a number on each, involves far more judgment than booking a trading loss. Not every incident sprawls like this, but the example shows why the definitions and the reporting process must be clear and followed carefully.

Comprehensive coverage

Basel also requires that internal loss data be comprehensive, capturing every material activity and exposure across all relevant subsystems and geographic locations (point d). The Committee does not define what counts as material, which leaves each firm to justify its own reporting threshold, as the next sections explain.

Most firms record operational incidents with much the same core set of fields. The sensible practice is to keep that set as short as possible: every extra field adds reporting effort for the business and rarely adds matching insight. The table below lists a typical set of fields and adds, for each one, the purpose it serves when the data is analysed later.

Each record should follow the firm’s taxonomy of risks, causes, impacts and controls. Drop-down lists and preset labels for risk type, cause and event type keep entries uniform, which is what makes the database easy to filter and aggregate. Free text has a place, since a narrative helps a reader understand what happened, but it should stay out of the key fields used for analysis. When two people describe the same kind of cause in slightly different words, automated grouping and aggregation become impossible. Some large banks now run natural language processing (NLP) and other machine-learning tools over years of accumulated free text to pull structured information out of it.

Basel fixes the regulatory floor for loss collection at €20,000, roughly $22,000, and firms are free to collect below it. Actual practice spreads across a wide range. Some firms set no threshold at all, either to capture every operational loss or simply because staff in the business units then do not have to estimate a loss before deciding whether to report. That approach is fading, particularly at large institutions, because the volume of tiny incidents creates a heavy workload for very little information. At the other end, some banks and insurers report only from 10,000 in dollars, euros or pounds, and a few go as high as the regulatory 20,000. The most common choices sit at 1,000 or 5,000 in the same currencies.

From a supervisor’s point of view any level can be acceptable as long as it does not weaken the credibility of the process or starve management of information. The choice must be justified, and it must not look like an attempt to cherry-pick events or engineer a lower capital requirement. In practice, the definition of what counts as a loss shapes the size and materiality of the dataset even more than the threshold itself does.

Regulation generally asks only for losses, meaning the negative financial effects that can be traced directly to an operational incident. Many firms go further and record incidents or risk events of every kind: accidental gains, events with no direct financial effect, and near misses where something failed but no loss followed. Only financial losses must be reported to the supervisor, but for management purposes it is good practice to log the so-called non-financial impacts as well, such as reputational harm, customer detriment, service disruption and the time senior people spend on the problem.

The label non-financial is misleading. Extra supervisory attention, unhappy customers, remediation programmes and diverted management effort usually turn into real and sometimes large costs. If those effects are never measured, the true cost of operational risk, and of weak operations in general, can be badly understated.

Direct and indirect losses

Direct losses arise from the event itself: the cost of fixing it, staff time lost, compensation to clients and other parties, regulatory fines and money lost on wrong transactions. Firms usually capture these well. Indirect losses come from what happens next, for example customers leaving after poor service or bad word of mouth, weaker staff morale and productivity, higher compliance costs under closer supervision, or dearer insurance after a claim. Indirect losses are often scored on a 1 to 4 impact rating that mirrors the firm’s impact assessment matrix. Putting a money value on them is something only firms with an advanced ORM framework tend to do, and doing so is important for accurate management information and sound decisions.

Grouped losses

Grouped losses are separate events that share one underlying cause. The same wrong advice given to many clients produces a stream of compensation claims; a single IT failure hits several departments in different ways; one misleading product sheet affects many sales. All of these must be recorded as one loss so that the data reflects the single failure behind them. Grouping is a regulatory requirement, it is sound practice for modelling, and it keeps management information accurate.

Basel asks banks to report the gross loss and the reference dates of every operational loss event, along with any recoveries and information on the drivers and causes of the event. Each incident carries four dates:

• Date of occurrence, the moment the event actually starts;
• Date of discovery, the point at which someone first notices it;
• Date of reporting, once it is logged in the incident database;
• Date of accounting, when its financial effect is booked in the general ledger.

Occurrence and discovery can be years apart. Small internal frauds that skim tiny amounts, sometimes called salami frauds, can run unnoticed for years. Data leaks by internal or external hackers, rogue trading and manipulation of suspense accounts (temporary accounts used in payments and settlement, and a favourite place to hide fraud) usually take months or years to surface. Equifax needed three months to identify its data leak, and Jerome Kerviel had traded without authorisation for 18 months before he was confronted. Slow discovery is usually the main reason for the long gap between occurrence and final payment, and it makes the damage much worse.

Supervisors do not insist on any one date for internal reporting, but each gap is informative. The time from occurrence to discovery shows how visible problems are inside the firm. The time from discovery to reporting shows how promptly staff escalate incidents to the risk function. Incident policies usually set a maximum reporting delay on a risk-based footing: material incidents within a few working days, minor ones in a periodic summary.

Settlement lags

The settlement date is reached when every effect of an incident has been booked in the general ledger, which for legal and regulatory matters can take years. An analysis of losses above $100 million at 31 banks that are, or were, global systemically important banks (G-SIBs) before the global financial crisis, covering 1996 to 2006, found that the most common duration between reporting and settlement was three to five years. So, part of what hits the accounts today happened years ago, under different managers and in a different business setting, and analysts should ask how relevant such old losses are for modelling future events and drawing management lessons.

A boundary event is a loss that shows up in one risk class even though its cause belongs to another. A credit loss made worse because collateral was recorded wrongly is one example; a market loss caused by booking a position by mistake is another. Basel gives two rules for sorting them.

RWAs are risk-weighted assets. The pragmatic stance taken by regulators is to leave credit events where they show up instead of trying to untangle the cause. That works for credit because regulatory credit models are calibrated on historical losses: even if part of a past credit loss was really operational, it is already covered by capital in the credit class, which is enough from a prudential point of view. Market risk models do not draw on these events in the same way, so market boundary losses must be recorded as operational losses.

Basel requires processes that independently review whether loss data is complete and accurate (point h). Completeness is the harder question, because nobody can see the losses that were never reported. The main answer is reconciliation: compare the operational loss database with other records that should reflect the same events.

The general ledger (GL), which captures every cash inflow and outflow including accidental losses and gains, is the best benchmark, provided its entries are labelled so that operational items can be picked out. Even so, the GL cannot replace a dedicated incident database. It records only direct financial effects, it does not tie indirect effects to the event, and it misses failures that show up as lower revenue, such as forgetting to charge a client a fee.

Several other sources serve either as proxies for operational incidents or as checks on completeness:

• IT logs, where P1 and P2 issues (the two highest priority ratings) usually count as operational incidents;
• provisions for pending lawsuits;
• customer complaints and compensation payments;
• negative press and media reports;
• issue logs kept by back-office, operations and administrative teams.

Why incidents go unreported

Underreporting remains a problem despite detailed rules, especially in smaller firms. Staff may lack clear guidance on what and how to report, may resist escalating their own errors, or may simply avoid a reporting process they find cumbersome. Firms respond with incentives that range from gentle encouragement to strict oversight. Self-reporting is usually mandatory, with disciplinary action if a hidden event is later found, and many firms include risk metrics in managers’ balanced scorecards, a practice regulators encourage. Internal audit also reviews the collection process regularly and recommends fixes where it is weak.

What makes operational loss data unusual

Compared with market and credit data, operational loss data is specific to each firm and largely unrelated to market movements; both the count and the size of its events can have extremely wide tails; it is scarce; it can stay hidden for a long time; and it can be hard to interpret once causes and knock-on effects are traced. These features shape everything about how operational risk is measured and modelled.

An RCSA asks a business, or part of one, to judge the likelihood and impact of the operational risks it faces at two levels. Inherent risk is the exposure before existing controls are taken into account. Residual risk is what remains once the controls in place have been evaluated. Most firms run RCSAs once a year, and some have moved to quarterly cycles because certain operational risks now change faster.

Some firms appoint a dedicated risk assessment unit (RAU) to run the exercise, which may sit within a business line, a division or a process. The classic format is a workshop in which the unit discusses its main inherent risks, the key controls that mitigate them, and how effective those controls are. Large organizations often collect the same judgments through questionnaires held in specialised risk software, where each business rates its own risks and controls. Preventive controls lower the likelihood of a risk, while corrective controls lower its impact. By the end, the firm should understand its inherent risks, how well its controls work, and the residual risks left over.

Strengths and weaknesses

RCSAs are qualitative and rest heavily on judgment, although mature firms ask for evidence of control testing before a control can be rated effective and before an inherent risk can be treated as reduced to a level within risk appetite. The same firms back-test assessments against their incident history, and such checks commonly find that participants underestimate likelihood and overestimate impact. Practitioners regularly question how reliable the exercise is and whether it is worth its cost. Even so, RCSAs are very widely used, and in Europe in particular they are a central assessment method, often because supervisors expect them.

Two variations

The point of a risk assessment is to decide which risks deserve attention first, and that only works if a “high” rating in one department means roughly the same as a “high” rating in another. Subjective judgment, behavioural bias and thin data all erode that consistency, and different units are often rated by different people. The remedy is a common risk description and precise definitions for every likelihood and impact rating. Without comparable ratings there is no ranking of risks and so no sound order for mitigation. RCSAs do not produce a precise measure of exposure, which a rough qualitative scale cannot deliver. They raise awareness of risks and controls, open a discussion of residual exposure under the current controls, and show where more mitigation is needed and in what order.

Impact scales

Impact scales usually cover four types of impact: financial, regulatory, customer and reputation. With the growing regulatory focus on resilience, many firms also rate the effect on continuity of service. Ratings normally run on four or five points, from “low” or “insignificant” up to “extreme” or “catastrophic”. The example below defines financial impact as a share of operating income and customer impact partly as a share of the client base.

RTO is the recovery time objective. Relative definitions make one scale usable across units of very different sizes, which matters for groups operating in several countries and divisions: a $100,000 loss can be serious for a medium-sized department yet minor for a large business unit. Regulatory, compliance and reputational impacts are usually defined in words, sometimes supported by metrics such as the share of customers affected or the number of negative articles.

Likelihood scales

Likelihood is expressed either as a probability or as a frequency (“once in x years”). Risk managers who speak of a 1-in-10-year event mean a 10% chance of the event happening in the coming year, not one occurrence somewhere over a ten-year horizon. The RCSA horizon is one year at most, and shorter for firms that reassess almost continuously, which matters for fast-moving risks such as cyberattacks, technology change and regulatory sanctions. History naturally informs likelihood, especially in stable environments, but a risk that has never materialized can still be quite likely, and the past is a weaker guide when conditions are unstable.

A heatmap, also called the RCSA matrix, crosses the likelihood scale with the impact scale and colours each cell by intensity, usually red, amber and green or red, amber, yellow and green. In many firms the colours encode risk appetite:

• Green: exposure within appetite; routine monitoring that activities and controls run as intended is enough.
• Yellow: still within appetite but nearing the limit; active monitoring and possibly more mitigation.
• Orange: above acceptable levels; an action plan must cut likelihood or impact, or management must escalate and formally accept the risk.
• Red: far beyond appetite; an urgent mitigation plan is required.

The colouring depends entirely on how the scales on each axis are defined. The ratings are non-linear and non-continuous, so they have indicative value when built with care but must be handled cautiously. A common mistake is to multiply likelihood by impact to get a single score. A frequent, low-impact risk scored 1×4 and a remote risk with extreme impact scored 4×1 both give 4, yet they are entirely different risks and call for different responses. On the matrix above, the first sits in yellow and the second in orange.

Outcome of an RCSA

Results are documented and signed off by business line managers, who own the output and any action plans. They obtain a ranking of their residual exposures after controls, and through inherent risk a view of what could happen if key controls failed. A risk above appetite first triggers a check of the assessment itself: is the mitigation really missing, and do others agree? If the excess is confirmed, further mitigation follows. Where the control environment is within appetite, the response is to keep the controls and continue monitoring.

Mitigation need not mean more controls. A firm can reduce transaction volumes, narrow access rights, redesign or improve existing controls, or buy insurance. Action plans should be owned by the first line, the business managers, rather than the second line risk function, because business ownership creates the right incentives to deliver. The second or third line then tracks progress much as it tracks audit recommendations.

A key risk indicator (KRI) is a metric that tells management whether the level of an operational risk is rising or falling at a given moment, either in likelihood or in potential impact. Decision-makers value KRIs because they promise early warning before a risk materializes. A preventive KRI tracks the strength or presence of a cause. A KRI of likelihood signals that the event is becoming more probable; a KRI of impact signals that the damage would be greater if the event occurred.

KPIs and KCIs

Key performance indicators (KPIs) measure performance against targets, that is, how well a unit runs. Examples include the downtime or uptime of IT systems, the rate of errors in retail transactions, client satisfaction or net promoter scores and complaint levels, and the share of services delivered within service level agreements (SLAs). Key control indicators (KCIs) measure how well controls are designed or performed, for instance due diligence items that were missed, duties that are not segregated, errors that slipped through four-eye checks (where two competent people review the same work), and identification files left incomplete.

The three families overlap heavily, especially when a metric shows that a minimum standard has been breached. Weak performance tends to create risk, and a failed key control is always a source of risk, so control failures are natural preventive KRIs. Overdue confirmations of financial transactions illustrate the point: they reveal poor back-office performance (KPI), make legal disputes, processing mistakes or rogue trading more likely (KRI), and show a control failing in transaction processing (KCI). Put simply, a KPI that turns amber becomes a KRI.

Thresholds, reporting and analytics

KRI thresholds express risk appetite and tolerance. Which metrics a firm watches shows what it cares about, and where it sets intervention points shows how strict it is; limits on key-person dependence, long vacancies, activity spikes, spare system capacity or untested back-ups state openly how much exposure the firm will accept. A typical report lists, for each process, the KRI, the related risk, lower and upper tolerance levels, the current score, a colour-coded risk level and comments or actions. A strong, preventive and reliable KRI programme is itself evidence of a mature risk function.

Data analytics and machine learning are improving KRI design by separating true risk signals from normal business noise through trends, patterns and outliers. This works best where data is plentiful and captured automatically, such as card fraud, systems operations, payments and trading. Text analytics such as NLP can pull keywords from incident descriptions to classify and group events. These applications are still young in operational risk but developing quickly.

Factor models assess operational risk by breaking a possible incident into the drivers of its likelihood and of its impact, and then estimating the range of outcomes. The central idea is to examine the layers of control that stand between a hazard and a loss, and how often each layer fails. Fault tree analysis (FTA), factor analysis of information risk (FAIR), the Swiss cheese model and the bowtie all belong to this family. They suit operational risk well because they focus discussion on controls and their effectiveness. Like any model, their output depends on the factors chosen and the probabilities attached to each, but splitting a scenario into explicit parts makes the estimate sturdier and the reasoning behind it easier to see.

These methods are a form of causal analysis, whose role in measuring operational risk and its capital has grown markedly since the 2010s. Causal analysis is used mainly for scenarios, meaning risks with tail potential that could hurt the firm badly. Such events are rare when mitigation works as intended, but real loss histories show that it does not always work. Instead of relying on the past distribution of losses, causal analysis looks forward at what determines likelihood and impact.

Supervisors know that sizing rare events is difficult, and because scenarios play a large part in operational risk capital, they want the process to be as reliable and defensible as possible. BCBS asks for a highly repeatable scenario process with consistent preparation and consistent use of qualitative and quantitative results. Firms are expected to keep subjectivity and bias to a minimum by grounding assumptions in empirical evidence, explaining the reasoning behind each parameter, and documenting assumptions and process properly. Assessing severity well means drawing on a large body of business data so that the evaluation is not distorted by exaggeration.

Fault tree analysis is a deductive method for analysing failures, developed in the 1960s and used mostly in safety engineering for high-risk industries such as nuclear power, pharmaceuticals and aviation. A fault tree lays out the external conditions and internal failures, such as control breakdowns, that must come together for a disaster to occur. The chain of failures behind a gas explosion or a train crash has clear parallels with those behind a successful cyberattack, a long systems outage or a compliance breach that costs a bank its licence, which is why the method is spreading into finance.

The tree combines conditions in two ways. An AND condition requires several conditions to occur together; an OR condition is met if any one of them occurs. When conditions are fully independent, the probability that all of them occur is the product of their individual probabilities.

Three independent key controls that each fail with 10% likelihood will all fail together with probability 10%^3 = 1/1000. Layering independent controls in this way is how high-reliability organizations such as power plants, airlines and emergency services keep disasters rare.

A fault tree for an insider data leak

Consider an accidental leak of confidential data by an employee, which needs five conditions to occur: a targeted phishing attack on the firm (P1); the firewalls fail (P2); an employee opens the phishing e-mail (P3); the controls that should detect suspicious network activity fail within a unit of time (P4); and the information leaves the firm, per unit of time (P5). P1 is close to 100% for an attractive target, since banks face many attacks every day. P3 depends on staff awareness; phishing tests in finance find that between 5% and 50% of employees click malicious links. P2, P4 and P5 depend on how sophisticated the firm’s cyber defence is.

Under independence the scenario probability is P1 × P2 × P3 × P4 × P5. That is the theoretical minimum, because real failures are rarely independent: the same team may design several security measures, or one unit may run them all. Conditional (Bayesian) probabilities give a much better sense of the true likelihood.

Exposure matters as much as probability

The tree gives the probability per employee. Scenario assessment multiplies that probability by the number of exposure units, not by the impact, which is a different quantity. Ignoring exposure can lead to severe underestimation.

In causal analysis the probability of each factor should come from external statistics and quantitative data wherever possible. Where the trigger is external, for example a natural disaster, earthquake, pandemic or terrorist attack, insurance and reinsurance tables supply the probabilities. For scenarios modelled on large losses in external databases, frequencies come from those databases and are then adjusted by the similarity judgment method. If a scenario struck one bank out of 100 during the last five years, a starting probability for one bank over a one-year horizon is 1/(100 × 5) = 1/500. Another useful source of scenarios is the most pessimistic version of risks discussed in the RCSA, with impacts pushed to the worst case and likelihoods informed by the control assessment.

When failures are not independent

Realistic fault trees allow control failures and impact factors to depend partly on one another. In the data-leak tree, the relevant quantity becomes the probability that the detection mechanism fails given that the firewalls have already failed. Models built on such conditional probabilities are called Bayesian, after the Reverend Thomas Bayes (1702 to 1761); in operational risk the term covers methods that update likelihoods as new information or events arrive. The probability of event A given that B has happened is written P(A|B).

Conditional reasoning is everywhere in operational risk. An employee under pressure to hit steep sales targets is more likely to commit internal fraud; a processing error is more likely once an IT breakdown has occurred. Conditional probabilities are used to update expert opinion with past data in risk and scenario assessment, and to evaluate dependent steps in fault trees.

The FAIR model is a factor model that many financial firms use to quantify operational risk, especially information risk. Like every factor model it works in three steps: identify the risk factors and how they relate, measure each factor, and combine the factors computationally. The result is a distribution of losses for a given scenario. A FAIR scenario must specify an asset at risk, a threat community (who or what poses the threat), a threat type (its nature) and an effect (the losses if it materializes). Business experts then estimate the loss event frequency and the probable loss magnitude, expressed as distributions rather than single values. Those distributions are the inputs to a Monte Carlo simulation whose output is the distribution of scenario losses.

Swiss cheese: why control independence matters

The Swiss cheese model, formally the cumulative act model, was popularised by the psychologist James Reason, whose work on human error is used in health care and other high-risk fields. In his picture (Reason, 2000), each line of defence is a slice of cheese. Ideally the slices would be solid, but in reality they have holes that keep opening, closing and moving. A hole in one slice rarely causes harm on its own. Damage happens when holes in several slices line up for a moment and let a hazard pass straight through to its victims.

The lesson for risk managers is to build control layers whose holes do not line up, meaning controls that fail independently and cover one another’s weaknesses, so the system acts as a tight safety net and not as a row of dominoes that all fall once the first one goes. Judging whether controls are independent of each other is at least as important as judging how reliable each control is on its own.

Supervisors expect firms to analyse the root causes of significant incidents and near misses. The first line should carry out the analysis consistently whenever an event caused, or could have caused, impacts above the materiality threshold, with the second line supporting or challenging it. Comparing the findings of many investigations adds even more value, because recurring causes reveal common weaknesses that one action plan can fix across the firm.

The bowtie, long used in industries such as oil and gas, is spreading in finance. It applies 5-why analysis, which means repeating the question “why?” at least five times to dig past the obvious cause. The event sits at the centre. Causes fan out to the left, from direct causes to indirect ones, until every root cause within the firm’s sphere of influence has been found. Impacts fan out to the right, from direct to indirect, giving the shape of a bowtie. Preventive controls are placed in front of each cause on the left, and detective and corrective controls are placed on the impact side on the right.

A 5-why chain

A bank loses $10 million when customer information is stolen. Why? A cyberattack exploited a weakness in its security. Why? A critical patch for that weakness existed but had not been installed on most computers with critical network access. Why? The administrator responsible for updates was on holiday and no one else with the right skills could do them. Why? The IT department had never set up dual responsibility for system upgrades. The root cause is a missing backup arrangement, not the attacker.

The bowtie is useful before events too. Firms use it to estimate the plausible range of impacts and the expected frequency of a risk given current mitigation, much as an FTA does. Because it maps detective controls, incident handling and both direct and indirect effects, it gives a structured view of the range of impacts and their drivers. Run in reverse as “5-then-what”, it explores consequences instead of causes. It also helps identify KRIs and reveals the speed of detection and the quality of the response in containing direct and indirect damage, which matters more and more in a world of cyberattacks, pandemics, armed conflict and extreme weather.

At the quantitative end of the spectrum sit stochastic models, used mainly by Tier-1 banks and insurers to compute economic capital. They fit statistical distributions to the frequency and severity of past internal and external losses, simulate the firm’s loss distribution, and set capital at the 99.9% percentile of that distribution. Of these, the loss distribution approach (LDA) dominates, with extreme value theory or scenarios added to model the tail.

Borrowed from actuarial science, the LDA separates losses into their count per period (frequency) and their size (severity). Treating each loss as one frequency observation and one severity observation doubles the data points available, which mattered greatly when operational loss data was scarce. Frachot, Georges and Roncalli, modellers at Credit Lyonnais, set out its use for operational risk in a 2001 paper, and it soon became standard. Statistical modelling took hold mostly in continental Europe, whereas US firms leaned more on scenarios and stress testing. The two distributions are estimated separately and then convoluted into an aggregate loss distribution, most often by Monte Carlo with a million or more random draws. Fast Fourier transform and Panjer recursion are equation-based alternatives that need more coding and mathematics but less computing time.

Frequency: the Poisson distribution

Frequency is discrete: the number of loss events per period, usually a year. The Poisson distribution is the usual choice and the simplest, with a single parameter λ, also known as the intensity or hazard rate, that equals both the mean and the variance. A 2009 BIS study of the 2008 loss data collection exercise found that nine out of ten AMA firms modelled frequency with a Poisson distribution, and the other 10% used a negative binomial.

Observations are assumed i.i.d. (independent and identically distributed), and the additivity property means two independent Poisson counts sum to a Poisson count whose intensity is the sum of the two.

Severity: the lognormal distribution

Severity is continuous, skewed and heavy-tailed, reflecting many small losses and a few very large ones. The lognormal is the most common choice, and heavier-tailed Weibull and generalized Pareto distributions (GPD) have gained ground. X is lognormal when Y = ln(X) is normal with parameters μ and σ; equivalently, if Y is normal then X = e^Y is lognormal, so X takes only positive values. μ is the location parameter and σ the scale parameter.

The LDA rests on strong assumptions. Frequency and severity are usually assumed independent, which is hard to square with the sharp contrast between how often small and large events occur. Losses are also grouped into units of measure (UoMs), or risk classes, within which they must be i.i.d.: no hidden correlation inside the class, and one data-generating mechanism for every loss. Given how varied operational events are, that is a demanding condition. It can be tested by forming clusters that are as homogeneous as possible and checking for the absence of autocorrelation.

Choosing and aggregating units of measure

Each UoM gets its own LDA, and the results are aggregated into a total loss distribution whose 99.9th percentile is the stand-alone operational risk capital when only the LDA is used. Possible UoMs combine event types and business lines, for example external fraud split by business line, damage to physical assets grouped across all lines, internal fraud per business entity under common senior management, and processing errors split by business line. A good UoM gathers events driven by the same factors, with similar dynamics and distributions. Finer segmentation yields more homogeneous data and more granular models, but also less data per model, more uncertain results and harder aggregation. The dependency structure has a large effect on capital, so validators and supervisors scrutinise it, and modellers should show that UoMs follow how the business is organised and how the model will be used. UoMs are combined with copulas, which generalise correlation and can capture tail dependence between extreme values.

Extreme value theory

LDA alone cannot capture the extreme skew of operational losses, so the tail is completed with extreme value theory (EVT), scenario data or both. EVT has two approaches. Block maxima (Fisher-Tippett) studies the largest loss in each equally spaced period and UoM. Peak-over-threshold (POT) studies only the observations above a high threshold u; a theorem (Balkema and de Haan, 1974; Pickands, 1975) shows that as u rises, the distribution of excesses, F_u(y), converges to the GPD.

A caution applies. EVT gives asymptotic results that hold only under regularity conditions. With losses drawn from many institutions, business lines and risk types, it is valid only if one mechanism generates every observed loss and will also generate future losses beyond anything yet seen, which would justify fitting one parametric model and extrapolating. That condition is unlikely to hold, so firms also turn to scenarios, causal models, fixed multipliers and macro drivers of operational risk.

Internal loss data is the backbone of the LDA. It supplies many data points for the body of the distribution, where events are frequent and moderate, and it is more relevant to the firm than outside data, with more detail and context to support cleaning, audit and model review. For causal models, a full internal database also shows exposures and causes of failure, which feed scenario structures and parameters.

The incident database kept by the risk function usually differs from the calculation dataset used by modellers, and each firm needs rules for what moves from one to the other, especially near misses, accidental gains and quickly recovered losses. The most conservative firms count them in both frequency and severity, using the absolute value or potential loss as severity; others count them in frequency only; others leave them out. Correct event classification, risk categories, business lines, dates and amounts are just as critical for reliable results.

External data

One firm’s history cannot show the full range of what might happen, so it is supplemented with external data. Public databases collect and classify events reported in the press and name the institutions involved, so a modeller can judge how relevant each event is, but public data is scarcer, which can make models less stable. Consortium databases pool members’ internal losses anonymously. ORX, with 103 international banks and insurers as members in 2022, leads the market with 800,000 loss events; ORIC International comes next, and its members are mainly insurers and asset managers. These databases are large but carry little detail per event.

Firms check that external losses come from comparable peers, looking at the geographic spread of activities, concentration by sector or business line, the length of data series by event type, classification and quality certification rules, and the membership’s number, type and size, as well as the database’s reporting threshold.

Mixing internal and external data

Each choice can move the final capital number substantially, so operational risk managers should understand them well enough to discuss them with modellers on an informed footing.

Since the new standardized approach took effect on January 1, 2023, banks no longer have to model operational risk capital. The largest institutions, however, hold more than 20 years of modelling experience and are moving those skills to Pillar 2 capital under the internal capital adequacy assessment process (ICAAP), to stress testing and to operational resilience.

Under the Advanced Measurement Approaches (AMA) used by banks, and their insurance counterpart the Internal Modeling Approaches (IMA), both now being phased out, firms set their own capital once each national supervisor had validated the model, covering operational losses up to a 99.9% confidence level over one year. Most adoptions came early and from Europe. Approval took on average three years of preparation, internal and external validation, and compliance with around 30 qualitative and quantitative criteria on day-to-day risk management. A qualifying model needed four inputs: internal loss data (ILD) and external loss data (ELD), which describe the past, plus scenario data (SD) and the business environment and internal control factors (BEICF), which add forward-looking information about exposures and controls.

One Tier-1 European bank’s model used all four inputs, fitting one distribution to the body of losses and a different one to the tail, split at a cut-off set by a frequency threshold, with regulatory capital read as the 99.9% VaR. RCSA results on control failure frequencies entered as BEICF. Its modellers chose simplicity and transparency over complexity, which made the model more stable, and they traced how each loss and scenario moved the final number, which made the model acceptable to the business. Such hybrid models, combining LDA on incident data with scenarios and adjustments for risk management quality, now support the bank’s Pillar 2 assessment.

ICAAP and scenario aggregation

ICAAP is the Pillar 2 requirement for a regulated firm to judge its own capital needs given its risk profile, business model, medium-term plan and controls. It covers all financial and non-financial risks and applies to firms of every size in the UK, continental Europe, Canada and Asia. The US relies instead on CCAR (Comprehensive Capital Analysis and Review), an annual firm-wide stress test for large banks that is conceptually similar. For operational risk, ICAAP means selecting relevant scenarios and quantifying them from business exposures, internal and peer evidence, and the control environment. A UK regulator’s outline for small and medium firms runs: executive summary; business model, strategy and forecasts; ICAAP process and governance; risk appetite, tolerances and monitoring; scenarios and their stressed assessment (Pillar 2A); capital assessment, planning and stress testing (Pillar 2B); internal challenge and adoption; wind-down planning; conclusion and next steps.

Many smaller firms aggregate scenarios with a simple multi-criteria rule: capital must exceed the loss of the largest single scenario and also the combined loss of each cluster of two to four scenarios that could happen together, such as a pandemic, a key employee departure and a cyberattack; a cyberattack and loss of a data centre; or loss of key staff and internal fraud. Larger firms use variance-covariance matrices or copulas. The report to the supervisor must justify each scenario chosen and each one discarded. Economic capital, the capital a firm itself judges necessary to survive or, in practice, to keep its credit rating, serves largely the same purpose as ICAAP.

A UK FinTech bank quantified a product launch failure for its ICAAP with a factor model (its figures were altered for confidentiality). Each factor takes one of four states, and the likelihood row gives the probability of each state.

The simulated loss distribution is right-skewed, with the following quantiles; the last column is derived.

Severe operational disasters, from systems failures and cyberattacks to physical damage and compliance breaches, can wipe out much of a firm’s revenue, damage its long-term earnings and reputation, and harm clients. Regulation asks firms to consider plausible disasters that may never have happened to them or their peers and to judge how resilient they would be. For modellers, such scenarios feed the tail and test whether capital is sufficient; for management, they reveal large exposures and needed controls. The BIS (2021) treats resilience as an outcome of sound operational risk management, since identification, assessment, mitigation and monitoring together reduce disruptions, so everyone in the ORM framework contributes to it.

Business continuity management (BCM) restores each process within a recovery time objective (RTO). Resilience instead protects important business services (IBS), such as services to customers, within impact tolerances, the equivalent of RTOs. BCBS (2021) describes resilience as keeping critical services running in all but the most extreme conditions, a phrase it does not define. The UK FCA treats a service as important if its disruption could cause intolerable harm to clients or put at risk the stability, soundness or resilience of UK finance or orderly markets, and defines impact tolerance as the maximum tolerable disruption, measured mainly in time, beyond which such harm would follow.

IBS depend on the firm: cash and payments for retail banks, order booking and trading for asset managers, short-term funding and FX execution for corporate banking. Not every service can be important. Mapping dependencies and critical paths is complex in large firms and needs ORM (usually coordinating), BCM, HR, third-party management, IT, operations and infrastructure. The key steps are: (1) identify IBS; (2) set impact tolerances; (3) map each IBS end-to-end with its resources; (4) design severe but plausible scenarios; (5) where tolerance is exceeded, learn lessons and act; (6) prepare internal and external communication plans; (7) produce an annual self-assessment signed off by the board. One large Canadian bank maps each stage of corporate lending (commitment, pre-funding, release, post-funding administration) to business and IT RTOs, systems and departments or third parties.

Business impact analysis and KRIs for resilience

A business impact analysis (BIA) assesses the operational and financial consequences of disrupting a process or service and the resources needed for recovery: lost or delayed sales and income, extra costs such as overtime or outsourcing, fines and contractual penalties, customer attrition and delayed business plans. It ranks restoration order, with the highest-impact processes restored first. Single points of failure (SPOF) are natural resilience KRIs: key staff without back-ups or documented processes (coders, pricing modellers, cybersecurity specialists, technical engineers), hard-to-replace suppliers (cloud, data, payment platforms, utilities) and critical systems nodes. SPOF should be removed through back-ups and redundancy, or else reflected in tolerances and stress tests.