CR 6 – Credit Scoring and Rating

Every credit risk model ultimately has to estimate three quantities: the probability of default (PD), the exposure at default (EAD) and the loss given default (LGD). Credit scoring and credit rating are the two families of tools that lenders use to deal with the first of these, the PD, and with the broader question of how creditworthy a corporate or consumer borrower really is. Both work from the information available about a borrower’s capacity to pay and willingness to pay a debt on the agreed terms, and both convert that information into a measure of risk. They are data-driven approaches, which sets them apart from purely judgmental credit decisions and from structural financial models of default, and they are by far the most widely used approach in practice.

The difference between the two lies mainly in the form of the output and the setting in which it is used. A credit scoring model produces a number, the credit score, for each borrower. The score is an indicator of creditworthiness and maps to a probability of default. A credit rating system produces a grade on an ordinal scale, such as AAA or BB, and every grade carries an empirically estimated PD that applies to all the borrowers placed in that grade. So, a score ranks borrowers on a continuous measure, while a rating sorts them into a limited set of risk buckets.

Where scores and ratings are used

Ratings are the familiar language of corporate loans, bond issues and sovereign borrowers, and many ratings are published for anyone to see. Scores, by contrast, are produced for the internal use of financial institutions and their corporate clients and are not made public. The production process also differs. A score comes out of an automated analytical model, whereas a rating can combine the output of such a model with the judgment of an experienced analyst, which makes the rating process more elaborate and more in depth.

Both tools reach well beyond the yes-or-no decision to approve a loan. They feed into loan pricing, into credit portfolio management (the measurement of risk exposure across an entire book of loans rather than one loan at a time), into monitoring of how a loan or a portfolio evolves over time, into internal and supervisory reporting, into setting the capital needed against expected and unexpected losses, and into customer relationship management. Four benefits follow from putting a formal scoring or rating system in place:

• the loan evaluation becomes less subjective, because the same evidence leads to the same assessment;
• credit risk can be managed analytically, including scenario analysis and stress testing;
• every customer is judged inside one common framework, which brings consistency and transparency;
• appraising a loan takes less time and costs less.

Internal models and external models

A bank can build its own scoring or rating model or buy the output of one from outside. Internal models are typical of banks and other credit institutions that hold enough historical data to develop the model and calibrate it to the profile of their own loan book. External models come from credit rating agencies, credit bureaus and consultancy firms, and their outputs are sold to banks, corporate clients and investors. Investors lean on external ratings when they assess bonds issued by companies and governments, and a company may use them to judge whether its own customers are good for the credit it extends.

The evidence on external ratings is mixed in an instructive way. As predictors of default they tend to be weaker than internal models or financial models, yet they carry good information about systematic risk, the risk that arises from shocks to the market as a whole. They are also useful as a benchmark for reviewing internal models. For that reason, an institution is usually better served by combining several scoring and rating systems, built in different contexts and from different inputs, than by relying on any single one.

The Basel Committee on Banking Supervision recognises two rating philosophies, and the difference between them is the time horizon over which a borrower is judged. A through-the-cycle assessment looks at the borrower over a long horizon that spans at least one full business cycle. It deliberately ignores temporary swings in conditions, so the rating changes rarely and is revised only when something more durable has changed. This long-term view is the one the major credit rating agencies say they follow.

A point-in-time assessment asks a narrower question: how likely is this borrower to default given its current and near-term condition? The horizon is predefined and in most cases is no longer than one year. Because the rating responds to the borrower’s present circumstances, it moves more often and is updated promptly whenever new information about the borrower arrives. Banks and other credit institutions typically build their internal systems on this basis, and point-in-time ratings are the natural tool for short-term loans and transactions.

Which philosophy predicts better

Research on the two approaches has produced mixed results on predictive power and accuracy, and neither has been shown to dominate. So, the right choice depends on what the analysis is for. A lender pricing a one-year working capital facility wants to know the risk over that year, whereas an investor holding a ten-year bond cares more about the borrower’s resilience through good years and bad.

There is also a modelling asymmetry. Default is easy to define in a point-in-time setting, for example as default within one year, so the target that the model must predict is clear. Extending the same model to a through-the-cycle setting is harder, because the time until default then has to be defined explicitly and modelled analytically rather than fixed at a single horizon.

The distinction returns in the criticisms of the rating agencies, whose supposedly through-the-cycle ratings have in practice moved with the cycle.

An issuer credit rating expresses a view, looking ahead, on how creditworthy the obligor is as a whole. It can be a long-term or a short-term rating depending on the horizon being examined, and it comes in several forms, such as a corporate credit rating, a sovereign credit rating or a counterparty credit rating. The rating attaches to the entity, so it describes the borrower’s general capacity to meet its financial commitments.

Issuers seek a rating for many reasons, but the commonest is access to capital markets and other sources of funding. A company placed at the top of the rating scale has shown that it repays its debts and carries little default risk. That reputation is an asset: it lowers the interest rates the company pays, makes external financing easier to obtain, and improves its standing with stakeholders such as suppliers and investors, who may offer better terms because they trust the firm to pay.

An issue-specific rating, in contrast, is an opinion on the borrower’s creditworthiness with respect to one particular obligation, such as a single loan or bond. It takes account of the terms of that obligation: its maturity, the collateral pledged, the purpose of the borrowing (for a mortgage, the value of the house) and any other feature that shapes the risk of that specific debt. Two loans to the same company can carry different issue-specific ratings when one is secured and the other is not, even though the company has only one issuer rating.

A standard application score is static: it is calculated once, when a new customer applies, from the data on the application form. Behavioral scoring takes the opposite approach. It is applied in consumer lending to customers the lender already has, whose conduct on their accounts can be observed, and it measures their risk from that observed behaviour. Inputs include account balances, missed payments and changes in employment status, among all the other information that becomes available as the relationship runs.

Behavioral scores are short-term assessments and are refreshed frequently, usually every month. So, instead of a single snapshot taken at the start of a loan, the lender has a running estimate of default risk that tracks how the customer pays and spends. Lenders put these scores to several uses beyond risk measurement: setting and adjusting credit limits, managing the collection of overdue debts, and marketing, for instance deciding which customers should be offered a new credit or financing product.

Profit scoring

Credit scores and behavioral scores both answer a risk question. Profit scoring asks a commercial one: will extending credit to this customer be profitable? It widens the traditional default model to take in marketing, pricing and operational decisions alongside the usual default prediction. A customer with a modest default risk who generates little revenue may be less attractive than a slightly riskier customer who uses many of the lender’s products.

There are two main types. Account-level profit scoring estimates the profit from one specific account, so it looks at a single financial product, ignoring any links with the other products the same customer holds. Customer-level profit scoring brings all of a customer’s accounts together, which gives a fuller picture of the customer’s value to the lender and helps the lender decide which new products to promote to that customer.

Peer-to-peer (P2P) lending, also called social lending, is one of the financing channels that financial technology (FinTech) has opened to companies and individuals. An online platform matches borrowers directly with lenders, with no bank standing in between, which makes credit easier to obtain than through a traditional bank. The first platforms appeared in the mid-2000s, with Zopa in the UK in 2005 and Prosper in the USA in 2006, and the market has grown fast since then. Data from the UK P2P Finance Association put the UK market at £2.16 billion at the end of 2014; by the first quarter of 2017 it had reached almost £8.5 billion, and business lending accounted for more than £5.1 billion of that total. Reports for the USA put the US P2P market above $68 billion.

Why P2P platforms need adapted credit models

Credit risk modelling for these platforms has become a research field of its own for two reasons. First, the P2P market is still largely unregulated and is not supervised in the way that banks are, so the discipline that regulation imposes on bank credit models is absent. Second, default risk on P2P platforms is generally higher than in traditional lending, and P2P loans usually come without collateral or guarantees, so a lender who misjudges a borrower has nothing to fall back on.

Scoring and rating models are therefore valuable on these platforms, but they must be adjusted to the special features of P2P lending and serve investors, lenders and borrowers alike. Two issues stand out. Platforms gather large volumes of information about applicants from online sources, so the models must be able to handle big data. And because lenders on a platform are often individuals with no access to the borrower, the platform must make the credit assessment transparent to every party involved.

Credit scoring and rating models in banks are not built in a vacuum. Local and international supervisors, central banks among them, set requirements for how such models are developed and used, and the Basel Committee rules lay down five basic specifications that any rating system has to meet.

1. Meaningful differentiation of risk

The system has to be designed to tell risk levels apart, not to squeeze regulatory capital as low as possible. Each grade must be clearly defined so that it represents a distinct level of credit risk, and borrowers inside the same grade should still be treated differently when the characteristics of their transactions differ. Exposures also need to be spread sensibly across the grades, so that no single grade holds an excessive concentration of the portfolio.

2. Continuous evaluation of borrowers

Every borrower and every loan in the portfolio should be re-rated at least once a year, using whatever new information has emerged about the borrower’s situation and progress.

3. Oversight of how the system operates

The system must be monitored continuously to confirm that it is working correctly. It needs sufficient and efficient controls, such as stress tests, and a feedback mechanism between the parties responsible for it, so that its integrity is maintained.

4. Correct selection of risk assessment attributes

Developers have to be able to show that the risk factors and attributes in the model really do allow the borrower’s creditworthiness to be analysed. The analysis should look ahead, judging how the borrower will perform in future on the basis of what is known today about the borrower and its external environment.

5. Collecting a substantial database

The data used to build and evaluate the system must be representative of reality. They should include historical data on the characteristics of borrowers, their past credit scores, ratings and PD estimates, credit migration data and payment histories.

These requirements shape the development process that follows: differentiation reappears when scores are cut into grades, and oversight explains why a model is never finished once implemented.

Building a scoring or rating model is a complex and data-intensive job that calls for the judgment of experienced analysts as much as for technical skill. The process runs through five stages, and the first three are repeated until the model performs well enough to move on.

Data collection and pre-processing

With the purpose of the model settled, the first task is to assemble the data. The result is a data set of m cases, each a borrower or a loan, written as pairs of an attribute vector and a known outcome:

The data must include both defaulted and non-defaulted cases. The defaulted group is usually made up of borrowers whose payments are overdue by at least 90 days, although each institution sets the definition of default to fit its own context and the uses of the model. Data come mainly from the institution’s internal historical databases, supplemented by external sources where needed.

Raw data are rarely ready for use. Pre-processing turns them into meaningful risk attributes, removes outliers and selects the attributes that will enter the analysis. Statistical tests and algorithmic procedures help measure how much information each attribute carries and how well it discriminates between good and bad borrowers. Expert judgment remains important at this point, because an attribute should not only discriminate but also make business sense.

Hard and soft data

Models typically mix “hard” quantitative attributes with “soft” qualitative information about the borrower. Qualitative inputs are valuable, but they bring the risk of subjectivity back into a process meant to remove it, so analysts must keep subjectivity to a minimum and treat qualitative data consistently at every stage of collection and processing.

For consumer loans, the usual information covers the borrower’s income and assets, existing loan payments and payment history, family, social and employment status, collateral and guarantees, and the type and purpose of the loan, such as a credit card or a mortgage. Corporate models draw on a wider set: accounting and financial statement information, transaction and behavioral data, the firm’s size and age, its competitive advantages and market conditions, share price data where the firm is listed, corporate governance, corporate news and analytics, and the external environment (business sector data and regional and country-level information). The next two sections take these corporate inputs in turn.

A company’s published financial statements (the balance sheet, the income statement and the cash flow statement) are the starting point for almost every corporate credit model. Analysts combine items from these statements into ratios that capture four aspects of financial strength: profitability, solvency, liquidity and management efficiency.

Profitability ratios show how efficiently a firm uses its assets and capital to generate profit. Return on assets, return on equity and the gross, operating and net profit margins are the standard measures. Solvency ratios measure the debt burden and show how far the firm relies on borrowed money; the typical choices are debt to assets, debt to equity and interest coverage, which divides operating income by interest expense. Liquidity ratios test whether the firm can meet its short-term obligations by comparing current assets with current liabilities. Management efficiency ratios measure how quickly the firm turns its assets into sales and cash, and how it manages the credit its suppliers give it.

Static ratios and growth indicators

Ratios are normally examined in a static way, using the latest statements available when the loan application arrives. That snapshot is supplemented by growth indicators, which give a dynamic view of where the firm is heading. These usually cover growth in sales and operations, in value and in volume, over a period of 3 to 5 years.

Financial statements describe a firm’s fundamental strength, but they are published at long intervals and look backwards. Corporate models add several other kinds of information, each filling a gap the accounts leave open.

Behavioral and transaction data

Transaction data follow a firm’s day-to-day dealings with its creditors, such as its suppliers and banks, with the emphasis on how it pays. Credit limits, current balances and delinquencies are typical indicators. Because they are tracked and refreshed far more often than accounting figures, they give a much more current view of the borrower’s creditworthiness.

Size and age

Studies repeatedly link corporate default to the size and age of the firm. Large companies can usually raise cheaper finance from banks or from the markets, for example by issuing bonds, and they negotiate better credit terms with suppliers. Small and young firms are more exposed to adverse conditions, often struggle to obtain the finance they need in their early years, and frequently lack the history that would let a lender assess them with confidence. Size is most often measured by total assets or revenues, and market capitalisation can be used for listed firms.

Market conditions and competitive advantages

A firm’s prospects depend on the market in which it operates, and that market keeps changing. Analysts consider the intensity of competition (the number and size of rivals and their market shares), how open the market is and how regulation affects competition, and the outlook and trends for the market. Against that background, the firm’s competitive advantages matter a great deal: strength in developing and adopting technology, research and development, innovation, a privileged position in the supply chain, and diversification of operations and products. The firm’s business network also counts. A company that depends heavily on suppliers or customers of low credit quality can be dragged into difficulty by their problems, because risk travels along the network.

Financial markets data

The standard objection to accounting data is that they are static and look backwards, so they record past performance rather than prospects. For listed firms, stock market data answer that objection, since share prices embody all the information market participants have about the firm’s condition and outlook. Variables found useful for credit risk include the volatility of stock returns, market capitalisation and valuation ratios such as the price to earnings ratio, the book to market value and earnings per share. The Black-Scholes-Merton structural model is the best-known example of a default model built on market data. Evidence suggests that market data work better over short horizons and accounting data over medium and long horizons.

Corporate governance

Failures such as Enron, WorldCom and Lehman Brothers drew attention to governance, meaning management practices and business ethics. Relevant aspects include the composition and structure of the board, the competence and background of the chairman, chief executive and chief financial officer, transparency and accountability in decision-making, and the protection of shareholder rights. Several studies find that good governance goes with stronger performance and financial health, so it tells the analyst something about the long-term stability of the borrower. It also sharpens the traditional review of management: its qualifications and experience, style, corporate culture, business planning, internal audit and controls, and human resources and customer relationship management.

Corporate news and analytics

The newest source is news about the company gathered online, including from social networks, which is relevant to consumer credit as well as corporate loans. It adds non-financial information on the firm’s operations, its market position, strategic moves such as mergers and acquisitions, and its links with customers and suppliers. The sheer volume of such material requires new feature extraction techniques, notably text analytics and social network analytics, and so leads towards big data systems.

Model fitting means finding the parameter values that make the model describe the training data as well as possible. Take a linear scoring model as the simplest case:

The vector of parameters is made up of the constant term and the coefficients on the attributes, and fitting must deliver the best estimates of them given the training data. In general form this is an optimisation problem:

The loss function L measures the gap between what the model outputs and the known classification of each training case. Strictly, problem (6.1) is only an empirical stand-in for the real objective, which is to minimise the loss over every borrower to whom the model will ever be applied. That population is not observable and it shifts over time, so minimising the loss on the available sample is a reasonable working basis. The problem resembles a regression, with one difference: the dependent variable is binary (default or non-default) rather than continuous.

Depending on the form of the prediction model and the loss function chosen, problem (6.1) can be written as unconstrained optimisation, as linear programming or as non-linear programming. Statistical methods, data mining and operations research techniques can all be used to solve it, and each brings its own strengths and limitations.

A model that fits its training data well may still fail on new borrowers, so it has to be checked on a separate sample, called the validation sample or holdout sample. That sample uses the same risk attributes as the training set but contains different borrowers or loans, and the aim is to estimate how the model will perform once it is used in practice. Both statistical and economic performance measures are applied. If the results disappoint, the analyst either repeats the fitting with a different algorithm or goes back to the data stage to choose other attributes or add new data. Data preparation, fitting and validation usually take several rounds before the outcome is satisfactory.

Back-testing

The main validation scheme is back-testing: a statistical evaluation of how well the model generalises and predicts, using historical data other than the data it was built on. Two kinds of test are involved. An out-of-sample test applies the model to borrowers who were not in the development sample. That is routine in statistical modelling, but for credit models it is not enough, because a model built at one date is then used for several later periods until it is replaced. So, the model must also pass an out-of-time test, in which it is applied to data from a later period than the one used in development: built on data up to period t, it is assessed on periods t + 1, t + 2 and so on. The best practice combines the two, testing on different borrowers and different periods together.

Walk-forward testing

Even a combined test can mislead, because a good result on one period and one sample may not carry over to others. Walk-forward testing addresses this by repeating the exercise over T + 1 consecutive runs. In the first run, data up to period t minus 1 are used to build the model, which is then tested on borrowers in period t. The window then moves forward one period: a new model is built on data up to t and tested on period t + 1, and so on. Resampling methods such as cross-validation and the bootstrap can be layered on top to reduce dependence on particular test samples and allow statistical analysis of the results.

Benchmarking

Benchmarking is an alternative to back-testing. Rather than comparing the model’s estimates with realised defaults by statistical methods, it compares them with external assessments, typically the ratings of credit rating agencies. The process is qualitative and judgmental. The goal is not to make the internal model reproduce the benchmark but to find large discrepancies and understand them. Some differences are natural, because the internal model and the external benchmark are built for different purposes and customer bases; others reveal that the internal model needs recalibration or improvement. Choosing a suitable benchmark is critical for the comparison to mean anything.

For a rating system, a validated model is not the end product. Its scores must be translated into rating classes, and at least ten categories are normally defined so that the risk scale is fine enough to be useful. Two conditions guide the design. The empirical PDs of neighbouring grades should be clearly separated, and borrowers should not be piled up in a few grades.

The empirical PD of a grade is simply the observed default frequency inside it:

A published illustration uses the PD from a scoring model to form nine grades, from low risk with PD < 0.25% to high risk with PD of 50% or more. In that sample no grade holds an excessive share of borrowers: even the two largest, central categories each hold less than 30% of the cases. The empirical PD roughly doubles from one grade to the next as risk rises, which shows good separation, and it sits inside the estimated PD range of each grade, which shows the grades are calibrated in line with the scoring model.

Validating the ratings and implementing the system

The rating scheme needs its own validation, just as the model did. Analysts check its stability over time, for instance through rating migration, the distribution of borrowers across the grades, and the agreement between the estimated PD of each grade and the empirical PD later observed among rated borrowers. If the ratings fail these checks, the first remedy is a different specification of the grades; building a new model is the last resort.

The final stage puts the model into a live scoring or rating system that produces real-time estimates for every new application. Performance is then monitored and reviewed frequently. Updates take one of two forms: recalibrating the ratings while leaving the decision model unchanged, or estimating a new model when the quality and accuracy of the risk estimates deteriorate significantly and persistently.

Credit rating agencies (CRAs) give individual and institutional investors information that helps them judge whether issuers of debt and fixed-income securities will meet their obligations. The instruments they rate include government and corporate bonds, certificates of deposit, municipal bonds, preferred stock, and structured products such as collateralized debt obligations and mortgage-backed securities. Their role has grown with the globalisation of financial markets, because the sheer range of debt issues is hard for investors, financial institutions and supervisors to monitor on their own.

The industry dates back to the early 1900s. Today three agencies, Moody’s, Standard and Poor’s (S&P) and Fitch, hold more than 90% of the global market between them. Moody’s and S&P each cover about 40% of the market, and Fitch about 15%, with approximately 350,000 outstanding ratings. Many smaller and specialised agencies serve local markets and individual countries.

The rating scales

The agencies rate corporate and sovereign debt on multi-grade ordinal scales running from the strongest issuers down to issuers in default. The scale splits into investment grades, for low-risk borrowers, and speculative grades, for borrowers of weaker credit quality and higher default risk.

Grade pairings as printed in the source table.

Historical default rates by grade

The agencies’ data show average one-year default rates below 0.2% for investment grades. For speculative grades the rate starts at around 0.7% for the top of the speculative range (Ba or BB) and exceeds 25% for the riskiest grades (C and D). The table below sets the three agencies side by side; the last column is derived from the Moody’s figures.

Sources: Moody’s Annual Default Study (1920 to 2016); S&P and Fitch default studies. S&P and Fitch report a single CCC/C bucket, which covers Moody’s Caa and Ca-C together.

Note that the S&P and Fitch CCC/C rates (26.78 and 28.13) should be compared with the Moody’s Ca-C rate of 25.62 as much as with Caa, because their bucket covers the whole bottom of the scale.

Rating horizons and inputs

Ratings are issued for different horizons. Short-term ratings assess credit risk over a short period, usually one year, and reflect the debtor’s ability to meet short-term obligations; medium-term and long-term ratings cover longer periods, and an agency may use a different scale for each horizon. Agency ratings combine analytical models with judgment and draw on the full range of inputs described earlier. The data come from proprietary databases and industry and market reports, and from on-site meetings between agency analysts and the executives of rated companies.

Investors, banks, regulators, supervisors and ordinary companies all rely on agency ratings, yet those ratings attract four recurring criticisms.

Transparency, accountability and conflicts of interest

Agencies generally do not disclose how they arrive at a rating, which leaves outsiders unable to judge the soundness of their methods. The business model adds a further concern: the agencies earn their revenue from fees paid by the issuers they rate, so the rater has a financial interest in keeping the rated entity satisfied.

Promoting debt explosion

Better scoring and rating improved risk management at banks and other lenders and produced sharper risk estimates. Lenders could then cut their risk premiums, which lowered the cost of borrowing and made debt far easier for firms and consumers to obtain. The downside is a build-up of systemic risk through credit bubbles and the credit crunches that follow them.

Poor predictive ability

The global credit crisis of 2007 to 2008, together with failures the agencies did not foresee, such as Lehman Brothers, Enron and WorldCom, raised serious questions about how well ratings predict default. Several empirical studies back those questions, finding that agency ratings forecast corporate failure less accurately than standard accounting-based and market-based models and than the internal systems that banks build for themselves.

Pro-cyclicality

The agencies describe their ratings as through-the-cycle opinions, independent of the state of the economy and reflecting the borrower’s underlying credit risk. Critics counter that ratings have been too generous in booms and too harsh in recessions. That pattern is pro-cyclical: it weakens the information value of ratings and means that downgrades in a downturn can make a crisis worse.

Why ratings are still used

Despite all this, agency ratings remain part of risk management systems across the business world. They are not meant to drive credit decisions on their own; they complement other measures within an integrated system, because credit risk has many dimensions and no single measure captures all of them. Ratings are especially useful for describing parts of systematic risk, as their effect on credit spreads shows.

What research has found

Empirical work broadly supports this balanced view. Research on scoring found that its adoption at an auto finance company sharply improved profitability by cutting lending to high-risk applicants (Einav and co-authors, 2013), and that at US banks it widened small-business lending because it lowered the cost of information flowing between lender and borrower (Frame and co-authors, 2001). A survey of 58 European microfinance institutions found more than half ready to adopt scoring systems (Forcella, 2015). On ratings, Kisgen (2006, 2009) showed that firms adjust capital structure around rating changes: downgraded firms cut their debt while upgrades have no such effect, which points to rating targets. Almeida and co-authors (2017) found that firms reduce investment and credit market exposure after a downgrade of their sovereign, whose rating acts as a ceiling on corporate ratings. Hilscher and Wilson (2017) found ratings to be strong indicators of systematic risk. The consensus is that agency ratings predict default less well than dedicated models but still contribute to an integrated view of credit risk.