Presentation amazonaws com
Boosting Financial Based Risk Measures with Nonfinancial Information Douglas Dwyer
27 October 2015
Movie Trivia
“The most valuable commodity I know of is information, wouldn’t you agree?”
What movie is this from?
Presentation Title, Date [Change in Slide Master]
2
Presentation Title, Date [Change in Slide Master]
3
Agenda
More data allows for more information to be used in assessing credit risk Combining Internal Bank Ratings with RiskCalc Extending RiskCalc to include usage and behavioral information So what?
4
More Information Correctly Weighted Yields Better PD Suppose the full information PD is given by
Φ ( β0 + β1 X1 + β 2 X 2 + β3 X 3 ) where •
X1 could be a financial information
•
X2 usage information
•
X3 payment behavior information
Presentation Title, Date [Change in Slide Master]
5
More Information Correctly Weighted Yields Better PD Suppose the full information PD is given by
Φ ( β0 + β1 X1 + β 2 X 2 + β3 X 3 ) These partial information PDs are all correct:
β +β X +β X 1 1 2 2 Φ 0 2 1 + β3 β +β X 0 1 1 Φ 1+ β 2 + β 2 2 3
but Gordon Gekko would want the full information one. (The technical condition for this illustrative example to work is that the X1,X2,…,XN are all are iid random variables drawn from a standard normal.)
Presentation Title, Date [Change in Slide Master]
6
Model Coefficients are Knowable Unknowns
One can estimate model coefficients for a model with three default drivers if one has a large data set containing all three drivers and knowledge of whether or not the firm defaulted.
One can estimate model coefficients for a model with two default drivers if one has a large data set containing the two drivers and knowledge of whether or not the firm defaulted.
One can not estimate the coefficients for all three drivers without a database with all three drivers.
Presentation Title, Date [Change in Slide Master]
7
Using All Information Incorrectly May Be Worse than Partial Information Suppose the full information PD is given by
Φ ( β0 + β1 X1 + β 2 X 2 + β3 X 3 ) This
β +β X +β X 1 1 2 2 Φ 0 2 1 + β3 may be better than this
Φ ( β0 + β1 X1 + β 2 X 2 + (0.6) X 3 ) where 0.6 is a ‘guess’ as to what the weight on β3 should be.
. Presentation Title, Date [Change in Slide Master]
8
In 2000, RiskCalc Excluded Specific Factors Risk Factors We Do Not Use • These factors include macroeconomic data and management quality…. too difficult to measure consistently and infer a statistical relationships and best left as independent calculations • No way of compiling such information in a database for statistical validation. • Our goal is a benchmark for credit risk. • Subjective information is neglected because its interpretation varies from person to person.
Eric Falkenstein, RiskCalc For Private Companies: Moody’s Default Model, May 2000 Presentation Title, Date [Change in Slide Master]
Regulators Expect Banks to Make Appropriate Use of All Information “Credit scoring models and other mechanical rating procedures generally use only a subset of available information. Although mechanical rating procedures may sometimes avoid some of the idiosyncratic errors made by rating systems in which human judgment plays a large role, mechanical use of limited information also is a source of rating errors. Credit scoring models and other mechanical procedures are permissible as the primary or partial basis of rating assignments, and may play a role in the estimation of loss characteristics. Sufficient human judgment and human oversight is necessary to ensure that all relevant and material information, including that which is outside the scope of the model, is also taken into consideration, and that the model is used appropriately” - Basel Committee on Banking Supervision (2004): International Convergence of Capital Measurement and Capital Standards (‘A Revised Framework’), Bank for International Settlements, Basel.
Presentation Title, Date [Change in Slide Master]
10
In 2010, We Developed a Framework to Compliment RiskCalc with Qualitative Factors Industry/Market Industry Market Conditions Customer Power Diversification of Products Competitive Position Company Years in Relationship Business Stage Supplier Power Credit History Conduct of Account Quality Management
Balance Sheet Factors Audit Method Inventory Valuation Debtor Risk/Accounts Receivable Risk Owner's Support Intrinsic Full Value of Intangibles Management Experience in Industry Financial Reporting and Formal Planning Risk Management Openness Risk Appetite Management Style & Structure
Presentation Title, Date [Change in Slide Master]
11
Banks Use Many Sources of Information to Monitor Credit Risk Monthly and Quarterly Financial Reports Business Plans and Pro-Forma Financial Statements Credit Bureau Reports Guarantor Financial Information Site Visits End of Year Financial Statements
When aware of financial distress, a bank will watch list a borrower. Such financial distress may or may not be indicated by the end of year financial statements.
Presentation Title, Date [Change in Slide Master]
12
Some Qualitative Information is Potentially Quantifiable
Using Big Data one can measure whether or not a firm is paying its utility bills a firm is using all available bank credit. a firm is current on its bank debt a firm has a history of lateness how established the borrower is More difficult to measure items include management quality firm has one core customer
Presentation Title, Date [Change in Slide Master]
13
“A More Informed PD”
14
More Informed PD » Banks use many sources of information to monitor credit risk. – If a bank is aware that a borrower is in distress, they will rate it as non-performing
» Moody’s Analytics Loan Accounting System collects non-performing information from banks
» RC EDFs are calibrated to exceed the overall observed default rate – However, on a sample restricted to non-performing borrowers, RC EDFs tend to under-predict default
» We estimate More-Informed PDs by taking the RC EDF as an input, and a dummy variable for performing / non performing and then estimate a simple probit model More Infor𝑚𝑚𝑚𝑚𝑚𝑚 𝑃𝑃𝑃𝑃 = 𝜙𝜙(𝛼𝛼0 + 𝛼𝛼1 ∗ 𝜙𝜙 −1 𝐸𝐸𝐸𝐸𝐸𝐸 + 𝛼𝛼2 ∗ 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 + 𝛼𝛼3 ∗ 𝜙𝜙 −1 𝐸𝐸𝐸𝐸𝐸𝐸
2
+ 𝛼𝛼4 ∗ 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 ∗ 𝐸𝐸𝐸𝐸𝐸𝐸) 15
Pass Borrowers Perform Better than Non Pass Borrowers Non Pass
Pass
The solid red (resp. blue) line and red (resp. blue) dots represent the predicted and actual default rate based on a model that controls for pass (resp. non-pass) borrowers and RiskCalc EDF. The dotted lines represent the +/- two standard errors of the expected default rate assuming defaults are iid to account for sampling variability. The observations have been bucketed by the RiskCalc implied Rating and are labeled accordingly.
16
The Behavioral Model
17
Adding Behavioral Information To RiskCalc 𝑃𝑃𝑃𝑃 = 𝑓𝑓(
𝑅𝑅𝑅𝑅 𝐸𝐸𝐸𝐸𝐸𝐸,
𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻 𝑜𝑜𝑜𝑜 𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝, 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 𝑜𝑜𝑜𝑜 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏,
𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈 𝑜𝑜𝑜𝑜 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙,
H𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 𝑤𝑤𝑤𝑤𝑤𝑤𝑤 𝑡𝑡𝑡𝑡𝑡 𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙) » History of late payments: – PD should increase if the borrower has been late on payments during the past year
» Payment on current balance: – PD should increase if the borrower is currently late on some of his payments
» Usage of credit lines: – PD should increase as the borrower draws down toward his credit limit
» History with lender: – If the lender has no past and/or current information on the borrower, PD should be adjusted higher
18
Accuracy Ratios for Different Combinations EDF Alone
EDF plus Payment
54%
Internal Rating+ EDF
64% Internal Rating
55%
62% 69% 65%
Payment Behavior
40%
Internal Rating+ Payment Behavior 19
Model Estimation and Performance
Driver
Impact
RiskCalc EDF Usage on Lines Of Credit Established Borrower
-
History of late payments
-
Late on present Balance
-
20
21
What does it mean?
22
Model Quality is Often Measured By Accuracy Ratio
Over Time Accuracy Ratios Have Improved
Some refer to Fra Luca Bartolomeo de Pacioli as the father of modern accounting. Pacioli lived in Italy from 14471517.
23
But What Does Accuracy Ratio Mean?
24
In Baseball, If You Win 57% of Your Games You Could Lead the Division
Toronto Blue Games are winning 57% of their games and are currently leading the division (as of end of regular 2015 season). The Baltimore Orioles win 50% of their games
25
The Saint Louis Cardinals are Currently Winning 62% of their Games
The Cardinals have the highest winning percentage in both the National League and the American League (as of the end of the regular 2015 season).
26
The Winningest Team in the History of Baseball is the Chicago Cubs in 1906 In 1906, the Chicago Cubs won 116 games out of 152 during the regular season. Their winning percentage was 76%.
27
Suppose this were the game? We randomly choose one door and placed a default behind it. Behind the other door is a good borrower. Please guess which one… Guess right and you win…
28
What Strategy Would You Use? If you flipped a coin and guessed the door on the left when the coin came up heads, you would be right 50% of the time. You would be as good as the Orioles.
29
What If You Had a Model? If you had a model that you believed in and it told which door was more likely to have the default, you would guess that door. How good of a model would you need to be as good as the Blue Jays? the Cardinals? the 1906 Chicago Cubs?
30
How Often You Win Depends on the Scores of the Bads and Goods In this case you would choose the door on the left EDF=35%
EDF=0.1%
31
How Often You Win Depends on the Scores of the Bads and Goods In this case you would choose the door on the left and win! EDF=35%
EDF=0.1%
32
How Often You Win Depends on the Scores of the Bads and Goods In this case you would choose the door on the left EDF=6%
EDF=5%
33
How Often You Win Depends on the Scores of the Bads and Goods In this case you would choose the door on the left and lose. EDF=6%
EDF=5%
34
Accuracy Ratio is Twice the Difference Between Your Winning Percentage and Winning One-Half Your Games Accuracy Ratio = 2*(Winning Percentage less 50%) A team that wins one-half of their games has an Accuracy Ratio of 0 A team that wins all their games has an Accuracy Ratio of 100% The 1906 Chicago Cubs had a winning percentage of 76%, which translates into an Accuracy Ratio of 52%.
35
Accuracy Ratio is Twice the Difference Between Your Winning Percentage and Winning One-Half Your Games Accuracy Ratio = 2*(Winning Percentage less 50%) or Winning Percentage = 0.5*Accuracy Ratio +50%
0%
50%
75%
100%
Winning Percentage
0%
50%
100%
Accuracy Ratio
36
How Good of a Model Would You Need to Be As Good As the Cardinals? Winning Percentage
AR
Perfection
100%
100%
RC plus Behavioral Information
81%
62%
RC v3.1
77%
54%
The 1906 Chicago Cubs
76%
52%
RC v1.0
75%
50%
PFM
73%
46%
Z-Score
71%
42%
Cardinals
62%
24%
Blue Jays
58%
16%
Coin Toss
50%
0%
Orioles
50%
0%
Model/Team
Accuracy Ratio is twice the difference between your winning percentage and winning one-half your games. Accuracy Ratio = 2 (Winning Percentage – 0.5) 37
What About Making Money?
If you have a better PD model than your competitor, how much more money you make depends on what loans you make and at what terms. What loans you are able to make depends on the terms your competitors offer We can illustrate the relationship with a few examples
38
Suppose You Have a Model
And your competitors do not. LGD is 50% and the average PD of the population is 2% The administrative costs of servicing the loan are 2% Your competitors charge a 3% spread over the risk free rate that is assumed to be zero (they just cover administrative costs) With a model, you would only lend to customers that have a PD of less than 2%. How much more money do you make?
39
Better Model Increases Rate of Origination and Lowers Expected Loss
Incremental Percent of Expected Gain ($) Applicants Loss on a $1B Declined Portfolio
Model
β1
AR
Expected Loss
Perfection RC plus Behavioral Information
∞
100.0%
0%
2%
-
0.54
62.4%
0.34%
30.2%
$3.43mm
$708,255
RC v3.1
0.44
54.1%
0.41%
33.2%
$4.14mm
$361,133
RC v1.0
0.40
50.2%
0.45%
34.5%
$4.50mm
$361,385
PFM
0.36
46.3%
0.49%
35.9%
$4.87mm
$366,800
Z-Score Coin Toss (or Orioles)
0.33
42.0%
0.52%
37.2%
$5.23 mm
0
0.0%
1%
?
$10.00mm
Moving from a model with an AR of 54% to 62% reduces expected loss from 0.41% to 0.34%, reduces the percent of applicants declined credit from 33.2% to 30.2%, and save $708,225 on a billion dollar portfolio. Based on simulations in which the PD is equal to N(β0 + β1 Z), where Z is a drawn from a standard normal distribution and β0 is chosen to ensure the that the population default rate stays fixed at 2% and . β1 is chosen to match the corresponding ratio. A higher β1 implies a better accuracy ratio. 40
Conclusion
41
Conclusion
Over time, banks can utilize more information to systematically manage credit risk. Both the “drivers” and “the weights for the drivers” are key pieces of information Factors that are not yet quantifiable still seem matter. Model accuracy improvements increase shareholder value
42
Appendix: Why Does the Math Work?
43
Why the Math Works Suppose the distribution of good and bad PDs look like this:
44
Why the Math Works (continued) One can compute the cumulative distribution functions of each
45
Why the Math Works From the cumulative distribution functions, one can compute the “ROC curve” For any value on the x axis, determine the PD such that x% of the good firms would have a worse PD. For the same PD find that % of bads that would have a worse PD. This would be a point on the ROC curve.
46
Why the Math Works
Suppose that the defaulter has a specific PD. The likelihood of winning is the probability that the good firm has a lower PD. This is just 1 minus the x value of the corresponding point on the ROC. It is indicated by the green line.
47
Why the Math Works
Draw a green line for every point on the ROC curve. The average of all these lines is the winning percentage. The average of these lines is also the area under the ROC curve. It is related to accuracy ratio by: AR = 2*(Area under ROC – 0.5)
48
© 2015 Moody’s Analytics, Inc. and/or its licensors and affiliates (collectively, “MOODY’S”). All rights reserved. ALL INFORMATION CONTAINED HEREIN IS PROTECTED BY COPYRIGHT LAW AND NONE OF SUCH INFORMATION MAY BE COPIED OR OTHERWISE REPRODUCED, REPACKAGED, FURTHER TRANSMITTED, TRANSFERRED, DISSEMINATED, REDISTRIBUTED OR RESOLD, OR STORED FOR SUBSEQUENT USE FOR ANY SUCH PURPOSE, IN WHOLE OR IN PART, IN ANY FORM OR MANNER OR BY ANY MEANS WHATSOEVER, BY ANY PERSON WITHOUT MOODY’S PRIOR WRITTEN CONSENT. All information contained herein is obtained by MOODY’S from sources believed by it to be accurate and reliable. Because of the possibility of human or mechanical error as well as other factors, however, all information contained herein is provided “AS IS” without warranty of any kind. Under no circumstances shall MOODY’S have any liability to any person or entity for (a) any loss or damage in whole or in part caused by, resulting from, or relating to, any error (negligent or otherwise) or other circumstance or contingency within or outside the control of MOODY’S or any of its directors, officers, employees or agents in connection with the procurement, collection, compilation, analysis, interpretation, communication, publication or delivery of any such information, or (b) any direct, indirect, special, consequential, compensatory or incidental damages whatsoever (including without limitation, lost profits), even if MOODY’S is advised in advance of the possibility of such damages, resulting from the use of or inability to use, any such information. The ratings, financial reporting analysis, projections, and other observations, if any, constituting part of the information contained herein are, and must be construed solely as, statements of opinion and not statements of fact or recommendations to purchase, sell or hold any securities. NO WARRANTY, EXPRESS OR IMPLIED, AS TO THE ACCURACY, TIMELINESS, COMPLETENESS, MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OF ANY SUCH RATING OR OTHER OPINION OR INFORMATION IS GIVEN OR MADE BY MOODY’S IN ANY FORM OR MANNER WHATSOEVER. Each rating or other opinion must be weighed solely as one factor in any investment decision made by or on behalf of any user of the information contained herein, and each such user must accordingly make its own study and evaluation of each security and of each issuer and guarantor of, and each provider of credit support for, each security that it may consider purchasing, holding, or selling.
49 49
27 October 2015
Movie Trivia
“The most valuable commodity I know of is information, wouldn’t you agree?”
What movie is this from?
Presentation Title, Date [Change in Slide Master]
2
Presentation Title, Date [Change in Slide Master]
3
Agenda
More data allows for more information to be used in assessing credit risk Combining Internal Bank Ratings with RiskCalc Extending RiskCalc to include usage and behavioral information So what?
4
More Information Correctly Weighted Yields Better PD Suppose the full information PD is given by
Φ ( β0 + β1 X1 + β 2 X 2 + β3 X 3 ) where •
X1 could be a financial information
•
X2 usage information
•
X3 payment behavior information
Presentation Title, Date [Change in Slide Master]
5
More Information Correctly Weighted Yields Better PD Suppose the full information PD is given by
Φ ( β0 + β1 X1 + β 2 X 2 + β3 X 3 ) These partial information PDs are all correct:
β +β X +β X 1 1 2 2 Φ 0 2 1 + β3 β +β X 0 1 1 Φ 1+ β 2 + β 2 2 3
but Gordon Gekko would want the full information one. (The technical condition for this illustrative example to work is that the X1,X2,…,XN are all are iid random variables drawn from a standard normal.)
Presentation Title, Date [Change in Slide Master]
6
Model Coefficients are Knowable Unknowns
One can estimate model coefficients for a model with three default drivers if one has a large data set containing all three drivers and knowledge of whether or not the firm defaulted.
One can estimate model coefficients for a model with two default drivers if one has a large data set containing the two drivers and knowledge of whether or not the firm defaulted.
One can not estimate the coefficients for all three drivers without a database with all three drivers.
Presentation Title, Date [Change in Slide Master]
7
Using All Information Incorrectly May Be Worse than Partial Information Suppose the full information PD is given by
Φ ( β0 + β1 X1 + β 2 X 2 + β3 X 3 ) This
β +β X +β X 1 1 2 2 Φ 0 2 1 + β3 may be better than this
Φ ( β0 + β1 X1 + β 2 X 2 + (0.6) X 3 ) where 0.6 is a ‘guess’ as to what the weight on β3 should be.
. Presentation Title, Date [Change in Slide Master]
8
In 2000, RiskCalc Excluded Specific Factors Risk Factors We Do Not Use • These factors include macroeconomic data and management quality…. too difficult to measure consistently and infer a statistical relationships and best left as independent calculations • No way of compiling such information in a database for statistical validation. • Our goal is a benchmark for credit risk. • Subjective information is neglected because its interpretation varies from person to person.
Eric Falkenstein, RiskCalc For Private Companies: Moody’s Default Model, May 2000 Presentation Title, Date [Change in Slide Master]
Regulators Expect Banks to Make Appropriate Use of All Information “Credit scoring models and other mechanical rating procedures generally use only a subset of available information. Although mechanical rating procedures may sometimes avoid some of the idiosyncratic errors made by rating systems in which human judgment plays a large role, mechanical use of limited information also is a source of rating errors. Credit scoring models and other mechanical procedures are permissible as the primary or partial basis of rating assignments, and may play a role in the estimation of loss characteristics. Sufficient human judgment and human oversight is necessary to ensure that all relevant and material information, including that which is outside the scope of the model, is also taken into consideration, and that the model is used appropriately” - Basel Committee on Banking Supervision (2004): International Convergence of Capital Measurement and Capital Standards (‘A Revised Framework’), Bank for International Settlements, Basel.
Presentation Title, Date [Change in Slide Master]
10
In 2010, We Developed a Framework to Compliment RiskCalc with Qualitative Factors Industry/Market Industry Market Conditions Customer Power Diversification of Products Competitive Position Company Years in Relationship Business Stage Supplier Power Credit History Conduct of Account Quality Management
Balance Sheet Factors Audit Method Inventory Valuation Debtor Risk/Accounts Receivable Risk Owner's Support Intrinsic Full Value of Intangibles Management Experience in Industry Financial Reporting and Formal Planning Risk Management Openness Risk Appetite Management Style & Structure
Presentation Title, Date [Change in Slide Master]
11
Banks Use Many Sources of Information to Monitor Credit Risk Monthly and Quarterly Financial Reports Business Plans and Pro-Forma Financial Statements Credit Bureau Reports Guarantor Financial Information Site Visits End of Year Financial Statements
When aware of financial distress, a bank will watch list a borrower. Such financial distress may or may not be indicated by the end of year financial statements.
Presentation Title, Date [Change in Slide Master]
12
Some Qualitative Information is Potentially Quantifiable
Using Big Data one can measure whether or not a firm is paying its utility bills a firm is using all available bank credit. a firm is current on its bank debt a firm has a history of lateness how established the borrower is More difficult to measure items include management quality firm has one core customer
Presentation Title, Date [Change in Slide Master]
13
“A More Informed PD”
14
More Informed PD » Banks use many sources of information to monitor credit risk. – If a bank is aware that a borrower is in distress, they will rate it as non-performing
» Moody’s Analytics Loan Accounting System collects non-performing information from banks
» RC EDFs are calibrated to exceed the overall observed default rate – However, on a sample restricted to non-performing borrowers, RC EDFs tend to under-predict default
» We estimate More-Informed PDs by taking the RC EDF as an input, and a dummy variable for performing / non performing and then estimate a simple probit model More Infor𝑚𝑚𝑚𝑚𝑚𝑚 𝑃𝑃𝑃𝑃 = 𝜙𝜙(𝛼𝛼0 + 𝛼𝛼1 ∗ 𝜙𝜙 −1 𝐸𝐸𝐸𝐸𝐸𝐸 + 𝛼𝛼2 ∗ 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 + 𝛼𝛼3 ∗ 𝜙𝜙 −1 𝐸𝐸𝐸𝐸𝐸𝐸
2
+ 𝛼𝛼4 ∗ 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 ∗ 𝐸𝐸𝐸𝐸𝐸𝐸) 15
Pass Borrowers Perform Better than Non Pass Borrowers Non Pass
Pass
The solid red (resp. blue) line and red (resp. blue) dots represent the predicted and actual default rate based on a model that controls for pass (resp. non-pass) borrowers and RiskCalc EDF. The dotted lines represent the +/- two standard errors of the expected default rate assuming defaults are iid to account for sampling variability. The observations have been bucketed by the RiskCalc implied Rating and are labeled accordingly.
16
The Behavioral Model
17
Adding Behavioral Information To RiskCalc 𝑃𝑃𝑃𝑃 = 𝑓𝑓(
𝑅𝑅𝑅𝑅 𝐸𝐸𝐸𝐸𝐸𝐸,
𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻 𝑜𝑜𝑜𝑜 𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝, 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 𝑜𝑜𝑜𝑜 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏,
𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈 𝑜𝑜𝑜𝑜 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙,
H𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 𝑤𝑤𝑤𝑤𝑤𝑤𝑤 𝑡𝑡𝑡𝑡𝑡 𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙) » History of late payments: – PD should increase if the borrower has been late on payments during the past year
» Payment on current balance: – PD should increase if the borrower is currently late on some of his payments
» Usage of credit lines: – PD should increase as the borrower draws down toward his credit limit
» History with lender: – If the lender has no past and/or current information on the borrower, PD should be adjusted higher
18
Accuracy Ratios for Different Combinations EDF Alone
EDF plus Payment
54%
Internal Rating+ EDF
64% Internal Rating
55%
62% 69% 65%
Payment Behavior
40%
Internal Rating+ Payment Behavior 19
Model Estimation and Performance
Driver
Impact
RiskCalc EDF Usage on Lines Of Credit Established Borrower
-
History of late payments
-
Late on present Balance
-
20
21
What does it mean?
22
Model Quality is Often Measured By Accuracy Ratio
Over Time Accuracy Ratios Have Improved
Some refer to Fra Luca Bartolomeo de Pacioli as the father of modern accounting. Pacioli lived in Italy from 14471517.
23
But What Does Accuracy Ratio Mean?
24
In Baseball, If You Win 57% of Your Games You Could Lead the Division
Toronto Blue Games are winning 57% of their games and are currently leading the division (as of end of regular 2015 season). The Baltimore Orioles win 50% of their games
25
The Saint Louis Cardinals are Currently Winning 62% of their Games
The Cardinals have the highest winning percentage in both the National League and the American League (as of the end of the regular 2015 season).
26
The Winningest Team in the History of Baseball is the Chicago Cubs in 1906 In 1906, the Chicago Cubs won 116 games out of 152 during the regular season. Their winning percentage was 76%.
27
Suppose this were the game? We randomly choose one door and placed a default behind it. Behind the other door is a good borrower. Please guess which one… Guess right and you win…
28
What Strategy Would You Use? If you flipped a coin and guessed the door on the left when the coin came up heads, you would be right 50% of the time. You would be as good as the Orioles.
29
What If You Had a Model? If you had a model that you believed in and it told which door was more likely to have the default, you would guess that door. How good of a model would you need to be as good as the Blue Jays? the Cardinals? the 1906 Chicago Cubs?
30
How Often You Win Depends on the Scores of the Bads and Goods In this case you would choose the door on the left EDF=35%
EDF=0.1%
31
How Often You Win Depends on the Scores of the Bads and Goods In this case you would choose the door on the left and win! EDF=35%
EDF=0.1%
32
How Often You Win Depends on the Scores of the Bads and Goods In this case you would choose the door on the left EDF=6%
EDF=5%
33
How Often You Win Depends on the Scores of the Bads and Goods In this case you would choose the door on the left and lose. EDF=6%
EDF=5%
34
Accuracy Ratio is Twice the Difference Between Your Winning Percentage and Winning One-Half Your Games Accuracy Ratio = 2*(Winning Percentage less 50%) A team that wins one-half of their games has an Accuracy Ratio of 0 A team that wins all their games has an Accuracy Ratio of 100% The 1906 Chicago Cubs had a winning percentage of 76%, which translates into an Accuracy Ratio of 52%.
35
Accuracy Ratio is Twice the Difference Between Your Winning Percentage and Winning One-Half Your Games Accuracy Ratio = 2*(Winning Percentage less 50%) or Winning Percentage = 0.5*Accuracy Ratio +50%
0%
50%
75%
100%
Winning Percentage
0%
50%
100%
Accuracy Ratio
36
How Good of a Model Would You Need to Be As Good As the Cardinals? Winning Percentage
AR
Perfection
100%
100%
RC plus Behavioral Information
81%
62%
RC v3.1
77%
54%
The 1906 Chicago Cubs
76%
52%
RC v1.0
75%
50%
PFM
73%
46%
Z-Score
71%
42%
Cardinals
62%
24%
Blue Jays
58%
16%
Coin Toss
50%
0%
Orioles
50%
0%
Model/Team
Accuracy Ratio is twice the difference between your winning percentage and winning one-half your games. Accuracy Ratio = 2 (Winning Percentage – 0.5) 37
What About Making Money?
If you have a better PD model than your competitor, how much more money you make depends on what loans you make and at what terms. What loans you are able to make depends on the terms your competitors offer We can illustrate the relationship with a few examples
38
Suppose You Have a Model
And your competitors do not. LGD is 50% and the average PD of the population is 2% The administrative costs of servicing the loan are 2% Your competitors charge a 3% spread over the risk free rate that is assumed to be zero (they just cover administrative costs) With a model, you would only lend to customers that have a PD of less than 2%. How much more money do you make?
39
Better Model Increases Rate of Origination and Lowers Expected Loss
Incremental Percent of Expected Gain ($) Applicants Loss on a $1B Declined Portfolio
Model
β1
AR
Expected Loss
Perfection RC plus Behavioral Information
∞
100.0%
0%
2%
-
0.54
62.4%
0.34%
30.2%
$3.43mm
$708,255
RC v3.1
0.44
54.1%
0.41%
33.2%
$4.14mm
$361,133
RC v1.0
0.40
50.2%
0.45%
34.5%
$4.50mm
$361,385
PFM
0.36
46.3%
0.49%
35.9%
$4.87mm
$366,800
Z-Score Coin Toss (or Orioles)
0.33
42.0%
0.52%
37.2%
$5.23 mm
0
0.0%
1%
?
$10.00mm
Moving from a model with an AR of 54% to 62% reduces expected loss from 0.41% to 0.34%, reduces the percent of applicants declined credit from 33.2% to 30.2%, and save $708,225 on a billion dollar portfolio. Based on simulations in which the PD is equal to N(β0 + β1 Z), where Z is a drawn from a standard normal distribution and β0 is chosen to ensure the that the population default rate stays fixed at 2% and . β1 is chosen to match the corresponding ratio. A higher β1 implies a better accuracy ratio. 40
Conclusion
41
Conclusion
Over time, banks can utilize more information to systematically manage credit risk. Both the “drivers” and “the weights for the drivers” are key pieces of information Factors that are not yet quantifiable still seem matter. Model accuracy improvements increase shareholder value
42
Appendix: Why Does the Math Work?
43
Why the Math Works Suppose the distribution of good and bad PDs look like this:
44
Why the Math Works (continued) One can compute the cumulative distribution functions of each
45
Why the Math Works From the cumulative distribution functions, one can compute the “ROC curve” For any value on the x axis, determine the PD such that x% of the good firms would have a worse PD. For the same PD find that % of bads that would have a worse PD. This would be a point on the ROC curve.
46
Why the Math Works
Suppose that the defaulter has a specific PD. The likelihood of winning is the probability that the good firm has a lower PD. This is just 1 minus the x value of the corresponding point on the ROC. It is indicated by the green line.
47
Why the Math Works
Draw a green line for every point on the ROC curve. The average of all these lines is the winning percentage. The average of these lines is also the area under the ROC curve. It is related to accuracy ratio by: AR = 2*(Area under ROC – 0.5)
48
© 2015 Moody’s Analytics, Inc. and/or its licensors and affiliates (collectively, “MOODY’S”). All rights reserved. ALL INFORMATION CONTAINED HEREIN IS PROTECTED BY COPYRIGHT LAW AND NONE OF SUCH INFORMATION MAY BE COPIED OR OTHERWISE REPRODUCED, REPACKAGED, FURTHER TRANSMITTED, TRANSFERRED, DISSEMINATED, REDISTRIBUTED OR RESOLD, OR STORED FOR SUBSEQUENT USE FOR ANY SUCH PURPOSE, IN WHOLE OR IN PART, IN ANY FORM OR MANNER OR BY ANY MEANS WHATSOEVER, BY ANY PERSON WITHOUT MOODY’S PRIOR WRITTEN CONSENT. All information contained herein is obtained by MOODY’S from sources believed by it to be accurate and reliable. Because of the possibility of human or mechanical error as well as other factors, however, all information contained herein is provided “AS IS” without warranty of any kind. Under no circumstances shall MOODY’S have any liability to any person or entity for (a) any loss or damage in whole or in part caused by, resulting from, or relating to, any error (negligent or otherwise) or other circumstance or contingency within or outside the control of MOODY’S or any of its directors, officers, employees or agents in connection with the procurement, collection, compilation, analysis, interpretation, communication, publication or delivery of any such information, or (b) any direct, indirect, special, consequential, compensatory or incidental damages whatsoever (including without limitation, lost profits), even if MOODY’S is advised in advance of the possibility of such damages, resulting from the use of or inability to use, any such information. The ratings, financial reporting analysis, projections, and other observations, if any, constituting part of the information contained herein are, and must be construed solely as, statements of opinion and not statements of fact or recommendations to purchase, sell or hold any securities. NO WARRANTY, EXPRESS OR IMPLIED, AS TO THE ACCURACY, TIMELINESS, COMPLETENESS, MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OF ANY SUCH RATING OR OTHER OPINION OR INFORMATION IS GIVEN OR MADE BY MOODY’S IN ANY FORM OR MANNER WHATSOEVER. Each rating or other opinion must be weighed solely as one factor in any investment decision made by or on behalf of any user of the information contained herein, and each such user must accordingly make its own study and evaluation of each security and of each issuer and guarantor of, and each provider of credit support for, each security that it may consider purchasing, holding, or selling.
49 49
