1. William Cook Abusing statistics in retail banks, and its contribution to the banking crisis

2. Introduction  Provide modelling support to banks and building societies within retail credit risk departments  Have worked with likes of Nationwide, Lloyds Banking Group... Bradford & Bingley, Northern Rock  Based on standard UK industry practices, will present here two examples of the abuse of statistics that contributed to the banking crisis 2

3. Background to credit risk  Within banks there are two buffers against losses: Provisions – against short term forecast losses Capital – a reserve for the worst case scenario  Provisions can be difficult to forecast, capital can be even harder to estimate  Miscalculation of retail capital contributed to the downfall of various banks in the UK 3

4. Capital calculations  Regulation for capital calculation comes from the FSA  Most large banks have opted for the “Internal Ratings Based” approach  This allows banks to make their own estimates, rather than using standard benchmark ratios  Since inception, the FSA has been keen for capital to be calculated at an individual account level under IRB  A key component of these account level models is the Probability of Default, or PD  Important for capital calculations that the PD can be ‘accurately’ modelled 4

5. Credit risk through time  Percent of UK mortgages 6-12 months in arrears 5

6. Problem 1 – The Gini coefficient  Account level PD modelling performed with logistic regression  Sample consists of all those who are not in default  Outcome is 0 for those that do not default in the following year and 1 for those that do  Independent variables come from many sources and include information such as age, performance on other products and credit reference agency data  Model power typically measured with the Gini 6

7.  Gini value = 0.72 Gini – example curve 7

8. Gini – overlapping distributions  Comparing ‘good’ and ‘bad’ distributions 8 badsgoods

9. Gini - two way table example  Can also look at 2 way table of predicted versus actual  If we wish to match total predicted with total actuals, need to find cut-off in predicted values  Example with 10,000 customers, 5,000 are ‘bad’  Gini = 0.72, sensitivity = 0.78 9 Actual GoodBadTotal PredictedGood3,8911,1084,999 Bad1,1093,8925,001 Total5,000 10,000

10. Gini – two way table example 2  In reality, typically trying to model an overall bad rate at 5% or less  Another example with 10,000 customers, but only 500 bad  Keep Gini = 0.72  Sensitivity now depends on prediction cut-off: If same cut-off used, remains same (as well as specificity) If new optimal cut-off used, sensitivity drops 10

11. Gini – two way table example 2  Same cut-off, sensitivity = 0.77  New optimal cut-off, sensitivity = 0.37 11 Actual GoodBadTotal PredictedGood7,3711147,485 Bad2,1293862,515 Total9,50050010,000 Actual GoodBadTotal PredictedGood9,1873139,500 Bad313187500 Total9,50050010,000

12. Gini conclusions  Ginis reported for credit risk models are often flattering compared to two way tables  Fundamentally trying to predict an outcome that is difficult to forecast  Often default is driven by future unemployment, which may not be picked up by independent variables 12

13. Problem 2 – Cross-sectional and time-series relationships  Calculating the correct amount of capital is very much a time-series problem  Logistic regression models used for predicting default are cross-sectional  Well known in statistics that relationships that hold for one type of analysis do not necessarily hold for the other  Simple example with earnings and savings 13

14. Cross-sectional and time-series example  Income and savings time series 14

15. Macro driver of credit risk  Comparing unemployment and mortgage arrears 15

16. Conclusions  Whilst logistic regression modelling does have its place in credit risk, the apparent power of such models did deceive the regulator and the industry  Since March of this year, the FSA has been making big changes in the way in which capital should be calculated for IRB banks  Rightly, there have been moves away from account level methodology to focus on long run time-series performance data 16