1. 1 Modeling and Calibration Errors in Measures of Portfolio Credit Risk Nikola Tarashev and Haibin Zhu Bank for International Settlements April 2007 The views expressed in this paper need not represent those of the BIS.

2. 2 Motivation ASRF model: a well-known model of portfolio credit risk Main reason for the model’s popularity: “portfolio invariance” of capital Assumption 1: perfect granularity of the portfolio Assumption 2: single common factor of credit risk Does ASRF allow for “bottom-up” calculation of economic capital ? Not really: estimating ρ i requires a global approach !!! To take full advantage of portfolio invariance  off-the-shelf values for ρ i : Pillar 1 of Basel II: ρ i = ρ IRB ( PD i ) ASRF-based capital measures are subject to: misspecification errors: ie, violated assumptions of the model calibration errors: eg, off-the-shelf values for ρ i.

3. 3 Related literature (BCBS WP No 15) Misspecification of the ASRF model: Granularity assumption: Martin and Wilde (2002), Vasicek (2002) Emmer and Tasche (2003), Gordy and Luetkebohmert (2006) Sector concentration and the number of common factors Pykhtin (2004), Duellmann (2006), Garcia Cespedes et al (2006) Duellmann and Masschelein (2006) Both assumptions: Heitfield et al (2006), Duellman et al (2006) Flawed calibration: Loeffler (2003), Morinaga and Shiina (2005)

4. 4 This paper Decomposes the wedge between target and shortcut capital measures. Exhaustive and non-overlapping components due to: Misspecification of the ASRF model Multifactor effect Granularity effect Flawed calibration the ASRF model Correlation dispersion effect Correlation level effect Plausible estimation errors Takes seriously the overall distribution of risk factors Gaussian versus fatter-tailed distributions The paper does not: (i) Discuss calibration of PDs or LGDs; (ii) address time variation in risk parameters

5. 5 Main results Applying ASRF model  Large deviations from target capital for realistic bank portfolios The main drivers of these deviations are calibration errors noise in the estimated value of ρ i wrong distributional assumptions Misspecification of the ASRF model has a much smaller impact exception: granularity effect in small portfolios

6. 6 Roadmap The ASRF model (overview) Alternative sources of error in capital measures (intuition) Empirical methodology Findings

7. 7 The ASRF model (an overview) PD i, LGD i, ρ i and weights w i are all known Assets (driven by a single common factor) drive defaults M ~ F (0,1), Z ~ G (0,1), V ~ H (0,1): weak restrictions Perfectly fine granularity: Idiosyncratic risk is diversified away (low w i & many exposures) Pairwise correlations: ρ i ρ k

8. 8 To attain solvency with probability (1- α): The ASRF model (implications) Portfolio invariance If V, M, Z are all normal and α = 0.001: which underpins the IRB approach of Basel II.999

9. 9 Errors in calculated capital charges The granularity effect (specification error 1) Stylized homogeneous portfolio: PD=1%, LGD=45%, ρ 2 = 10% Granularity  ASRF capital undershoots target capital target capital ASRF capital

10. 10 The multi-factor effect (specification error 2) Stylized homogeneous portfolio: PD=1%, LGD=45% Two sectors, weight = ω. Intra-sector correlation = 0.2; inter-sector = 0 Owing to its single-CF structure, ASRF model undershoots target Errors in calculated capital charges (cont’d)

11. 11 Correlation level effect (calibration error 1) Stylized homogeneous portfolio: PD=1%, LGD=45% Higher correlation raises the capital measure Errors in calculated capital charges (cont’d)

12. 12 Correlation dispersion effect (calibration error 2) Start with a homogeneous portfolio: PD=1%, LGD=45% ρ varies within Then, let PDs vary too, within [0.5%,1.5%] Errors in calculated capital charges (cont’d)

13. 13 Methodology. Decomposing the gap Select a portfolio: Realistic distribution of exposures across industrial sectors representative portfolio of large US banks, Heitfield et al. (2006) Size Large: 1000 exposures (homogeneous weights) Small: 200 exposures (homogeneous weights) For each portfolio, quantify the difference between two extremes: Target capital Shortcut capital, implied by off-the-shelf calibration of ASRF model Three additional capital calculations bring up the (i) multifactor (ii) granularity (iii) correlation-level (iv) correlation-dispersion effects

14. 14 Target capital N firms,  (σ) + Monte Carlo simulations Shortcut capital N = , + ASRF model Methodology: decomposing the four effects N firms, R(  ) (1-factor best fit) + copula Multi-factor effect For all measures: homogeneous portfolios, the same {PD i } i N and LGD, Gaussian distributions

15. 15 R(  ) is the “one-factor approximation” of  (σ) Loadings (  i ): allowed to differ across exposures Approximation matches well average correlation Approximation could miss correlation dispersion Fitting a single-factor structure

16. 16 Target capital N firms,  (σ) + Monte Carlo simulations N firms, R(  ) (one-factor best fit) + copula Shortcut capital N = , + ASRF model Multi-factor effect Methodology: decomposing the four effects N = , R(  ) + ASRF model Granularity effect N = , average  + ASRF model Correlation dispersion effect For all measures: homogeneous portfolios, the same {PD i } i N and LGD, Gaussian distributions Correlation level effect

17. 17 Data 10,891 non-financial firms worldwide: 40 industrial sectors mostly unrated firms Risk parameters: from Moody’s KMV, for July 2006 two sets of mutually consistent estimates: EDF: 1-year physical PD, at exposure level Global Correlation (GCORR) asset return correlations Estimated based on a multi-factor loading structure LGD = 45%

18. 18 Match sectoral distribution of the representative portfolio of US wholesale banks. Heitfield et al. (2006) Two sizes: large (1000 exposures) and small (200 exposures) 3000 simulations (for each size): to average out sampling errors mean (%) LargeSmall Average PD2.422.28 Std dev of PD5.165.05 Median PD0.260.24 Average correlation9.7810.49 Std dev of loadings9.3310.54 Correl.(PD, loading)-20.0-19.8 Simulated Portfolios

19. 19 Findings Dissecting deviations from target capital: Mean (%)Add-on (%)95% interval (%) Target capital Multi-factor effect Granularity effect Correlation dispersion effect Correlation level effect Shortcut capital (correlation = 12%) Total difference 2.95 -0.03 -0.11 0.35 0.55 3.71 0.76 -1.02 -3.73 11.86 18.64 25.76 [2.64, 3.27] [-0.09, 0] [-0.14, -0.09] [0.27, 0.43] [0.44, 0.66] [3.37, 4.06] Correlation level effect if: correlation = 6% correlation = 18% correlation = 24% -0.96 2.01 3.47 -32.54 68.14 117.63 [-1.11, -0.83] [1.84, 2.18] [3.23, 3.72]

20. 20 For small portfolios: similar results, except for the granularity effect Large Mean (%) Small Mean (%) Target capital Multi-factor effect Granularity effect Correlation dispersion effect Correlation level effect Shortcut capital Correlation level effect if correlation = 6% correlation = 18% correlation = 24% 2.95 -0.03 -0.11 0.35 0.55 3.71 -0.96 2.01 3.47 3.35 -0.04 -0.53 0.38 0.36 3.52 -1.07 1.76 3.15 add-on: large -3.73%; small -15.8% Findings, continued

21. 21 Findings, continued Discrepancy: shortcut minus target capital Explaining the variation of the discrepancy across simulated portfolios

22. 22 Delving into calibration errors Small-sample estimation errors in calibrated correlations Conduct the following exercise Design true model:  2 = 9.78%, PD = 1%, Gaussian variables, N equal exposures Draw from the true model: T periods of asset returns Adopt the point of view of a user (has data, estimates parameters) Construct sample correlation matrix (PD, etc: known) Fit a one-factor model and calculate ASRF-implied capital Repeat 1000 times for each (N,T)

23. 23 Small-sample estimation errors are significant Meaningful reduction of errors requires unrealistic sample sizes Delving into calibration errors, contd.

24. 24 Estimation errors affect substantially capital charges Bias Noise Delving into calibration errors, contd.

25. 25 The importance of distributional assumptions Stylized fact: fat tails of asset returns Gaussian (thin tails) assumption  too low capital measures Quantify the bias: Student t distributions General ASRF formula:

26. 26 Alternative distributions Gaussian assumption  negative bias in measured capital

27. 27 Conclusions Large deviations from target capital for realistic portfolios The main drivers of the deviations: calibration errors not model misspecification Challenges for risk managers and supervisors