Russian banks sovereign ratings: a comparative study S.Smirnov, A. Kosyanenko, V. Naumenko, V. Lapshin, E. Bogatyreva Higher School of Economics, Moscow

2 Introduction The aim of the research was to assess credit ratings’ quality for the purpose of Bank Counterparties Credit Risk Assessment, in order to use them in credit risk models under IRB approach. Presentation plan:  General requirements to external credit ratings  Properties of migration matrices  Assessment of conditional intervals for PD  Entropy measures  Mapping to model-based PD  Conclusions

3 Using credit ratings in models Probability of default (PD) is one of the major building blocks in credit risk management. External credit rating can be used as an input variable in PD-models. Using credit ratings in addition to other sources of information about borrower's credit risk (e.g. financial ratios, market-based information) may improve the prediction power of credit risk models (see [Kealhofer, 2003], [Loffler, 2007]) By using credit ratings as input to credit risk models one should assess the uncertainty of these variables (see [Basel II]). Capital requirements, Pricing, Provisions PD External rating

4 Basel II requirements for ratings Under the IRB approach different exposures should be treated separately, e.g. corporate, sovereign, bank, retail, and equity (Basel II, §215). “Irrespective of whether a bank is using external, internal, or pooled data sources, or a combination of the three, for its PD estimation, the length of the underlying historical observation period used must be at least five years for at least one source. If the available observation period spans a longer period for any source, and this data are relevant and material, this longer period must be used” (Basel II, § 463) Ratings assigned by for external credit assessment institution (ECAI) should be recognized by regulator and satisfy 6 eligibility criteria: Objectivity, Independence, International access/Transparency, Disclosure, Resources, Credibility (Basel II, § 91). :

5 National rating agencies in Russia There are four largest national rating agencies recognized by the Russian Central Bank: RusRating, Expert RA, National rating agency (NRA) and AK&M. Historical data for the purpose of the research was provided by RusRating and AK&M. Information about ratings assigned by NRA and Expert RA was taken from their web-sites. Rating data contains monthly information about rating assigned to Russian banks from January, 2001 to May, 2010.

6 Dynamics of rating assignment In October 2008 Russian Central Bank recognized ratings assigned by national rating agencies (RusRating, Expert RA, NRA and AK&M) for the purpose of granting unsecured loans. As of May 1, 2010 there were the following numbers of efficient credit ratings in the banking sector: RusRating – 51 AK&M – 33 NRA – 65 Expert RA - 113

7 Rating Transitions Rating agencies are not likely to revise their ratings: since 2001 there were only few rating downgrades. The number of rating upgrades is much more substantial. TotalSince Down grades Up grades Obs No. Down grades Up grades Obs No. RusRating Expert RA NRA AK&M There are several cases when ratings withdrawals due to termination of the contract followed rating downgrades in a month or two. In the beginning of 2010 Expert RA had 5 such facts.

Rating philosophies Ratings Point in Time indicate the current probability of issuer’s default. They are likely to change significantly during the bad times. Ratings Through the Cycle indicate the average probability of default during the long period of time. They are not likely to change during the economic cycle. However it’s likely that default probabilities associated with ratings grades do change.

Transition matrix computation Cohort approach takes into account only the initial and terminal states of the institution in question. Duration approach takes into account time spent in every rating grade. Under conditions of first order Markov process, time homogenous matrix structure for Russian agencies these approaches coincide.

10 Migration matrices: the case of S&P Typical S&P migration matrix (expressed in monthly transition probabilities for the purpose of comparison with Russian rating agencies): Key features are:  Distinct diagonal line (taking into account aggregation of rating classes).  Existence of non-diagonal elements which gives evidence of rating upgrades and downgrades. Source: S&P 2008 Global Corporate Default Study and Rating Transitions

11 Migration matrices: RusRating Data on rating transitions had monthly frequency. Each number in a diagonal cell of a migration matrix shows probability of the fact that rating will not change in a month period of time. Numbers below the diagonal line show the probability of rating upgrades, above –downgrades. RusRating migration matrix demonstrates sufficient amount of both upgrades and downgrades.

12 Migration matrices: Expert RA, AK&M Migration matrices for Expert RA and AK&M have much less non-diagonal elements than migration matrix for RusRating. Expert RA AK&M

13 Migration matrices: NRA Rating history of NRA has almost no downgrades.

14 Confidence intervals Confidence interval is an interval with lower (L) and upper (U) bounds that covers the unknown true parameter, i.e. L < p < U with some predefined probability: Prob{L < p < U} = 1 − α. Confidence intervals is a standard industry tool to assess uncertainty of PD estimations (see, for example [OeNB, 2004]). One of the major factors that influence the length of confidence intervals for PD is the amount of data available. There are research papers that show that to some extent it is impossible to statistically distinguish investment grade rating classes (see [Lawrenz, 2008]).

15 Confidence interval methodology To calculate confidence intervals for PD one should:  Fit a priori unconditional PD distribution from external data (Russian Deposit Insurance Agency PD model) as Beta distribution – very good agreement. Estimated parameters (a,b) = (1,16.3).  Regard each month for each bank with given rating as a trial: success if no default, failure if default. Form posterior distribution for PD: Beta (number of defaults + 1, number of non- defaults ).  Find 95% confidence interval for Beta distribution with estimated parameters and plot together with (number of observations)-1.

16 Confidence intervals: RusRating

17 Confidence intervals: Expert RA

18 Confidence intervals: NRA

19 Confidence intervals: AK&M

20 Conditional entropy Conditional entropy measures new information (in bits) contained in each successive rating value (randomly selected). To understand what happened to credit quality of the rating object (3 possibilities: whether it improved, deteriorated or remained the same) it is necessary to obtain data over the period of (months): RusRating – 9; NRA– 11; Expert RA– 13; AK&M – 21. Given migration matrix p i,j and unconditional probabilities p i (expected) conditional entropy is

21 Mapping to model assessed PD Ratings were mapped to PD estimates derived from econometric model based on balance sheet data. This model is used by Deposit Insurance Agency to assess PD of banks –participants of Deposit Insurance System. The following measures were calculated in order to estimate the accuracy of ratings:  average PD for each rating grade;  confidence intervals for PD according to each rating grade;  probability that PD associated with different rating grades will coincide.

22 Rating grades comparison methodology  Given PD samples for 2 different rating values, test a hypothesis: “these 2 samples really come from the same PD distribution”.  Non-parametric Kolmogorov-Smirnov test using as test statistics.  Enter the p-value for each pair of rating values (including general population) in a table.

23 Mapping: RusRating Probability of PD coincidence

24 Mapping: Expert RA Probability of PD coincidence

25 Mapping: NRA Probability of PD coincidence

26 Mapping: AK&M Probability of PD coincidence

27 Conclusions  It is reasonable to use external credit rating as an input parameter in credit risk models. Accuracy of these rating assessment should be taken into account.  However according to our findings we can not recommend to use ratings assigned by national rating agencies in credit risk models as the only source of information due to the lack of credibility:  rating are not likely to be downgraded;  sometimes there is no uniform dependence between rating grades and PD;  in most cases we can not differentiate between rating grades.  When new data will be accumulated it will be possible to estimate rating accuracy once more and probably use ratings as an alternative source of credit quality information.

28 References 1.Basel Committee on Banking Supervision. International Convergence of Capital Measurement and Capital Standards. A Revised Framework. Bank for International Settlements. June 2006 (Basel II). 2.Kealhofer, Quantifying Credit Risk I: Default Prediction. Finandal Analysts Journal, 59, pp Loffler, The Complementary Nature of Ratings and Market- Based Measures of Default Risk. The Journal of Fixed income, pp OeNB (Oesterreichische Nationalbank), Rating Models and Validation in Guidelines on Credit Risk Management. 5.Lawrenz J. Assessing the estimation uncertainty of default probabilities.// Kredit und Kapital Vol. 41 (2). pp

29 Thank you for your attention!