1. Loss Reserve Estimates: A Statistical Approach for Determining “Reasonableness” Mark R. Shapland, FCAS, ASA, MAAA EPIC Actuaries, LLC Casualty Loss Reserve Seminar Chicago, Illinois September 8-9, 2003

2. Scope of the Paper Definition of Terms Measures of Risk What is “Reasonable”? Risk Concepts, Assumptions & Considerations Models for Calculating Ranges Practical Considerations Conclusions

3. Definition of Terms From the Statements of Statutory Accounting Principles (SSAP): Management’s Best Estimate – Management’s best estimate of its liabilities is to be recorded. This amount may or may not equal the actuary’s best estimate. Ranges of Reserve Estimates – When management believes no estimate is better than any other within the range, management should accrue the midpoint. If a range can’t be determined, management should accrue the best estimate. Management’s range may or may not equal the actuary’s range. Best Estimate by Line – Management should accrue its best estimate by line of business and in the aggregate. Recognized redundancies in one line of business cannot be used to offset recognized deficiencies in another line of business.

4. Definition of Terms From the Actuarial Statement of Principles No. 36 (ASOP 36): Risk Margin – An amount that recognizes uncertainty; also known as a provision for uncertainty. Determination of Reasonable Provision – When the stated reserve amount is within the actuary’s range of reasonable reserve estimates, the actuary should issue a statement of actuarial opinion that the stated reserve amount makes a reasonable provision for the liabilities.

5. Definition of Terms From the Actuarial Statement of Principles No. 36 (ASOP 36): Range of Reasonable Reserve Estimates – The actuary may determine a range of reasonable reserve estimates that reflects the uncertainties associated with analyzing the reserves. A range of reasonable estimates is a range of estimates that could be produced by appropriate actuarial methods or alternative sets of assumptions that the actuary judges to be reasonable. The actuary may include risk margins in a range of reasonable estimates, but is not required to do so. A range of reasonable reserves, however, usually does not represent the range of all possible outcomes.

6. Definition of Terms Other Definitions Offered in the Paper: Reserve – an amount carried in the liability section of a risk- bearing entity’s balance sheet for claims incurred prior to a given accounting date. Liability – the actual amount that is owed and will ultimately be paid by a risk-bearing entity for claims incurred prior to a given accounting date. Loss Liability – the expected value of all estimated future claim payments. Risk (from the “risk-bearers” point of view) – the uncertainty (deviations from expected) in both timing and amount of the future claim payment stream.

7. Measures of Risk From statistics: Variance, standard deviation, kurtosis, average absolute deviation, Value at Risk, Tail Value at Risk, etc. which are measures of dispersion. Other measures useful in determining “reasonableness” could include: mean, mode, median, etc. The choice for measure of risk will also be important when considering the “reasonableness” and “materiality” of the reserves in relation to the capital position.

8. Measures of Risk Types of Risk: Process Risk – the randomness of future outcomes given a known distribution of possible outcomes. Parameter Risk – the potential error in the estimated parameters used to describe the distribution of possible outcomes, assuming the process generating the outcomes is known. Model Risk – the chance that the model (“process”) used to estimate the distribution of possible outcomes is incorrect or incomplete.

9. What is “Reasonable”? A range, by itself, creates problems: A range (arbitrary or otherwise) can be misleading to the layperson – it can give the impression that any number in that range is equally likely. A range can also give a false sense of security to the layperson – it gives the impression that as long as the carried reserve is “within the range” anything is reasonable (and therefore in compliance) as long as it can be justified by other means.

10. What is “Reasonable”? A range, by itself, creates problems: There is currently no specific guidance on how to consistently determine a range within the actuarial community (e.g., +/- X%, +/- $X, using various estimates, etc.). A range, in and of itself, has insufficient meaning without some other context to help define it.

11. What is “Reasonable”? $70M $140M

12. What is “Reasonable”? We should define a “reasonable” range based on probabilities of the distribution of possible outcomes. This can be translated into a range of liabilities that correspond to those probabilities. Premise:

13. What is “Reasonable”? A probability range has several advantages: The “risk” in the data defines the range. Adds context to other statistical measures. A “reserve margin” can be defined more precisely. Can be related to risk of insolvency and materiality issues. Others can define what is reasonable for them.

14. What is “Reasonable”? Comparison of “Reasonable” Reserve Ranges by Method Relatively Stable LOBMore Volatile LOB MethodLowEVHighLowEVHigh Expected +/- 20%8010012080100120 50 th to 75 th Percentile9710011590100150

15. What is “Reasonable”? Comparison of “Normal” vs. “Skewed” Liability Distributions

16. What is “Reasonable”? Comparison of Aggregate Liability Distributions

17. What is “Reasonable”? Liability Estimate Probability Expected Value Median Mode 50 th Percentile 75 th Percentile Reasonable & Prudent Margin Reasonable & Conservative Margin

18. What is “Reasonable”? Comparison of “Reasonable” Reserve Ranges with Probabilities of Insolvency “Low” Reserve Risk Corresponding Surplus Depending on Situation Loss ReservesSituation ASituation BSituation C AmountProb.Amount Prob. Of Ins.Amount Prob. Of Ins.Amount Prob. Of Ins. 10050%8040%12015%1601% 11075%7040%11015%1501% 12090%6040%10015%1401%

19. What is “Reasonable”? Comparison of “Reasonable” Reserve Ranges with Probabilities of Insolvency “Medium” Reserve Risk Corresponding Surplus Depending on Situation Loss ReservesSituation ASituation BSituation C AmountProb.Amount Prob. Of Ins.Amount Prob. Of Ins.Amount Prob. Of Ins. 10050%8060%12040%16010% 12075%6060%10040%14010% 14090%4060%8040%12010%

20. What is “Reasonable”? Comparison of “Reasonable” Reserve Ranges with Probabilities of Insolvency “High” Reserve Risk Corresponding Surplus Depending on Situation Loss ReservesSituation ASituation BSituation C AmountProb.Amount Prob. Of Ins.Amount Prob. Of Ins.Amount Prob. Of Ins. 10050%8080%12050%16020% 15075%3080%7050%11020% 20090%-2080%2050%6020%

21. What is “Reasonable”? Satisfying Different Constituents: Principle of Greatest Common Interest – the “largest amount” considered “reasonable” when a variety of constituents share a common goal or interest, such that all common goals or interests are met; and the Principle of Least Common Interest – the “smallest amount” considered “reasonable” when a variety of constituents share a common goal or interest, such that all common goals or interests are met.

22. What is “Reasonable”? Liability Estimate Probability Expected Value Median Mode 50 th Percentile $70M $140M

23. Risk Concepts, Assumptions And Considerations Concept 1: For each accident year, the coefficient of variation should be the largest for the oldest (earliest) year and will, generally, get smaller when compared to more and more recent years. Concept 2: For each accident year, the standard error (on a dollar basis) should be the smallest for the oldest (earliest) year and will, generally, get larger when compared to more and more recent years.

24. Risk Concepts, Assumptions And Considerations Concept 3: The coefficient of variation should be smaller for all accident years combined than for any individual year. Concept 4: The standard error (on a dollar basis) should be larger for all accident years combined than for any individual year.

25. Risk Concepts, Assumptions And Considerations Concept 5: The standard error should be smaller for all lines of business combined than the sum of the individual lines of business – on both a dollar basis and as a percentage of total liabilities (i.e., coefficient of variation). Concept 6: In theory, it seems reasonable to allocate any overall “reserve margin” (selected by management) based on the standard error by line after adjusting for covariances between lines.

26. Risk Concepts, Assumptions And Considerations Concept 7: Whenever simulated data is created, it should exhibit the same statistical properties as the real data. In other words, the simulated data should be statistically indistinguishable from real data.

27. Risk Concepts, Assumptions And Considerations Assumption 1: For lines of business with small payment sizes (e.g., Auto Physical Damage) Normality might be a reasonable simplifying assumption. Assumption 2: For most lines of business, the distribution of individual payments, or payments grouped by incremental period, is skewed toward larger values. Thus, it would be better to model the claim payment stream using a Lognormal, Gamma, Pareto, Burr or some other skewed distribution function that seems to fit the observed values.

28. Risk Concepts, Assumptions And Considerations Assumption 3: Estimating the distribution of loss liabilities assuming normality could produce misleading results. Assumption 4: Estimating the distribution of loss liabilities assuming normality, but “simulating” the loss distribution using a lognormal distribution (or some other skewed distribution) is marginally better.

29. Risk Concepts, Assumptions And Considerations Consideration 1: The “extra” information in the case reserves is generally believed to add value by giving a “better” estimate of the expected mean. However, does this “extra” information really change the estimate of the expected value of the payment stream (by year), or does it give a better “credibility adjusted” estimate of the likely outcome (by year) as the additional information comes to light and leave the expected value of the payments unchanged?

30. Risk Concepts, Assumptions And Considerations Consideration 2: Consider two identical books of business with two different insurance companies. They are identical except that one company sets up case reserves on the claims and the other does not. The estimates of the total liabilities (IBNR vs. case plus IBNR) are identical. Will the deviations of actual from the expected value of the future claim payments be any different?

31. Risk Concepts, Assumptions And Considerations Consideration 3: Since measuring the variations in the incurred claims does not directly measure the variations in the payment stream, should risk measures based on incurred claims be used to quantify risk for management?

32. Models For Calculating Ranges Many good probability models have been built using “Collective Risk Theory” Each of these models make assumptions about the processes that are driving claims and their settlement values None of them can ever completely eliminate “model risk” Models used to calculate liability ranges are grouped into four general categories: 1) Multiple Projection Models, 2) Statistics from Link Ratio Models, 3) Incremental Models, and 4) Simulation Models

33. Models For Calculating Ranges Description: Uses multiple models, data, assumptions Assume various estimates are a good proxy for the variation of the expected outcomes 1) Multiple Projection Models: Primary Advantage: Better than no range at all

34. Models For Calculating Ranges Problems: It does not provide a measure of the density of the distribution for the purpose of producing a probability function The “distribution” of the estimates is a distribution of the models and assumptions used, not a distribution of the expected future claim payments. Link ratio models only produce a single point estimate and there is no statistical process for determining if this point estimate is close to the expected value of the distribution of possible outcomes or not. 1) Multiple Projection Models:

35. Models For Calculating Ranges Problems: Since there are no statistical measures for these models, any overall distribution for all lines of business combined will be based on the addition of the individual ranges by line of business with judgmental adjustments for covariance, if any. 1) Multiple Projection Models:

36. Models For Calculating Ranges Uses: Data limitations may prevent the use of more advanced models. A strict interpretation of the guidelines in ASOP No. 36 seems to imply the use of this “model” to create a “reasonable” range 1) Multiple Projection Models:

37. Models For Calculating Ranges Suggested addition to ASOP No. 36: “Whenever a range of expected values is produced as the range of reasonable estimates, and the actuary has no further means of producing a reasonable distribution of possible outcomes, then the midpoint of the range of expected values should be used as the minimum acceptable reserve.” 1) Multiple Projection Models:

38. Models For Calculating Ranges Description: Calculate standard error for link ratios to calculate distribution of outcomes / range Typically assume normality and use logs to get a skewed distribution Examples: Mack, Murphy and others 2) Statistics from Link Ratio Models: Primary Advantages: Significant improvement over multiple projections Focused on a distribution of possible outcomes

39. Models For Calculating Ranges Problems: The expected value still based on multiple models Often assume link ratio are normally distributed and constant by (accident) year – this violates three concepts Provides a process for calculating an overall probability distribution for all lines of business combined, still requires assumptions about the covariances between lines 2) Statistics from Link Ratio Models:

40. Models For Calculating Ranges Uses: If data limitations prevent the use of more sophisticated models Caveat: Suggested language for ASOP No. 36 applies to the expected value portion of the calculations 2) Statistics from Link Ratio Models:

41. Models For Calculating Ranges Description: Directly model distribution of incremental claims Typically assume lognormal or other skewed distribution Examples: Finger, Hachmeister, Zehnwirth, England, Verrall and others 3) Incremental Models: Primary Advantages: Overcome the “limitations” of using cumulative values Modeling of calendar year inflation (along the diagonal)

42. Models For Calculating Ranges Problems: Actual distribution of incremental payments may not be lognormal, but other skewed distributions generally add complexity to the formulations Correlations between lines will need to be considered when they are combined Main limitation to these models seems to be only when some data issues are present 3) Incremental Models:

43. Models For Calculating Ranges Uses: Usually, they allow the actuary to tailor the model parameters to fit the characteristics of the data. An added bonus is that some of these models allow the actuary to thoroughly test the model parameters and assumptions to see if they are supported by the data. They also allow the actuary to compare various goodness of fit statistics to evaluate the reasonableness of different models and/or different model parameters. 3) Incremental Models:

44. Models For Calculating Ranges Description: Dynamic risk model of the complex interactions between claims, reinsurance, surplus, etc., Models from other groups can be used to create such a risk model 4) Simulation Models: Primary Advantage: Can generate a robust estimate of the distribution of possible outcomes

45. Models For Calculating Ranges Problems: Models based on link ratios often exhibit statistical properties not found in the real data being modeled. Usually overcome with models based on incremental values or with ground-up simulations using separate parameters for claim frequency, severity, closure rates, etc. As with any model, the key is to make sure the model and model parameters are a close reflection of reality. 4) Simulation Models:

46. Practical Considerations Some may not agree with the statement “a reasonable range” is meaningless without some other context. Context is provided by the ASOP No 36 phrase, “that could be produced by appropriate actuarial models or alternative sets of assumptions that the actuary judges to be reasonable.” In other words, “The reasonable range is from $A to $B” must make sense in light of reasonable statements about the history of cost drivers and about the history of loss development. Are Reasonable Assumptions Enough?

47. Practical Considerations What makes selecting $A as the final reserve any more or less “reasonable” than $B or any other number in between?” Without any further guidance do we, as a profession, have any basis for selecting one number in the range over another? Are Reasonable Assumptions Enough? All of the subjectiveness cannot be removed, so setting an absolute percentile may not be a good idea. But theoretically at least, the expected value seems to be a logical minimum for a reasonableness standard.

48. Practical Considerations A standard that is less than the expected value would be akin to recommending to a casino that they set the odds at something less than in their favor. Some constituents may consider a percentage lower than the expected value to be a reasonable lower bound However, the principle of greatest common interest would suggest that other interested parties would likely insist on at least an expected value standard as the minimum for the reasonable probability range. Are Reasonable Assumptions Enough?

49. Practical Considerations Current guidelines seem to say that if you can document the reasonableness of the models and assumptions used to arrive at a “possible outcome” then, ipso facto, that “possible outcome” is reasonable. Shouldn’t we look at the reasonableness of that “possible outcome” in relation to all other possible outcomes? No matter how reasonable a given model and assumptions are, is that “possible outcome” reasonable if it is less than the expected value given a reasonable distribution of possible outcomes? Are Reasonable Assumptions Enough?

50. Conclusions Users of actuarial liability estimates based on probability ranges will get much more information for risk evaluation and decision-making, The width of the dollar range will be directly related to the potential volatility (uncertainty) of the actual data, The concept of materiality can be more directly related to the uncertainty of the estimates, Risk-Based Capital calculations could be related to the probability “level” of the reserves,

51. Conclusions Both ends of the “reasonable” range of reserves will be related to the probability distribution of possible outcomes in addition to the “reasonableness” of the underlying assumptions, The concept of a “prudent reserve margin” could be related to a portion of the probability range and will then be directly related to the uncertainty of the estimates, and The users of actuarial liability estimates would have the opportunity to give more specific input on what they consider “reasonable.”

52. Conclusions To implement the advantages of the statistical approach, the actuarial profession should consider adding wording similar to the following to ASOP No. 36: “Whenever the actuary can produce a reasonable distribution of possible outcomes, a reasonable reserve estimate should not be less than the expected value of that distribution.”

53. Conclusions ASOP No. 36 definition of Risk Margin could be improved by adding wording similar to the following: “A risk margin should include a statistically calculated amount to reflect both ‘process’ and ‘parameter’ risk and it should also include a judgmental amount to reflect ‘model’ risk.”

54. Conclusions Other issues that should be addressed in our standards include: 1)the need to consider language to more directly require testing of the assumptions for different models, 2)a more definitive solution for how to consistently disclose the relative reserve risk, and 3)a more precise definition of “material change” as it relates to reserve risk.

55. Questions?