1. Dealing with the Differences in Hurricane Models Catastrophe Risk Management Seminar October 7 & 8, 2002 Ronald T. Kozlowski Martin M. Simons William Gardner

2. Agenda F Kozlowski - ASOP #38 - Using Models Outside the Actuary’s Area of Expertise F Simons - Why are hurricane models different? F Gardner - Quantification of model uncertainty and variations in model results

3. ASOP #38 - Development Background F 11-15-95 American Academy of Actuaries Casualty Practice Council Meeting F ASB Casualty Council Why is a standard needed: Regulators and other users...question the applicability of these analyses when they are based in large part on the results of models that are either outside of the normal range of actuarial work or for which key parts of the model contain proprietary information and are not subject to normal disclosure processes.

4. ASOP #38 - Development Background F ASB Casualty Council believes that a new standard on the Use of Complex Models in Actuarial Practices is required.... Address the level of understanding of the model and any underlying theories... Also address what control processes must be used by the actuary in establishing a reliance on the model’s output. F 1996 - ASB Task Force on Complex Models F Adopted by the Actuarial Standards Board June 2000

5. ASB Task Force on Complex Models F Karen F. Terry, Chair F Kay A. Cleary F Alice H. Gannon F Paul E. Kinson F Ronald T. Kozlowski F Godfrey Perrott F David A. Lalonde F Jeffrey F. McCarty F Daniel M. Scheibenreif F A. Eric Thorlacius F Joan M. Weiss F Kurt Reichle

6. Issues F Is a standard needed? F Should it address when to use a model? F Who should standard apply to? F Raising the bar F Reliance on experts F Proprietary issues

7. Using Models Outside the Actuary’s Area of Expertise (P/C) F Section 1 - Purpose, Scope, Cross References, and Effective Date F Section 2 - Definitions F Section 3 - Analysis of Issues and Recommended Practices F Section 4 - Communications & Disclosures F Appendices

8. Section 1 - Scope This standard applies to actuaries who use models that incorporate specialized knowledge outside of the actuary’s own area of expertise when performing professional services in connection with property and casualty insurance coverages... This standard applies to all models whether or not they are proprietary in nature. The standard is intended to be used in conjunction with other actuarial standards of practice (ASOPs) providing guidance for actuarial work,... If a conflict exists between this standard and applicable law, compliance with applicable law is not considered to be a deviation from this standard.

9. Section 3.1 - Introduction When using a model that incorporates specialized knowledge outside the actuary’s own area of expertise, the actuary should: FDetermine appropriate reliance on experts FHave basic understanding of the model FEvaluate whether model is appropriate for intended application FConfirm appropriate validation has occurred FDetermine appropriate use of model Level of effort in understanding and evaluating should be consistent with intended use and its materiality to results of actuarial analysis

10. Section 3.2 - Appropriate Reliance on Experts Actuary should consider whether: F individual is an expert F model has been reviewed by experts in field F there are standards that apply to the model or to the testing and validation of the model... certified as having met such standards

11. Section 3.3 - Understanding of the Model “The actuary should be familiar with the basic components of the model and understand both the user input and the model output...” FModel Components (familiar with basic components, basic understanding, is it generally accepted, how tested or validated, level of independent expert review) FUser Input (detail required to produce results consistent with intended use) FModel Output (consistent with intended use)

12. Section 3.4 - Appropriateness of the Model for the Intended Application Is the model appropriate for particular actuarial analysis? May consider: F Applicability of historical data F Developments in relevant fields. Impossible to remain up to date if not an expert in the field

13. Section 3.5 - Appropriate Validation The actuary should evaluate the user input and reasonableness of the model output F Directed to ASOP 23, Data Quality F Reasonability may consider  Results of alternative models or methods  Historical observations  Consistency of relationships  Sensitivity to variations in assumptions

14. Section 3.6 - Appropriate Use of the Model Having completed sections 3.2-3.5, the actuary should use judgment to determine whether it is appropriate to use the model results, making any compensating adjustments as necessary.

15. Section 3.7 - Reliance on Model Evaluation by Another Actuary F An actuary may rely on another actuary who followed this standard F Satisfied that other actuary’s evaluation performed in accordance and is appropriate for intended purpose F Disclose such reliance

16. HURRICANE MODELS WHY ARE THEIR RESULTS DIFFERENT?

17. MODEL COMPONENTS PRIMARY COMPONENTS w Meteorology w Vulnerability w Actuarial SECONDARY COMPONENTS w Statistical w Computer

18. METEOROLOGICAL COMPONENTS w Hurricane Frequencies geographical frequencies frequency by magnitude directional frequencies w Hurricane Tracks initial position initial direction directional movements

19. METEOROLOGICAL COMPONENTS - CONTINUED w Hurricane Characteristics central pressure wind speed forward velocity radius of maximum winds radius of hurricane force winds far field pressure other characteristics –Rankine vortex –Beta parameter

20. METEOROLOGICAL COMPONENTS - CONTINUED Surface water temperature Geographical impacts Distance from landfall Surface roughness topography buildings trees

21. VULNERABILITY CHARACTERISTICS w Data used to derive damage functions Insurance claim data Post hurricane inspections Wind tunnel tests Engineering judgment w Data used to verify damage functions Insurance claim data Post hurricane inspections

22. VULNERABILITY CHARACTERISTICS - CONTINUED w Adjustments to damage functions New building codes Mitigation measures Individual structural characteristics Roof type Cladding type Structure height Many others

23. ACTUARIAL COMPONENTS w Policy provisions Policy limits Deductibles Coinsurance w Insurer practices What constitutes a covered claim? What constitutes a total loss?

24. STATISTICAL TESTS w VERIFICATION w statistically reasonable results? w UNCERTAINTY w Causes of the variations? w SENSITIVITY w How does each component impact result?

25. STATISTICAL TESTS - CONTINUED w Verification of reasonable results Chi-square goodness-of-fit test Kolmogorov-Smirnov test Cramer-von Mises test w Sensitivity and Uncertainty Latin Hypercube Sampling

26. SO WHY DO MODEL RESULTS DIFFER? w Incomplete knowledge w Myriad of variables w Very short historical period w Constantly changing environment Structures Population shifts w Policy variations w Insurer practice variations