Dealing with the Differences in Hurricane Models Catastrophe Risk Management Seminar October 7 & 8, 2002 Ronald T. Kozlowski Martin M. Simons William Gardner

Agenda F Kozlowski - ASOP #38 - Using Models Outside the Actuary’s Area of Expertise F Simons - Why are hurricane models different? F Gardner - Quantification of model uncertainty and variations in model results

ASOP #38 - Development Background F American Academy of Actuaries Casualty Practice Council Meeting F ASB Casualty Council Why is a standard needed: Regulators and other users...question the applicability of these analyses when they are based in large part on the results of models that are either outside of the normal range of actuarial work or for which key parts of the model contain proprietary information and are not subject to normal disclosure processes.

ASOP #38 - Development Background F ASB Casualty Council believes that a new standard on the Use of Complex Models in Actuarial Practices is required.... Address the level of understanding of the model and any underlying theories... Also address what control processes must be used by the actuary in establishing a reliance on the model’s output. F ASB Task Force on Complex Models F Adopted by the Actuarial Standards Board June 2000

ASB Task Force on Complex Models F Karen F. Terry, Chair F Kay A. Cleary F Alice H. Gannon F Paul E. Kinson F Ronald T. Kozlowski F Godfrey Perrott F David A. Lalonde F Jeffrey F. McCarty F Daniel M. Scheibenreif F A. Eric Thorlacius F Joan M. Weiss F Kurt Reichle

Issues F Is a standard needed? F Should it address when to use a model? F Who should standard apply to? F Raising the bar F Reliance on experts F Proprietary issues

Using Models Outside the Actuary’s Area of Expertise (P/C) F Section 1 - Purpose, Scope, Cross References, and Effective Date F Section 2 - Definitions F Section 3 - Analysis of Issues and Recommended Practices F Section 4 - Communications & Disclosures F Appendices

Section 1 - Scope This standard applies to actuaries who use models that incorporate specialized knowledge outside of the actuary’s own area of expertise when performing professional services in connection with property and casualty insurance coverages... This standard applies to all models whether or not they are proprietary in nature. The standard is intended to be used in conjunction with other actuarial standards of practice (ASOPs) providing guidance for actuarial work,... If a conflict exists between this standard and applicable law, compliance with applicable law is not considered to be a deviation from this standard.

Section Introduction When using a model that incorporates specialized knowledge outside the actuary’s own area of expertise, the actuary should: FDetermine appropriate reliance on experts FHave basic understanding of the model FEvaluate whether model is appropriate for intended application FConfirm appropriate validation has occurred FDetermine appropriate use of model Level of effort in understanding and evaluating should be consistent with intended use and its materiality to results of actuarial analysis

Section Appropriate Reliance on Experts Actuary should consider whether: F individual is an expert F model has been reviewed by experts in field F there are standards that apply to the model or to the testing and validation of the model... certified as having met such standards

Section Understanding of the Model “The actuary should be familiar with the basic components of the model and understand both the user input and the model output...” FModel Components (familiar with basic components, basic understanding, is it generally accepted, how tested or validated, level of independent expert review) FUser Input (detail required to produce results consistent with intended use) FModel Output (consistent with intended use)

Section Appropriateness of the Model for the Intended Application Is the model appropriate for particular actuarial analysis? May consider: F Applicability of historical data F Developments in relevant fields. Impossible to remain up to date if not an expert in the field

Section Appropriate Validation The actuary should evaluate the user input and reasonableness of the model output F Directed to ASOP 23, Data Quality F Reasonability may consider  Results of alternative models or methods  Historical observations  Consistency of relationships  Sensitivity to variations in assumptions

Section Appropriate Use of the Model Having completed sections , the actuary should use judgment to determine whether it is appropriate to use the model results, making any compensating adjustments as necessary.

Section Reliance on Model Evaluation by Another Actuary F An actuary may rely on another actuary who followed this standard F Satisfied that other actuary’s evaluation performed in accordance and is appropriate for intended purpose F Disclose such reliance

HURRICANE MODELS WHY ARE THEIR RESULTS DIFFERENT?

MODEL COMPONENTS PRIMARY COMPONENTS w Meteorology w Vulnerability w Actuarial SECONDARY COMPONENTS w Statistical w Computer

METEOROLOGICAL COMPONENTS w Hurricane Frequencies geographical frequencies frequency by magnitude directional frequencies w Hurricane Tracks initial position initial direction directional movements

METEOROLOGICAL COMPONENTS - CONTINUED w Hurricane Characteristics central pressure wind speed forward velocity radius of maximum winds radius of hurricane force winds far field pressure other characteristics –Rankine vortex –Beta parameter

METEOROLOGICAL COMPONENTS - CONTINUED Surface water temperature Geographical impacts Distance from landfall Surface roughness topography buildings trees

VULNERABILITY CHARACTERISTICS w Data used to derive damage functions Insurance claim data Post hurricane inspections Wind tunnel tests Engineering judgment w Data used to verify damage functions Insurance claim data Post hurricane inspections

VULNERABILITY CHARACTERISTICS - CONTINUED w Adjustments to damage functions New building codes Mitigation measures Individual structural characteristics Roof type Cladding type Structure height Many others

ACTUARIAL COMPONENTS w Policy provisions Policy limits Deductibles Coinsurance w Insurer practices What constitutes a covered claim? What constitutes a total loss?

STATISTICAL TESTS w VERIFICATION w statistically reasonable results? w UNCERTAINTY w Causes of the variations? w SENSITIVITY w How does each component impact result?

STATISTICAL TESTS - CONTINUED w Verification of reasonable results Chi-square goodness-of-fit test Kolmogorov-Smirnov test Cramer-von Mises test w Sensitivity and Uncertainty Latin Hypercube Sampling

SO WHY DO MODEL RESULTS DIFFER? w Incomplete knowledge w Myriad of variables w Very short historical period w Constantly changing environment Structures Population shifts w Policy variations w Insurer practice variations