2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO

2 Accomplishing Business Goals Project Management Implementation Future

3 Project Management Determine business processes that support strategic goals – Underwriting decisions – Pricing decisions Develop project plan aligned with strategic goals – Model Building – Technology Development – Implementation Phases Determine project needs Monitor actual vs. planned costs/milestones

4 Project Needs Team Skills – Data management – Analytical/statistical – Technology – Business Knowledge Data Statistical Tools Computer Capacity

5 Prior to Modeling Formulate the Problem Evaluate Possible Data Sources Prepare the Data Explore the Data with Simple Modeling Techniques

6  25%  50%  75%  85% What percent of a model building project is the data preparation and data management?

7 Prepare the Data Do quality checks in level of detail needed for project Understand how to prepare individual variables for use in models Need to be practical about number of classification categories models can handle Need to decide on truncation and bucketing of variables that are continuous Create new variables

8 Data Management Issues Matching additional internal policy information to premium/loss data – Different points in time – Tracking & balancing audited exposures Different summarization keys – handling of mid-term endorsements Address scrubbing Matching to external data for correct point in time Significance of missing values within variable

9 Modeling Procedures and Diagnostics Basic modeling training – GLM, Data Mining Decide on appropriate diagnostics Evaluate diagnostics

10 Modeling Process Business Knowledge Data Linking Data Cleansing Analyze Variables Determine Predictive Variables Evaluation Data Gathering Modeling

11 Business Questions What goals are you trying to achieve? What results do you expect to see? How will you know if the results are reasonable? How do you ensure sufficient knowledge transfer to business staff?

12 Model Performance

13 Model Input/Output Model input considerations – Access to data – Robustness/quality of data – Timeliness of refreshed data Design Model output for users – Definition of output – expected loss ratio, pure premium, loss ratio relativity? – Provide support for output – reason codes

14 Business Implementation of Model Model usage determined by strategic goals – Underwriting risk decision – Pricing of risks – Support of market growth Integration of Model into business workflow decisions – Consistency in underwriting/pricing decisions – Compliance with regulations based on implementation decisions

15 Implementation of Model Workflows: Underwriting – New Business – Renewal business Rating – Pricing – Coverage Adjustment

16 Implementation of Model New Business decision options – Write risk – Request additional info on risk – Decline risk – Adjust price/coverage Consider model output alone or along with other information available from application Model output needed within seconds for quick decision

17 Implementation of Model Renewal decision options – Automatic renewal – Flag for non-renewal – Adjust coverage level for risk – Adjust pricing for risk Initial Year – review all in-force policies on weekly or monthly basis Subsequent years – establish schedule for reevaluation based on specific underwriting guidelines

18 Implementation of Model Rating Model O/P represents relative loss ratio factor Determine rating selections Determine rating process – Modify application of IRPM plan – Implement new rating factors based on Model – Tier risks into different insurers within insurer group

19 Technology Development Incorporate business implementation decisions Decide on how Model will be accessible electronically – Web-based interface – Integrated into existing workflow – Batch processing Develop/Modify Systems – Phase-in technology Model uses information from a third-party vendor Determine I/P and O/P criteria

20 Rollout Implementation of Model Prepare Announcement/Training Material for Internal & External Customers Coordinate Implementation Phases Monitor Feedback/Adjust Implementation Monitor Results against Strategic Goals

21 Future of Predictive Modeling More refined rating plans – Industry-sourced or internally developed – Combination of internally-developed & industry- sourced risk component variables Ongoing updating and maintenance of Models – Refresh data – New data sources/variables – New tools/techniques – React to new market environments