1. Predictive Modeling Spring 2005 CAMAR meeting Louise Francis, FCAS, MAAA Francis Analytics and Actuarial Data Mining, Inc www.data-mines.com

2. 2 Objectives Introduce Predictive modeling Why use it? Describe some methods in depth Trees Neural networks Clustering Apply to fraud data

3. 3 Predictive Modeling Family Predictive Modeling Classical Linear Models GLMsData Mining

4. 4 Why Predictive Modeling? Better use of insurance data Advanced methods for dealing with messy data now available

5. 5 Major Kinds of Modeling Supervised learning Most common situation A dependent variable Frequency Loss ratio Fraud/no fraud Some methods Regression CART Some neural networks Unsupervised learning No dependent variable Group like records together A group of claims with similar characteristics might be more likely to be fraudulent Some methods Association rules K-means clustering Kohonen neural networks

6. 6 Two Big Specialties in Predicative Modeling Data Mining Trees Neural Networks Clustering

7. 7 Modeling Process Internal Data Data Cleaning External Data Other Preprocessing Build ModelValidate ModelTest Model Deploy Model

8. 8 Data Complexities Affecting Insurance Data Nonlinear functions Interactions Missing Data Correlations Non normal data

9. 9 Kinds of Applications Classification Prediction

10. 10 The Fraud Study Data 1993 Automobile Insurers Bureau closed Personal Injury Protection claims Dependent Variables Suspicion Score Number from 0 to 10 Expert assessment of liklihood of fraud or abuse 5 categories Used to create a binary indicator Predictor Variables Red flag indicators Claim file variables

11. 11 Introduction of Two Methods Trees Sometimes known as CART (Classification and Regression Trees) Neural Networks Will introduce backpropagation neural network

12. 12 Decision Trees Recursively partitions the data Often sequentially bifurcates the data – but can split into more groups Applies goodness of fit to select best partition at each step Selects the partition which results in largest improvement to goodness of fit statistic

13. 13 Goodness of Fit Statistics Chi Square  CHAID (Fish, Gallagher, Monroe- Discussion Paper Program, 1990) Deviance  CART

14. 14 Goodness of Fit Statistics Gini Measure  CART i is impurity measure

15. 15 Goodness of Fit Statistics Entropy  C4.5

16. 16 An Illustration from Fraud data: GINI Measure

17. 17 First Split All Claims p(fraud) = 0.36 Legal Rep = Yes P(fraud) = 0.612 Legal Rep = No P(fraud) = 0.113

18. 18 Example cont:

19. 19 Example of Nonlinear Function Suspicion Score vs. 1 st Provider Bill

20. 20 An Approach to Nonlinear Functions: Fit A Tree

21. 21 Fitted Curve From Tree

22. 22 Neural Networks Developed by artificial intelligence experts – but now used by statisticians also Based on how neurons function in brain

23. 23 Neural Networks Fit by minimizing squared deviation between fitted and actual values Can be viewed as a non-parametric, non- linear regression Often thought of as a “black box” Due to complexity of fitted model it is difficult to understand relationship between dependent and predictor variables

24. 24 The Backpropagation Neural Network

25. 25 Neural Network Fits a nonlinear function at each node of each layer

26. 26 The Logistic Function

27. 27 Universal Function Approximator The backpropagation neural network with one hidden layer is a universal function approximator Theoretically, with a sufficient number of nodes in the hidden layer, any continuous nonlinear function can be approximated

28. 28 Nonlinear Function Fit by Neural Network

29. 29 Interactions Functional relationship between a predictor variable and a dependent variable depends on the value of another variable(s)

30. 30 Interactions Neural Networks The hidden nodes pay a key role in modeling the interactions CART partitions the data Partitions capture the interactions

31. 31 Simple Tree of Injury and Provider Bill

32. 32

33. 33 Missing Data Occurs frequently in insurance data There are some sophisticated methods for addressing this (i.e., EM algorithm) CART finds surrogates for variables with missing values Neural Networks have no explicit procedure for missing values

34. 34 More Complex Example Dependent variable: Expert’s assessment of liklihood claim is legitimate A classification application Predictor variables: Combination of claim file variables (age of claimant, legal representation) red flag variables (injury is strain/sprain only, claimant has history of previous claim) Used an enhancement on CART known as boosting

35. 35 Red Flag Predictor Variables

36. 36 Claim File Variables

37. 37 Neural Network Measure of Variable Importance Look at weights to hidden layer Compute sensitivities: a measure of how much the predicted value’s error increases when the variables are excluded from the model one at a time

38. 38 Variable Importance

39. 39 Testing: Hold Out Part of Sample Fit model on 1/2 to 2/3 of data Test fit of model on remaining data Need a large sample

40. 40 Testing: Cross-Validation Hold out 1/n (say 1/10) of data Fit model to remaining data Test on portion of sample held out Do this n (say 10) times and average the results Used for moderate sample sizes Jacknifing similar to cross-validation

41. 41 Results of Classification on Test Data

42. 42 Unsupervised Learning Common Method: Clustering No dependent variable – records are grouped into classes with similar values on the variable Start with a measure of similarity or dissimilarity Maximize dissimilarity between members of different clusters

43. 43 Dissimilarity (Distance) Measure Euclidian Distance Manhattan Distance

44. 44 Binary Variables

45. 45 Binary Variables Sample Matching Rogers and Tanimoto

46. 46 Results for 2 Clusters

47. 47 Beginners Library Berry, Michael J. A., and Linoff, Gordon, Data Mining Techniques, John Wiley and Sons, 1997 Kaufman, Leonard and Rousseeuw, Peter, Finding Groups in Data, John Wiley and Sons, 1990 Smith, Murry, Neural Networks for Statistical Modeling, International Thompson Computer Press, 1996

48. Data Mining CAMAR Spring Meeting Louise Francis, FCAS, MAAA Louise_francis@msn.com www.data-mines.com