1. FAKE GAME updates Pavel Kordík kordikp@fel.cvut.cz

2. 2/67 FAKE GAME concept

3. 3/67 Automated data preprocessing For each feature, optimal sequence of preprocessing methods is evolved…

4. 4/67 Preprocessing methods implemented in FAKE GAME

5. 5/67 Evolving preprocessing sequences

6. 6/67 More on automated preprocessing

7. 7/67 FAKE GAME concept

8. 8/67 Automated Data Mining The GAME engine – automated evolution of models

9. 9/67 Example: Housing data CRIM ZN INDUS NOX RM AGE DIS RAD TAX PTRATIO B LSTA MEDV Per capita crime rate by town Proportion of owner-occupied units built prior to 1940 Weighted distances to five Boston employment centers Input variables Output variable Median value of owner-occupied homes in $1000's

10. 10/67 Housing data – records CRIM ZN INDUS NOX RM AGE DIS RAD TAX PTRATIO B LSTA MEDV Input variables Output variable 24 0.00632 18 2.31 53.8 6.575 65.2 4.09 1 296 15.3 396.94.98 21.6 0.02731 0 7.07 46.9 6.421 78.9 4.9671 2 242 17.8 396.99.14 … … … A B C A = Training set … to adjust weights and coefficients of neurons B = Validation set … to select neurons with the best generalization C = Test set … not used during training

11. 11/67 Housing data – inductive model CRIM ZN INDUS NOX RM AGE DIS RAD TAX PTRATIO B LSTA MEDV Input variables Output variable sigmoidgauss ? sigmoidexplinear … …… MEDV=a 1 *PTRATIO+ a 0 MEDV=1/(1-exp(-a 1 *CRIM+ a 0 ))

12. 12/67 Housing data – inductive model CRIM ZN INDUS NOX RM AGE DIS RAD TAX PTRATIO B LSTA MEDV Input variables Output variable sigmoid Validation error: 0.13Validation error: 0.21 MEDV=1/(1-exp(-5.724*CRIM+ 1.126))MEDV=1/(1-exp(-5.861*AGE+ 2.111)) ?

13. 13/67 Housing data – inductive model CRIM ZN INDUS NOX RM AGE DIS RAD TAX PTRATIO B LSTA MEDV Input variables Output variable sigmoid Error: 0.13 Error: 0.21 sigmoid Error: 0.26 linear Error: 0.24 polyno mial Error: 0.10 MEDV=0.747*(1/(1-exp(-5.724*CRIM+ 1.126))) +0.582*(1/(1-exp(-5.861*AGE+ 2.111))) 2 +0.016

14. 14/67 Housing data – inductive model CRIM ZN INDUS NOX RM AGE DIS RAD TAX PTRATIO B LSTA MEDV Input variables Output variable sigmoid linear polyno mial polyno mial linear expo nential Validation error: 0.08

15. 15/67 Optimization of coefficients (learning) x 1 x n x 2...    0 1 1 2 2 1 2 * 1aeay n n i ii a ax n         Gaussian(GaussianNeuron) We have inputs x 1, x 2, …, x n and target output y in the training data set We are looking for optimal values of coefficients a 0, a 1, …, a n+2 y’ The difference between unit output y’ and the target value y should be minimal for all vectors from the training data set

16. 16/67 Which optimization method is the best? It depends on –dataset –type of unit –location of unit –configuration of method Use them all!

17. 17/67 Remember the Genetic algorithm optimizing the structure of GAME?

18. 18/67 More on optimization …

19. 19/67 Experimental results supporting “use them all” strategy Atrial fibrillation (AF) is the most common clinically significant arrhythmia … number of affected patients is expected to rise to 3.3 million by 2020 and 5.6 million in 2050

20. 20/67 Input features Number of inflection points in particular A-EGM signal (IP). Mean value of number of inflexion points in the found SCs in particular A-EGM signal (MIPSC). Variance of number of inflexion points in the found SCs in particular A-EGM signal (VIPSC). Mean value of width of found SCs in particular A-EGM signal (MWSC). Number of inflection points in found SCs in particular A-EGM signal (IPSC). IPSC normalized per number of found SCs in particular A-EGM signal (NIPSC). IP + MIPSC (IPMIPSC). pIPSC2 + TDM2 (IPSCTDM). Number of zero-level crossing points in found SCs in particular A-EGM signal (ZCP). Maximum of IPSC in particular A-EGM signal (MIPSC). Time domain method (see below) with rough (unfiltered) input A-EGM signal (TDM). 12. Time domain method using input A-EGM signal filtered by above described wavelet filter (FTDM).

21. 21/67 Output 3 experts manually annotated the signal (Class I, II, III, IV).

22. 22/67 Prepared Data Avg Cls – average of experts ranking (regression) Class I,II,III,IV – majority ranking (classification) x36x37x39x40Avg ClsClass IClass IIClass IIIClass IV 0.0071570.0039040.114460.06558611000 0.0123620.0039040.167790.0655861.31000 0.0149640.0078070.195090.06172811000 0.0162650.0097590.215360.07484611000 0.0175670.0091090.224950.0617281.31000 0.0156150.0039040.21940.0974551.71000 0.010410.0045540.146860.06558611000 0.0084580.0052050.132230.07330211000 0.0097590.0045540.147450.07749811000

23. 23/67 Experimental Setup Regression, Classification 10 fold crossvalidation not enough to get statistically significant results - repeated 10 times Each boxplot – average RMS or Classif. Accuracy from 100 models Compared with WEKA methods

24. 24/67 Regression – GAME RMSE GAME configurations Conclusions: ensemble is more accurate and stable linear regression sufficient all performs slightly worse Configurations: lin – linear units only, QN std – subset of unit types, QN quick – std with 5 epochs only all – all units, all methods ens3 postfix – ensemble of 3 models

25. 25/67 Comparison with WEKA Prefix w - WEKA Conclusions: w-RBFN fails in spite of tunning linear regression best game-all3 not bad RMSE GAME configurations

26. 26/67 Classification – GAME Classification accuracy [%] GAME configurations lin – fails ens3 – good enough all-ens best

27. 27/67 Comparison with WEKA Conclusions: GAME slightly better than decesion tree j48

28. 28/67 It takes a lot of time Do it in parallel, employ all cores

29. 29/67 More on distributed computing

30. 30/67 Parallel threads synchronized by join()

31. 31/67 Experiment

32. 32/67 Speedup for two cores

33. 33/67 Speedup for 8 cores

34. 34/67 Speedup limitations For infinite number of cores, the speedup will be limited to 5.82 ! The speedup for N cores according to Amdahl’s law: The speedup for 2 cores is 1.7, for 8 cores just 3.5 … P

35. 35/67 FAKE GAME concept

36. 36/67 Information that might be useful Extracted from data –Data statistics – hist,…plots, matrices, feature ranking, etc. Extracted from automated preprocessing –Outliers –Data transformations –Data reduction Extracted from models –Accuracy, 10xcv –2d structure – type of units, opt methods –Formulas –3D structure of model - behavior –Feature ranking –Behavior of models –Credibility of prediction/classification

37. 37/67

38. 38/67

39. 39/67

40. 40/67

41. 41/67

42. 42/67 Data projections 2D 3D

43. 43/67 Feature ranking

44. 44/67 More on feature ranking

45. 45/67 Model structure …

46. 46/67

47. 47/67 Model structure, behavior of units

48. 48/67 Play FAKE GAME with your data

49. 49/67 Log messages Evolving preprocessing sequences … Evolving ensemble of inductive models … Evolving “interesting” visualizations … Generating report … Done … in 2009