1. Pour insérer une image : Menu « Insertion / Image » ou Cliquer sur l’icône de la zone image Pour personnaliser « nom événement et auteur » : « Insertion / En-tête et pied de page » Personnaliser la zone de de pied de page Cliquer sur appliquer partout FUZZY INFERENCE SYSTEM AND LEARNING 08 JULY 2014 DataSense Digicosme | CORNEZ Laurence

2. PLAN | PAGE 2 DataSense Digicosme | CORNEZ Laurence I.Brief introduction on Fuzzy Logic and Fuzzy inference system (FIS) II.Real context and database III.Fuzzy rules and Sugeno’s classifier IV.Implementation in three steps VI.Visualization of FIS obtained VII.Perspectives in terms of intelligibility and performance

3. Fuzzy logic and applications 1965, Zadeh proposes fuzzy concept : one object can be simultaneously in two different classes This allows the imperfections (natural language), imprecisions and uncertainties (data) Applications (since 1974): washing machine, ABS, autofocus camera…) | PAGE 3 DataSense Digicosme | CORNEZ Laurence

4. Fuzzy expert systems Goal: three parts to reproduce the cognitive reasoning of an expert: Rules base: -expression of the knowledge of the expert through « If-then » inference rules. -directly expressed by expert or learnt via databases Inputs Inference engin able to integrate these rules and these inputs to produce specific outputs. | PAGE 4 DataSense Digicosme | CORNEZ Laurence Inference engin inputs outputs

5. Fuzzy inference system: example Rules base (Jang97) IF temperature=low THEN cooling valve=half open IF temperature=medium THEN cooling valve=almost open input moteur d’inférence 18° low 1 0 half open 1 0 1 0 medium 1 0 almost open T° d% 0,2 0,5 1 0 d% Implementation DataSense Digicosme | CORNEZ Laurence | PAGE 5 70%

6. WORK POSITION / DEFINITION DataSense Digicosme | CORNEZ Laurence | PAGE 6

7. Seismicity map (+/- France) | PAGE 7 DataSense Digicosme | CORNEZ Laurence Earthquakes Marine explosions Quarry blasts Rock bursts How to class a new event automatically with good interpretability for the expert ?

8. Database stutied  French seismic metropolitan data from 1997 and 2003  Inputs (high level features):  Hour : circular variable [0;24]  Latitude : quantitative variable [42;51]  Longitude : quantitative variable [-5;9]  Magnitude : quantitative variable [0.7;6.0]  Date : qualitative variable with 3 modalities {Working day, Saturday, Sunday and bank holiday}  Classification output (3 possible classes):  Earthquakes (9349 events)  Quarry blasts (3485 events)  Rock bursts (1075 events) | PAGE 8 DataSense Digicosme | CORNEZ Laurence

9. Model proposed | PAGE 9 DataSense Digicosme | CORNEZ Laurence  Aggregation of rules (Sugeno order 0) - If magnitude is middle and event is nocturnal then event is earthquake - If magnitude is high then event is surely earthquake How to generate these rules automatically ? An example of the input space Weight of the rule k Membership degree of x to the rule k Issue of the rule k (unit vector)  Sugeno’s classifier (normalized) defined as:

10. MODEL IMPLEMENTATION DataSense Digicosme | CORNEZ Laurence | PAGE 10

11. Model implementation: first step (1/2) | PAGE 11 DataSense Digicosme | CORNEZ Laurence  Soft Clustering = modelling class density by gaussian mixture  Mountain clustering (Chiu 94) The algorithm learns : Gaussian number Location of gaussian centers magnitude hour

12. Model implementation: first step (2/2) | PAGE 12 DataSense Digicosme | CORNEZ Laurence  Results:  Good classification rate with « winner takes all » method  5-fold cross-validation databases MethodLearning rate (%)Test rate (%)Cluster number 5 quantitatives (quali. va. misused) 85.40 +/- 0.8784.84 +/- 1.4732 ; 36 ; 35 ; 38 ; 37 4 quantitatives85.66 +/- 0.8584.65 +/- 1.7625 ; 26 ; 28 ; 28 ; 29 What about the qualitative variable ? Similar good classification rates Less clusters

13. Model implementation: second step (1/2) | PAGE 13 DataSense Digicosme | CORNEZ Laurence Probability estimations of each modality for each cluster With: Associated Sugeno’s classifier

14. Model implementation: second step (2/2) | PAGE 14 DataSense Digicosme | CORNEZ Laurence MethodLearning rate (%)Test rate (%) 4 quanti. + 1 quali.86.99 +/- 0.4886.02 +/- 0.91  Results after step II: Good classification rates not significantly improved Well classified point Ill classified point One cluster Semi optimal (cluster juxtapositions) Not optimal (absence of clusters)

15. Model implementation: third step (1/2) | PAGE 15 DataSense Digicosme | CORNEZ Laurence  Improvement of parameters with EM « Expectation- Maximization » (Jordan et Jacobs 1993)  Input space is virtually augmented by adding a hidden variable, the cluster of interest  EM garantees improvement after each step  Computation of new parameters: ={weights, centers and standard deviations }

16. Model implementation: third step (2/2) | PAGE 16 DataSense Digicosme | CORNEZ Laurence  Results after step III:  50 iterations for EM  The same 5-fold cross-validation database MethodLearning rate (%)Test rate (%) 4 quant.+ 1 qual.93.67+/- 0.6093.12 +/- 1.66 Improvement for good classification rate significant improvement for cluster locations

17. Visualization | PAGE 17 DataSense Digicosme | CORNEZ Laurence X sum X X X Estim. Proba GaussiansRule output Weight Product One rule One example Class of the example Classe decided EQ [ 92.70% 0.00% 7.32%]

18. PERSPECTIVES IN TERMS IN INTELLIGIBILITY AND PERFORMANCE DataSense Digicosme | CORNEZ Laurence | PAGE 18

19. FIS 95,19% DT 94,88% Fuzzy DT 95,19% Comparison with previous works | PAGE 19 DataSense Digicosme | CORNEZ Laurence 1998 : S. Muller fuzzy Controller codage MLP 92.5% well classified 1999 : F. Gravot FIS - Mixture of gausians - Gradient-based descent 90,5% well classified 2005 : R. Quach et D. Mercier fuzzy controller codage MLP : 95,9% well classified SVM : 96,5% well classified Intelligibility Performance cNF+RN 92,5% cNF+MLP/SVM ~96% FIS 90,5% 2006 : L. Cornez DT 94,88% well classified 95,19% well classified 2007 : L. Cornez FIS 3 steps 95,19% well classified Objective

20. How improve intelligibility ? | PAGE 20 DataSense Digicosme | CORNEZ Laurence 20 According to fold cross validation database, the coverage is different

21. Improve intelligibility and stability | PAGE 21 DataSense Digicosme | CORNEZ Laurence  More the model is stable and more the model fit with cognitive representation more the expert can accept it  Generative Gaussian Graph (M. Aupetit) to identify complex clusters

22. CEA Tech Département Métrologie, Instrumentation et Information Laboratoire d’Analyse de Données et Intelligence des Systèmes Commissariat à l’énergie atomique et aux énergies alternatives Institut Carnot CEA LIST Centre de Saclay | 91191 Gif-sur-Yvette Cedex T. +33 (0)1 69 08 18 00 Etablissement public à caractère industriel et commercial | RCS Paris B 775 685 019 | PAGE 22 DataSense Digicosme | CORNEZ Laurence THANKS ! QUESTIONS ?

23. Best fuzzy decision tree | PAGE 23 DataSense Digicosme | CORNEZ Laurence