Presentation is loading. Please wait.

Presentation is loading. Please wait.

Re-thinking Modelling: a Call for the Use of Data Mining in Data-driven Social Simulation Samer Hassan Javier Arroyo Celia Guti é rrez Universidad Complutense.

Published byNickolas Perry Modified over 9 years ago

Similar presentations

Presentation on theme: "Re-thinking Modelling: a Call for the Use of Data Mining in Data-driven Social Simulation Samer Hassan Javier Arroyo Celia Guti é rrez Universidad Complutense."— Presentation transcript:

1 Re-thinking Modelling: a Call for the Use of Data Mining in Data-driven Social Simulation Samer Hassan Javier Arroyo Celia Guti é rrez Universidad Complutense de Madrid

2 Samer Hassan SS@IJCAI 2009 2 Contents Data-driven ABM DM-assisted Methodology Case Study: Mentat Application Conclusions

3 Samer Hassan SS@IJCAI 2009 3 Research Aim

4 Samer Hassan SS@IJCAI 2009 4 Research Aim Theoretical KISS Structural Validation Abstract General

5 Samer Hassan SS@IJCAI 2009 5 Research Aim Data-driven Non-KISS Empirical Validation Specific (case study) Expressive Theoretical KISS Structural Validation Abstract General

6 Samer Hassan SS@IJCAI 2009 6 Classical Logic of Simulation

7 Samer Hassan SS@IJCAI 2009 7 Data-Driven Logic

8 Samer Hassan SS@IJCAI 2009 8 Data-driven Approach Complexity Large amounts of Data Auxiliary AI: Fuzzy Logic Ontologies Evolutionary Computation Data Mining

9 Samer Hassan SS@IJCAI 2009 9 Data Mining Extracting patterns and relevant information from large amounts of data Pre-processing of empirical data Cluster finding Discovery of hidden patterns Locates redundancies Post-processing of simulation output Clustering: Discovery of hidden patterns Validation of clusters Locates inconsistencies Classification Cluster matching

10 Samer Hassan SS@IJCAI 2009 10 Contents Data-driven ABM DM-assisted Methodology Case Study: Mentat Application Conclusions

11 Samer Hassan SS@IJCAI 2009 11 Methodology for DM-assisted ABM

12 Samer Hassan SS@IJCAI 2009 12 Methodology for DM-assisted ABM Data Collection Initial point Validation points Necessarily ≠ initial Type Explicit Externalised Empirical distributions Secondary sources Methods Quantitative E.g. surveys Qualitative E.g. interviews

13 Samer Hassan SS@IJCAI 2009 13 Methodology for DM-assisted ABM Analysis Preprocessing of empirical data Roles Domain expert Guide DM exploration Interpretation DM expert Confirm or refine theories

14 Samer Hassan SS@IJCAI 2009 14 Methodology for DM-assisted ABM Selection of Relevant Data Filtering Adaptation of data Normalisation Discretisation Domain Expert Theory DM Redundancies Overlooked independent variables

15 Samer Hassan SS@IJCAI 2009 15 Methodology for DM-assisted ABM Data Analysis Large data collections Guided by theory Types Cluster analysis Principal Component Analysis Time series methods Association rules

16 Samer Hassan SS@IJCAI 2009 16 Methodology for DM-assisted ABM Interpretation of results Theory expert Relate results to theory New findings are added to the findings base

17 Samer Hassan SS@IJCAI 2009 17 Methodology for DM-assisted ABM ABM Building Based on Findings Modeller Steps Formalisation Data-driven Design Implementation Initialisation

18 Samer Hassan SS@IJCAI 2009 18 Methodology for DM-assisted ABM Simulation Fine tuning the ABM Sensitivity analysis Intensive testing Output Record agent trace

19 Samer Hassan SS@IJCAI 2009 19 Methodology for DM-assisted ABM Validation Analysis of the results Empirical validation Theoretical consistency Roles DM expert Analyse the data Domain expert Extract conclusions Iterative cycle

20 Samer Hassan SS@IJCAI 2009 20 Contents Data-driven ABM DM-assisted Methodology Case Study: Mentat Application Conclusions

21 Samer Hassan SS@IJCAI 2009 21 The Problem Aim: simulate the process of change in social values in a period in a society Plenty of factors involved Inertia of generational change: To which extent the demographic dynamics explain the mental change? Inter-generational: Agent characteristics remain constant Macro aggregation evolves

22 Samer Hassan SS@IJCAI 2009 22 Mentat: architecture Agent : Mental State attributes Life cycle patterns Demographic micro-evolution: Couples Reproduction Inheritance

23 Samer Hassan SS@IJCAI 2009 23 Mentat: architecture World: 3000 agents Grid 100x100 Demographic model 8 indep. parameters Social Network: Communication with Moore Neighbourhood Friends network Family network

24 Samer Hassan SS@IJCAI 2009 24 Contents Data-driven ABM DM-assisted Methodology Case Study: Mentat Application Conclusions

25 Samer Hassan SS@IJCAI 2009 25 Data Collection in Mentat Initial data: EVS-1980 Representative sample of Spain Qualitative info Empirically-grounded demographic equations Validation data: EVS-1990 EVS-1999

26 Samer Hassan SS@IJCAI 2009 26 Analysis in Mentat Selection of relevant data EVS-1980,1990,1999 Options: 1.Algorithm for the best subset of variables 2.Rely on domain expert Tested domain knowledge (2) chosen Variables adaptation Normalisation NameTypeRange gendercategorical agenumeric≥18 studiesnumeric≥5 civil statecategorical economynumericreal ideologyordinal1-10 conf. churchordinal1-4 church att.Ordinal1-7 relig. personcategorical

27 Samer Hassan SS@IJCAI 2009 27 Analysis in Mentat Data Analysis Algorithm selection Wrapped k-means Explore different k (# of clusters) Discarded variables Gender & Age provokes appearance of irrelevant clusters E.g. widowed women Economy is redundant High correlation with Education

28 Samer Hassan SS@IJCAI 2009 28 Analysis in Mentat Interpretation Sociological research Religious typology (RLGTYPE) Based on 3 variables Ecclesiastical, low-intensity, alternatives & non-religious Clusters found (1980, 1999) Based on the 9-3=6 variables 5 clusters with sociological meaning Consistent with RLGTYPE Theoretical observations of the pattern evolution: Religiosity strength falls Ideological spectrum twists to the left education & economy Newest type of religiosity, “alternatives” rise youngsters

29 Samer Hassan SS@IJCAI 2009 29 Analysis in Mentat

30 Samer Hassan SS@IJCAI 2009 30 Validation in Mentat Mentat re-building & simulation explored Mentat output clusterised Same 5 clusters found Similar evolution trends 3 theoretical observations shown Inconsistencies detected Liberal cluster % do not match although aggregated they do Graphics show less youngsters Liberal clusters deeply affected Guide to re-design

31 Samer Hassan SS@IJCAI 2009 31 Contents Data-driven ABM DM-assisted Methodology Case Study: Mentat Application Conclusions

32 Samer Hassan SS@IJCAI 2009 32 Conclusions DM-assisted ABM methodology Suitable for DDABM Complexity Large amounts of data Limitations KISS Qualitative sources Uses Build new ABM Re-thinking existing DDABM Revealing hidden facts Detect inconsistencies

33 Samer Hassan SS@IJCAI 2009 33 Thanks for your attention! Samer Hassan samer@fdi.ucm.es Universidad Complutense de Madrid

34 Samer Hassan SS@IJCAI 2009 34 Contents License This presentation is licensed under a Creative Commons Attribution 3.0 http://creativecommons.org/licenses/by/3.0/ You are free to copy, modify and distribute it as long as the original work and author are cited

Download ppt "Re-thinking Modelling: a Call for the Use of Data Mining in Data-driven Social Simulation Samer Hassan Javier Arroyo Celia Guti é rrez Universidad Complutense."

Similar presentations

The Robert Gordon University School of Engineering Dr. Mohamed Amish

The Robert Gordon University School of Engineering Dr. Mohamed Amish
Agent-Based Social Modelling and Simulation with Fuzzy Sets Samer Hassan Collado Luis Garmendia Salvador Juan Pavón Mestras ESSA 2007 Dep. Ingeniería del.

Agent-Based Social Modelling and Simulation with Fuzzy Sets Samer Hassan Collado Luis Garmendia Salvador Juan Pavón Mestras ESSA 2007 Dep. Ingeniería del.
Understanding the Research Process

Understanding the Research Process
Data-Driven Agent-Based Social Simulation of Moral Values Evolution Samer Hassan Universidad Complutense de Madrid University of Surrey.

Data-Driven Agent-Based Social Simulation of Moral Values Evolution Samer Hassan Universidad Complutense de Madrid University of Surrey.
Huge Raw Data Cleaning Data Condensation Dimensionality Reduction Data Wrapping/ Description Machine Learning Classification Clustering Rule Generation.

Huge Raw Data Cleaning Data Condensation Dimensionality Reduction Data Wrapping/ Description Machine Learning Classification Clustering Rule Generation.
Integrating Bayesian Networks and Simpson’s Paradox in Data Mining Alex Freitas University of Kent Ken McGarry University of Sunderland.

Integrating Bayesian Networks and Simpson’s Paradox in Data Mining Alex Freitas University of Kent Ken McGarry University of Sunderland.
Erasmus University Rotterdam Frederik HogenboomEconometric Institute School of Economics Flavius Frasincar.

Erasmus University Rotterdam Frederik HogenboomEconometric Institute School of Economics Flavius Frasincar.
Friends Forever: Social Relationships with a Fuzzy Agent-Based Model Samer Hassan Mauricio Salgado Juan Pav ó n Universidad Complutense de Madrid University.

Friends Forever: Social Relationships with a Fuzzy Agent-Based Model Samer Hassan Mauricio Salgado Juan Pav ó n Universidad Complutense de Madrid University.
Civil and Environmental Engineering Carnegie Mellon University Sensors & Knowledge Discovery (a.k.a. Data Mining) H. Scott Matthews April 14, 2003.

Civil and Environmental Engineering Carnegie Mellon University Sensors & Knowledge Discovery (a.k.a. Data Mining) H. Scott Matthews April 14, 2003.
1 ACCTG 6910 Building Enterprise & Business Intelligence Systems (e.bis) Introduction to Data Mining Olivia R. Liu Sheng, Ph.D. Emma Eccles Jones Presidential.

1 ACCTG 6910 Building Enterprise & Business Intelligence Systems (e.bis) Introduction to Data Mining Olivia R. Liu Sheng, Ph.D. Emma Eccles Jones Presidential.
An Introduction to METHODOLOGY in Social Sciences.

An Introduction to METHODOLOGY in Social Sciences.
Video Mining Learning Patterns of Behaviour via an Intelligent Image Analysis System.

Video Mining Learning Patterns of Behaviour via an Intelligent Image Analysis System.
Verification and Validation

Verification and Validation
Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Data mining and the knowledge discovery process Summer Course 2005.

Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Data mining and the knowledge discovery process Summer Course 2005.
GUHA method in Data Mining Esko Turunen Tampere University of Technology Tampere, Finland.

GUHA method in Data Mining Esko Turunen Tampere University of Technology Tampere, Finland.
Enterprise systems infrastructure and architecture DT211 4

Enterprise systems infrastructure and architecture DT211 4
LÊ QU Ố C HUY ID: QLU13069 1. OUTLINE  What is data mining ?  Major issues in data mining 2.

LÊ QU Ố C HUY ID: QLU OUTLINE  What is data mining ?  Major issues in data mining 2.
Overview of Distributed Data Mining Xiaoling Wang March 11, 2003.

Overview of Distributed Data Mining Xiaoling Wang March 11, 2003.
The design process z Software engineering and the design process for interactive systems z Standards and guidelines as design rules z Usability engineering.

The design process z Software engineering and the design process for interactive systems z Standards and guidelines as design rules z Usability engineering.
Data Mining Chun-Hung Chou

Data Mining Chun-Hung Chou

Similar presentations

Ads by Google