15 June 2010ABM Workshop -Leeds Salem Adra and Phil McMinn Automated Discovery of Emergent Misbehaviour in Agent-Based Models

15 June 2010ABM Workshop -Leeds Outline Background: Agent-Based modelling Agent-Based modelling Applications Testing Agent-Based Models: Motivations and Challenges Automated Discovery of Emergent Misbehaviour: Our Vision Testing Strategies

15 June 2010ABM Workshop -Leeds Background What is Agent-Based computational modelling? Increasingly popular modelling paradigm Main Idea: Simulates the interactions of autonomous entities (agents) with each other and their local environment to predict higher level emergent patterns Bottom-up procedure The focus goes into the individual interacting units Benefits: Deeper understanding of the interacting agents and their emergent behaviour

15 June 2010ABM Workshop -Leeds Background Example of some Agent-Based Modelling Frameworks: FLAME (C Based) MASON (Java Based) RePast (Java Based) Swarm (Objective C)

15 June 2010ABM Workshop -Leeds Background Agent-Based Models Examples The Epitheliome Project: (Part of the Physiome Project, a worldwide public domain effort to provide a computational framework for understanding human physiology) Goal: Modelling the Emergent Behaviour of Epithelial Cells Benefits: Better Understanding of Cell Proliferation and self-organisation Capability to explore the roles of a single parameter in a complex biological system Helps in the development of reconstruction methods for artificial human skin

15 June 2010ABM Workshop -Leeds Background Prediction and validation about Epithelial scratch wound healing process Agent-Based Models Examples

15 June 2010ABM Workshop -Leeds Background Modelling of cardiac cells and tissues Agent-Based Models Examples

15 June 2010ABM Workshop -Leeds Background Macroeconomic Policy Modelling Agent-Based Models Examples

15 June 2010ABM Workshop -Leeds Background Social Insects: Pharaohs Ant Agent-Based Models Examples

15 June 2010ABM Workshop -Leeds Testing Agent-Based Models Motivation: ABM are increasingly used to make predictions that can affect organisations and even society Hidden errors lurking in such models can be very costly E.g. In the banking sector, losses made by NatWest, Barclays and Deutsche Morgan Grenfell totalling tens of millions of pounds were blamed on decisions that involved economic model errors The correctness of such models and their thorough testing is vital No Formal Testing Technique for ABM has yet been established

15 June 2010ABM Workshop -Leeds Agents vs. Objects “Object-oriented programmers often fail to see anything novel or new in the idea of agents. … While there are obvious similarities, there are also significant differences between agents and objects.” N. R. Jennings, K. Sycara, and M. Wooldridge, “A roadmap of agent research and development,” Journal of Autonomous Agents and Multi-Agent Systems, vol. 1, no. 1, pp. 7–38, Testing Agent-Based Models

15 June 2010ABM Workshop -Leeds Agent vs. Objects: Sources: (1)James J. Odell: "Objects and Agents Compared", in Journal of Object Technology, vol. 1, no. 1, May-June 2002, pp (2)Parunak, H. Van Dyke, "Go to the Ant": Engineering Principles from Natural Agent Systems, Annals of Operations Research, 75, 1997, pp Evolution of Programming Approaches Testing Agent-Based Models

15 June 2010ABM Workshop -Leeds Agents vs. Objects: Main Differences: AUTONOMY and FLEXIBILITY ObjectsAgents Behaviour controlled by external entitiesSelf-governed (goals and rules) Always Say “Yes” - “No” is an errorAllowed to Say “No” Predictive behaviour (Static Functionality – which facilitates inheritance) Unpredictable - Behave differently in different scenarios (can learn from experience and evolve) Some examples of differences between Objects and Agents Testing Agent-Based Models

15 June 2010ABM Workshop -Leeds Testing Agent-Based Models Challenges: ABM are hard to test Models often built to predict previously unseen emergent behaviour Modellers themselves are not sure what to expect Main purpose for modelling in the 1 st place Model Complexity Simulating interactions of thousands and millions of agents is common Tracing back certain model behaviour to certain agent or event is very hard

15 June 2010ABM Workshop -Leeds Errors and Artefacts in Agent-Based Modelling Galan et al. (2009), Journal of Artificial Societies and Social Simulation vol. 12, no. 1 Testing Agent-Based Models

15 June 2010ABM Workshop -Leeds Without visual inspection this bug would have not been caught Most modelling work involve batch processing tasks (i.e. no visualisation) Testing Agent-Based Models

15 June 2010ABM Workshop -Leeds Some Suggested Informal Validation Solutions: 1.Application of an ABM to extreme scenarios 2.Re-implementing an ABM using: different programming languages, programming paradigms agent-based modelling frameworks Testing Agent-Based Models

15 June 2010ABM Workshop -Leeds The Misbehaviour Project: Project’s Aim: Automated Discovery of Emergent Misbehaviour (ADEM) Develop testing techniques and frameworks for agent-based software that can lead to the detection of systems’ Emergent Misbehaviour Towards Formalising ABM testing Testing Agent-Based Models

15 June 2010ABM Workshop -Leeds Approach: Automated Testing Framework composed of 2 main steps Step 1- Learn and Build up an abstract model of the ABM to be tested Formulate fitness functions (simpler for task oriented ABMs) and constraints that define the model’s normal behaviour) Tools: Daikon (invariant detection), statistical and visual analysis methods Step 2- Search-Based Testing Process (using a black box approach) Search for scenarios, parameters or settings that might force the ABM to uncover “Misbehaviour” or bring it close to an extreme behaviour (i.e. violating a fitness functions and/or constraint) Tools: Metaheuristic techniques (e.g. GAs, Memetic Algorithms, … etc) Towards Formalising ABM testing Testing Agent-Based Models

15 June 2010ABM Workshop -Leeds ADEM Procedure Testing Agent-Based Models Learn Model (DAIKON) Misbehaviour Detected? Found Bug No Yes Produce new Behaviour (GAs) Max Iterations Reached? Start End No No Bug Found Yes

15 June 2010ABM Workshop -Leeds Case Studies: Cancer Research and BT 1 - Learn real model (i.e. invariants, agent’s properties and normal behaviour) 2 - Produce buggy models out of the real models How? Mutation Testing: Introduction of artificial bugs 3 – Apply ADEM Procedure: Repeat until Misbehaviour Found or maximum iterations reached IF any Misbehaviour already detected or any Invariant violated Task accomplished ELSE Deploy metaheuristics to search for Misbehaviour END

15 June 2010ABM Workshop -Leeds The End. Thank You.