Advantage Business Group ‘The Barbican’, East Street, Farnham, Surrey GU9 7TB

Combining Cognitive Maps with other OR methods to support Decision-Makers ISMOR 21, September 2004 Colin Drysdale

Advantage Technical Consulting 2004 Slide 3 Presentation Outline Introduction Home Office – Crime Reduction Model MOD, Dstl – C 3 I system Benefits Model –(Command Control Communications & Intelligence) Final Discussion

Advantage Technical Consulting 2004 Slide 4 Introduction

Advantage Technical Consulting 2004 Slide 5 Decision making – common themes Governmental decision makers Account rationally for decisions Possibility of some form of ‘public’ scrutiny

Advantage Technical Consulting 2004 Slide 6 Case Study 1 Home Office – Crime Reduction Model

Advantage Technical Consulting 2004 Slide 7 Home Office – Crime Reduction Model Responsibilities include reducing the level of crime in England and Wales Culture change ‘evidence based policy making’ Justification

Advantage Technical Consulting 2004 Slide 8 Spending Review 2004 Knowledge existed across a number of groups Some of this was locked in experts’ minds Seemed like a job for SODA cognitive mapping –(Strategic Options Development and Analysis )

Advantage Technical Consulting 2004 Slide 9 Cognitive Mapping - Justification

Advantage Technical Consulting 2004 Slide 10 Intermediate step List Example Intervention Goal Cognitive Map - Concepts

Advantage Technical Consulting 2004 Slide 11 Weak Link Connotative Possible Link Causal - Negative Causal Logical AND & & Cognitive Map - Links Correlation

Advantage Technical Consulting 2004 Slide 12 Intervention Goal Intervention Cognitive Map - Constructs

Advantage Technical Consulting 2004 Slide 13 Intervention Goal Intervention Argument Constructs

Advantage Technical Consulting 2004 Slide 14 Intervention Goal Intervention Evidence summary and URL link to supporting Documentation in a pop-up box Memo ArgumentEvidence Constructs

Advantage Technical Consulting 2004 Slide 15 Intervention Goal Intervention Evidence summary and URL link to supporting Documentation in a pop-up box Memo ArgumentEvidence Constructs = Case+

Advantage Technical Consulting 2004 Slide 16 Approach ~30 Interviews –Generated initial maps interviewees invited to discuss: –causes of crime –policy goals –interventions –… –evidence sources / quality Synthesis –To a single map (~ concepts)

Advantage Technical Consulting 2004 Slide 17 Initial benefits Knowledge model Single resource Linked to exiting documents Shows where more research & analysis is required

Advantage Technical Consulting 2004 Slide 18 Next issue Interventions that should be funded ? Defines the “Problem” Time stepped simulation

Advantage Technical Consulting 2004 Slide 19 Further insights Exploit causal pathways Interventions to goals ~1800 concepts MS Excel with VBA tool developed Enabled analysis of –Contradicting interventions –Loops in the causal logic –Completely dependent interventions

Advantage Technical Consulting 2004 Slide 20 Dependent interventions

Advantage Technical Consulting 2004 Slide 21 Findings Many logical anomalies different interpretations of “intervention” Showed that – more clarity needed –for quantitative modelling

Advantage Technical Consulting 2004 Slide 22 Case Study 1 – Further benefits Cognitive modelling helped define ‘the problem’ and requirements for more quantitative modelling In this role SODA cognitive mapping should not be viewed ‘soft’ OR –it enables a ‘soft-to-harder’ transition

Advantage Technical Consulting 2004 Slide 23 ‘soft-to-harder’ methodologies subjective objective quantitativequalitative Soft Hard

Advantage Technical Consulting 2004 Slide 24 ‘soft-to-harder’ methodologies subjective objective quantitativequalitative Soft Hard

Advantage Technical Consulting 2004 Slide 25 ‘soft-to-harder’ methodologies subjective objective quantitativequalitative Soft Hard

Advantage Technical Consulting 2004 Slide 26 ‘soft-to-harder’ methodologies subjective objective quantitativequalitative Soft Hard

Advantage Technical Consulting 2004 Slide 27 ‘soft-to-harder’ methodologies subjective objective quantitativequalitative Soft Hard tacit explicit

Advantage Technical Consulting 2004 Slide 28 Case Study 2 MOD - C3I system Benefits Modelling

Advantage Technical Consulting 2004 Slide 29 Benefits Modelling Dstl name Causal mapping & MCDA C 3 I systems

Advantage Technical Consulting 2004 Slide 30 The Task Aim - Develop a C 3 I system benefits model Purpose – support contractor down-select & OR programme definition Implementation – constrained to perform calculations in Excel with VBA for ease of maintenance in-house

Advantage Technical Consulting 2004 Slide 31 The concept SODA COGNITIVE maps contain CAUSAL logic –and additional information MCDA networks for modelling C 3 I can get complicated ….

Advantage Technical Consulting 2004 Slide 32 MCDA nets for C 3 I get complicated Investments (SRD / ADD Level) Investments (URD Level) System Attributes Organisational Attributes Military capabilities Value System

Advantage Technical Consulting 2004 Slide 33 The concept SODA cognitive maps contain causal logic –and additional information MCDA networks for modelling C 3 I can get complicated Can’t use a Commercial Off The Shelf (COTS) MCDA tool

Advantage Technical Consulting 2004 Slide 34 The concept SODA cognitive maps contain causal logic –and additional information MCDA networks for modelling C 3 I can get complicated Can’t use a Commercial Off The Shelf (COTS) MCDA tool Why not use the cognitive map as the front end to an Excel with VBA MCDA model

Advantage Technical Consulting 2004 Slide 35 What happened Workshop - “false start” ! Unavoidable part of the problem exploration process How to recover –Stay calm –Needs skill and experience

Advantage Technical Consulting 2004 Slide 36 Recovery Method was unfamiliar but …. –Resembled familiar information flows Solution was to exploit familiar information flows Converted these “off-line” High level structure

Advantage Technical Consulting 2004 Slide 37 Benefits Modelling subjective objective quantitativequalitative Soft Hard causal mapping MCDA

Advantage Technical Consulting 2004 Slide 38 Final Discussion

Advantage Technical Consulting 2004 Slide 39 Final Discussion Take-up of Soft OR techniques has not been universal Combining cognitive mapping with other techniques –quantified logical case –pass independent scrutiny Both studies illustrate that cognitive mapping supports problem formulation The second study illustrates that cognitive mapping requires skill & experience

Advantage Technical Consulting 2004 Slide 40 References / Reading Description of the SODA technique see: “Rational Analysis for a Problematic World”, Ed. J Rosenhead, Chapter 2, John Wiley & Sons, “Multiple Criteria Decision Analysis – Practically the only way to choose”, V Belton, OR 32, Sep 1990, Published in “Operational Research Tutorial Papers: 1990” “Benefits Analysis – A Robust Assessment Approach”, G Mathieson, Dstl, 2002, Unclassified.