1.3 Modeling process and outcomes Walter Sermeus, RN, PhD Catholic University Leuven Belgium Witten, Fri Session 1: 11:00-12:30 Session 2: 13:30-15:00

The European Academy of Nursing Science2 Background: MRC CI-framework Modeling

The European Academy of Nursing Science3 What does the MRC-CI framework say on Modelling process and outcomes? Modelling a complex intervention prior to a full scale evaluation can provide important information about the design of both the intervention and the evaluation. One useful approach to modelling is to undertake a pre-trial economic evaluation. This may identify weaknesses and lead to refinements, or it may show that a full-scale evaluation is unwarranted, for example because the effects are so small that a trial would have to infeasibly large. Formal frameworks for developing and testing complex interventions, such as MOST or RE_AIM may be a good source of ideas, and the National Institute for Health and Clinical Excellence has produced detailed guidance on the development and evaluation of behaviour change

The European Academy of Nursing Science4 What will we do in this session ? What & why Models – Tools Economic evaluation Examples Excercises

The European Academy of Nursing Science5 What kind of models ? No scale models Rather causal models: –A causal model is an abstract model that uses cause and effect logic to describe the behaviour of a system. –Focus on understanding, relationships, which effects might be expected –Using theories, from various disciplines

Why models ? Refining conceptual models –influences, components, relations, consequences Generating (tentative) estimates of effect size Identifying barriers Optimising combinations of components of the intervention The European Academy of Nursing Science6 (Source: Campbell et.al., BMJ, 2007)

Relation between context, problem definition, intervention and evaluation The European Academy of Nursing Science7 (Source: Campbell et.al., BMJ, 2007)

MODELS: PRIME PRocess Modelling in ImpleMEntation research –Identifying active ingredients in professional behaviour change –Process modelling: understanding of factors underlying clinical practice, in order to identify what sorts of processes should be targeted in implementation interventions –4 levels: individuals, teams, organisations, systems –Different kind of theories The European Academy of Nursing Science8 (Source: Walker AE et.al., BMC HSR, 2003)

Example of PRIME Theories - individual level The European Academy of Nursing Science9

Normalization process model Theoretical framework for understanding complex interventions Normalization vs adoption / rejection 4 dimensions: –Interactional workability: effect on interactions between people and practices –Relational integration: relation to existing knowledge and relationships –Skill-set workability: effect on current division of labour –Contextual integration: relation to the organisation The European Academy of Nursing Science10 May C, BMC Health Services Research, 2006

Example of NPM to telehealth services The European Academy of Nursing Science11 May C, BMC Health Services Research, 2006

Tools: RE-AIM DimensionLevelDescription ReachIndividualProportion of target population that participated in the intervention Efficacy / effectiveness IndividualSuccess rate; positive / negative outcomes AdoptionOrganizationProportion of settings that will adopt the intervention ImplementationOrganizationExtent to which intervention is implemented as intented in real world MaintenanceI&OExtent to which a program is sustained over time The European Academy of Nursing Science12 Source: Glasgow R. et.al., 1999 ( The goal of RE-AIM is to encourage program planners, evaluators, funders, and policy-makers to pay more attention to essential program elements including external validity that can improve the sustainable adoption and implementation of effective, generalizable, evidence-based interventions.

TOOLS : MOST Multiphase Optimization Strategy –Screening phase: Components that are candidates for inclusion in an intervention (active components) Based on significant effect, effect size, cost,… –Refining phase: Fine tuning e.g. optimal level –Confirming phase: Evaluating the intervention as a package The European Academy of Nursing Science13 (Source: Collins et.al. Am J Prev Med, 2007)

Economic evaluation Source: R. Taylor, Masterclass Complex Interventions, Exeter 2010

Incremental Cost Effectiveness Ratio (ICER) cost [complex intervention – usual care] outcome [complex intervention – usual care] ICER = Quality adjusted life years (QALYs)

Cost utility analysis (CUA) Utility Life expectancy A B Intervention A = 5 QALYs perfect health death 0 Intervention B = 7 QALYs QALY difference = +2 QALYs 0.5

EQ-5D (EuroQoL) (

EQ-5D scoring guide

Example of Modelling exercise ProActive causal model (Hardeman et.al., Health Education Research, 2005) Aim: increase physical activity among individuals at risk of Type 2 diabetes The model provides: –Rational guide for appropriate measures –Intervention points –Intervention techniques The European Academy of Nursing Science19

Generic causal model Epidemiology –Defining health outcomes and precise objective measures, target group, likely impact of achievable behaviour change on physiological and biochemical variables Psychology –Theory-based determinants, intervention points, techniques to support behaviour, measures of change in behavioural determinants The European Academy of Nursing Science20

Methods used The European Academy of Nursing Science21

Theory of Planned Behaviour (Ajzen 1991) beliefs

Results : ProActive causal model The European Academy of Nursing Science23

… continued The European Academy of Nursing Science24

Hardeman causal modelling case study: extending the MRC framework Concise one-page representation of causal pathways Guides the choice of intervention points and measures Assists in choice of behaviour change techniques Informs the assessment of ‘fidelity’ to theories Enables statistical modelling of the relationships between behaviours and health outcomes

Exercises Exercise 1: Joint arthoplasty surgery Exercise 2: diabetes The European Academy of Nursing Science26