How to design and analyse empirical research for and in modelling Behavioural Operational Research Summer school June 18th 2019 Dr. Inge Bleijenbergh Dr. Hubert Korzilius Radboud University

Topics Design Analysis Research objective Research questions Research strategies Analysis Deriving or testing CLD’s Building and testing formal models

Design: research objective Broad context of which the research is (a small) part Theory-oriented research project: overview of discussion within scientific area or discipline Practice-oriented research: overview of practical problems within organization(s) or policy area Complex and elaborate matter Steering is necessary Please note: practice-oriented based PhD research also contributes to theory; theory-oriented PhD research always has practical implications.

Theory-oriented research Theory developing research Gaps in the construction of a theory Contribution to the construction of a theory Theory testing research Testing, adjusting and refining existing theory

Practice-oriented research Intervention in order to change an existing situation Five stages Problem analysing research Diagnostic research Design oriented research Intervention oriented research Evaluation research

Design: research objective Formulation: the aim of this research is reaching… (a) … by … (b) … Alternative formulation examining … (b) … in order to… (a) … (the content remains similar) Part (a): contribution of your research project  external goal Part (b): indication of the knowledge, information or insight needed internal goal Refers to the knowledge necessary to provide this contribution.

Design: research objective Theory-oriented research: contribute to the theoretical understanding of/test theoretical explanations of … by … Practice-oriented research: contribute to diagnosing, analyzing, designing, implementing, evaluating policies … by ... Effective useful realistic feasible clear informative

Criteria Research Objective Useful means that it must be convincing that you indeed contribute to the theoretical problem addressed or the practical problem to be solved. Realistic means that you narrow down your contribution to an external goal to a specific aspect of it or a specific incidence of this problem. Feasible means that it must seem feasible to acquire the knowledge in the time available for the PhD research and with the resources available. Clear means that it is precisely defined what the project contributes to the theoretical or practical problem. Informative means that the knowledge to be generated is defined more or less exactly.

Design: research objective Interactive assignment What is the research objective of the papers by Herrera et al. (2016) and Schweiger et al. (2018)? To what extent are they effective: useful; realistic; feasible; clear; informative

Research objective Herrera et al., 2016, p. 1285

Schweiger et al., 2018, p. 1

Design: research questions Research objective External goal (goal of) Practical problem Research questions Internal goal (goal within) Knowledge problem (shaped like questions or hypotheses) 12

Elaborating research questions Corroborative types of knowledge: Descriptive: how does a phenomenon look like? Explanatory: how is a phenomenon explained? Predictive: to what extent do causes explain a phenomenon? Evaluative: how is a phenomenon to be assessed? Prescriptive: what is an effective policy/instrument to change a phenomenon?

Design: research questions Interactive assignment Regarding Herrera et al. (2016) and Schweiger et al. (2018): What are the research questions and how would you characterize them in terms of knowledge types?

Research question Herrera et al., 2016, p. 1280

Research question Schweiger et al., 2018, p. 2

Design: research strategies Qualitative Quantitative Case-study Experiment Survey Number of research units small limited large Characteristics to be studies many few Context natural natural/ artificial Methods for accessing sources unstructured interviews, (participant) observation, content analysis questionnaire, observation questionnaire, structured interview Measurement longer period pretest, posttest single moment

Design: research strategies Key decisions Breadth versus depth Qualitative versus quantitative research Empirical versus desk research

Sources and methods for accessing sources individual people questioning media observation social reality measurement instruments documents content analysis literature search method

Design: research strategies Herrera et al. (2016) and Schweiger et al. (2018) What research strategy? Which key decisions? Which sources and methods for accessing sources?

Design: research strategies Herrera et al. (2016, p. 1287) Experiment Two groups Multimethod, SODA, GMB Single method, GMB No randomization ‘Blocking’ on age, gender, professional background

Design: research strategies Herrera et al. (2016, p. 1288)

Design: research strategies Schweiger et al. (2018, p. 3)

Design: data collection and analysis methods Schweiger et al. (2018, p

Analysis: Herrera et al. (2016, pp. 1296, 1303)

Analysis: Schweiger et al. (2018, pp. 12-13)

Recommendations A sound research design should have: A research objective that is useful; realistic; feasible; clear; informative Research questions that are: Efficient, Steering A research strategy that is Well underpinned With a clear division between sources and methods Data analysis methods which are: Coherent with former choices Most efficient way to achieving the research objective

Discussion What about your own project? Theory oriented or practice oriented? If theory oriented: theory development? Theory testing? If practice oriented: which phase in the empirical cycle? Where could you be more Specific Efficient Coherent?

Recommended literature Denscombe, M. (2012). Research proposals. A practical guide. Maidenhead Berkshire: McGraw-Hill. Sterman, J. D. (2000). Business dynamics. Systems thinking and modelling for a complex world. Boston: McGraw-Hill. Vennix, J. A. M. (1996). Group model building. Facilitating team learning using System Dynamics. Chichester: Wiley. Verschuren, P., & Doorewaard, H. (2010). Designing a research project. The Hague: Eleven International Publishing.