1. How to design and analyse empirical research for and in modelling
Behavioural Operational Research Summer school
June 18th 2019
Dr. Inge Bleijenbergh
Dr. Hubert Korzilius
Radboud University

2. Topics Design Analysis Research objective Research questions
Research strategies
Analysis
Deriving or testing CLD’s
Building and testing formal models

3. Design: research objective
Broad context of which the research is (a small) part
Theory-oriented research project: overview of discussion within scientific area or discipline
Practice-oriented research: overview of practical problems within organization(s) or policy area
Complex and elaborate matter
Steering is necessary
Please note: practice-oriented based PhD research also contributes to theory; theory-oriented PhD research always has practical implications.

4. Theory-oriented research
Theory developing research
Gaps in the construction of a theory
Contribution to the construction of a theory
Theory testing research
Testing, adjusting and refining existing theory

5. Practice-oriented research
Intervention in order to change an existing situation
Five stages
Problem analysing research
Diagnostic research
Design oriented research
Intervention oriented research
Evaluation research

6. Design: research objective
Formulation: the aim of this research is
reaching… (a) … by … (b) …
Alternative formulation
examining … (b) … in order to… (a) …
(the content remains similar)
Part (a): contribution of your research project  external goal
Part (b): indication of the knowledge, information or insight needed internal goal
Refers to the knowledge necessary to provide this contribution.

7. Design: research objective
Theory-oriented research: contribute to the theoretical understanding of/test theoretical explanations of … by …
Practice-oriented research: contribute to diagnosing, analyzing, designing, implementing, evaluating policies … by ...
Effective
useful
realistic
feasible
clear
informative

8. Criteria Research Objective
Useful means that it must be convincing that you indeed contribute to the theoretical problem addressed or the practical problem to be solved.
Realistic means that you narrow down your contribution to an external goal to a specific aspect of it or a specific incidence of this problem.
Feasible means that it must seem feasible to acquire the knowledge in the time available for the PhD research and with the resources available.
Clear means that it is precisely defined what the project contributes to the theoretical or practical problem.
Informative means that the knowledge to be generated is defined more or less exactly.

9. Design: research objective
Interactive assignment
What is the research objective of the papers by Herrera et al. (2016) and Schweiger et al. (2018)?
To what extent are they effective:
useful; realistic; feasible; clear; informative

10. Research objective Herrera et al., 2016, p. 1285

11. Schweiger et al., 2018, p. 1

12. Design: research questions
Research objective
External goal (goal of)
Practical problem
Research questions
Internal goal (goal within)
Knowledge problem (shaped like questions or hypotheses) 12

13. Elaborating research questions
Corroborative types of knowledge:
Descriptive: how does a phenomenon look like?
Explanatory: how is a phenomenon explained?
Predictive: to what extent do causes explain a phenomenon?
Evaluative: how is a phenomenon to be assessed?
Prescriptive: what is an effective policy/instrument to change a phenomenon?

14. Design: research questions
Interactive assignment
Regarding Herrera et al. (2016) and Schweiger et al. (2018):
What are the research questions and how would you characterize them in terms of knowledge types?

15. Research question Herrera et al., 2016, p. 1280

16. Research question Schweiger et al., 2018, p. 2

17. Design: research strategies
Qualitative
Quantitative
Case-study
Experiment
Survey
Number of research units
small
limited
large
Characteristics to be studies
many
few
Context
natural
natural/
artificial
Methods for accessing sources
unstructured interviews,
(participant) observation,
content analysis
questionnaire,
observation
questionnaire, structured interview
Measurement
longer period
pretest, posttest
single moment

18. Design: research strategies
Key decisions
Breadth versus depth
Qualitative versus quantitative research
Empirical versus desk research

19. Sources and methods for accessing sources
individual people
questioning
media
observation
social reality
measurement
instruments
documents
content analysis
literature
search method

20. Design: research strategies
Herrera et al. (2016) and Schweiger et al. (2018)
What research strategy?
Which key decisions?
Which sources and methods for accessing sources?

21. Design: research strategies Herrera et al. (2016, p. 1287)
Experiment
Two groups
Multimethod, SODA, GMB
Single method, GMB
No randomization
‘Blocking’ on age, gender, professional background

22. Design: research strategies Herrera et al. (2016, p. 1288)

23. Design: research strategies Schweiger et al. (2018, p. 3)

24. Design: data collection and analysis methods Schweiger et al. (2018, p

25. Analysis: Herrera et al. (2016, pp. 1296, 1303)

26. Analysis: Schweiger et al. (2018, pp. 12-13)

27. Recommendations A sound research design should have:
A research objective that is
useful; realistic; feasible; clear; informative
Research questions that are:
Efficient, Steering
A research strategy that is
Well underpinned
With a clear division between sources and methods
Data analysis methods which are:
Coherent with former choices
Most efficient way to achieving the research objective

28. Discussion What about your own project?
Theory oriented or practice oriented?
If theory oriented: theory development? Theory testing?
If practice oriented: which phase in the empirical cycle?
Where could you be more
Specific
Efficient
Coherent?

29. Recommended literature
Denscombe, M. (2012). Research proposals. A practical guide. Maidenhead Berkshire: McGraw-Hill.
Sterman, J. D. (2000). Business dynamics. Systems thinking and modelling for a complex world. Boston: McGraw-Hill.
Vennix, J. A. M. (1996). Group model building. Facilitating team learning using System Dynamics. Chichester: Wiley.
Verschuren, P., & Doorewaard, H. (2010). Designing a research project. The Hague: Eleven International Publishing.