Postgraduate Course 7. Evidence-based management: Research designs

Postgraduate Course Empirical circle

Postgraduate Course Research designs: 4 elements Randomization Randomization Comparison Comparison Before vs after Before vs after Prospective vs retrospective Prospective vs retrospective

Postgraduate Course Randomized controlled study Randomized controlled study Controlled before-after study Controlled before-after study Cohort / panel study Cohort / panel study Before-after study Before-after study Post-test only study Post-test only study Survey Survey Case study Case study Experiment Experiment Quasi-experiment Quasi-experiment Observational study Observational study Longitudinal study Longitudinal study Cross-sectional study Cross-sectional study Qualitative study Qualitative study Research designs: terms

Postgraduate Course CBA: controlled before-after study In a controlled before-after study two or more groups are compared with each other, usually comprising one group in which an intervention is carried out (experimental group) and one group where no or an alternative intervention is conducted (control group).

Postgraduate Course = CBA: controlled before-after study

Postgraduate Course In the case of randomization, the groups compared with each other are selected entirely randomly, for example by drawing lots. This means that each participant (or other unit such as a team, department or company) has an equal chance of being in the intervention or control group. In this way, the influence of any distorting factors is spread over both groups so that these groups are as comparable as possible with each other with the exception of the intervention. Better (but not always feasible): randomization

Postgraduate Course = even oneven Randomized controlled study (RCT)

Postgraduate Course Cohort / panel study Starting point: intervention/exposure (independent variable) Longitudinal study where large groups of people or companies (cohort / panel) are followed over a long period to see (prospective) whether differences occur among the groups. =

Postgraduate Course Case-control study Starting point: outcome (dependent variable) Longitudinal study in which one group of people or companies with a particular outcome is compared retrospectively with a group that does not have this outcome. = VS VS

Postgraduate Course Examples of (flawed) case-control studies

Postgraduate Course versus Study in which data are obtained or particular characteristics are measured of a population before versus after an intervention / exposure / event, to measure the effect or correlation. Before-after study

Postgraduate Course = Posttest-only

Cross-sectional study Study in which data of a statistically significant sample of a population (managers, CEOs, employees) is gathered at one point in time. It provides a snapshot of the current condition but does not explain cause and effect. Cross-sectional studies include surveys

Postgraduate Course Cross-sectional study ? ? ?

Postgraduate Course Study in which a large number (qualitative or quantitative) of aspects of a single case (organization or team) was investigated in depth over a long period within the cases own context. Case study

Postgraduate Course Experimental research True experiment: RCT True experiment: RCT Quasi experiment: CBA Quasi experiment: CBA Experimental research refers to studies where the researcher manipulates one (or more) variable and controls the other variable(s) to determine whether there is a causal relation between the manipulated variable and the outcome.

Postgraduate Course Observational research Natural experiment Natural experiment Cohort / panel study Cohort / panel study Case-control study Case-control study Observational research refers to studies where the researcher merely observes but does not intervene, with the intention of finding correlations among the observed data

Postgraduate Course Qualitative research explores and tries to understand people's beliefs, experiences, attitudes, behaviour and interactions. It generates non-numerical data. The best- known qualitative research-methods include in-depth interviews, focus groups, documentary analysis and participant observation. Qualitative research includes Case studies Case studies Ethnographic studies Ethnographic studies Field research Field research Grounded theory approach Grounded theory approach Qualitative research

Postgraduate Course Longitudinal research Cohort / panel study Cohort / panel study Repeated measures Repeated measures Times series Times series A longitudinal study is an observational study that involves repeated observations (measurements) of the same variable(s) over long periods of time (sometimes years or even decades).

Postgraduate Course Mixed methods

Postgraduate Course Better than a single study: a replication study Better than a replication study: a systematic review / meta analysis If there were 100 studies, 99 of which gave a negative result (where, say, the new intervention appeared to be not effective), while one had a positive result (were the intervention appeared effective), it would obviously be a mistake to consider only the single positive study. But ….

Postgraduate Course Systematic review / meta-analysis The intention behind a systematic review is to identify as fully as possible all the scientific studies of relevance to a particular subject and to assess the validity and authority of the evidence of each study separately. As the name indicates, a systematic review takes a systematic approach to identifying studies and has the methodological quality critically appraised by multiple researchers independently of each other, as a consequence of which the review is transparent and reproducible and can be monitored. The use of statistical analysis techniques in a systematic review to pool the results of the individual studies numerically in order to achieve a more accurate estimate of the effect is termed a meta-analysis.

Postgraduate Course ? Systematic review / meta-analysis

Postgraduate Course 1.Randomized controlled study 2.Controlled before-after study 3.Cohort / panel study 4.Before-after study 5.Post-test only study 6.Survey 7.Case study A.Experiment B.Quasi-experiment C.Observational study D.Longitudinal study E.Cross-sectional study F.Qualitative study Which terms belong together?