SIT094 – The Collection & Analysis of Quantitative Data Week 7 Luke Sloan Quantitative Research Design Week 7 Luke Sloan Quantitative Research Design

Introduction What is Research Design Experimental Designs Cross-Sectional Designs Longitudinal Designs Case Study Designs Mixed-Methods Designs (Hybrid)

What is Research Design I There is no such thing as a ‘quantitative research design’! The research design is the framework in which we design our data collection instruments Data collection instruments can collect qualitative or quantitative data There is nothing inherently quantitative about surveys and there is nothing inherently qualitative about case-studies You should generate the questions and hypotheses first – then the appropriate research design will become clear

What is Research Design II Topic Area Research Question 1 Hypotheses 1 Research Question 2 Hypotheses 2 Hypotheses 3 Hypotheses 4 Hypotheses 5 Hypotheses 6 The data we collect must give us the information we need to answer our hypotheses However it is not until we have generated our hypotheses that we know what data we need! Until we have framed the problem then we do not know which research design to use either The data we collect must give us the information we need to answer our hypotheses However it is not until we have generated our hypotheses that we know what data we need! Until we have framed the problem then we do not know which research design to use either Best text for research design is: de Vaus, D. (2011) Research Design in Social Research London: Sage Best text for research design is: de Vaus, D. (2011) Research Design in Social Research London: Sage

Measure attitudes towards racism, split sample into two groups, one receives the intervention and one doesn’t, measure attitudes again and compare Create metric for measuring racism, measure attitudes of men and women, compare differences Repeatedly measure level of racism in the same group of people over time Choose a single or a handful of people with racist attitudes, investigate their background in depth What is Research Design III Attitudes towards racism have changed over time Men are more racist than women Racism is the result of environmental factors Racism can be reduced though participation in an intervention programme HYPOTHESES HOW TO TEST?

Case Study Design Longitudinal Design Cross-Sectional Design Experimental Design Measure attitudes towards racism, split sample into two groups, one receives the intervention and one doesn’t, measure attitudes again and compare Create metric for measuring racism, measure attitudes of men and women, compare differences Repeatedly measure level of racism in the same group of people over time Choose a single or a handful of people with racist attitudes, investigate their background in depth What is Research Design IV Attitudes towards racism have changed over time Men are more racist than women Racism is the result of environmental factors Racism can be reduced though participation in an intervention programme HYPOTHESES HOW TO TEST? RESEARCH DESIGN

Experimental Designs I For measuring the effect of an ‘intervention’ One experimental group and one control group (placebo) Measure before and after (t1 and t2) Allows you to control for random fluctuations through a comparative element and random group assignment You only change a single variable – everything else is constant Common in medical sciences and psychology (normally involving classical music!) and could use a range of data collection tools But… it can be difficult (perhaps impossible) to run strict experiment in social research – how do you only change one variable? What is the intervention?

Experimental Designs II You must measure exactly the same thing before and after (same questions) Normally uses scales rather than dichotomies/nominal categories Suggest using a 10pt scale rather than 5pt likert Split between experimental and control group must be random (aka. no significant differences in characteristics) Two sets of analyses: – Compare experimental and intervention group to check they are drawn from the same population at t1 and t2 – Compare before and after to test whether intervention worked and whether similar changes were present in control group (temporal fluctuations)

Cross-Sectional Designs I No intervention – relies on existing differences Variable(s) of interest should vary in sample (hypothetically…) Data collected a t1 only (a snapshot) Surveys are the typical tool of this design Quick and cheap – effective if sampled well Common in social and market research But… the lack of a temporal dimension prevents controlling for random fluctuations as does the lack of a ‘control group’

Cross-Sectional Designs II You should know in advance what groups you are going to compare (male/female, ethnicities, social class…) You can only take a snap-shot, so think about your questions (current salary vs. past salary – how could someone answer the latter?) Essentially all of your analysis will be based on comparing groups, but you might also want to split the data… – Male/female AND ethnicity – Splitting groups may lead to low ‘n’

Longitudinal Designs I Repeatedly measure the same sample over a period of time This allows temporal and potential causal links to be made Perhaps there is an intervention, but often it is not that simple Examples include the many birth cohort studies Extremely powerful datasets e.g National Child Development Study surveyed at ages 7, 11, 16, 23, 33, 42, 46 and 50 plus subset of DNA, 1971 and 1981 Census data and examination results But… very time consuming and extremely expensive normally involving substantial investments from governments or research councils. Datasets can also be tremendously complex and require specialist skills (e.g. geographical transience)

Longitudinal Design II Many of the large-scale national datasets are longitudinal, but not all use the same cohort (panel studies) Mainly involving secondary data, so your problem has to be answerable using data collected by someone else Data analysis is more complex – not just comparing groups, but over time and space through linking records Looking at things over time is salient for causation (strong probabilistic) A single ‘phase’ of a longitudinal study is cross-sectional!

Case Study Designs In-depth examination of a given case Uses contextual information to inform causal processes typically (nut not always) using interviews It is possible to compare two cases, but normally after individual examination Think of it as building up a historical narrative (telling a story) and linking it to a theory Normally qualitative in nature, but I’ve run qual/quant case studies (historical electoral records and context of three London Boroughs) But… the focus on a single ‘case’ means that there is no comparison or control group. If comparisons are made, the focus on contextual factors tends to create a situation in which you are not comparing like with like

Mixed-Method Designs (Hybrid) Much more common than you might think… Use cross-sectional survey to identify case studies? Secondary analysis of longitudinal data to inform experimental hypothesis? Cross-sectional study to test how experimental results workout in the social world? Experimental design to confirm longitudinal hypotheses? Confirm case study observations with experiment to test hypothesis? Enhance a longitudinal study with a cross-sectional investigation for data not already collected? Enhance a longitudinal study with an in-depth case study – or vice versa? Best text for mixed-methods is: Creswell, J. & Plano Clark, V. (2011) Designing and Conducting Mixed Methods Research Thousand Oaks: Sage

Summary There are no ‘qualitative’ or ‘quantitative’ research designs – it’s just that some designs tend to lend themselves to specific data collection tools a majority of the time However there is also nothing inherently ‘quantitative’ about a survey Nor is there anything inherently ‘qualitative’ about interviews for that matter Choose the research design that best fits your question Understand that there are different quantitative considerations for each design that are essential to bare in mind

Any Questions?

Workshop Go to the ESDS Longitudinal website at: Select a study from the list and explore the design/methodology section Also use this session to ask me any questions about the course so far