1. Copyright, 1995-2002 1 Invitation to Research RESEARCH TECHNIQUES at the SCIENTIFIC/ INTERPRETIVIST BOUNDARY Roger Clarke, Xamax Consultancy, Canberra Visiting Professor, CSIS, Uni of Hong Kong Visiting Fellow, Australian National University http://www.anu.edu.au/people/Roger.Clarke/......Res /54-SciInt.ppt ebs, 16-20 January 2003

2. Copyright, 1995-2002 2 Research Techniques at the Boundary of Scientific and Interpretivist Research A Taxonomy (5) Field Study Questionnaire-Based Survey Interview-Based Survey Case Study Secondary Research

3. Copyright, 1995-2002 3 Field Study – Scientific/Positivist Theory-Driven Theoretical Inferences => Hypotheses Targeted Observation Recording of primarily Quantitative Data Statistical Analysis

4. Copyright, 1995-2002 4 Field Study – Interpretivist Theory-Agnostic or Theory-Driven Descriptive detail is crucial, and lays the foundation for interpretation Data primarily Qualitative Interpretation may be performed intuitively, or using some form of discipline, e.g. Grounded Theory (Strauss) Critical Theory (Habermas)

5. Copyright, 1995-2002 5 Questionnaire-Based Survey The Traditional, Positivist Model

6. Copyright, 1995-2002 6 Questionnaire Survey – Positivist Theory-Driven Theoretical Inferences => Hypotheses Questions Mostly Closed-Ended Recording of primarily Quantitative Data Statistical Analysis

7. Copyright, 1995-2002 7 Quality Tests of Research Method Design Content Validity (all aspects included) Construct Validity (effective operationalisation of the Concepts, i.e. close correspondence between behaviour of the real world and of the measure) Internal Validity Integrity of the Relationships among the Concepts (commonly focussing on causal relationships) External Validity Generalisability within a Definable Domain Reliability Ability to Repeat and Get Similar Results

8. Copyright, 1995-2002 8 Research Statistics Tabulation (Counts, Percentages) Measures of Central Tendency and of Dispersion / Std Deviation / Variance Association and Correlation Tests for Differences ( X 2, t, z) Multivariate: Co-Variance, Regression, Discriminant Analysis Non-Parametric Tests (for non-normal distributions)

9. Copyright, 1995-2002 9 Measurement Bias Systematic Measurement Error Measurement Bias from: Social Context e.g. presence of significant others, data sensitivity such as illicit drugs, sex Recording Bias Intrument Bias, e.g. the set of responses available Procedure Bias, e.g. the sequence of questions Response Bias Systematic tendency to respond based on something other than what the researcher intended Non-Response Bias Non-random refusals, e.g. under-reporting of privacy-lovers attitudes to privacy Item Non-Response Bias Non-random refusals to individual questions

10. Copyright, 1995-2002 10 Unreliability Random Measurement Error The Data Source Respondent States (e.g. boredom) Respondent Recall Respondent Comprehension The Context Physical Conditions (e.g. distractions) Social Conditions The Researcher Comprehension Recording Error Premature Data Reduction Researcher Differences The Measure Insensitivity Instrument and Procedure Variation

11. Copyright, 1995-2002 11 Question Design Foddys TAP Guide Topic – Must be defined, so that each respondent clearly understands what is being talked about Applicability To Each Respondent – Must be established, i.e. respondents should not be asked to give information that they do not have Perspective – The perspective that respondents should adopt when answering the question, must be specified, so that each respondent gives the same kind of answer

12. Copyright, 1995-2002 12 Questionnaire-Based Survey The Symbolic Interactionist View

13. Copyright, 1995-2002 13 Questionnaire Survey – Interpretivist Theory-Agnostic or Theory-Driven Descriptive detail is crucial, and lays the foundation for interpretation Data primarily Qualitative Questions Mostly Open-Ended Interpretation may be performed intuitively, or using some form of discipline, e.g. Grounded Theory (Strauss) Critical Theory (Habermas)

14. Copyright, 1995-2002 14 Questionnaire-Based Survey Process Define Objectives Develop Instrument Define Population Test Instrument Define Sample- Size Acquire Sample Distribute Follow Up Analyse Report

15. Copyright, 1995-2002 15 Interview-Based Survey Similarities to Questionnaire-Based Scientific tends towards structured sequence of carefully-prepared, closed-ended questions Interpretivist is less structured, open-ended questions, with much more interaction Similar approaches to Population, Sampling Frame, Sample

16. Copyright, 1995-2002 16 Elements of the Interview Process Context, Purpose, Common Language Question-Types: Sequence-Controlled cf. Free-Flowing Unstructured cf. Structured If structured: Closed-Ended cf. Open-Ended Means of Recording Feedback, Clarification, Confirmation

17. Copyright, 1995-2002 17 Case Study In-Depth Study of a Particular Setting Some Focus, But Reasonably Broad in Scope Multiple Sources => Triangulation Documents Managers Interviews Subordinates Observation Those Affected Integration of Diverse Materials Preferably Longitudinal, not Snapshot or Cross-Sectional; but possibly Retrospective

18. Copyright, 1995-2002 18 Case Study – Alternative Perspectives Positivist Orientation Theory-Driven Variables Pre-Defined Purposeful Collection Quantitative Data Causality Replicability Interpretivist Orientation Theory-Driven or Not Variables Emerge A Moderate Focus Qualitative Data Understanding Likely Ambiguity

19. Copyright, 1995-2002 19 Single-Case, Multi-Cases, Mini-Cases Single-Case Must be Deep and Rich Justifiable only if Unique or Revelatory Multiple-Case Must be Deep and Rich Must Have Enough (4-10?) They Must Be Comparable Mini-Case Studies / Vignettes Achieves Greater Spread for the Same Resources Superficial, with Limited Scope Information Reliability Very Low Possible Value as a Preliminary Study, to provide insight into diversity, and indications of key variables

20. Copyright, 1995-2002 20 Secondary Research aka Available Data Studies Documents or Data are acquired that were previously captured (probably by someone else, and probably for a different purpose) The Documents or Data are re-analysed Generally, a theoretical framework is used that is different from that originally used