Johan Mouton© February 2006 C Hart 2007

Exploratory questions What are the most important variable that have an effect on learner achievement? What happens in the classroom i.t.o. teaching pedagogy? What conceptions do different educators have about “ authentic pedagogy ” ? Appropriate Designs Exploratory surveys In-depth case studies Ethnographic studies in the school Descriptive/Correlational Questions Bivariate: Do learners in small classes do better than children in large classes Multivariate: Are children from single- parent families & rural areas less likely to perform well than … ? Appropriate Designs Cross-sectional surveys Predictive Questions What would the effect of x be on y? Appropriate Designs Experiments & Field Experiments

Causal questions What is the differential effect of parental SES on children in large/small classes & within rural & urban schools? Which causal/multiple regression model best explains differences in learner performance? How does an effective teacher produce/achieve situated pr0blem- solving in the classroom? Appropriate Designs Analytical surveys (theoretically driven surveys + statistical modeling) Experimental & field-experimental studies In-depth case studies incl. Extensive participant observation Evaluative Questions ( outcome / impact Questions) Will a change in teaching training method (intervention) produce more effective teaching (outcome)? Multivariate Appropriate Designs Evaluation studies (comprehensive evaluations/quasi- experimental designs)

 The what ’ of your study – what object, phenomenon, entity, process or event  Unit of analysis typically also the unit of observation  Unit of analysis – things we examine to construct summary description of all such units  Must be clear to what your unit of analysis is  Common social science units of analysis: 1. Individuals 2. Groups 3. Organizations & Institutions 4. Social artifacts/cultural objects 5. Social actions 6. Interventions

1. Characteristics Individuals - Gender, age, height, marital status, deformities, region of origin, or hearing ability Groups & Organisations – size, structure, location & aggregated descriptions of their members Artifacts – size, colour, weight, characteristics of people associating with them – social events; where they occur what type of people involved, etc. 2. Orientations Individuals – attitudes, beliefs, values, personality traits, prejudices, religious, political, intellectual, superstitious Social Groups – purposes, policies, regulations/procedures 3. Actions Observe directly or second-hand accounts of actions i.e. voting

 Rationale for research design: Plan & structure research project for validity of findings through minimizing and/or eliminating potential error  Validity = ‘ best approximation to the truth ’  Key: How does one recognize valid research? 1. Id key dimensions of validity 2. Argue maximizing validity if by minimizing/ eliminating foreseeable threats to validity  Validity to be applied to entire research process

Stage in research process Sources of errorMethodological ‘ move ’ / ‘ strategy ’ (objective research Outcome/goal/end- product Epistemic (validity- related quality or criterion Conceptualization (conceptual analysis) Complex notions, Vagueness, Ambiguity, Abstract concepts Thorough literature review, Clear & logical definitions. Concepts/definitionsTheoretical validity (clarity/scope) OperationalisationPoor sampling of items, Leading questions, Scaling errors Scale validation, Face Validity, Pilot test. Measuring instrumentsMeasurement validity (construct validity) SamplingBias, Heterogeneous populations, Incomplete sampling frame Probability sampling Stratification, Optimal sample size SampleRepresentativeness Data collectionsObservation effects, Interviewer bias, Respondent bias, Context effects Multi-method Proper training of fieldworkers Data setsReliability Analysis/interpretationCompeting/rival conclusions or explanations Appropriate techniques of analysis Thorough understanding of literature Conclusion/results/ findings Inferential (conclusions) validity

1. Theoretical validity 2. Measurement validity 3. Representativeness 4. Reliability & 5. Inferential validity 1. Conceptual vagueness 2. Measurement error 3. Biased samples 4. Unreliable data & 5. Invalid conclusions

 Key validity criterion for data collection is ‘ reliability ’  “ Will the same method as used by different researchers and/or at different times produce the same results? Smith, 1975:58)  Reliability of data is effected by: 1. Researchers, experimenters, interviewers/ observers 2. Individuals/subjects participating in the research 3. Measuring instruments – questionnaires, scales, tests, interviewing & observation schedules 4. Research context/circumstances under which the research was conducted

1. Researcher 1.2 Researcher orientations (attitudes, values, options, preferences 3. Measuring Instrument (questionnaire, interview, schedule, observation, categories, scale, index) 4. Research Context 2. Participant 2.1 Participant characteristics ( gender, nationality, age, educational, level, income level) 2.2 Participant orientations (attitudes, preferences, opinions 1.1 Researcher characteristics (gender, nationality, age, socio-economic status, educational level)