Qualitative Classroom Research: "Blind" Observation and Analysis Damian Bebell Boston College

Case Study Methodology in Technology Studies Often only post-technology (i.e. intervention) inquiry Researcher can be unintentionally biased An example to use throughout: –How does teaching and learning change when a new technology is introduced? –(Russell, Bebell, Cowan, and Corbelli; 2002)

3 Ways to Improve Case Studies collecting data over an extended period of time before the technology is placed into the classroom(s) using observers who are "blind" to the intervention engaging students in documenting their environment and experiences.

Date Collection Before Intervention PROS –Allows the measurement of change!!! –Less reliant upon specific curricula –Increased comfort level with teachers and students CONS –Time consuming and expensive –Potentially confounds “blindness” or objectivity

For example: Study Pre - Post change Increased use of AlphaSmarts and computers Increased peer conferencing

Pre-Post change

“Blind” Observer PROS –Decreases unintentional bias –Increases validity –Less training of the observers CONS –Observer is likely to figure out intervention anyway –Big learning curve (less training) –Interpretation may be off –Difficult finding observers who are not aware of research

Engaging Students in Documenting Their Environment and Experiences Common Ways: –Student Surveys –Student Interviews Less Common Ways: –Student Drawings (Haney, Bebell and Russell; in press) Of classroom Of processes (learning) Low cost Easy for students Engaging for students, researchers, teachers and decision makers

What Students Drew

Code the drawings reliably with emergent analytic coding techniques See: Wheelock, Bebell and Haney, 2000

Code the drawings reliably with emergent analytic coding techniques

Future directions in our case study observation Use of videos Using FileMaker Pro databases –Pre-made templates –Not blinded approach Extension of AlphaSmart (1:1 ratio) work –Successful for multiple observers –Include measurement of time –Random checklists

Overview Page

Phase Notes Page

Checklist Page