1. C R E S S T / U C L A Qualitative Indicators: Classroom Observations and Classroom Practice Ann M. Mastergeorge UCLA Graduate School of Education & Information Studies Center for the Study of Evaluation National Center for Research on Evaluation, Standards, and Student Testing Education Writers Association Conference “How To Tell If Schools Are Really Improving” December 8, 2000

2. C R E S S T / U C L A An Overview Classroom Observation “ What Things to Look For?” An Overview  A Researcher Perspective: Classroom Culture, Classroom Practice, Classroom Domains  Shaping Classroom Observations: Field notes and Protocols  Meaning in the Classroom: What do the words and ratings mean?

3. C R E S S T / U C L A Characteristics of Qualitative Observations   Natural Setting As Direct Source of Data   Researcher = Key Instrument   Descriptive   Concerned with Process   Analyze Inductively   Meaning in Participant Perspectives

4. C R E S S T / U C L A Classroom Observations: To See or Not to See   What is important to see in and out of the classroom?   How do I structure my fieldnotes and interviews?   Where is the story and how do I let it unfold?

5. C R E S S T / U C L A Classroom Observations Creative Tensions: Issues and Paradoxes   Understanding the classroom as a culture versus the classroom as a classroom   Entering the world as a ”novice" versus the world as an “expert”   Knowing the story versus finding the story and letting the story unfold

6. C R E S S T / U C L A Classroom Observations (con’t.)  Notes versus fieldnotes  The whole picture versus a piece of the picture  The research community versus the school community  Quality of observations versus quantity of observations

7. C R E S S T / U C L A Classroom Observations (con’t) Creative Chaos: The Collision of Observers and Participants   The World As We See It: How to Share a Vision   Creating Understandings of the Observations: Perspectives, Insights, Barriers, Limitations   The Right Answer: Where is it?

8. C R E S S T / U C L A A Day in the Life of a School Issues to Consider   The (sub)cultures   Perspectives   Chaos   Negotiation and Choices   Making the Familiar Strange   Making the Strange Familiar   Finding and Unfolding the story

9. C R E S S T / U C L A Issues and Dilemmas in Fieldwork   At what level do I describe reality (e.g. how specific, how general should I be)?   How inferential should my field observations be?   How do I differentiate between what I see and hear andwhat I infer from my observations?

10. C R E S S T / U C L A Issues and Dilemmas in Fieldwork (con’t)  Should I count "things" that I see?  When have I observed enough?  Should I take my participants points of view or my own when I do fieldwork?  What's important about what I am observing?  Have I seen everything, or is there more to see?

11. C R E S S T / U C L A Issues and Dilemmas in Fieldwork ( con’t)   Should I take notes while I am observing? If so, should they be narratives, shorthand accounts, outlines, or....?   What should I watch--people, interactions, settings, all of the above?   Should I concentrate on watching a few things, or should I sample from many? If the latter, how do I sample?

12. C R E S S T / U C L A Issues and Dilemmas in Fieldwork ( con’t)  Should I quote verbatim what people say or just the gist of theirwords?  What should be my style of observation/recording? Should I be a holistic ethnographer, a radical empiricist, both, neither?

13. C R E S S T / U C L A Characteristics of Field Notes   Rough (jotted, mental notes) versus Finished Notes   Use of Mnemonics   Noting what is missed   Level of Detail

14. C R E S S T / U C L A Characteristics of Field Notes (con’t.)  Observer Comments  Summarizing  Posing New Questions  Timely Write-up

15. C R E S S T / U C L A Field Note Focus   Activities   Individuals   One Individual’s Life   Roles Dyads Times of Day Periods in Routine Types of Acts “Windows” on Behavior Observation Domains:  Descriptive  Focused  Selective Observation Types :

16. C R E S S T / U C L A Field Note Example 1

17. C R E S S T / U C L A Field Note Example 1 (con’t.)

18. C R E S S T / U C L A Field Note Example 2

19. C R E S S T / U C L A Field Note Example 2 (con’t.)

20. C R E S S T / U C L A äOpportunities for engagement in discussions äMonitoring students’ work äCritical thinking skills äQuality feedback äDialogue between students and teachers Domains of Engagement

21. C R E S S T / U C L A Domains of Engagement (con’t.) äExcitement äUse of prompts äIntegrated approach äEngaging implementation of lesson/activity äCreative process and thinking

22. C R E S S T / U C L A CLASSROOM OBSERVATIONS CPOP Classroom Practice Observation Protocol Domains of Classroom Practice:  Challenge of the Lesson Activities  Implementation of Lesson Activity  Proportion of Students On-Task  Opportunity to Participate in Conversation  Student Participation in Conversation  Quality of Instructional Feedback

23. C R E S S T / U C L A

36. TEACHER INTERVIEWS Emerging Themes  Learning that promotes student inquiry and provides students with meaningful experiences  Learners that are active, independent, and empowered  An expectation that test scores will improve and a concern that test scores have to improve

37. C R E S S T / U C L A Student Technology Assignments Teacher Ratings of Technology Integration Assignments

38. C R E S S T / U C L A

41. Classroom Observations Methodological Framework Documentation Interviews Observations