EDU 5818 DATA COLLECTION AND ANALYSIS Dr Ramli Basri Faculty of Educational Studies University Putra Malaysia

CONTENT 1. Problems in Observation 2. Data Collection Procedures 3. Data Analysis Approach

1. PROBLEMS IN OBSERVATION Pitfalls in collecting complete and accurate data from classroom observation. Inability to collect complete records: issue when data critical to analysis is not recorded. Subjectivity: data is subject to supervisor values, experience or mood. Record observation not interpret observation. Presence of supervisor: Alters teacher’s and students’ behavior Changing focus of observation: appropriate data collection technique for each observation focus.

2. DATA COLLECTION PROCEDURES: FROM A WIDE TO A NARROW FOCUS Focus of Observation Widest Focus Narrowest Focus Educational Criticism Anecdotal/ Short Narrative Record Verbatim Selective Verbatim Observation Systems

(a) Educational Criticism Tell story of classroom (like a game commentator on radio) Supervisor presents their perception on the characters Purpose to find the hidden meaning of classroom events and disclose them to verify them with the teacher Common use: research that addresses context of schooling.

(b) Anecdotal Record Objective or factual descriptions of classroom events Broad but selective set of data for a narrow focus Exclude interpretation and summery of observation Research technique for ethnography and neo-traditionalist supervisor Can be used for assisting and assessing teachers.

(c) Verbatim Recording/Selective Verbatim Records specific verbal behaviors in the classroom (types of teacher or students responses) Emphasis on effective teaching Simple to implement

(d) Observation Systems Focus on specific behavior that can quantified Requires data gathering instrument with a prescribed procedure. Instrument based on frequency and inference.

Features of Observation Systems Focus on specific teacher/students behaviors (verbal & non verbal behaviors) Amount of data and degree of inference varies during observation Prescribed procedures and precision of data gathering varies during observation Quantifiable data can be summarized and presented to teachers Amount of training to acquire skills varies Data gathering device/instrument made by teacher/supervisor or design by professional.

Observation Systems Design/Format Category or Sign Format Frequency or Sequence Format Event or Time Dependent Format.

a) Category Format Set categories comprise all the possible behaviors of the type to be observed. All behaviors must fit into one of the categories Categories are mutually exclusive Eg. Teacher Questioning (behavior); Categories (memory/ convergent/divergent/evaluative) (according to Bloom’s taxanomy)

b) Sign Format Designated behaviors may or may not occur during observation Observer makes notation every time the designated behavior occurs Eg. Using foreign language in classroom (behavior); observer notes word “okey” used by teacher. Use of BM/English in PPSMI class

c) Frequency Format d) Sequence Format Record frequency of a behavior during class Record behavior every time it occurs Eg. Motivating (M) and Demotivating (D) behavior of teacher towards students Tally M and D d) Sequence Format Record sequence of behavior during class MMMDMDDDM

e) Event-Dependent Format Record each time a particular event occurs without regards to when it occurs Eg. Record the type of question each time the teacher asks one (like usual note taking) f) Time-Dependent Format Records events at set interval Eg. Records student attentive (A)/ not attentive (N)

3. DATA ANALYSIS APPROACH Types of Data Analysis Analysis conducted by supervisor Focus on the structure of teaching/learning behaviors Discern (identify) possible relationships between teacher and students; and Compare to teachers perception and with other observations.

Analysis conducted by teacher Identify the structure of teaching and the impact of teacher behaviors on students outcome Generate hypothesis that guide further investigation of classroom acts ( what next?) Analysis by Supervisor and Teacher Find agreements on structure of teaching and learning behaviors and impact on students and teaching practice.

Data Gathering and Analysis Approach a) Neo-Traditional Approach Supervisor has set criteria (eg. effective teaching) Observe using anecdotal record the events in the classroom to reflect teachers decision (non judgmental) Compare observation with criteria Supplemented with observation instrument.

b) Instructional Objective Approach Function-Observe and record data as evidence that students have obtained an agreed upon instructional objective (assessment of student performance such as test, essay or other efforts or students behavior); and Data analysis focus on students mastery of content. Examine teacher strategies which contributes or obstruct attainment of objective (Anecdotal record).

c) Teacher Concern Model Identify observation instrument based on teachers concern Pupose of Instrument: Verify & demonstrate to the teacher events in the classroom relating to the teachers concern Verify the impact of a new teacher action on teachers original concern.

SUMMARY Accurate observation data are objective and representative of events in the classroom Data collection procedures range from the widest (educational criticism) where the observer is fully immersed in the classroom setting and presents the perception to the participants Anecdotal technique captures objective data on teacher and students behavior without interpretation, favored by neo-traditionalists for assessing and assisting teachers

Verbatim or selective verbatim records verbal behaviors Observation systems is the narrowest lens. Examined specific behaviors that can be quantified Collected data are analyzed by the supervisor; teacher; or teacher and supervisor during post conference Analyzed data will indicate effective or ineffective teaching behaviors

Neo-Traditional Analysis looks for indicators of effective behaviors; instructional objectives analysis looks for evidence of objective achievement Supervisor must write observation report (assessing or assisting) immediately after post conference (with teachers collaboration) Report must be brief with specific supporting data.

REFERENCE Observing Classroom Teaching: Data Collection and Analysis. (Reference: Chapter 8 in Tracy, S. J., MacNaughton, R. (1993). Assisting and Assessing Educational Personel. The Impac of Clinical Supervision. Boston: Allyn and Bacon