Collecting and Analyzing Data

Collecting data First step in the acting stage of action research 2 types of data: –Qualitative – narrative – words –Quantative

Qualitative Data Descriptive/Narrative accounts – words Examples Interview transcripts Observational notes Journal entries Transcriptions of audio/video tapes Existing documents, records, reports

Observations Carefully watching and systematically recording in a particular setting –What is seen –What is heard Example: –checking student nonverbal reactions to something occurring in classroom –Understand how students interact and communicate in small group work

Observations Structured – look for specific –Behaviors –Reactions –interactions Semi-structured and Unstructured –Free flowing –Allows teacher to shift focus from one event to another as new events arise Leedy & Ormrod, 2005

Advantages of Observations Gather data about actual student behavior Teacher sees things that student might not be able to report on himself Using video tape allows teacher to see more than normally would with own eyes Schmuck, 1997

Limitations of Observations Presence of teacher (with notebook, pencil, video camera) can change student behavior May have to wait for extended periods of time to observe desired behavior Different observers may see different things even while observing the same event Schmuck, 1997

Fieldnotes Written observations of what see taking place in classroom Stop thinking – just write what you see Observing + recording = begin to focus on things that are interesting or important Observations over time = patterns emerge Johnson, 2005

Fieldnotes Leedy & Ormrod, 2005; Bogdan & Biklen, 1998 Record actual observations Note preliminary interpretations of what observed - (observer’s comments) Often shed light on emerging patterns from observational data What you think the observed event means. Interpretations may change over time as you collect more data Record of changing interpretations is invaluable in interpreting and reflecting on the project

Interviews Conversations between researcher and participants –Individual –Group Interview guide – specific or general questions to be asked – prior to interview Structured, semi-structured, open-ended

Interviews Structured –Specific set of predetermined questions –Only those questions are asked of each person being interviewed –= consistency

Interviews Semi-structured –Base questions –Follow up questions –Optional questions Keep questions –Brief –Clear –Stated in simple language Johnson, 2005; Schwalbach, 2003

Interviews Open-ended –Few questions –Very broad in nature –Intent = gather different kinds of information from different individuals Example: –What does “school” mean to you? –What do you like about school? Johnson, 2005; Schwalbach, 2003

Interviews Focus group –Simultaneous interview of people –Usually no more than people –1 – 2 hours in length Each participant should have an opportunity to speak and share perspective Johnson, 2005; Schwalbach, 2003

Interviews interviews –Series of questions in message –Reply = transcription of answers –Lack anonymity and confidentiality Informal –May be spontaneous –Take place throughout data collection process –Can be part of the daily interactions with students in classroom Johnson, 2005; Schwalbach, 2003

Data journals Provide info about workings of classroom Sense of daily thoughts, perceptions, experiences Narrative account of professional reflections on practice Reflect on feelings and interpretations associated with observations

Existing documents and records Curriculum materials, textbooks, instructional manipulatives, attendance records, test scores, previous grades, discipline records, cumulative folders Attendance rates, retention rates, graduation rates, newspaper stories about school events, minutes from faculty or school board meetings, standardized test scores (disaggregated by grade level, gender, ethnicity)

Existing documents and records Record data on a common data form Develop a form for your specific purposes to compile data Classroom artifacts –Written or visual source of data –Contained within the classroom –Contribute to understanding of what is occuring Example: –Student portfolios and/or products

Accuracy, credibility, dependability Ensure quality of data – precise, accurate Validity – extent to which data was collected accurately and measures what you purport to measure Trustworthiness – accuracy, believability of data Credibility – believable from perspective of participant in research Dependability – need for researcher to account for changing context within which the research occurs – researcher must describe changes that occur and how changes affected the way the research approached the study.

Analysis To break something down into its component parts so that it can be understood

Action Research Analyzes data Organizes data into categories Purpose: so that others can understand the reality you are trying to represent

Elements of Data Analysis Accuracy and credibility Validity, reliability, triangulation Inductive analysis

Accuracy and Credibility “This is what is” Accuracy = data you collect creates fairly true picture of the reality you are observing –Helps make decisions for your situation Credibility = trustworthy or capable of being believed –Use of data with confidence by self and others

7 Tips to Establish Accuracy and Credibility Record your observations carefully and precisely –Double check to make sure you are recording exactly what you are seeing

7 Tips to Establish Accuracy and Credibility Describe all phases of data collection and analysis –While recording, recount all the steps used in collecting and analyzing data –In reporting you want to create a level of clarity so that another person can duplicate your steps

7 Tips to Establish Accuracy and Credibility Make sure you record and report everything that is of importance –Do not omit data that may be counter to what you believe –Goal is to understand fully all aspects of what you are observing

7 Tips to Establish Accuracy and Credibility Be as objective as possible in describing and interpreting what you see –Pronounced biases or hidden agendas are easily spotted prevent seeing all aspects of what your are trying to study make AR less accurate and credible

7 Tips to Establish Accuracy and Credibility Use enough data sources –Similar patterns found from two or more forms of data = accurate + credible

7 Tips to Establish Accuracy and Credibility Use the right kinds of data sources –Should provide most accurate understanding possible of your topic –Example: middle school students use of humor Interview classroom teachers (only one perspective) Audiotapes of actual conversations Observing middle school students/descriptive field notes Student survey – self report kinds of humor used and instances when used

7 Tips to Establish Accuracy and Credibility Look long enough and deep enough –Long observations provide more chances to see and confirm patterns –Your question dictates length of observation and data collected –Final criterion = “Have you created an accurate picture of what was studied?”

Validity Reliability Triangulation

Validity The degree to which a thing measures what it reports to measure

Validity Example: assess writing ability of elementary students –Standardized test of grammar/punctuation = read series of questions/choose one of 4 responses –Accurate results quantified and comparable –Problem: doesn’t look at writing in authentic context = not valid measure of student ability to organize and communicate ideas in writing –Valid assessment = actual writing samples assessed with form that rates specific elements

Reliability Degree to which a study or experiment can be repeated with similar results Traditional experimental research = –Controlled variables –X causes Y beyond all doubt each time –Results generalize to similar situations

Reliability Action research = –Messy real world events –Humans are unpredictable –Expect to see different things each time we research –Repetition = repeating patterns and themes Noticing recurring items, themes or patterns emerging from data –Findings are not generalized broadly –Help us: Understand particular situations Inform similar situations

Triangulation Collecting more than one form of data and Looking at something from more than one perspective Interview with Susan Observations of Susan Susan’s questionnaire responses

Triangulation Ensures you are seeing all sides of a situation Provides greater depth and dimension Enhances accuracy and credibility

Triangulation achieved by: Collecting different types of data Using different data sources Collecting data at different times Having other people review your data to check for accuracy Adjust your findings

Inductive Analysis To look at a field or group of data and try to induce or create order by organizing what is observed into groups or categories.

Look for: Recurring items Themes Patterns Code similar things into initial categories Categories = flexible to allow for change as data collection progresses Categories that form initially inform further data collection

Categories: Define each category Describe each category See examples on page 78 – 81 of Methods of Analyzing Data under Inductive Analysis: Resources for Inductive Analysis On the Course Documents Page

Deductive reasoning Adapted from illiam M.K. Trochim 2002

Deductive reasoning is more narrow in nature and is concerned with testing or confirming hypotheses.

Deductive reasoning works from the more general to the more specific Sometimes this is informally called a "top- down" approach.

Begin with a theory about our topic of interest Narrow into more specific hypotheses that can be tested Narrow further by collecting observations to address the hypotheses. Leads to testing the hypotheses with specific data = confirm (or not) original theories.

Inductive reasoning Adapted from illiam M.K. Trochim 2002

Inductive reasoning, by its very nature, is more open-ended and exploratory, especially at the beginning.

Inductive reasoning works the other way, moving from specific observations to broader generalizations and theories.

Informally, we sometimes call this a "bottom up" approach

Inductive reasoning: –specific observations and measures –detect patterns and regularities, –formulate some tentative hypotheses for exploration –Develop some general conclusions or theories.

Inductive Analysis Exercise Randomly generate a list of 25 nouns (on board)

Inductive Analysis Exercise In small groups, put order to this field by moving them into groups or categories Describe your field in terms of categories and numbers in each category