Data Analysis

Types of Validity Internal validity degree to which results are true for the participants External validity degree to which results can be generalized beyond the participants in the study Test validity Degree to which a test or assessment measures what it is supposed to measure

Validity Considerations Type of ValidityQuestions Truth-valueIn what ways can I ensure my results are accurate and truthful? Applicability/transferabilityHow might my results be useful beyond my particular classroom and with other students? Consistency/dependabilityWould I get the same results with other students similar to mine? Neutrality/confirmabilityHow do I show results are indicative of what actually occurred during the study and do not reflect my personal desires or biases? Democratic validityWhich stakeholders should collaborate or have a voice in the study? In what ways should I involve them? Outcome validityHow will I use results for continued planning, ongoing reflection, and deepening my personal understanding? Process validityWhat do I need to do to ensure I have looked deeply at the problem so I understand the ways context and processes have impacted results? Catalytic validityIn what ways will processes and outcomes change my practice? Dialogic validityDo I need a critical friend to discuss my study and results with so I can determine whether my planning and interpretations are on track?

Ways to Increase Validity Peer debriefing Discuss study with a colleague not invested in study to gain their insights Persistent and prolonged observations Helps to determine whether the intervention is effective after the newness wears off providing credibility Record data accurately Be sure to include detailed notes to help answer questions Member checks Discuss your interpretations of data to determine whether your findings accurately represent the participants Triangulation of data The use of multiple sources to corroborate findings Researcher bias Define any preconceived ideas you have and note them Engage in ongoing reflective planning Allows a researcher to change data collection strategies based on experiences during the data collection phase.

Obtaining Permission Will protect your participants, school and university Parents will want to know if child is involved in action research Always extend courtesy of obtaining permission from the participants in the study regardless of whether they are children or adults

Example of Consent Form Dear Parents, I will be conducting a study in our classroom to determine [include purpose here]. I am writing to ask permission to use the data I collect from your child during this process. Participation in this study involves only regular classroom activities. You may contact me at any time regarding your child’s participation. The principal of the school has approved this study. The purpose of the study is to [briefly describe purpose here]. The study will take place at [name location] and will last for [give time frame]. During the study, I will collect various forms of data to determine whether [name of intervention] was successful. Possible types of data I will collect include [list data collection strategies such as samples of student work, survey/questionnaires, interviews, observations, test scores, etc.]. Benefits of participating in this study include [describe the benefits]. Only [name(s) of researcher and collaborators] will have access to the data collected in this study. Your child’s participation in this project is strictly confidential. Only [and names of collaborators and/or supervising professor if study is part of a university assignment] will have access to your child’s identity and to information that can be associated to your child’s identify. [If applicable, state when data or documentation will be destroyed]. Use of data from your child is voluntary. You may contact me at any time if you do not wish to have your child’s data included in this study. Please check the appropriate choice below, sign the form and return the form with your child. ____ I give permission for my child’s data to be used in this study. I understand that I will receive a signed copy of this consent form. I have read this form and understand it. ____ I do not give permission for my child’s data to be included in this project. ______________________ _____________________________ ________________ Student’s Name Signature of parent/guardian Date

Data Collection Any form of information, observations, or facts that are collected or recorded Should be collected systematically Plan should be in place prior to collection Describes what data will be collected How, when and how often it will be collected May want to use a calendar or checklist to record when and how data is collected

Types of Data Collection Log or research journal Notebook used to record thoughts and observations related to all parts of research Field notes Written observations of what is taken place in the classroom Checklists A list that specifies certain attributes such as behaviors, traits, assignments, or skills Checked off when attribute is seen Interviews Students respond to a specific set of questions that are prepared in advanced Questions are asked in same manner and order each time to maintain consistency Data retrieval charts Visual organizer used to collect and organize data

Continued… Student products or performances A representative sample of each student’s work at different time periods to provide feel of student work Surveys Allow you to get a variety of information quickly Scores and other Quantifiable Data Scores on tests, homework, grades Attitude and rating scales Provide respondents with a question or statement for which they are asked to select one of several answers that determines the strength of response Five point rating scale Example: 5= strongly agree to 1= strongly disagree Archival data Past scores or grades

Strategies for Data Analysis Interim analysis Ongoing analysis of the data throughout the study Looking at and thinking about data as it is collected and then making changes or additions to strategies if necessary Allows the researcher to make changes to data collection strategies during the study based on problems or questions that arise when analyzing the data

Analysis of Quantitative Data Quantitative data Any data that can be counted or averaged the number of responses for each item Example: number of readers who indicated that they are strong, average, or weak readers –Strong = 3, average=2, weak=1 –Provide each student with a score and then find the percentage of class that felt they were a 3,2, or 1 –Could average scores to find the mean score of the class to look at the average score of how students’ perceived their reading ability

Steps of Analysis for Quantitative Data Reporting Displaying the data and describing what it shows Comparing Looking at various attributes and comparing to see if any differences appear Example: Does gender make a difference in the scores? What does the data show? Example: Sample one as compared to sample two Is there a difference between the two samples? For example, did scores increase based upon what was implemented? (pre and post data) Displaying data Use charts and graphs to visual display data collection Microsoft – analyzing data using excel 2007 tutorial: csubmenuxi6 csubmenuxi6

Steps for Organizing and Analyzing Data Gather together data sources and record information Record data in a grade book, journal or computer spreadsheet Create graphical displays of data Display data so that it is easy to understand Charts, tables, graphs Examine displayed data to determine the best ways to explain results Be sure to accurately describe data and data comparisons Share your analyses and graphical displays with collaborators or peers Colleagues can help you determine whether your analyses, charts, figures, graphs make sense Make notes in your journal about how your analyses can be used to answer research questions Refer often to your research question Make sure to analyze data in ways to help answer the research question

Qualitative Data Analysis The process of making meaning from data sources that can be interpreted in various ways Requires a deeper analysis of data Must first be converted to text Look for patterns or themes within the data

Steps for Analyzing Qualitative Data Data must be written as text Textual data are studied for patterns to determine ways to code the data Textual data and codes are analyzed so that patterns and themes can be developed Answers to research questions are offered based on themes found in the data

Coding Examples (Bogdan and Bilken 2003) Setting/context Provides descriptive information on aspects of the research setting Participants’ perspectives Describes what participants think about certain issues Definition of situation Illustrates participants’ understanding of setting and context Ways of thinking Describes participants’ understanding of self and/or others Processes Explains patterns of behavior over time Activities Illustrates recurring, typical behaviors Events Describes specific or particular nonrecurring, meaningful events Strategies Describes methods used by participants to accomplish certain tasks Relationships Defines social roles and typical behavioral patterns among or between people

Triangulating Data Sources and Drawing Conclusions from Data Triangulation Determining whether data sources corroborate with one another Draw reasonable conclusions based on the interpretation of the data Include the answers to the such questions as –Was the intervention effective? Why or why not? –What would you do differently? –You need to include future plans regarding how you intend to use the results and conclusions of the study to inform your practice as an educator.