School Counselors Doing Action Research Jay Carey and Carey Dimmitt Center for School Counseling Outcome Research UMass Amherst CT Guidance Leaders March 12, 2004

What is Action Research? ► Action research is a valuable and powerful way to use data to impact educational practice. ► Systematically collecting information about what is happening, evaluating the effectiveness of interventions, and disseminating the results helps school counselors become more intentional and thus more successful in helping all children learn.

Action Research Process ► Action research allows school counselors to evaluate programs and interventions in their specific contexts. ► Practices can change immediately and in an ongoing manner as the data is collected and analyzed.

Action Research Process 1. Identify a problem 2. Review what is known about the problem 3. Develop research hypotheses or questions about the problem and plan the research process accordingly 4. Gather the data 5. Analyze the data 6. Interpret results; disseminate and use findings 7. Evaluate the research process

Conducting Action Research: Identify the Problem Conducting Action Research: Identify the Problem 1. Identify the problem and determine the purpose of the research. Ask yourself:  What is the problem I want to solve?  What is the question I want to answer?  What do I wish was different?

Conducting Action Research: Identify the Problem ► Define the problem/question in clear, specific language. The question may be general or specific.  “Does the new Math Lab increase math grades or math test scores?”  “Which guidance curricula are having the greatest positive impact on student behavior?”  “Does a study skills intervention impact 7 th grade failure rates?”  “Does the Career Exploration Guidance Curriculum unit improve student knowledge of career choices?”

Conducting Action Research: Identify the Problem ► Purpose of the study: Once the research question is identified, consider what you will do with the solutions and/or answers to your question.  What educational practices would you like to impact?  What student outcomes are you hoping to influence?  What decisions will the study help you make?

Conducting Action Research: What is Already Known? 2. Review what is known about the problem.  Has anyone in the school asked this question before?  Who might have information?  What is the relevant research in professional journals?  What does an internet search find on this topic?

Conducting Action Research: What is Already Known? ► Evaluating findings from research literature and internet searches:  What is the source? How reliable is it?  What are the strengths/weaknesses of the research design, sampling, effect size, measures used, treatment fidelity, researcher bias, instrument reliability and validity?

Conducting Action Research: Develop Hypotheses Conducting Action Research: Develop Hypotheses 3. Develop research hypotheses or questions about the problems and plan the research process accordingly.  Ask yourself what you think the answer(s) to your question(s) will be.  Identifying your hypotheses helps you identify your biases, which may impact your process.  What is the opposite of your hypothesis (the “null hypothesis”)? What would the data look like if you were wrong?

Conducting Action Research: Gather Data Conducting Action Research: Gather Data 4. Gather the data.  What information do you need in order to answer your question or solve your problem?  Use multiple sources of data, or multiple outcome measures, wherever possible.  Decide whether you need to consider student achievement data, psychosocial data, career data, school data, process data, perception data, and/or results data.

Conducting Action Research: Gather Data ► Where is the data?  Does it already exist? ► School records, Teacher records  Will you generate your own data? ► Surveys ► Interviews ► Observations

Conducting Action Research: Gather Data ► How accurate is the data you’ve chosen to use? What’s missing? ► Multiple data sources help you more accurately get at the complexity of a situation, whereas one measure or data source will give you a snapshot view.

Conducting Action Research: Gather Data ► Select and/or develop the instruments you will use to gather the data. Possibilities include:  Surveys  Tests (of achievement, aptitude, attitude, etc.)  Behavioral checklists or observations  Performance assessments  interviews

Conducting Action Research: Gather Data ► Reliability and validity need to be considered when selecting instruments. Does it measure what it’s supposed to? Have others used the instrument and found it to be reliable and valid? ► Just because an instrument exists doesn’t mean it’s well done!

Conducting Action Research: Gather Data ► Identify and follow ethical and legal standards:  No participant should be exposed to physical or psychological harm.  Permission to use confidential data must be obtained.  Participation in a study is always voluntary.  Participants may withdraw from the study at any time.  Participants’ privacy rights must be respected.

Conducting Action Research: Gather Data ► Identify the group/sample to be studied:  Ideally either the entire sample is involved in the study (the class, grade, or school) or the group studied is a random sample of the population.  Stratified sampling uses a smaller sample which has the same proportions as the larger sample.  Systematic random sampling is when every x number of students is chosen from the whole population (every 3 rd, or every 4 th for example).

Conducting Action Research: Gather Data ► Often there are context constraints which limit the sample size. ► One way to strengthen the findings is to compare the group who received the intervention with a comparable group who didn’t receive the intervention.

Conducting Action Research: Gather Data ► Once data sources and measures are identified, ethical standards are considered, and the sample is identified, data can be gathered!

Conducting Action Research: Analyze the Data 5. Analyze the data:  Before data can be analyzed it may need to be edited, coded and organized for analysis.  Data is often entered into a software program such as Excel or SPSS.

Conducting Action Research: Analyze the Data ► Descriptive statistics describe the data and can provide information about how a group has changed over time:  Measures of central tendency ► Mean, median, mode  Measures of variability ► Variance, standard variation, range  Measures of relative standing ► Percentile rank

Conducting Action Research: Analyze the Data ► Inferential statistics provide additional information:  Looking carefully at how a group changes over time ► Use t-tests or Chi-Square  Looking at differences between control and intervention groups ► Use t-tests or Chi-Square  Looking at differences among more than 2 groups ► Use Analysis of Variance (ANOVA)

Conducting Action Research: Analyze the Data ► Finding parent, colleague, or university partner(s) who already know how to input and analyze data can make this part of the research process easier.

Conducting Action Research: Interpret Research Results 6. Interpret research results, disseminate, and use findings to inform practice. Develop an action plan.  What do the results of your analysis mean?  What did you find out?  Were your hypotheses correct?

Conducting Action Research: Interpret Research Results ► How can the results be communicated so they most effectively impact behavior or understanding?  Decide what’s relevant to each audience.  Present all relevant results even if they don’t support your hypotheses.  Relate your findings to the purpose of the study, your hypotheses and previous research.

Conducting Action Research: Interpret Research Results ► One of the distinguishing features of Action Research is that its primary goal is to take action based on the research findings. Some options include:  Make recommendations that will resolve the problem.  Make plans and decisions about interventions based on the findings.  Make program plans based on the findings.  Develop action plans based on the findings.

Conducting Action Research: Interpret Research Results ► Action research seeks to use research-based knowledge and findings to actively change behavior. The ultimate question is: what will you do that is similar and what will you do that is different based on what you learned in your research?

Conducting Action Research: Evaluate the Research Process 7. Evaluate the Research Process with all involved:  What would we do differently next time?  Does the instrument need revising?  Did the research generate the information we wanted?  What new questions did the research generate?

Conducting Action Research: Evaluate the Research Process ► Mills (2000) and Kemmis and Wilkinson (1998) suggest the following questions:  Does the project clearly address a problem or issue in practice that needs to be solved?  Did the researcher collect sufficient data to help address the problem?  Did the plan of action build logically from the data?  Did the action research actually lead to a change or did a solution to a problem make a difference?

Conducting Action Research: Evaluate the Research Process ► Asking participants what they learned from their involvement empowers them to think of themselves as active learners in an environment where evaluation is something to be used supportively for growth rather than something to be feared.