1. Copyright © Allyn & Bacon 2007 An Evidence-Based Approach to the Practice of Educational Leadership First Edition Ronald W. Rebore Angela L. E. Walmsley Saint Louis University This multimedia product and its contents are protected under copyright law. The following are prohibited by law: Any public performance or display, including transmission of any image over a network; Preparation of any derivative work, including the extraction, in whole or part, of any images; Any rental, lease, or lending of this program.

2. Copyright © Allyn & Bacon 2007 Chapter 5: Developing Best Practice through Research

3. Copyright © Allyn & Bacon 2007 Instructional Objectives To learn how to create a culture of evidence in a school district that will enhance the decisions within a school To provide information that can be used to engage stakeholders in discourse, which will strengthen the data collection and decision making within a school To understand the value and importance of using research in administrative decision making

4. Copyright © Allyn & Bacon 2007 Instructional Objectives To understand the various types of research available for use, including the broad categories of quantitative and qualitative research To understand the value of action research and program evaluation in school settings, and to be able to perform both types of research

5. Copyright © Allyn & Bacon 2007 Instructional Objectives To understand ethical implications in conducting research in a school setting To understand the steps needed to complete an applied research study and develop an action plan To gain knowledge about the values of meta-analysis

6. Copyright © Allyn & Bacon 2007 Introduction To develop best practice, an administrator must have research and data available to make good decisions. Administrators must: make valid decisions based on evidence, understand how research is collected, and understand how research can be used to create a plan that can be fine-tuned based on new data.

7. Copyright © Allyn & Bacon 2007 Introduction to Research Accountability The general public, the government, and the stakeholders involved all want documentation that a district is performing well. It has become imperative that teachers and administrators become researchers in order to provide thorough analyses of data pertaining to the students.

8. Copyright © Allyn & Bacon 2007 Types of Research Quantitative: statistical analysis of numerical data through a deductive approach –Benefits include: large sets of data can be easily collected in a relatively short amount of time with a relatively low budget, and offers “hard data”

9. Copyright © Allyn & Bacon 2007 Types of Research Qualitative: analysis of data described in words through an inductive approach –Benefits include: may dig deeper into the thoughts of individuals, helpful when little is known about a specific issue and finding past evidence is difficult It is imperative that administrators be able to use both quantitative and qualitative research methods.

10. Copyright © Allyn & Bacon 2007 Types of Research Historiography: the study of historical events This method critically analyzes past events by studying a variety of material to search for a connection between the past and present.

11. Copyright © Allyn & Bacon 2007 Types of Research Ethnography: typically involves understanding the culture of a classroom, a grade level, a building, or a district “Ethnography is traditionally or classically defined as the discovery and comprehensive description of the culture of a group of people” B. Johnson and L. Christensen, Educational Research: Quantitative, Qualitative, and Mixed Approaches (Boston: Allyn & Bacon, 2004), p. 369.

12. Copyright © Allyn & Bacon 2007 Types of Research Case Study and Group Study: focusing a research question or issue on an individual or a specific group of people. The greatest detriment to case study analysis is that the information lacks generalizability, often only applying to one student.

13. Copyright © Allyn & Bacon 2007 Types of Research Action Research: research while action is taking place Action research results in solid research that can be completed by busy administrators and teachers. Action research provides the solid documentation to make curriculum or educational changes as necessary.

14. Copyright © Allyn & Bacon 2007 Types of Research Program Evaluation: examines a specific program to decide its direction or worth Similar to action research as it is typically completed during the regular course of the work day Program evaluation is the most crucial type of research performed by administrators because it provides evidence for all types of programs, which directly affect the school or district budget.

15. Copyright © Allyn & Bacon 2007 Types of Research Experimental and Nonexperimental Research: –Nonexperimental: a study where no changes are made to participants –Experimental: usually involves two groups, control and experimental. The control group receives no change in treatment, while the experimental group receives a treatment change.

16. Copyright © Allyn & Bacon 2007 Types of Research Ethics in Research While groups may receive different treatments in teaching during research, it is imperative that both groups receive quality teaching despite the differences. Students cannot be harmed when participating in school activities.

17. Copyright © Allyn & Bacon 2007 Types of Research If a study may be perceived as harmful in any way, the informed consent of the students and guardians must be obtained. Teachers may perform action research studies within the context of their duties. Any other kind of study besides action research must be discussed with and approved by an administrator.

18. Copyright © Allyn & Bacon 2007 Applied Research In order to clearly provide examples of applied research, the following steps can be taken: 1.Defining the problem or issue 2.Framing research questions 3.Designing research projects 4.Evaluating research results

19. Copyright © Allyn & Bacon 2007 Applied Research Documentation will be easiest if a definition of the problem can be established. Specific research questions allow the administrator to focus on data collection for those questions. These questions allow the administrator to decide in what direction to go when designing the project.

20. Copyright © Allyn & Bacon 2007 Applied Research Defining the Problem or Issue Begin by thinking about problems or issues that may arise or have arisen in the school.

21. Copyright © Allyn & Bacon 2007 Applied Research Framing Research Questions Develop one or more specific research questions that the study is intended to answer. These questions should be narrow, yet defined enough for an effective study to be designed.

22. Copyright © Allyn & Bacon 2007 Applied Research Designing Research Projects Data collection may include: –Interview questions –Observation –Collection of test results to analyze The method of data collection must be matched with the research question.

23. Copyright © Allyn & Bacon 2007 Applied Research A quasi-experimental design can be used when participants cannot be randomized. Types of design include: –Comparison group –Pretest-posttest –One-group, pretest-posttest

24. Copyright © Allyn & Bacon 2007 Applied Research Designs categorized by treatment: –A-B-A: baseline is recorded, treatment is imposed, and baseline is recorded after the treatment has shown change. Johnson and Christensen, Educational Research, p. 313 –A-B-A-B: experiment ends with the changes made implemented Results are always impacted by external factors, such as natural maturation.

25. Copyright © Allyn & Bacon 2007 Evaluating Research Results Quantitative study: analysis is in the form of statistical testing to analyze the original hypothesis. Qualitative study: data must be coded and analyzed by looking at responses from surveys, transcribing interviews, and analyzing notes from document analysis.

26. Copyright © Allyn & Bacon 2007 Evaluating Research Results Selecting and Using Research Computer Software Software available for analyzing qualitative data: ATLAS, the Ethnograph, and NUD*IST (Nonnumerical, Unstructured Data Indexing, Searching, and Theorizing) Ibid, p. 520. These program aid in storing and organizing data.

27. Copyright © Allyn & Bacon 2007 Evaluating Research Results Quantitative data programs will complete a data analysis, but a qualitative data program only organizes the data to be analyzed by the researcher. Quantitative data is more easily analyzed when using a computer program, such as SPSS (Statistical Package for the Social Sciences).

28. Copyright © Allyn & Bacon 2007 Developing an Action Plan An action plan does not deal with problems, but is a plan of action for any program within the school. An action plan is one of the most powerful ways an administrator can use research. It is always supported by evidence and can be changed year to year.

29. Copyright © Allyn & Bacon 2007 Developing an Action Plan There are six steps in developing an action plan: 1.Establishing data-driven goals and objectives 2.Developing a timeline 3.Choosing assessment instruments 4.Showing measurable evidence 5.Redefining goals and objectives 6.Communicating the plan

30. Copyright © Allyn & Bacon 2007 Developing an Action Plan Establishing Data-Driven Goals and Objectives Action plans can be written so an evaluation of the action can be performed. An administrator must create organized and data driven plans and goals for the school.

31. Copyright © Allyn & Bacon 2007 Developing an Action Plan The purpose is not to “explore” an issue, but to describe what needs to be achieved. The evaluation of an action plan is typically performed for a specific reason: to see if the goals and objectives were met.

32. Copyright © Allyn & Bacon 2007 Developing an Action Plan Developing a Timeline The project should be designed in order for a person to be able to collect and analyze the data within a timeframe. The timeline should begin with dissemination of the plan to those involved.

33. Copyright © Allyn & Bacon 2007 Developing an Action Plan The timeline should also include time for action to take place and time intervals that data will be collected. It should include the ending date for both data collection as well as a final evaluation to be written and provided to those involved.

34. Copyright © Allyn & Bacon 2007 Developing an Action Plan Choosing Assessment Instruments The assessment instruments must match the goals and objectives. Every goal and objective should have one or more data collection suggestions to show whether that goal and objective is met.

35. Copyright © Allyn & Bacon 2007 Developing an Action Plan Numerical data, or “hard data”, is preferred by evaluators due to ease of analysis and lack of bias. Interview or observation data is also called “soft data”. Both are useful when documented. No matter what types of data are collected, the data must show evidence.

36. Copyright © Allyn & Bacon 2007 Developing an Action Plan Showing Measurable Evidence Measurable evidence is any official data collected with as little bias as possible about the particular goal or objective. It may or may not be favorable towards the action plan, but will help the administrator make new decisions within the action plan.

37. Copyright © Allyn & Bacon 2007 Developing an Action Plan Redefining Goals and Objectives The final part of the plan is to revaluate parts or the entire action plan, and to redefine goals and objectives for the plan. This leads to next year’s action plan.

38. Copyright © Allyn & Bacon 2007 Developing an Action Plan Communication of the Plan Communication of the plan should include eventual documentation of results and the redefined goals and objectives. Administrators must communicate results and actions based on evidence from an action plan to all stakeholders.

39. Copyright © Allyn & Bacon 2007 Developing an Action Plan Administrators may find the development and implementation of an action plan easier by creating a matrix showing crucial step in the action planning process: Goal / Objective Planned Activity TimelineData Collected Outcome

40. Copyright © Allyn & Bacon 2007 Developing an Action Plan An administrator must be able to use both applied research and action plans to make decisions and lead the school. An administrator must also expect teachers to be able to use action research or applied research in the context of their teaching.

41. Copyright © Allyn & Bacon 2007 Understanding Meta-Analysis Meta-research designs allow the administrator to interpret various similar studies together to make an effective decision. Meta-analysis is a useful way to interpret and analyze multiple studies together. An educational statistical consultant would likely perform a meta-analysis, but an administrator must understand the importance of using meta- analysis for decision making.

42. Copyright © Allyn & Bacon 2007 Discussion Questions and Statements 1.What are the four steps to an applied research model? 2.Compare and contrast qualitative and quantitative research methods. 3.Explain the differenced between action research and program evaluation. 4.Discuss why the issue of accountability in educational setting is appropriate currently.

43. Copyright © Allyn & Bacon 2007 Discussion Questions and Statements 5. What is the importance of developing a timeline in a research study? 6. What steps are involved in designing an action planning matrix? 7. What is the difference between summative and formative evaluations? 8. Explain how meta-analysis can be used in educational research.