IRIS CENTER Learning Outcomes for IRIS Online Modules Used in College Courses Project # H325F OSEP Project Directors’ Conference Washington, DC July 21, 2010

IRIS CENTER What is IRIS? IDEA ‘04 and Research for Inclusive Settings National Center for Training Enhancements Provides free, online resources for pre-service training and professional development

IRIS CENTER The Array of IRIS Materials Includes: Modules Case Studies Other Resources Information Briefs Student Activities Film Search Tool Module Media Search Tool Podcasts On-Line Dictionary

IRIS CENTER The STAR Legacy Cycle and IRIS Modules Grounded in learning sciences (How People Learn theory)

IRIS CENTER Three Studies Study 1: Do pre-service students learn course content when using IRIS modules? Study 2: Is there a difference in student learning outcomes for instructor-led (high HPL) versus independent (low HPL) module use? Study 3: Can the outcomes of Studies 1 and 2 be replicated with other modules?

IRIS CENTER Study 1 Procedures Success in Sight module Control: Only the textbook chapter on visual disabilities. Experimental: Textbook chapter on visual disabilities + module as independent homework. Pre/posttest: IRIS Challenge video about two students with visual impairments followed by three open-ended questions that addressed Instructional Considerations, Setting Up the Classroom, and Assistance with Instruction.

IRIS CENTER Study 1 Subjects n = 659

IRIS CENTER Fall 2004 Winter 2005 Spring 2005 Summer 2005 Fall 2005 Instructor 1O 1 O 2 O 1 X E O 2 Instructor 2O 1 O 2 O 1 X E O 2 Instructor 3O 1 O 2 O 1 X E O 2 O 1,O 2 = Pretest, posttest administered X E = Experimental condition – use of module as independent assignment Study 1 Design

IRIS CENTER Open-ended Questions What advice would you give Ms. Milton about preparing for Evan and Emily? Include issues surrounding: 1.Setting up her classroom – List as many considerations as you can; be as specific as possible. 2.Instructional considerations - List as many as you can; be as specific as possible. 3.Types of assistance she can anticipate with Evan and Emily’s education - List as many as you can; be as specific as possible.

IRIS CENTER Sample Scoring Rubric

IRIS CENTER Online Learning Outcomes Two types: 1)Enhance course content 2)Replace course content

IRIS CENTER Instructor A Instructor B Instructor C Pre and Posttest Means on Instructional Considerations Content: Textbook and Module No ModuleModule *Difference between posttest means p < , ES +.90 Enhancing Course Content

IRIS CENTER Instructor A Instructor B Instructor C Pretest and Posttest Means on Instructional Considerations Items: Module Only No Module Module *Difference between posttest means p < 0.001, ES Replacing Course Content

IRIS CENTER Results and Implications Students in the module condition outperformed students in the no module condition – Confirmed the value-added benefits of the module as a course enhancement. Modules are an effective method to convey and reinforce knowledge to pre-service students.

IRIS CENTER Grounded in learning sciences (How People Learn theory) The STAR Legacy Cycle and IRIS Modules

IRIS CENTER Study 2 Is there a difference in student learning outcomes for instructor-led (high HPL) versus independent (low HPL) module use?

IRIS CENTER Study 2 Procedures SOS: Helping Students Become Independent Learners module Instructor-enhanced condition: Viewed module in class, discussed initial thoughts as a group, worked module independently, discussed final thoughts as a group. Independently-viewed condition: Did module as homework, no instructor interaction. Pre/posttest assessed students’ factual knowledge, responses to clearly defined problems, and responses to ambiguously (or ill-) defined problems.

IRIS CENTER Sample Questions Factual: Which of the following statements best summarizes the concept of self- regulation? a) Self-regulation is a type of learning strategy that utilizes components of behavioral intervention and academic skill building to improve school outcomes for students with disabilities. b) Self-regulation is a learner-directed strategy in which students use self-talk and goal setting to improve academic skills. c) Self-regulation is a teacher-directed strategy that helps students monitor self- made behavior goals. d) Self-regulation is a student-directed strategy that promotes the ability to improve both educational and behavioral outcomes.

IRIS CENTER Sample Questions Well-defined: Teaching students how to manage their own behavior is not effective for all types of problem behaviors. From the list below, select two problems that are MOST likely to be improved by students learning to manage their own behavior. Provide a clear explanation for each choice. Vaguely-defined: Watch video of classroom. Identify the problem behavior displayed by the boy wearing the blue shirt. As a teacher, what strategy(ies) would you use to change his behavior? Explain the reasons for your choice. List the specific steps you would take to implement the strategy(ies) you chose.

IRIS CENTER O 1,O 2 = Pretest, posttest administered X = Type of module use Study 2 Design Spring 2006Fall 2006Spring 2007 Condition Instructor enhanced O 1 X IE O 2 Independently viewed O 1 X IV O 2 Posttest only O 2

IRIS CENTER

Results/ Implications There were no significant differences in learning outcomes between instructor-led and independent module use. Instructors can use the modules outside of class (as homework) and still anticipate positive learning outcomes.

IRIS CENTER What does this mean for college instructors? IRIS modules continue to produce significant learning outcomes for college students in pre-service courses. The modules are just as effective when used as independent homework as when they are used in class settings.

IRIS CENTER Study 3 Now that we have demonstrated that the IRIS modules are effective in teaching the material And that they can be used effectively as independent assignments Can we replicate these findings with two of our most popular modules?

IRIS CENTER Study 3 Procedures Three instructors taught an introductory special education course (n=216) 2 instructors taught in the traditional lecture setting 1 taught the course at a distance (online) All 3 instructors agreed to use the same 2 IRIS modules as part of the course requirements: RTI (Part 1): An Overview and Addressing Disruptive and Noncompliant Behaviors (Part 1): Understanding the Acting Out Cycle

IRIS CENTER Procedures (cont.) During the first week of class, the pre/posttest was administered The pre/posttest presented 2 scenarios: one related to RTI and the other related to the acting out cycle All students responded to both scenarios as a pretest Two instructors (1 traditional and 1 distance) assigned the RTI module; the other instructor assigned the module on the acting out cycle Following the completion of the module assignments, all instructors asked students to complete the pre/posttest

IRIS CENTER O 1,O 2,O 3 = Pre/posttest administered X = Module used Study 3 Design Time 1Time 2Time 3 Order 1 (RTI, BI1) (n=123) O1O1 X RTI O2O2 X BI1 O3O3 Order 2 (BI1, RTI) (n=52) O1O1 X BI1 O2O2 X RTI O3O3

IRIS CENTER Pre/posttest Scenario 1: Presented a situation based on a hypothetical school. Respondents were asked to describe the overall RTI process. Scenario 2: Presented a situation in which a student was acting out as his class work became more difficult. Respondents were asked how the teacher could effectively address the student’s behavior without causing it to escalate.

IRIS CENTER Pre/posttest (cont.) Students were given 15 minutes to respond to both scenarios The instructions encouraged students to bullet their responses versus writing complete sentences.

IRIS CENTER O 1,O 2, O 3 = Pretest, posttest administered X = Module used Study 3 Design Time 1Time 2Time 3 Order 1 (RTI, BI1) O1O1 X RTI O2O2 X BI1 O3O3 Order 2 (BI1, RTI) O1O1 X BI1 O2O2 X RTI O3O3

IRIS CENTER Expected Performance

IRIS CENTER Pre and Posttest Means on RTI and BI1 Items

IRIS CENTER Distance versus Traditional Instruction

IRIS CENTER Results/New Directions Students effectively learn content from independent use of IRIS modules in both traditional and distance learning settings. Lingering questions when BI1 was completed first New research question: How can students’ learning be scaffolded so that they recognize when and how to apply new knowledge?

IRIS CENTER Summary of IRIS Web Site User Data

IRIS CENTER IRIS Center: for Technical Support Peabody College Vanderbilt University (800) Toll Free for Training Claremont Graduate University (CGU) (866) 626-IRIS (4747) Toll Free