Year One Evaluation Outcomes Presented by Kathleen Rhoades Boston College Research Team Year One Evaluation Outcomes

Cross State Survey Responses New Hampshire = 3 This table is based on the number of participants in all 8 states during year 1. NH is state #3. Of the 55 total people who took OPEN NH courses, only 23, less than half, took both the pre and post course surveys. The evaluation statistics are only based on those who answered the applicable questions in whichever surveys applied. Data Collected: June 1 to December 31, 2006

Background Variables Race The participants who completed both surveys are predominantly white. This was also true for 7 of the 8 states in the e-Learning Initiative. Background Variables Race Cross-State New Hampshire

Background Variables Certification & Previous Experience/Course Content NH is very similar to the other states in the e-Learning initiative. Most teachers taking our courses are certified. They say they never took a professional development course in the same subject matter as their course. Cross-State Cross-State

Background Variables Students Receiving Free/Reduced Lunch NH teachers claimed that less than 50% of our students receive free/reduced lunch. It was expected that the majority would say that more than 50% do. Background Variables Students Receiving Free/Reduced Lunch Cross-State Cross-State Standardized Residuals tells where the differences are. Expected: create based on the total number that teachers report, not what the schools report.

NH teachers claimed that taking the OPEN NH course increased their content knowledge. They claimed they knew more about the content after they took the course than they did prior to taking the course. e-Learning for Educators Impacts Changes in Teachers’ Content Knowledge Pre Survey: Based on the description, how knowledgeable are you in the content before you take the course. Post: Having taken the workshop how knowledgeable are you now.

Changes in Content Knowledge The graph of content knowledge before and after taking a course shows a pattern of increases in content knowledge. Changes in Content Knowledge

Course to Classroom Intentions to Transfer Content All teachers who responded to these questions thought that the content from the course was easily transferable to their classrooms. They also expected they would implement the knowledge gained from the course in the classes they teach. Yes/No answer, so missing means they did not answer.

e-Learning for Educators Impacts Workshop Expectations and Outcomes The pre-survey asked if they expected to accomplish an outcome, and the post-survey asked if they felt they had accomplished it. The post response is given first. Values closer to 1 represent a greater number of “yes” responses. In most cases, the outcome exceeded the expectation. Pre: 16 A – E: What they expected to get out of the course. Post: 3 A – E: What they got out of the course. Pedagogical expectations and outcomes. Check all that apply.

e-Learning for Educators Impacts Technology Changes The bar graph reports technical skills that were improved by taking an OPEN NH online course. While there were marked improvements noted in navigating websites and in participating in threaded discussions, teachers reported improvement in all these skills.

Course Quality Ratings The mean of 4.17 overall rating given by New Hampshire teachers of the courses they took fell between “Very Good” (4) and “Excellent” (5) on a 5 point scale. Course Quality Ratings

Course Quality Ratings This bar graph shows how NH teachers reported the effectiveness of various components of the course they took. All components rated between “Effective” (3) and “Very Effective” (4) on a 4 point scale. Course Quality Ratings

Facilitator Quality Ratings The graph represents the frequency of instructor quality ratings on a 4 point scale. The majority either “Agreed” (3) or “Strongly Agreed” (4) with options given concerning the quality of their instructors, resulting in a mean instructor rating of 3.5. Facilitator Quality Ratings Post: 29-31 Average of three questions that look at the facilitator quality.

The participants rated the overall quality of their course as just above “Very Good” (4) on a 5 point scale. They rated the success of their instructor as just above “Successful” (3) on a 4 point scale. e-Learning for Educator Course & Facilitator Quality (Cross-State Charts) Scale: 1=VUnsucc., 2=Unsucc., 3=Succ., 4=VSucc. Scale: 1=Poor, 2=Fair, 3=Good, 4=VGood, 5=Excell.