World History Hybrid Course – PILOT- Units 1 and 2 Students and Teachers Pre-and-Post Surveys and Online Comments Kavita Mittapalli MN Associates, Inc. February 1, 2011

2 Overview of Data Collection Activities Google documents - 56 responses (majority students; 1-2 teachers); 4 counties completed Pre and Post Surveys- Teachers - total responses - 10 and 6, respectively; 7 counties completed Pre and Post Surveys- Students- total responses- 135 and 77, respectively (as of Jan 5 th ); 5 counties completed SurveyMonkey site was kept open for up to 3 weeks 3 reminder s were sent to non-respondents Winter holidays and late start of pilot in some LEAs caused delays in completion.

3 Data analyses Descriptives and frequencies- demographic questions and questions related technology usage and knowledge and its applications in learning WH (pre) and Level of Satisfaction with Units, Role of technology in teaching WH, and Student engagement (post). Paired t-tests- significant mean differences in the pre-and –post student surveys regarding technology usage and integration, knowledge of WH before and after browsing the units, and comfort levels of technology implementation at work and classrooms. Caveat Low and uneven response rate for the pre-and-post surveys. No pre and post test done for teachers surveys (small and uneven sample) Students worked at their own pace. Classroom environment and teachers’ level of instruction/facilitation greatly varied across the responding LEAs. Pilots conducted in a “non-controlled” environment and context per LEA/classroom not taken into account The results should be considered exploratory and may not be conclusive or generalizable across the LEAs.

Teachers Pre- and –Post Surveys 4

5 Findings Demographics and General questions Pre-survey Avg. about 15 years of teaching experience Majority have a Masters degree 7 out of 10 had not taken an online course before. Those who had taken a course have attended lectures, discussion forum, reflections, and have completed assignments 7 out of 10 had not taught an online OR a hybrid course 80% are in the high intermediate range of technology proficiency and frequently use technology mainly for professional productivity (research, class materials, etc)

6 Challenges using technology to teach WH (pre-survey contd) Having technology that works Limited access: Current funding tends to allow 1 computer for every 4 or 5 students. Many students do not have computers & internet access at home. Compared to other content areas, there seems to be a lack of materials that are dedicated to world history curricula. The level of technological proficiency that students have is lacking. They can perform basic Google searches and use basic functions of Word and Powerpoint, but their skills with productive software seem very limited. Limited ability for linking creativity and the content. Having students who have a wide range of technological abilities. Reliability of wireless network at school

7 PRE Agree PRE Strongly Agree POST Agree POST Strongly Agree I feel comfortable using technology to learn WH 11%78%33%67% Technology can help teachers address the diverse learning needs of students in WH. 64%27%67%17% Using technology in instruction can increase students’ motivation and enthusiasm about learning WH. 63%25%50%33% Use of technology and online resources can enhance a teacher’s understanding of the WH unit/s 29%0%33% Use of technology and online resources can enhance a students' understanding of the unit/s 56%22%67%17% Using technology in the classroom with students can increase overall student scores in WH. 33%11%50%17% Using technology in the classroom with students can improve motivation of students to learn WH. 58%25%50%33%

8 Post- Units 1 and 2 (n=6)- Quality of Content in the Units Unit 1 Agree Strongly AgreeUnit 2 Agree Strongly Agree The pace of the unit was appropriate 67%17%67%17% Unit course material was relevant 0%50%67%33.3% The length and detail of each section was appropriate 33.3% (50% Neither) 17%33.3%17% The unit is sufficiently differentiated 50% 33.3% Neither) 0% 33.3% (50% Neither) 0% The unit was well organized 50%33.3%67%17% Presentation (layout, style, and format) of the material 60%40%75%25% Material was conducive to learning the unit goals and objectives 50%17%50%17%

9 Unit 1 Agree Strongly AgreeUnit 2 Agree Strongly Agree Assessments were related to the material covered 50%33.3%50%17% The type and number of activities per unit was adequate The reading level of the units was adequate for my students 50%17% 33% (50% - Disagree) 17% Post- Units 1 and 2 (n=6)- Quality of Content in the Units Contd…

Students Pre- and –Post Surveys 10

Demographics Majority females (more than 50%) Majority in Grade 11 (63%) Majority have not taken an online course (92%) (those who have- financial literacy, Ap Psychology, History, Government and English, HAS, Microsoft Professional) Readings, Discussion forum, and Assignments Majority Intermediate Ability in using technology (59%); High/Advanced (35%) Majority use technology “frequently” 45%, Very frequently 29% 11

Qualities in a Social Studies teacher you liked/learned a lot Made learning fun Many activities Group work Passionate and knowledgeable about the subject Class projects to explain a topic area Told stories Little homework Creative Used worksheets to teach yourself Positive attitude 12

Biggest Challenge with the Content of World History Keeping the events in order; hard to remember Finding a usage of WH in real life Tests and Quizzes Hard to pay attention in class to WH Not challenging enough Too much memorization Vocabulary 13

Expectations from this online learning experience That it would be exciting and interesting None Help me understand better To learn about different cultures Grab my attention to learn WH Maybe more updated from the lessons Visual approach may help me learn more and retain better To learn and advance my knowledge of WH Learn at my own pace 14

15 Post- Units 1 and 2 (n=77)- Level of Satisfaction with Units Unit 1 Agree Strongly AgreeUnit 2 Agree Strongly Agree The pace of the unit was appropriate 51%8% 50% (28% Neither) 9% Unit course material was relevant 57%9%53%5% The length and detail of each section was appropriate 48%12%47%9% The unit is sufficiently differentiated 56%4%52%3% The unit was well organized 58%4%59%12% Presentation (layout, style, and format) of the material 57%10%57%7% Level of assessments 51%8%51%7%

16 Unit 1 Agree Strongly AgreeUnit 2 Agree Strongly Agree Frequency of assessments 48%9%49%9% Overall satisfaction with the unit 57%7%53%9% Post- Units 1 and 2 (n=77)- Level of Satisfaction with Units Contd…

17 Post- Units 1 and 2 (n=77)- Online format has changed/enhanced my knowledge in WH Unit 1 Agree Strongly AgreeUnit 2 Agree Strongly Agree Understanding the concepts 57%10%53%19% Actively engaging me 53%7%47%19% Reflecting on the topics 67%3%51%12% Incorporating strategies to my understanding and learning 54%7%53%11% Providing/sharing thoughts and online resources with fellow students 50%7%56%7% Continually evaluate and reflect on my learning 53%8%49%10%

Online Learning, Use of Technology, and World History 18 AgreeStrongly Agree I feel comfortable using technology along with classroom instruction 53%19% I feel comfortable learning using technology with classroom instruction 47%19% Online learning can help students address their diverse learning needs 51%12% Using technology in World History instruction in a classroom may increase my motivation and enthusiasm about learning WH. 53%11% Using online mode of instruction can improve and enhance my understanding of Units 1 and 2 in WH. 56%7% Using technology with classroom instruction can increase my achievement in social studies. 49%10%

Paired- t-test There were significant differences (p<0.05) in pre and post surveys for components – a) I feel comfortable using technology along with classroom instruction (p=0.006)and b) I feel comfortable learning using technology with classroom instruction (p=0.027) Almost - Using technology in WH instruction in a classroom may increase students’ motivation and enthusiasm about learning WH (p=0.07). 19

20 Narratives- Google Documents (Teachers) Utility of the Historical Thinking Skills Graphic Organizer- include sources that have markedly different accounts of the same event, which would require students to think critically about the multiple perspective, bias, and credibility aspects of historical thinking. All things considered, I think that this is a great lesson. A way to view the PowerPoint in full screen mode.

21 Narratives- Teachers Positive aspects of the course- Comments were related to the overall format, layout and content, resources and organization of the material, and usefulness of the course in classroom instruction. A majority of the respondents appreciated the fact that students could work on the modules at their own pace and were able to collaborate with team members on some of the assignments.

22 Specifically… In what ways do you think you could use, if at all, the content/s of the units to teach in your classroom? and Overall satisfaction with the Units Could use pieces from the Units. Varying student technology proficiency levels affected technology usage in WH course Material too basic for the grade- had to infuse the online curriculum with a number of extra activities to ensure student learning. Material overall pretty good, May need to modify at her school since WH is taught at junior level Falls short in practice; needs organization Liked the course navigator- took some time getting used to Need more interactive activities; more differentiated instruction within the units Liked that it was self-paced; content was in-depth; definitions handy Overall presentation was good

23 Briefly describe any changes you would make to this unit (e.g., layout, content, style, format, presentation). Students had trouble with the map always coming up. It did not show on some of the computers. Add extension materials for honors-level courses or for students who work quickly. Some interactives are short and therefore create a bit of a "jump" between activities that some students may struggle to follow. Adding activities that are not paper and pencil, as well as more quick check for understanding interactive assessments. Scavenger hunt type of interactive activity. I would like to see more of the "color" of history. i.e. the anecdotal stories, the human element, personal stories, the stuff that is not in the textbook. A wider variety of types of activities needs to be made available. The content needs to be deeper and more historically enriched. Pacing needs to be addressed. Certain topics were given too much attention while other topics were not addressed sufficiently.

24 Students Strengths and Other general comments: Own pace Great visuals and photos to teach WH concepts Keeps my attention- liked it Interaction keeps our interest Liked the reflection piece- helps us to relate Great details Lot of information in a short period of time. Learned a lot Boring Don’t like WH No difference between online and classroom instruction

Next Steps Content edits as suggested by teachers and students Make units and lessons more interactive and engaging Differentiated instruction Ways to improve response rates (reminders to supervisors) Tighter deadlines for completion Pilot next Units in 3-4 LEAs - More rigorous selection of schools and classrooms - Comparison across LEAs, Schools, and Classrooms 25

26 Questions/Comments Suggestions