1. The Assessment of Blended Courses: Gathering and Using Faculty and Student Feedback to Maximize Program Effectiveness Orly Calderon, PsyD, Long Island University, CW Post Campus Amy Ginsberg, PhD, Long Island University, Brooklyn Campus Kathleen Morley, PhD., Long Island University, University Center Liz Ciabocchi, EdD, Long Island University, University Center

2. Introduction Faculty and student feedback on blended courses is instrumental to improving blended courses and programs. We will present the methodology and results of the WLP student and faculty surveys and the faculty focus groups. Finally, we will discuss next steps for blended learning at LIU for 2011-12 and beyond.

3. Three Parts of WLP Assessment Student Feedback: WLP Student Survey Student Feedback: WLP Student Survey Direct Measures: University Outcomes Assessment Direct Measures: University Outcomes Assessment Faculty Feedback: WLP Faculty Survey & Focus Groups Faculty Feedback: WLP Faculty Survey & Focus Groups

4. Need for direct measures of learning outcomes Program-level vs. course-level assessment Academic program vs. blended pilot program Blended component vs. the entire course Direct Measures: Challenges

5. WLP Student Survey Online, self-report, quantitative and qualitative questions Three parts: A.demographic data B.26 Likert-scale items that ask students to rate their experiences in blended courses and provide comments regarding: Blended vs. f2f courses Overall quality of blended course Learning value of specific course components C.three open-ended questions inviting students to reflect on their overall experiences in the blended course

6. Relative to f2f courses WLP Student Survey: Content Areas Amount of interaction with students and instructor Quality of interaction with students and instructor Adequacy of resources Clarity of syllabus with regard to online work Overall satisfaction Time management Workload Difficulty Integration of f2f & online Syllabus In-class material presentation On-line material presentation In-class interaction On-line interaction In-class quizzes & tests On-line quizzes & tests Textbook On-line readings Learning value of:

7. Open-Ended Questions – What was the most effective aspect of this blended learning course? – What was the least effective aspect of this blended learning course? – What advice would you give to a student considering a blended learning course for the first time?

8. The Foundation for the WLP Faculty Survey Rubric for Online Instruction Created at California State University, Chico to answer the question: What should a quality online course look like? www.csuchico.edu/celt/roi Licensed under Creative Commons Intended uses: – As a faculty self-evaluation tool (existing courses) – In the design on new online courses – For recognition of exemplary online instruction

9. WLP Faculty Survey Categories Learner Support and Resources Online Organization and Design Instructional Design and Delivery Assessment and Evaluation of Student Learning Innovative Teaching with Technology Faculty Use of Student Feedback Administered and scored through StudentVoice 30 items; 3 response choices (exemplary, effective, baseline)

10. Faculty and administrators Convened at the end of each semester Three main questions: – What are the faculty’s reactions to the WLP faculty survey and student survey results? – How do these results influence faculty thinking about their courses in subsequent semesters? – What resources are needed for faculty to implement the improvements they seek in their blended courses? Faculty Feedback: Focus Groups

11. WLP Student Survey Results (1) Important considerations when choosing to participate in LIU Blended Program: Flexibility of being able to complete assignments anyplace/anytime. The convenience of not having to come to campus as often.

12. Blended courses vs. F2F courses – workload in blended courses is somewhat heavier compared to f2f courses. Specific components of blended courses are moderately helpful. Overall, students are moderately satisfied with blended courses at LIU. WLP Student Survey Results (2)

13. WLP Student Survey Results (3) Students’ perceptions of the sufficiency of university resources for blended learning is the greatest predictor of students’ overall satisfaction with their blended course. In contrast, students’ rating of the learning value of specific course components was not a significant predictor of overall course satisfaction.

14. Student Survey Results (4) Most effective: – Flexible schedule is helpful with time management – Opportunities for independent work – Opportunities for interaction with the instructor and with peers – Good organization of the course and course materials – Instructor’s sympathetic attitude towards students Least effective: – The challenge of independent time management – Lack of opportunities to interact interpersonally with the instructor and with peers – Confusion (assignments, schedule of F2F sessions) – Technical difficulties (Blackboard, low computer literacy)

15. WLP Faculty Survey Results Overall, the faculty perceived their blended course to be effective. Items with highest ratings: o Organization of the course o clarity of the syllabus o clarity of learning objectives o goodness of fit between learning activities and learning objectives Items with lowest ratings: o Availability of information on online learning and campus resources o Pedagogy in blended courses o support for different learning styles o use of multimedia o opportunities for student feedback

16. Focus Groups Results The focus groups brought to life the needs of each program, some of which are universal and others are unique. Themes that emerged during the focus groups: – need for more faculty development in terms of pedagogy specific to blended instruction – enhanced support for students in terms of resources, computer literacy, organizational and time management skills.

17. Individual Experiences & Audience Input http://vimeo.com/25820531

18. Next Steps (under consideration)  A blended faculty training course leading to internal certification and delivered within Blackboard to develop instructional competency  Web-based faculty training courses such as facilitating online discussions, creating and using rubrics for assessment of blended/online courses, and using Blackboard and Web 2.0 tools  Access to Adobe Connect Pro for synchronous communications  Access to a dedicated server and faculty training in the development of multimedia content, including podcasts  A new Blended Learning website with resources for students and faculty  Better communication to faculty and students regarding on-campus and online resources