Sharing the Secret Recipe: Deconstructing Key Performance Indicators for Quality Distance Social Work Education Social Work Distance Education Conference Indianapolis, Indiana April 15, 2015 Debora Rice, Ph.D., MSW., LCSW Pablo Arriaza, Ph.D., MSW., LICSW Sally Brocksen, Ph.D., MSW., MBA

Presentation Objectives Critically evaluate performance measures used in traditional higher education Identify factors for determining quality in online education around teaching, learning, and support Present key performance indicators (KPIs) used in Walden University’s Social Work Program Discuss how three experienced online social work professors interpret and use KPIs to identify quality and encourage performance

Reluctance to Embrace Online Education Only 28% of higher education leadership say their faculty accept the “value and legitimacy of online education” 66% of faculty consider online courses as inferior or somewhat inferior to a comparable face-to-face course 79% of faculty say they have concerns about the quality of online education at for-profit institutions Social work faculty specifically appear to have an embedded hesitation to embrace social work education in an online environment References: Allen, I. E. & Seaman, J. (2012). Conflicted: Faculty and online education. United States: The Sloan Consortium. Allen, I. E. & Seaman, J. (2014). Grade level: Tracking online education in the United States. United States: The Sloan Consortium. To change the mindset, hard data about quality are needed. The indicators of quality and performance have to adapt to the new environment. Walden’s model provides the evidence of performance and quality. Faculty and accrediting bodies might be surprised to discover there are two levels of additional metrics for quality online education beyond those of traditional bricks & mortar Sources: Allen, I. E. & Seaman, J. (2012). Conflicted: Faculty and online education. United States: The Sloan Consortium. Allen, I. E. & Seaman, J. (2014). Grade level: Tracking online education in the United States. United States: The Sloan Consortium.

Conceptualizing Quality Performance Traditional Early Distance Education Walden’s Online Learning outcome measures Student evaluation of faculty Faculty self-evaluation N/A Absent Learning outcome measures Student evaluation of faculty Faculty self-evaluation Evaluation of technology delivery system Absent Learning outcome measures Student evaluation of faculty Faculty self-evaluation Evaluation of technology delivery system Automated student accountability Faculty Key Performance Indicators (KPIs) Compare and contrast the information used to evaluate quality performance in the traditional face-to-face classroom, the early distance education courses, and Walden’s current online program. Two additional metrics used in Walden’s online environment are: Automated capture of student attendance, participation, documentation of no plagiarism, etc. Faculty KPIs

Today’s online environment requires: Evidence-based benchmarks specific to the online environment Online social work education requires: A baseline for accreditation and reaffirmation of hybrid and online programs

Questions for Today What factors (beyond the traditional) help determine quality performance in online instruction? How can Walden University's KPIs and Lead Faculty Model demonstrate successful evaluation and high quality of teaching in the online environment?

Factors That Help Determine the Quality of Online Instruction Responsiveness & availability Frequency & quality of presence Facilitation of discussion Prompt, quality feedback Developing cooperation among students Responding to diverse talents & styles Management of administrative aspects Setting high expectations References: Boling, E.C., Hough, M., Krinsky, H., Saleem, H., Stevens, M. (2012). Cutting the distance in distance education: Perspectives on what promotes positive, online learning experiences, The Internet and Higher Education,15(2), March 2012, 118-126. Crawford-Ferre, H.G. and West, L.R. (2012). Effective online instruction in higher education, The Quarterly Review of Distance Education, 13 (1), 2012,11-14. This list assumes quality technology is available, accessible, etc. Walden’s KPIs evaluate these factors Sources: Boling, E.C., Hough, M., Krinsky, H., Saleem, H., Stevens, M. (2012). Cutting the distance in distance education: Perspectives on what promotes positive, online learning experiences, The Internet and Higher Education, 15(2), March 2012, 118-126. Crawford-Ferre, H.G. and West, L.R. (2012). Effective online instruction in higher education, The Quarterly Review of Distance Education, 13 (1), 2012,11-14.

Operationalizing Walden’s Model KPI data goes to the Program Director weekly In cases where KPIs are not met, data goes to the individual professor and Lead Faculty for immediate action Lead Faculty serve as: Subject matter experts (SME) First line problem solvers Evaluators Coaches/Cheerleaders

Walden University’s Key Performance Indicators (KPIs)

Walden’s Key Performance Indicators (KPIs) Log into class 4 out of 7 days Participate in discussion posts 4 out of 7 days Respond to at least 2/3 of student posts to each discussion question Timely grade submissions with specific, substantial feedback Assignments and grades for discussion questions returned within 7 calendar days from the due date on the course calendar

Additional Walden KPIs Personalize the classroom Upload picture, CV, and brief narrative Review explicit class expectations Provide substantial feedback to promote/enhance students’ critical thinking Respond to students’ questions on the Class Café and Contact the Instructor tabs within 48 hours Use Announcements to update students, provide relevant external links/resources, and promote a sense of presence Create and maintain a respectful and safe environment in the classroom

The Trajectory and Experiences of Three Core Faculty Professors Interpreting, Embracing, and Applying Walden’s Model of Excellence in Distance Education The Trajectory and Experiences of Three Core Faculty Professors _ _ _ _ ___ _______ __________

Debbie’s Story Pablo’s Story Sally’s Story Three Stories Debbie’s Story Pablo’s Story Sally’s Story Three faculty with significant online teaching experience will discuss how they interpret, translate, and apply the KPIs in their roles as faculty and program director.

Questions? Additional indicators relevant to social work?

References Allen, I. E. & Seaman, J. (2012). Changing course: Ten years of tracking online education in the United States. United States: The Sloan Consortium. Allen, I. E. & Seaman, J. (2012). Conflicted: Faculty and online education. United States: The Sloan Consortium. Allen, I. E. & Seaman, J. (2014). Grade level: Tracking online education in the United States. United States: The Sloan Consortium. Boling, E.C., Hough, M., Krinsky, H., Saleem, H., Stevens, M. (2012). Cutting the distance in distance education: Perspectives on what promotes positive, online learning experiences, The Internet and Higher Education, 15(2), March 2012, 118-126. Council on Social Work Education (CSWE) (2012/2013). Annual report. Retrieved 24 October, 2014 from www.cswe.org/File.aspx?id=68989 Crawford-Ferre, H.G. and West, L.R. (2012). Effective online instruction in higher education, The Quarterly Review of Distance Education, 13 (1), 2012,11-14.

References Cummings, S.M., Chaffin, K.M., Cockerham, C. (2015). Comparative analysis of an online and a traditional MSW program: Educational outcomes, Journal of Social Work Education, 51:109-120. Flynn, M. Maiden, P. R. , Smith W., Wiley, J. & Wood. G. (2013). Launching the virtual academic center: Issues and challenges in innovation. Journal of Teaching in Social Work, 33, 331–338. Lackey, K. (2011). Faculty development: An analysis of current and effective training strategies for preparing faculty to teach online, Online Journal of Distance Learning Administration, 14(4). Retrieved from http://westga.edu/distance/ojdla/wnter144/lackey144.html Shelton, Kaye. (2011). A review of paradigms for evaluating the quality of online education programs. Online Journal of Distance Learning Administration, IV(I), Retrieved from http://www.westga.edu/~distance/ojdla/spring141/shelton141.html Thyer, B. A. and Polk, G. (1997). Distance learning in social work education: A preliminary evaluation, Journal of Social Work Education, 33(2), 363-368.