The Lycurgus Group: Instructional Effectiveness Survey System

Slides:



Advertisements
Similar presentations
School Improvement Through Capacity Building The PLC Process.
Advertisements

Assessing Student Learning Outcomes In the Context of SACS Re-accreditation Standards Presentation to the Dean’s Council September 2, 2004.
Implementation of the PA Core Standards. Effective Communication Guiding Principle 1 Design and establish systems of effective communication among stakeholders.
Training Evaluation Presentation by Ranjith Menon.
Training Kirkpatrick’s Four Levels of Evaluation Kelly Arthur Richard Gage-Little Dale Munson Evaluation Strategies for Instructional Designers.
Performance Management
Chapter 2 Strategic Training
Charting a course PROCESS.
Central Kentucky Partnership in Mathematics and Science (CKPIMS) Central Kentucky Partnership in Mathematics and Science (CKPIMS) Central Kentucky Education.
PD-1 PROFESSIONAL DEVELOPMENT. PD-2 PROFESSIONAL DEVELOPMENT Instructional Leadership Development Framework for Data-driven Systems QUALITY STUDENT PERFORMANCE.
March 26-28, 2013 SINGAPORE CDIO Asian Regional Meeting and Workshop on Engineering Education and Policies for Regional Leaders Programme Evaluation (CDIO.
EVALUATION OF HRD PROGRAMS Jayendra Rimal. The Purpose of HRD Evaluation HRD Evaluation – the systematic collection of descriptive and judgmental information.
ELearning Committee Strategic Plan, A Brief History of the ELC Committee Developed and Charged (2004) CMS Evaluation and RFP Process (2004)
Copyright  2005 McGraw-Hill Australia Pty Ltd PPTs t/a Australian Human Resources Management by Jeremy Seward and Tim Dein Slides prepared by Michelle.
The University of Kentucky Program Review Process for Administrative Units April 18 & 20, 2006 JoLynn Noe, Assistant Director Office of Assessment
Instructional Strategies Teacher Knowledge, Understanding, and Abilities The online teacher knows and understands the techniques and applications of online.
The Standards-based IEP Process: What You Need to Know Standards-Based IEP State-Directed Project - January 2011.
Assessment and Development: The Keys to Quality DLA2007 Distance Learning Administration Conference 8:00 AM on Monday, June 25, 2007.
Middle States Re-Accreditation Town Hall September 29, :00-10:00 am Webpage
Copyright 2006 © Lycurgus Group, Inc.
Implementing QM towards Program Certification
Contents Playbook Objectives Playbook Value Details Playbook Design
Creating Faculty Evaluation and Mentoring Programs Using Brightspace
Designing Effective Evaluation Strategies for Outreach Programs
Assessment Basics PNAIRP Conference Thursday October 6, 2011
Self Assessment for Pastoral Care
First-Year Experience Seminars: A Benchmark Study of Targeted Courses for Developmental Education Students.
Monitoring and Evaluation
Bridges To Success “Effective Advising in Guided Pathways: Executing advising plans that transform departments and institutions to help students achieve.
Introduction to evaluating and measuring impact in career development Presented by – Date – Doubt, the essential preliminary of all improvement and.
Achieving the Dream Mark A. Smith.
How did WE work? Assessing Collaborative Projects in the Online or Hybrid Classroom
RECOGNIZING educator EXCELLENCE
ASSESSMENT OF STUDENT LEARNING
Continuous Improvement through Accreditation AdvancED ESA Accreditation MAISA Conference January 27, 2016.
Evaluating performance management
Evaluation of HR – Why evaluate?
Project-Based Learning
BEST PRACTICES IN LIBRARY INSTRUCTION FORUM November 7, 2007
Project-Based Learning
Designing and Implementing Local Faculty Development Programs
Overview – Guide to Developing Safety Improvement Plan
MTM Measurement Initiative
Human Resources Competency Framework
Overview – Guide to Developing Safety Improvement Plan
Krista Jackson AET/560 February 15, 2016 Dr. Nortz
TAP CENTRAL ADMINISTRATIVE UNIT CUSTOMER SERVICE SURVEY
Strategic Planning Setting Direction Retreat
recommendations for new teachers
Orientation and Training
الجــودة الشاملة في مؤسسات التعليم العالي
Management and Evaluation Programs for Faculty
Orientation and Training
Engaging Institutional Leadership
Standard Four Program Impact
Chicago Public Schools
Course Evaluation Ad-Hoc Committee Recommendations
Administrator Evaluation Orientation
Parent-Teacher Partnerships for Student Success
Using Data to Assess High Quality Professional Development Findings From Interviews With Massachusetts Educators May 2015 Massachusetts Department.
MAZARS’ CONSULTING PRACTICE Helping your Business Venture Further
TECHNOLOGY ASSESSMENT
Orientation and Training
Welcome to Your New Position As An Instructor
Internal Audit Who? What? When? How? Why? In brief . . .
Strategic Planning.
Using Logic Models in Project Proposals
Balanced Scorecard Process Writing Outcome Objectives and Measures
Bridging the ITSM Information Gap
Bridging the ITSM Information Gap
Presentation transcript:

The Lycurgus Group: Instructional Effectiveness Survey System LEAP System 9:00 AM on Wednesday, June 7, 2006

Copyright 2006 © Lycurgus Group, Inc. The Lycurgus Group, Inc. An organizational effectiveness and development consulting firm that uses scientific and research approaches to assist organizations in the areas of organizational assessment, change, development and online learning. Copyright 2006 © Lycurgus Group, Inc.

What you can expect from us We will... Design tailored programs to measure and manage Student, Faculty and Staff experience and effectiveness Deploy hundreds of surveys Conduct focus groups/interviews Develop faculty training and development programs Conduct industry studies/identify benchmarks Measure financial impact Determine instructional design needs Partner with your organization to achieve distance education excellence! Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. Services Provided Our services are focused in the following areas: change management organizational and leadership development training and development e-learning organizational metrics performance management survey assessment and research Copyright 2006 © Lycurgus Group, Inc.

One small step for education, one giant LEAP for Introducing the LEAP System™ One small step for education, one giant LEAP for distance learning excellence Copyright 2006 © Lycurgus Group, Inc.

Lycurgus Evaluation Assessment Process The LEAP System™ The LEAP System™ is a systematic approach in assessment, change implementation and evaluation. Assessments measure learning outcomes and institutional effectiveness. Data is gathered and used to identify planned change efforts and needed improvements. All key stakeholders are included in the feedback process. The evaluation system is linked to measuring quality standards and best practices in distance education. The 12 key factors are based on research leveraging organizational effectiveness and best practices in distance education. Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. Proactive Evaluation Focus on Quality Standards Formative/Summative Evaluation Formative/Summative Evaluation Best Practices in Online Learning Kirkpatrick’s Evaluation Model Copyright 2006 © Lycurgus Group, Inc. $16,200/year

Traditional Evaluation Methods Formative=Ongoing Institutional Effectiveness Formative evaluations can be accomplished through: This can be accomplished by obtaining feedback from students and faculty on their satisfaction with online learning. Improvement of programs Evaluate strengths/weaknesses and areas of concern Summative=Completion Course Effectiveness Synthesis of data to make decisions on course improvements Summative evaluations It is important to be able to improve the course design to facilitate learning in a more effective way. Evaluate course objectives, experience and facilitation style Improvement of programs Form (process) and summative (outcomes) Formative evaluation: Is an on-going process to be considered at all stages of instruction. Will enable the instructor to improve the course as he/she proceeds. Facilitates course and content adaptation. Will identify major gaps in the instructional plan or the need for minor adjustments. Summative evaluation: Assesses overall effectiveness of the finished product or course. Can be a springboard in developing a revision plan. Can be a baseline of information for designing a new plan, program, or course. Will not help current students since it is conducted upon course completion. Some questions that educators may want to ask students when collecting summative data include: List five weaknesses of the course. List three (or five) strengths of the course. Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. Quality Framework Quality framework: enables institutions to set goals, measure progress in reaching goals and implement required changes The focus on quality and best practices are the basis of our evaluation system. Applied to each individual institution uniquely by identifying needs/goals Following organizations identified quality standards: American Council of Education, American Distance Education Consortium, Sloan-C, Institute for Higher Education Sloan Consortium: A consortium of institutions and organizations committed to quality online education Copyright 2006 © Lycurgus Group, Inc.

Kirkpatrick’s Four-Level Model Evaluation model focused on results Level 1: Evaluate reaction to instruction/satisfaction Level 2: Evaluate learning Level 3: Evaluate behavior Were performance objectives achieved? Level 4: Evaluate results/impact on organization (Schankman, 2004) Level 5 added: ROI Missing: Fails to consider necessary improvements Considering the importance of motivation Level 1: Evaluate reaction to instruction/Satisfaction-survey Level 2: Evaluate learning-survey Level 3: Evaluate behavior-survey Were performance objectives achieved? Level 4: Evaluate results/impact on organization (Schankman, 2004)-Interview/performance data Copyright 2006 © Lycurgus Group, Inc.

Evaluation System: Based on Quality standards in Online Education Learning Effectiveness Student Satisfaction Faculty Satisfaction Cost Effectiveness Main Measure: End of Course Survey Also, can use focus groups and interviews Student Survey on Institutional Effectiveness Faculty Survey of online learning experience/Instructional Effectiveness *Goal is to cut or control costs *Reduce teacher turnover, lower training costs and improve technologies *Increase retention *Number of students obtaining degrees -Good to develop dashboard of results as a communication tool -important to utilize internal and external benchmarks n Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. Leap System™ Steps Step 1: Determine goals, objectives and standards of excellence. Step 2: Identify internal benchmarks. (These benchmarks are used to compare to future evaluations and to compare with other schools.) Step 3: Collect data from all stakeholders. Assessment includes surveys as well as interviews and focus groups when applicable. Step 4: Report results and make recommendations. Step 5: Identify top areas for planned improvements. Step 6: Implement changes. Step 7: Reevaluate. Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. 1. Determine Goals/Objectives Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. Goals/Objectives It is important to determine: Why you are doing the evaluation? What do you hope to accomplish? What is success? Student performance, satisfaction, retention What will you do with the data? Who do you want to share the results with? Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. 2. Identify Internal Benchmarks Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. Internal Benchmarks Important to identify key internal measures Such as student and faculty satisfaction and retention Will there be longitudinal comparisons made? Comparisons conducted within classes/departments/curriculum? What are internal messages of success? Or causes for alarm? Also, valuable to identify comparisons against established external benchmarks when applicable Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. 3.Collect Data from All Stakeholders Copyright 2006 © Lycurgus Group, Inc.

Data Collection Options Identify who will be involved in data collection process: Staff, faculty and/or students These are key stakeholders Also, graduates can provide informative data Identify best methods of data collection Different surveys, focus groups, or interviews, document evaluation Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. Survey Assessments The nucleus of the system LEAP System™ Faculty/Staff & Student Effectiveness Survey & End of course Survey Web-Based Platform – Quickly gather critical data about your organization. Proprietary 12 Key Factors – Tested best practices. Researched and designed over the last 10 years. Detailed reports based on Data – Have the knowledge to make crucial decisions. Recommendations made on Key Areas – Know how to make improvements. Web-based Custom Dashboards – Know critical information about your organization anytime and anywhere in the world. Copyright 2006 © Lycurgus Group, Inc.

Survey Assessment Benefits Can involve many individuals in the feedback process in a short timeframe Able to get detailed feedback Ease in administration Utilize web to contact participants Ensures confidentiality of feedback obtained Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. 4. Report Results/Make Recommendations Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. Reports Different levels of reports can be provided that will allow for different levels of action planning Strengths can be identified Can see overall picture of institutional performance Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. 5. Identify Top Areas for Planned Improvements Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. Planned Improvements Top areas of concern can be discovered Focus of top action plans Use data to identify planned improvements, timeline and who will be involved in making needed changes Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. 6. Implement Changes Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. Changes to Implement Utilizing an organizational effectiveness approach Identified from all data & participants Examples of changes made working with other schools include: Implementation of professional development activities Construction of new student orientation materials Changes to online courses/programs Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. 7. Reevaluate Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. Evaluation Revisited Important to evaluate to see if improvements have occurred, if new changes need to be made or if new issues have developed Also, important to continually celebrate accomplishments! Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. Other Key Services Training of Online Instructors Development of Materials to assist with new students new to online learning Online Course Design Selection/Recruitment of Adjunct Professors Copyright 2006 © Lycurgus Group, Inc.

Copyright 2006 © Lycurgus Group, Inc. Contact Us to discuss how we can help you! We can present solutions for your most difficult problems. www.lycurgusgroup.com info@lycurgusgroup.com ddimatteo@lycurgusgroup.com jgibson@lycurgusgroup.com Phone: (310) 543-0909 Fax: (310) 543-0050 Copyright 2006 © Lycurgus Group, Inc.