1. The Lycurgus Group: Instructional Effectiveness Survey System
LEAP System
9:00 AM on Wednesday, June 7, 2006

2. Copyright 2006 © Lycurgus Group, Inc.
The Lycurgus Group, Inc.
An organizational effectiveness and development consulting firm that uses scientific and research approaches to assist organizations in the areas of organizational assessment, change, development and online learning.
Copyright 2006 © Lycurgus Group, Inc.

3. What you can expect from us
We will...
Design tailored programs to measure and manage Student, Faculty and Staff experience and effectiveness
Deploy hundreds of surveys
Conduct focus groups/interviews
Develop faculty training and development programs
Conduct industry studies/identify benchmarks
Measure financial impact
Determine instructional design needs
Partner with your organization to achieve distance education excellence!
Copyright 2006 © Lycurgus Group, Inc.

4. Copyright 2006 © Lycurgus Group, Inc.
Services Provided
Our services are focused in the following areas:
change management
organizational and leadership development
training and development
e-learning
organizational metrics
performance management
survey assessment and research
Copyright 2006 © Lycurgus Group, Inc.

5. One small step for education, one giant LEAP for
Introducing the LEAP System™
One small step for education,
one giant LEAP for
distance learning excellence
Copyright 2006 © Lycurgus Group, Inc.

6. Lycurgus Evaluation Assessment Process The LEAP System™
The LEAP System™ is a systematic approach in assessment, change implementation and evaluation.
Assessments measure learning outcomes and institutional effectiveness.
Data is gathered and used to identify planned change efforts and needed improvements.
All key stakeholders are included in the feedback process.
The evaluation system is linked to measuring quality standards and best practices in distance education.
The 12 key factors are based on research leveraging organizational effectiveness and best practices in distance education.
Copyright 2006 © Lycurgus Group, Inc.

7. Copyright 2006 © Lycurgus Group, Inc.
Proactive Evaluation
Focus on Quality Standards
Formative/Summative Evaluation
Formative/Summative Evaluation
Best Practices in Online Learning
Kirkpatrick’s Evaluation Model
Copyright 2006 © Lycurgus Group, Inc.
$16,200/year

8. Traditional Evaluation Methods
Formative=Ongoing
Institutional Effectiveness
Formative evaluations can be accomplished through:
This can be accomplished by obtaining feedback from students and faculty on their satisfaction with online learning.
Improvement of programs
Evaluate strengths/weaknesses and areas of concern
Summative=Completion
Course Effectiveness
Synthesis of data to make decisions on course improvements
Summative evaluations
It is important to be able to improve the course design to facilitate learning in a more effective way.
Evaluate course objectives, experience and facilitation style
Improvement of programs
Form (process) and summative (outcomes)
Formative evaluation:
Is an on-going process to be considered at all stages of instruction.
Will enable the instructor to improve the course as he/she proceeds.
Facilitates course and content adaptation.
Will identify major gaps in the instructional plan or the need for minor adjustments.
Summative evaluation:
Assesses overall effectiveness of the finished product or course.
Can be a springboard in developing a revision plan.
Can be a baseline of information for designing a new plan, program, or course.
Will not help current students since it is conducted upon course completion.
Some questions that educators may want to ask students when collecting summative data include:
List five weaknesses of the course.
List three (or five) strengths of the course.
Copyright 2006 © Lycurgus Group, Inc.

9. Copyright 2006 © Lycurgus Group, Inc.
Quality Framework
Quality framework: enables institutions to set goals, measure progress in reaching goals and implement required changes
The focus on quality and best practices are the basis of our evaluation system.
Applied to each individual institution uniquely by identifying needs/goals
Following organizations identified quality standards:
American Council of Education, American Distance Education Consortium, Sloan-C, Institute for Higher Education
Sloan Consortium: A consortium of institutions and organizations committed to quality online education
Copyright 2006 © Lycurgus Group, Inc.

10. Kirkpatrick’s Four-Level Model
Evaluation model focused on results
Level 1: Evaluate reaction to instruction/satisfaction
Level 2: Evaluate learning
Level 3: Evaluate behavior
Were performance objectives achieved?
Level 4: Evaluate results/impact on organization (Schankman, 2004)
Level 5 added: ROI
Missing:
Fails to consider necessary improvements
Considering the importance of motivation
Level 1: Evaluate reaction to instruction/Satisfaction-survey
Level 2: Evaluate learning-survey
Level 3: Evaluate behavior-survey
Were performance objectives achieved?
Level 4: Evaluate results/impact on organization (Schankman, 2004)-Interview/performance data
Copyright 2006 © Lycurgus Group, Inc.

11. Evaluation System: Based on Quality standards in Online Education
Learning Effectiveness
Student Satisfaction
Faculty
Satisfaction
Cost Effectiveness
Main Measure: End of Course Survey
Also, can use focus groups and interviews
Student Survey on Institutional Effectiveness
Faculty Survey of online learning experience/Instructional Effectiveness
*Goal is to cut or control costs
*Reduce teacher turnover, lower training costs and improve technologies
*Increase retention
*Number of students obtaining degrees
-Good to develop dashboard of results as a communication tool
-important to utilize internal and external benchmarks n
Copyright 2006 © Lycurgus Group, Inc.

12. Copyright 2006 © Lycurgus Group, Inc.
Leap System™ Steps
Step 1: Determine goals, objectives and standards of excellence.
Step 2: Identify internal benchmarks. (These benchmarks are used to compare to future evaluations and to compare with other schools.)
Step 3: Collect data from all stakeholders. Assessment includes surveys as well as interviews and focus groups when applicable.
Step 4: Report results and make recommendations.
Step 5: Identify top areas for planned improvements.
Step 6: Implement changes.
Step 7: Reevaluate.
Copyright 2006 © Lycurgus Group, Inc.

13. Copyright 2006 © Lycurgus Group, Inc.

14. Copyright 2006 © Lycurgus Group, Inc.
1. Determine Goals/Objectives
Copyright 2006 © Lycurgus Group, Inc.

15. Copyright 2006 © Lycurgus Group, Inc.
Goals/Objectives
It is important to determine: Why you are doing the evaluation? What do you hope to accomplish? What is success?
Student performance, satisfaction, retention
What will you do with the data?
Who do you want to share the results with?
Copyright 2006 © Lycurgus Group, Inc.

16. Copyright 2006 © Lycurgus Group, Inc.
2. Identify Internal Benchmarks
Copyright 2006 © Lycurgus Group, Inc.

17. Copyright 2006 © Lycurgus Group, Inc.
Internal Benchmarks
Important to identify key internal measures
Such as student and faculty satisfaction and retention
Will there be longitudinal comparisons made?
Comparisons conducted within classes/departments/curriculum?
What are internal messages of success? Or causes for alarm?
Also, valuable to identify comparisons against established external benchmarks when applicable
Copyright 2006 © Lycurgus Group, Inc.

18. Copyright 2006 © Lycurgus Group, Inc.
3.Collect Data from All Stakeholders
Copyright 2006 © Lycurgus Group, Inc.

19. Data Collection Options
Identify who will be involved in data collection process:
Staff, faculty and/or students
These are key stakeholders
Also, graduates can provide informative data
Identify best methods of data collection
Different surveys, focus groups, or interviews, document evaluation
Copyright 2006 © Lycurgus Group, Inc.

20. Copyright 2006 © Lycurgus Group, Inc.
Survey Assessments
The nucleus of the system LEAP System™
Faculty/Staff & Student Effectiveness Survey &
End of course Survey
Web-Based Platform – Quickly gather critical data about your organization.
Proprietary 12 Key Factors – Tested best practices. Researched and designed over the last 10 years.
Detailed reports based on Data – Have the knowledge to make crucial decisions.
Recommendations made on Key Areas – Know how to make improvements.
Web-based Custom Dashboards – Know critical information about your organization anytime and anywhere in the world.
Copyright 2006 © Lycurgus Group, Inc.

21. Survey Assessment Benefits
Can involve many individuals in the feedback process in a short timeframe
Able to get detailed feedback
Ease in administration
Utilize web to contact participants
Ensures confidentiality of feedback obtained
Copyright 2006 © Lycurgus Group, Inc.

22. Copyright 2006 © Lycurgus Group, Inc.
4. Report Results/Make Recommendations
Copyright 2006 © Lycurgus Group, Inc.

23. Copyright 2006 © Lycurgus Group, Inc.
Reports
Different levels of reports can be provided that will allow for different levels of action planning
Strengths can be identified
Can see overall picture of institutional performance
Copyright 2006 © Lycurgus Group, Inc.

24. Copyright 2006 © Lycurgus Group, Inc.
5. Identify Top Areas for Planned Improvements
Copyright 2006 © Lycurgus Group, Inc.

25. Copyright 2006 © Lycurgus Group, Inc.
Planned Improvements
Top areas of concern can be discovered
Focus of top action plans
Use data to identify planned improvements, timeline and who will be involved in making needed changes
Copyright 2006 © Lycurgus Group, Inc.

26. Copyright 2006 © Lycurgus Group, Inc.
6. Implement Changes
Copyright 2006 © Lycurgus Group, Inc.

27. Copyright 2006 © Lycurgus Group, Inc.
Changes to Implement
Utilizing an organizational effectiveness approach
Identified from all data & participants
Examples of changes made working with other schools include:
Implementation of professional development activities
Construction of new student orientation materials
Changes to online courses/programs
Copyright 2006 © Lycurgus Group, Inc.

28. Copyright 2006 © Lycurgus Group, Inc.
7. Reevaluate
Copyright 2006 © Lycurgus Group, Inc.

29. Copyright 2006 © Lycurgus Group, Inc.
Evaluation Revisited
Important to evaluate to see if improvements have occurred, if new changes need to be made or if new issues have developed
Also, important to continually celebrate accomplishments!
Copyright 2006 © Lycurgus Group, Inc.

30. Copyright 2006 © Lycurgus Group, Inc.
Other Key Services
Training of Online Instructors
Development of Materials to assist with new students new to online learning
Online Course Design
Selection/Recruitment of Adjunct Professors
Copyright 2006 © Lycurgus Group, Inc.

31. Copyright 2006 © Lycurgus Group, Inc.
Contact Us
to discuss how we can help you!
We can present solutions for your most difficult problems.
Phone: (310) Fax: (310)
Copyright 2006 © Lycurgus Group, Inc.