1. QUALITY ASSURANCE IN HIGHER EDUCATION AND PROFESSIONAL TRAINING How to assess an e-learning project? Application to CVG- « Virtual Campus in Management » Jean-Marie DUJARDIN and Severine MARON HEC-University of Liège, Belgium

2. CVG-Virtual Campus in Management (created in 2000) 3 « Business Schools » as partners Project sponsored by the European Social Fund and Walloonia

3. Developed courses : –Accounting –Marketing –Supply Chain Management –Strategy –Leadership –Organisational changes –Human resources –Finances –E-business Target groups: –Trainees (and students)

4. Plan 1. Building of a methodology to make a quantitative and qualitative assessment 2. Trying to apply this methodology to 2 courses: Supply chain management course and Leadership course 3. Next step : Applying the methodology systematically

5. 1. Building of a methodology To make a quantitative and qualitative assessment

6. CVG assessment methodology

7. RESOURCES

8. LEARNINGS

9. "STAKEHOLDERS"satisfaction

10. PROJECT DASHBORD

11. QUALITY ASSURANCE FOR DISTANCE LEARNING

12. 2. Trying to apply this methodology to 2 courses: Supply chain management course and Leadership course

13. Quantitative evaluation of Leadership course (2002-2006) INPUTS

14. OUTPUTS Leadership course Total amount of developped hours for Leadership course: 40 hours Total amount of students/trainees from 2004 to 2006 (nobody between 2002-2004) –Students : 650 –Trainees :9 Total amount of training hours for this course –Students : 650*40= 26 000 –Trainees 9*40= 360

15. What can we conclude of this? Relation between INPUTS and OUTPUTS- Leadership course Average cost (of development ) of a course for every : –Trainee (FSE target): 68.486,63 /9= 7.609,63€ –User( Students and Trainees) = 68.486,63 / (650+9)= 104 euros Average cost of one hour e-learning production –68.486,63 coût total /40(hours)= 1712 euros Human resources cost rate (compared to the entire costs) –50%

16. Qualitative evaluation of Leadership course

17. SCORE PRE-TEST ON 15 POINTS SCORE POST-TEST ON 15 POINTSDIFFERENCE Student 1 712+5 Student 2 119-2 Student 3 1314+1 Student 4 1413 Student 5 1314+1 Student 6 1415+1 Student 7 1312 Student 8 128-4 Student 9 13 0 Student 10 149-5 Average 12,4/1511,9/15

18. Evaluation of training effectiveness by the participants “Hot evaluation” : After the seminar “Cold evaluation” : Back at work…10 months later REMIND What specifically did the learners learn or be usefully reminded of Leadership can be performed with different styles  adaptation. Importance of preparing the interviews with the subordinates. Every communication has a content and relationship aspect  analyze both aspects. To be clear, positive minded, assertive in communication situations. To be more transparent. Communication is critical : listen, involve, explain once or twice why as well as what needs to be done. Take time to listen to and really understand people. Have faith in people. IMPLEMENT What do learners expect to implement on their return to work / actual implementation of the training course Define the objectives, the timing and the structure of each meeting. Not to camp on his position. Define objectives and negotiation strategy. Reformulate. To be clear ourselves about we mean Not to let the problems nor the doubts take place in work relationships. Analyze in depth each problem to find the real problem and tough the best solution. Managing meetings more effectively.

19. Evaluation of training effectiveness by the participants After the seminarBack at work…10 months later RESSOURCES Resources at their disposal to implement the concepts learned during the seminar HR department and colleagues Experimentation : keep on learning by your errors Training course. they are really supported in this by the top managers. Management By Objectives implemented in 2007. Change management process which put them in leadership situations : follow-up interviews… DifficultiesKeep on being natural in the choice of the right leadership style. Listen, let the other express himself. Be careful to separate the content of a problem and the relationship aspect. Be able to keep time to listen to people problems, ideas… Handle with conflicts. Lack of time… we are in the action. To have a vision shared by everyone about the best way to manage people. To find the best way to communicate with subordinates : e-mail, intranet, meetings, …

20. Evaluation of training effectiveness by the top management Back at work…10 months later Thanks to the seminar, benefits that are pointed by the top management Increase commitment to organization future. Leadership is a job in itself. It’s not just something more than you ask to do to someone. It’s a very difficult job especially on the RH point of view. Need for follow-up training courses. Need to adapt to mentality, to new management tools. The seminar really responded to a need; it was not just fads and trends. Traineeship take a big part in the total quality management. Creation of monthly management meetings about RH issues only. Change in the recruitment policy for new managers.

21. Evaluation of training satisfaction by the participants After the seminarBack at work…10 months later +Interactions Time for thinking, for learning  for other things than operational tasks. To connect with other middle managers who are dealing with the same problems than ourselves. The different themes of the seminar are well reflecting our role of middle managers. Improve self-confidence. We have tools to deal with the new firm policies : change management, management by objectives, total quality management… ImprovementsSometimes, role playing is so different from what we are living at work. E-learning tools are not always appropriate for the ones who are not use to computer. Meetings to share field problems, solutions with other middle managers. Organize a follow-up training course for the ones who need it.

22. What can we conclude of this? Is there a difference between –pre- and post-test? –« hot » and « cold » evaluation? Do the results match with the course objectives? Are the students/trainees satisfied?

23. Quantitative evaluation of SCM course (2002-2006) INPUTS

24. OUTPUTS (2002-2006) Total amount of developped hours for SCM Management course: 90 hours Total amount of students/trainees from 2004 to 2006 (nobody between 2002-2004) –Students :/ –Trainees : 63 Total amount of training hours for all courses –Students :/ –Trainees : 90*63=5670

25. What can we conclude of this? For Supply Chain Management course Relation between INPUTS and OUTPUTS Average cost (of development ) of a course for every –Student :/ –Trainee: 139.354,65€ /63= 2211Euros Average cost of one hour e-learning production: 139.354,65€ /90 = 1548 euros Human resources cost rate (compared to the entire costs) –46%

26. Qualitative evaluation of SCM course ( self-evaluation) Just before the course, the teacher asked the trainees to evaluate their knowledge level for each part of his course, for example: Transportation (theory basis) Transportation (application) Stocks management …. On a scale: low-middle-good-excellent After the course, the teacher will ask the trainees to do the same thing, so we could compare…

27. Try to apply this methodology to the global project:

28. OUTPUTS Global project Total amount of developped hours for all courses: 350 hours (10 courses) Total amount of students/trainees from 2004 to 2006 (nobody between 2002-2004) –Students : about 2200 –Trainees (FSE target) : 261 Total amount of training hours for all courses –Students : about 2200*30=66000 –Trainees: over 12000 hours

29. What can we conclude of this?

30. Average cost (of development) of the courses for every –Student :/ –Trainee 1010500,38 /261=3871 euros Average cost of one hour e-learning production: 1010500,38/ 350= 2887 euros …but the diffusion only begins !!!! Human resources cost rate (compared to the entire costs) –45% Dropout Rate:?? What can we conclude of this? Relation between INPUTS and OUTPUTS Global project

31. 3. Next steps … How to improve the methodology and its application for next years ? Apply the methodology to all courses systematically for quality insurance!

32. Interests of the methodology Approach also focused on outputs Professional and systematical management of the project Discovery of new information and different view on the project Guidelines for budget and development Needs much time and resources Lack of information Drawbacks of the methodology

33. Do you have questions ? Visit our website www.campusvirtuel.be Or write an e-mail Cvg@campusvirtuel.be