1. Sociedade Portuguesa de Inovação Information Society Technologies Programme Accompanying Measures (IST-2001-32633) Sponsor Innovation in e-Learning Seminar

2. Sociedade Portuguesa de Inovação To disseminate, stimulate discussion and creative suggestions with regard to identified best practices in e- Learning Objective Innovation in e-Learning Seminar

3. Sociedade Portuguesa de Inovação 9:15Introduction 9:30Project Objectives and Applied Methodology 10:30 Break 10:45Project Results – Case Studies 11:30Roundtable Discussion 12:00Lunch 13:15Presentations and Input From Participants 15:15Break 15:30 Next Steps 16:00Closing Comments Agenda Innovation in e-Learning Seminar

4. Sociedade Portuguesa de Inovação Project Objectives & Applied Methodology Innovation in e-Learning Seminar

5. Sociedade Portuguesa de Inovação Project Objectives & Applied Methodology  Project Objectives General Objective General Objective Specific Objective Specific Objective  Applied Methodology Identification of e-Learning Sites Identification of e-Learning Sites Initial Evaluation Initial Evaluation -Structure -Applications Final Evaluation Final Evaluation -Learner Models -Soft Skills & Informal Learning -Communities of Learning Dissemination Activities Dissemination Activities December 2002

6. Sociedade Portuguesa de Inovação Project Objectives To stimulate creativity, and to enhance education and training systems for lifelong learning. (IST 2001 Key Action 3.2.1 – Self Learning for Work) General Objective December 2002

7. Sociedade Portuguesa de Inovação Project Objectives To conduct a study on innovative structures and applications of e-Learning within the United States and Europe. To conduct a study on innovative structures and applications of e-Learning within the United States and Europe. To foster innovative e-Learning development through dissemination of “best practice” innovative structures and applications. To foster innovative e-Learning development through dissemination of “best practice” innovative structures and applications. Specific Objectives December 2002

8. Sociedade Portuguesa de Inovação Applied Methodology e-Learning Sites: Internet based learning that is focused on the delivery of content (text, audio, video, animation, simulation) in a training format.Internet based learning that is focused on the delivery of content (text, audio, video, animation, simulation) in a training format. Identification of e-Learning Sites Description Reasoning Reflection December 2002

9. Sociedade Portuguesa de Inovação Applied Methodology  Utilized the four leading search engines based on the Jupiter Media Matrix’s ratings for “audience reach”, 2002 (searchenginewatch.com). Microsoft NetworkMicrosoft Network YahooYahoo GoogleGoogle America OnlineAmerica Online Identification of e-Learning Sites Description Reasoning Reflection December 2002

10. Sociedade Portuguesa de Inovação Applied Methodology  Conducted a search with the term “e-learning” and selected only sites with a focus on students as the end user. About 50 sites identifiedAbout 50 sites identified Identification of e-Learning Sites Description Reasoning Reflection December 2002

11. Sociedade Portuguesa de Inovação Applied Methodology  Search engines are the most common method for identifying Internet sites. Theme “Self-learning for work” - accessible existing sites.  Assumption: leading e-Learning sites have the knowledge to leverage search engine and directories to attract end users.  Assumption: the term “e-learning” is a universal term applied across Europe and the United States for Internet based learning. Identification of e-Learning Sites Description Reasoning Reflection December 2002

12. Sociedade Portuguesa de Inovação Applied Methodology  Restricted search due to assumptions – the term “e- learning” and e-learning sites ability to leverage search engines. Other options are also highly restricted such as surveys of members of relevant associations (European Distance Education Network – EDEN) that rely on a significant response rate. There is always a low interest for participating in evaluations!Other options are also highly restricted such as surveys of members of relevant associations (European Distance Education Network – EDEN) that rely on a significant response rate. There is always a low interest for participating in evaluations! Identification of e-Learning Sites Description Reasoning Reflection December 2002

13. Sociedade Portuguesa de Inovação Applied Methodology  Restricted search due to assumptions – the term “e- learning” and e-learning sites ability to leverage search engines. (Continued) Possibly limited to English based sites – 6 % of the resulting sites were non English based.Possibly limited to English based sites – 6 % of the resulting sites were non English based. Identification of e-Learning Sites Description Reasoning Reflection December 2002

14. Sociedade Portuguesa de Inovação Applied Methodology  Resulting sites represented a diversity of organisation structures (private for profit, associations, etc.) but no traditional universities. A lack of university focus on the general public market.A lack of university focus on the general public market. Opportunity to transfer innovative activities to the traditional university sector – dissemination activities.Opportunity to transfer innovative activities to the traditional university sector – dissemination activities. Identification of e-Learning Sites Description Reasoning Reflection December 2002

15. Sociedade Portuguesa de Inovação Applied Methodology Initial Evaluation  Identified a set of criteria to evaluate the sites’ structures and applications.  Evaluated the sites based on the set of criteria.  Identified the top 5 sites with regard to structures.  Identified the top 5 sites with regard to applications.  Identified the top 20 sites overall (structures / applications). Description Reasoning Reflection December 2002

16. Sociedade Portuguesa de Inovação Applied Methodology Initial Evaluation  Analysis of structures accounted for 7 specific criteria: Objectiveness of the Mission – clearly stated/understood Objectiveness of the Mission – clearly stated/understood Unique Design and Structure of the Interface Unique Design and Structure of the Interface Aesthetics of the Web site – draw interest and appealing Aesthetics of the Web site – draw interest and appealing Usability and Navigation – user friendly Usability and Navigation – user friendly Description Reasoning Reflection December 2002

17. Sociedade Portuguesa de Inovação Applied Methodology Initial Evaluation  Analysis of structures accounted for 7 specific criteria (continued): Adaptability to the Web – course content adapted or simply a written document Adaptability to the Web – course content adapted or simply a written document Appropriateness of the Course Structure – take into account subject (Accounting includes problem solving simulations) Appropriateness of the Course Structure – take into account subject (Accounting includes problem solving simulations) Support Available – (contact information, call centre) Support Available – (contact information, call centre) Description Reasoning Reflection December 2002

18. Sociedade Portuguesa de Inovação Applied Methodology Initial Evaluation  Analysis of applications accounted for 4 specific criteria: Technological Characteristics –course specific (Audio, video, animation, whiteboards) Technological Characteristics –course specific (Audio, video, animation, whiteboards) Degree of Interaction with Other Students (Synchronous interaction via audio/video/chat, email) Degree of Interaction with Other Students (Synchronous interaction via audio/video/chat, email) Degree of Interaction with a Tutor (Synchronous interaction via audio/video/chat, email) Degree of Interaction with a Tutor (Synchronous interaction via audio/video/chat, email) Pedagogical Tools –course specific (exercises, quizzes, simulations) Pedagogical Tools –course specific (exercises, quizzes, simulations) Description Reasoning Reflection December 2002

19. Sociedade Portuguesa de Inovação Applied Methodology Initial Evaluation  Previous works: Previous studies within the e-learning field: LGUIDE – The e-Learning ExpertsPrevious studies within the e-learning field: LGUIDE – The e-Learning Experts Thesis: “A Framework for Comparing Web- Based Learning Environments”Thesis: “A Framework for Comparing Web- Based Learning Environments” Description Reasoning Reflection December 2002

20. Sociedade Portuguesa de Inovação Applied Methodology Initial Evaluation  Subjective component to the evaluation, although limited.  Short list of criteria sources – a need to review the sources utilized with other existing sources – American Society for Training & Development, Expert Papers  Clear need to develop widely accepted criteria to assess e-Learning practices – DG Education & Culture “eLearning call for Proposals”, September 2002 Description Reasoning Reflection December 2002

21. Sociedade Portuguesa de Inovação Applied Methodology - Recap December 2002

22. Sociedade Portuguesa de Inovação Applied Methodology Final Evaluation  Evaluated the top 20 e-Learning sites with regard to the following 3 areas: Learner Models – focus on flexible, personal models for learning.Learner Models – focus on flexible, personal models for learning. Soft Skills and Informal Learning – focus on the development of soft skills (communication, teamwork, leadership, project management, etc) and applying informal learning.Soft Skills and Informal Learning – focus on the development of soft skills (communication, teamwork, leadership, project management, etc) and applying informal learning. Communities of Learning – focus on identifying and implementing virtual communities, individuals with similar learning needs, within and across organisational boundaries.Communities of Learning – focus on identifying and implementing virtual communities, individuals with similar learning needs, within and across organisational boundaries. Description Reasoning Reflection December 2002

23. Sociedade Portuguesa de Inovação Applied Methodology Final Evaluation Description Reasoning Reflection  Learner Models evaluation was based on 10 criteria: Skills Gap Analysis – skill level evaluation prior to course Skills Gap Analysis – skill level evaluation prior to course Tutor Availability Tutor Availability Student Flexibility – course scheduling, course navigation Student Flexibility – course scheduling, course navigation Control of Screen Information – student level interface Control of Screen Information – student level interface Defined Objectives – course purpose, expected performance Defined Objectives – course purpose, expected performance December 2002

24. Sociedade Portuguesa de Inovação Applied Methodology Final Evaluation Description Reasoning Reflection  Learner Models evaluation was based on 10 criteria (continued): Self Assessment – periodically preferred Self Assessment – periodically preferred Content Integration – audio, video, slide show, animation Content Integration – audio, video, slide show, animation Document Management – bibliography, e-books Document Management – bibliography, e-books Help Facilities – beyond a tutor such as links, help desk Help Facilities – beyond a tutor such as links, help desk Simple User Interface – user friendly, consistent interface Simple User Interface – user friendly, consistent interface December 2002

25. Sociedade Portuguesa de Inovação Applied Methodology Final Evaluation Description Reasoning Reflection  Soft Skills and Informal Learning evaluation was based on 7 criteria: Interaction Activity – promoted and actual interaction Interaction Activity – promoted and actual interaction Reward – recognition or reinforcement of accomplishments: diploma, certificate Reward – recognition or reinforcement of accomplishments: diploma, certificate Transfer – relate course to the student’s “real world” Transfer – relate course to the student’s “real world” Environment – positive environment, tolerance of errors, appropriate feedback Environment – positive environment, tolerance of errors, appropriate feedback December 2002

26. Sociedade Portuguesa de Inovação Applied Methodology Final Evaluation Description Reasoning Reflection  Soft Skills and Informal Learning evaluation was based on 7 criteria (continued): Communication Interactivity – level of interaction: student / student and student / instructor Communication Interactivity – level of interaction: student / student and student / instructor Motivation – ability to motivate the student, justify the need to learn Motivation – ability to motivate the student, justify the need to learn Student Centred – ability to manage depth and scope of learning, schedule, contribution. Student Centred – ability to manage depth and scope of learning, schedule, contribution. December 2002

27. Sociedade Portuguesa de Inovação Applied Methodology Final Evaluation Description Reasoning Reflection  Communities of Learning evaluation was based on 8 criteria: Clearly Stated Purpose – establish a foundation for the community of learning Clearly Stated Purpose – establish a foundation for the community of learning Experienced Moderators – ability to initiate and guide discussions Experienced Moderators – ability to initiate and guide discussions Registration System – to allow students to present themselves to the community and provide the community a structure Registration System – to allow students to present themselves to the community and provide the community a structure December 2002

28. Sociedade Portuguesa de Inovação Applied Methodology Final Evaluation Description Reasoning Reflection  Communities of Learning evaluation was based on 8 criteria (continued): Synchronous Tools – allow real-time interaction (chat rooms, discussion boards, virtual meetings) Synchronous Tools – allow real-time interaction (chat rooms, discussion boards, virtual meetings) Asynchronous Tools – ability to initiate and guide discussions (e-mail, message posting on web forums) Asynchronous Tools – ability to initiate and guide discussions (e-mail, message posting on web forums) Group Problem Solving – foster groups with objectives within the course structure Group Problem Solving – foster groups with objectives within the course structure December 2002

29. Sociedade Portuguesa de Inovação Applied Methodology Final Evaluation Description Reasoning Reflection  Communities of Learning evaluation was based on 8 criteria (continued): Scheduled Events – ability to increase student awareness of group activities on particular topics Scheduled Events – ability to increase student awareness of group activities on particular topics Level of Interaction – final result: existence of actual communities with activity Level of Interaction – final result: existence of actual communities with activity December 2002

30. Sociedade Portuguesa de Inovação Applied Methodology Final Evaluation Description Reasoning Reflection  EC defined key topics of learning  Previous works: Previous studies within the e-learning field: LGUIDE – The e-Learning ExpertsPrevious studies within the e-learning field: LGUIDE – The e-Learning Experts Thesis: “A Framework for Comparing Web-Based Learning Environments”Thesis: “A Framework for Comparing Web-Based Learning Environments” Various articles and reports from “experts”Various articles and reports from “experts” December 2002

31. Sociedade Portuguesa de Inovação Applied Methodology Final Evaluation Description Reasoning Reflection  Subjective component to the evaluation, although limited.  Short list of criteria sources – a need to review the sources utilized with other existing sources – American Society for Training & Development  Clear need to develop widely accepted criteria to assess e-Learning practices – DG Education & Culture “eLearning call for Proposals”, September 2002 December 2002

32. Sociedade Portuguesa de Inovação Applied Methodology Letter of Interest to about 750 individuals involved/interested in Distance Learning Letter of Interest to about 750 individuals involved/interested in Distance Learning About 60 responses directly interested in the project findings About 60 responses directly interested in the project findings SeminarSeminar E-mail final reportsE-mail final reports Website: spi.pt/innoelearningWebsite: spi.pt/innoelearning Dissemination Activities December 2002

33. Sociedade Portuguesa de Inovação Contacts December 2002 Sociedade Portuguesa de Inovação Edf. Les Palaces Rua Júlio Dinis, 242 – Sala 208 4050-318 Porto, Portugal Tel: 351-22-6076400 Fax: 351-22-6099164 E-mail: markspinoglio@spi.pt www.spi.pt