Design, Implementation and Evaluation of Brokerage Systems of Learning Resources Lampros K. Stergioulas Department of Information Systems & Computing Brunel University, UK

2 Overview of presentation Universal Brokerage Platform Universal Brokerage Platform Value chain and user benefits Value chain and user benefits Platform Architecture Platform Architecture Service and LR Evaluation Service and LR Evaluation The need for Brokerage: eLearning business: far from lean eLearning business: far from lean Replication & Waste Replication & Waste Re-Use is spectacularly absent Re-Use is spectacularly absent

3 Universal Brokerage Platform The Universal Brokerage Platform is a distributed, multi-lingual system supporting the exchange of learning resources Challenges: Neutral business model To interface with heterogeneous content delivery systems such as – Web servers, – Streaming media servers, – Videoconference systems, and – Learning Management Systems

4 The UNIVERSAL Project: Universal Exchange for Pan-European Higher Education The UNIVERSAL Project: Universal Exchange for Pan-European Higher Education Focus on Higher Education Institutes (HEIs) Focus on Higher Education Institutes (HEIs) Brokerage Service: Not just a catalogue Brokerage Service: Not just a catalogue Aims to Serve: Aims to Serve: HEIs HEIs Corporate Training Corporate Training Individual learners Individual learners Consortium: Consortium: 17 members from the EU 17 members from the EU Higher Education Institutions with engineering and business studies Higher Education Institutions with engineering and business studies Technology supplier/integrator & Network service operators Technology supplier/integrator & Network service operators external collaborators in Canada and Iceland external collaborators in Canada and Iceland

5 Scope of work Key innovation: Flexible and open inter- organisational information system; tight integration of various content and live delivery systems, such as learning management systems, web server, media streaming server and video conferencing tools. Key innovation: Flexible and open inter- organisational information system; tight integration of various content and live delivery systems, such as learning management systems, web server, media streaming server and video conferencing tools. Learning Resource (Def.): a form (material or activity) of highly specialised academic content. A learning resource can be a short video or a complete course. Learning Resource (Def.): a form (material or activity) of highly specialised academic content. A learning resource can be a short video or a complete course.

6 Service:Brokerage of Learning Resources Approach: - Open exchange of LRs - Interface between brokerage platform and different delivery systems - Compatibility with various business models - Collaboration catalyst between Higher Education Institutions and companies - Standardised descriptions of pedagogical, administrative, and technical characteristics of LRs Platform Features: Enquiries Enquiries LR Provision LR Provision LR Booking LR Booking LR Delivery LR Delivery

7 Contributions of Universal The Technology for building knowledge exchanges and educational brokers is known as the Universal Brokerage Platform (UBP). The Technology for building knowledge exchanges and educational brokers is known as the Universal Brokerage Platform (UBP). The Service for sharing learning resources among faculty members in higher education is known as EducaNext. The Service for sharing learning resources among faculty members in higher education is known as EducaNext.

8

9 Benefits to Learners Higher degree of specialisation and variety of approach Higher degree of specialisation and variety of approach Wider choice of LRs Wider choice of LRs Lower cost and higher quality of teaching Lower cost and higher quality of teaching

10 Benefits to HEIs/Corporate Training Providers Faculties & organisations can easily re-use each other’s LRs enrich courses with external material, lowering the cost and raising the quality enrich courses with external material, lowering the cost and raising the quality Foster national and international academic alliances and exchanges Foster national and international academic alliances and exchanges Enable/reinforce partnerships between institutions of higher education and industry Enable/reinforce partnerships between institutions of higher education and industry Improve their teaching materials through feedback from users Improve their teaching materials through feedback from users Maximise use of their LRs, minimise “waste” Maximise use of their LRs, minimise “waste” Gain reputation in a constantly growing community Gain reputation in a constantly growing community Access new academic distribution channels Access new academic distribution channels Open Universities to extend range and depth Open Universities to extend range and depth

11 EU user-centred value objectives Openness of UBP (technical) Richness of LRs (pedagogical) Diversity of Users (marketing) Scope of delivery sys. supported - topics - country of origin - up-to-date - existing curricula Scope of geogr./ling. distribution

12 Broker Open System Architecture Open System Architecture for Material Content Delivery system (e.g. web server, Learning Management System) Metadata Learning resource Content Delivery system (e.g. Web server, Streaming Server, Video Conferencing Tool) Provider PC Provider system System interface

13 Open System Architecture for Live Activities 4. Consumption Delivery system for live educational activities (e.g. ISABEL) 2. Provision UNIVERSAL Brokerage Platform 1. Announce- ment 3. Booking 4. Consumption

14 Platform Architecture

15 User Interface Engine Main Functions To provide a user friendly dialogue capability between users and the platform To provide a user friendly dialogue capability between users and the platform To exchange information with other system components (engines), as required. To exchange information with other system components (engines), as required. Multilingual: English, French, German, and Slovenian Multilingual: English, French, German, and Slovenian Quick edit mode allows instantaneous adaptations for web page editors Quick edit mode allows instantaneous adaptations for web page editors

16 Administration Engine Administration Engine Acts as a records repository. Thus its main operational objectives are to keep records of users and transactions ( detailed access and transaction statistics) keep records of users and transactions ( detailed access and transaction statistics) provide data to other engines provide data to other engines Initiate, organise, and sustain actions in the platform Initiate, organise, and sustain actions in the platform Advanced user management Advanced user management notification management tools notification management tools Contract & Booking Engine Contract & Booking Engine CE: Enables formal transactions between users and platform and invokes contract formulation, acceptance and billing mechanisms Tracks Learning Resource Usage (Research, Teaching, Course Preparation, Preview) Tracks Learning Resource Usage (Research, Teaching, Course Preparation, Preview) Personalized List of Booked Learning Resources Personalized List of Booked Learning Resources Triggers Delivery (and Assessment) Triggers Delivery (and Assessment) Notifies Learning Resource Provider Notifies Learning Resource Provider

17 User Profile Engine is responsible for creating and maintaining users’ profiles and user authentication files LR Profile Engine LR Profile Engine This engine keeps LR information. Its main operational objectives are to store LR data store LR data search LR data and provide information in response to user requests. search LR data and provide information in response to user requests.

18 Evaluation Engine The Evaluation engine supports quality evaluation of learning resources and platform activities. In doing so, it is focused on the following operational objectives: to collect evaluation data and to collect evaluation data and provide search and analysis capability on evaluation and other user-related data provide search and analysis capability on evaluation and other user-related data

19 Delivery Engine The brokerage platform does not store LR contents; instead it offers an interface layer which provides communication functionality between the platform and various delivery systems. This is the responsibility of the delivery engine, which provides authentication and authorization services, delivery negotiation and delivery supervision. Presently, the platform provides support to the following set of delivery systems: Standard web server Standard web server Hyperwave’s E-learning Suite Hyperwave’s E-learning Suite RealNetworks Real server RealNetworks Real server Isabel Teleconferencing Suite Isabel Teleconferencing Suite

20 Delivery

21 Platform Data Management User Database User Database Learning Resource Database Learning Resource Database Evaluation Database Evaluation Database System Log Database System Log Database Booking Database Booking Database Evaluation Data LR Quality evaluation Questionnaires LR evaluation Data UPB Quality evaluation Questionnaires UBP evaluation Data

22 Evaluation Services This activity is focused on: Evaluation of LR quality and collection of feedback that can be used to: Evaluation of LR quality and collection of feedback that can be used to:  improve LRs  encourage acceptability (and even accreditation) of LRs by professional institutions and educational organisations (HEI’s). Evaluation of the effectiveness and completeness of the brokerage service, which will provide a measured understanding of the quality of brokerage service and leads to future evolutionary/improved versions of the system. Evaluation of the effectiveness and completeness of the brokerage service, which will provide a measured understanding of the quality of brokerage service and leads to future evolutionary/improved versions of the system.

23 LR Evaluation Service In LR Evaluation Service, the platform collects from users LR evaluation data which is then stored in the LR evaluation database. This data is analyzed by the evaluation engine and is transformed into meaningful LR evaluation information, which adds effectively a unique “LR quality” dimension to the UNIVERSAL brokerage operation. In LR Evaluation Service, the platform collects from users LR evaluation data which is then stored in the LR evaluation database. This data is analyzed by the evaluation engine and is transformed into meaningful LR evaluation information, which adds effectively a unique “LR quality” dimension to the UNIVERSAL brokerage operation. Fast feedback support via spot light reviews and meta reviews, “ How useful was this review? ” Fast feedback support via spot light reviews and meta reviews, “ How useful was this review? ” Option to create customised evaluation questions and integrate them into a powerful evaluation tool Option to create customised evaluation questions and integrate them into a powerful evaluation tool

24 LR evaluation data is collected from the following users: Learners (students – consumers) Learners (students – consumers) HEI administrators and academic staff (faculty consumers) HEI administrators and academic staff (faculty consumers) LR providers LR providers The evaluation data collection process is initiated by the administration engine. The administration engine logs all platform usage activity, including LR delivery, and is in a position therefore to activate the LR evaluation data collection process on-line. LR evaluation: Data Collection LR evaluation: Data Collection

25 Platform/Service evaluation Technical evaluation Technical evaluation Functionality evaluation Functionality evaluation User Satisfaction User Satisfactionvia Automatic logging activities Automatic logging activities User feedback/comments/complaints User feedback/comments/complaints Automatic alert system Automatic alert system User Driven User Driven System Activated System Activated

26 Current Status Service available on-line since April 2002 Service available on-line since April 2002 LR Usage and LR quantity growing exponentially LR Usage and LR quantity growing exponentially Continuous Evaluation and Refinement Continuous Evaluation and Refinement Peer Reviews Peer Reviews MISSION STATEMENT: UNIVERSAL is a world-class brokerage service linking educators for exchanging/distributing learning resources.

27 Universal Brokerage EducaNext

28 Further information can be Brokerage Service for Learning Resources available at