Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006,

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Demetrios G Sampson Advanced e-Services for the Knowledge Society Research Unit (A.S.K.) Informatics & Telematics Institute (I.T.I.) The Centre of Research and Technology Hellas (C.E.R.T.H.) TenCompetence The European Network for lifelong competence development This work is licenced. This work is licenced under the Creative Commons Attribution-NoDerivs-NonCommercial. To view a copy of this licence, visit or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Outline (1/2) Competence-based Learning: The need for a new generation of Learning Services The TenCompetence Approach –Competence Development Lifecycle –The TenC Cycle for Lifelong Competence Development –TenC Competence Domain Model

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Outline (2/2) WP6 Learning Activities & Units of Learning –TenC key problems tackled in WP6 –Overall WP6 Objectives –WP6 Contributions to TenCompetence Domain Model –Overall WP6 Activities WG-A (IMS LD Authoring) WG-B (Assessment Model) WG-C (IMS LD Run-time) WP8 Task 1: Competence Observatory –TenC key problems tackled in WP8 T1 –Scope of the TenC Competence Observatory –Targeted User Groups

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, TenCompetence Project facts Duration: December 2005-November 2009 An Integrated Project funded under the IST Programme, Sixth Framework Programme, European Commission Estimated Budget: 14MEuro total with 8,5 MEuro public funding 13 partners from 9 different countries – Open Universiteit Nederland, University of Bolton, CERTH, INSEAD, Universitat Hannover, Universitat Pompeu Fabra, Universiteit van Amsterdam, Sofia University –Software de Base, LogicaCMG, GIUNTI, Stichting SURF, Synergetics

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Central TenCompetence Problem The knowledge society demands continuous competence development and management at the individual-, group- and organizational level Competence development and management for individuals, groups and organizations are distinct fields with their own approaches and tooling Integrated support for informal and formal learning is missing

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, TenCompetence Goal TENCompetence will support networks of individuals, groups and organisations in Europe in lifelong competence development by establishing the most appropriate technical and organisational infrastructure, using open source standards-based, sustainable and innovative technology

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Aspects to be integrated Knowledge Resource Sharing & Management Learning Activities & Units of Learning Competence Development Programmes Networks for Lifelong Competence Development

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Integration Levels

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, The TenCompetence Approach Support –Individuals –groups –organizations In –Lifelong competence development By –Pedagogical Models –Organizational Processes –Technological infrastructure Using –Open-source, standards-based technology [TenCompetence DoW pp. 5, 10]

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Competence Development Lifecycle Identification of Job Roles and relevant competencies 1.Assess individual’s competencies. 2.Gap Analysis between existing competencies and the required competencies for a specific job role. Design Competence Development Programmes to minimize the identified gaps Continuous Performance Monitoring and Assessment to confirm improvement.

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, TenC Cycle for Lifelong Competence Development Competence Programme Units of Learning Knowledge Resources populated with as a synthesis of Organizations Network 1.Specify relevant Job Roles 2.Define required competencies for each job role 3.Assess the competencies of their employees 4.Define and deliver competence development programmes 5.Monitor their employees performance Competencies Description Competencies Assessment Gap Analysis Performance Assessment design a Individuals Network 1.Join community of relevant Job Role 2.Assess own competencies for the specific Job Role 3.Search/Find relevant Competence Development Programme 4.Share experiences with other members of the network execute WP5 WP6 WP7

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, TenC Core Functional Services WP 8 WP 7 WP 6 WP 5 refer to use populated with

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Key role for 'competences' The concept of competence can bridge the world of education, training, knowledge management, human resource management & informal learning Many definitions, problem for system development - Competence: Effective performance in a domain at different levels of proficiency - Competency: Undefined (we prefer to avoid the term and use more specific terms when needed, eg, a skill) We use a stipulative definition and model in TENCompetence to avoid endless discussions about the concept

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, TenC Competence Domain Model [Rob Koper, TenC Domain Model v1.0, 19 June 2006]

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, TenC WorkPackages WP 1 WP 2 WP 3 WP 4 WP 10 WP 5 WP 6 WP 7 WP 8 WP 9

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Issues to be tackled per Work Package WPIssue 3 integrate isolated models and tools for competence development into a common, easy to use infrastructure 4 run pilots to ensure the validity and viability of the approach 5 develop software for the effective support of users who create, store, use and exchange knowledge resources within a learning network 6 develop new innovative pedagogical approaches and assessment models for lifelong competence development develop software for the effective support of users who create, store, use and exchange learning activities and units of learning within a learning network 7 develop software for the effective support of users who create, store, use and exchange competence development programmes within a learning network 8 develop new innovative organisational models for lifelong competence development 9 deliver training programs to learn users how to work with the infrastructure, and to train instructors and companies (specifically SMEs) to deliver services using the infrastructure 10 build a growing network of associated partners to ensure large-scale use in Europe

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Work Package 6 Learning Activities & Units of Learning

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, TenC key problems tackled in WP6 TenC Issue #2: For individuals, groups and organisations in Europe, it is still hard to get an overview of all possible formal and informal knowledge resources, units of learning, programmes and learning networks that are available, and to identify the most appropriate for their needs. WP6 Specific Objective: Research and develop innovative, standards-based methods and tools for the exchange and quality rating of formal and informal learning activities and units of learning. TenC Issue #4: For an organisation in Europe it is still hard to assess the competencies of applicants, employees and learners who have studied and worked in a variety of settings. WP6 Specific Objective: Develop a formal specification model and supporting tools that combines new assessment types and the ones included in the IMS QTI (providing input for standards development). TenC Issue #5: Current e-learning and knowledge management environments provide too little effective support to the users in their various tasks. WP6 Specific Objective: Research and develop innovative, standards-based methods and tools for the creation, storage and use of formal and informal learning activities and units of learning.

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Overall WP6 Objectives Research and develop innovative, standards-based methods and tools for the creation, storage, use, exchange and quality rating of formal and informal learning activities and units of learning, including methods and tools for the assessment of the learning process and learning outcomes.

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, WP6 Contributions to TenCompetence Domain Model (1/2) WP6 related items

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, WP6 Contributions to TenCompetence Domain Model (2/2) Component B Component A Component C Assessment Component Phase 1: Specification Definition and proof-of-concept tool Phase 2: Specification Elaboration and TenC Assessment Tool Phase 3: Preparation for International Specification Authoring Component Phase 1: Specification of IMS LD Authoring Tool Phase 2: TenC Authoring Tool v1.0 (implementing IMS LD) Phase 3: TenC Authoring Tool v2.0 (implementing IMS LD + the new Assessment Specification) Run-time Component Phase 1: Connector Protocol Definition Phase 2: TenC IMS LD Run-time Engine (implementing the protocol) and Protocol Elaboration Phase 3: TenC Run-time Engine (implementing IMS LD + the protocol + new Assessment Specification) Working Group B Working Group A Working Group C

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, WP6: WG-A (IMS LD Authoring) Activities Issue to be tackled: Although there are a number of existing IMS LD Authoring Tools (such as Reload LD Editor, CooperAuthor), in TenCompetence a higher-level graphical Learning Flow tool is needed to enable practitioners visualize and assemble Units of Learning easily. The design paradigm of this tool should be closer to the common practice on designing pedagogical scenarios, rather than to the XML-based structure of the IMS Learning Design specification. Main activities for Working Group A: a)Definition of an algorithm for machine (automatic) transformation of practitioners’ workflow design paradigms to the lower XML language of IMS LD and vice versa b)Architectural design and implementation of the TenC IMS LD Authoring Tool prototype c)Creation of a set of examples to be used for testing the TenC IMS LD Authoring Tool d)Planning and execution of a usability evaluation study (in respect to the provided flexibility for learning designers and practitioners to define their own pedagogical scenarios) of the TenC IMS LD Authoring Tool

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, WP6: WG-B (Assessment Model) Activities Issue to be tackled: Assessment in lifelong competence development is a complex process with many actors, which can and needs to be supported in a variety of ways. The scope of the TenC Assessment Specification is to provide the means for defining assessment processes, as an internal part of the design process of a Unit of Learning, by combining new types of assessment (e.g. 360-degree assessment, portfolio assessment) with the ones already included in the IMS QTI specification. Main activities for Working Group B: a)Definition of an Assessment Specification and integration with the IMS Learning Design specification b)Development of a proof-of-concept Assessment Authoring Tool for demonstrating the use of the TenC Assessment Specification and its integration with IMS QTI c)Creation of a set of examples to be used for validating the TenC Assessment Specification d)Planning and execution of validation testing of the TenC Assessment Specification via the use of the Assessment Authoring Tool

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, WP6: WG-C (IMS LD Run-time) Activities Issue to be tackled: The range of services available in the IMS Learning Design specification is restricted. The current structure for a service in the IMS LD specification essentially maps an open set of roles to a fixed set of service roles, limiting the ability of IMS LD Run-time engines to connect to external communication and collaboration services (e.g Instant Messaging, Chat services). Main activities for Working Group C: a)Definition of a specification to connect communication and collaboration services to IMS LD Runtime Engines b)Enhancement of existing IMS LD Run-time Engines (namely, Reload Player and Coopercore) to support the TenC Services Connection Specification c)Creation of a set of examples to be used for validating the TenC Services Connection Specification d)Planning and execution of validation testing of the TenC Services Connection Specification via the use of the enhanced IMS LD Runtime Engines

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Work Package 8 – Task 1 Competence Observatory

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, TenC key problems tackled in WP8 T1 TenC Issue #4: For an organization in Europe it is still hard to assess the competencies of applicants, employees and learners who have studied and worked in a variety of settings. To this end TENCompetence will provide models and software tools to assess the competencies of individuals, groups and organizations in an exchangeable way. An interoperable mechanism to express and exchange learners’ competencies will be developed, especially for informally acquired knowledge and competencies. Core aspects of such a mechanism are methods and technologies for interoperable person and group portfolio's and personal development plans, and interoperable competence definitions. WP8 Task 1 Specific Objective: Develop a Competence Observatory to monitor and capture the competences that have to be acquired in different professional and academic fields, in an interoperable way following the current state of art specifications namely IMS RDCEO and HR-XML.

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, TenC Competence Domain Model [Rob Koper, TenC Domain Model v1.0, 19 June 2006] WP8 T1 Competence Observatory

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, TenC Competence Observatory Scope Monitor and capture the competencies that have to be acquired in different professional and academic fields Provide the infrastructure for specifying competencies in an interoperable way Provide the communication tools, which will allow competence experts and practitioners to discuss and exchange opinions about competence-related issues

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, TenC Competence Targeted User Groups The targeted groups of the TENC Competence Observatory are: –Enterprises or organizations who are interested in defining job profiles for their employees –Individuals who have a specific profession and they want to assess their proficiency level by comparing their own competencies with reference ones for their profession Organizations and Individuals can be seen as “Communities related with specific job roles and they want to share experiences related to required competencies for a specific job role” The TenC Competence Observatory needs to support Communities of Professions

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Main Functionalities Information ServicesCommunication ToolsResource Management NewsSearch EnginesForumChatWikiLinksResource Library  Competence related functionalities: –Searching specific competencies, and exporting them as XML files –Creating competencies descriptions –Editing/updating competencies descriptions Basic functionalities:

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Using the TenC Competence Observatory

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Review of Competence Models and Observatory Tools

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Definition of Competency and Competency Models A Competency can be defined as “A set of skills, related knowledge and attributes that allow an individual to perform a task or an activity within a specific function or job” [1] The Competency Description Models define an information model for describing, referencing, and exchanging definitions of competencies. Two existing specifications for describing competencies. –IMS RDCEO specification from e-learning domain –HR-XML specification from Human Resources domain [1] United Nations Industrial Development Organization, “UNIDO Competencies”, 2002, Available Online at: CompetencyModel-Part1.pdf, Last retrieved in 13 April 2006

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, IMS RDCEO Key Elements (1) Identifier: A globally unique label that identifies this definition of competency or educational objective. The “Identifier” element consists of two other sub-elements: “Catalog” and “Entry”. Title: A short name for this competency or educational objective. The “Title” may be repeated in multiple languages. Description: A narrative description of the competency or educational objective. The “Description” may be repeated in multiple languages. Definition: A structured description that provides a more complete definition of the competency or educational objective, using a collection of statements that determine a competency or an educational objective. Typically, such models define a competency or educational objective in terms of a “statement, conditions, criteria”, “proficiency, criteria, indicators”, “standards, performance indicators, outcomes”, “abilities, basic skills, content, process”, and similar sets of statements. The “Definition” consists of two other sub-elements: “Model Source” and “Statement”. Metadata: Optional meta-data record that further describe the RDCEO. The meta- data records must be conform to IEEE (IEEE Learning Object Metadata).

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, IMS RDCEO Key Elements (2)

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Disadvantages of IMS RDCEO How to represent the level of a competency? The IMS-RDCEO specification supports the representation of a competency level, within the “title” element. The information stored within this element is in a narrative format, thus, it is not machine understandable and limits the scope of interoperability among different systems. How to represent the grading scale of a competency? The IMS-RDCEO specification does not provide a way to represent the “grading scale” of a competency, thus, provides limited support for the assessment of complex competencies. How to represent the success threshold of a competency? The IMS-RDCEO specification does not support the definition of a “success threshold” for a competency. Therefore, a learning system cannot interpret the existence of a competence. How to describe complex competencies in an interoperable way? The IMS-RDCEO already supports the definition of complex competencies (that is, any competency consisting of other – simple or complex – competencies) through the use of “metadata/relation” element. However, it does not provide a way to represent the weighting factor of sub-competencies when assessing a complex one, thus, provides limited support for the assessment of complex competencies.

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, HR-XML Key Elements (1) Name: A short name for the related competency. Description: A narrative description of the competency. Required: A boolean used to indicate whether the CompetencyEvidence is mandatory for a particular position or given context. CompetencyId: An identification code assigned to identify or classify the competency. TaxonomyId: A code that identifies the taxonomy of the competency. CompetencyEvidence: A text label that is used to capture information to substantiate the existence, sufficiency, or level of a Competency. CompetencyEvidence might include test results, reports, performance appraisals, evaluations, certificates, licenses, or a record of direct observation, such as a report given by a former supervisor or other employment reference. CompetencyWeight: A text label that allows the capture of information on the relative importance of the Competency. Competency: Competencies can be recursive. A competency may include other competencies. One competency might be decomposed into several component competencies, each of which might be separately measurable. UserArea: Personal information about the individual that holds the defined competency.

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, HR-XML Key Elements (2)

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Disadvantages of HR-XML How to represent the level of a competency? The HR-XML specification supports the representation of the proficiency level of a competency, within the “CompetencyWeight” element. This element also permits the capture of the minimum and the maximum value for the proficiency level of the relative competency. How to represent the grading scale of a competency? The HR-XML specification does not provide a way to represent the “grading scale” of a competency, thus, provides limited support for the assessment of complex competencies. How to represent the success threshold of a competency? The HR-XML specification does not support the definition of a “success threshold” for a competency. Therefore, a learning system cannot interpret the existence of a competence. How to describe complex competencies in an interoperable way? The HR-XML supports the definition of complex competencies through the use of “Competency” element. However, it does not provide a way to represent the weighting factor of sub- competencies when assessing a complex one, thus, provides limited support for the assessment of complex competencies.

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Mapping of IMS-RDCEO and HR-XML (1)

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Mapping of IMS-RDCEO and HR-XML (2)

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, ASK-CDM Key Elements (1) Identifier: A globally unique label that identifies this definition of competency or educational objective. The “Identifier” element consists of two other sub-elements: “Catalog” and “Entry”. Title: A short name for this competency. The “Title” may be repeated in multiple languages. Description: A narrative description of the competency or educational objective. The “ Description ” may be repeated in multiple languages. ProficiencyLevel: The proficiency level of the competency. There may be multiple instances of this category. Classification: This category describes where this competency falls within a particular classification system. To define multiple classifications, there may be multiple instances of this category. Relation: Competencies can be recursive. A competency may refer to other competencies.

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, ASK-CDM Key Elements (2)

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Case Study: The Europass Europass has recently adopted by the European Parliament, in order to help citizens to convey their qualifications and competences in an effective way, by providing a comprehensive tool for users based on an accessible electronic format. The Europass consists of five documents: –Europass CV –Europass Language Passport –Europass Mobility –Europass Diploma Supplement –Europass Certificate Supplement

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006,

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006, Presenter’s Contact Information Advanced e-Services for the Knowledge Society Research Unit (A.S.K.) Informatics & Telematics Institute (I.T.I.) The Centre of Recearch and Technology Hellas (C.E.R.T.H.) Postal Address: 42 Arkadias & Taygetou Str., GR-15234, Chalandri, Athens, Greece TEL: /17, FAX: , URL: