Using Open Technical E-learning Standards and Service-orientation to Support New Forms of E-assessment Yongwu Miao, Colin Tattersall, Judith Schoonenboom, Krassen Stevanov, Adelina Aleksieva-Petrova Manchester, January 12, 2007

Overview Introduction to New Forms of Assessment Characteristics of New Forms of Assessment A Standard-based Approach to Model New Forms of Assessment Delivering New Forms of Assessment in Today’s Open Infrastructure Comparison with Software Development Approaches to Support New Forms of Assessment Conclusions and Future Work

New Forms of Assessment Traditional forms of assessment multiple-choice, filling-in-blank, sliding, ranking, matching, …… New forms of assessment (NFA) self-/peer assessment, 360 degree feedback, portfolio assessment, …… Competence assessment is usually based on new forms of assessment

Characteristics of New Forms of Assessment Involvement of multiple roles/users Variety in task types Complex process control Exchange of information Explaining through a peer assessment example

A Peer Assessment Example Students are instructed how to conduct this peer assessment Each student selects a different paper and reads it Each student then prepares a brief article ( words) about their chosen paper Pairs of students then exchange articles and review each other’s articles Each student writes a response to the review, revise the article, and then submit documents including paper, original article, reviewed article with the reviewer’s comments, response and the final version of their article to the tutor The tutor then marks on students’ exercises

learner1learner2tutor Involvement of multiple roles/users A Peer Assessment Example

instruction selecting/reading paper1 writing article1 reviewing article2 responding review1 instruction selecting/reading paper2 writing article2 reviewing article1 responding review2 learner1learner2tutor Variety in task types final assessment2final assessment1 A Peer Assessment Example

instruction selecting/reading paper1 writing article1 reviewing article2 responding review1 instruction selecting/reading paper2 writing article2 reviewing article1 responding review2 final assessment2final assessment1 learner1learner2tutor Complex process control A Peer Assessment Example

Exchange of information instruction selecting/reading paper1 writing article1 reviewing article2 responding review1 instruction selecting/reading paper2 writing article2 reviewing article1 responding review2 final assessment2final assessment1 learner1learner2tutor paper2 article2 review1 response2 paper1 article1 review2 response1 A Peer Assessment Example

A Standard-based Approach A combined use of IMS LD, IMS QTI, and assessment- specific services –Using IMS LD to model multi-role/user-involved assessment processes –Using IMS QTI to model traditional assessment tasks and using assessment-specific services to model special assessment tasks –Using IMS LD to model a learning flow with complicated process controls –Using IMS QTI, IMS LD, and assessment-specific services to model complicated dataflow As a consequence, a competence assessment process can be modeled as a unit of assessment, a specific unit of learning which contains assessment-specific elements (e.g., QTI documents and assessment- specific services

learning design (or assessment design) unit of learning (or unit of assessment) roleactivityenvironmentproperty component methodcondition expressionaction play act role-part learningsupportservicelearning objectstafflearner internalchange-property notification show/hide assesseeassessor assessment- specific service item activity-description resource QTI_type external assessment An extended IMS LD model assessment_itemoutcome item_body assessment_test interactionresponse response_processing A simplified IMS QTI model

Delivering New Forms of Assessment Using today’s open technical e-learning infrastructure Including IMS LD compatible engine like CopperCore Including IMS LD compatible client like SLeD Including IMS QTI compatible player like APIS Including assessment-specific services like a concept-mapping tool and a simulation tool

Delivering New Forms of Assessment A screenshot of execution of peer assessment example

Comparison with Traditional Software Development Approaches Many software tools supporting new forms of assessment have been developed and used. In comparison with these tools, our approach has advantages in Supporting for interoperability Supporting for flexibility Supporting for seamless integration with learning processes

Conclusions and Future Work Conclusions: –New forms of assessment can be modelled as units of assessment through a combined use of IMS LD, IMS QTI, and assessment-specific services –In comparison with traditional software development approaches, the standard-based approach fosters interoperability, flexibility, and seamless integration with learning activities Future Work: –developing a high-level assessment authoring tool which enables practitioners to define their own customized assessment models –Developing mechanisms to transform such assessment models into units of assessment automatically

Thanks for attention Questions?