STEM 2012 Automatically assessing free-form diagrams in E-assessment systems Pete Thomas Kevin Waugh and Neil Smith Open University, UK.

Slides:

Advertisements

Similar presentations

E-books and E-journals Off-campus This presentation will show you how to log in and access Oxford Brookes Library e-books and e-journals when youre off.

Advertisements

Forward Back 23 April 2014Alex MacDonald Q- Project.1 Genre WebQuest. You can print off these instructions, or ask for a paper copy from your teacher.

E-books and E-journals Off-campus This presentation will show you how to log in and access Oxford Brookes Library e-books and e-journals when youre off.

Chapter 11 Designing the User Interface

EiS – Education iT Services Our passion in EiS is to make a real difference in education and ultimately childrens lives by providing innovative solutions.

Visual Scripting of XML

ARCH-05 Application Prophecy UML 101 Peter Varhol Principal Product Manager.

Use Case - Example University library system requirements

Use Case Diagram © copyright 2001 SNU OOPSLA Lab..

2-1 © Prentice Hall, 2007 Chapter 2: Introduction to Object Orientation Object-Oriented Systems Analysis and Design Joey F. George, Dinesh Batra, Joseph.

Summary Class responsibility cards can be used to help allocate responsibilities between different classes. The use of stereotype classes, such as entity,

Practical Object-Oriented Design with UML 2e Slide 1/1 ©The McGraw-Hill Companies, 2004 PRACTICAL OBJECT-ORIENTED DESIGN WITH UML 2e Chapter 5: Restaurant.

Design Patterns in Java Appendix D UML at a Glance Summary prepared by Kirk Scott 1.

Which of these images best illustrates Peer Assessment?

Essentials of interaction diagrams Lecture 23 & 24.

© 2005 Prentice Hall4-1 Stumpf and Teague Object-Oriented Systems Analysis and Design with UML.

Component and Deployment Diagrams

© Copyright Eliyahu Brutman Programming Techniques Course.

System Analysis and Design

03/12/2001 © Bennett, McRobb and Farmer Use Case Diagrams Based on Chapter 6 of Bennett, McRobb and Farmer: Object Oriented Systems Analysis and.

IT Skills Use Case Diagrams. Objectives Richard Hancock  Define what a use case is  Describe the common elements of a use case diagram  Be.

Firat Batmaz, Chris Hinde Computer Science Loughborough University A Diagram Drawing Tool For Semi–Automatic Assessment Of Conceptual Database Diagrams.

Structured Vs. Object Oriented Analysis and Design SAD Vs. OOAD

Lawson System Foundation 9.0

MASTERS THESIS DEFENSE QBANK A Web-Based Dynamic Problem Authoring Tool BY ANN PAUL ADVISOR: PROFESSOR CLIFF SHAFFER JUNE 2013 Computer Science Department.

Smith’s Aerospace © P. Bailey & K. Vander Linden, 2005 Architecture: Component and Deployment Diagrams Patrick Bailey Keith Vander Linden Calvin College.

Intended for novice users as an introduction to the online catalog’s capabilities. The guide would be available on the New Brighton Public Library’s website.

Document that explains the chosen concept to the animator.

14 Chapter 11: Designing the User Interface. 14 Systems Analysis and Design in a Changing World, 3rd Edition 2 Identifying and Classifying Inputs and.

Moodle (Course Management Systems). Forums, Chats, and Messaging.

High-Level Design With Sequence Diagrams COMP314 (based on original slides by Mark Hall)

1 Object orientation. 2 What benefits does OO give? Primarily –Encapsulation (Associates data & operations) –Types & specialisation –Software re-use.

SHAUN WERSTED’S CAPSTONE PRESENTATION. INTRODUCTION n PROBLEM: Teachers and students need a resource for rampant absenteeism.

L8 - March 28, 2006copyright Thomas Pole , all rights reserved 1 Lecture 8: Software Asset Management and Text Ch. 5: Software Factories, (Review)

Chapter 5 Models and UML Notation for The Object-Oriented Approach.

Software Engineering Lecture 10 UML vs. ERD.

Collaboration Diagrams CSIS3600. What is a Collaboration Diagram Collaboration diagrams illustrate interactions between objects The collaboration diagram.

Carmen David, Senior BA Business Analysis Carmen David, Senior BA Business Analysis Foundation in Business Analysis Session 7 MODELLING REQUIREMENTS.

Sequence Diagrams And Collaboration Diagrams HungNM.

Appendix D UML at a Glance Summary prepared by Kirk Scott 1.

Welcome This is a document to explains the chosen concept to the animator. This will take you through a 5 section process to provide the necessary details.

Assessment Analysis Using MarkIt CECIL Users Meeting 6 May 2004.

Storyboard, Design, Layout.  Create storyboards  Decide upon navigation  Prepare short page summaries.

OOAD (part 2) diagrams CSCI577a TAs: Alexey. Outline UML diagrams: – Use case diagrams – Robustness diagrams – Sequence diagrams – Artifacts diagrams.

 What to do if you want to build a new house? › Buy a bunch of wood and nails and start immediately. › Or, put some blueprints to follow, and plan of.

Object Oriented Analysis and Design Introduction to Rational Rose.

 ActiveLearn GCSE Edexcel Maths. Online Homework & Revision Online Homework & Revision Matched to Course Specification Matched to Course Specification.

George Wang, Ph.D. COMP 380/L Lesson 2. Use Case Use cases are a way to capture system functionalities (i.e., functional requirements) Based on use case.

Data Informed Practice. Problem…  School performing well below state and national averages in Reading.  PM Reading Levels across multiple grades below.

1 M206 Chapter 31: An Overview of Software Development 1.Defining the problem 2.Analyzing the requirement – constructing initial structural model 3.Analyzing.

Requirements Models Representing the Product in Ways Other than Text.

Computer Representation of Venn and Euler Diagrams Diunuge B. Wijesinghe, Surangika Ranathunga, Gihan Dias Department of Computer Science and Engineering,

Document that explains the chosen concept to the animator

Business Process and Functional Modeling

Collaboration Diagrams

The Movement To Objects

Evolution of UML.

Chapter 11: Collaboration Diagram - PART1

Unified Modeling Language

Object-Oriented Systems Analysis and Design Using UML

Instructor Name Instructor Title Library Name

James Miller, Julia John

The Process of Object Modeling

H5P: Using an Interactive Assessment Tool in Moodle

Basic know how ???.

Requirements To Design In This Iteration

Analysis models and design models

Powered by: Jazz Presentation

The Moodle eLearning platform -University of Pitesti-

SDMX IT Tools SDMX Registry

Presentation transcript:

STEM 2012 Automatically assessing free-form diagrams in E-assessment systems Pete Thomas Kevin Waugh and Neil Smith Open University, UK

STEM 2012 Aim To extend the capabilities of e-assessment systems (online tests) to include questions that require students to draw free-form diagrams in answers. In particular: –Minimal constraints on what might be drawn –Not multiple choice –Support for a variety of diagram types –Automatic marking –Moodle VLE (via OpenMark)

STEM 2012 Example Student draws diagram in response to question

STEM 2012 What is a diagram? Free-form –Not free-hand –Uses a drawing tool Specialised for the domain by the teacher As unconstrained as the teacher wishes Graph based –Shows relationships between objects

STEM 2012 Biological Flow Diagram Illustrates the flow of hormones between glands.

STEM 2012 Entity-Relationship Diagram Shows the relationships (links) between entities (boxes)

STEM 2012 Use case diagram (UML) Shows the actions that an actor Book Borrower takes part in when using a library

STEM 2012 Spray Diagram Illustrates how UK farms have diversified in recent times.

STEM 2012 Spatial Relationships It is the relative positions of boxes that determines whether a relationship exists. Adult and Child are inside Person – inside indicates subtype relationship..

STEM 2012 Sequences time The messages, represented by arrows, are sequenced in time.

STEM 2012 Question in VLE test This is what the user sees in their browser

STEM 2012 Run the Drawing Tool Up pops the drawing tool – an applet

STEM 2012 Automatic marker provides feedback Feedback from the automatic marker

STEM 2012 Get Answer

STEM 2012 Automatic Marking The students diagram is compared with the model solution(s) to see how well the objects and the relationships match. Similarity measures of how well each element on the student diagram matches with an element on the model solution. –A mixture of NLP and heuristics A marking scheme is applied. –Designed by the question setter from a template

STEM 2012 Does it work? – Experiment diagrams from a real exam - marked out of 7. Difference between auto and human markers Number % Cumulative % Main corpus Diagrams only have entity types and relationships Published in 2011

STEM 2012 Comparison Human – Auto (Distribution of marks)

STEM 2012 Support Tools An authoring tool for teacher to design the question, the model solution(s) and the mark scheme. A specification tool to define a type of diagram. –Used to specialise both the drawing tool and the automatic marker A marking tool for marking sets of diagrams off-line.

STEM 2012 Authoring Tool for Questions, Model Solutions and Marking Schemes Question (text) Model Solution (diagram)

STEM 2012 Specification Tool A graphical user interface for defining a new type of diagram

STEM 2012 Summary A new question type for online tests Free-form diagrams (graph based) Automatic marking Implemented in Moodle Set of support tools

STEM 2012 Thank You We always want more examples of types of diagram. Send me your suggestions. There is a website where you can try this technology out. Please contact me for details.