LTR, The Marking System of CourseMaster Colin Higgins, Pavlos Symeonidis, Athanasios Tsintsifas Learning Technology Research (LTR), School of Computer Science and I.T., The University of Nottingham

LTR, Contents  Background  The CourseMaster system  Demonstration  Results  Future work

LTR, Background - CBA  Teaching can be fun  Marking is monotonously repetitive  Automatic assessment:  Saves time and costs  Enhances learning experience for students  Allows lecturers to concentrate on their teaching and research

LTR, LTR - Learning Technology Research  The LTR group is focusing its research in finding ways to facilitate teaching and enhance the learning experience  For the last four years, LTR has been developing the CourseMaster system  The Ceilidh system, the ancestor of CourseMaster, has been in use for over 10 years by more than 100 universities in the world First version, C module Ceilidh 2, C, C++ modules SML SQLPascal Diagrammatical Assessment Java X-Interface, Software-Tools WWW Interface, Prolog Z Course Master CD-ROM 2000

LTR, Problems with Ceilidh  Ceilidh is based on Unix  User interface is based on ASCII terminals  Ceilidh has scaling limitations  As Ceilidh evolved, it became:  harder to support  harder to extend

LTR, The CourseMaster System  CourseMaster is a courseware system  It provides:  Administration of courses  Assessment of student work  The presentation of information  It covers the various stages of automatic assessment:  development,  execution,  administration,  and management.

LTR, The CourseMaster System  CM is written in Java:  Portability,  Networking (with Remote Method Invocation – RMI),  Concurrency (with Threads),  Security,  Friendly GUI interface.  Runs on: Windows 95/98/NT/2000/XP, Solaris, Linux

LTR, Users’ Roles in CourseMaster  Students:  Setup exercises  Solve exercises  Develop and submit solutions  Receive feedback  Tutors / Lab assistants:  Register students  Monitor course state Less rights and responsibilities More rights and responsibilities StudentsTutors / Lab assistants TeachersSystem administrator Course developers

LTR, Users’ Roles in CourseMaster  Teachers:  Register tutors  Set / edit exercises  Monitor class’ progress  Course Developers:  Create courses / units  Author exercises  System administrators:  Monitor system logs  Install new courses  Give exercise extensions

LTR,  Automatic Marking in CM:  Based on criteria and metrics  Programming languages (C, C++, Java)  Diagrams (o-o design, circuit design, flowcharts)  In theory: marks any type of document (needs suitable metrics) CourseMaster Marking

LTR, Demonstration

LTR, Experience CM at Nottingham:  Has been running for 4 years  It has been used by 293 students in 2001/2002  Student feedback is positive – with minor exceptions  Teachers are very happy that they no longer need to mark hundreds of programs every weekend  Course administration and monitoring are very effective

LTR, CourseMaster at other sites  CM is running at: – The National University of Singapore – Helsinki University of Technology (Finland) – Kings College London (UK) – 15 other institutions around the globe  Used by up to 1500 students

LTR, Results : Benefits  Practical:  Saves time, most importantly with larger numbers of students (assessment, feedback, …)  Saves money (cost effective)  Can be administered by teaching assistants, thus freeing lecturers for teaching  Good use of course resources (materials, exercises, …)

LTR, Results : Benefits  Pedagogical:  Consistent assessment  Equitable assessment  Incremental assessment (they learn more)  Instant feedback to students (keeps them informed)  Detection and prevention of plagiarism

LTR, Future Work  Automatic ing of student progress to tutors  Assessment of object-oriented coursework (refactoring, o-o heuristics)  Assessment of Java programs with GUIs  A uPortal channel for CM  Composite Exercises

LTR, Summary  CourseMaster helps with the presentation, management and assessment of student work  CM is configurable for different educational requirements  Students are happy and academic staff even happier

LTR, Web Addresses  CourseMaster’s main page  CourseMaster’s on-line web client  CM’s Diagrammatical assessment information