1. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk The Marking System of CourseMaster Colin Higgins, Pavlos Symeonidis, Athanasios Tsintsifas Learning Technology Research (LTR), School of Computer Science and I.T., The University of Nottingham ltr@cs.nott.ac.uk http://www.cs.nott.ac.uk/CourseMaster

2. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk Contents  Background  The CourseMaster system  Demonstration  Results  Future work

3. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk Background - CBA  Teaching can be fun  Marking is monotonously repetitive  Automatic assessment:  Saves time and costs  Enhances learning experience for students  Allows lecturers to concentrate on their teaching and research

4. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk LTR - Learning Technology Research  The LTR group is focusing its research in finding ways to facilitate teaching and enhance the learning experience  For the last four years, LTR has been developing the CourseMaster system  The Ceilidh system, the ancestor of CourseMaster, has been in use for over 10 years by more than 100 universities in the world 198819901992199919951998 First version, C module Ceilidh 2, C, C++ modules SML SQLPascal Diagrammatical Assessment Java X-Interface, Software-Tools WWW Interface, Prolog Z Course Master CD-ROM 2000

5. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk Problems with Ceilidh  Ceilidh is based on Unix  User interface is based on ASCII terminals  Ceilidh has scaling limitations  As Ceilidh evolved, it became:  harder to support  harder to extend

6. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk The CourseMaster System  CourseMaster is a courseware system  It provides:  Administration of courses  Assessment of student work  The presentation of information  It covers the various stages of automatic assessment:  development,  execution,  administration,  and management.

7. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk The CourseMaster System  CM is written in Java:  Portability,  Networking (with Remote Method Invocation – RMI),  Concurrency (with Threads),  Security,  Friendly GUI interface.  Runs on: Windows 95/98/NT/2000/XP, Solaris, Linux

8. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk Users’ Roles in CourseMaster  Students:  Setup exercises  Solve exercises  Develop and submit solutions  Receive feedback  Tutors / Lab assistants:  Register students  Monitor course state Less rights and responsibilities More rights and responsibilities StudentsTutors / Lab assistants TeachersSystem administrator Course developers

9. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk Users’ Roles in CourseMaster  Teachers:  Register tutors  Set / edit exercises  Monitor class’ progress  Course Developers:  Create courses / units  Author exercises  System administrators:  Monitor system logs  Install new courses  Give exercise extensions

10. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk  Automatic Marking in CM:  Based on criteria and metrics  Programming languages (C, C++, Java)  Diagrams (o-o design, circuit design, flowcharts)  In theory: marks any type of document (needs suitable metrics) CourseMaster Marking

11. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk Demonstration

12. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk Experience CM at Nottingham:  Has been running for 4 years  It has been used by 293 students in 2001/2002  Student feedback is positive – with minor exceptions  Teachers are very happy that they no longer need to mark hundreds of programs every weekend  Course administration and monitoring are very effective

13. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk CourseMaster at other sites  CM is running at: – The National University of Singapore – Helsinki University of Technology (Finland) – Kings College London (UK) – 15 other institutions around the globe  Used by up to 1500 students

14. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk Results : Benefits  Practical:  Saves time, most importantly with larger numbers of students (assessment, feedback, …)  Saves money (cost effective)  Can be administered by teaching assistants, thus freeing lecturers for teaching  Good use of course resources (materials, exercises, …)

15. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk Results : Benefits  Pedagogical:  Consistent assessment  Equitable assessment  Incremental assessment (they learn more)  Instant feedback to students (keeps them informed)  Detection and prevention of plagiarism

16. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk Future Work  Automatic e-mailing of student progress to tutors  Assessment of object-oriented coursework (refactoring, o-o heuristics)  Assessment of Java programs with GUIs  A uPortal channel for CM  Composite Exercises

17. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk Summary  CourseMaster helps with the presentation, management and assessment of student work  CM is configurable for different educational requirements  Students are happy and academic staff even happier

18. pxs@cs.nott.ac.uk© 2002 LTR, ltr@cs.nott.ac.uk Web Addresses  CourseMaster’s main page http://www.cs.nott.ac.uk/~cmp/CourseMaster/index.htm  CourseMaster’s on-line web client http://zeus.cs.nott.ac.uk/login.html  CM’s Diagrammatical assessment information http://www.cs.nott.ac.uk/~azt/daidalos/