Testovid - an environment for testing almost any aspect of student assignments I. Pribela, S. Tošić, M. Ivanović, Z. Budimac Risan, September 2007

Introduction Continuous increase of enrolled students How to avoid straining of teachers:  decrease number of students  increase number of teachers  transfer some work load from teachers to computers Recent advances in computer technology made third option possible One of major conditions is ability to represent student work on the computer

Automated assessment Assessment phases  question / assignment selection  question / assessment deployment  answer / solution gathering  answer / solution grading  student / teacher notification  statistical analysis of results

Advantages Large student groups  faster assessment  moving work load from teachers  leaving more time for more productive aspects of teaching

Advantages Small student groups  self-assessment allows students to monitor their own progress to point to difficult parts of lectures early  teachers can easily note common problems and difficult topics  question selection can be better adjusted to individual needs  assessment results are immediate  same questions (assignments) can be repeated as much as needed  questions can contain graphics, sound, animations, and other multimedia content  increase of assessment objectivity

Disadvantages Type of assessments supported  short multiple choice tests  fill-in-the-blanks tests with fixed correct answers More appropriate for natural sciences than for social sciences

Assessment of programming Programming assignments are perfect example for automated assessment Besides generic, special systems for assessing of computer programs were also developed There are two distinct types of computer program assessment:  correctness assessment does the student program solve given problem?  optimality assessment how efficient the student program is?

Assessing correctness Oldest correctness assessment method is  running student programs against various sets of input data  comparing output form student program with correct one Input data  created by teacher and given to the student  created by the student  hidden from the student Selecting input data  fixed  chosen randomly every time Output data  compared with fixed data corresponding to given input  compared with output from correct solution

Assessing optimality Early systems focused on execution speed and precision of numeric results Recently attention has shifted towards readability and code style

Testovid Batch system Students can test solutions by themselves Teacher can test all tasks at once Generates execution reports Uses Apache Ant

Apache Ant Apache Ant is a tool for automatization on a build proceses Looks like Make, but written in Java Demands Java virtual machine Most appropriate for Java projects Uses XML for description of build processes Open source

Implementation Client-server architecture  Server  Clients (students, teacher) Student Instructor Server Computer lab Interactive testing Batch testing

Implementation Directory structure of a server a) Instructors test data b) Log files containing test results d) Student directories for storage of latest attempts c) Temporary directories

Implementation Two Ant build files  File for single testiranje  File for batch testing Two configuration files  File with testing modules  File with weights of every module

Testing module targets.all=test01,test02,test03,test04,test05,t est06, test07,test08 test01.name=Kompajliranje test01.score=1 test02.name=Up test02.score=1 test03.name=Down test03.score=1 test04.name=Up - Gornje ogranicenje test04.score=2 test05.name=Down - Gornje ogranicenje test05.score=2 test06.name=Sigurnost test06.score=1 test07.name=Greske u stilu test07.score=1 test08.name=Greske ili upozorenja u stilu test08.score=1 List of used modules Name of every module Number of points for each module All marks are binary

Testing module implementation One module is one Ant target Result of execution is string that contains message about error

Further Work Integration of system with Svetovid environment Creation of base of usefull modules for testing Adding system for plagiarism detection

The end Thank you for your attention