Results from using eTesting methods in CS education Workshop on Teaching Methods in Computer Science Courses Results from using eTesting methods in CS education Goce Armenski, Marjan Gusev {armenski, marjan}@ii.edu.mk Institute of Informatics – PMF University “St. Cyril and Methodius” - Skopje

Results from using eTesting methods in CS education 1. Results from using eTesting methods in CS education 1. OVERVIEW Changes in Higher Education (educating more students) - problems in giving lectures, delivering the material, assessing student knowledge Solutions to the problems - distance learning (for the lectures) - web publishing (for the materials) - computer based assessment (for the assessment) Results from other studies: - increased effectiveness (automatic feedback) - more time for teachers - lower the time for taking tests - have more time to concentrate on real problems

Results from using eTesting methods in CS education 2. Results from using eTesting methods in CS education 2. GOAL Focus of the research: To implement a system for computer based assessment and evaluate the results from its use Global goal: Identify the influence of this system to the teachers and students Main goal: Is computer based assessment more effective and objective than the traditional assessment Oxford, Cambridge and RSA Examinations: The effectiveness of the system can be analyzed through few criteria - cost reduction - provide better quality of the assessment process

3. Practical implementation and data gathering Results from using eTesting methods in CS education 3. Practical implementation and data gathering Implementation The use of technology in education is very dependent of the organization in which it is implemented - the logistic which will be offered - the cultural changes which will arouse Synchronization with other systems Technical infrastructure Data gathering System was implemented in 2001 at the University - 12 courses - 327 examinations - 5291 students were involved

4. Effectiveness of the eTesting system Results from using eTesting methods in CS education 4. Effectiveness of the eTesting system Costs reduction Paper reduction Printing reduction Time reduction Technical infrastructure Software development and maintenance Administration training Technical staff Teachers perspective Development of the question bank was time consuming - more than 1300 questions per course Time reduction after development of the course questions

4. Effectiveness of eTesting system 5. Results from using eTesting methods in CS education 4. Effectiveness of eTesting system Students perspective Survey with 10 questions - Likert type of questions with 5 values answers (1-5) - 236 students toke the survey Question 1

Results from using eTesting methods in CS education 6. Results from using eTesting methods in CS education Question 2

Results from using eTesting methods in CS education 7. Results from using eTesting methods in CS education Question 3

Results from using eTesting methods in CS education 8. Results from using eTesting methods in CS education Question 4

Results from using eTesting methods in CS education 9. Results from using eTesting methods in CS education Question 5

Results from using eTesting methods in CS education 10. Results from using eTesting methods in CS education Question 6

Results from using eTesting methods in CS education 11. Results from using eTesting methods in CS education Question 7

Results from using eTesting methods in CS education 12. Results from using eTesting methods in CS education Question 8

Results from using eTesting methods in CS education 13. Results from using eTesting methods in CS education Question 9

Results from using eTesting methods in CS education 14. Results from using eTesting methods in CS education Question 10

Results from using eTesting methods in CS education 15. Results from using eTesting methods in CS education Summary of the results Прашање N Ср. Вр. Std The use of the system for electronic testing is: 235 3,7149 0,8865 The electronic testing is __________ compared to the traditional one 233 3,7296 0,7368 I prefere assessment using the system for electronic testing, than the traditional one 236 4,3686 0,9154 Marking on the system for electronic testing is objective (same for all) 4,3305 0,8748 The use of the module for online learning with often knowledge assessment is helping with the material 4,3814 0,7428 Using the module for online learning with often knowledge assessment helps me achive more knowledge 4,0128 0,8986 I recommend use of the system for electronic testing for assessment in other courses 4,4378 0,8288