1. An Introduction to Computer- assisted Assessment Joanna Bull and Ian Hesketh CAA Centre Teaching and Learning Directorate

2. Overview What is CAA? Why use it? Objective tests Integration: assessment methods & strategies Hands-on session

3. Computer-assisted Assessment I Computer-assisted assessment (CAA) is a common term for the use of computers in the assessment of student learning.

4. Computer-assisted assessment II Deliver, mark and analyse assignments or examinations (computer or web-based) Collate and analyse data gathered from optical mark readers (OMR) Record, analyse and report on achievement Collate, analyse and transfer assessment information through networks

5. Assessment Formative primarily to provide feedback to students and promote student learning Summative contributes to final grade for course/module Diagnostic to determine ability and understanding Self-assessment to allow students to evaluate their learning

6. Why use CAA ? What are the benefits to using CAA?

7. Why use CAA ? Increasing the frequency of assessment Increasing feedback Widening the range of assessment methods Provide objective and consistent marking Decrease marking Administrative efficiency

8. Increasing the frequency of assessment Motivating students to learn Encouraging students to practice skills Broadening the knowledge assessed

9. Increasing feedback Students to motivate students needs to be timely, accurate and constructive Staff about student performance about test and question performance

10. Widen range of assessment methods Different assessment methods for different skills, abilities and knowledge Balance of assessment methods Utilisation of multimedia to create innovative assessments

11. Provide objective and consistent assessment Assessment can be unreliable, subjective and inconsistent Automated marking enables consistent, objective assessment for large groups Rigorous evaluation of questions ensures overall consistency and standards.

12. Decrease marking Potential time savings: dependent on level of implementation must be offset against investment in design may represent a re-distribution of workload may demand a cultural shift

13. Administrative efficiency Automatic transfer of data to MIS, or SRS Examination papers and handling Speed of results

14. Objective Tests Examples of question types Guidelines for writing items Assessing skills and abilities Feedback and scoring

15. Objective Tests Require students to provide a response to a question with a pre-determined answer: select a solution from a set of choices identify an object or position supply brief numeric or text responses

24. Question Types Multiple choice Multiple response Assertion-reason Graphical hotspot Text/numerical Matching/ranking Sore finger questions Sequencing Assembly Graphing

25. Designing questions I Questions single, definitive statement avoid unnecessary or irrelevant material use clear language use negatives sparingly put as much of the question in the stem do not use double negatives

26. Designing questions II Distractors ensure there is only one correct answer use plausible distractors avoid clues to the correct answer avoid all of the above and none of the above base distractors on common student errors avoid ambiguity

27. Blooms Taxonomy knowledge, comprehension, application, analysis, synthesis and evaluation Example questions

28. Feedback Objective tests allow provision of : timely, specific formative feedback to students fast and detailed feedback to staff Enables a consistent level of feedback to large student groups Can focus and direct learning through anticipated problem areas Enables curriculum design modification during a course

29. Scoring One score per correct answer Issue of guessing Negative marking

30. Discrimination and facility Discrimination power of an individual question to differentiate between weaker and better students Facility measures the difficulty of a question by dividing the number of correct responses by the total number of responses.

31. Integration: assessment methods & strategies Appropriateness and balance Quality of feedback and student learning Extending the skills and knowledge assessed

32. Test structure II Analyse material content difficulty assessment type question style Identify areas of low coverage and uneven spread

33. CAA Website CAA Centre http://caacentre.ac.uk email: info@caacentre.ac.uk

34. CAA in business studies Introduction to financial accounting Intermediate/advanced financial accounting Principles of finance Production and operations management Professional certificate in management

35. Advantages of CAA- business studies respondents Speed Remote access Anonymity Neutrality Quick feedback Wide course coverage

36. Disadvantages of CAA-business studies respondents Inflation of students' marks Difficult to set appropriate questions Steep learning curve/time investment Fails to develop students' argumentation skills

37. CAA Website CAA Centre http://caacentre.ac.uk email: joanna.bull@luton.ac.uk