Assessment and Feedback using IT Types of assessment Formative Summative Norm-referenced Criterion-referenced

Formative vs. Summative Formative Advice or feedback to students; feedback to designers of materials No contribution to final student marks Summative Graded feedback Counts towards final mark

Norm-referenced vs. Criterion-referenced Norm Use the achievements of a group to set the standards for specific grades In use in most universities! Criterion Establish the standard (criterion) and mark students against it Recent moves towards this at Sydney

Examples of Assessment Formative Continuous Peer Self Summative End of course exams Continuous (Peer)

Use of computers and the Web in assessment Delivered and marked questions for both formative and summative assessment WebMCQ Revision materials Sydney Biology SAMs Tutorials with quiz sections Assignment submission

Course Management Tools WebCT BlackBoard

Pedagogical issues in designing student assessment Different levels of learning Different levels of teaching materials Objective testing Advantages and disadvantages of objective testing Standard tests and concept inventories

Different levels of learning Assessment usually drives learning — not necessarily a bad thing! The assessment tools should reinforce the level of learning you want Shallow learning Memorize by rote Focus on discrete elements Treat tasks as impositions Deep learning Interact with content Relate new ideas to previous Relate evidence to conclusions

Classification of teaching materials Bloom’s Taxonomy of understanding Knowledge Comprehension Application Analysis Synthesis Evaluation. Findlay’s Taxonomy for mathematics Knowledge Representation Manipulation Algorithmic procedure Attack and structuring.

Objective testing Started out in intelligence testing — required validity, reliability, prediction First educational use, entrance testing for universities — sifting, ranking, selecting Nowadays “objective” testing of learning by: Two-state (true/false; yes/no) Multiple choice (key plus distracters) Supply (simple completion) Clustered sets or any or all of the above

Objective testing (2) Advantages Fair Broad coverage Easily automated Easily administered Easily evaluated Disadvantages Take time to create Need much evaluation Superficial? Lead to fragmentation of knowledge? Encourage students to work backwards?

Standardized tests and concept inventories Many teaching communities now share item banks of questions Useful in comparing students across universities and countries Concept inventories used for research, show that objective testing can test understanding

Some Examples of Assessment using IT Drill and Practice Embedded self assessment Graded multiple choice Assessment in course managers

Drill and Practice Balancing Chemical Equations An example of where repetitive, drill and practice exercises are useful in learning necessary skills. Published by Launchpad Multimedia Pty Ltd, P.O. Box 106 Geebung, Queensland.

Embedded Self Assessment Self Assessment Modules (SAMs), Biology, University of Sydney. Exercises created with four levels of difficulty so that each is testing a different ability with respect to the content. ResourceCentre/CAL.html

Graded Multiple Choice Practice question sets to allow students of Psychology (University of Sydney) to monitor their level of understanding, using the software system WebMCQ. See:Dalziel, J., R. and Gazzard, S., “WebMCQ: A web-based multiple choice assessment system”, ” 1998, Proc. University Science Teaching and the Web Workshop.

Assessment in Course Management Software WebCT A course management system which incorporates several different kinds of assessment capabilities. Available at: