1. IITE Professional Development Course Lucknow University (6/4/2010) Professor Tim Keirn timkeirn@csulb.edu Module 6: Assessment

2. A Review: Standards-Based Approaches and Learning Outcomes Programme learning outcomes Course learning outcomes Program curricular map w/ sequenced papers for two certifications Physical Science: Teacher Ed, Chemistry, Mathematics and Physics Biology/Life Science: Teacher Ed, Chemistry, Botany and Zoology

3. Review Continued Learning outcomes for each paper Design an example of a lesson within a paper that is: Inquiry-based Aligned to a paper specific learning outcome Engages students with materials from the web

4. Review Continued Design an assessment that is aligned to the inquiry-based lesson and the specified paper learning outcome Design a rubric for the aforementioned specific assessment Publish materials to the portal on the web

5. General Introduction to Assessment Do students learn what faculty believe they are teaching? How do you know? On what evidence do you substantiate your claims? As an employer of a candidate with an upper second B.Sc from Lucknow in e.g. Botany -- what do I know ‘they know’ and what do I know ‘they can do’? What more do they know and what more can they do than someone with a ‘lower second’ and compared to some one with a ‘first’?

6. Introduction to Assessment Can I assume that someone who did the same paper with Vivek ‘knows and can do’ the same as a student of Nalini? If so -- how can you substantiate these claims? Think-Pair-Share Strategy: Identify and discuss the origins of three weaknesses in the current means by which students are assessed at Lucknow University This may not be an exhaustive list!

7. 3 Weakness of Current Assessment DescriptionImpact of Weakness Exams testing factual knowledge and asked to reproduce knowledge The exams are the same each year; responding without a deeper understanding of the concepts; not training/ developing skills Evaluation of the exam is effected by the readers mood, quality of other papers Unreliable evaluation Evaluation is not continuous and comprehensive, reliable and valid Students and employers don’t have a reliable confidence in what a student could actually do

8. Traditional Assessment Traditional assessment is inseparable from traditional modes of teaching and learning PH.D. provides discretion as to what is taught Stand and deliver Design assessment to measure knowledge retention Assign marks based on the ‘volume’ of knowledge retained PH.D provides discretionary authority to assess the ‘volume’ itself

9. Problems with Traditional Assessment Serve to discriminate between students as opposed to demonstrating competencies Almost always measures the reproduction of factual knowledge Little if any variance in both the method of assessment and the modality of learning Assessment is never deployed as a learning tool The secret handshake Blame the learner, not the teacher

10. Problems with Traditional Assessment (Cont) Assessment is infrequent and heavily weighted (high stakes) Summative over formative assessment Limited measurement of teaching efficacy: Did the instructor get the content ‘across’? Did the students read and ‘remember’ the book?

11. Alternative Forms of Assessment Standards-, disciplinary- and inquiry-based approaches to teaching and learning require a different approach to assessment Seek to measure: Thinking and skill > factual retention Production and application of knowledge > reproduction of knowledge What is learned (aligned to SLO) > What is taught

12. Alternative Assessment (Cont) Standards-based assessments: Are designed to measure task competence and degrees of proficiency > ranking and discriminating between students Are done in multiple forms to measure multiple modalities of learning Are learning tools in support of instruction and are transparent to students Are on-going and used to support reflection and improvement in teaching practice

13. Alternative Assessment Practicum In disciplinary groups -- design a draft of both a formative and summative assessment aligned to specific student outcome from a paper in the programme Specify the SLO Discuss what dimensions of a task are specifically measured in your standards-based assessments

14. SLO Demonstrate

15. Different Forms of Assessment and Methodologies Formative Assessments Aligned to learning outcome and to summative assessment Should provide appropriate feedback to student in preparation for the summative assessment Provide appropriate feedback to instructor about the efficacy of the pedagogic methodology Monitoring for comprehension in lecture Think-pair-share Short prompts

16. Other types of formative assessment Multiple-choice quizzes Short exercises and prompts Meeting the challenge of marking Be specific about nature of feedback and limits of time Peer evaluation Rubrics

17. Multiple Choice Questions Design questions that assess thinking and skill > factual content Bloom’s taxonomy Develop ‘justified’ multiple choice questions that demonstrate thinking and process Develop distracters that demonstrate & identify student (mis)understandings Questions that task students to substantiate or challenge claims

18. Bloom’s Taxonomy Bloom’s pyramid and active verbs Recall (list) Application (show) Analysis (compare) Synthesis (predict) Evaluation (dispute and/or substantiate)

19. Authentic Assessment Performance assessments tied to authentic disciplinary- tasks -- students produce knowledge as opposed to reproducing knowledge Laboratory practicum Research projects Assessment constructed as a problem Evaluating the validity of different interpretations and conclusions and their evidentiary basis Counterfactual questions and prompts

20. Rubrics - A Scoring Guide that Provides Criteria to Describe Levels of Student Performance The advantages of using rubrics: Instructors marks more accurately, reliably and quickly Requires greater accuracy about the criteria of student performance Serves as a learning tool and provides better feedback to students and makes the standard of performance explicit Creates better reliability across sections

21. Challenges to Using Rubrics Initially time-consuming (but in long-run saves time) Difficulty to find exact language that distinguishes between levels of performance and establishes criteria May require revision in initial implementation

22. Rubric Practicum Identify the dimensions of competence in the task that can be both delineated and demonstrated in the student performance (aligned with SLO) Holistic versus analytic (and the advantages of the latter within limits) Weight and scale the dimensions within the task

23. Rubric Practicum Cont. Establish criteria for competent performance of each specified dimension of the task Establish a scale of criteria performance How many clearly identifiable scales? E.g., Competent and Not Competent Not Proficient, Proficient, Excellent Not Proficient, Developing, Proficient, Beyond Proficient, Exemplary # of scales needs to be justified by clearly delineated performances of each dimension of the task

24. Rubric Practicum Continued The ideal process Create draft of rubric Implement and refine with evaluation of samples of student work Calibrate with other faculty Mark!

25. Rubric Exercise In disciplinary groups -- create a draft rubric for a laboratory practicum with three scales of performance for each dimension Teacher education faculty -- to do the same but for a pre-service teacher’s design of a laboratory practicum

26. SLO: Laboratory Practicum DimensionsCriteria Not ProficientProficient/Baseline skills Exemplary Lab preparationDESCRIPTION DESCRIPT Execution of methodology