New Faculty Orientation August 2019 Assessment: Challenges and Opportunities Title page for the presentation Thank you for invitation Thanks to Dr. Kesarwani fro supporting an event The need for faculty across all factulties to have an opportunity .. To address challenging issues

Assessment from the Latin assidere which means “to sit beside” “To sit beside” brings to mind such verbs as to engage, to involve, to interact, to share, to trust. It conjures up team learning, working together, discussing, reflecting, helping, building, collaborating. It makes one think of cooperative learning, community, communication, coaching, caring, consultation… “Sitting beside” implies dialogue and discourse, with one person trying to understand the other’s perspective before giving value judgments. (Braskamp and Ory, 1994) This not current However very much embraces the sprit of new authentic or alternative

Second Optime (pass men) Inferiores (charity passes) Historical Baggage: Optime (honor men) Second Optime (pass men) Inferiores (charity passes) Pejores (unmentionables) Yale 1783 Prior to this read from the book

Pillars of Sound Assessment Transparency Standardization

Learning Outcomes State the relationship between Learning Outcomes and Assessment Activities Identify methods for incorporating Bloom’s Cognitive Taxonomy into assessment methods Identify strategies for constructing effective MCQ test items including methods for writing items at higher cognitive levels Apply the results of Item Analysis to ensure the effectiveness of test items and to help identify items which test at higher cognitive levels Identify strategies to help test takers perform at optimal levels to ensure valid test scores

Bloom’s Taxonomy Knowledge: Recalling facts Comprehension: Demonstrating understanding Application: Transferring previous learning to solve new problems Analysis: Understanding the organizational structure Synthesis: Applying knowledge or skills to create something new Evaluation: Applying judgment

Types of Assessments Formative assessment The goal of formative assessment is to monitor student learning Summative assessment The goal of summative assessment is to evaluate student learning at the end of an instructional unit Criterion what does the student know Norm what does he know related to others What sustains bell curving Meyer 1908 50 med 22 above superior 3 excellent 22 inferior 3 failure Mensa and Densa The appeal of nature’s distribution Grade inflation Fear of success

Assessment Drives BOTH Teaching and Learning Teaching / Learning BUILDING THE BRIDGE Assessment Methods Impacts on student perception of the role of assessment Makes assessment a method for grades Experienced as divorced from learning

RELIABILITY What Reliability represents How we establish Reliability Criteria for scoring Acknowledging subjectivity Sufficient opportunities The concept of reliability addresses the question to what extent does a particular score a candidate obtains in an assessment context actually reflect the candidate’s “true score”. The true score refers to the score a candidate would get in a perfect world where such factors such as the technical characteristics of the test, the abilities of the assessors, the testing conditions and the motivation of the candidate were perfect in nature. Reliability co-efficient ranged from 0.0 to 1.00. The higher the co-efficient the more reliable the test scores are and consequently the more valid the pass-fail decisions. Coefficients in the range of .70 to .80 are highly desirable for licensure decisions. There are a number of reliability co-efficients one could use but the most common are internal consistency estimates of reliability. Such coefficients provide evidence of how well the assessment instrument is measuring a single underlying construct such as case history skills or physical examination skills. These coefficients are generated through the application of statistical techniques and are commonly part of the output of any item analysis procedure. In performance-based examinations (like ours) the issue of reliability is most concerned with the reliability of the judgements and therefore scores made by the assessors. In order to enhance reliability co-efficient and therefore the confidence we have in our past /fail decisions there are a number of strategies we employ. First, we have well developed checklists that are clear and unambiguous. Second, we have highly competent practitioners as examiners who receive extensive training for each administration. Finally, the reliability of our scores is enhanced by our practice of reviewing the videos of candidate's who have fallen below the cut-scores.

VALIDITY What Validity represents How we establish Validity Meaningful activities Representativeness Appropriate format

USABILITY Provision for feedback Appropriate weightings Reasonable frequency Provision for feedback Appropriate weightings You can probably elaborate on these areas quite well

Types of Scoring Methods Criterion-referenced Provides absolute comparisons Describes student performance in relation to pre-set standard Norm-referenced Provides relative comparisons Describes individual performance in comparison to others Criterion what does the student know Norm what does he know related to others What sustains bell curving Meyer 1908 50 med 22 above superior 3 excellent 22 inferior 3 failure Mensa and Densa The appeal of nature’s distribution Grade inflation Fear of success

GRADING Grading that reflects the actual ability of the student Grading as a personal communication Show video sequences

Feedback on your Assessment Selective response Item Analysis Constructive response Descriptive stats Peer/Student feedback

Item Analysis Example Prop. Answering Correctly: 0.74 Discrimination Index: 0.42 Point Biserial: 0.40 Alt. Total Ratio Low (27%) High (27%) Point Biserial Key A 0.74 0.44 0.86 0.40 * B 0.09 0.17 0.03 -0.13 C 0.04 0.06 0.08 -0.03 D 0.13 0.33 -0.38 E 0.00 Other We need to reflect

COLLABORATIVE LEARNING SCORING RUBRIC 4- Thorough Understanding 3- Good Understanding 2- Satisfactory Understanding 1- Needs Improvement

COLLABORATIVE LEARNING SCORING RUBRIC 4- Thorough Understanding Consistently works toward group goals Is sensitive to needs of all the group Willingly fulfills individual role within the group Consistently contributes knowledge, opinions, skills Values the knowledge of all group members Helps group identify necessary group changes

COLLABORATIVE LEARNING SCORING RUBRIC 1- Needs Improvement Works only when prompted Contributes only when prompted Needs reminders to be sensitive to others Participates in needed changes when prompted

Summary Suggestions Integrate assessment into your teaching Seek out feedback regarding your assessment techniques Attempt to introduce one new assessment technique