6/12/ Introduction to Assessment of Student Learning Kathleen M. Morley, Ph.D. University Director of Assessment

6/12/ Overview I. Assessing Learning II. Goals & Objectives III. Measures IV. Rubrics

6/12/ I. Assessing Learning

6/12/ Why Assess Student Learning? “Natural human, scholarly act” – Barbara Walvoord, 2009 Identify student strengths and weaknesses National movement that is not going away

6/12/ What is Assessment? “Effective assessment is best understood as a strategy for understanding, confirming, and improving student learning” – Higher Learning Commission

6/12/ Long Island University Mission Campus Mission Academic Program Goals Program Effectiveness (e.g., admissions, curriculum, satisfaction) Student Learning (e.g., student knowledge, skills, and attitudes) ASSESSMENT FRAMEWORK School/College Mission

6/12/ Student Learning Assessment Cycle Clear Learning Goals Clear Learning Objectives (i.e., how goal is demonstrated) Identify Measures Collect Data Analyze Findings Action for Improvement

6/12/ Assessment of Student Learning How do we know students are achieving program learning outcomes? Establish program learning goals Apply curricular mapping Implement a program-level assessment plan

6/12/ II. Goals & Objectives

6/12/ Definitions Goal: “General statement about knowledge, skills, attitudes, and values expected in graduates” * Objective: “A clear concise statement that describes how students can demonstrate their mastery of a program goal” * * Allen, Mary J., Assessing Academic Programs in Higher Education. Anker Publishing Company, 2004.

6/12/ Examples Goal: Students will utilize quantitative methods in social scientific research. Objective: Students will independently design and carry out correlational research that yields valid results.

6/12/ Bloom’s Taxonomy Bloom’s Taxonomy: classification for levels of learning moves from the least to most complex has list of verbs associate with each level: my.htm

6/12/ Bloom’s Taxonomy Complexity

6/12/ StudentVoice.Com Writing Objectives: The 3 Ms How does the objective support the department goal or mission? Meaningful What is needed to foster the achievement of the objective? Is the objective realistic? Manageable How will you know if the objective has been achieved? What will be the assessment method? Measurable

6/12/ Characteristics of a Good Objective Explains how students can demonstrate the mastery of a learning goal Uses an active verb Specifies definite observable behaviors Is clearly linked to a goal

6/12/ III. Measures

6/12/ Why Grades Are Insufficient Do not tell you about student strengths and weaknesses in learning content Do not tell you about student strengths and weaknesses in learning content Cannot use average grade to tell you where to make changes in the curriculum Cannot use average grade to tell you where to make changes in the curriculum

6/12/ Definitions Direct Measure: A measure that directly demonstrates that students achieved a learning objective A measure that directly demonstrates that students achieved a learning objective Indirect Measure: A measure that offers students’ opinion about meeting the learning objective

6/12/ Examples Direct Measure: Set of test items Set of test items Written assignment scored with a rubric Written assignment scored with a rubric Direct observation scored with a rubric Direct observation scored with a rubric Indirect Measure: Student reflections/self-assessments Student reflections/self-assessments Student opinion surveys Student opinion surveys Focus groups Focus groups

6/12/ Choosing Measures Begin with what you are already assessing Begin with what you are already assessing Use what will tell you about student strengths and weaknesses Use what will tell you about student strengths and weaknesses Look for culminating experiences Look for culminating experiences

6/12/ Example Assessment AY09-10 Phase I – due November 15, 2009Phase II – due May 31, 2010 Program Learning Goal: Students will … Student Learning Objective s Students will … Direct and Indirect Measures Findings (Learning Outcomes) InterpretationActions Planned Goal 1 (continued ): Read proficientl y Learning Objective 2: Analyze college- level readings proficientl y Measure 1: Scores for the interpretive questions on the Comprehensio n portion of the Nelson- Denny Reading Test Measure 1: On average, the 210 ENG XX students tested correctly answered XX% of the interpretive comprehension questions they attempted within the time limit, or XX% of all the interpretive comprehension questions. There was a considerable spread of results from low to high scores. Measure 1: In contrast with their fine performance comprehending the literal meaning of texts (see above measure), ENG XX students on average tend to be quite challenged in interpreting meanings that are not literally spelled out. Measures 1 & 2: The (program) will provide (faculty) with pedagogical resources on inference building, such as a handout for students on reading skills (e.g., connotation, denotation, etc.). Senior (faculty) will be leading a series of seminars in to educate faculty both within (the Department) and across campus on reading pedagogy. The Department also plans to expand its assessment of students ’ reading proficiency by administering the Nelson- Denny Reading Test to students at other stages in the (program). Measure 2: A rubric assessing students ’ written summaries of readings Measure 2: On a 5-point scale, the Exit Exam essays of 481 ENG XX students most frequently received ratings of 3 (XX%) or 4 (XX%), with an “ average ” rating of X.XX. Measure 2: The rubric defines a rating of 4 as signifying a “ good, fairly concise synthesis of most of the key points, ” and a rating of 3 as signifying a “ fair, less focused synthesis that may misidentify key points. ” ENG XX students ’ grasp of key points in readings could be better developed.

6/12/ IV. Rubrics

6/12/ Introduction to Rubrics by Stevens and Levi, 2005

6/12/ Definition Rubric: “At its most basic, a rubric is a scoring tool that lays out the specific expectations for an assignment.” - Stevens and Levi, 2005

6/12/2016 Descriptive Scoring Guides Student Learning *Adapted from Suskie, L. (2008). Assessing Student Learning: A Common Sense Guide, 2nd Edition. Jossey-Bass, San Francisco, CA, p Well Done (5) Clearly, concisely written. Logical, intuitive progression of ideas and supporting information. Clear and direct cues to all information. Motivating questions and advance organizers convey main idea. Information is accurate. Presents overall topic. Draws in audience with compelling questions or relating to audience’s interests or goals. Satisfactory (4-3) Logical progression of ideas and supporting information. Most cues to information are clear and direct. Includes persuasive information Clear, coherent, and related to topic. Needs Improvement (2-1) Vague in conveying viewpoint and purpose. Some logical progression of ideas and supporting information, but cues are confusing or flawed. Includes persuasive information with few facts. Some structure but does not create a sense of what follows. May be overly detailed or incomplete. Somewhat appealing. Incomplete (0) Lacks a clear point of view and logical sequence of information. Cues to information are not evident. Information is incomplete, out of date, or incorrect. Does not orient the audience to what will follow. Organization Persuasiveness Introduction Etc. Exhibit 9.6. Descriptive Rubric for a Slide Presentation on Findings from Research Sources*

6/12/ Do You Need A Rubric? Carpal tunnel from writing the same comments? Weeks behind in your grading? Worry last papers may be graded differently than first? - Stevens and Levi, 2005

6/12/ Benefits of a Rubric Timely feedback Increased likelihood of feedback being read Encourages critical thinking Helps to refine teaching skills - Stevens and Levi, 2005

6/12/ Rubric Components 1. Assignment description 2. Scale 3. Dimensions 4. Level of achievement - Stevens and Levi, 2005

6/12/ Rubric Components 3 Level Rubric Task Description: DimensionsExemplary 3 Competent 2 Developing 1

6/12/ Rubric Components Scoring Guide Rubric Task Description: DimensionsDescription of highest level of performance CommentsPoints

6/12/ Rubric Components Scoring Guide Rubric Task Description: DimensionsDescriptionCommentsPoints ▫ 2 ▫ 1

6/12/ Rubric Components Scoring Guide Rubric with Boxes and Circles Task Description: DimensionsDescriptionCommentsPoints ▫ text, text, text text, text, text 2 ▫ text, text, text 1

6/12/ Look for Models ( ce.htm#gened_rubrics) ce.htm#gened_rubrics

6/12/ Steps to Creating a Rubric 1. Reflecting on the assignment 2. Listing learning objectives 3. Grouping and labeling 4. Applying to rubric grid - Stevens and Levi, 2005

Resources on Rubrics* How to create a rubric: Create_rubric.html Create_rubric.html Rubric for a given task: Designing scoring rubrics: Scoring rubrics: Recommendations for scoring rubrics: Scoring rubric development: validity and reliability: Rubrics for web lessons: management/rubrics/4521.html Creating rubrics: Performance criteria: Student Learning *Suskie, L. (2008). Assessing Student Learning: A Common Sense Guide, 2nd Edition. Jossey-Bass, San Francisco, CA, p. 154.

6/12/ Resources Kathleen M. Morley University Director of Assessment x4258 Assessment Website: Affairs/Outcomes-Assessment.aspx