1. Introduction to Assessment of Student Learning
Kathleen M. Morley, Ph.D.
University Director of Assessment
11/20/2018

2. Overview I. Assessing Learning II. Goals & Objectives III. Measures
IV. Rubrics
11/20/2018

3. I. Assessing Learning
11/20/2018 3

4. Why Assess Student Learning?
“Natural human, scholarly act” – Barbara Walvoord, 2009 Identify student strengths and weaknesses National movement that is not going away
11/20/2018

5. What is Assessment?
“Effective assessment is best understood as a strategy for understanding, confirming, and improving student learning”
– Higher Learning Commission
11/20/2018 5

6. ASSESSMENT FRAMEWORK Long Island University Mission Campus Mission
School/College Mission
Academic Program Goals
Program Effectiveness (e.g., admissions, curriculum, satisfaction)
Student Learning
(e.g., student knowledge, skills, and attitudes)
11/20/2018 6

7. Student Learning Assessment Cycle
Clear Learning Goals
Action for Improvement
Clear Learning Objectives (i.e., how goal is demonstrated)
Analyze Findings
Identify Measures
Collect Data
11/20/2018 7

8. Assessment of Student Learning
How do we know students are achieving program learning outcomes?
Establish program learning goals
Apply curricular mapping
Implement a program-level assessment plan
11/20/2018 8

9. II. Goals & Objectives
11/20/2018 9

10. Definitions
Goal:
  “General statement about knowledge, skills, attitudes, and values expected in graduates” *
Objective:
“A clear concise statement that describes how students can demonstrate their mastery of a program goal” *
* Allen, Mary J., Assessing Academic Programs in Higher Education. Anker Publishing Company, 2004.
11/20/2018 10

11. Examples
Goal: Students will utilize quantitative methods in social scientific research. Objective: Students will independently design and carry out correlational research that yields valid results.
11/20/2018 11

12. Bloom’s Taxonomy Bloom’s Taxonomy:
classification for levels of learning
moves from the least to most complex
has list of verbs associate with each level:
11/20/2018 12

13. Bloom’s Taxonomy Complexity
11/20/2018

14. Writing Objectives: The 3 Ms
How does the objective support the department goal or mission?
Meaningful
What is needed to foster the achievement of the objective?
Is the objective realistic?
Manageable
How will you know if the objective has been achieved?
What will be the assessment method?
Measurable
11/20/2018
StudentVoice.Com 14

15. Characteristics of a Good Objective
Explains how students can demonstrate the mastery of a learning goal
Uses an active verb
Specifies definite observable behaviors
Is clearly linked to a goal
11/20/2018 15

16. III. Measures
11/20/2018 16

17. Why Grades Are Insufficient
Do not tell you about student strengths and weaknesses in learning content
Cannot use average grade to tell you where to make changes in the curriculum
11/20/2018 17

18. Definitions Direct Measure:
A measure that directly demonstrates that students achieved a learning objective  
Indirect Measure:
A measure that offers students’ opinion about meeting the learning objective
11/20/2018 18

19. Examples Direct Measure: Set of test items
Written assignment scored with a rubric
Direct observation scored with a rubric
Indirect Measure:
Student reflections/self-assessments
Student opinion surveys
Focus groups
11/20/2018 19

20. Choosing Measures Begin with what you are already assessing
Use what will tell you about student strengths and weaknesses
Look for culminating experiences
11/20/2018

21. Example Assessment AY09-10 11/20/2018 21
Phase I – due November 15, 2009
Phase II –due May 31, 2010
Program Learning Goal:
Students will…
Student Learning Objectives
Direct and Indirect Measures
Findings (Learning Outcomes)
Interpretation
Actions Planned
Goal 1 (continued):
Read proficiently
Learning Objective 2:
Analyze college-level readings proficiently
Measure 1:
Scores for the interpretive questions on the Comprehension portion of the Nelson-Denny Reading Test
On average, the 210 ENG XX students tested correctly answered XX% of the interpretive comprehension questions they attempted within the time limit, or XX% of all the interpretive comprehension questions. There was a considerable spread of results from low to high scores.
In contrast with their fine performance comprehending the literal meaning of texts (see above measure), ENG XX students on average tend to be quite challenged in interpreting meanings that are not literally spelled out.
Measures 1 & 2:
The (program) will provide (faculty) with pedagogical resources on inference building, such as a handout for students on reading skills (e.g., connotation, denotation, etc.).
Senior (faculty) will be leading a series of seminars in to educate faculty both within (the Department) and across campus on reading pedagogy.
The Department also plans to expand its assessment of students’ reading proficiency by administering the Nelson-Denny Reading Test to students at other stages in the (program).
Measure 2:
A rubric assessing students’ written summaries of readings
On a 5-point scale, the Exit Exam essays of 481 ENG XX students most frequently received ratings of 3 (XX%) or 4 (XX%), with an “average” rating of X.XX.
The rubric defines a rating of 4 as signifying a “good, fairly concise synthesis of most of the key points,” and a rating of 3 as signifying a “fair, less focused synthesis that may misidentify key points.” ENG XX students’ grasp of key points in readings could be better developed.
11/20/2018 21

22. IV. Rubrics
11/20/2018 22

23. Introduction to Rubrics by Stevens and Levi, 2005
11/20/2018 23

24. Definition
Rubric:
“At its most basic, a rubric is a scoring tool that lays out the specific expectations for an assignment.”
- Stevens and Levi, 2005
11/20/2018 24

25. Descriptive Scoring Guides
Student Learning
Descriptive Scoring Guides
Exhibit 9.6. Descriptive Rubric for a Slide Presentation on Findings from Research Sources*
Organization
Persuasiveness
Introduction
Etc.
Well Done (5)
Clearly, concisely written. Logical, intuitive progression of ideas and supporting information. Clear and direct cues to all information.
Motivating questions and advance organizers convey main idea. Information is accurate.
Presents overall topic. Draws in audience with compelling questions or relating to audience’s interests or goals.
Satisfactory (4-3)
Logical progression of ideas and supporting information. Most cues to information are clear and direct.
Includes persuasive information
Clear, coherent, and related to topic.
Needs Improvement (2-1)
Vague in conveying viewpoint and purpose. Some logical progression of ideas and supporting information, but cues are confusing or flawed.
Includes persuasive information with few facts.
Some structure but does not create a sense of what follows. May be overly detailed or incomplete. Somewhat appealing.
Incomplete (0)
Lacks a clear point of view and logical sequence of information. Cues to information are not evident.
Information is incomplete, out of date, or incorrect.
Does not orient the audience to what will follow.
*Adapted from Suskie, L. (2008). Assessing Student Learning: A Common
Sense Guide, 2nd Edition. Jossey-Bass, San Francisco, CA, p. 144.
11/20/2018

26. Do You Need A Rubric? Carpal tunnel from writing the same comments?
Weeks behind in your grading?
Worry last papers may be graded differently than first?
- Stevens and Levi, 2005
11/20/2018 26

27. Benefits of a Rubric Timely feedback
Increased likelihood of feedback being read
Encourages critical thinking
Helps to refine teaching skills
- Stevens and Levi, 2005
11/20/2018 27

28. Rubric Components 1. Assignment description 2. Scale 3. Dimensions
4. Level of achievement
- Stevens and Levi, 2005
11/20/2018 28

29. Rubric Components 3 Level Rubric Task Description: Dimensions
Exemplary 3
Competent 2
Developing 1
11/20/2018 29

30. Description of highest level of performance
Rubric Components
Scoring Guide Rubric
Task Description:
Dimensions
Description of highest level of performance
Comments
Points
11/20/2018 30

31. Rubric Components Scoring Guide Rubric ▫ 2 1 Task Description:
Dimensions
Description
Comments
Points ▫ 2 1
11/20/2018 31

32. Rubric Components Scoring Guide Rubric with Boxes and Circles
Task Description:
Dimensions
Description
Comments
Points
▫ text, text, text
text, text, text 2
▫ text, text, text 1
11/20/2018 32

33. Look for Models (
11/20/2018 33

34. Steps to Creating a Rubric
1. Reflecting on the assignment
2. Listing learning objectives
3. Grouping and labeling
4. Applying to rubric grid
- Stevens and Levi, 2005
11/20/2018 34

35. Student Learning
Resources on Rubrics*
How to create a rubric:
Rubric for a given task:
Designing scoring rubrics:
Scoring rubrics:
Recommendations for scoring rubrics:
Scoring rubric development: validity and reliability:
Rubrics for web lessons:
Creating rubrics:
Performance criteria:
*Suskie, L. (2008). Assessing Student Learning: A Common Sense Guide, 2nd Edition.
Jossey-Bass, San Francisco, CA, p. 154.

36. Resources Kathleen M. Morley University Director of Assessment
x4258
Assessment Website:
11/20/2018 36