1. Youngstown State University September 2011 Susan Hatfield Winona State University Winona, MN SHatfield@winona.edu Assessing your Assessment Plan

2. YSU Added note: the first four slides are introduction slides

6. What is the focus of your assessment efforts?

7. Accreditation Reporting Number & Amount Quality & Utility Interpreting Learning Internal QuestionsExternal Questions Collecting it Using it ComplianceCommitment

8. Promoting a Culture of Learning

9. BEGINNING PROGRESS MATURING

10. Workshops, Speakers and Consultants on student learning Ongoing dialogue on student learning, development and measurement of learning outcomes Student learning has become central to the institution and student learning, performance, and achievement are celebrated Promoting a Culture of Learning BEGINNING PROGRESS MATURING

11. Activities Events SystemsProcesses Traditions Promoting a Culture of Learning BEGINNING PROGRESS MATURING

12. OccasionalUsual ExternalInternal AccreditationImprovement SurfaceEmbedded IndividualStructural IsolatedPervasive Promoting a Culture of Learning BEGINNING PROGRESS MATURING

13. OccasionalUsual ExternalInternal AccreditationImprovement SurfaceEmbedded IndividualStructural IsolatedPervasive LEADERSHIP KNOWLEDGE RESOURCES COMMUNICATION COMMITMENT Promoting a Culture of Learning BEGINNING PROGRESS MATURING

14. What are you measuring?

15. The relationship between values and measures Measures Values

16. Do you have measurable, meaningful program level student learning outcomes?

17. Reviewing Student Learning Outcomes A: Student Learning or Program Effectiveness?

18. What the student / faculty / staff will √ Know √ Do Assessing Assessing Learning

19. What the course / program /department / area will do or achieve Participation Support Improvement Placement Retention Development Satisfaction. Assessing Program Effectiveness

20. Satisfaction surveys Exit interviews & focus groups Observations Follow-up studies Retention and transfer studies Length of time to degree Efficiency, accuracy of services performed Number of students served, repeat visits Self report of impact on those served Graduation and transfer rates Placement rates.

21. Reviewing Student Learning Outcomes A: Student Learning or Program Effectiveness? B: Reasonable Number?

22. Exertion without Intention

23. Exhaustion

24. Intention without Exertion

25. Intention and Exertion

26. Outcomes can overlap

27. Outcome Common Core

28. Outcome Program A

29. Outcome Program B

30. Outcome Program C

31. Outcome Program D

32. Reviewing Student Learning Outcomes A: Student Learning or Program Effectiveness? B: Reasonable Number? C: Tied to mission / goals?

33. Mission General Education Academic Programs Vision Student Affairs External Agencies

34. Reviewing Student Learning Outcomes A: Student Learning or Program Effectiveness? B: Reasonable Number? C: Tied to mission / goals? D: Appropriate format?

35. Format for learning outcomes: Students / Faculty / Staff will be able to > >

36. Students / Faculty / Staff will be able to,, and,,, and.

37. Reviewing Student Learning Outcomes A: Student Learning or Program Effectiveness? B: Reasonable Number? C: Tied to mission / goals? D: Appropriate format? E: Cognitively appropriate?

38. KNOWLEDGE COMPREHENSION APPLICATION ANALYSIS SYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate

39. KNOWLEDGE COMPREHENSION APPLICATION ANALYSIS SYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Su rize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Lower level course outcomes

40. KNOWLEDGE COMPREHENSION APPLICATION ANALYSIS SYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Advanced Course / Program outcomes

41. Student Learning Outcomes Students will have a basic knowledge of human anatomy.

42. Student Learning Outcomes Interpret a financial statement

43. Student Learning Outcomes Question inconsistent results

44. Student Learning Outcomes Demonstrate the ability to solve problems, to work independently and to work as members of a team.

45. Student Learning Outcomes Be recognized for excellent preparation and will be vigorously recruited by employers.

46. Student Learning Outcomes Students will be able to identify, define, and evaluate……

47. Student Learning Outcomes Students will be able to identify, define, and evaluate……

48. Student Learning Outcomes Students will be able to select, describe, and interpret…..

49. Reviewing Student Learning Outcomes A: Student Learning or Program Effectiveness? B: Reasonable Number? C: Tied to mission / goals? D: Appropriate format? E: Cognitively appropriate? F: Do faculty agree on the definition of the outcome? (components)

50. Components What are you looking for in student work to be able to tell if they “get it” Grading criteria

51. Goal

52. Outcome

53. Communication RelateSpeakListenParticipateWrite

54. Component Communication RelateSpeakListenParticipateWrite

55. Component Communication RelateSpeakListenParticipateWrite

56. Component Communication RelateSpeakListenParticipateWrite

57. Component Communication RelateSpeakListenParticipateWrite

58. Component Communication RelateSpeakListenParticipateWrite

59. The Reality of Assessing Learning Outcomes: Why you need common components.

60. Speaking

61. volume poise conclusion Speaking teacher1

62. volume poise conclusion Speaking teacher2 teacher1 gestures rate evidence

63. volume poise conclusion Speaking teacher2 teacher1teacher3 gestures rate evidence sources examples organization

64. volume poise conclusion Speaking teacher4teacher2teacher1teacher3 eye contact style appearance gestures rate evidence sources examples organization

65. volume poise conclusion Speaking teacher5 eye contact style appearance gestures rate evidence sources examples organization transitions verbal variety attention getter teacher4teacher2teacher1teacher3

66. volume poise conclusion eye contact style appearance gestures rate evidence sources examples organization transitions verbal variety attention getter Can our students deliver an effective Public Speech?

67. volume poise conclusion eye contact style appearance gestures rate evidence sources examples organization transitions verbal variety attention getter Can our students deliver an effective Public Speech?

68. volume poise conclusion eye contact style appearance gestures rate evidence sources examples organization transitions verbal variety attention getter Can our students deliver an effective Public Speech?

69. Are your outcomes differentiated by degree?

70. Calculate X Calculate X Calculate X Calculate X Apply C Apply C Apply C Apply C Describe Y Describe Y Describe Y Describe y Evaluate W Evaluate W Evaluate W Evaluate W Diagnose Z Diagnose Z Diagnose Z Diagnose Z CertificateAssociatesBachelorsMasters

71. Define X Name C Identify Y Recite W List Z Describe X Translate C Compare Y Report W Explain Z Apply X Produce C Calculate Y Evaluate W Manage Z Evaluate X Validate C Synthesize y Create W Diagnose Z CertificateAssociatesBachelorsMasters

72. Calculate X Calculate X Calculate X Calculate X Apply C Apply C Apply C Apply C Describe Y Describe Y Describe Y Describe y Evaluate W Evaluate W EvaluateSEvaluateS Evaluate W Diagnose Z Diagnose Z Diagnose Z Diagnose Z Major 1Major 2Major 3Major 4

73. Calculate X Calculate X Calculate X Calculate X Describe Y Describe Y Describe Y Describe y Diagnose Z Diagnose Z Diagnose Z Diagnose Z Major 1Major 2Major 3Major 4

74. Are your outcomes supported by the curriculum?

75. 7 1 2 3 4 5 6 Program Level Student Learning Outcomes

76. 1xx 7 1 2 3 4 5 6 Program Level Student Learning Outcomes 1xx 2xx 3xx 4xx Capstone

77. 1xx 7 1 2 3 4 5 6 Program Level Student Learning Outcomes 1xx 2xx 3xx 4xx Capstone

78. 1xx X X X X X X X 7 1 2 3 4 5 6 Program Level Student Learning Outcomes 1xx X X X X X X X 2xx X X X X X X X X X X X X X X X X X X X X X 3xx X X X X X X X X X X X X X X X X X X X X X 4xx X X X X X X X Capstone X X X X X X X

79. 1xx X X X 7 1 2 3 4 5 6 Program Level Student Learning Outcomes 1xx X X X 2xx X X X X X 3xx X X X X X X 4xx X X X X Capstone X X

80. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K 2xx A A S A A K 3xx A K A A K A 4xx S A K S Capstone S S

81. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K 2xx A A S A A K 3xx A K A A K A 4xx S A K S Capstone S S

82. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K 2xx A A S A A K 3xx A K A A K A 4xx S A K S Capstone S S

83. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K 2xx A A S A A K 3xx A K A A K A 4xx S A K S Capstone S S

84. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K 2xx A A S A A K 3xx A K A A K A 4xx S A K S Capstone S S

86. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K 2xx A A S A A K 3xx A K A A K A 4xx S A K S Capstone S S

87. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K 2xx A A S A A K 3xx A K A A K A 4xx S A K S Capstone S S

88. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K K 2xx A A A A A K K 3xx A K K K A A S S A S S A 4xx S A S S Capstone S S option cluster

89. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K K 2xx A A A A A K K 3xx A K K K A A S S A S S A 4xx S A S S Capstone S S

90. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes 1xx S K K K 2xx A A A A A K K 3xx A K K K A A S S A S S A 4xx S A S S Capstone S S Build a curriculum K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation

91. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes 1xx S K K K 2xx A A A A A K K 3xx A K K K A A S S A S S A 4xx S A S S Capstone S S Diagnose Learning Analyze Curriculum K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation

92. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes 1xx S K K K 2xx A A A A A K K 3xx A K K K A A S S A S S A 4xx S A S S Capstone S S Diagnose Learning K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation

93. Do students know the program has outcomes?

94. Program Level Outcomes In the catalog On the website In department related publications On the syllabi

95. Do you have a plan for assessing the outcomes?

96. Assessment Plan Assessment does NOT mean Every outcome for Every student in Every course Every semester by Every faculty member

97. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K K 2xx A A A A A A K 3xx A K K A A A S S A S S A 4xx S A S S Capstone S S S

98. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K K 2xx A A A A A A K 3xx A K K A A A S S A S S A 4xx S A S S Capstone S S S

99. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K K 2xx A A A A A A K 3xx A K K A A A S S A S S A 4xx S A S S Capstone S S S

100. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K K 2xx A A A A A K K 3xx A K K K A A S S A S S A 4xx S A S S Capstone S S

101. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K K 2xx A A A A A K K 3xx A K K K A A S S A S S A 4xx S A S S Capstone S S

102. 1xx K K K 7 1 2 3 4 5 6 Program Level Student Learning Outcomes K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation 1xx S K K K 2xx A A A A A K K 3xx A K K K A A S S A S S A 4xx S A S S Capstone S S

103. How will know if your students have learned?

104. Student Learning Outcome 3 2 1 4

105. Student Learning Outcome 3 2 1 4 Test Questions 1234 5678 101112 1713141516 9 18

106. Student Learning Outcome 3 2 1 4 Assignment 1 2 3 4

107. Student Learning Outcome 3 2 1 4 1 2 Assignment

108. Student Learning Outcome 3 2 1 4 Experience

109. Matching Outcomes to Objects Logical decision ? Individual vs. group assignment ? Amount of feedback provided in preparation of the assignment ? Timing / program / semester

110. Student Learning Outcome 3 2 1 4

111. Student Learning Outcome 3 2 1 4 Test Questions 12345678910 11121314151617181920

112. Student Learning Outcome 3 2 1 4 Assignment 1 2 3 4

113. Student Learning Outcome 3 2 1 4 Experience

114. Are you studying patterns of evidence?

115. Interpreting Data Consistency - over time Consensus - different populations Distinctiveness - different situations/ variables / items

116. Interpreting Data Consistency - over time Consensus - different populations Distinctiveness - different situations/ variables / items

117. Consistency Examines the same practice of and individual or group over time Key question: Has this person or group acted, felt, or performed this way in the past / over time?

118. Consistency How well are students performing on the learning outcome? High performance Low performance 0405060708091011

119. Consistency How many students utilize our services? High performance Low performance 0405060708091011

120. Consistency High performance Low performance 0506070809101104 How many students utilize our services?

121. Consensus Comparison to or among groups of students Variation between disciplines, gender, other demographic variables Key questions: What is the general feeling, outcome, attitude, behavior? Do other groups of people act, perform or feel this way?

122. Consensus How well are students performing on the learning outcome? FemalesMalesTransfersOTA High performance Low performance

123. Consensus How effectively are we meeting the needs of our students ? First GenInt’lNon residentialDistance High performance Low performance

124. Distinctiveness Examines individual or cohort perspectives across different situations, categories Key Question : Does a person or group respond differently based upon the situation, item, issue?

125. Distinctiveness How well are our students achieving our Learning Outcomes? High Performance Low Performance SLO3SLO4SLO2SLO5SLO1SLO6

126. Distinctiveness How satisfied are our student with the services we provide? High Performance Low Performance S3S4S2S5S1S6

127. Are we measuring achievement of the outcomes?

128. Low performance High performance SLO3SLO4SLO2SLO5SLO1SLO6 Have our students achieved our learning outcomes?

129. SLO3SLO4SLO2SLO5SLO1SLO6 Low performance High performance Have our students achieved our learning outcomes?

130. Have our students achieved our learning outcomes? SLO3SLO4SLO2SLO5SLO1SLO6 Low performance High performance

131. What are you doing to close the loop?

132. Closing the Loop

133. Development Faculty, Staff, Student Infrastructure Policy, Process, Planning Curriculum Learning Opportunities

134. Closing the Loop Development Faculty, Staff, Student Infrastructure Policy, Process, Planning Curriculum Learning Opportunities

135. Closing the Loop Development Faculty, Staff, Student Infrastructure Policy, Process, Planning Curriculum Learning Opportunities

136. Closing the Loop Development Faculty, Staff, Student Infrastructure Policy, Process, Planning Curriculum Learning Opportunities

137. The Seven Principles for Good Practice in Undergraduate Education 1. Student-Faculty Contact 2. Cooperative Learning 3. Active Learning 4. (Prompt) Feedback 5. Time on Task 6. High Expectations 7. Respect for Diverse Talents and Ways of Learning Zelda Gamson and Arthur Chickering Wingspread Journal 1987

138. Increase Reduce Improve Determine Maintain

139. Increase Reduce Improve Determine Maintain the Number Amount Percentage Ratio Level Rate

140. Increase Reduce Improve Determine Maintain Number Amount Percentage Ratio Level Rate Someone Students Faculty Staff Community members Employer of the

141. Increase Reduce Improve Determine Maintain Number Amount Percentage Ratio Level Rate Someone Students Faculty Staff Community members Employer Something applications grants Interviews request workshops reports satisfaction engagement of the

142. Increase Reduce Improve Determine Maintain Number Amount Percentage Ratio Level Rate Someone Students Faculty Staff Community members Employer Something applications grants Interviews request workshops reports satisfaction engagement > Something of who that with the

143. Increase Reduce Improve Determine Maintain Number Amount Percentage Ratio Level Rate Something applications grants Interviews request workshops reports satisfaction engagement Contribute to the annual campaign of who that with Someone Students Faculty Staff Community members Employers > Something the

144. Increase Reduce Improve Determine Maintain Number Amount Percentage Ratio Level Rate Someone Students Faculty Staff Community members Employer Something applications grants Interviews request workshops reports satisfaction engagement Submit Grant Applications of who that with > Something the

145. Increase Reduce Improve Determine Maintain Number Amount Percentage Ratio Level Rate Paper, Glass and Plastic Are not recycled of who that with Someone Students Faculty Staff Community members Employers > Something the

146. Increase Reduce Determine Maintain Number Amount Percentage Ratio Level Rate of who that with Someone Seniors Faculty Staff Community members Employers > Something Conduct Independent Research Something the

147. Increase Reduce Determine Maintain Number Amount Percentage Ratio Level Rate of who that with Someone Seniors Faculty Staff Community members Employers > Something Pass the licensing exam on the first try Something the

148. Increase Reduce Determine Maintain Number Amount Percentage Ratio Level Rate of who that with Someone Seniors Faculty Staff Community members Employers > Something Pass the licensing exam on the first try Something theby Action

149. Educating….. Assisting ….. Explaining ….. Assisting ….. Developing ….. Consulting ….. Delivering ….. Facilitating ….. Creating ….. Nurturing ….. Negotiating ….. Sponsoring ….. Training ….. Translating ….. Validating ….. Updating ….. Enforcing ….. Testing ….. Assessing ….. Reporting ….. Issuing ….. Communicating ….. Integrating ….. Responding ….. Arranging ….. Coordinating ….. Providing ….. Organizing ….. Producing ….. Training ….. Maintaining ….. Supporting ….. Directing ….. Designing ….. Collecting ….. Developing ….. Responding ….. Reviewing ….. Studying ….. Distributing ….. Executing ….. Recruiting ….. Generating ….. Implementing ….. Investigating ….. Designing ….. Collaborating ….. Revising ….. Scheduling ….. Sharing ….. Increase Reduce Improve Determine Maintain Number Amount Percentage Ratio Level Rate Something applications grants Interviews request workshops reports satisfaction engagement by of who that with Someone Students Faculty Staff Community members Employers > Something the

150. Increase Reduce Determine Maintain Number Amount Percentage Ratio Level Rate of who that with Someone Seniors Faculty Staff Community members Employers > Something Pass the licensing exam on the first try Something theby Action

151. Increase Reduce Determine Maintain Number Amount Percentage Ratio Level Rate of who that with Someone Seniors Faculty Staff Community members Employers > Something Pass the licensing exam on the first try Something theby Action

152. Youngstown State University September 2011 Susan Hatfield Winona State University Winona, MN SHatfield@winona.edu Assessing your Assessment Plan