1. Timothy S. Brophy, Ph.D., Director of Institutional Assessment University of Florida Office of the Provost

2. Today’s Goals Describe and explain assessment systems Review UF’s program assessment components Discuss program assessment systems and steps to strengthen them Review examples

3. Assessment Systems A system is a complex whole in which there are multiple parts or phases that can stand alone but are stronger when linked logically into a cohesive combination The system can use linear and nonlinear sequences of events or processes, or a combination of both Assessment systems consist of curricular, educational, and procedural components The UF Assessment System is one example

4. UF Assessment System Establish Mission, Goals, and Outcomes Assessment Planning Implement the Plan and Gather Data Interpret and Evaluate the Data Modify and Improve May – Assessment Plans and Effectiveness Documentation Reports submitted for the next AY October - Assessment Data, results, and use of results for previous AY entered into Compliance Assist! Assessment and Institutional Effectiveness Data Reporting

5. Elements of program assessment systems Catalog ALCs Graduation requirements Assessment Plans SLOs Measurement tools Curriculum Maps and Assessment matrices Compliance Assist! Goals Results Use of results Student work samples

6. The Academic Learning Compact 2012-13 UF Undergraduate Catalog Academic Learning Compacts identify the skills students should acquire if they follow their major’s prescribed course of study. These skills, known collectively as Student Learning Outcomes, describe the core learning expectations that UF is required to assess for each baccalaureate degree program. Core learning expectations identify communication, critical thinking, and content knowledge skills, as well as additional learning outcomes specific to the major.

7. Student Learning Outcomes What students are expected to learn by the completion of their baccalaureate degree

8. Individual Student Assessments The different ways in which UF will measure whether students have successfully completed the learning outcomes for a major. These assessments can include a passing score on a particular test or final project, term paper, portfolio, etc. - 2012-13 UF Undergraduate Catalog

10. Academic Learning Compact What we sometimes have SLO Assessment Course

11. What we should have in all programs Academic Learning Compact SLOAssessmentCourseAssessmentCourse SLOAssessmentCourseAssessmentCourseSLOAssessmentCourseAssessmentCourse

12. How do we get from ALC SLOAssessmentCourseAssessmentCourse SLOAssessmentCourseAssessmentCourseSLOAssessmentCourseAssessmentCourse SLOSLO Asse ssm ent C o ur se Academic Learning Compact SLOSLO SLOSLO Asse ssm ent C o ur se THIS TO THIS?

13. And avoid this when dealing with assessment? SLOs!! Plans!!! Rubrics!!!

14. Start with the ALC program description Components of a strong ALC program description: Establishes learning goals for the degree program – the “core learning expectations” Addresses the relationship of the degree to the job or career

15. Characteristics of Program Learning Goals found in program descriptions These are broad statements that present multiple measurable components so the statement itself may be “unmeasurable” These often use verbs like “understand” and “learn” and “gain the ability to…” etc. in the program description Why? These are the broad LEARNING GOALS of the program – not the LEARNING OUTCOMES SLOs represent the measurable components of the SLOs – they are “S.M.A.R.T.” SLOs are stated using active verbs However, the SLOs and goals share common language and intent

16. This model allows you to develop assessments that measure the outcomes, and that then connect directly to the program goals Specific Learning Outcome – the course-level SLO General Learning Outcome – the program-level SLO – these describe what students will do to demonstrate the learning goals Learning Goal Level – programs establish learning goals for the degree – at UF these are described in the Academic Learning Compact Developing an Assessment System: A Three-level Model (Carriveau, 2010)

17. A sample description: Materials Science Engineering The major enables you to develop an understanding of materials systems and their role in engineering. Emphasis is placed on the ability to apply knowledge of mathematics, science and engineering principles to materials science and engineering; to design and conduct experiments, as well as to analyze and interpret data; and to design a system, component or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability. Learning Goals Connection to the job or career

18. Students who complete the MSE degree will: Understand materials systems and their role in engineering Gain the ability to apply knowledge of mathematics, science and engineering principles to materials science and engineering Level 1: Learning Goals based on the MSE ALC

19. Content Knowledge Apply knowledge of mathematics, science and engineering principles to materials science and engineering. Design and conduct materials science and engineering experiments and analyze and interpret the data. Critical Thinking Design a materials science and engineering system, component or process to meet desired needs within realistic economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability constraints. Communication Communicate technical data and design information effectively in speech and in writing to other materials engineers. Level 2 – Program Student Learning Outcomes for MSE

20. Learning Goals: Understand materials systems and their role in engineering Ability to apply knowledge of mathematics, science and engineering principles to materials science and engineering. Student Learning Outcomes: Design a materials science and engineering system, component or process to meet desired needs within realistic economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability constraints. Communicate technical data and design information effectively in speech and in writing to other materials engineers MSE: Connecting Goals to Outcomes Goal SLO

21. Learning Goals: Understand materials systems and their role in engineering Ability to apply knowledge of mathematics, science and engineering principles to materials science and engineering. Student Learning Outcomes: Apply knowledge of mathematics, science and engineering principles to materials science and engineering. Design and conduct materials science and engineering experiments and analyze and interpret the data MSE: Connecting Goals to Outcomes Level 1 Level 2

22. Connecting Program SLOs to Courses MSE Curriculum Map Assessments in the boxes marked A are conducted using specific homework, exam, or assignment questions aligned with that SLO. Source: 2011-12 MSE Academic Assessment Plan

23. Completing the System Align course objectives with program SLOs Develop assessments of the course objectives Select course assessments as assessments of the program SLOs – vet these with the program faculty

24. Completing the System Collect data Analyze and interpret Modify Report to Compliance Assist!

25. A synopsis… Program Description Learning Goals Program SLOs Course objectives and assessments Data collection, analysis, and interpretation Annual Reporting and Planning

26. All of this leads to SUCCESS!