Timothy S. Brophy, Ph.D., Director of Institutional Assessment University of Florida Office of the Provost

Today’s Goals Describe and explain assessment systems Review UF’s program assessment components Discuss program assessment systems and steps to strengthen them Review examples

Assessment Systems A system is a complex whole in which there are multiple parts or phases that can stand alone but are stronger when linked logically into a cohesive combination The system can use linear and nonlinear sequences of events or processes, or a combination of both Assessment systems consist of curricular, educational, and procedural components The UF Assessment System is one example

UF Assessment System Establish Mission, Goals, and Outcomes Assessment Planning Implement the Plan and Gather Data Interpret and Evaluate the Data Modify and Improve May – Assessment Plans and Effectiveness Documentation Reports submitted for the next AY October - Assessment Data, results, and use of results for previous AY entered into Compliance Assist! Assessment and Institutional Effectiveness Data Reporting

Elements of program assessment systems Catalog ALCs Graduation requirements Assessment Plans SLOs Measurement tools Curriculum Maps and Assessment matrices Compliance Assist! Goals Results Use of results Student work samples

The Academic Learning Compact UF Undergraduate Catalog Academic Learning Compacts identify the skills students should acquire if they follow their major’s prescribed course of study. These skills, known collectively as Student Learning Outcomes, describe the core learning expectations that UF is required to assess for each baccalaureate degree program. Core learning expectations identify communication, critical thinking, and content knowledge skills, as well as additional learning outcomes specific to the major.

Student Learning Outcomes What students are expected to learn by the completion of their baccalaureate degree

Individual Student Assessments The different ways in which UF will measure whether students have successfully completed the learning outcomes for a major. These assessments can include a passing score on a particular test or final project, term paper, portfolio, etc UF Undergraduate Catalog

Academic Learning Compact What we sometimes have SLO Assessment Course

What we should have in all programs Academic Learning Compact SLOAssessmentCourseAssessmentCourse SLOAssessmentCourseAssessmentCourseSLOAssessmentCourseAssessmentCourse

How do we get from ALC SLOAssessmentCourseAssessmentCourse SLOAssessmentCourseAssessmentCourseSLOAssessmentCourseAssessmentCourse SLOSLO Asse ssm ent C o ur se Academic Learning Compact SLOSLO SLOSLO Asse ssm ent C o ur se THIS TO THIS?

And avoid this when dealing with assessment? SLOs!! Plans!!! Rubrics!!!

Start with the ALC program description Components of a strong ALC program description: Establishes learning goals for the degree program – the “core learning expectations” Addresses the relationship of the degree to the job or career

Characteristics of Program Learning Goals found in program descriptions These are broad statements that present multiple measurable components so the statement itself may be “unmeasurable” These often use verbs like “understand” and “learn” and “gain the ability to…” etc. in the program description Why? These are the broad LEARNING GOALS of the program – not the LEARNING OUTCOMES SLOs represent the measurable components of the SLOs – they are “S.M.A.R.T.” SLOs are stated using active verbs However, the SLOs and goals share common language and intent

This model allows you to develop assessments that measure the outcomes, and that then connect directly to the program goals Specific Learning Outcome – the course-level SLO General Learning Outcome – the program-level SLO – these describe what students will do to demonstrate the learning goals Learning Goal Level – programs establish learning goals for the degree – at UF these are described in the Academic Learning Compact Developing an Assessment System: A Three-level Model (Carriveau, 2010)

A sample description: Materials Science Engineering The major enables you to develop an understanding of materials systems and their role in engineering. Emphasis is placed on the ability to apply knowledge of mathematics, science and engineering principles to materials science and engineering; to design and conduct experiments, as well as to analyze and interpret data; and to design a system, component or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability. Learning Goals Connection to the job or career

Students who complete the MSE degree will: Understand materials systems and their role in engineering Gain the ability to apply knowledge of mathematics, science and engineering principles to materials science and engineering Level 1: Learning Goals based on the MSE ALC

Content Knowledge Apply knowledge of mathematics, science and engineering principles to materials science and engineering. Design and conduct materials science and engineering experiments and analyze and interpret the data. Critical Thinking Design a materials science and engineering system, component or process to meet desired needs within realistic economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability constraints. Communication Communicate technical data and design information effectively in speech and in writing to other materials engineers. Level 2 – Program Student Learning Outcomes for MSE

Learning Goals: Understand materials systems and their role in engineering Ability to apply knowledge of mathematics, science and engineering principles to materials science and engineering. Student Learning Outcomes: Design a materials science and engineering system, component or process to meet desired needs within realistic economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability constraints. Communicate technical data and design information effectively in speech and in writing to other materials engineers MSE: Connecting Goals to Outcomes Goal SLO

Learning Goals: Understand materials systems and their role in engineering Ability to apply knowledge of mathematics, science and engineering principles to materials science and engineering. Student Learning Outcomes: Apply knowledge of mathematics, science and engineering principles to materials science and engineering. Design and conduct materials science and engineering experiments and analyze and interpret the data MSE: Connecting Goals to Outcomes Level 1 Level 2

Connecting Program SLOs to Courses MSE Curriculum Map Assessments in the boxes marked A are conducted using specific homework, exam, or assignment questions aligned with that SLO. Source: MSE Academic Assessment Plan

Completing the System Align course objectives with program SLOs Develop assessments of the course objectives Select course assessments as assessments of the program SLOs – vet these with the program faculty

Completing the System Collect data Analyze and interpret Modify Report to Compliance Assist!

A synopsis… Program Description Learning Goals Program SLOs Course objectives and assessments Data collection, analysis, and interpretation Annual Reporting and Planning

All of this leads to SUCCESS!