1. UNIVERSITY OF WISCONSIN–MADISON
ASSESSMENT FRAMEWORK
Instructions – 15 min about how we are doing assessment; panel with UMinn and I think OSU member.
Hello.
I’m very pleased to join this panel today.
Thank you for the invitation.
Like OSU, University of Wisconsin-Madison is a large research university.
Almost undergraduates and total students in 125 departments and nearly 500 academic programs.
Not quite as big as OSU, but similar enough to share the challenge of implementing a coherent assessment framework at scale across a very large university.
I’ll tell you a bit about what we are doing and I’m looking forward to learning from you and my fellow panel members some ways we might advance our efforts.

2. Presenter
Mo Bischof, Associate Vice Provost, Director of Assessment | provost.wisc.edu/assessment

3. 01 | OVERVIEW OF CONTEXT FOR ASSESSMENT OF STUDENT LEARNING IN HIGHER EDUCATION
02 | STUDENT LEARNING ASSESSMENT FRAMEWORK AT UW-MADISON
03 | TIMELINE, PROGRESS AND CHALLENGES
04 | ALIGNMENT WITH MAJOR CAMPUS INITIATIVES
05 | QUESTIONS AND DISCUSSION

4. NATIONAL CONTEXT

5. National Context
Focus on student learning in higher education has shifted – “value question”
Emerging pedagogies/technology – learner-centered approaches
Shift from inputs (delivering content) to outputs (learning impact)
Cross-functional processes, less reliance on local only
Re-accreditation guidelines (more on that later)

6. Bottom Line
Expectations require evidence to demonstrate student learning and the value of a degree.

7. ASSESSMENT AT UW-MADISON

8. Milestones Gen Ed 2003 Math, VALUE A-Plan Writing Rubrics WI-X
2003
A-Plan
Math,
Writing
A-Council
A-Fund
Gen Ed
HLC
VALUE
Rubrics
WI-X
update
WI-X
MIU
Big10/
NILOA
LO collection
A-Plans
Adopted
ELOs,
“Convergence”
Program
Review
Planning
REACH
Educational
Innovations
Hired
A-Staff
2015
Assessment
Plan
AACU
LEAP
A-Reports,
HLC Prep
VSA-College Portrait
Student
Digital
Ecosystem
You can see here a 25 year timeline, with some of the key milestones in student learning assessment.
I’m not going to go into depth on all of these projects and initiatives – it’s a bit of an alphabet soup.
The key point here is that assessment isn’t something new – it’s something we’ve been working on and talking about since at least 1991.
The assessment program began then with a focus on math and writing, and an Assessment Council with school/college representation and an assessment fund.
And assessment is by now a deep-rooted effort that most of our academics have at least passing experience with over time.
And we have by now a number of excellent examples of where assessment approaches make a marked difference in student learning and the student experience.
For Engineering programs, the requirements of ABET accreditation have helped establish assessment as routine.
And in pockets where faculty have invested enough to see the value, ongoing assessment of learning and student experience are well established.
But like other large universities its been a challenge to establish a sustained, consistent and comprehensive approach.
We are quite decentralized, and the responsibility for implementing previous assessment plans has been at the level of the deans offices of each school/college, and that has resulted in some unevenness in implementation.
So over the past few years we’ve been focused on establishing a consistent level of assessment across all academic programs.
…………..[next slide]……………….

9. Milestones Gen Ed 2003 Math, VALUE A-Plan Writing Rubrics WI-X
2003
A-Plan
Math,
Writing
A-Council
A-Fund
Gen Ed
HLC
VALUE
Rubrics
WI-X
update
WI-X
MIU
Big10/
NILOA
LO collection
A-Plans
Adopted
ELOs,
“Convergence”
Program
Review
Planning
REACH
Educational
Innovations
Hired
A-Staff
2015
Assessment
Plan
AACU
LEAP
A-Reports,
HLC Prep
VSA-College Portrait
Student
Digital
Ecosystem
In recent years we’ve made some changes to get to a pervasive use of assessment approaches across all of our programs.
About four years ago we hired our first full time staff person to work on a rubric-based assessment project of undergraduate writing.
That position evolved into what is now the assessment coordinator position under Mo Bischof, who directs the assessment program.
It’s been very helpful to has someone giving full-time attention to this effort.
In early 2015 our faculty approved a new assessment plan, which sets our current assessment framework.
Over the past two years, as we began to implement that assessment plan we have seen a positive interaction with a range of other initiatives.
Three of those other initiatives – all benefiting from the leadership of Steve Cramer VPTL - are especially relevant and I’ll say a bit more about them later -
the REACH project, which aims to transform large, introductory, lecture-based courses into sustainable, active learning environments that increase students’ engagement in their own learning.
Implementation of a Student Digital Ecosystem, which connects the student information system, learning management and curriculum management systems, and online assessment tools in a way that allow for interoperability and flexibility, improve information accessibility and consistency, and support academic improvement efforts at scale.
An update to the language for what we call the Wisconsin Experience, a melding of a learning experience rich in high impact practices and experiential learning with the Wisconsin Idea
But for now I want to turn back to the assessment plan and the features of the plan that are proving to support successful implementation.

10. 2015 Assessment Plan Principles
Activities of academic life are evaluative; systematic approaches make them assessment
Student learning assessment supplements program and curricular review
Assessment informs decision-making across institutional, school/college, division, department/program and course-level
The 2015 assessment plan is based on principles that are carried forward from prior plans.
This is helpful because these principles are consistent over time and reflect our assessment culture.
I want to call our a few highlights -
- evaluative activity is part of regular academic life – what makes it assessment is making these activities systematic and explicit
is faculty driven – the university plan has the endorsement of faculty governance and faculty involvement is an expectation. Program and course LOs are approved through governance
Assessment is student-centered
assessment of learning is required at the program level, and we recognize that there is a lot to be gained by examining the student experience and curricular review
- Assessment is connected to regular program review and to curricular improvement … proposals for change supported by evidence.

11. 2015 Assessment Plan Principles
Assessment if ongoing and iterative; it promotes program excellence and explores questions about students’ educational experiences
Assessment activities are faculty driven
Collaboration between academic departments and co-curricular programs identifies and aligns opportunities for assessment student learning across the students’ educational experience
The 2015 assessment plan is based on principles that are carried forward from prior plans.
This is helpful because these principles are consistent over time and reflect our assessment culture.
I want to call our a few highlights -
- evaluative activity is part of regular academic life – what makes it assessment is making these activities systematic and explicit
is faculty driven – the university plan has the endorsement of faculty governance and faculty involvement is an expectation. Program and course LOs are approved through governance
Assessment is student-centered
assessment of learning is required at the program level, and we recognize that there is a lot to be gained by examining the student experience and curricular review
- Assessment is connected to regular program review and to curricular improvement … proposals for change supported by evidence.

12. 2018-19 UW-Madison Re-accreditation Higher Learning Commission
Criterion
1. Mission & Integrity
1. Mission
2.Resource & Infrastructure
2. Integrity: Ethical & Responsible Conduct
3. Student Learning & Effective Teaching
3. Teaching & Learning: Quality, Resources & Support
4. Acquisition, Discovery & Application of Knowledge
4. Teaching & Learning: Evaluation & Improvement
5. Engagement & Service
5. Institutional Effectiveness
Student Learning – HLC Accreditation

13. WWHSW Model What – Articulate student learning outcomes
Where – Identify where in the curriculum the learning takes place
How – Develop an assessment plan; engage in at least one assessment activity per year; direct assessment within a 1-3-year period
So What – Report findings annually to the Office of the Provost; use findings as a basis for improvement, program review, curricular change
And in the context of those durable principles - we did something quite different for our university.
I mentioned before that the implementation of assessment had been delegated to the dean’s offices.
In the process of preparing the 2015 assessment plan the school/college reps for assessment did something that surprised us.
They unanimously requested that we centralize the implementation of the plan.
And they endorsed a prescriptive approach to assessment.
At the course of our assessment plan then is a prescribed approach for each academic degree/major -
- every academic program must articulate learning outcomes
We mandated a deadline for faculty-approved learning outcomes and collected them starting in summer 2015
We have almost 100% compliance
All programs will have their learning outcomes published in the Guide starting Summer 2017
Every degree/major program must have an assessment plan that includes a curricular map of where learning takes place
- may include co-curricular activities
- all COURSES are required to have learning outcomes – the vision is that program and course learning outcomes are nested
Work in progress - implementation of the various components of the Student Digital Ecosystem
About 60% participation so far in submission of assessment plans, expected to improve as we can provide better tools
Starting in Fall 2017, program faculty will all need to report on assessment activities, findings, and plans for improvement.

14. WWHSW Model (WWHSW) Academic Program Assessment SO WHAT
Interpret/Report and Use Findings
WHAT
Develop/Revise Learning Goals
WHERE
Map Learning Goals to Curriculum & Student Experiences
HOW
Develop & Conduct Assessments
(metrics, measures)

15. Learning Assessment Timeline
July 2015 – Programs submitted learning outcomes, published to Guide (nearly 100% reported)
July 2016 – Programs submitted 3-year assessment plans (~70% UGs, 67% Masters, 80% Ph.Ds)
Nov – Assessment annual reports due, report template (in-design); report through AEFIS tool
Ongoing – Program assessment reports will be submitted annually to the Office of the Provost
The assessment program is now asking program faculty to participate in consistent and sustained assessment activities.
Although this has been an expectation for many years, now we are insisting. So we have to put some infrastructure in place to help program faculty and staff participate.
By removing the dean’s offices as a conduit for information, we’ve gained efficiencies.
The messages we provide are consistent and we make an effort to keep them as simplified as possible.
(Getting “Oh that’s what you mean?” response).
We’ve built collection systems for learning outcomes, and for assessment plans, and we have created standard templates and forms.
We’ve established training and outreach programs for both groups and also for departments that want
And we are building better on-line way-finding to data resources that support assessment and an understanding of the student experience – getting them lined to information that used to be routed through the deans offices.
-reports and analysis from student and curricular records and surveys
We are implementing a enterprise-wide course evaluation system that when fully implemented will support assessment.
We can’t do the direct measures of learning for departments, but we can given them support for indirect assessment and tools for direct assessment.

16. Learning Outcomes Practices
Work directly with departments, not via dean’s offices; one consistent message
Centralized infrastructure in place to collect learning outcomes, assessment plans, assessment reports
Common templates for curricular maps, assessment plans, assessment reports
Training sessions, group workshops, one-on-one workshops
The assessment program is now asking program faculty to participate in consistent and sustained assessment activities.
Although this has been an expectation for many years, now we are insisting. So we have to put some infrastructure in place to help program faculty and staff participate.
By removing the dean’s offices as a conduit for information, we’ve gained efficiencies.
The messages we provide are consistent and we make an effort to keep them as simplified as possible.
(Getting “Oh that’s what you mean?” response).
We’ve built collection systems for learning outcomes, and for assessment plans, and we have created standard templates and forms.
We’ve established training and outreach programs for both groups and also for departments that want
And we are building better on-line way-finding to data resources that support assessment and an understanding of the student experience – getting them lined to information that used to be routed through the deans offices.
-reports and analysis from student and curricular records and surveys
We are implementing a enterprise-wide course evaluation system that when fully implemented will support assessment.
We can’t do the direct measures of learning for departments, but we can given them support for indirect assessment and tools for direct assessment.

17. Infrastructure, Resources and Support
Work directly with departments, not via dean’s offices; one consistent message
Centralized infrastructure in place to collect learning outcomes, assessment plans, assessment reports
Common templates for curricular maps, assessment plans, assessment reports
Training sessions, group workshops, one-on-one workshops
The assessment program is now asking program faculty to participate in consistent and sustained assessment activities.
Although this has been an expectation for many years, now we are insisting. So we have to put some infrastructure in place to help program faculty and staff participate.
By removing the dean’s offices as a conduit for information, we’ve gained efficiencies.
The messages we provide are consistent and we make an effort to keep them as simplified as possible.
(Getting “Oh that’s what you mean?” response).
We’ve built collection systems for learning outcomes, and for assessment plans, and we have created standard templates and forms.
We’ve established training and outreach programs for both groups and also for departments that want
And we are building better on-line way-finding to data resources that support assessment and an understanding of the student experience – getting them lined to information that used to be routed through the deans offices.
-reports and analysis from student and curricular records and surveys
We are implementing a enterprise-wide course evaluation system that when fully implemented will support assessment.
We can’t do the direct measures of learning for departments, but we can given them support for indirect assessment and tools for direct assessment.

18. Infrastructure, Resources and Support
Assessment tools, including an enterprise implementation of on-line course/learning assessment solutions
Institutional Shared Data: APIR data for undergraduates (NSSE data, enrollment, degree trends, grade gap analysis, curricular data, etc.)
Institutional Shared Data: Graduate School Exit Surveys
The assessment program is now asking program faculty to participate in consistent and sustained assessment activities.
Although this has been an expectation for many years, now we are insisting. So we have to put some infrastructure in place to help program faculty and staff participate.
By removing the dean’s offices as a conduit for information, we’ve gained efficiencies.
The messages we provide are consistent and we make an effort to keep them as simplified as possible.
(Getting “Oh that’s what you mean?” response).
We’ve built collection systems for learning outcomes, and for assessment plans, and we have created standard templates and forms.
We’ve established training and outreach programs for both groups and also for departments that want
And we are building better on-line way-finding to data resources that support assessment and an understanding of the student experience – getting them lined to information that used to be routed through the deans offices.
-reports and analysis from student and curricular records and surveys
We are implementing a enterprise-wide course evaluation system that when fully implemented will support assessment.
We can’t do the direct measures of learning for departments, but we can given them support for indirect assessment and tools for direct assessment.

19. Challenges – Cultural Change
For programs that are not subject to disciplinary accreditation, assessment may perceived as a extra work – something imposed.
Cultural change is evolutionary
Keys to success - context
Illustrate student benefit
Programs in control of their own assessment data
Infrastructure that facilitates participation
Communication, communication, communication

20. Aligning Assessment & Major Initiatives
Student Digital Ecosystem - connects the student information system, learning management system , curriculum management systems, and online assessment tools for interoperability and flexibility; support academic improvement efforts at scale
REACH – transforming large, introductory, lecture-based courses into sustainable, active learning environments that increase students’ engagement in their own learning.
Wisconsin Experience – updated to efforts to meld a learning experience rich in high impact practices and experiential learning with the Wisconsin Idea
What we are now seeing is that the assessment program is integrated into our major teaching and learning initiatives as a matter of course.
Three projects to mention
the REACH project, which aims to transform large, introductory, lecture-based courses into sustainable, active learning environments that increase students’ engagement in their own learning.
Implementation of a Student Digital Ecosystem, which connects the student information system, learning management and curriculum management systems, and online assessment tools in a way that allow for interoperability and flexibility, improve information accessibility and consistency, and support academic improvement efforts at scale.
An update to the language for what we call the Wisconsin Experience, a melding of a learning experience rich in high impact practices and experiential learning with the Wisconsin Idea

21. Student Digital Ecosystem
Advising Gateway
Student Digital Ecosystem
Student info system
BI tools, Tableau
Curriculum Management System
Academic policy
Curricular structure
Academic approvals
Course scheduling
Program LOs, Course LOs
T&L Tools – Unizin plus
LMS (Migration to Canvas)
Integrated course-level Learning Analytics
Content sharing
Other tools
Career Services
Public
FERPA
Student Learning Assessment
Programs/Course
[Student Digital EcoSystem, From Steve Cramer, ; revised for APLU ]
(Skip if time is short).
I mentioned the infrastructure – this “bubble chart” is what we use on campus to reference a set of interconnected technology systems that are transforming the way we work and support teaching and learning, and the student experience.
What we are striving for is seamless interoperability.
The curriculum management system will serve as a “single source of truth” for the curriculum and academic program requirements.
It will house both learning outcomes for courses and academic programs, and eventually provide the tools for integrated curricular mapping.
Starting in May 2017, those learning outcomes that are required for all degree/major programs will be published in the Guide, our first major overall of academic catalogs in more than 25 years.
Our LMS, Canvas, will be a resource for learning analytics. We are explorers at this point and are taking a deliberate, planful approach.
Assessment tools, including an integrated course evaluation and survey tool, is included in the digital ecosystem – this change from pen-and-pencil/scantron format a full-scale, enterprise-wide, online system has catalyzed/provoked some of the most valuable and vital conversations about the role of assessment to date.
Culture change.
Integration with advising tools, career services tools, and a re-build of our data management and analysis environment.
Private

22. REACH Initiative
Large enrollment, introductory course, lecture transformation project
Currently 9 courses in expanding to 12 and beyond through 2021– impacting over 15,000 students per year
Active learning and detailed assessment proceeding in an integrated manner.

23. Wisconsin Experience Empathy and Humility Intellectual Confidence
Develop and demonstrate cultural understanding of self and others
Engage locally, nationally, and globally in respectful and civil manner
Appreciate and celebrate one another’s abilities, views, and accomplishments
Relentless Curiosity
Actively learn with expert teachers, scholars, and peers
Engage in creative inquire, scholarship, and research
Develop resilience and foster courage in life and learning
Intellectual Confidence
Develop competence, depth and expertise in a field of study
Integrate ideas and synthesize knowledge across multiple contexts
Exercise critical thinking and effective communication
Purposeful Action
Apply knowledge and skills to solve problems
Engage in public service, partner with others, and contribute to community
Lead for positive change

24. Faculty Perspectives
“Course redesign begins with the careful identification of learning outcomes and of ways to assess students’ progress towards those outcomes. Assessment at the course level can then be aligned with program- and campus-level assessment effort.”
“As we explore new pedagogies and new instructional technologies, UW’s systematic approach to assessment will keep student learning front and center, where it belongs.”
Professor John Zumbrunnen Political Science, UW-Madison
I’ll close with this observation from a faculty member who has been very involved with our curricular redesign efforts.
His highlights many of the strengths of our teaching and learning improvement efforts
- Faculty involvement
Student learning at the center
Alignment of course-level, program-level and campus-level efforts
And lots of exploration and experimentation

25. THANK YOU.