1. Assessment Where Learning Matters
David Marshall, PhD, Senior Scholar
National Institute for Learning Outcomes Assessment (NILOA)

2. ● Surveys ● Web Scans ● Case Studies ● Focus Groups
NILOA
NILOA’s mission is to discover and disseminate effective use of assessment data to strengthen undergraduate education and support institutions in their assessment efforts.
● Surveys ● Web Scans ● Case Studies ● Focus Groups
● Occasional Papers ● Website ● Resources ● Newsletter ● Presentations ● Transparency Framework ● Featured Websites ● Accreditation Resources ● Assessment Event Calendar ● Assessment News ● Measuring Quality Inventory ● Policy Analysis ● Environmental Scan ● Degree Qualifications Profile ● Tuning

4. Involved in so much And we are exhausted – involved in tons of initiatives, mostly disconnected from each other – and when we look at the results of work external groups are not impressed – we respond with more committees
Why are we in this space?? How did we get here and how do we get out?

5. Assessment Cube of Misunderstandings
Uses/Questions
Purposes/Value
Definitions
Levels/Focus

6. Some say it’s because of the tensions to the ends of education – work, democracy, good life – others say it’s because of the tension between improvement and accountability – a topic that is not new to our field and one in which Peter Ewell updated a paper of his from I believe the 1980s on improvement and accountability

7. Outcomes Frameworks

8. Principles of Local Practice
Develop specific, actionable learning outcomes statements.
Connect learning outcomes with actual student demonstrations of their learning.
Collaborate with relevant stakeholders, beginning with the faculty.
Design assessment approaches that generate actionable evidence about student learning that key stakeholders can understand and use to improve student and institutional performance.
Focus on improvement and compliance will take care of itself.
In 2016 we looked across everything we learned and offered up a policy statement with 5 principles to inform practice – culminating in if we focus on improvement, compliance will take care of itself. So why the tension and struggle still??
I think it’s due in part to something I loving refer to as the assessment cube of misunderstanding

10. Focus on shift in approach to assessment over time – now along the way we know by the end where they are…because of an embedded approach
Requires a purpose and focus of assessment
Diversity of approaches and institutions
Formative embedded assessment
Organization and support (structures)
Curriculum mapping and course taking patterns
Increasing role of technology

11. A Focus on Assignments Most valuable assessment approaches:
Classroom-based assessment
National student surveys
Rubrics

12. Institutional or Program Improvement

14. Learning Improvement

15. Assignments as Assessments
Faculty professional staff are working to create a curriculum that intentionally builds in integrated learning opportunities over time for students to apply and practice as well as transfer their knowledge and skills through assignments, in and out of courses.
Can’t assume coverage in a course
How embed – can’t wait until the end as summative assessment, needs to be formative along the way (mention workshop on assignment design)

17. The Learning Systems Paradigm
Framework to think about assignment and some things to consider that are missed often

18. Assignments as a way in Assignment Learning Outcomes
Scaffolding Learning
Evaluative Criteria
Learning Outcomes
Assignments as assessments – what can be done in one assignment where else to look or partner

19. How assignments connect
Scaffolding Learning
Evaluative Criteria
Learning Outcomes
Assignment
Scaffolding Learning
Evaluative Criteria
Learning Outcomes
Assignment
Scaffolding Learning
Evaluative Criteria
Learning Outcomes

20. Program View General Education Major Assignment Learning Outcomes
Scaffolding Learning
Evaluative Criteria
Learning Outcomes
General Education
Major

21. Implications for transcripts, career development, and pathways
Connections
Assignment
Scaffolding Learning
Evaluative Criteria
Learning Outcomes
Employers
Co-curricular
Implications for transcripts, career development, and pathways

22. Consensus-based
Through faculty-led conversations, reflections, and explorations with employers, alumni, students, and others, a shared understanding and consensus is reached on learning outcomes. This shared understanding serves as the foundation for revising outcomes for enhanced clarity and designing educational experiences.
Basically – what are we all talking about? How do we understand our similarities and our differences?
Consensus does not mean we all agree – or we get to 100% word smithing agreement – but that we have a shared understanding that allows us to move forward as a collective – I know what you are doing and you know what I am doing and where we differ and or agree

23. Alignment
Using the agreed upon learning outcomes, faculty and staff align educational experiences throughout the institution for intentional integration, coherence, and fostering of multiple pathways. Alignment involves curriculum mapping, scaffolding, assignment design, mapping of career pathways, and co-curricular engagement.
Once we have some shared understanding, we need to figure out how our different bits fit together – this is alignment – how do our disparate elements come together in a collective whole?

24. Alignment
How do you ensure alignment between our assignments and a given learning outcome for a learning experience?
How do we create assignments and activities that will elicit student demonstration of a specific learning outcome?
How do we know that we have mapped our assignment to rubric criteria?
Where do students engage with these outcomes across a curriculum or collegiate experience and how are we capturing that learning?
Does this assignment or course introduce/reinforce/demonstrate mastery?
Where will it be practiced later? Is there a chance to try again based on feedback?
How does the assignment relate to evaluation criteria (rubric or otherwise)?

25. Verbs (and I don’t mean Bloom’s)
Alignment and fitness of method occur in relation to the verbs identified in the learning outcome statement
In your own assignment – what are you asking students to actually do or demonstrate?

26. Learner-Centered
The educational system reorganizes educational experiences around all students and their learning. Taking a student view includes consideration of issues of equity, learning-focused transfer, alternative delivery models, flexibility in offerings, integration of prior-learning assessment, ensuring stackable credentials, and building multiple pathways.
Alll is for naught it we don’t keep the students at the center – but an individual learner view
I like to share the story of the students on the bus and the ones coming in early found on analytics – don’t make assumptions – involve the students
If it’s about learning – then all sorts of possibilities open up

27. Involving students
Assessment is not something we do to students it is something we do with students.

28. Timing and Placement of Assignments
Do we consider student movement through curriculum in terms of the placement of signature assignments? Do we also consider transfer points and convergence between general education and the major?

29. Communication
Communication and collaboration with students and other audiences through transparent discussions around the outcomes and educational system works to make the implicit explicit. Communication involves exploration and integration with advising, alternative transcripts, admissions, and employers.
But if they don’t know about the design – then what’s the point? The ones that would have been successful and figure it out will but what about the others – if we really want every student that steps in for whatever period of time to learn something, then we need to tell them about it
Advising is a big one here

30. Are students even aware?
Student focus groups reveal that most students are not even aware they are being assessed. But upon learning about assessment…wish they had been told about it earlier.

31. Transparency in Assignments
Transparency in Teaching and Learning:
Purpose
Skills you’ll practice by doing this assignment
Content knowledge you’ll gain from doing this assignment
How you can use these in your life beyond the context of this course, in and beyond college
Task
What to do
How to do it (Are there recommended steps? What roadblocks/mistakes should you avoid?)
Criteria
(Are you on the right track? How to know you’re doing what’s expected?)
Annotated examples of successful work
(What’s good about these examples? Use the checklist to identify the successful parts.)

32. Rubrics
Do we share our rubrics or criteria with students and actively engage them in the review process?
Rubric Criteria
Student Evaluation
Faculty Feedback
Rubric Content
Stipulate why gave score did
Faculty stipulate why gave score did
Stipulate what they need to do to advance
Targeted feedback to improve
The case of the CIC project – the big ten schools

33. Questions and discussion