1. Assignment Design Workshop: Best of the Transparency Frameworks
Jillian Kinzie and Mary-Ann Winkelmes  

3. Overview
PURPOSE:
Learn to apply research and best practices for promoting students’ learning equitably through assignment design
TASKS:
Review: summary of research findings
Apply: to your assignments; your institutional contexts
CRITERIA:
You’ll leave with
Understanding of their research
Strategies and tools for implementing on your campus

4. TILT Higher Ed an educational development and research project;
helps faculty to implement a transparent teaching framework that promotes college students' success.
The Project's activities include:
workshops for both faculty and students,
online surveys that help faculty to gather, share and promptly benefit from current data about students' learning by coordinating their efforts across disciplines, institutions and countries,
confidential reporting of survey results to faculty
collaborative research on students' learning experiences.

5. Equity of Access Equity of Experience
Underrep, First Gen, Low Income:
half as likely to complete in 4 years
High-achievement in HS can frustrate college success
Equity of Access Equity of Experience
Black, Hispanic, Native American and Pacific Islander students are just about half as likely to complete a four-year college degree as their White and Asian classmates (U.S. Department of Education January 2014). Completion rates for low-income students lag far behind those of students whose family incomes are above the bottom quartile (Tough 2014). And first generation college students are 51% less likely to graduate in four years than students whose parents completed college (Ishtani 2006). PREACHING TO CHOIR: HIPs
Well-prepared novices don’t think like experts
Gatekeepers stunt research

6. Your challenges Your strategies
Understanding/Preparation
little research experience
unclear assignments
don’t understand expectations
provide, discuss examples of good work
review relevance to real life
offer rubric
begin work in class
class time for questions/clarification
chance to redo work after feedback
Motivation
fear of failure
don’t’ see relevance of academic work
low self-esteem
low motivation
flexible formats (audio recording, writing, etc.)
feedback at frequent check-in points
onsite and online follow-ups
rewards, extra credit
Time Management
planning and starting early enough
allowing enough time
conflicting commitments (work, family)
procrastination
break work into smaller pieces
regular check-ins, reminders
process maps, calendar plans

7. How we know Transparent Assignments boost student success:
How we know Transparent Assignments boost student success: (Peer Review, 2016)
AAC&U and TILT Higher Ed collaboration
7 MSIs, 1800 students, 35 faculty
425 First generation students
402 non-white students
479 low-income students
297 multiracial students
2 x small teaching intervention (2 assignments)
Boosted students’ learning in 3 important ways (medium-large effect for underserved students):
Academic confidence
Sense of belonging
Skills valued most by employers
One-page research summary on page 10 of your handout contains the description I’m providing you here.
Increased GPA, retention

8. What is Transparency in Learning and Teaching?
Transparent teaching and learning methods explicitly focus on how and why students are learning course content in particular ways.
Transparent methods benefit students who are unfamiliar with college success strategies by explicating learning/teaching processes.
Greater benefits for underrepresented and first-generation students
Winkelmes. Liberal Education 99, 2 (Spring 2013)
Winkelmes et al. Peer Review 18, 1/2 (Winter/Spring 2016)

9. What does Transparent Assignment Design look like?
Faculty/Instructors agreed (in national study, 7 MSIs) to discuss with students in advance:
Purpose
Skills practiced long-term relevance to students’ lives
Knowledge gained connection to learning outcomes
Task
What students will do
How to do it (steps to follow, avoid)
Criteria for success
Checklist or rubric in advance so students can self-evaluate
What excellence looks like (annotated examples where students/faculty apply those criteria)
Winkelmes et al, Peer Review (Winter/Spring, 2016) }
Problem-
centered
Mary-Ann

10. Amount of Transparency
ES=.80
Employer-valued Skills*
Academic Confidence
ES=.50
Sense of Belonging
ES=.64
KEY: N: number of students responding |__|: one standard error
ES: effect size (Hedges’ G) Effect sizes of 0.25 standard deviations or larger are
“substantively important” (US Dept of Education WWC, 2014, p. 23).
Less Transparent: mean perceived transparency < 3.3/4
More Transparent: mean ≥ 3.3/4
* Hart Associates employer surveys, 2015, 2013.
For example, ethnically underrepresented (African-American) first-year college students who completed an exercise that aimed to increase their feelings of social belonging earned higher GPAs in the next three years, reduced the racial achievement gap, reduced their feelings of self-doubt, increased their confidence and were more likely to be in the top 25% of their college class (Walton Walton, Gregory M., and Geoffrey L. Cohen “A Brief Social-Belonging Intervention Improves Academic and Health Outcomes of Minority Students.” Science 331 (6023): pp
). In addition, struggling college students increased their test scores after endorsing the belief that intelligence is not fixed but rather malleable. One year later, these students were 80% less likely to drop out of college, and their GPAs continued increasing (Aronson, J., C. Fried, and C. Good “Reducing the Effects of Stereotype Threat on African American College Students by Shaping Theories of Intelligence.” Journal of Experimental Social Psychology 38: 113–125.). For both white and African American first-year college students, sense of belonging can indirectly increase students’ persistence behaviors (Hausmann, Leslie R. M., Feifei Ye, Janet Ward Schofield and Rochelle L Woods “Sense of Belonging and Persistence in White and African American First-Year Students. Research in Higher Education 50, 7: ). Hart Research Associates Falling Short? College Learning and Career Success. Washington, DC: AAC&U.
____ ____ It Takes More than a Major: Employer Priorities for College Learning and Student Success. Washington, DC: AAC&U.

13. Impact on students’ views of learning

14. Implications for Transparent Design
Where does Transparent Assignment Design Come From?
Research on Learning
Implications for Transparent Design
Elbow, Jaschik/Davidson, Mazur
Ambrose, Bergstahler
Gregorc, Kolb
PURPOSE: Low stakes for greater creativity / risk
Varied and/or flexible formats appeal equitably to students’ strengths; inclusive
AAC&U HIPs, Bass, Bloom, Colomb, Felder, Perry
PURPOSE: Build critical thinking skills in sequence.
Target feedback to phase, don’t overwhelm
Doyle, Felder, Tanner, Winkelmes
PURPOSE: Specify knowledge/skills, criteria and encourage self-monitoring
Fisk/Light, Tanner
TASK: Provide annotated examples of successful work w/criteria applied, before students begin work 4
Aronson, Dweck, Fisk, Light, Schnabel, Spitzer, Steele, Treisman, Yeager/Walton, Vygosky
TASK: Structure and require peer instruction, feedback; positive attribution activities
Finley/McNair
Winkelmes et al
Yeager, Walton
CRITERIA: Explicate purposes, tasks, criteria in advance.
Give students a compass, set expectations;
Explicate applicability, relevance; Engage students in applying shared criteria to increase belonging

15. Assignments to Elicit Outcomes
How an assignment is designed to elicit an example of student demonstration of the outcome in question is important.
Scaffolding of outcome in a course or across courses
Number of outcomes addressed in an assignment
Assignment directions to students – is the outcome implicit or explicit?
Preparation – how are we preparing students to complete the assignment?

16. How do we know that we have mapped assignments to rubric criteria?
Alignment
How do you ensure alignment between assignments and a given learning outcome for a learning experience?
How do we create assignments and activities that elicit student demonstration of a specific learning outcome?
How do we know that we have mapped assignments to rubric criteria?
Where do students engage with these outcomes across a curriculum or collegiate experience and how are we capturing that learning?
Does this assignment or course introduce/reinforce/demonstrate mastery?
Where will it be practiced later? Is there a chance to try again based on feedback?
How does the assignment relate to evaluation criteria (rubric or otherwise)?

17. Desirable Characteristics of Assignments
Form:
Simple and easily understood
Focused, with minimum distractions from the main task
Contains appropriate information needed to frame a good response
Does not address too many DQP proficiencies
Appropriate balance between DQP and course/discipline outcomes
Hutchings, P., Jankowski, N. A., & Ewell, P. T. (2014). Catalyzing assignment design activity on your campus: Lessons from NILOA’s assignment library initiative. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).

18. Desirable Characteristics of Assignments
Content:
Engages student interest and supports learning
Helps student see underlying structure of the problem
Reflects the actual learning experiences that students have had
Provides opportunities for small successes within the main task (e.g. for partial credit)
Provides opportunities for correction after feedback
Is unbiased with respect to student backgrounds and circumstances
Allows originality in response
Hutchings, P., Jankowski, N. A., & Ewell, P. T. (2014). Catalyzing assignment design activity on your campus: Lessons from NILOA’s assignment library initiative. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).

19. Desirable Characteristics of Assignments
Level of Challenge:
Is appropriately located on a developmental continuum
Contains scaffolding appropriate to the level of challenge
Might be a series of related assignments with ascending levels of challenge as the student progresses through them
Can determine what level of performance signifies mastery of the proficiency
Hutchings, P., Jankowski, N. A., & Ewell, P. T. (2014). Catalyzing assignment design activity on your campus: Lessons from NILOA’s assignment library initiative. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).

20. Apply all that research to designing transparent, equitable assignments!

21. Sample Assignments Sample A (p. 7)
Purpose
Skills practiced long-term (problem-centered) relevance to students’ lives
Knowledge gained connection to learning outcomes
Task: What to do
How to do it (steps to follow, avoid)
Criteria
Checklist or rubric in advance to help students to self-evaluate
What excellence looks like (multiple annotated examples) }

22. Sample Assignments Sample B (p. 7)}
Purpose
Skills practiced relevance to students
Knowledge gained connection to LOs
Task: What to do
How to do it
Criteria
What excellence looks like (annotated)
Criteria in advance to help students to self-evaluate

23. Sample Assignments Sample C (p. 8)}
Purpose
Skills practiced relevance to students
Knowledge gained connection to LOs
Task: What to do; How to do it
Criteria
What excellence looks like (annotated)
Criteria in advance to help students to self-evaluate
It’s A, revised

24. Sample Assignments Sample D (p. 8)}
Purpose
Skills practiced relevance to students
Knowledge gained connection to LOs
Task (problem-based, relevant)
What to do; How to do it
Criteria
What excellence looks like (annotated)
Criteria in advance to help students to self-evaluate
Compare to B

25. Break

26. Gather Feedback on Your Own Assignment
Why are we doing this now?
Purpose
Knowledge: share feedback, insights;
promote student success
Skills: apply transparency; engage community of practice
Task
Four steps, 2-4 min each, in pairs / 3s
Criteria
draft you can use in your course
helpful insights from colleagues as novices

27. Set up
Sit with a disciplinary stranger

28. Describe an Assignment from Your Course
from 1st half of the term
after students are acquainted with basic tools and terminology the course uses
when students are starting to apply those and try them out
Describe this assignment to your partner(s)
(3 min each) [12 total]

29. Feedback on Your Assignments, part 1 of 3
As a novice student, offer feedback on the Purpose
(4 min each; 12 min total)
Five years after taking your course,
What essential knowledge should students retain from doing this assignment?
What skills should students be able to perform from doing this assignment? (p. 2 may help)
Why are these important to students?

30. Train the trainers: What Happened?
Multiple skills?
Complex skills?
Potential for smaller, scaffolded assignments?
Did “teachers” spend time “correcting” or revising?
How to prepare for your faculty?

31. Feedback on Your Assignments, part 2 of 3 handout page 6
As a novice student, offer feedback on the Task
(2 min each; 6 total)
As a novice, list the steps you’d take to do the assignment.

32. Train the trainers: What Happened?
Did “students” identify steps out of order?
Did “students” identify unanticipated (good) steps?
Did “teachers” spend time “correcting” or revising?
How to prepare for your faculty?

33. Feedback on Your Assignments, part 3 of 3 handout page 6
As a novice student, offer feedback on the Criteria
In groups, discuss and define (3 min each; 9 total)
As a novice:
Are you confident you are doing the task effectively?
Are you confident you are doing excellent work?
Do you have annotated good examples?
To answer yes, what would you need?

34. Train the trainers: What Happened?
Do students need things that teachers don’t have?
Time-efficient ways to accommodate?
How to prepare for your faculty?

35. Additional Research-based Strategies
Offer feedback in groups, (3 min each; 9 total)
Which additional research-based methods
could be used (TILT research review chart)?
Which Desirable Characteristics (NILOA) could be added?

36. Courses/Programs: Units: Networks:
Tables: How can TILT/NILOA assignment resources help you boost student success in these contexts?
Courses/Programs:  
Intro (large,small), Gateway, High DFW, Majors/Pathways, Gen Ed
Units:
collaboration among student support, faculty affairs
Networks:
MSIs, Community Colleges, Research Intensive, Liberal Arts, Regional, Teaching/Learning Centers, STEM Ed Ctrs, Disciplinary Association conferences...

37. What would you need to try this as a team/group?
materials and resources (online)
NILOA assignment library
TILT materials
organizational structure
onsite and online seminars
NILOA; TILT Transparent 2nd Tuesdays
TILT online surveys, analysis, reporting
NILOA assignment library comments/feedback
other?

38. What would you need to try this as an individual/
Instructor Participants:
materials and resources (online)
NILOA assignment library
TILT materials
TILT online signup
Researchers / Publishers:
Publish an assignment, NILOA Assignment Library
Join TILT research team:
complete CITI human subjects certification
identify research interests, join a team
access TILT database, analyze
publish

39. Sign up for online Updates
NILOA
TILT

40. Train the trainers: How did we do?
Purpose
Knowledge: share feedback, insights
Skills: apply transparency; engage community of practice
Task
Four steps, min each, in pairs / 3s
Criteria
draft you can use in your course
helpful insights from colleagues as novices
Strategies
Mary-Ann