Universal Design for Evaluation Summit on VR Program Evaluation & Quality Assurance September 16, 2013 Jennifer Sullivan Sulewski, Ph.D Institute for Community Inclusion University of Massachusetts, Boston June Gothberg, Ph.D Michigan Center for Educational Networking Background: UDE, UDE checklist, myself & June.

IMAGE: Jennifer and June at the American Evaluation Association conference, holding a sign saying “I will be responsive to culture in my evaluation work by… using Universal Design for Evaluation.” Our work started from a panel Jen organized in 2009 Past Co-Chairs of the Disabilities and Other Vulnerable Population TIG at the American Evaluation Association.

Introductions Name Affiliation One thing you are hoping to get from today

What is Universal Design? Universal Design (UD) is the design of products and environments to be usable by all people, to the greatest extent possible, without the need for adaptation or specialized design. IMAGE: silhouettes of people including a wheelchair user, someone pushing a baby stroller, someone dragging a suitcase, a pregnant woman. Center for Universal Design, North Carolina State University

“When you include the extremes of everybody, that’s to say differently abled people of all sorts, then you produce things that are better for all of us” ~ Michael Wolff UD improves the experience across the broadest range of users. Why Universal Design? IMAGE: several photos in a collage, including several of people, a baby stroller, a guide dog, a person wearing a hearing aid, a person using a wheelchair, a person using a cane. ©2010 Norsk Designråd DOGA

Designed to be Useable by ‘All’ Velcro Electric toothbrush Single tap facets Flexible drinking straws Audio books Automatic doors Q-drums IMAGES: five images in a vertical column. Velcro shoes, electric toothbrush, single tap faucet, flexible drinking straw, Q-drum (a rolling drum for carrying/dragging water) http://www.optimastudio.com/disenosparatodos/en/index.php

Principles of Universal Design Equitable use Flexibility in use Simple and intuitive use Perceptible information Tolerance for error Low physical effort Size and space for approach and use Center for Universal Design, North Carolina State University

Principle One: Equitable Use Image: cartoon of someone shoveling stairs while the ramp is still covered in snow. Kid in wheelchair: “Could you please shovel the ramp?” Janitor: “All these other kids are waiting to use the stairs. When I get through shoveling them off, then I will clear the ramp for you.” Kid: “But if you shovel the ramp, we can all get in!” The design is useful and marketable to people with diverse abilities.

Consent process - need for consent from guardians for those under guardianship - present information verbally as well as on paper - process of negotiation with the IRB IMAGE: consent form for “Making the Transition to Work” project.

Principle Two: Flexibility in Use The design accommodates a wide range of individual preferences and abilities. IMAGE: staircase with integrated ramp.

Multiple methods and sources State Local provider agency Local provider agency Local provider agency Individual Individual Individual Individual Individual Individual Case study approach allows for inclusion of multiple perspectives - people with & without disabilities - embedded to create a story Individual Individual Individual

Principle Three: Simple & Intuitive As we show you examples of the seven principles, please jot down notes to how they might apply to your evaluation or research project. Use of the design is easy to understand, regardless of the user's experience,knowledge, language skills, or current concentration level. Eliminate unnecessary complexity. IMAGE: cup of instant noodles with text & pictorial cooking directions.

Different survey versions for different audiences IMAGE: Post-training survey form for parents participating in a transition training.

Different survey versions for different audiences IMAGE: simpler post-training survey for students.

Principle Four: Perceptible Information The design communicates necessary information effectively to the user,regardless of ambient conditions or the user's sensory abilities. IMAGE: pictorial restroom sign.

IMAGE: “Real People Real Jobs” website with text & video employment success stories. http://www.realworkstories.org/home

Principle Five: Tolerance for Error The design minimizes hazards and the adverse consequences of accidental or unintended actions IMAGE: “Undo” button on a computer software menu

Use of probes to clarify questions Thinking about the past year, what three issues do you think are at the top of the policy agenda for DDS? PROBE: What are three things that DDS cares about most? PROBE: They say they care about (insert example here) – do you agree with that?

Principle Six: Low Physical Effort As we show you examples of the seven principles, please jot down notes to how they might apply to your evaluation or research project. The design can be used efficiently and comfortably, and with a minimum of fatigue. IMAGE: Young boy dragging a Q-drum, a rolling, donut-shaped water drum. ©Pieter Hendrikse www.optimastudio.com/disenosparatodos/en/

IMAGE: public transit bus on a city street. http://www.fotopedia.com/items/flickr-2320045609

Principle Seven: Size and Space for Approach and Use Appropriate size and space is provided for approach, reach, manipulation, and use regardless of user's body size, posture, or mobility. IMAGE: Man in a wheelchair working in an accessible kitchen space. http://www. foreveractivemed.com

IMAGE: woman in a wheelchair at a meeting.

Meet people where they are at. IMAGE: winding road through a forest. flickr.com/photos/robotbrainz/2535755932/

Thoughts or questions? IMAGE: Silhouette of a manwith a question mark over his head.

Discussion Questions

What, if anything, was new information in the introduction to Universal Design for Evaluation?

In your evaluation work, what steps have you taken to ensure inclusion of a diverse range of participants?

What lessons have you learned in working with people with disabilities and other vulnerable populations? What tips would you like to share with the group?

Jennifer Sullivan Sulewski Institute for Community Inclusion University of Massachusetts, Boston (617) 287-4356 Jennifer.sulewski@umb.edu