1. Current R&D at the NTID Center on Access Technology E. William Clymer November 18, 2010 1

2. Center on Access Technology www.rit.edu/ntid/cat www.rit.edu/ntid/cat Rochester Institute of Technology National Technical Institute for the Deaf 52 Lomb Memorial Drive Rochester, NY 14623 2

3. Recent & Ongoing Access Technology Projects NSF Enrichment CAT Innovation Lab and Director PDA Projects Cisco Grant – Three Strands Cisco Evaluation – Desktop Systems NTID Technology Symposium 3

4. NTID’s Location Image: Map of Northeast USA showing Rochester, NY and RIT and NTID campus drawings. 4 The RIT Campus Lyndon Baines Johnson Building (LBJ) Located in Rochester, New York

5. 5 College of Engineering College of Liberal Arts College of Imaging Arts & Sciences College of Applied Science & Technology College of Science College of Business National Technical Institute for the Deaf CAST COB COE CIAS CLA COS NTID  The Eight Colleges of RIT  ≈17,000 Students  The Eight Colleges of RIT  ≈17,000 Students GCCIS Golisano College of Computer & Information Sciences Rochester Institute of Technology

6. 6 CAST COBCOE CIAS CLA COS NTID Supports and Educates 1,300 Deaf Students Rochester Institute of Technology

7. 7 CAST COBCOE CIAS CLA COS NTID Supports and Educates 1,300 Deaf Students Bachelors & Masters Degrees 454 Students (41%) Transfer Programs 118 Students (11%)

8. 8 College of Engineering College of Liberal Arts College of Imaging Arts & Sciences College of Applied Science & Technology College of Science College of Business National Technical Institute for the Deaf CAST COB COE CIAS CLA COS NTID GCCIS Golisano College of Computer & Information Sciences Direct Service Model NTID Teachers Sign Teach Students Directly Teachers Sign Teach Students Directly Rochester Institute of Technology

9. 9 CAST COB COE CIAS CLA COS NTID GCCIS Support Model For Bachelor’s and Master’s Degree Students Access Services Provided Academic Support Provided Access Services Provided Academic Support Provided Rochester Institute of Technology

10. Access Services Interpreting Captioning (C-Print) Notetaking Tutoring Counseling 10

11. Interpreting Services  Interpreting services provided by: – 122 FTE staff interpreters (October 2010) – Freelance interpreters – Deaf interpreters for Deaf-blind students – Students from interpreter training program 11

12. Classroom Interpreting 12

13. Notetaking  Notetaker coordinators  Trained student notetakers (paid)  Notes are scanned to the web  Support faculty evaluate the content matter of the notes 13

14. Classroom Notetaking 14

15. Web Distribution of Notes 15 Classes Scanning Center Web Site

16. Captioning Captioning Options – Speech to Text Systems – C-Print – CART 50 C-Print operators at RIT CART is outsourced http://www.pepnet.org/fa q/pdf/FAQ04.pdf 16

17. Access Services http://myaccess.rit.edu 17

18. Course Listings Interpreter, Notes and C-print 18

19. Manager 19

20. Interpreter Schedule 20

21. Student Download Notes and Captions 21

22. C-Print Captions File 22

23. Example of Notes and Captions 23

24. Mainstreamed Students & Support Services Hours of Service #StudentsInterpretingNotetaking Academic SupportC-Print AY 2008-09635+110,00063,29017,63617,330 24

25. Center on Access Technology www.rit.edu/ntid/cat www.rit.edu/ntid/cat Rochester Institute of Technology National Technical Institute for the Deaf 52 Lomb Memorial Drive Rochester, NY 14623 25

26. Mission To investigate, evaluate, and report on the most effective and efficient use of access technologies To train individuals in their use in order to accelerate the widespread implementation of best practices within deaf education at the postsecondary level. To focus its efforts on technologies that have a high likelihood of improving access to postsecondary educational opportunities for deaf students within the next several years. Four strands include: – Classroom Access Technologies – Mobile Technologies – Audio and Sound Technologies of Interest to Hard-of-Hearing Persons – Training and Evaluation Services 26

27. Mission The Center seeks projects that fall within the following parameters: – Those that can adapt/ adopt existing technologies – Those that utilize existing professional networks – Those that involve education and training – Those that bring together faculty, staff, students and others at NTID, RIT and external groups 27

28. NSF Enrichment Grant Testing the concept of a virtual Alliance for deaf and hard of hearing STEM students at the postsecondary level 28

29. Specifics NSF Award # HRD-0927586 18 Months – 9/1/2010 through 2/28/2011 $199,585 http://www.rit.edu/ntid/cat/enrichment 29

30. Team Bill Clymer Jim DeCaro Peter Lalley Gary Behm Gail Hyde Elizabeth Lawley Walter, Foster McKee Lang, Brooks, Ladner, Gan 30

31. Goal Prepare for an NSF Alliance grant proposal in the amount of $3M due Feb. 2, 2011. Purpose of an Alliance…. Alliances use evidence-based practices, as well as beta-testing novel interventions, to increase the quality and quantity of students with disabilities completing associate, baccalaureate and graduate degrees in STEM disciplines and entering either STEM graduate degree programs or our nation’s science and engineering workforce. 31

32. Concept Use cyberinfrastructure to establish a “virtual” RDE regional alliance to provide student support to deaf and hard of hearing STEM students. – Communication Support Interpreting and captioning – Academic Support Tutoring Notetaking – Advising and mentoring 32

33. Communication Support Academic Community 33

34. Specifically this grant will… Examine national baseline data on deaf students in STEM programs Analyze programs that have deaf students enrolled in STEM programs Conduct 4 focus groups Indentify 4-5 postsecondary institutions as potential partners Identify elements of successful Alliances Develop and pilot rudimentary cyberinfrastructure system for a virtual alliance 34

35. Expected Work Products Summary report of baseline data Annotated list of programs Recommendation report from focus groups List of 4 or 5 postsecondary programs to participate Analysis and summary of features, format, participants and budget for a virtual alliance Target plans for an alliance Functional model of a web-based application for communication and academic support 35

36. Time Line Year 1 (12 Months) – Project website – Data search – D/HOH STEM students – Focus groups – PEPNet – Review best practices – Review platforms for delivery of services 36

37. Time Line Year 2 (6 Months) – Identify potential partners – Social network (academic community) prototype – Dissemination – Write and submit Alliance proposal 37

38. Evaluation Plan Dr. Barbara McKee Formative evaluation – Timely and accurate completion of project goals Compliance with RIT and NSF reporting guidelines Summative evaluation 38

39. Dissemination Web site – http://www.rit.edu/ntid/cat/enrichment http://www.rit.edu/ntid/cat/enrichment National presentations – NTID Techsym (June 2010) – NSF JAM (June 2010) – Accessing Higher Ground (November 2010) 39

40. Budget Original Budget Percentage Spent – 51.4% One year no-cost extension? 40 AwardedSpent Personnel98,87746,069 Benefits23,51512,988 Travel6,4001,788 Materials, etc13,2612,439

41. CAT Innovation Laboratory The NTID Center on Access Technology Innovation Laboratory ("CAT Lab"), a first-of-its-kind initiative, provides a place for students to get involved in the innovation process. The CAT Lab is a place where faculty and associate degree-level deaf and hard-of-hearing students can collaborate on multidisciplinary projects related to developing and adapting access and instructional technologies, health care technologies and services, and more. 41

42. Personnel James J. DeCaro – Director of CAT E. William Clymer – Associate Director of CAT Peter A. Lalley – Researcher / Investigator Gary W. Behm – Director of CAT Innovation Lab Gail Hyde – Project Manager 42

43. Types of Projects Funded – Internally (RIT or NTID) – Externally Proposed to funding agencies Concept 43

44. PDA / Cell Phone Notification System Funded – Internally (NTID) NamesRoles Dr. Jim DeCaroProject Sponsor / Owner Bill ClymerSupervisor Gary BehmEngineering Project Manager Wendy DannelsEngineering Subject Matter Expert Joe StanislowEngineering Subject Matter Expert Cory BehmEngineering Design Assistant Lead Matthew PoeEngineering Design Assistant 44

45. 45 Testing the Concept of a Virtual Alliance for Postsecondary Level STEM Students who are Deaf and Hard-of-Hearing  NTID Center on Access Technology  PEPNet  Lab for Social Computing at RIT  Other NSF-supported regional alliances Funded – National Science Foundation RolesNames PIBill Clymer Co-PIGary Behm Co-PIPeter Lalley Project ManagerGail Hyde NSF Grant Partners

46. Improving Distance Learning for Deaf Students by use of a 3D Signing Avatar  NTID Center on Access Technology  RIT Computer Science Department  NTID Department of Engineering Studies  Other NSF-supported alliances Proposed – National Science Foundation RolesNames PIJoe Geigel Co-PIGary Behm Project ManagerGail Hyde NSF grant partners Whiteboard Instructor Deaf Student Sign Language Interpreter 3D Signing Avatar Hearing Student Whiteboard 3D Avatar Conferencing Service Undivided Attention 46

47. Developing an Innovative See-Through Face Mask for More Effective Communication  NTID Center on Access Technology  NTID Department of Engineering Studies  Department of Chemistry in the College of Science  Packaging Science in College of Applied Science and Technology  Rochester General Health System (RGHS)  RIT IP Management  IBM Proposed – National Science Foundation RolesNames PIJim DeCaro Co-PIGary Behm Co-PIChangfeng Ge Co-PIMassoud Miri Project ManagerGail Hyde NSF grant partners IBM has agreed to assign this patent to RIT 47

48. Digital Image Relocation Device to Assist Individuals with Low Vision Proposed – NTID, RIT/RGHS Alliance, NSF NSF Grant Partners Co-PI - Gary BehmProject Lead / Developer NTID Center on Access Technology Project Mgr. - Gail HydeProject ManagementNTID Center on Access Technology - Bill TravisConsultant End User PI - Dr. Andreas SavakisDeveloper RIT Department of Computer Engineering - Dr. Andreas Savakis’ TeamDeveloper RIT Department of Computer Engineering Co-PI - Dr. Gwen SternsOphthalmologist RGH Ophthalmology’s Laboratory - Dr. Gwen Sterns’ TeamOphthalmologists RGH Ophthalmology’s Laboratory - Deaf Co-op StudentSoftware Engineering NTID Department of Engineering Studies - Deaf Co-op StudentDesign Assistant NTID Department of Engineering Studies 48

49. Preliminary Investigation of Cisco Technologies and Access Solutions for Deaf and Hard-of-Hearing Individuals State of the Art and Recommendations Related to 911-411-211 Telephone Response Systems Evaluation of Possible Use of Avatars to Enhance Direct Communication Support for Deaf and Hard- of-Hearing Users Evaluation of TelePresence Technologies for Face-to-Face and Remote Communication for Deaf and Hard-of-Hearing Users This project is the start of a multi-year effort. Funded – Cisco Foundation  NTID Center on Access Technology  Cisco Foundation  NTID Department of Engineering Studies  RIT Golisano College of Computing & Information Studies Partners 49

50. A Investigation of Cisco Technologies & Access Solutions 50

51. Cisco and RIT/NTID Cisco has supported RIT Information Science and computer programs Jim Ebenhoch and NTID past president Hurwitz visited Cisco, with introductions to Cisco’s Accessibility, Compliance and Certification office by Len Mudrock ’84 who is a senior software engineer at Cisco 51

52. Proposal Three strands One year effort Establish our understanding and areas where we can contribute to Cisco product development Development of position papers and a call for further research Deaf perspective 52

53. Strands of R&D Provide Cisco with a deaf perspective on their products and the services they support Three strands – 911-411-211 Communication – Avatars – TelePresence 53

54. Award From the Silicon Valley Community Foundation, following the recommendation of the Cisco Accessibility Team To the NTID Center on Access first year of what is anticipated to be a multi-year effort Approximately $65,000 cash, and $35,000 equipment 54

55. But… TelePresence equipment donation was upped to the equivalent of $300,000! – One 65” HD screen – Three 65” HD screens – System installation and furniture 55

56. General Structure of Work Form teams of RIT/NTID, local and national experts Conduct a literature review to understand issues Produce “White Paper” and disseminate for comment Convene meeting(s) and focus groups Revise “White Paper” Produce final recommendations 56

57. Web-based Resources http://www.rit.edu/ntid/cat/cisco 57

58. 911-411-211 Communication Project Team – Bill Clymer, Project Leader – Gail Hyde, Project Coordinator – Kelly Masters, Focus Group and Evaluation RIT/NTID experts External experts Cisco experts 58

59. Issues Access to Emergency Information and Services through 9-1-1 emergency public safety answering points (PSAP’s) through receipt of text and video Identify barriers and offer solutions Next generation 9-1-1 communication Recognize the “community” of interested persons Fail safe technology that emergency responders, telecommunication companies, equipment companies and users can accept 59

60. Major Tasks Develop set of issues from literature Convene focus groups at RIT – Policy – Technology – Users Literature review & “White paper” Post and collect comments Review and comment by experts Publish final report 60

61. Signing Avatar Project Team – Joe Geigel, Project Co-­‐Leader – Gary Behm, Project Co-­‐Leader – Gail Hyde/Bill Clymer, Project Coordination – Kelly Masters, Focus Group Expert and Writer RIT/NTID experts External experts Cisco experts 61

62. Signing Avatar Goal is to explore: – Possibility of signing avatars, guided by voice-to-text or other input processes, to be used effectively for signing communications support (over videophone, TelePresence or conferencing systems) – Similar to what is provided by live, professional sign language interpreters. 62

63. Signing Avatar An avatar is a computer representation of a person in the form of a three- dimensional model used in virtual space. Signing avatar – use of a 3D avatar for displaying signed messages. http://www.vcom3d.com/vault_files/making_forest_asl/ 63

64. Applications Automated sign language translation in public spaces Education Remote sign language support – Virtual spaces – Teleconferencing http://www.visicast.co.uk/demo/tessa/tessa_demo.htm 64

65. Idealized Signing Avatar System Speech to text Text to SL SL to gestures Motion capture Live interpreter audio textSign language gestures AVATAR 65

66. Expertise Required Sign Language / Linguistics Current signing Avatar systems Speech to text 3D Graphics / Animation – Motion capture (body and hand) – Facial analysis, modeling, and animation Deaf communities – Cyber/virtual – Educational 66

67. Deliverables “White Paper” – State of the art – Future possibilities – Identify key questions and challenges – Recommendations Focus Group Meeting – Discussion among experts in areas previously listed – Refine “White Paper” based on discussions 67

68. TelePresence Evaluation Goal is to explore: – Possibilities and applications of TelePresence to support communication: in instructional, laboratory and social environments for students who are deaf or hard-of-hearing. of closed caption, presentation, signing avatars, social networking, and others. 68

69. TelePresence Telepresence refers to a set of technologies which allows a person to feel as if they were present, to give the appearance that they were present, or to have an effect, at a location other than their true location. TelePresence makes remote or distant people appear or feel present and part of the local activity by using telematics technology. http://matthewwall.typepad.com/.a/6a00d8341c68d853ef0111 68a477f0970c-800wi 69

70. Applications Education – Classroom and Laboratory Direct Indirect – Tutoring / Mentoring Remotely (different colleges) – Job Interview Remotely Social – Networking Communication Support – Remote Sign language virtual spaces – Teleconferencing – Closed Captions – Others? 70

71. TelePresence Network System 71

72. Expertise Required Instructors for Deaf / Hard-of-Hearing people Deaf / Hard-of-Hearing Students (direct and indirect) Sign Language / Linguistics 3D Graphics / Animation / Avatar – Motion capture – Facial analysis, modeling, and animation Communication Support – Closed captions / presentation – Other Deaf Communities – Cyber / virtual – Educational 72

73. Deliverables White Paper – State of the art – Future possibilities – Key questions and challenges identified – Recommendations Focus Group Meeting – Discussion among experts – White paper refined, based on discussions 73

74. Team Gary Behm, NTID Wendy Dannels, NTID Kelly Masters, outside consultant Gail Hyde / Bill Clymer, NTID 74

75. Instructional Technology Symposium http://www.rit.edu/ntid/vp/techsym/ June 21-23, 2010 in Rochester 75