Using the Web for Bilingual/Bicultural Education of Deaf Children Sonia Martinez, Vicki Hanson & Susan Crayne IBM T. J. Watson Research Center New York, USA

HandsOn Project The HandsOn project sought to provide schools with computer technology to support bilingual/bicultural programs. HandsOn II a Web based application Project purpose is to use the 1 st language (stronger language) to help in learning a 2 nd language.

HandsOn The original HandsOn program used laser disc technology to present full screen, real people signing ASL Goals: –Develop a research instrument to investigate how deaf children used ASL and English cooperatively –Provide schools with computer technology that supported bilingual/bicultural programs for signing children Story based activities allowed students to interact with both languages ASL and English presented simultaneously on one monitor Sonia: Handson consisted of Story based activites Sonia: Handson consisted of Story based activites

HandsOn Activities Sonia: Handson consisted of 4 activities Sonia: Handson consisted of 4 activities Play a StoryRead a Story Caption a StoryWrite a Story

ASL Translation to English Stories translated from ASL to English Translations at a sentence level, not word-for-word translations to capture grammar and avoid “dictionary” approaches to language learning. ASL signs are dependent on the English context For Example: GIVE Dictionary Definition : GIVE (TO YOU. HIM, ME, ALL OF YOU) GIVE (GIFT) The stories used in HandsOn where translated between ASL and English Translations where done at a sentence level, not on a word for word basis. We wanted to avoid dictionary approaches to language learning. Since ASL is dependant on the English context. For ex. English word give, movement of sign depends on the meaning of the word or who is giving and receiving. The stories used in HandsOn where translated between ASL and English Translations where done at a sentence level, not on a word for word basis. We wanted to avoid dictionary approaches to language learning. Since ASL is dependant on the English context. For ex. English word give, movement of sign depends on the meaning of the word or who is giving and receiving.

HandsOn

Sonia: This graph represents three schools and the percentage of time the schools spent on each activity. We can see that most of the time was spent on the Read and Caption activity. Sonia: This graph represents three schools and the percentage of time the schools spent on each activity. We can see that most of the time was spent on the Read and Caption activity.

Language Choices Used Play a Story –Mostly ASL, deaf children rarely asked for English text Read a Story –Frequently asked for ASL, varied by the level of reading ability Sonia: We observed that in play a story, ASL was mostly used. In Read a story it depended on the reading level of the child, although ASL was usually used. Sonia: We observed that in play a story, ASL was mostly used. In Read a story it depended on the reading level of the child, although ASL was usually used.

Conclusions Used ASL (stronger language) to help with difficult English text. An unevenness in the use of the two languages. The interactivity afforded by computers motivated students to engage in the reading and writing tasks Sonia: We concluded that students showed an unevenness…. Sonia: We concluded that students showed an unevenness….

Comments from Students “I understood the ASL” “The woman signing helped me a lot” “It helped me read better” “I liked writing English from ASL signs” “It helped me learn ASL” “I would like more stories” “I would like to make it smaller and make it simple to take other places so I can play on it in the car.”

Comments from Teachers “It has been a good tool for us as teachers to sharpen our ASL skills.” “HandsOn impacts many areas of the curriculum at the same time. Students have fun while they are watching, reading, or captioning a story. The fact that they are learning about science or social studies subjects at the same time is an added bonus.” “Students complain when they have to stop.” “The biggest strength of HandsOn is the ability to foster improved self-esteem by working with materials designed especially for deaf students.” “HandsOn directly engages and motivates students to try something most do not enjoy doing, such as writing.”

Address Concerns Will children attempt to “write ASL” when captioning ASL sentences? ASL: True, happen 1956 Will children attempt to “write English” when captioning ASL sentences? English: It happened in 1956 Sonia: We wanted to address the criticism of children writing ASL rather than correct English. Sonia: We wanted to address the criticism of children writing ASL rather than correct English. OR

Observations from Story Captioning Sentences were longer and more grammatically correct –Improved morphology –More inclusion of articles –Improved use of function words –More complex English sentences Results attributed to: –Lessening of cognitive demand –Visual memory support necessary to organize ideas while writing English (Hanson & Padden, 1990; Mozzer-Mather, 1990; Kelly et al, 1994) There were 2 studies done which concluded that when captioning ASL, longer and more grammatically correct sentences were created.

Updating HandsOn The original HandsOn project employed videodisc technology, which is no longer generally used The Web-based version called HandsOn II has the potential to reach many more users Video Streaming techniques used on the Web –Real ASL signing –Ideal for language learning –Limited download time, signer presented in a window rather than full screen As technologies advanced, the original HandsOn application that used videodisc technology was outdated. It was limited in the audience it could reach. Therefore we turned to the web. Current version of handson uses video streaming technologies to present ASL stories over the web.

HandsOn II Read a Story Activity Sonia: A signer is showned in a window on the web page. The signer and the story text are displayed on the same screen. A book metaphor is used in the application. Sonia: A signer is showned in a window on the web page. The signer and the story text are displayed on the same screen. A book metaphor is used in the application.

Creating English Translations ASL stories translated to English Synchronized translations from new digital video format Used our Authoring tool: –Indicate the beginning and ending of signed sentences on the video –Enter the English text translation for that video segment ASL: DO? JOIN DECIDE PLAN BOYCOTT English: They decided to plan a boycott. Here is an example of an English translation of ASL. We produced synchronized translations from the video format. We started with ASL stories and created their English translations using our own authoring tool which allows you to indicate the beginning and ending of a signed sentence taken from the video. Here is an example of an English translation of ASL. We produced synchronized translations from the video format. We started with ASL stories and created their English translations using our own authoring tool which allows you to indicate the beginning and ending of a signed sentence taken from the video.

HandsOn II Application From the main function menu, choose an activity: –Successful activities from previous HandsOn –Play a Story, Read a Story, Caption a Story A Java applet is embedded in each activity –Applet manages the video streaming –Applets work on both Mac and Windows computers Sonia: The activities in HandsOn II consist of the successful activities from the previous HandsOn version, PaS,RaS,CaS. Sonia: The activities in HandsOn II consist of the successful activities from the previous HandsOn version, PaS,RaS,CaS.

HandsOn II Architecture QuickTime is used for video streaming Two sizes of video tracks embedded in movie –Dial-up (164x136) –Broadband (220x180) Bigger Video