The effects of captions on deaf students’contents comprehension, cognitive load and motivation in online contents 21 June 2010 Joong-O Yoon Heaeun Choi Korea Employment Agency for the Disabled 2010 International Symposium

Table of Contents Ⅰ Introduction 1. Background of the study 2. Research questions Ⅱ Methods 1. Subjects 2. Experiment Design 3. Test materials 4. Measurements 5. Data analysis Ⅲ Results Ⅳ Conclusions and Discussion V Recommendations

Ⅰ. Introduction

1. Background of the study Linguistic deficiency Limitation in linguistic-information collection Limitation in spoken language ability Limitations in Verbal learning Challenges in learning Low literacy skills Text comprehension Cognitive development Restricted access to information & knowledge in learning environments Multimedia learning materials Visual learning aids (Luckner, 1992) Captions Sign language Video clips Deafness

Content comprehension? Cognitive load? Motivation? 2. Purpose of Study To examine the effects of captions when it is provided along with Sign language video clips in multimedia learning Online material with both captions and sign language video clips is effective for deaf students whose first language is sign language and literacy skills are limited ?

Having more visual tools in multimedia learning aids affects learners’ cognitive load negatively 2. Research questions When captions are incorporated into presentation, the amount of verbal information is greater and leads to improved content comprehension Having more visual tools in multimedia learning materials raises learners’ motivation in learning Do online learning contents with Sign language video clips and captions result in increase of content comprehension / cognitive load / motivation compared to learning contents only provides Sign language clips?

Hypothesis 2. Research Hypothesis There will be a rise in content comprehension, cognitive load, and motivation when both captions and sign language video clips are provided at the same time in multimedia learning materials. - The experimental group(Sign language clips+Captions) will show a high score than the control group(exposed only to Sign language clips) in content comprehension, cognitive load, and motivation

Ⅱ. Methods

1. Subjects 62 deaf people - Age : (Avg. 25) - Gender : 40 male, 22 female - Schooling : 58 high school graduates, 4 college graduates 62 deaf people - Age : (Avg. 25) - Gender : 40 male, 22 female - Schooling : 58 high school graduates, 4 college graduates

2. Research design Caption using/ not using Caption using/ not using Content Comprehension Independent variable Cognitive load Motivation Posttest-only control group design Dependent variable Deaf people

Randomly assignment process for all deaf participants Randomly assignment process for all deaf participants Divide into two groups TreatmentPost Control group (Sign language only) RO Experiment group (Sign language+captions) RХO Dependent : Content Comprehension, Cognitive load, Motivation 3. Experimental Design As for the experimentThe subjects were

4. Test Materials (Online contents) - Subject Content : Investment techniques Sign language ( Displayed all the time ) Sign language ( Displayed all the time ) Captions (On / Off) Captions (On / Off)

4. Measurements Written test(10 questions, multiple-choice) Content comprehension Cognitive load Instruction Materials Motivation Survey (Keller, 1993) MotivationFeedbacks Individual in-depth interview(24 people) Learning Load Measurement Survey (Ryu, 2009) 5 sub-sections : Physical effort, mental effort, difficulty of perceived assignment, self-evaluation, and availability 4 sub-sections : Attention, Relevance, Confidence, Satisfaction To see the effects of the closed captions, 4 measurement tools were used

5. Data Analysis Computation on average and standard deviation of content comprehension, cognitive load, and motivation had been conducted. The difference between the two groups was analyzed using t-test and MANOVA at the.05 significance level. Computation on average and standard deviation of content comprehension, cognitive load, and motivation had been conducted. The difference between the two groups was analyzed using t-test and MANOVA at the.05 significance level.

Ⅲ. Results

1. Contents Comprehension groupNMSDdftp Control group (Sign language) *.035 Experimental group (Sign language+Captions) *p<.05 Notable statistical difference (t=-2.16, p<.05) between two groups Recorded.565(>.5) in effect size, Notable statistical difference (t=-2.16, p<.05) between two groups Recorded.565(>.5) in effect size, 7.1

2. Effects on Cognitive load Fp Caption using/not using Physical Effort Mental Effort Difficulty of Perceived assignment Self-evaluation Availability of Data structure No significant difference between two groups

3. Effects on Motivation Fp Caption using/not using Attention Relevance Confidence Satisfaction No significant difference between two groups

“The pace of the program was too fast” 12 3 “Some word to hard to understand in the captions but it helped” “ I looked at the captions only when I had to” 4. In-depth Interview

Ⅳ. Conclusions & Discussion

Result Statistically significant difference Exposure to both captions and sign language video clips resulted in the transmission of a greater amount of information in online learning Subjects used the captions to complement their understanding of the contents along with sign-language video clips. Relative effectiveness of interpreted and captioned formats may differ depending on the literacy skills and sign language skills of deaf learner Content Comprehension 1. Conclusions and Discussion

Result Contrasted to Modality effect & Redundancy effect Due to deaf people’s different learning style Due to lower readability level of the material itself that raised their intrinsic cognitive load might have affected their later cognitive load. Quality of captions ( readability, placement, spacing, presentation rate, sound effects) Cognitive load 2. Conclusions and Discussion No significant difference Such outcome seems to be a result from

Result Presence of captions did not affect motivation of deaf learners Constant barrage of contents and difficult vocabulary might have led the deaf to quickly lose interest. Instructional design of the contents might have failed to meet the unique needs of deaf learners in terms of motivation and learning interests Motivation 3. Conclusions and Discussion This can be seen as No significant difference

V. Recommendations

Captions should be provided along with sign-language video clips to meet the unique needs of Deaf learners The type and quality of captions should be taken into consideration in terms of literacy levels of the learners. Poorly captioned materials will not provide equal access to information and captioning should occur during the production stage Video Typography when presenting animation or video clips for the Deaf as it switches reading information into visual information - Increasing emotional understanding of an image by describing emotions, movement, facial expression, and other emotional and linguistic sense with written letters. (Kwon & Huh, 2009) 1.Readability level and quality of captions to meet the unique needs of deaf learners to meet the unique needs of deaf learners 2. Using Video Typography for motivation Based on these results, I would like to make 3 suggestions for further research

A systematic approach based on instructional technology should be considered to be responsive to deaf learners needs in the process of developing learning materials. Developing tailor-made instructional design, evaluation of the application, and further research should be emphasized to optimize their learning. 3. Tailor-made instructional design for the Deaf Lastly, I would like to suggest

Thank you