Architecture of the Cross-Modal Bilingual Mental Lexicon

Architecture of the Cross-Modal Bilingual Mental Lexicon
Third International Symposion on Bilingualism Bristol, April 2001 Architecture of the Cross-Modal Bilingual Mental Lexicon Klaudia Grote University of Cologne / Aachen Technical University Center for Cultural Studies “Media and Cultural Communication” I my talk I would like to discuss the architecture of the Cross-modal bilingual lexicon, that is the lexicon of people who grew up with two languages in two different modalities: 1. The visual-gestural modality of Sign Language and 2. The vocal-auditive modality of spoken language. (I think that the comparison of signed and spoken languages offers a very interesting field for the study of bilingualism, because the difference in modalities between signed and spoken languages may have important and critical consequences for our understanding of language representation and processing in the bilingual mind.) I will present different empirical studies regarding the semantic relations between concepts in a lexicon based on Sign Language, Spoken Language and in a lexicon based on both Languages, spoken and signed. I investigated three different groups of participants: 1. Hearing participants who speak an oral Language, but don´t have a knowledge of signed languages. 2. Deaf participants, who use Sign Language as their preferred means of Communication. They are usual bilingual to some extent because they live within a dominant spoken language culture, but their knowledge of the spoken language will vary and is always incomplete because they cannot hear and speak. 3. The third group of participants investigate is the group of CODAS, the children of deaf Adults and this group is the group I call cross-modal bilingual because they grow up with Sign Language and Spoken Language simultaneously presented to them, so they acquire both languages at the same time and use them both regularly. I performed three studies with these three groups of participants, regarding the following research questions: 3rd International Symposion on Bilingualism

3rd International Symposion on Bilingualism
Introduction Research Question Are the architectures of the semantic systems based on sign language, spoken language and sign/spoken languages the same or is there language specificity? Can we find evidence that the use of languages in two different modalities, visual-gestural and audio-vocal, influences conceptualization processes? .... 1. Are the architectures of the .... And second... 2. Can we find... I´m not suggesting that there is no cognitive unity underlying languages in general, e.g. there are universal organizational principles in the lexicon, but I suggest that the modality differences between signed and spoken languages may influence the strength and nature of the relations between semantic concepts. This thesis, of course, refers to the hypothesis of ‘linguistic relativity’, which claims in its more moderate version that the structure of a language influences semantic categorization or the organization of the internal semantic knowledge system. Since the meaning of linguistic signs is bound up with the linguistic form, one might hypothesize that the use of a different modality leads to different language structures, because every modality has its own constrains and principles and this might influence the relations between words/signs in the mental lexicon and in turn in the conceptual system.. I will report some empirical findings I gained in an experiment regarding this question. 3rd International Symposion on Bilingualism

Introduction Modality Specific Differences between Signed and Spoken Language Length of Articulation (Bellugi/Fischer, 1972; Grosjean, 1980; Emmorey,1995) Signs = ca msec Words = ca. 500 msec Type of Articulation (Hall, 1992; Cutler u.a.,1985; Sandler, 1993; Emmorey,1995) Signs = Preference for simultaneous morphology Words = Preference for sequential morphology So first of all I´d like to point out the modality specific differences between signed and spoken languages: 1. Length of Articulation: One consistent result found by different researchers is that monomorphemic signs take longer to articulate than words. Signs approximately about 1000ms and words about 500ms. This modality difference would usually slow down the production rate of signs compared to spoken language and it would also have dramatic consequences for the memory span of the working memory because its sensitive to the articulation time of lexical items, that is the longer the words or signs the less one can keep in the articulatory loop. This is called the word length effect. So the relatively slow articulators and the limit of processing time puts pressure on signed systems to avoid a great reliance on sequentially expressed information. They rather rely on nonlinear structures, that is they compensate the long articulation time through simultaneously signing a large amount of information. (Example: Katze, Kuh) 2. So, signers have a preference for simultanous morpholody whereas speakers have a preference for linear structures. (Example: Vase vs. große Vase; arbeiten vs. hart arbeiten; A car, which is driving through a twisting street very slowly up the hill and then very fast down the hill. As you could see, a lot of information can be incorporated in one sign, either be the modulation of the movement of the sign or by using a classifier, as I did in the example with the car. All these mentioned differences lead to one interesting fact I investigated in my research, that is, that a simultaneously signed concepts which is based on iconic features or iconic classifiers of the sign, is always a syntagmatically related one, like attributes, actions or locations. So if Signers have a preference for simultaneous morphology they have a preference for merging together syntagmatically related concepts.And this could lead to different strength of semantic relations in the lexicon. This is the thesis I investigate: 3rd International Symposion on Bilingualism

Introduction General Thesis The simultaneous expression of information is an important characteristic of languages in the visual-gestural modality. Syntagmatically related concepts which can be expressed simultaneously in Sign Language are stronger related to each other than paradigmatically related concepts, which can only be expressed sequentially. Based on the observation that the modulation of the perceived iconic features of the signs and that only syntagmatically related concepts can be expressed simultanously in Sign Language, it might be that syntagmatically related concepts are stronger related to each other than paradigmatically related concepts, which can only be expressed sequentially. I investigated this thesis in two studies: 3rd International Symposion on Bilingualism

Experiments with deaf, hearing and bilingual participants Studies with deaf, hearing and bilingual participants - Verification Task with Signs vs. Spoken Words Pictures - Triad-Comparison-Task - Semantic Assoziation - Iconicity - Lexical Decision In two different studies I investigated three different groups of participants: First 1. Deaf signers of German Sign Language, second 2. Hearing speakers of German Spoken Language and finally 3. Bilinguals or also called ‘codas’, which is an abbreviation of ‘Children of deaf adults’ . They learned German Sign Language and German Spoken Language at the same time and use both languages regularly as a means of communication. The first study I performed was a verification task with signs versus spoken words and pictures. This is a paradigm typically used for such purposes and its functions in the way that two items are presented one at a time and the test participant has to decide as fast and as accurate as possible if these two items are semantically related or not. The shorter the RTs the stronger the semantic relation between the two items. In the second study, another verification task with signs/words and pictures, I investigated the influence of iconicity on conceptualization processes. Studies with deaf participants - Verification Task with Signs (Verbs and Adjectives) - Memory Task 3rd International Symposion on Bilingualism

Experiments with deaf, hearing and bilingual participants Study 1: Verification-Task with signs vs. spoken words Question: Do deaf, hearing and bilinguals show differences in semantic similarity judgements? Method: Measurement of the reaction time required by participants to determine whether there is a semantic relation between two signs or two words, respectively. The shorter the response time (RT), the stronger the semantic relation between the two presented concepts. 3rd International Symposion on Bilingualism

Yes-Answer No-Answer Study 1: Example Stimulus Set shirt shirt
clothing iron fruit eat trousers light lamp fast In study 1, the deaf, hearing and bilingual participants had to judge the semantic similarity of several pairs of items. The deaf had to judge two signs, whereas the hearing had to judge two spoken words. Here you can see an example stimulus set. For example, the sign or word ‘shirt’ was combined with a superordinate, coordinate and action and attribute. Always when they see such combinations they are supposed to answer with yes and if they Each study contained 160 runs, grouped in three series of 60, 50 and 50 trials. The subjects were told to respond as fast and as accurately as possible. Thus, this method can be used to assess the strength of semantic relations within the mental lexicon. Paradigmatic Syntagmatic Semantic Relation

In Study one the subjects were sitting in front of an computer, holding a response pad in their hands. They were told that they would see several pairs of semantically related or unrelated signs,- the first one at the top of the screen, the second at the bottom. They were instructed to press the right button if they could recognize a certain semantic relation, and the left button if they could not identify a semantic relation. Example: Dog

Experiment 1: Konzepte und Konzeptbildung
The they saw ‚cat‘ and of course they had to press the yes-button, because this is a coordinate. Konzepte und Konzeptbildung

‚Bed‘

‚to fry‘ For the hearing participants ist quite similar, but instead of seeing a vidoe they had to listen to word which was accompanied by an icon, so that theay could see, which item was the stimulus item and which one the target item.

‘dog

‘cat’

‘bed’

‘salad’

3rd Symposium on Iconicity in Language and Literature
Study 1 Verification Task: Procedure Material: 20 Stimulus Sets (20 Signs/Words + 80 Test Items + 80 Distractors) Procedure: 10 Trials and 160 Runs IV: Hearing Status (3) x Type of Semantic Relation (4) DV: Response Time Participants: 20 Deaf + 20 Hearing + 20 Bilinguals 10 trials 60 runs 50 runs 50 runs As already mentioned the experimental material I used consisted of 20 Stimulus sets, so we had 20 signs/words combined with 60 relatedpictures and 60 unrelated pictures. After 10 trials the testitems were randomly grouped in three series of 40 runs and the subjects were told to respond as fast and as accurate as possible. We tested 20 deaf, 20 hearing and 20 bilinguals, which were matched with respect to schooling, gender and age. The independent variables were the hearing status with 3 grades and type of picture with two grades. The dependent variable is the Response time and accuracy. 3rd Symposium on Iconicity in Language and Literature

Study 1: Verification Task with Signs vs. Spoken Words Hypothesis 1: In the verification task the reaction times in the visual-gestural modality are shorter for syntagmatic responses than for paradigmatic responses. In the auditory-vocal modality the reaction times have a different relation. H1: RT (GL/BI)syntag RT (GL/BI)paradig RT (H)syntag RT (H)paradig < = Based on my general thesis, that the structural diversities between signed and spoken language lead to differences in the semantic relatedness of concepts in the semantic lexicon, my hypothesis for the first study was, that in the verification task in the visual-gestural modality the reaction-times for syntagmatically related items were shorter than for paradigmatically related items and that this would be different in the audio-vocal modality . Method: Repeated measures two-way analysis of variance and t-test for paired samples 3rd Symposium on Iconicity in Language and Literature

Significantly faster Response Times for syntagmatic
Study 1: Results Significantly faster Response Times for syntagmatic semantic relations for deaf participants Verification Task with Signs: Deaf Participants The results of study one for the deaf group of participants revealed significantly different response times for every type of semantic relation. Deaf: In this diagram you see the averaged response times for superordinates and coordinates (which are paradigms) and actions and attributes, which are syntagmatically related items for the group of deaf participants. The analysis of the data showed a significant difference between the syntagmatically related items and the paradigmatically related item. The judgements of the actions and attributes were about 350 ms faster than the judgement of the superordinates and coordinates.

Response Times are not significantly faster for syntagmatic
Study 1: Results Response Times are not significantly faster for syntagmatic semantic relations for hearing participants Verification Task with Spoken Words: Hearing Participants Hearing: For the hearing the results were somewhat different. The responses to superordinates were the fastest, followed by the RTs to actions and coordinates and the slowest responses were given to attributes. For the hearing too, we could find a significant difference between syntagmatically related and paradigmatically related items, but this time in the opposite direction. Thus, the outcome supports the hypothesis, that the semantic relations of a concept differ in strength in a lexicon based on sign and a lexicon based on spoken words. But when we look closer to the general averaged RT, then it is remarkable that it is much longer for signs than for words. This is due to the fact that the articulation of a single sign takes more time than the articulation of a spoken word. In average the time for the videos is about 750 ms longer than the time for the audios.

Significantly different pattern of results for
Study 1: Results Significantly different pattern of results for bilingual participants compared to deaf participants Verification Task with Signs: Deaf and Bilingual Participants So how performed the cross-modal bilinguals? The bilinguals participated in both studies, that is with signs and with spoken words.

Significantly different pattern of results for
Study 1: Results Significantly different pattern of results for bilingual participants compared to hearing participants Verification Task with Words: Hearing and Bilingual Participants The participants of the experiments were 20 deaf and hearing paid voluteers. All the deaf participants were prelingually and profoundly deaf and they reported that German Sign Language was their preferred means of communication system. The subjects were matched with respect to schooling and age: 8 out of each group finished Junior-High-School, 6 graduated from high-school and 6 went to college.

Verification Task with Signs and Words: Bilingual Participants
Study 1: Results Significant Interaction: Pattern of results in the visual-gestural and audio-vocal modality for bilinguals are different Verification Task with Signs and Words: Bilingual Participants The participants of the experiments were 20 deaf and hearing paid voluteers. All the deaf participants were prelingually and profoundly deaf and they reported that German Sign Language was their preferred means of communication system. The subjects were matched with respect to schooling and age: 8 out of each group finished Junior-High-School, 6 graduated from high-school and 6 went to college.

Study 1: Results Study 1: Summary
Significantly faster Response Times for syntagmatic semantic relations for deaf, but not for hearing and bilingual participants. Significantly different pattern of results for bilingual participants compared to deaf and hearing participants Bilinguals performed different in the visual-gestural modality compared to the audio-vocal modality. The participants of the experiments were 20 deaf and hearing paid voluteers. All the deaf participants were prelingually and profoundly deaf and they reported that German Sign Language was their preferred means of communication system. The subjects were matched with respect to schooling and age: 8 out of each group finished Junior-High-School, 6 graduated from high-school and 6 went to college.

Study 2: Verification Task with Pictures Hypothesis 2: Mean Response Times per type of semantic relation show the same pattern in the language-presentation-mode (1) and the picture-presentation-mode (2) in each group of participants. And for the study with the pictures I had the hypothesis, that... Method: Repeated measures two-way analysis of variance and t-test for paired samples 3rd Symposium on Iconicity in Language and Literature

Experiment 1: Konzepte und Konzeptbildung
Then all groups of participants had to perform the same task with pictures. They first saw ‚cup‘ and then ‚Geschirr‘ Konzepte und Konzeptbildung

Experiment 1: Konzepte und Konzeptbildung

Experiment 1: ‘shirt’ Konzepte und Konzeptbildung

Experiment 1: ‘to iron’ Konzepte und Konzeptbildung

Verification Task with Signs vs. Pictures: Deaf Participants
Study 2: Results Significant Interaction: Different Pattern of results for deaf participants in the verbal vs. pictorial mode. Verification Task with Signs vs. Pictures: Deaf Participants r=.62** The participants of the experiments were 20 deaf and hearing paid voluteers. All the deaf participants were prelingually and profoundly deaf and they reported that German Sign Language was their preferred means of communication system. The subjects were matched with respect to schooling and age: 8 out of each group finished Junior-High-School, 6 graduated from high-school and 6 went to college.

Verification Task with Words vs. Pictures: Hearing Participants
Study 2: Results Non-Significant Interaction: Similar pattern of results, when RTs for ‘action-items’ excluded. Verification Task with Words vs. Pictures: Hearing Participants r=.57** The participants of the experiments were 20 deaf and hearing paid voluteers. All the deaf participants were prelingually and profoundly deaf and they reported that German Sign Language was their preferred means of communication system. The subjects were matched with respect to schooling and age: 8 out of each group finished Junior-High-School, 6 graduated from high-school and 6 went to college.

Signs and pictures: Significant interaction
Study 2: Results Signs and pictures: Significant interaction Words and pictures: No significant interaction Verification Task with Signs, Words and Pictures: Bilingual Participants The participants of the experiments were 20 deaf and hearing paid voluteers. All the deaf participants were prelingually and profoundly deaf and they reported that German Sign Language was their preferred means of communication system. The subjects were matched with respect to schooling and age: 8 out of each group finished Junior-High-School, 6 graduated from high-school and 6 went to college.

Deaf and Bilinguals: No significant interaction
Study 2: Results Deaf and Bilinguals: No significant interaction Hearing and Bilinguals: Significant interaction Verification Task with Pictures: Deaf, Hearing and Bilingual Participants The participants of the experiments were 20 deaf and hearing paid voluteers. All the deaf participants were prelingually and profoundly deaf and they reported that German Sign Language was their preferred means of communication system. The subjects were matched with respect to schooling and age: 8 out of each group finished Junior-High-School, 6 graduated from high-school and 6 went to college.

Study 2: Results Summary
The pattern of results for deaf participants are different in the verbal compared to the pictorial mode of presentation. For the hearing participants we don´t find a different pattern of results (RTs for action-items excluded). The pattern of results for bilinguals in the verbal-word mode and the pictorial mode are similar, but differ from the verbal-sign mode. In the pictorial task the pattern of results for deaf participants is similar to the pattern of results of the bilingual participants. >>>> Pictorial categorization influenced by architecture of the lexicon? >>>> Verification based on Word-Lexicon. >>>> Pictorial categorization is influenced by the sign- and the word-lexicon.

General Conclusions There is empirical evidence for structural diversity in the organization of a lexicon based on signs, words or both languages. The results support the hypothesis that the use of a language in a visual gestural modality reinforces the syntagmatic relations in the mental lexicon and thus, that language modality influences the structure of the semantic system. Apparently, the categorization of pictures is influenced by the used language system. Cross-modal bilinguals categorize on the basis of both languages. The strength of semantic relations between concepts in cross-modal bilinguals is influenced by both language modalities in a language specific way. The participants of the experiments were 20 deaf and hearing paid voluteers. All the deaf participants were prelingually and profoundly deaf and they reported that German Sign Language was their preferred means of communication system. The subjects were matched with respect to schooling and age: 8 out of each group finished Junior-High-School, 6 graduated from high-school and 6 went to college.

Acknowledgment Special Thanks To: Deaf and Sign Language
Ludwig Jäger Klaus Willmes Walter Huber Horst Sieprath Isa Werth Anke Huckauf Erika Linz Luise Springer Gisela Fehrmann and Karen Emmorey Deaf and Sign Language Research Team - Aachen

Frage: Stehen diese beiden Items in einer inhaltlichen Beziehung zueinander? ...followed by the picture of a ring. The response should still be yes, but not as fast as for example for the picture of a necklace.

Frage: Stehen diese beiden Items in einer inhaltlichen Beziehung zueinander?

Frage: Stehen diese beiden Items in einer inhaltlichen Beziehung zueinander? Finally this is an example for a item pair with an instructor. First ‘train’ and then ....

Frage: Stehen diese beiden Items in einer inhaltlichen Beziehung zueinander? .... The sign for football

Frage: Stehen diese beiden Items in einer inhaltlichen Beziehung zueinander? The same procedure was used for the hearing subjects, but in contrast to the deaf participants the hearing subjects heard a word, for example ’cat’ accomponied by an visual marker and then they see..

Frage: Stehen diese beiden Items in einer inhaltlichen Beziehung zueinander? ...and then they see the same pictures as the deaf subjects do, in this case a cat.

Frage: Stehen diese beiden Items in einer inhaltlichen Beziehung zueinander? Onother example is the word for baby and then the picture of the baby.

Frage: Stehen diese beiden Items in einer inhaltlichen Beziehung zueinander? The bilinguals performed on two different days both experiments: Half of the group started with the signs, the other half started with the word.

Study 1 Verification Task: Procedure Material: 20 Stimulus Sets (20 Signs/Words+ 60 Test Items + 60 Distractors) Procedure: 10 Trials and 120 Runs Participants: 20 Deaf + 20 Hearing + 20 Bilinguals IV: Hearing Status (3) x Type of Item (2) DV: Response Time and Accuracy Rate 10 trials 40 runs 40 runs 40 runs As already mentioned the experimental material I used consisted of 20 Stimulus sets, so we had 20 signs/words combined with 60 relatedpictures and 60 unrelated pictures. After 10 trials the testitems were randomly grouped in three series of 40 runs and the subjects were told to respond as fast and as accurate as possible. We tested 20 deaf, 20 hearing and 20 bilinguals, which were matched with respect to schooling, gender and age. The independent variables were the hearing status with 3 grades and type of picture with two grades. The dependent variable is the Response time and accuracy. 3rd Symposium on Iconicity in Language and Literature

Study 1 Verification Task with Signs / Spoken Words and Pictures Hypothesis In the verification task, the Response Times (RTs) of the deaf and bilingual participants for the pictures which match the iconic features of the signs, are shorter than the RTs for the pictures which do not match the features. The RTs of the hearing participants do not differ for both types of pictures. H1: RT (D)iconic-features RT (D)no iconic-features RT (H) iconic-features RT (H) no iconic-features < = RT (BI) iconic-features RT (BI) no iconic-features Based on my general thesis, that the iconicity of a sign retains an influence on the semantics of the sign, my hypothesis for the first study was, that in the verification task, the Response Times (RTs) of the deaf and bilingual participants for the pictures which match the iconic features of the signs, are shorter than the RTs for the pictures which do not match the features. The RTs of the hearing participants do not differ for both types of pictures. Method: Repeated measures two-way analysis of variance and t-test for paired samples 3rd Symposium on Iconicity in Language and Literature

Results: Study 1 Deaf Participants: Significantly faster response times (RTs) for pictures which reflect iconic features of the signs. T(19)=-7.378; p=.000 The results of study one for the deaf group of participants revealed significantly different response times for the two types of pictures. Deaf: In this diagram you see the averaged response times for pictures which reflect iconic features and pictures which do not reflect iconic features. As you can see the deaf participants need in average about 722 ms to respond to pictures which reflect the iconic features and about 850 ms to respond to the second class of pictures. The analysis of the data showed a significant difference between the two types of pictures. The responds to the iconic feature-picture were about 130 ms faster than the responses to the no-iconic feature-picture. This means 3rd Symposium on Iconicity in Language and Literature

Results: Study 1 Deaf Participants: Significantly faster response times (RTs) for pictures which reflect iconic features of the signs. This means that the deaf participants react significantly faster to a picture like you see in the middle, when the sign for ‘cow’ highlightens the horn of a cow. That means that the attribute ‘has horns’ for the concept ‘cow’is a central one. Ist more important than the attribute ‘has an utter’ or ‘has spotted hide’. No Iconic Features Iconic Features No Iconic Features 3rd Symposium on Iconicity in Language and Literature

Results: Study 1 Hearing Participants: No significant difference between RTs for pictures which reflect or do not reflect iconic features of the signs. This is different for the hearing participants The responses to both types of pictures were around 750 ms. The data analysis showed no significant difference between the two classes of items. 3rd Symposium on Iconicity in Language and Literature

Results: Study 1 Hearing Participants: No significant difference between RTs for pictures which reflect or do not reflect iconic features of signs. For the deaf participants this means that horns are not that central for concept of a ‘cow’, rather, at least in Germany the participants respond very fast to the picture presenting a spotted black and white hide. At least for this area in Germany its the most important attribute of a cow. No Iconic Features Iconic Features No Iconic Features 3rd Symposium on Iconicity in Language and Literature

Results: Study 1 Bilingual Participants (Signs): Significantly faster response times for pictures which reflect iconic features of the signs T(19)= ; p=.000 Now its interesting to look at the bilinguals. How do they respond to the pictures in the two different tasks? In the verification task with signs they react similar as the deaf subjects, although a little bit slower. Yet, we find that they respond significantly faster to the pictures which match the signs with regard to the iconic features. They are about 130 ms faster. 3rd Symposium on Iconicity in Language and Literature

Results: Study 1 Bilingual Participants (Words): Significantly faster response times for pictures which reflect iconic features of the signs T(19)=-3.371; p=.000 Interestingly they also show a significant difference in response times in the verification task with words. Although there is no relation between the words and the picture in regard to iconicity the bilingual still respond fasterto the iconic-feature-picture. This means that even with a change of language modality the effect of iconicity on semantic concepts retains. 3rd Symposium on Iconicity in Language and Literature

Results: Study 1 Summary: Results Study 1 The deaf participants show faster response times for pictures which reflect the iconic features of the matched sign. The hearing participants show the same response times for all types of semantically related pictures. The bilingual participants show faster response times for pictures which reflect the iconic features of the matched sign. This effect is found not only in the verification task with signs but also in the task with words. To sum up: The results of study one are: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3rd Symposium on Iconicity in Language and Literature

Study 2 Triad Comparison Task: Procedure Material: 20 Stimulus Sets (20 Signs/Words+ 20 Test Items) Procedure: 10 Trials and 120 Runs Participants: 20 Deaf + 20 Hearing + 20 Bilinguals IV: Hearing Status (3) x Type of Item (2) DV: Relative Number of Choices in % 10 trials 20 runs 20 runs 20 runs In the second study, the Triad comparison task, we used the same experimental materials as in study one. The items were grouped in triads of four series of 10 trials and three times 20 runs. We randomly combined a sign/word with two related pictures and the subject shoud decide if the left or right picture had astronger relationsship to the presented linguistic sign. The dependent variable was the number of choices for either type of pictures. 3rd Symposium on Iconicity in Language and Literature

Frage: Welches Bild hat die engste inhaltliche Beziehung zur Gebärde?
I show you a short demonstration of the test procedure.... First the subject saw the sign for ‘shirt’ and then saw two pictures

Frage: Welches Bild hat die engste inhaltliche Beziehung zur Gebärde? This is called in German ‘Bündchen’, sorry I don´t know the englisch word.. 3rd Symposium on Iconicity in Language and Literature

Frage: Welches Bild hat die engste inhaltliche Beziehung zur Gebärde? And the second picture is a collar. 3rd Symposium on Iconicity in Language and Literature

Frage: Welches Bild hat die engste inhaltliche Beziehung zum Wort? The same for hearing subjects...They hear the word for ‘man’ and see two related pictures. 3rd Symposium on Iconicity in Language and Literature

Frage: Welches Bild hat die engste inhaltliche Beziehung zum Wort? ‘hat’ 3rd Symposium on Iconicity in Language and Literature

Frage: Welches Bild hat die engste inhaltliche Beziehung zum Wort? And ‘tie’. 3rd Symposium on Iconicity in Language and Literature

Study 2 Triad-Comparison Task with Signs vs. Spoken Words Hypothesis In the verification task the relative number of choices (NC) of the deaf and bilingual participants for the pictures which match the iconic features of the signs is greater than the NCs for the pictures which do not match the features. The NCs of the hearing participants do not differ for both types of pictures. H1: NC (D)iconic-features < NC (D)no iconic-features The hypothesis is: >>>>>>>>>>>>> NC (H) iconic-features = NC (H) no iconic-features NC (BI) iconic-features < NC (BI) no iconic-features Method: Repeated measures two-way analysis of variance and t-test for paired samples 3rd Symposium on Iconicity in Language and Literature

Results: Study 2 Deaf Participants: Significantly more choices for pictures which reflect iconic features of the signs. T(19)=11.741; p=.000 The results support the outcomes of study one. Deaf participants chose significantly more pictures which matched the sign in regard to some iconic features. 3rd Symposium on Iconicity in Language and Literature

Results: Study 2 Deaf Participants: Significantly more choices for pictures which reflect iconic features of the signs. For example they chose more often the picture in the middle which means murder in the sense of stabbing somebody. And thery hardly chose the picturefor shooting somebody or to strangle. No Iconic Features Iconic Features No Iconic Features 3rd Symposium on Iconicity in Language and Literature

Results: Study 2 Hearing Participants: No significant difference between NCs for pictures which reflect or do not reflect iconic features of the signs. Again the hearing subjects´relative number of choicesdidn´t differ in regard to type of pictures. 3rd Symposium on Iconicity in Language and Literature

Results: Study 2 Hearing Participants: No significant difference between NCs for pictures which reflect or do not reflect iconic features of the signs. For this example they actually chose the picture which presents ‘shooting sombody’ as the most typical picture to express murder. No Iconic Features Iconic Features No Iconic Features 3rd Symposium on Iconicity in Language and Literature

Results: Study 2 Bilingual Participants (Signs): Significantly more choices for pictures which reflect iconic features of the signs. T(19)=4.720; p=.001 The bilingual´relative number of choices was significantly greater for the iconic feature pictures in both tasks. In the task with signs they chose about 60 % of the time the picture which resembled the sign . Compared to about 40 % the other type of picture. 3rd Symposium on Iconicity in Language and Literature

Results: Study 2 Bilingual Participants (Words): Significantly more choices for pictures which reflect iconic features of the signs. T(19)=6.545; p=.000 We gained the same result in the task with the words. So here again we see that even in a different language the attribute highlighted by the signi still plays a dominant role for the semantic concept corresponding to the word. 3rd Symposium on Iconicity in Language and Literature

Results: Study 2 Summary: Results Study 2 The deaf participants make significantly more choices for pictures which reflect the iconic features of the matched signs. The hearing participants make the same number of choices for all types of semantically related pictures. The bilingual participants make more choices for pictures which reflect the iconic features of the matched sign. This effect is found not only in the verification task with signs but also in the task with words. To sum up the results of study two: >>>>>>>>>>>>>>>>>>>>>>> 3rd Symposium on Iconicity in Language and Literature

Conclusions / Points to discuss 1. There is empirical evidence for the influence of sign language iconicity on the strength of the semantic relations in the conceptual system. 2. The results support the hypothesis that the characteristics or aspects of an entity which are accentuated by the iconic feature of the sign, play a central role in the corresponding concept. 3. Iconicity not only reflects conceptual and perceptual structures. In fact, the results indicate that the influence is also the other way around, i.e. the form of the sign can have an impact on semantic structures. Based on these findings we conclude that >>>>>>>>>>> So, the impact of primary iconicity is not limited to a motivational aspect in the process of sign creation... The results rather support the hypothesis..... >>>>>>>>>>>>> That means that iconicity... >>>>>>>>>>>>>>>>>>>>>>> 3rd Symposium on Iconicity in Language and Literature

Acknowledgment Deaf and Sign Language Research Team - Aachen/Cologne
Special Thanks go to the Deaf and Sign Language Research Team - Aachen/Cologne 3rd Symposium on Iconicity in Language and Literature

Architecture of the Cross-Modal Bilingual Mental Lexicon

Similar presentations

Presentation on theme: "Architecture of the Cross-Modal Bilingual Mental Lexicon"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Architecture of the Cross-Modal Bilingual Mental Lexicon

Similar presentations

Presentation on theme: "Architecture of the Cross-Modal Bilingual Mental Lexicon"— Presentation transcript:

Similar presentations

About project

Feedback