Iconicity, Frequency, and Sign length effects in TSL lexical processing Yi-Hsuan Chiu Cognitive Neuropsychology Laboratory, National Yang-Ming University, Taipei, Taiwan Cognitive Neuropsychology Laboratory, National Yang-Ming University, Taipei, Taiwan Institute of Neuroscience, School of Life Science, National Yang-Ming University, Taipei, Taiwan

An animal or a tool? DOGKEY

Iconicity of sign language The components of The components of Location Location Handshape Handshape Movement Movement constrain the meaning of the objects! Language Thought

Iconicity of sign language The components of The components of Location Location Handshape Handshape Movement Movement constrain the meaning of the objects! Language Thought iconicity

Iconicity of sign language The components of The components of Location Location Handshape Handshape Movement Movement constrain the meaning of the objects! iconicity Language --- Thought

The usage of Sign languages in Taiwan Sign language in Taiwan Sign language in Taiwan Nature sign language (Taiwan Sign Language) Nature sign language (Taiwan Sign Language) Arbitrary Arbitrary Iconicity Iconicity Vision-based Vision-based Manipulation-based Manipulation-based {the spatial information (location, handshape, movement) is meaningful} Chinese Character Sign Language Chinese Character Sign Language

Iconicity in Sign Language

Visuion-based Manipulation- based handshape All possible shape-for-shape iconicity GINGER Path-for-shape iconicity VACE Ease articulator shape-for-shape iconicity KITCHEN-KNIFEGLASS location body-for-body iconicity ELEPHANT Neutral: VACE Neutral: KEY, GLASS movement atemporal motion Fictive motion Path-for-path iconicity Factive motion

Sign lexical access Task: Picture signing, picture delay signing, picture naming, picture delay naming Task: Picture signing, picture delay signing, picture naming, picture delay naming Subjects: deaf signer, interpreter (hear signer), and normal (hear non-singer) Subjects: deaf signer, interpreter (hear signer), and normal (hear non-singer) Independent variables Independent variables Iconicity: manipulation, vision Iconicity: manipulation, vision Frequency: high, low Frequency: high, low Sign length: short, long Sign length: short, long Dependent variable: reaction time Dependent variable: reaction time +&$ Release button to sign/ name out RT

Picture signing / picture naming +&$ Release button to sign/ name out RT Picture delay signing / picture delay naming +&$ Release button to sign/ name out RT * 1s

Behavior study

Picture naming/ signing Language modality Language modalityNamingSigning Visual processing/ Concept 0-150ms  Lexical access ms (lemma)  ms (Phonological encoding) Phonetic/Articulation ms 

Picture naming/ signing NamingSigning Concept Iconicity: no difference (all noun) Lexical access Sign length: Rt (Short < Long) Frequency: Rt (High < Low) Articulation RT

Picture delay naming/ signing Delay Naming Delay Signing Concept Iconicity: no difference (all noun) Lexical access Sign length: Rt (Short = Long) Frequency: Rt (High = Low) Articulation RT Language modality

Subjects DeafInterpreterNormal Signing Naming? Hearing ability  Dif.: Hearing lossDif.: Language experience However, what is the interpreter’s mother tongue?

Deaf signing (n=24) All subjects learned TSL before their puberty, and took TSL as their daily language. All subjects learned TSL before their puberty, and took TSL as their daily language. M=14, F=10 M=14, F=10 Mean age= years-old (19-57ys) Mean age= years-old (19-57ys)

Deaf and normal (N=24) Deaf and normal (N=24)D/SN/Np* Subject (deaf/ normal) ** Task (signing/delay signing) ** Interaction0.0000**

Deaf and normal (N=24) Deaf and normal (N=24) p* Interaction0.0000** sign-de. sign ** name- de. name ** sign-name ** de. sign- de. name ** Iconicity Articulation

N=24DeafInterpreterNormal Signing/GesturingIconicity*** Frequency****** Sign length ****** Delay signing Iconicity Frequency Sign length NamingIconicity** Frequency**** **** Delay naming Iconicity Frequency Sign length *

ERP study

Picture naming/ signing Iconicity: (verb < noun)~ (manipulation < vision) N400 Iconicity: (verb < noun)~ (manipulation < vision) N400 Frequency: ms (High < Low) Frequency: ms (High < Low) ms (High < Low) N ms (High < Low) N400 Sign length: ms (Short < Long) Sign length: ms (Short < Long)

Time window Window (ms) IconicityFrequency Sign length

Location (channel) Brain areas ChannelsIco.Fre.Leng. Frontal cortex F1, FCz, F2 M>VM<VH<L Inferior frontal (44)/ premotor cortex (6) FC5, FC6 M>V Motor cortex C3, C4 M>V Occipital cortex O1, Oz, O2 M<VS<L

Deaf Signing (N=8) and Normal naming (N=12)

Nouns and verbs Federmeier et al Federmeier et al Noun Verb N400: N>V

Deaf signing (Iconicity: M/V) (N=8)

Frequency (H<L) : H<L : H<L : H<L : H<L Muente et al Muente et al. 2001

Deaf signing (Fre.: H/L) (N=8) ms ms:H<L

Sign length (S< L) ms: S<L ms: S<L Assadollahi et al MEG Short word Long word

Deaf signing (Leng.: S/L) (N=8) ms ms:S<L

ERP data summary IconicityFrequency Sign length N400 N>V class V>M ico H<L60-120S<L Deaf signing *** Normal naming * Interpreter signing * S>L ? Interpreter naming *

Summary Frequency, sign length effects in TSL lexicon~ Frequency, sign length effects in TSL lexicon~ Chinese lexicon (spoken language lexicon) Sign language Iconicity ~ semantic Sign language Iconicity ~ semantic RT: signing< naming, delay signing~ delay naming RT: signing< naming, delay signing~ delay naming ERP: , ms, signing< naming ERP: , ms, signing< naming Normal gesture: RT: M<V Normal gesture: RT: M<V Iconicity: Iconicity: ERP: Manipulation/Vision~ Verbs/Nouns ERP: Manipulation/Vision~ Verbs/Nouns