Coding Gestures PSYC 453 Meeting 6 Photo: Koç University Longitudinal Language Development Database (KULLDD) Coding Gestures PSYC 453 Meeting 6

What is coded as a gesture One motion, one handshape, one place. A single gesture entry has one meaningful motion, one handshape (but see exception), and one place. Exception: If the motion is a "within hand" motion (e.g., hand open, hand close, fingers wiggle, hand bends, etc.), there is only one gesture entry (i.e., not one entry for the hand when open and one entry for the hand when closed)

Only symbolic gestures Only symbolic gestures are coded. Nonsymbolic movements are not coded. Nonsymbolic motions include imitations (i.e., a gesture that imitates the immediately preceding gesture of another person); performances (i.e., repetition of gesture or group of gesture at insistence of other person in order to "get it right"); rituals (i.e., pantomimes that belong to games or songs or poems); and functional acts (i.e., actual actions [push a toy, scratch an itchy spot, cover a mouth to cough, pull Mom's hand]).

Indications that a gesture may have occurred a hand or body part moves or changes its place a hand that does not move is acted upon by a second hand  an unusual handshape or motion form  Note: Just because a handshape does not "look like" a particular object does not mean it does not represent that object: There are other clues aside from handshape that can make the identity of the gesture clear.  

Gesture types Show ELAN annotation (i.e., coding) Attı(m) + point deictic-point Indication of a distal object through directional extension of the finger, hand, limb, or occasionally, head. point, palm, continuous point Attı(m) + point 15 months Elif Q: why can this be considered a two-act utterance? Show ELAN annotation (i.e., coding) ELAN is a professional tool for the creation of complex annotations on video and audio resources 

Gesture types deictic-show Drawing attention to an object or body part by holding it up towards the interlocutor. hold

Gesture types Give/BEG Requesting an object by reaching towards the interlocutor with the hand palm-up.

Gesture type conventional Culturally-determined or ritualized gestures that serve as symbolic replacements for ideas and are produced in a similar form every time. nod, shake, come, wait, shh, thumbs up, çık in Turkish (chin up)

Gesture type representational Depicting attributes or actions associated with objects through spontaneous (mostly manual) movement. iconic: The relationship between the form of the sign and the meaning of the sign is representational. That is, the sign form looks like the sign meaning (e.g., HIT, TWIST, FLY).

Gesture type Iconic/representational Attım Go to ELAN 19 months, Elif Q: Did you notice the mom’s gesture? What kind of gesture? How does it combine with her speech? Q: how does E’s gesture combine with her speech? Go to ELAN

Different types of speech-gesture combinations Match (reinforce) Disambiguate Deictics, for example, need to be disambiguated Supplement

Reinforcing/Matching gestures Reinforcing referent Gesture reinforces the info provided in speech. “Cat” +[point to cat] Reinforcing attribute Gesture reinforces an adjective in speech describing perceptual features. “Here is a blue one” +[point to blue block] Gesture reinforces the spoken predicate (verb) describing an action the referenced object performsp “Crying” +[point to crying baby in book]

Disambiguating gestures Gesture disambiguates speech “Put it there” +[point to table] “These are mine” +[toys on palm ] Senem, 36 months Oraya oturttur Make Maviş sit there

Language-Specificity and Interplay of Action, Gesture, and Speech in Early Language Development of Caused Motion Reyhan Furman Radboud University, Nijmegen Aslı Özyürek Radboud University and Max Planck Institute for Psycholinguistics, Nijmegen Aylin Küntay Koç University, Istanbul

Background Few cross cultural studies on the early development of gestures and their relation to language development No inquiry into the effects of type of language on this development

General Research Questions Is gesture development and its relation to language similar across cultures ? What is the possible effect of typology of language in the early interplay between gesture, speech and action? Are the type of gestures children produce and the relationship between speech and gesture influenced by the language that children learn to speak? Early acquisition of verbs, so are there more iconic gestures? Noun/Argument omissions, so are gestures used in a more supplementary function to speech?

“The girl pushed the box into the building.” Caused Motion Events Caused motion events are basic events. “The girl pushed the box into the building.” AGENT ACTION FIGURE PATH GOAL 17

Expression of Motion Events Across Languages (Talmy, 1985) Satellite-framed languages: Action in the verb, Path in the satellite. The boy put the book into his bag. Verb-framed languages: Action and Path in the verb. Kız kitab-ı çanta-ya sok-tu. girl book-acc bag-dat insert-past “The girl put the book into the bag.” Turkish is a language in which verbs are acquired early (Aksu-Koc & Slobin, 1986) and the arguments of a verb can easily be omitted in speech.

Specific Research Questions Do Turkish children talk about caused motion events in language-specific ways in early language development? How do Turkish children gesture when they talk about caused motion events? What types of gestures are used? How does the early relationship speech-gesture relationship look like in Turkish?

Data Set Using the Koç University Longitudinal Language Development Corpus, we sampled the spontaneous speech and co-speech gestures of 8 Turkish-speaking children in Istanbul. For each child, one-hour sessions were sampled every month between age 1 and 3. ~20 sessions per child (160 hrs) Children were videotaped while engaged in daily activities at home with caregivers.

Coding: Speech 1778 utterances transcribed and coded for the type of construction that children used. Verb Only Action: Koydum. “(I) put (it)”. Action+Path: Soktuk. “(We) put (it) in.” Verb+Arguments (e.g., Figure, Object, Goal) Action+Figure: Bunu koyuyoruz. “(We) put this one.”

Coding: Gesture 468 co-speech gestures coded for: Type Point Show Iconic Functional Act Speech-gesture relation (Özçalışkan & Goldin-Meadow, 2005; 2009). Match Disambiguate Supplement

Speech: Do Turkish children begin with Verb- Only Speech: Do Turkish children begin with Verb- Only? Light blue = 14-26 months; Dark blue= 27-36 months Verb-only constructions are used more frequently at earlier ages and Verb+Noun (Argument) constructions are used more frequently later on, ps =.000. Verb-only constructions are used more frequently at earlier ages, p=.000. Furman, Özyürek & Küntay, 2010

Mean number of gestures per session Light blue = 14-26 months; Dark blue= 27-36 months Overall, showing gestures are used less frequently than points, p = .001. No age differences in the production of different gesture types.

Gesture functions: Do gestures express elements not in speech Gesture functions: Do gestures express elements not in speech ? Light blue = 15-26 months; Dark blue= 27-36 months Overall, disambiguating gestures are used less than matching and supplementary ones, ps = .001. Older children use more disambiguating gs, p= .027.

Gesture: How do different gesture types function with regard to speech? Match: Iconic and FA, p < .05. Supplement: All Disambiguate: Point and Show, p < .05. There are no age differences.

Conclusions SPEECH Language-specificity is evident from the start of development. Around 18 months of age, Turkish children start to talk about caused motion events using only verbs, as in other V-framed languages (Choi & Bowerman, 1991; Narasimhan & Brown, 2009; Slobin et al., 2008) and unlike S-framed languages (i.e. with prepositions). Although arguments appear in children’s speech after the age of 27 months, argument omission still continues at later ages. 27

Conclusions GESTURE Type of Gesture Gestures also appear to be language-specific from the start of development. Between the ages of 1 and 3, Turkish children use equal numbers of deictic and iconic gestures, unlike their English-speaking counterparts who use only deictics early on (Özçalışkan & Goldin-Meadow, in press). Turkish children produce iconics earlier (around 20-22 months) than children speaking other languages. The early use of verbs in Turkish may be triggering the early use of iconic gestures. Iconic gestures appear in utterances with verbs but not in Argument-Only utterances. 28

Conclusions GESTURE Speech-Gesture Relationship Gestures and even manipulative actions reveal more aspects of the caused motion event than speech from early on (verb-only stage). Even at the age of 3, supplementary gestures are still used as frequently as in earlier ages due to argument omission possibilities. In contrast, the supplementary gestures of English-speaking children decrease with age (Özçalışkan & Goldin-Meadow, 2005; 2009). 29

Conclusions GESTURE Speech-Gesture Relationship Unlike American and Italian children who use only deictic gestures to supplement their speech (Iverson et al., 2008), Turkish children use all different types of gestures. With age, gestures help more to disambiguate similar to English-speaking children (Özçalışkan & Goldin-Meadow, 2005; 2009). 30

Lab report I Will be assigned tomorrow (October 19) Will be due by Friday (October 22, 9:30 AM) Estimated time needed: 3 hours Will need to download ELAN http://www.lat-mpi.eu/tools/elan/download