1. Coding Gestures PSYC 453 Meeting 6
Photo: Koç University Longitudinal Language Development Database (KULLDD)
Coding Gestures
PSYC 453
Meeting 6

2. What is coded as a gesture
One motion, one handshape, one place. A single gesture entry has one meaningful motion, one handshape (but see exception), and one place.
Exception: If the motion is a "within hand" motion (e.g., hand open, hand close, fingers wiggle, hand bends, etc.), there is only one gesture entry (i.e., not one entry for the hand when open and one entry for the hand when closed)

3. Only symbolic gestures
Only symbolic gestures are coded. Nonsymbolic movements are not coded.
Nonsymbolic motions include imitations (i.e., a gesture that imitates the immediately preceding gesture of another person); performances (i.e., repetition of gesture or group of gesture at insistence of other person in order to "get it right"); rituals (i.e., pantomimes that belong to games or songs or poems); and functional acts (i.e., actual actions [push a toy, scratch an itchy spot, cover a mouth to cough, pull Mom's hand]).

4. Indications that a gesture may have occurred
a hand or body part moves or changes its place
a hand that does not move is acted upon by a second hand
 an unusual handshape or motion form
 Note: Just because a handshape does not "look like" a particular object does not mean it does not represent that object: There are other clues aside from handshape that can make the identity of the gesture clear.

5. Gesture types Show ELAN annotation (i.e., coding) Attı(m) + point
deictic-point
Indication of a distal object through directional extension of the finger, hand, limb, or occasionally, head.
point, palm, continuous point
Attı(m) + point
15 months Elif
Q: why can this be considered a two-act utterance?
Show ELAN annotation (i.e., coding)
ELAN is a professional tool for the creation of complex annotations on video and audio resources 

6. Gesture types deictic-show
Drawing attention to an object or body part by holding it up towards the interlocutor.
hold

7. Gesture types Give/BEG
Requesting an object by reaching towards the interlocutor with the hand palm-up.

8. Gesture type conventional
Culturally-determined or ritualized gestures that serve as symbolic replacements for ideas and are produced in a similar form every time.
nod, shake, come, wait, shh, thumbs up, çık in Turkish (chin up)

9. Gesture type representational
Depicting attributes or actions associated with objects through spontaneous (mostly manual) movement.
iconic: The relationship between the form of the sign and the meaning of the sign is representational. That is, the sign form looks like the sign meaning (e.g., HIT, TWIST, FLY).

10. Gesture type Iconic/representational Attım Go to ELAN 19 months, Elif
Q: Did you notice the mom’s gesture? What kind of gesture? How does it combine with her speech?
Q: how does E’s gesture combine with her speech?
Go to ELAN

11. Different types of speech-gesture combinations
Match (reinforce)
Disambiguate
Deictics, for example, need to be disambiguated
Supplement

12. Reinforcing/Matching gestures
Reinforcing referent
Gesture reinforces the info provided in speech.
“Cat” +[point to cat]
Reinforcing attribute
Gesture reinforces an adjective in speech describing perceptual features.
“Here is a blue one” +[point to blue block]
Gesture reinforces the spoken predicate (verb) describing an action the referenced object performsp
“Crying” +[point to crying baby in book]

13. Disambiguating gestures
Gesture disambiguates speech
“Put it there” +[point to table]
“These are mine” +[toys on palm ]
Senem, 36 months
Oraya oturttur
Make Maviş sit there

14. Language-Specificity and Interplay of Action, Gesture, and Speech in Early Language Development of Caused Motion
Reyhan Furman
Radboud University, Nijmegen
Aslı Özyürek
Radboud University and
Max Planck Institute for Psycholinguistics, Nijmegen
Aylin Küntay
Koç University, Istanbul

15. Background
Few cross cultural studies on the early development of gestures and their relation to language development
No inquiry into the effects of type of language on this development

16. General Research Questions
Is gesture development and its relation to language similar across cultures ?
What is the possible effect of typology of language in the early interplay between gesture, speech and action?
Are the type of gestures children produce and the relationship between speech and gesture influenced by the language that children learn to speak?
Early acquisition of verbs, so are there more iconic gestures?
Noun/Argument omissions, so are gestures used in a more supplementary function to speech?

17. “The girl pushed the box into the building.”
Caused Motion Events
Caused motion events are basic events.
“The girl pushed the box into the building.”
AGENT
ACTION
FIGURE
PATH
GOAL 17

18. Expression of Motion Events Across Languages (Talmy, 1985)
Satellite-framed languages:
Action in the verb, Path in the satellite.
The boy put the book into his bag.
Verb-framed languages:
Action and Path in the verb.
Kız kitab-ı çanta-ya sok-tu.
girl book-acc bag-dat insert-past
“The girl put the book into the bag.”
Turkish is a language in which verbs are acquired early (Aksu-Koc & Slobin, 1986) and the arguments of a verb can easily be omitted in speech.

19. Specific Research Questions
Do Turkish children talk about caused motion events in language-specific ways in early language development?
How do Turkish children gesture when they talk about caused motion events?
What types of gestures are used?
How does the early relationship speech-gesture relationship look like in Turkish?

20. Data Set
Using the Koç University Longitudinal Language Development Corpus, we sampled the spontaneous speech and co-speech gestures of 8 Turkish-speaking children in Istanbul.
For each child, one-hour sessions were sampled every month between age 1 and 3.
~20 sessions per child (160 hrs)
Children were videotaped while engaged in daily activities at home with caregivers.

21. Coding: Speech
1778 utterances transcribed and coded for the type of construction that children used.
Verb Only
Action: Koydum.
“(I) put (it)”.
Action+Path: Soktuk.
“(We) put (it) in.”
Verb+Arguments (e.g., Figure, Object, Goal)
Action+Figure: Bunu koyuyoruz.
“(We) put this one.”

22. Coding: Gesture 468 co-speech gestures coded for: Type
Point
Show
Iconic
Functional Act
Speech-gesture relation (Özçalışkan & Goldin-Meadow, 2005; 2009).
Match
Disambiguate
Supplement

23. Speech: Do Turkish children begin with Verb- Only
Speech: Do Turkish children begin with Verb- Only? Light blue = months; Dark blue= months
Verb-only constructions are used more frequently at earlier ages and Verb+Noun (Argument) constructions are used more frequently later on,
ps =.000.
Verb-only constructions are used more frequently at earlier ages, p=.000.
Furman, Özyürek & Küntay, 2010

24. Mean number of gestures per session Light blue = 14-26 months; Dark blue= 27-36 months
Overall, showing gestures are used less frequently than points,
p = .001.
No age differences in the production of different gesture types.

25. Gesture functions: Do gestures express elements not in speech
Gesture functions: Do gestures express elements not in speech ? Light blue = months; Dark blue= months
Overall, disambiguating gestures are used less than matching and supplementary ones, ps = .001.
Older children use more disambiguating gs, p= .027.

26. Gesture: How do different gesture types function with regard to speech?
Match: Iconic and FA, p < .05.
Supplement: All
Disambiguate: Point and Show, p < .05.
There are no age differences.

27. Conclusions
SPEECH
Language-specificity is evident from the start of development.
Around 18 months of age, Turkish children start to talk about caused motion events using only verbs, as in other V-framed languages (Choi & Bowerman, 1991; Narasimhan & Brown, 2009; Slobin et al., 2008) and unlike S-framed languages (i.e. with prepositions).
Although arguments appear in children’s speech after the age of 27 months, argument omission still continues at later ages. 27

28. Conclusions GESTURE Type of Gesture
Gestures also appear to be language-specific from the start of development.
Between the ages of 1 and 3, Turkish children use equal numbers of deictic and iconic gestures, unlike their English-speaking counterparts who use only deictics early on (Özçalışkan & Goldin-Meadow, in press).
Turkish children produce iconics earlier (around months) than children speaking other languages.
The early use of verbs in Turkish may be triggering the early use of iconic gestures.
Iconic gestures appear in utterances with verbs but not in Argument-Only utterances. 28

29. Conclusions GESTURE Speech-Gesture Relationship
Gestures and even manipulative actions reveal more aspects of the caused motion event than speech from early on (verb-only stage).
Even at the age of 3, supplementary gestures are still used as frequently as in earlier ages due to argument omission possibilities.
In contrast, the supplementary gestures of English-speaking children decrease with age (Özçalışkan & Goldin-Meadow, 2005; 2009). 29

30. Conclusions GESTURE Speech-Gesture Relationship
Unlike American and Italian children who use only deictic gestures to supplement their speech (Iverson et al., 2008), Turkish children use all different types of gestures.
With age, gestures help more to disambiguate similar to English-speaking children (Özçalışkan & Goldin-Meadow, 2005; 2009). 30

31. Lab report I Will be assigned tomorrow (October 19)
Will be due by Friday (October 22, 9:30 AM)
Estimated time needed: 3 hours
Will need to download ELAN