1. Information Technology – Dialogue Systems Ulm University (Germany) http://www.dialogue-systems.de Speech Data Corpus for Verbal Intelligence Estimation Kseniya Zablotskaya, Steffen Walter, Wolfgang Minker

2. www.dialogue-systems.de | LREC 2010 | May 2010 Page 2 Outline -Introduction -Improvement of Spoken Language Dialogue Systems -Verbal Intelligence Estimation -Monologues Collection -Hamburg Wechsler Intelligence Test -Dialogues Collection -Transcription Standards -Result Table for Each Candidate -Participants -Conclusions and Future Work

3. www.dialogue-systems.de | LREC 2010 | May 2010 Page 3 Introduction Analysis of speech Emotion Gender Age Verbal Intelligence Social class Personality Different ways to describe the same event: - “Excuse me, could you tell me the way to the railway station?” - “Hey you, show me where the railway station is.” … What our words can say about us:

4. www.dialogue-systems.de | LREC 2010 | May 2010 Page 4 Improvement of Spoken Language Dialogue Systems Spoken language dialogue system Acoustic front-end Speech recognition Linguistic analysis Dialogue management Text generationSpeech synthesis Application Cognitive processes of the user Cognitive processes of the user Adaptation to the user - estimation of cognitive processes of the user - adaptation to the user

5. www.dialogue-systems.de | LREC 2010 | May 2010 Page 5 Verbal Intelligence Estimation VI=80 VI=105 VI=120 Monologues Dialogues VI=120 VI=105 VI=80 Transcribed speech Evaluation % Feature extraction Model / Classifier VI-test VI VI Mod

6. www.dialogue-systems.de | LREC 2010 | May 2010 Page 6 Monologues Collection Two short films (Galileo): Craziest hotels in the world: -necessary to memorize the names -necessary to memorize the order Experiment on how long people could stay awake -possible to describe the film without certain details -descriptions are informative

7. www.dialogue-systems.de | LREC 2010 | May 2010 Page 7 Dialogues Collection Duration: at least 10 minutes Topic: the education and the school system in Germany - interesting - participants know a lot about it - participants have different opinions Dialogues Discussions Contra-Discussions Answering Questions

8. www.dialogue-systems.de | LREC 2010 | May 2010 Page 8 Hamburg Wechsler Intelligence Test Information (25 questions) -measures general knowledge -questions from a particular culture For example: Who is president of Russia?

9. www.dialogue-systems.de | LREC 2010 | May 2010 Page 9 Hamburg Wechsler Intelligence Test Comprehension (10 questions) - social awareness - common-sense For example: What would you do if you lost your way in a forest?

10. www.dialogue-systems.de | LREC 2010 | May 2010 Page 10 Hamburg Wechsler Intelligence Test Digital Span -forward -backward - auditory short memory - concentration - attention For example: Please listen to the fallowing digits and repeat them: 5 7 2 4 6

11. www.dialogue-systems.de | LREC 2010 | May 2010 Page 11 Hamburg Wechsler Intelligence Test Arithmetic (10 questions) -mental alertness -attention and concentration while manipulation mental mathematical problems For example: Seven envelopes cost twenty five cents. How many envelopes can you buy if you have one dollar?

12. www.dialogue-systems.de | LREC 2010 | May 2010 Page 12 Hamburg Wechsler Intelligence Test Similarities in Dissimilar Objects (12 questions) - abstract reasoning - power of conceptualization For example: Please find a similarity among a dog and a lion?

13. www.dialogue-systems.de | LREC 2010 | May 2010 Page 13 Hamburg Wechsler Intelligence Test Vocabulary (42 questions) - comprehension of meanings - relation between the expressive words For example: What does the word “zebra” mean?

14. www.dialogue-systems.de | LREC 2010 | May 2010 Page 14 Advertisement University of Ulm is seeking German Native Speakers Tasks: - to watch a film and to describe it - to make a dialogue with a partner - to take a verbal test Age: at least 16 years old Duration: approx. 1 hour Allowance: 10 Euro

15. www.dialogue-systems.de | LREC 2010 | May 2010 Page 15 Transcription Standards “?” – interrogative word intonation and rising tone “.” – completed thoughts and falling tone “,” – short pauses in the speech, but with a continuation of the main idea “;” – interrupted thoughts Example: “no no. or yes? you say; understand,” All monologues and dialogues were transcribed according to the standards by Mergenthaler. The punctuation marks in transcripts are used to show rhythmical and syntactical speech interruptions:

16. www.dialogue-systems.de | LREC 2010 | May 2010 Page 16 Result Table for Each Candidate Sub-TestPoints Information 18 out of 25 Comprehension 19 out of 20 Digit Span 15 out of 17 Arithmetic 13 out of 14 Similarities in dissimilar Objects 24 out of 24 Vocabulary 73 out of 84 Verbal IQ 122 - candidate’s points for each verbal task and the verbal IQ. - verbal IQ is measured according to the special tables of the HAWIE.

17. www.dialogue-systems.de | LREC 2010 | May 2010 Page 17 Participants 56 candidates: men - 27, women - 29 Age: 16 – 75 Language: German 71 monologues (3 hours 30 minutes) 30 dialogues (6 hours 30 minutes)

18. www.dialogue-systems.de | LREC 2010 | May 2010 Page 18 Conclusions and future directions Speech data corpus: - 56 candidates; - 10 hours of audio data; Approaches which can be applied to the collected data: - Word usage, abstracts, emotion words; - Analysis at different linguistic levels: morphology, lexicology, syntax, semantics, and discourse; - Linguistic styles; - Content words; - Degree of speakers’ immersion in monologues and dialogues; - “Good story” criteria; - Status in a conversation; - Levels of agreement; -Kelly’s repertory grids; We are going to find more candidates and to continue these recordings.

19. www.dialogue-systems.de | LREC 2010 | May 2010 Page 19 Thanks for your attention!