1. People & Speech Interfaces CS 260 Wednesday, October 4, 2006

2. People & Speech Born to be wild and understand speech Speech as an instrument of cooperative action What literacy has to do with speech

3. A video

4. Wired for Speech (Nass & Brave) “Humans are automatic experts at extracting the social aspects of speech.” Our brains associate voice with social relationships: so what happens when we communicate with technology through speech?

5. Our brains can’t tell the difference Gender stereotyping: the man (woman?) behind the curtain –pitch, pitch range Personality: opposites (don’t) attract – volume, pitch, pitch range, speed rate

6. Prosody vs. Pragmatics

7. Hello? Hey. Hey! Hey, what’s up? Not much. How are you? Okay! So…what’s going on?

8. “Our talk exchanges do not normally consist of a succession of disconnected remarks, and would not be rational if they did. They are characteristically…cooperative efforts.” ~ H. Paul Grice

9. Conversational Maxims Maxims of Quantity –Make your contribution as informative as necessary. –Do not make your contribution more informative than is necessary. Maxims of Quality –Do not say what you believe to be false. –Do not say that for which you lack adequate evidence. Maxims of Relevance –Be relevant. Maxims of Manner –Avoid obscurity. –Avoid ambiguity. –Be brief. –Be orderly.

10. How we use language The maxims describe how speech should be produced –Cooperative users But speech is often subordinate to some kind of action A sentence serves a syntactic function A dialogue act serves a pragmatic function

11. Sentence Types Constituent interrogatives Yes/no interrogatives Imperatives Assertions Each of these sentence types serve a different syntactic function

12. “ Would you please pass the salt?”

13. Dialogue Acts Task Management Acts –Constitutive acts Expressives (complimenting someone) Declaratives (sentencing someone to prison) –Informative acts Assertives (stating a fact) Interrogatives (asking for information) –Obligative acts Directives (requests) System directives (calling the help system) Commissives (offering something) Dialogue Management Acts – Flow-regulating acts (beginning an exchange) – Grounds-keeping acts (clarifying a point)

14. Task /Dialogue Management Example System: How may I help you? DM: exchange initiator TM: offer Caller: I was trying to place a call and must have dialed the wrong number DM: acknowledgment TM: acceptance, assertion can I get credit for that? DM: turn assignmentTM: request System: Do you need me to give you credit? DM: acknowledgment, turn assignment TM: offer Caller: Yes. DM: confirmation, turn releaseTM: acceptance

15. “The situation in which words are uttered can never be passed over as irrelevant to the linguistic expression.” ~ Bronislaw Malinowski

16. Prosody vs. Pragmatics

17. “Illiterate”

20. Illiterate people also… Live in poverty Have not had formal schooling Are marginalized Feel inferior to educated people May not have regular jobs Live in literate societies

21. Cognition & Literacy #1 All Kpelle men are rice farmers. Mr. Smith is not a rice farmer. Is he a Kpelle man? Illiterate Subject: I don’t know the man in person. I have not laid eyes on the man myself…If you know a person, if a question comes up about him you are able to answer. But if you do not know the person, if a question comes up about him, it’s hard for you to answer it. #2 In the far north all bears are white; Novaya Zemyla is in the far north. What color are the bears there? Illiterate Subject: You should ask the people who have been there and seen them…We always speak of only what we see; we don’t talk about what we haven’t seen. Experiential, empirical, situational knowledge

22. Translating an interface into illiterate (1)Replace all text with icons (2)Done!

23. It’s not that simple… Associating an identifier (icon, number) with a concept is a literate task Reluctance to learning how to do arbitrary tasks 2 apples 1.Oranges 2.Apples 3.Bananas Please enter your selection:

24. Translating into illiterate, take two (1)Turn it into a speech-based telephone application. (2)Done!

25. Issues with speech only interfaces Can’t support browsing Difficult to represent spatial and temporal data Requires user to remember a lot of information Forced through a deep decision tree Memorize command for abstract tasks

26. Our own experience Environmental factors –Reluctance –Attrition Memorization of commands –Clarifications –Dialect

27. An ideal interface for illiterate users