People & Speech Interfaces CS 260 Wednesday, October 4, 2006

People & Speech Born to be wild and understand speech Speech as an instrument of cooperative action What literacy has to do with speech

A video

Wired for Speech (Nass & Brave) “Humans are automatic experts at extracting the social aspects of speech.” Our brains associate voice with social relationships: so what happens when we communicate with technology through speech?

Our brains can’t tell the difference Gender stereotyping: the man (woman?) behind the curtain –pitch, pitch range Personality: opposites (don’t) attract – volume, pitch, pitch range, speed rate

Prosody vs. Pragmatics

Hello? Hey. Hey! Hey, what’s up? Not much. How are you? Okay! So…what’s going on?

“Our talk exchanges do not normally consist of a succession of disconnected remarks, and would not be rational if they did. They are characteristically…cooperative efforts.” ~ H. Paul Grice

Conversational Maxims Maxims of Quantity –Make your contribution as informative as necessary. –Do not make your contribution more informative than is necessary. Maxims of Quality –Do not say what you believe to be false. –Do not say that for which you lack adequate evidence. Maxims of Relevance –Be relevant. Maxims of Manner –Avoid obscurity. –Avoid ambiguity. –Be brief. –Be orderly.

How we use language The maxims describe how speech should be produced –Cooperative users But speech is often subordinate to some kind of action A sentence serves a syntactic function A dialogue act serves a pragmatic function

Sentence Types Constituent interrogatives Yes/no interrogatives Imperatives Assertions Each of these sentence types serve a different syntactic function

“ Would you please pass the salt?”

Dialogue Acts Task Management Acts –Constitutive acts Expressives (complimenting someone) Declaratives (sentencing someone to prison) –Informative acts Assertives (stating a fact) Interrogatives (asking for information) –Obligative acts Directives (requests) System directives (calling the help system) Commissives (offering something) Dialogue Management Acts – Flow-regulating acts (beginning an exchange) – Grounds-keeping acts (clarifying a point)

Task /Dialogue Management Example System: How may I help you? DM: exchange initiator TM: offer Caller: I was trying to place a call and must have dialed the wrong number DM: acknowledgment TM: acceptance, assertion can I get credit for that? DM: turn assignmentTM: request System: Do you need me to give you credit? DM: acknowledgment, turn assignment TM: offer Caller: Yes. DM: confirmation, turn releaseTM: acceptance

“The situation in which words are uttered can never be passed over as irrelevant to the linguistic expression.” ~ Bronislaw Malinowski

Prosody vs. Pragmatics

“Illiterate”

Illiterate people also… Live in poverty Have not had formal schooling Are marginalized Feel inferior to educated people May not have regular jobs Live in literate societies

Cognition & Literacy #1 All Kpelle men are rice farmers. Mr. Smith is not a rice farmer. Is he a Kpelle man? Illiterate Subject: I don’t know the man in person. I have not laid eyes on the man myself…If you know a person, if a question comes up about him you are able to answer. But if you do not know the person, if a question comes up about him, it’s hard for you to answer it. #2 In the far north all bears are white; Novaya Zemyla is in the far north. What color are the bears there? Illiterate Subject: You should ask the people who have been there and seen them…We always speak of only what we see; we don’t talk about what we haven’t seen. Experiential, empirical, situational knowledge

Translating an interface into illiterate (1)Replace all text with icons (2)Done!

It’s not that simple… Associating an identifier (icon, number) with a concept is a literate task Reluctance to learning how to do arbitrary tasks 2 apples 1.Oranges 2.Apples 3.Bananas Please enter your selection:

Translating into illiterate, take two (1)Turn it into a speech-based telephone application. (2)Done!

Issues with speech only interfaces Can’t support browsing Difficult to represent spatial and temporal data Requires user to remember a lot of information Forced through a deep decision tree Memorize command for abstract tasks

Our own experience Environmental factors –Reluctance –Attrition Memorization of commands –Clarifications –Dialect

An ideal interface for illiterate users