Pardon Me, Your Computer’s Showing Using speech to speed and streamline desktop computing Kimberly Patch President, Redstart Systems SpeechTek West February 22, 2007

Keys to Using Speech to Speed and Streamline Computer Control Minimizing steps Making commands easy to remember Enabling combinations

Human-Machine Grammar Follows the way the brain works Designed for controlling a computer

Keys to Using Speech to Speed and Streamline Computer Control Minimizing steps

Tap existing knowledge, but don’t let experience confine new methods of communication.

11 steps Retrieving a Picture Keyboard and Mouse Steps Click Insert Click Picture Click From File Click Look In Click Program Files Click Redstart Systems Click Utter Command Click Demo Click Bird Pictures Left Arrow Enter

Think outside the silicon box

11 steps Retrieving a Picture Keyboard/Mouse Style in the Real World Stand up Walk to the file cabinet Open the third drawer Go to the "B" section Find the Bird folder Open the Bird folder Find the Pictures subfolder Find the Redstart Singing picture Take it out of the folder Put in my hand Let go

Efficient Speech # 1Efficient Speech # 2 Under i p f Bird Pictures Folder 1 Right Enter Redstart Singing File All Copy to Word New 3 steps 2 steps Retrieving a Picture

Keyboard and Mouse Click Outlook Express icon Click Message Click New Message Type “David, Linda” Tab Key Type “Eric” Tab Key Type “Paris trip” Tab Key Type “David, Linda,” Enter Address and Add Subject, First Line 12 steps

Keyboard and MouseEfficient Speech Click Outlook Express icon Click Message Click New Message Type “David, Linda” Tab Key Type “Eric” Tab Key Type “Paris trip” Tab Key Type “David, Linda,” Enter Express David Linda CC Kim Paris trip 1 Tab 12 steps 3 steps Address and Add Subject, First Line

Accommodating vs. Direct

Keys to Using Speech to Speed and Streamline Computer Control Minimizing steps Making commands easy to remember

How do you remember commands?

Two Fry

Counter Trey Right

Unit 26, Code 11-31, 13th and Vine

Delta 265, Clear to Land, Runway 3-0

Chair Dance Teeth Grow

Blues 2

Human-HumanHuman-Computer Two Fry Counter Trey Right Delta 265, Clear to land, Runway 3-0 Chair Dance Blues 2 Word Open Backspace Left Delete Bird Pictures Folder Window 0 by 20 Window Close No Efficiently Worded Instruction

Important Factors How the brain processes words

Important Factors How the brain processes words Memory chunking

Important Factors How the brain processes words Memory chunking Six degrees of separation

Human-Machine Grammar No synonyms

Human-Machine Grammar No synonyms Logical rules to minimize wording possibilities

Human-Machine Grammar No synonyms Logical rules to minimize wording possibilities Follows the way the brain uses language

UC Top 60 Command Words (plus numbers and labels) All Caps Another By Cap Check Clear Compound Copy Cut Go Graph Insert Menu Message Microphone Mouse New Nope Paste Redo Screen Seconds Short Site Speech Spell This Touch Touch 2 Tray 1-20 Under Undo Volume Win(dow) Word Words Paired words Left – Right Lefts – Rights Before – After Befores – Afters Up – Down Ups – Downs Line – Line Up Lines – Line Ups On – Off Open – Close Top – Bottom Max(imize) – Min(imize)

Most Common HMG Rules Match words used for a command as closely as possible with what the command does Be consistent Eliminate synonyms Follow the way people naturally adjust language to fit a situation Follow the order of events

Common Commands Speech On Line Copy 3 Before Window Close Screen Clear

Keys to Using Speech to Speed and Streamline Computer Control Minimizing steps Making commands easy to remember Enabling combinations

Computer Command Steps

Carnegie Mellon Study 74% of users prefer a structured rather than natural language approach to speech recognition Speech Graffiti Vs. Natural Language: Assessing the User Experience, Stephanie Tomko and Roni Rosenfeld, Carnegie Mellon University

Command Language Based on Efficient Human Process Minimal steps

Command Language Based on Efficient Human Process Minimal steps Easy-to-remember commands

Command Language Based on Efficient Human Process Minimal steps Easy-to-remember commands Combinations

Beyond the Keyboard and Mouse Speech links Document access and navigation Favorites-like lists Virtual calculator

Word Help – Insert a Special Character Table of ContentsSearch Function 1 Click Table of Contents Click Working with Text Click Symbols, Characters…" Click "Insert a symbol or…" Click Close Function 1 Special Characters Click "Insert a symbol or..." Click "Insert a special… " Click Close 8 steps 7 steps

Beyond the Keyboard and Mouse Speech links Document access and navigation Favorites-like lists Virtual calculator

Human-Machine Grammar Papers and presentations Utter Command – speech interface software that works the way your brain does