Multiple Input/Output Connector Shalini Agarwal Oxygen Conference January 8 th, 2002

L C SL C S 2 Multiple Input/Output Connector Shalini Agarwal Oxygen Workshop, January, 2002

L C SL C S 3 Multiple Input/Output Connector Agenda Goals Desired Functionality Valid Inputs/Outputs System Overview Multiple Output Demo Future Work

L C SL C S 4 Multiple Input/Output Connector Goals Scenario: Multiple input devices; multiple applications Challenge: Route input to correct application Flexible design –Dynamically add input devices (and output) –Basic socket design to allow for different applications Transparent use of different inputs and outputs

L C SL C S 5 Multiple Input/Output Connector We want three types of functionality Single Input Multiple Output Applications Multiple Inputs Single Output Application Multiple Inputs Multiple Output Applications Single input, Multiple output Multiple input, Single output Multiple input, Multiple output

L C SL C S 6 Multiple Input/Output Connector What kind of input can we accommodate? Not all input modes are compatible, but some are: –Speech can be converted to text –Palm Pilot graffiti can be converted to text –T9 converts numbers to text Implicit input such as body gestures and facial expressions cannot be converted to text –Outside the scope of this project stage

L C SL C S 7 Multiple Input/Output Connector Issues Involved – Input Side Understanding different input types –Inherent differences in information world Processing method –Can it be generic? –How much customization is required?

L C SL C S 8 Multiple Input/Output Connector What kind of output can we accommodate? We must be able to control the application through another program so that our event handler can do the action requested by the input Must take in at least one of the inputs that fit the input requirements

L C SL C S 9 Multiple Input/Output Connector Issues Involved – Output Side Choosing the output node –Similar output applications could lead to the wrong decision. –What if two applications want the input? How should we define how many connections to make? Different output formats –How much customization is required for each application?

L C SL C S 10 Multiple Input/Output Connector First Generation: Putting input and output together Appl. 1Appl. 2Appl. 3 Speech

L C SL C S 11 Multiple Input/Output Connector Second Generation: Switchbox Approach Appl. 1Appl. 2Appl. 3 Speech Switchbox

L C SL C S 12 Multiple Input/Output Connector Third Generation: Processing Input, Choosing Output Events Appl. 1 Appl. 2 Appl. 3 Speech KeyboardT9

L C SL C S 13 Multiple Input/Output Connector Mutiple Output Demo Speech Input Multiple speech domain applications –Broadcast speech messages –Choose the best fit –Allows for controlling different layers Galaxy speech processing system

L C SL C S 14 Multiple Input/Output Connector Use Galaxy Components Individually Utilizing server within Galaxy hub to access individual components Hub RecognizerNatural Language Processor Other Inputs We can add processing of other inputs by using the natural language processing component of the Galaxy Galaxy speech processing system system –Consistent method of choosing correct output –Modular functionality

L C SL C S 15 Multiple Input/Output Connector Multiple Output Demo Diagram Appl. 1 Appl. 2 Appl. 3 Speech Processing Galaxy Hubs

L C SL C S 16 Multiple Input/Output Connector Future Work Error tolerant system –Requires understanding of valid input and output types Modular –Hard because each input modality requires different type of processing –Plug and play?