Active Active, a platform for building intelligent software Dr. Charles Baur (EPFL) Adam Cheyer (SRI International) Didier Guzzoni (EPFL) Active
Presentation Plan Introduction Active Framework Applications Problem space Active proposition Active Framework Active Ontologies Implementation Methodologies Applications Conclusion Introduction Active
Motivation Our information environment is rich and complex Ubiquitous access to a wealth of data and services Software and hardware industry constant innovations UIs have not changed: Simple click-and-do approach not enough Need for computer assistants Interact naturally with humans Can be delegated complex tasks Observe, understand, anticipate and act Introduction Active
Intelligent Systems Resolve Anticipate Interpret Plan action Understand Resolve Anticipate Interpret Plan action Observe Act Effectuate Communicate Listen Vision Sense Intelligent Systems : Naturally collaborates with human users to deliver services and contents through adaptable, efficient, multimodal user interfaces. Introduction Active
Difficult Task Requires numerous AI techniques Language processing Plan execution Dynamic service brokering (MAS) Implementation is difficult HCI components (speech recognition, vision, robotics) Large teams of specialists Different programming languages and platforms Testing, debugging and maintenance is difficult Performance is likely to be affected Lack of an integrated tool and methodology to easily and effectively build intelligent systems Introduction Active
Goal of Active Provide programmers with an integrated framework and a methodology to build complex AI-based systems Capable of encapsulating AI techniques Language processing, plan execution and agent type techniques Programmer friendly Small teams Based on popular programming languages (Java/Javascript) Offers an IDE (code, test, debug and deploy) Open and standard compliant SOA-based (SOAP, REST, RMI) Deployment (Java, J2EE) Active Framework Active
Basic Concept : Active Ontologies Ontology : A data structure Formal representation for domain knowledge Classes, attributes, relations P movie P genre P actor P rating Active Ontology : A processing environment Processing elements arranged according to ontology notions Communication channels Active Framework Active
Active Ontology : Processing Production Rule System Rules Sets Rules (Conditions, actions) Data store (facts) Current state of the system Evaluation Engine Evaluation passes Active Innovations Organizing rules around Ontologies helps design and debug Developer friendly rule language. Enhanced Java/Javascript with unification P movie rule set rule condition action Active Framework Active
Active Application Design One or more Active Ontologies Hosted on Active runtime Typically : Language processing Plan execution Dynamic service brokering Service oriented (SOA) Loosely coupled Sensors (user interfaces, speech recognition, vision) Actuators (robots, user interfaces) Reusable Dynamically swapped Active runtime Optionally hide. Spend less time on it. services Active Framework Active
Implementation Active Server Active Editor Active Console Hosts Active Ontologies Maintains a fact store Runs evaluation engine Extensions Active Editor IDE Code, deploy, test Pluggins Active Console Manages Active Server Active Editor Active Ontology Active Server deploy Facts store debug Evaluation Engine monitor Active Ontology Active Console Active Ontology Active Ontology Implementation Active
Active Server Runs and hosts Active Ontologies Implementation Evaluation Engine Fact Store Implementation Java application/J2EE webapp SOAP / RMI interface Rule language is Java/JavaScript enhanced by unification Extensions Encapsulate pre-compiled complex operations Active Server Evaluation Engine Active Ontologies Fact store Extensions Connect out SOAP/RMI interface services Implementation Active
Active Editor IDE Active Server Connection Plugins Graphical editing of ontologies Specialized concept and rule editors Active ontology definition files saved locally (XML) Active Server Connection Deploy/undeploy edited ontologies Integrated test/debug Plugins Automatically creates concepts and rules based on interactive wizards Show relationships names options. Implementation Active
Active Console Management tool Query (read) panel Store (write) panel Monitor and configure deployed Active Ontologies SOAP/RMI interface Query (read) panel Construct complex queries to Active Server Tabular result sets Store (write) panel Stimulates Active Ontologies by sending events to the server Could be hidden Implementation Active
Active Methodologies Language Processing Agent techniques Chart Parsing Event based Agent techniques Delegated computing Dynamic service selection Plan execution Process Execution Engine Reactive Planning Methodologies Active
Language Processing : Grammar-Based Chart parsing Advantages Formal parsing (Mathematical expressions) Deterministic Disadvantages Not flexible Not robust to missing words Not well suited for non-reliable input modalities (Speech recognition) Add: Screen Shot from chart parser Methodologies Active
Language Processing : Domain-Based Implementation Bottom Up Leaves : Word set, regex Nodes : Gather, Select Context Kept among utterances Errors, Suggestions Advantages Robust to syntax Ports well to different languages Wizards Easy modifications “find action movies in San Francisco” “nearby Chinese restaurants” Popup a leaf wizard in the animation. Second utterance. Shared address -> Context. Half-life: stay in context get gradually weaker. Change probability to confidence. Final command should not stay red. Methodologies Active
Activity/Dialog Modeling Dialogs, Activities, Behaviors Full-featured workflow management State / Transitions Flow instances, Instance space Basic flow constructs Start, End Wait, Switch (branching) Parallel, sequences Active Implementation Set of plugins (Editor Wizards) Extension (Java/JavaScript) to access flow variables Has the core primitives such as BPEL. Popup Wizard Just do by drag and drop. Front end : language processing. Second tier: Use the term business logic term. Backend : dynamic services delegation. Then you can build your entire application in Active. Methodologies Active
Dynamic Service Brokering Delegated Computing What instead of how or who Service Registry Service categories cross-ontology references Service instances (providers) Broker Parallel, Sequential, Broadcast Third-party meta-agents Active Implementation Specialized Active Ontology Server Extension, IDE Wizards Enable Fandango. One more Wizard : Test panel. Methodologies Active
Prototypes / Demonstration Information Retrieval Assistant Meeting Organizer Assistant Operating Room Assistant Prototypes Active
Information Retrieval Assistant Language Processing Plan Execution Delegation Active Server SOAP Extensions Email gmail SMTP/POP server Google Movies Yahoo! Local City Information Opentable Prototypes Active
Meeting Organizer Assistant Active Server Language Processing Plan Execution Delegation Extensions IM SOAP Email gmail calendar gmail SMTP/POP server Prototypes Active
Operating Room Assistant Mouse 3D Speech Recognizer Patient vital signs probes Context History Hand/Head Tracker Language Processing Plan Execution Powered Endoscope Actions through Delegation User Interface Gesture Recognizer Text to Speech Prototypes Active
Active Advantages One platform to learn for programmers Can build all three tiers of applications in Active One language and tool to learn Easier to debug, test and deploy AI Encapsulation-Lowering the bar Methodologies encapsulated as Active Wizards and Extensions NLP, process execution, service brokering Componentized reusable sensors and actuators TTS, Speech, Gesture recognizers, vision systems Open and Standard-based SOA design Ease of integration through SOAP and REST Reuse of existing components in multiple applications Bringing AI to the masses. Conclusion Active
Ongoing Work Research Topics Active implementation and features Combine activity recognition with process execution Implement and evaluate BDI-like behaviors with Active (goal and intention stack) Active implementation and features Scalability Performance optimizations Lightweight embedded Active Server User Evaluations Put some of our systems on-line Measure feedback from surgeons (Intelligent Operating Room) Conclusion Active
Thank You ! Questions ? Suggestions ? Remarks ? Conclusion Active