A Speech Interface to Virtual Environment Authors Scott McGlashan and Tomas Axling Swedish Institute of Computer Science.

Slides:

Advertisements

Similar presentations

CONCEPTUAL WEB-BASED FRAMEWORK IN AN INTERACTIVE VIRTUAL ENVIRONMENT FOR DISTANCE LEARNING Amal Oraifige, Graham Oakes, Anthony Felton, David Heesom, Kevin.

Advertisements

National Technical University of Athens Department of Electrical and Computer Engineering Image, Video and Multimedia Systems Laboratory

Structured Design The Structured Design Approach (also called Layered Approach) focuses on the conceptual and physical level. As discussed earlier: Conceptual.

Alina Pommeranz, MSc in Interactive System Engineering supervised by Dr. ir. Pascal Wiggers and Prof. Dr. Catholijn M. Jonker.

Cognitive Systems, ICANN panel, Q1 What is machine intelligence, as beyond pattern matching, classification and prediction. What is machine intelligence,

ARCH-05 Application Prophecy UML 101 Peter Varhol Principal Product Manager.

Requirements Engineering n Elicit requirements from customer  Information and control needs, product function and behavior, overall product performance,

Spoken Dialogue Technology How can Jerry Springer contribute to Computer Science Research Projects?

The Travails of Visually Impaired Web Travelers Presented by Chih-Tang Lee By Carole Goble Simon Harper Robert Stevens.

Introduction to Software Architecture. What is Software Architecture?  It is the body of methods and techniques that help us to manage the complexities.

Understanding and Conceptualizing Interaction Chapter 2.

Chapter 2: Understanding and conceptualizing interaction

Spatial reasoning in a multi-modal user guide for a complex machine Nadejda Soudzilovskaia, Rafael Bidarra, Frederik W. Jansen Delft University of Technology,

What Is Object-Oriented Design? (Chapter 1). Software Development Life Cycle 1. Problem statement and requirements 2. Solution specification 3. Code design.

Computing ESSENTIALS     CHAPTER Ch 9Copyright 2003 The McGraw-Hill Companies, Inc Graphics, Multimedia, and Artificial Intelligence computing.

1 Case Study: Starting the Student Registration System Chapter 3.

Methodology Conceptual Database Design

THE BASICS OF THE WEB Davison Web Design. Introduction to the Web Main Ideas The Internet is a worldwide network of hardware. The World Wide Web is part.

Software Architecture premaster course 1.  Israa Mosatafa Islam  Neveen Adel Mohamed  Omnia Ibrahim Ahmed  Dr Hany Ammar 2.

Section 2.1 Compare the Internet and the Web Identify Web browser components Compare Web sites and Web pages Describe types of Web sites Section 2.2 Identify.

Chapter 2: Understanding and conceptualizing interaction Question 1.

Chapter 6 System Engineering - Computer-based system - System engineering process - “Business process” engineering - Product engineering (Source: Pressman,

Communication Degree Program Outcomes

Ciarán O’Leary Wednesday, 23 rd September Ciarán O’Leary School of Computing, Dublin Institute of Technology, Kevin St Research Interests Distributed.

Requirements Analysis

GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004.

Some Thoughts on HPC in Natural Language Engineering Steven Bird University of Melbourne & University of Pennsylvania.

1 Web Basics Section 1.1 Compare the Internet and the Web Compare Web sites and Web pages Identify Web browser components Describe types of Web sites Section.

The NISO Question/Answer Transaction Protocol (QATP) AVIAC January 2004 Donna Dinberg Library and Archives Canada Mark Needleman Sirsi Corporation.

Author: James Allen, Nathanael Chambers, etc. By: Rex, Linger, Xiaoyi Nov. 23, 2009.

Computer Graphics Lecture 28 Fasih ur Rehman. Last Class GUI Attributes – Windows, icons, menus, pointing devices, graphics Advantages Design Process.

© 2007 Tom Beckman Features:  Are autonomous software entities that act as a user’s assistant to perform discrete tasks, simplifying or completely automating.

Using Business Scenarios for Active Loss Prevention Terry Blevins t

Situated Design of Virtual Worlds Using Rational Agents Mary Lou Maher and Ning Gu Key Centre of Design Computing and Cognition University of Sydney.

3231 Software Engineering By Germaine Cheung Hong Kong Computer Institute Lecture 12.

The Effectiveness of Web Components Presented By: Geoffrey Zimmerman Computer Science Capstone Fall 2004/Spring 2005 Mentor: Dr. C. David Shaffer.

A Cognitive Substrate for Natural Language Understanding Nick Cassimatis Arthi Murugesan Magdalena Bugajska.

ELA Common Core Shifts. Shift 1 Balancing Informational & Literary Text.

1 PLAN RECOGNITION & USER INTERFACES Sony Jacob March 4 th, 2005.

User-Centered Development Methodology A user interface comprises “ those aspects of the system that the user comes in contact with.” ● Moran [1981]

Towards Cognitive Robotics Biointelligence Laboratory School of Computer Science and Engineering Seoul National University Christian.

The NISO NETREF Protocol Mark H Needleman Product Manager- Standards Sirsi Corporation LITA National Conference 2004.

Odyssey A Reuse Environment based on Domain Models Prepared By: Mahmud Gabareen Eliad Cohen.

RELATIONAL FAULT TOLERANT INTERFACE TO HETEROGENEOUS DISTRIBUTED DATABASES Prof. Osama Abulnaja Afraa Khalifah

Approaching a Problem Where do we start? How do we proceed?

AVI/Psych 358/IE 340: Human Factors Interfaces and Interaction September 22, 2008.

Computers in Police Cruisers Article in Pervasive Computing FIRST RESPONSE Authors: Andrew L. Kun, W. Thomas Miller III, and William H. Lenharth ECE in.

Supporting Scenario-Based Requirements Engineering IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 24, NO. 12, DECEMBER, 1998 A. G. Sutcliffe, N. A. M.

Software Engineering User Interface Design Slide 1 User Interface Design.

E.g.: MS-DOS interface. DIR C: /W /A:D will list all the directories in the root directory of drive C in wide list format. Disadvantage is that commands.

Agents that Reduce Work and Information Overload and Beyond Intelligent Interfaces Presented by Maulik Oza Department of Information and Computer Science.

Digital Libraries1 David Rashty. Digital Libraries2 “A library is an arsenal of liberty” Anonymous.

Digital Video Library Network Supervisor: Prof. Michael Lyu Student: Ma Chak Kei, Jacky.

Chapter 5:User Interface Design Concepts Of UI Interface Model Internal an External Design Evaluation Interaction Information Display Software.

Secure middleware patterns E.B.Fernandez. Middleware security Architectures have been studied and several patterns exist Security aspects have not been.

RULES Patty Nordstrom Hien Nguyen. "Cognitive Skills are Realized by Production Rules"

ELACC7W1 Write arguments to support claims with clear reasons and relevant evidence.

SEESCOASEESCOA SEESCOA Meeting Activities of LUC 9 May 2003.

Speech Processing 1 Introduction Waldemar Skoberla phone: fax: WWW:

Computer Science and Engineering Department The University of Texas at Arlington MavHome: An Intelligent Home Environment.

Computer Systems Architecture Edited by Original lecture by Ian Sunley Areas: Computer users Basic topics What is a computer?

1 Nicholas Vidovich Software Developer Battelle Memorial Institute Intuitive, Interactive Data Visualization Applications Not for distribution or publication.

Chapter 5 Process Modeling By Muna Shabaneh. What is a Model? What is a process? What is a Process modeling? What are the Perspectives in process representation.

CIRP Annals - Manufacturing Technology 60 (2011) 1–4 Augmented assembly technologies based on 3D bare-hand interaction S.K. Ong (2)*, Z.B. Wang Mechanical.

AUTHOR PRADEEP KUMAR B.tech 1 st year CSE branch Gnyana saraswati college of eng. & technology Dharmaram(b)

International Telecommunication Union The Fully Networked Car Geneva, 3-4 March 2010 Human Machine Interface (HMI) and signal processing for Intelligent.

Mixed Reality Server under Robot Operating System

Geography Matters… to All of Us

Multimodal Human-Computer Interaction New Interaction Techniques 22. 1

Presentation transcript:

A Speech Interface to Virtual Environment Authors Scott McGlashan and Tomas Axling Swedish Institute of Computer Science

Presentation Agenda Introduction The TALKING AGENT system DIVE SR/TTS Agent Modeling Framework Interaction Metaphor Reference Resolution Future Work Conclusion

Purposes of this paper Analyze the technical and design issues to combine a virtual world with a speech interface. Describe system architecture of the TALKING AGENT system.

Problems of Integration Speech Recognition : Limited vocabulary to gain accuracy. Language Understanding : Limited knowledge to maximize the understanding. Interaction Metaphor : Who does the user talk to? (Above questions are discussed in detail in the authors’ last paper “Speech Interface to Virtual Reality”.)

Innovation of this System Combining intelligent agent and speech interface to carry out specialized functions in the VR World. Functions have been implemented : Transporting objects Fetching objects Painting objects Increasing the size of objects

System Architecture

DIVE-Virtual Reality System DIVE(Distribute Interactive Virtual Environment) is a multi-user virtual environment. DIVE allow users and environment interact in real- time. DIVE contains a database composed of hierarchically organized objects.

DIME- DIVE Meeting Environment

Speech Recognition SR with limited pre-defined phrases promises good recognition performance. Using grammar to set constraint to search space. Using commercial SR-engine (Nuance).

Agent Modeling Framework High-level languages do not support complex symbolic computations. Oz is well suited for this purpose. Using ODI as interface between Oz and DIVE. The parent agent consists basic functions. We can define more specific agent by extend parent agent.

Agent Modeling Framework

Interaction Metaphor Direct manipulation -Personal Presence. Various metaphors for spoken interaction have been proposed. Proxy Divinity Telekinesis Interface Agent This system adopt the Proxy metaphor.

The DIVERSE System-Interface Agent

Addressing Agent Inside the user’s eye-sight Dialogue initiated by clicking on the agent. Outside the user’s eye-sight Phone agent-First press the phone agent then connect to remote agent

Feedback Given speech input,system should give the visual feedback to the user. If the agent listening or not? What is the feedback when talking to agent far away?

Reference Resolution Given some descriptions, the reference resolution engine maps them to object which user is referring to. Considerations Object focus. Property Perception. Discourse Modeling.

Robust Interaction When errors don’t matter User can view the results and current them by direct manipulation. Safety-critical applications Confirm user command. Clarifying incomplete or ambiguous commands.

Future Work Agent behavior should related to its previous action. Add mental components. Talking to agent by aura-driven. Evaluate this system with realistic scenario. Ex: virtual travel agency.

Conclusions Add a speech interface to VR-system. Using constraint SR to achieve high accuracy. Developing an appropriate metaphor. The agents modeled in this system provide specific functions in the virtual world.

Q & A

Paper Source McGlashan, S Speech Interfaces to Virtual Reality in Proceedings of the Second Conference on the Military Applications of Synthetic Environments and Virtual Reality, Stockholm, Sweden, 1995.