Opportunity Knocks: A Community Navigation Aid Henry Kautz Don Patterson Dieter Fox Lin Liao University of Washington Computer Science & Engineering

Outline 1. Need for navigation aids 2. Limitations of current devices 3. Advancing the State of the Art 4. Prototype: Opportunity Knocks 5. Future plans

The Need: Community Access for the Cognitively Disabled

Problems in Using Public Transportation Learning bus routes and numbers

Problems in Using Public Transportation Learning bus routes and numbers Transfers, complex plans

Problems in Using Public Transportation Learning bus routes and numbers Transfers, complex plans Recovering from mistakes

Result Need for extensive life-coaching Need for point-to-bus service

Result Need for extensive life-coaching Need point-to-bus service Isolation

Current GPS Navigation Devices Designed for drivers, not bus riders! Should I get on this bus? Is my stop next? What do I do if I miss my stop? Requires extensive user input Keying in street addresses no fun! Device decides which route is best Familiar route better than shorter one

Vision Can we build a system that... Automatically learns the daily pattern of the users transportation plans – no typing! Provides proactive help in carrying out the plans Helps user recover from mistakes Is compatible with todays transportation infrastructure

Approach User carries GPS cell phone System infers bus use from Position (near bus stop?) Velocity (on foot? in a vehicle?) Bus route information Over time system learns about user Important places Common transportation plans Mismatches = possible mistakes

Inferring Goals

Transportation Plans BA Goals work, home, friends, restaurant, doctors,... Trip segments Home to Bus stop A on Foot Bus stop A to Bus stop B on Bus Bus stop B to workplace on Foot Work

User Model System learns a probabilistic model of the users pattern of transportation use Robust even if... Data is missing Behavior varies User is predictable, but not rigid! x k-1 z k-1 zkzk xkxk m k-1 mkmk t k-1 tktk g k-1 gkgk

Goal Prediction

Error Detection: Missed Bus Stop

Prototype: Opportunity Knocks GPS camera-phone Knocks when opportunity to help Can I guide you to a likely destination? I think you made a mistake! This place seems important – would you photograph it?

Example

Example (continued)

Status Proof of concept prototype Can use machine learning to create a smarter, more useful personal navigation system for disabled persons Basic scientific contributions on predicting human behavior Best paper award at national computer science conference, AAAI

Next Steps User interface design, testing Audio Graphical Adaptive Develop & deploy Have begun discussions with METRO Seek commercial partnerships

END

Probabilistic Model: Dynamic Bayesian Network x k-1 z k-1 zkzk xkxk m k-1 mkmk Transportation mode x= GPS reading t k-1 tktk g k-1 gkgk Goal Trip segment

Error Detection Approach: model-selection Run two trackers in parallel Tracker 1: learned hierarchical model Tracker 2: untrained flat model Estimate the likelihood of each tracker given the observations

Novelty Detection