Opportunity Knocks: A Community Navigation Aid Henry Kautz Don Patterson Dieter Fox Lin Liao...

Opportunity Knocks:A Community Navigation

Aid

Opportunity Knocks:A Community Navigation

Aid

Henry KautzDon Patterson

Dieter FoxLin Liao

University of WashingtonComputer Science & Engineering

OutlineOutline

1. Need for navigation aids2. Limitations of current devices3. Advancing the State of the Art4. Prototype: Opportunity Knocks5. Future plans

The Need: Community Access for the Cognitively

Disabled

The Need: Community Access for the Cognitively

Disabled

Problems in Using Public Transportation

Problems in Using Public Transportation

•Learning bus routes and numbers

Problems in Using Public Transportation

Problems in Using Public Transportation

•Learning bus routes and numbers

•Transfers, complex plans

Problems in Using Public Transportation

Problems in Using Public Transportation

•Learning bus routes and numbers

•Transfers, complex plans

•Recovering from mistakes

ResultResult

•Need for extensive life-coaching

•Need for point-to-bus service

ResultResult

•Need for extensive life-coaching

•Need point-to-bus service

•Isolation

Current GPS Navigation Devices

Current GPS Navigation Devices

Designed for drivers, not bus riders!Should I get on this bus?Is my stop next?What do I do if I miss my stop?

Requires extensive user inputKeying in street addresses no fun!

Device decides which route is “best”

Familiar route better than shorter one

VisionVisionCan we build a system that...

Automatically learns the daily pattern of the user’s transportation plans – no typing!Provides proactive help in carrying out the plansHelps user recover from mistakesIs compatible with today’s transportation infrastructure

ApproachApproach

User carries GPS cell phoneSystem infers bus use from

Position (near bus stop?)Velocity (on foot? in a vehicle?)Bus route information

Over time system learns about user

Important places Common transportation plans

Mismatches = possible mistakes

Inferring GoalsInferring Goals

Transportation Plans Transportation Plans

BA

Goalswork, home, friends, restaurant, doctor’s, ...

Trip segmentsHome to Bus stop A on FootBus stop A to Bus stop B on BusBus stop B to workplace on Foot

Work

User ModelUser Model

System learns a probabilistic model of the user’s pattern of transportation useRobust even if...

Data is missingBehavior varies

User is predictable, but not rigid!

xk-1

zk-1 zk

xk

mk-1 mk

tk-1 tk

gk-1 gk

Goal Prediction

Error Detectio

n: Missed

Bus Stop

Prototype: Opportunity KnocksGPS camera-phone“Knocks” when opportunity to help

Can I guide you to a likely destination?I think you made a mistake!This place seems important – would you photograph it?

ExampleExample

ExampleExample

ExampleExample

Example(continued

)

Example(continued

)

Example(continued

)

Example(continued

)

Example(continued

)

Example(continued

)

StatusStatus

Proof of concept prototypeCan use machine learning to create a smarter, more useful personal navigation system for disabled persons

Basic scientific contributions on predicting human behavior

Best paper award at national computer science conference, AAAI

Next StepsNext Steps

User interface design, testingAudioGraphicalAdaptive

Develop & deployHave begun discussions with METROSeek commercial partnerships

ENDEND

Probabilistic Model:Dynamic Bayesian

Network

Probabilistic Model:Dynamic Bayesian

Network

xk-1

zk-1 zk

xk

mk-1 mk Transportation mode

x=<Location, Velocity>

GPS reading

tk-1 tk

gk-1 gk Goal

Trip segment

Error DetectionError Detection

Approach: model-selectionRun two trackers in parallel

Tracker 1: learned hierarchical modelTracker 2: untrained flat modelEstimate the likelihood of each tracker given the observations

Novelty DetectionNovelty Detection