16
Thrust IIA: Environmental State Estimation and Mapping Dieter Fox (Lead) Nicholas Roy URI 8 ickoff Meeting 2007

Thrust IIA: Environmental State Estimation and Mapping

  • Upload
    sani

  • View
    25

  • Download
    0

Embed Size (px)

DESCRIPTION

Thrust IIA: Environmental State Estimation and Mapping. Dieter Fox (Lead) Nicholas Roy. MURI 8 Kickoff Meeting 2007. Task Objective: Human-Centered Maps. Observation: Automatic map-building (SLAM) is solved sufficiently well Goal : Describe environments by higher-level concepts: - PowerPoint PPT Presentation

Citation preview

Page 1: Thrust IIA: Environmental State Estimation and Mapping

Thrust IIA: Environmental State Estimation and Mapping

Dieter Fox (Lead)

Nicholas Roy

MURI 8Kickoff Meeting 2007

Page 2: Thrust IIA: Environmental State Estimation and Mapping

Task Objective: Human-Centered Maps

Observation: Automatic map-building (SLAM) is solved

sufficiently well

Goal: Describe environments by higher-level concepts:

Places (room, hallway, street, walkway, parking lot, …)

Objects (tree, person, building, car, wall, …)

Key challenges:

Estimating concept types is mostly a discrete problem

Complex features and relationships

MURI 8 Kickoff Meeting 2007

University of Washington

Page 3: Thrust IIA: Environmental State Estimation and Mapping

Existing Technology

Human-centered mapping requires integration of high-dimensional, continuous

features from multi-modal sensor data reasoning about spatial and temporal

relationships

Conditional Random Fields provide extremely flexible probabilistic framework for learning and inference

MURI 8 Kickoff Meeting 2007

University of Washington

Page 4: Thrust IIA: Environmental State Estimation and Mapping

Conditional Random Fields

Discriminative, undirected graphical model

Introduced for labeling sequence data to overcome weaknesses of

Hidden Markov Models [Lafferty-McCallum-Pereira: ICML-01]

Applied successfully to

Natural language processing [McCallum-Li: CoNLL-03], [Roth-Yih: ICML-05]

Computer vision [Kumar-Hebert: NIPS-04], [Quattoni-Collins-Darrel: NIPS-05]

Robotics [Limketkai-Liao-Fox: IJCAI-05], [Douillard-Fox-Ramos: IROS-07]

MURI 8 Kickoff Meeting 2007

University of Washington

Page 5: Thrust IIA: Environmental State Estimation and Mapping

Conditional Random Fields

Directly models conditional probability p(x|z)

(instead of modeling p(z|x) and p(x), and using Bayes rule to infer p(x|z)).

No independence assumption on observations needed!

k 1xHidden states x

Observations z

MURI 8 Kickoff Meeting 2007

University of Washington

Page 6: Thrust IIA: Environmental State Estimation and Mapping

Online Object Recognition

MURI 8 Kickoff Meeting 2007 [Douillard-Fox-Ramos: IROS-07, ISRR-07]

Page 7: Thrust IIA: Environmental State Estimation and Mapping

From Laser Scans to CRFs

MURI 8 Kickoff Meeting 2007

Object type of laser beam 1

Shape and appearance

Object type of laser beam 2

Object type of laser beam 3

Object type of laser beam 4

Object type of laser beam n

Shape and appearance

Page 8: Thrust IIA: Environmental State Estimation and Mapping

Temporal Integration

MURI 8 Kickoff Meeting 2007

…………

k-2 k-1 k k+1

Taking past and future scans into account can improve labeling accuracy.

Match consecutive laser scans using ICP.

Associated laser points are connected in CRF.

Can perform online filtering or offline smoothing via BP.

Page 9: Thrust IIA: Environmental State Estimation and Mapping

Example Trace: Car vs. Others

MURI 8 Kickoff Meeting 2007

Trained on 90 labeled scans

Inference via filtering in CRF

Page 10: Thrust IIA: Environmental State Estimation and Mapping

7 Class Example Labeling

MURI 8 Kickoff Meeting 2007

Page 11: Thrust IIA: Environmental State Estimation and Mapping

Proposed Technical Advances

Integrate recognition results into maps Improve results by leveraging web training

data and high level object detectorss Add object types suited for target scenario Improve CRF training

MURI 8 Kickoff Meeting 2007

University of Washington

Page 12: Thrust IIA: Environmental State Estimation and Mapping

Situation Awareness via Wearable Sensors

Microphone Camera

Light sensors

2 GB SD card

Indicator LEDs

Records 4 hours of audio, images (1/sec), GPS, and sensor data (accelerometer, barometric pressure, light intensity, gyroscope, magnetometer)

Page 13: Thrust IIA: Environmental State Estimation and Mapping

Soldier Activity Recognition

Automatic generation of mission summaries Motion type (linger, walk, run, drive, …) Environment (inside, outside building) Events (conversations, marked via keyword)

Technical challenges High-dimensional, continuous observations /

features Different data rates: (1 Hz - 256 Hz) Getting labeled training data Different persons / environments

MURI 8 Kickoff Meeting 2007

Page 14: Thrust IIA: Environmental State Estimation and Mapping

Data Visualization / Summarization

MURI 8 Kickoff Meeting 2007

GPS traces

Image sequence(currently in car)

Timeline of soldier activities

Page 15: Thrust IIA: Environmental State Estimation and Mapping

Milestones

Goals: Real time wearable interface on cell phone Data sharing among soldiers and robots Real time display on remote laptop

MURI 8 Kickoff Meeting 2007

Page 16: Thrust IIA: Environmental State Estimation and Mapping

Milestones

Year 1: Real time data sharing between wearable sensor

platforms Integration of object recognition into mapping

Year 2: Real time data sharing between soldiers, robots,

and remote laptop Detection of specific soldier states / activities

(moving, incapacitated, ...)