Upload
rosalind-williamson
View
218
Download
3
Tags:
Embed Size (px)
Citation preview
Agile Robotics Agile Robotics Program ReviewProgram Review
AR TeamAR TeamMIT, Lincoln Lab, Draper Lab, BAEMIT, Lincoln Lab, Draper Lab, BAE
August 8, 2008August 8, 2008
Agile Robotics TeamAgile Robotics Team
Albert Huang
Luke Fletcher
Matthew Walter
Lenny ParitskyTroy JonesMitch Leammukda
Steve ProulxRob Truax
Missy Cummings
Jake Crandall
Birsen Donmez
Nicholas RoyRuss Tedrake
Edmund Williams
Ed Corbett
Mike Boulet
Bryt BradleyNira Manokharan
Seth Teller Matt Antone
Emma Brunskill
Geoff Carrigan
Mofe Uku
Jeong hwan JeonEmilio Frazzoli Jon How
Scott CyphersStephanie SeneffTara Sainath Lee HetheringtonAlex Gruenstein
Howard ShrobeAndrew CorreaJim GlassRandy Davis
Platformbuildup
Safety& intent
Situationalawareness
Planning& control
Supervisorinterface
Programsupport
Matthew Walter
Seth TellerMitch LeammukdaLenny Paritsky
Jim Liu
Paul Pepin
Brandon Luders
Daniela RusNicholas Roy
Whole-SSAsimulation
John Leonard
Ryne Barry
USA LIA
DDRE
CASCOM
ARL
AgendaAgenda0900 – 0930: Arrive MIT Kiva room (32G-449)
0930 – 0945: Informal introductions
0945 – 1030: Summary goals and status
1030 – 1045: Break, walk to demonstration venues
1045 – 1145: Demonstrations (Hangar, Holodeck, Kiva)
1200 – 1300: Lunch (Highlights of other MIT robotics)
1300 – 1415: Technical briefings
1415 – 1430: Break
1430 – 1530: Technical briefings
1530 – 1630: Feedback and discussion
1630: Main group adjourns
1630 – 1700: Program management discussion
1700: Program review adjourns
Administrative NotesAdministrative Notes
• Name placards
• Room security
• Kiva conference room
• Bathrooms
• Handouts / briefing charts
AgendaAgenda0900 – 0930: Arrive MIT Kiva conference room (32G-449)0930 – 0945: Informal introductions0945 – 1030: Summary goals and status 1030 – 1045: Break, walk to demonstration venues1045 – 1145: Demonstrations (Hangar, Holodeck, Kiva) 1200 – 1300: Lunch (Highlights of other MIT robotics)1300 – 1415: Technical briefings1415 – 1430: Break 1430 – 1530: Technical briefings1530 – 1630: Feedback and discussion1630: Main group adjourns1630 – 1700: Program management discussion1700: Program review adjourns
Project GoalsProject Goals
• Build upon foundation of DARPA Challenge – Real-time perception, planning, and control– Avoid brittle aspects (prior map, GPS) of DGC
• Collaborate with LIA, CASCOM– Develop “Agile Robotics” autonomous forklift
• Introduce new mobile manipulation aspects – Whole-pallet manipulation (Year 1)– Sub-pallet manipulation (Years 2-3)
• Demonstrate prototype at end of Year 1
• Migrate capability to Army vehicle Years 2,3
Summary StatusSummary Status• Tackling problem along four fronts, in parallel:
– Requirements analysis: LIA, CASCOM, Ft. Campbell– Simulation studies for sensing, dataflow, throughput– Mockup experiments, drive-by-wire elements– Full-scale prototype development, data collection
• At present, substantially ahead of schedule– Several September, December milestones achieved– Demonstrated components in simulation, on mockup– (Partially) working full-scale prototype by fall 2008
• Anticipated fruitful directions for Years 2, 3– Improved manipulation, reasoning, sensing, scaling
1. Autonomous vehicle (flatbed truck) arrives at reception area with two pallets2. Autonomous forklift meets arriving truck at reception area3. Forklift unloads one pallet, transports it to specified outdoor bulk lot ASL4. Forklift unloads second pallet, moves it to queueing area in pickup zone5. Later, forklift loads queued pallet onto indicated truck in pickup zone6. Autonomous truck departs pickup zone
Bulk lot
PickupReception
1
Scenario: A Commandable ForkliftScenario: A Commandable Forklift(Notional SSA Layout)
1. Autonomous vehicle (flatbed truck) arrives at reception area with two pallets2. Autonomous forklift meets arriving truck at reception area3. Forklift unloads one pallet, transports it to specified outdoor bulk lot ASL4. Forklift unloads second pallet, moves it to queueing area in pickup zone5. Later, forklift loads queued pallet onto indicated truck in pickup zone6. Autonomous truck departs pickup zone
Bulk lot
PickupReception
Forklift
Supervisor
2
Demonstration ScenarioDemonstration Scenario
Demonstration ScenarioDemonstration Scenario
Bulk lot
PickupReception
ASL
3
1. Autonomous vehicle (flatbed truck) arrives at reception area with two pallets2. Autonomous forklift meets arriving truck at reception area3. Forklift unloads one pallet, transports it to specified outdoor bulk lot ASL4. Forklift unloads second pallet, moves it to queueing area in pickup zone5. Later, forklift loads queued pallet onto indicated truck in pickup zone6. Autonomous truck departs pickup zone
Demonstration ScenarioDemonstration Scenario
Bulk lot
PickupReception
4
QA
1. Autonomous vehicle (flatbed truck) arrives at reception area with two pallets2. Autonomous forklift meets arriving truck at reception area3. Forklift unloads one pallet, transports it to specified outdoor bulk lot ASL4. Forklift unloads second pallet, moves it to queueing area in pickup zone5. Later, forklift loads queued pallet onto indicated truck in pickup zone6. Autonomous truck departs pickup zone
Demonstration ScenarioDemonstration Scenario
Bulk lot
PickupReception
5
1. Autonomous vehicle (flatbed truck) arrives at reception area with two pallets2. Autonomous forklift meets arriving truck at reception area3. Forklift unloads one pallet, transports it to specified outdoor bulk lot ASL4. Forklift unloads second pallet, moves it to queueing area in pickup zone5. Later, forklift loads queued pallet onto indicated truck in pickup zone6. Autonomous truck departs pickup zone
Demonstration ScenarioDemonstration Scenario
Bulk lot
PickupReception
6
1. Autonomous vehicle (flatbed truck) arrives at reception area with two pallets2. Autonomous forklift meets arriving truck at reception area3. Forklift unloads one pallet, transports it to specified outdoor bulk lot ASL4. Forklift unloads second pallet, moves it to queueing area in pickup zone5. Later, forklift loads queued pallet onto indicated truck in pickup zone6. Autonomous truck departs pickup zone
Design RequirementsDesign Requirements• Develop plausible prototype capability
– Compatible with existing Army platforms– Safe, acceptable to Army personnel– Capable of locating, moving, placing pallets– Affordable (acceptable cost increment)– Rugged (operates outdoors on uneven terrain)
Principal DeliverablesPrincipal Deliverables
• Prototype Demonstration (March 2009)– Single voice-, gesture-commandable forklift– Whole-pallet manipulation: truck/ground to ground/truck
• Documentation of effort results– Detailed drive-by-wire conversion procedure– Study of task requirements, sensor choices– Report on system architecture and algorithms
• Plans for continued capability development– Technology transition plan (Years Two and Three)– Extend capability along several dimensions
• Fine-grain manipulation; higher-level reasoning; multiple forklifts and supervisors; adverse environments; GPS-denial
Working TimelineWorking Timeline
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr
Y2 Year 1
Requirements analysis, system & interface design, safety, user testing
Simulation development
Mockup forklift
Full-scale prototype
Capability migration
Visit to Visit to Visit to CASCOM Incursion Seamless Shouted Testing & LIA CASCOM Ft. Campbell Visit to MIT detection handoff warnings validation
Port DGC Speech Gesture Pallet Situational Whole-SSA codebase integration support detection awareness simulation
Drive-by-wire Sensor Planning Pallet prototyping placement and control engagement
Drive-by-wire Sensor Planning Pallet Pallet Vehicle prototyping placement and control engagement mobility interaction
Lincoln FMTV Laboratory studies
KickoffMeeting
Program Review
Year 1 Demo
(Rented forklift) (Purchased forklift)
Program Review
Today
Preliminaries: Drive-by-Wire Platform Development
Prototype AutonomousCapability Development
Milestone Status Milestone Status ((= completed)= completed)
• Capability 0.0: 30 Jun 08 Analyze and summarize existing SSA operational practices Demonstrate low-fidelity simulation of terrain, pallets, and trucks Develop prototype speech, gesture interface to command forklift
• Capability 0.1: 30 Sep 08 Build a partially-actuated mockup forklift
• Gather data on pallet approach, pallet and slot detection / localization– Develop prototype interface, perception, planning, control algorithms– Demonstrate mockup working in a real-world scenario to identify, localize,
select, engage, lift, transport, and place one pallet at a time as directed• Capability 0.2: 31 Dec 08
– Convert a stock manual forklift to drive-by-wire control Identify model to purchase Lease example of target forklift• Procure suitable sensor suite and compute server• Instrument leased forklift with sensor suite• Transfer drive-by-wire elements to prototype forklift
– Team members complete required OSHA forklift training– Understand degrees of freedom, sensing and control of forklift– Develop functional interface, perception, planning, control algorithms
• Capability 0.3: 31 Mar 09– Prototype demonstration, venue to be determined (tentatively Ft. Belvoir)
Working TimelineWorking Timeline
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr
Y2 Year 1
Requirements analysis, system & interface design, safety, user testing
Simulation development
Mockup forklift
Full-scale prototype
Capability migration
Visit to Visit to Visit to CASCOM Incursion Seamless Shouted Testing & LIA CASCOM Ft. Campbell Visit to MIT detection handoff warnings validation
Port DGC Speech Gesture Pallet Situational Whole-SSA codebase integration support detection awareness simulation
Drive-by-wire Sensor Planning Pallet prototyping placement and control engagement
Drive-by-wire Sensor Planning Pallet Pallet Vehicle prototyping placement and control engagement mobility interaction
Lincoln FMTV Laboratory studies
KickoffMeeting
Program Review
Year 1 Demo
(Rented forklift) (Purchased forklift)
Program Review
Today
Preliminaries: Drive-by-Wire Platform Development
Prototype AutonomousCapability Development
Understanding the TaskUnderstanding the Task• MIT team visits LIA, CASCOM, Ft. Campbell• Collect, study relevant Dept. of Army pamphlets• CASCOM forklift operator visits MIT• OSHA forklift operator certification for team members
DGC System ArchitectureDGC System Architecture
Perception
NavigatorMDF
Goal
Trajectory
Steer, gas, brake
Vehicle states
Drivable surface, Lane markings, Obstacles,
Traffic vehicles
Local mapDrivableSurface, Hazards
SituationalPlanner
Vehicle Controller
Landrover LR3
Vehicle State Estimator
SensorsSensorsSensorsSensorsSensorsSensorsSensorsRNDF
AR System ArchitectureAR System Architecture
• Existing elements (from DGC codebase)– Sensor handlers and terrain perception– Ethernet and CANbus networks
• Novel elements– Whole-SSA environment model– Interpretation of supervisor speech and gestures– Detection of trucks, pallets, loads, pallet slots– Forklift mast planning and control– Seamless autonomy handoff, return– Shouted warning detection– Visible and audible apparent intent
AR System ArchitectureAR System Architecture
Drive-by-Wire Modifications
Mobility and ManipulationPlanning and
Control
Sensing andSituationalAwareness
Supervisor
Speech,Gestures
Failure: Diagnostic Information
Success: Confirming InformationTo bystanders,via annunciators
Standard Forklift Forklift
Operator
SupervisorInterface/
InterpreterTablet
AR System ArchitectureAR System Architecture
Perception
Navigator
Goal
Trajectory
Steer, gas, brake, transmission, parking brake, mast
Vehicle state
Local mapTerrain,People,Trucks,Pallets
SituationalPlanner
Vehicle, mast Controller
Forklift
Vehicle State Estimator
SensorsSensorsSensorsSensorsSensorsSensorsSensorsSupervisorInterface/
Interpreter
Safety:Incursions,
ShoutedWarnings
Speech &Gesture
Interpretation
SensorsSensorsSensorsSensorsSensorsSensorsMicrophones
Whole-SSAEnvironment
Model
Visible, AudibleApparent Intent
(Novel elements(Novel elements shown in shown in red boxesred boxes))
Sensor data
Working TimelineWorking Timeline
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr
Y2 Year 1
Requirements analysis, system & interface design, safety, user testing
Simulation development
Mockup forklift
Full-scale prototype
Capability migration
Visit to Visit to Visit to CASCOM Incursion Seamless Shouted Testing & LIA CASCOM Ft. Campbell Visit to MIT detection handoff warnings validation
Port DGC Speech Gesture Pallet Situational Whole-SSA codebase integration support detection awareness simulation
Drive-by-wire Sensor Planning Pallet prototyping placement and control engagement
Drive-by-wire Sensor Planning Pallet Pallet Vehicle prototyping placement and control engagement mobility interaction
Lincoln FMTV Laboratory studies
KickoffMeeting
Program Review
Year 1 Demo
(Rented forklift) (Purchased forklift)
Program Review
Today
Preliminaries: Drive-by-Wire Platform Development
Prototype AutonomousCapability Development
Simulation StudiesSimulation Studies
• Experiments with:– Sensor placement– Truck, pallet, slot detection– SSA structure and layout– Planning, control algorithms– Supervisor interface & bi-directional dataflow
Whole-SSA SimulationWhole-SSA Simulation
• Simulated whole-SSA operation– Multiple forklifts, supervisors, customers
• Predict task throughput, wait times for a variety of operating parameters– Speed, # trucks, # supervisors, failure rate &c.
Working TimelineWorking Timeline
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr
Y2 Year 1
Requirements analysis, system & interface design, safety, user testing
Simulation development
Mockup forklift
Full-scale prototype
Capability migration
Visit to Visit to Visit to CASCOM Incursion Seamless Shouted Testing & LIA CASCOM Ft. Campbell Visit to MIT detection handoff warnings validation
Port DGC Speech Gesture Pallet Situational Whole-SSA codebase integration support detection awareness simulation
Drive-by-wire Sensor Planning Pallet prototyping placement and control engagement
Drive-by-wire Sensor Planning Pallet Pallet Vehicle prototyping placement and control engagement mobility interaction
Lincoln FMTV Laboratory studies
KickoffMeeting
Program Review
Year 1 Demo
(Rented forklift) (Purchased forklift)
Program Review
Today
Preliminaries: Drive-by-Wire Platform Development
Prototype AutonomousCapability Development
Mockup ForkliftMockup Forklift• Enables experiments with:
– Forklift elements to be converted to Drive-by-Wire:• Parking brake and latch release• Pedal brake• Steering wheel and column• Mast raise, lower, tilt• Mast tine spread and side-shift• Rapid transfer to prototype forklift when available
– Sensing, planning and control algorithms for:• Rear-wheel steering• Pallet detection, localization, approach• Pallet engagement (tine insertion), transport, placement• Unloaded, loaded mobility• Incursion detection (operator override)
Working TimelineWorking Timeline
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr
Y2 Year 1
Requirements analysis, system & interface design, safety, user testing
Simulation development
Mockup forklift
Full-scale prototype
Capability migration
Visit to Visit to Visit to CASCOM Incursion Seamless Shouted Testing & LIA CASCOM Ft. Campbell Visit to MIT detection handoff warnings validation
Port DGC Speech Gesture Pallet Situational Whole-SSA codebase integration support detection awareness simulation
Drive-by-wire Sensor Planning Pallet prototyping placement and control engagement
Drive-by-wire Sensor Planning Pallet Pallet Vehicle prototyping placement and control engagement mobility interaction
Lincoln FMTV Laboratory studies
KickoffMeeting
Program Review
Year 1 Demo
(Rented forklift) (Purchased forklift)
Program Review
Today
Preliminaries: Drive-by-Wire Platform Development
Prototype AutonomousCapability Development
Rental, Purchase ForkliftRental, Purchase Forklift• Enables experiments with:
– Sensor placement and sensor data under pallet approach
– Electrical interfaces to existing (electrical) DBW elements
– Characterization of suspension dynamics when under load
– Annunciator visibility/audibility • Purchase forklift as base platform
for eventual demonstration prototype– Toyota 8FGU-15 lift truck ordered May 19, 2008 – Toyota arranged expedited delivery: July 23, 2008 – Transfer actuated mechanisms from mockup in August – Begin integrated outdoor experiments August 2008
Principal Research AdvancesPrincipal Research Advances
• Operation in semi-structured environment
• Hierarchical task-level autonomy
• Voice/gesture interface from bot’s-eye-view
• Seamless autonomy handoff & return
• Annunciation of apparent intent
• Detection of shouted warnings
Semi-Structured EnvironmentSemi-Structured Environment
Idea: Give forklift a narrated, guided tour!
Manually-drivenforklift path
Operator utteranceand / or gesture
Hierarchical Task-Level AutonomyHierarchical Task-Level Autonomy
• Not tele-operated; Not fully autonomous
• But: at a useful operating point in between– Summon forklift to working area– Direct it toward one truck (of several)– Direct it toward one aspect (of several)– Direct it toward one pallet (of several)– Help it localize pallet, slots (if necessary)– Direct it to destination (bulk lot, issue area etc.)
• Rich, incremental path to full autonomy
Voice and Gesture InterfaceVoice and Gesture Interface
• PDA interface for commanding forklift
• Interpretation of supervisor’s speech
• Supervisor uses stylus gestures to: – Summon and
direct forklift – Confirm or edit
motion paths – Indicate which
pallet, slots areto be engaged
Autonomy Hand-off and ReturnAutonomy Hand-off and Return
• Fundamental design constraint:– Forklift pauses if human approaches– Relinquishes autonomy if s/he enters cabin– Operates indistinguishably from manual forklift– Returns to autonomous mode after human exit
• Perhaps after an explicit go-ahead from human
• Six distinct, independent safety layers– Described later in briefing
• Challenging implications for planning layer– Described later in briefing
Apparent IntentApparent Intent
• Visible/Audible mode annunciators – Words, symbols, LEDs on cabin exterior– Speakers for audible annunciation– Redundant modes for safety
• Robot announces when:– It is about to move– It is stuck and needs help– It is paused, awaiting
supervisor’s instructions
DemonstrationsDemonstrations0900 – 0930: Arrive MIT Kiva conference room (32G-449)0930 – 0945: Informal introductions0945 – 1030: Summary goals and status 1030 – 1045: Break, walk to demonstration venues1045 – 1145: Demonstrations (Hangar, Holodeck, Kiva) 1200 – 1300: Lunch [Highlights of other MIT robotics]1300 – 1415: Technical briefings1415 – 1430: Break 1430 – 1530: Technical briefings1530 – 1630: Feedback and discussion1630: Main group adjourns1630 – 1700: Program management discussion1700: Program review adjourns