Upload
willow-garage
View
13.779
Download
2
Embed Size (px)
DESCRIPTION
Alex Sorokin's presentation on his work this summer building a server to annotate data collected from the Willow Garage PR2 robot. This server would submit jobs to people on Amazon's Mechanical Turk service to get labeled training data.
Citation preview
Summer 09 is over.
When did that happen?
What have I done?
• Tools for annotating data• Tools to for data access• Executive to capture data autonomously• Annotation lifting• PR2 challenge
Data annotation
New modes of annotation
• Several new ways toannotate data– Multiple fast
bounding boxes– Zoom-in interface for
detailed polygonalannotation
– Grouping images totrack people
– Free text labels
http://vm7.willowgarage.com/mt/good_results/bottles-p-2/p1/
New modes of annotation
• Several new ways toannotate data– Multiple fast
bounding boxes– Zoom-in interface for
detailed polygons– Grouping images to
track people– Free text labels
http://vm7.willowgarage.com/mt/good_results/bottles-p-2/p1/
New modes of annotation
• Several new ways toannotate data– Multiple fast
bounding boxes– Zoom-in interface for
detailed polygonalannotation
– Grouping images totrack people
– Free text labels
New modes of annotation
• Several new ways toannotate data– Multiple fast
bounding boxes– Zoom-in interface for
detailed polygonalannotation
– Grouping images totrack people
– Free text labels
Grading!
• Just filter out poorresults
Much easier thanannotating yourself
• New task to run gradingon MechTurk
• Random sample ofthumbnails
• Grading by worker• Track grading conflicts
http://vm7.willowgarage.com/mt/grading/thumbnails_random/prf-jul-08-L2-p/
Grading!
• Just filter out poorresults
Much easier thanannotating yourself
• New task to rungrading on turk
• Random sample ofthumbnails
• Grading by worker
http://vm7.willowgarage.com/mt/grading/thumbnails_random/prf-jul-08-L2-p/
Data annotation server
• Django server to manage dataannotation process
• Stable server• Consistent user interface• Scalable task architecture• First 5 users• ROS integration across 2+ different masters
– Image in (over HTTP post)– cv_mech_turk/ExternalAnnotation out
http://vm7.willowgarage.com/
Data access
• Bag server– Random access into a bag file– Replay of select messages
• Bag image view– Browsing images off a bag– Example client to the bagserver
Data lifting
• 2D to 3D correspondence• Annotation of laser clouds with object
labels, image coordinates an image IDs• Image storage server• Static map of all annotations• Whiteboard detector from static object
map
Camera “calibration”
• Create a virtual cameraand move it to align withthe laser cloud or robotmodel
• Fix major calibrationerrors
http://pr.willowgarage.com/wiki/camera_offsetter
before
after
Data collection executive
• High level executive based on M2 pythonexecutive– Move to waypoints– Position head– Record data– Record action state
• Select images when the “record” action wasactive and annotate them.
PR2 challenge
• Integrated ROS components into a completebehavior.
• Preemptable high level state machine:– Allows to switch between autonomy and teleop– Preempts child actions– External GUI to watch and control the executive
• Great prototype for hardware in the loop test
PR2 challenge• Integrated ROS
components into acomplete behavior.
• Preemptable high levelstate machine:– Allows to switch between
autonomy and teleop– Preempts child actions– External GUI to watch
and control the executive• Great prototype for
hardware in the looptest and learning
Thank you