Optical Soft Skin For Soft Object Manipulation

Akihiko Yamaguchi

Robotics Institute, Carnegie Mellon Universityhttp://akihikoy.net/

http://reflectionsintheword.files.wordpress.com/2012/08/pouring-water-into-glass.jpg

http://schools.graniteschools.org/edtech-canderson/files/2013/01/heinz-ketchup-old-bottle.jpg

http://old.post-gazette.com/images2/20021213hosqueeze_230.jpg

http://img.diytrade.com/cdimg/1352823/17809917/0/1292834033/shampoo_bottle_bodywash_bottle.jpg http://www.nescafe.com/

upload/golden_roast_f_711.png

For Robust Behavior Generation

A library of skills (primitives)

Strategies to use skills (behavior graphs)

Planning/learning methods for adjusting skill parameters and selecting skills (Graph-DDP)

Richer sensing

9

Sensing In Pouring

We used mostly vision (RGB, RGB-D, Depth)Container posesAmount poured into receiving container Some robot pouring use external weight scale

E.g. Tamosiunaite, Neme, Ude, Worgotter, 2011: Learning to pour with a robot arm combining goal and shape learning for dynamic movement primitives, Robotics and Autonomous Systems.

Flow of liquid/particle

We ignored:Amount in source container Some robot pouring use force-torque sensor

E.g. Rozo, Jimenez, Torras, 2013: Force-based robot learning of pouring skills using parametric hidden Markov models, RoMoCo.

Tapping detection

10

Tactile Sensing Is Useful In Soft Object Manipulation

Contact/collision detectionTapping a container with a finger

Contact force estimationGrasping strength (e.g. banana, egg)

Perceiving occluded stateCutting food with a structure (e.g. seed of peach)

Detecting cutting board

Perceiving state through toolsCutting with a knife, Calligraphy with a brush

15

Argus Project

16

Hyakumehttps://ja.wikipedia.org/wiki/百目

Argus Panopteshttp://www.greekmythology.com/Myths/Creatures/Argus_Panoptes/argus_panoptes.html

Optical Skin for Fingers

High resolution force field

Multimodal

Force field

Proximity vision

Low-cost and easy to manufacture

Physically robust

17

Design and Installation

18

Physical Robustness

19

External force is appliedonly to skinnot to imagers

Low-Cost and Easy to Manufacture

20

Silicone and Its Processing Matter

21

Force Estimation

Marker (blob) detection

Tracking: comparing initial and current marker position and size

22

23

Proximity VisionProximity vision makes in-hand movement detection easier

e.g. Deformation of grasp, Slip

24

Application To Cutting

25

Grasping banana piece

28

Related Work

29

OptoForcehttp://optoforce.com/technology/

Fingertip sensor for NAIST handhttp://robotics.naist.jp/research/naisthandhp_enadj/NAIST-Hand.htm

TacTiphttp://www.brl.ac.uk/researchthemes/medicalrobotics/tactip.aspx

GelSighthttp://news.mit.edu/2014/fingertip-sensor-gives-robot-dexterity-0919

Discussion - How To Make Smaller?

Cameras could be small

NanEye camera

Lens? Custom lens is expensive?

Processing units

30

http://www.slideshare.net/WenQianPeng/magic-finger-39687355

https://techxplore.com/news/2016-04-flexible-camera-radically-approach-imaging.html

Summary: Optical Skin Is Useful In Manipulation

Force filed estimation

Autonomous cutting control

Detecting too-strong force, cutting board

Proximity vision

Slip detection

31

Future Work

Stereo vision for each fingerMore manipulation tasks (grasping, peeling, pouring (tapping), etc.)Different types of sensorsWhole body & camera network

32