Project Hands.DVI - KickOff Meeting

Italian Institute of Technology, IIT, Genova, 24.01.2011

HANDS.DVI

Centro di Ricerca “E. Piaggio” – University of Pisa

The project HANDS.DVI will contribute to the developmentof a unified structure for programming and controlling robotic hands based on a number of fundamental primitives, andabstracting, to the extent possible, from the specifics of their kinematics, mechanical construction, sensor equipment

The project will propose a new DeVice-Independent programming and control framework for robotic HANDS

This project builds upon well-founded results on sensorimotor synergies developed in neuroscience and preliminary used in some special robotic applications. The project HANDS.DVI will exploit initial results of the research activity developed in the European Project The Hand Embodied (THE)

Introduction to project HANDS.DVI

The project THE is focused on the study of how the embodied characteristics of the human hand and its sensors, the sensorimotor transformations, affect and determine the learning and control strategies we use for such fundamental cognitive functions as exploring, grasping and manipulating

The project hinges about the conceptual structure and the geometry of such enabling constraints, or synergies: correlations in redundant hand mobility (motor synergies), correlations in redundant cutaneous and kinaesthetic receptors readings (multi-cue integration), and overall sensorimotor system synergies

These key ideas aim at advancing the state of the art in artificial systems for robotic manipulation, haptic, and neuroprosthetic interfaces

Motivation and objective of project THE

The mapping between the paradigmatic hand and the robotic hands can be obtained as the result of an optimal control strategy. A possible solution is to organize this function as two hierarchical levels:

in the high level stage only a few elements, the synergies, are controlled

in the low level the synergies are mapped to the actual kinematic variables of the robotic hand we want to control

Motivation and objective of project HANDS.DVI

The device independent mapping will be mediated by a model of an anthropomorphic robotic hand (paradigmatic hand) able to capture the idea of synergies in human hands. This model will be used to design the new control strategies for both anthropomorphic and non–anthropomorphic robotic hands

Force control of robotic grasping: the role of hand synergies in the optimal choice of grasping forces

Extending the paradigmatic hand model: modelling and control of biomorphic fingers using tendon actuation and rolling contacts

Mapping synergies between the paradigmatic hand and robotic hands with different kinematics: “DLR/HIT Hand II” (DLR) and “THE First Hand” (UNIPI) - direct kinematic mapping of the contact points or definition of an optimal function based on grasping forces

Synergies definition: the Motion Capture system and statistical methods for data analysis

Contents of the presentation

On the Role of Hand Synergies in the Optimal

Choice of Grasping Forces

Objective of the present study

Investigate analytically and numerically the role that limiting the DoFs of an hand via synergistic underactuation have on:

making possible precision/whole-hand grasps;

grasping force distribution;

grasp optimality evaluated with respect to grasp quality indices;

robustness with respect to physical characteristics (e.g., compliance).

Fundamental Equations

(object)

Equilibrium Equations

where:

(hand)

Congruence Equations

(Grasp matrix: maps contact forces to object wrench )

(maps contact forces to joint torques )

(Hand Jacobian: maps joint velocities to fingertip twists )

(maps the EE twist to twists of the object contact points )

(object)

(hand)

Contact compliance model

Penetration of the fingertip point into the object surface

j-th limb

object

Contact force (penalty formulation)

By properly juxtaposing the expression for all the contact forces

(contact stiffness matrix)

(contact compliance matrix)

Joint compliance model

Joint torque costitutive equation

By properly juxtaposing the expression for all the joints

(joint stiffness matrix)

joint reference positions

joint actual positions

(joint compliance matrix)

Soft Synergy Model

We introduce a synergy vector whose components can be interpreted as “high level knobs” controlling joint coordination patterns;

In the classical synergistic model:

the hand configuration is rigidly constrained to a dimensional sub-manifold of the joint configuration manifold. Therefore the virtual variations allowed are

Synergy matrix found through Principal Component Analysis (PCA) of recorded grasping experiments [Santello et al. 1998];

In the soft synergy model (proposed) synergistic displacements command joint reference positions

The actual hand configuration depends on hand-object interaction (hand adaptability to different grasping conditions).

(synergy matrix)

Soft-Synergy Concept

synergistic displacements (actual DoF)

coupling defines the synergy vector on references

actual joint position

Solution of the Grasping Problem with Synergies

For a general grasping system applying a wrench to the object, i.e.,

the contact force distribution is given by

where:

= particular solution (minimizes the strain energy)

( weighted right inverse of )

= active internal forces that can be commanded via synergistic displacements

If no underactuation takes place (independent joint control):

= passive (preload) internal contact forces

Internal Controllable Forces

Internal controllable forces directly dependent on synergistic variations

Defining it is possible to write also

(coefficient of the linear combination)

The above form turns out to be very convenient in optimizing the grasping forces.

Direct employment of is not efficient since it entails the explicit calculation of .

A more efficient algorithm can be employed based on the intersection of subspaces.

Qualitative analysis of Hand Embodiment

(increases)

(a plateau is reached)

1) Assume each has non-null projection onto , then

3) For the underactuated case, increase the no. of synergies engaged one by one (increase the number of columns in ):

For

For

Therefore if is “small” (2, 3), embodiment with an equal number of synergies does not endanger ability to exert the same grasping forces with great control simplification.

2) Consider a grasp for which indep. joint control bears

Grasping Force Optimization Problem

Given a grasp characterized by

weighted right inverse of the Grasp Matrix ;

basis for the subspace of internal controllable forces by employing a given set of synergies;

basis for the subspace on internal passive (preload) forces by employing a given set of synergies;

External wrench to be resisted

Grasping Force Optimization Problem (GFOP)

In the tests performed: , [N]

Problem feasibility force-closure

(normal components)

Paradigmatic Hand Model

Paradigmatic Hand Model features:

model of the Cyberglove employed in the work of Santello et al. (1998) for which high quality experimental data were recorder and made available to us;

the only “plant” for which postural synergies are rigorously defined;

15 DoFs

4 DoFs [T]: Rot., Abd., Metac., Interph.;

3 DoFs [I, R, L]: Abd., Metac., Prox. Interph.;

2 DoFs [M]: Metac., Prox. Interph.

limbs represented as cylinders;

anthropometric data relative to one subject employed in the experimental tests.

[T]

[I]

[M][R]

[L]

Numerical tests – Precision grasp

As a representative of precision grasp layout we consider the grasp of a cherry.

Grasp analysis (independent joint control):

3 contact points: [T] Interph., [I] & [M] Prox. Interph.;

contact constraints: PCWF;

;

; (internal)

; (internal, controllable)

; (internal, passive)

force-closure: yes!

Questions:

What is the effect of hand embodiment for this grasp?

Is the grasp still force-closure? For which synergy/synergies?

What is the effect of synergies on grasp optimality criteria?

active limbs

cherry.avi

Precision grasp – Contact Stiffness Var.ns

Variation of the cost function (2-norm of contact force vector) as more synergies are engaged into the grasp.

The grasp is force-closure already for the 1-st synergy.

Limited effect of Contact Stiffness variation, units [N/mm].

Variation of the cost function (2-norm of contact force vector) as more synergies are engaged into the grasp.

The grasp is force-closure already for the 1-st synergy.

Limited effect of Joint Stiffness variation, units [N/mm].

Precision grasp – Joint Stiffness Var.ns

Numerical tests – Power grasp

As a representative of power grasp layout we consider the grasp of an ashtray.

Grasp analysis (independent joint control):

11 contact points: [T] 3 cps; [I], [M], [R], [L] 2cps each;

contact constraints: PCWF;

;

; (internal)

; (internal, controllable)

; (internal, passive)

force-closure: yes!

Questions:

What is the effect of hand embodiment for this grasp?

Is the grasp still force-closure? For which synergy/synergies?

What is the effect of synergies on grasp optimality criteria?

ashtray.avi

Power grasp – Contact Stiffness Var.ns

Variation of the cost function (2-norm of contact force vector) as more synergies are engaged into the grasp.

The grasp is not always force-closure for the 1-st synergy.

Some effects of Contact Stiffness variation, units [N/mm].

Power grasp – Joint Stiffness Var.ns

Variation of the cost function (2-norm of contact force vector) as more synergies are engaged into the grasp.

The grasp is not always force-closure for the 1-st synergy.

Some effects of Joint Stiffness variation, units [N/mm].

Conclusions

Theoretical results:

a way has been proposed to solve the force decomposition and optimization problem for hands with synergies;

a way to map synergies from the kinematic domain (where have been observed) to the force domain (soft-synergies).

Numerical results:

force-closure properties of grasps strongly depends on which synergies are used to control the hand;

if the first few synergies (PCs) are not actively controlled, force-closure can be obtained only in many more DoFs;

no improvement beyond the first three synergies for precision grasp, continuous but small improvements in the whole-hand grasp case;

results are consistently robust with respect to different values of the stiffness parameters (uncertainty in their knowledge/control).

Modelling and Control of a Tendon Actuated Biomorphic Hand

Biomorphic reference scheme

Contact points move over the surfaces

[M. Gabiccini, A. Bicchi, GRSSP Workshop, 2008]

Object

Palm

Contact frames

Convective coords

FingerObject/finger contact

Total wrench on each limb

Resultant wrench on the -th body:

where :

is the wrench applied by the tendons expressed in body frame

is the wrench applied by the contact forces expressed in body frame

The vector must be chosen by means of an optimization routine to fulfill all the constraints. This can be formulated as a convex optimization problem.

Solution of the problem

We search a vector such that the system is in equilibrium and all the friction/tendon constraints are satisfied (tendon tension must be positive):

If we consider the complete system, we obtain:

the system must be in equilibrium under an external wrench , therefore:

where

is the particular solution;

is the homogeneous solution that not affect the overall system eq.

Control scheme

TRAJECTORY GENERATION

INITIAL CONDITIONS

COMPUTED TORQUE

SYSTEM DYNAMICS

OPTIMIZATION ROUTINE

CONTACT KINEMATICS

SOLUTION(TENDON TENSIONS)

System layout

The hand/object system consists of:

• Three-fingered hand grasping a ball;

• Each finger is a 4-d.o.f. serial manipulator with unilateral joints;

• The first joint (MCP) is a 2-d.o.f. hyperboloidic (saddle joint);

• The 2-nd and 3-rd joints are hinge joints with 1-d.o.f.

• There are 8 tendons which guaranteed the integrity of the system

Task

Task for the hand/object system:

• the object has to track a prescribed trajectory (e.g., G follows ellipse in vert. plane);

• integrity of the hand/object contacts and integrity of the finger limbs have to be

guaranteed by proper co-contraction forces in the tendons;

• integrity meaning:

• contact forces within friction cone boundaries;

• max/min contact force below/above prescribed limits;

• max/min tendon tensions below/above prescribed limits (e.g., tendons

cannot pull)

Videos of simulations

Move the Center of Gravity of the ball along an ellipse in the vertical plane

Rotate the ball around a vertical axis

Move the Center of Gravity of the ball up and down

16 DoFs Robotic Hand Prototype

Hand Features

16 DoFs Five fingered hand Tendon driven One rotoidal spring for each joint

(only one tendon is needed) Low cost Human sized Human-like movements Servo controlled

Kinematic model of the “First Hand” is very similar to the paradigmatic hand model

Hand Schematics

PHALANGES

Index, Middle, Ring and LittleABDUCTIONS

PALM

Proximal phalanges are independently controlled w.r.t intermediate and distal phalanges

Intermediate and distal phalanges are dependently controlled

THUMB

Phalanges are independently controlled

Middle finger doesn’t have the abduction DOF

Thumb and Little abductions are independently controlled

Two rotoidal joints to close palm on itself

Grasping Trials - 1

Grasping Trials - 2

Hand Synergies

First Synergy Second Synergy

first.mp4 second.mp4

Hand Posture Tracking:Motion Capture System

Motion capture system:active leds, impulse cameras, server computer, led controllers, led base station

Easy to reconfigure and calibrate

Fast and reliable data transmission (up to 480 Hz)

Possibility to connect several clients and integrate measurements of grasping forces

Problems to overcome:

Optimal marker positioning (phalanges or joints) Kinematic hand model that best fits experimental data Observability, model parameters estimation and posture reconstruction using nonlinear filters (EKF) Statistical methods for data analysis

Experimental Setup

Statistical Methods

Modelling grasping movements using statistical methods: Hidden Markov Models and Gaussian Mixture Models

Two Objectives:

Task replication using regression techniques: Gaussian Mixture Regression

Tasks classification from continuous movements while grasping different objects

Problem of data dimension reduction (15 dofs for the paradigmatic hand) using linear and nonlinear PCA

Hand posture reconstruction with inaccurate data using a-priori information and its application to a low cost glove (sensory synergies).

Thanks for your attention