Helping Hand: An Anatomically Accurate Inverse Dynamics ...karan/pdf/HandPaper_SCA.pdf · 2 Winnie Tsang, Karan Singh, Eugene Fiume / Helping Hand: An Anatomically Accurate Inverse

Eurographics/ACM SIGGRAPH Symposium on Computer Animation (2005), pp. 1–10K. Anjyo, P. Faloutsos (Editors)

Helping Hand: An Anatomically Accurate Inverse DynamicsSolution For Unconstrained Hand Motion

Winnie Tsang, Karan Singh, Eugene Fiume

Department of Computer Science, University of Toronto{wtsang,karan,elf}@dgp.toronto.edu

Abstract

We present a realistic skeletal musculo-tendon model of the human hand and forearm. The model permits directforward dynamics simulation, which accurately predicts hand and finger position given a set of muscle activations.We also present a solution to the inverse problem of determining an optimal set of muscle activations to achieve agiven pose or motion; muscle fatigue, injury or atrophy can also be specified, yielding different control solutionsthat favour healthy muscle. As there can be many (or no) solutions to this inverse problem, we demonstrate howthe space of possible solutions can be filtered to an optimal representative. Of particular note is the ability of ourmodel to take a wide array of joint interdependence into account for both forward and inverse problems. Givenkinematic postures, the model can be used to validate, predict or fill in missing motion and improve coarselyspecified motion with anatomic fidelity. Lastly, we address the visualization and understanding of the dynamicallychanging and spatially compact musculature using various interaction techniques.

1. Introduction

The human hand is an exquisitely flexible biomechanical de-vice that has evolved from a multiplicity of function overmillennia, beginning with our move to bipedalism, the use ofhand-held implements, and the development of verbal, writ-ten, gestural and musical communication [MM00]. As withmany other marvels of evolution, it is easy to ignore the re-markable utility of our hands–until, that is, we sustain eventhe smallest injury that inhibits their function. The complex-ity of the human hand makes the construction of accuratevisual and physical models an extremely challenging goal.Such a model is, however, necessary in exacting disciplinessuch as surgery, the study of hand pathologies, biomechan-ics, education and paleo-archaeology, and they can greatlybenefit expressive character animation.

The exploration of dynamic hand function requires an ac-curate model that is visually, biomechanically and anatomi-cally valid. As work in computer graphics takes on a deepercollaborative role with fields such as anatomy, surgery, edu-cation, archeology and biology, the outcomes of our physi-cal models and simulations will need to be more than visu-ally persuasive. For example, the pathology of a repetitive

stress injury likely has and will require a visual manifesta-tion; however, predicting an injury, modeling therapies, orsimulating preventative measures, all require a hand modelthat attends to anatomical and physical validity. That said,a hand model need not be an exact physical replica. This isunlikely to be necessary even if it were possible. Instead werequire a synthetic biomechanical model of the hand that canbe used to test hypotheses and to operate in a manner that isconsistent with the observable physical expectations of handfunction. Our goal is to choose the simplest model necessaryto meet such task requirements. There are further bounds tophysical modeling. A strictly physical model is for exampleunlikely to predict pain, or the preferred hand position for aguitar chord, or the accepted fingering of a piano piece. Weinstead contribute a robust model that accurately estimatescontraction impulses and physical parameters required by areal human hand to assume a given posture or to perform acertain task, and to allow domain experts on hand functionfrom various disciplines to work with these models and drawconclusions based on their expertise and needs.

The ability to simulate and animate biomechanical hand mo-tion, and to explore muscle activations needed for a handto perform tasks is the basis of our contribution. We cannot

submitted toEurographics/ACM SIGGRAPH Symposium on Computer Animation (2005)

2Winnie Tsang, Karan Singh, Eugene Fiume / Helping Hand: An Anatomically Accurate Inverse Dynamics Solution For Unconstrained Hand Motion

(a) (b) (c) (d) (e) (f)

Figure 1: Musculosketal architecture of the hand in (a), is explored by spreading, highlighting and annotating selected musculesin (b). Clinical application: (c) and (d) show inverse simulation solutions to motion drills.Animation application: (e) and (f)show an inverse solution to the motion capture of the letters "A","S", performed in A.S.L. A ghosted hand shows the fitting error.

model what we do not know, and there is much that is stillnot understood about the neuro-muscular control of the hu-man hand. Here perhaps our models may help to close theloop with experimental sciences to improve our overall un-derstanding of hand function through both simulation andvisualization.

Our goal is thus to build a skeletal musculo-tendon modelof the hand that combines computer graphics, physical me-chanics and anatomical and clinical measurements for un-constrained hand motion. We restrict ourselves to uncon-strained hand motion so as to avoid attempting to measureand environmental forces and isolate them from forces gen-erated by the hand. Such a model would provide forwardsimulation capabilities for visualization and pose prediction,and from which we could derive inverse dynamics controlsolutions for hand animation. This is of particular interest ina clinical setting to explore biomechanical hand function. Arealistic, robust inverse dynamic model for a given muscula-ture also allows us to address an increasingly important topicin character animation, one of validation. Just as computercharacters today, real or fictional, are modeled with a virtualskeleton for kinematic control, they may in the near futurebe constructed with an anatomic musculature of the kind wedescribe in this paper. The musculature could then be useddynamically for simulation, as an interpolation mechanismto fill in coarsely keyframed data, or as a reality check tovalidate whether the character is capable of performing agiven animated motion. An anatomic hand musculature cangreatly augment vision and hand motion capture systems tofill in missing motion data resulting from occlusion that ischaracteristic for complex hand motion.

Our hand model consists of a complex 3D musculoskeletalmodel with parameterized musculo-tendons accurately at-tached to skeletal components. To animate a model, our sys-tem incorporates anatomical and biomechanical data to per-form (forward) simulation; given a time sequence of muscle

activations. Hand motion is computed by solving the equa-tions of motion and integrating them in time using a stableimplicit integrator. To effect control over a model, we needto solve the inverse dynamics problem, which recovers mus-cle activations given a set of desired joint orientations. Away is needed to traverse the large space of physical con-trol solutions to find biomechanically valid if not optimalsolutions. We discuss several possible techniques below butbecause we wish to permit a wide range of unrestricted handmotion, we select an optimization approach based on mini-mizing the error between the target and output joint anglessubject to inequality constraints on the parameters of the ob-jective function; those parameters are the muscle activationswe seek. Using animations either from simulation or motioncapture data, the target joint angles and muscle effort effi-ciencies are formulated into objective functions, which arethen minimized to recover contraction values.

The contribution of our work is a biomechanical, animatedhuman hand model (see Figure1) containing:

• a comprehensive biomechanically realistic human handand forearm architecture with real-time physical simula-tion using rigid body dynamics. Control forces from theactuation of 41 musculo-tendon units around 16 jointsproduces bone rotation based on mechanical laws andexperimental data from studies of human hand anatomy.Joint interdependence is anatomically modeled.

• a constraint based inverse dynamics solution that recov-ers muscle actuation values for the purposes of analyzingmuscle function and hand animation.

• a graphical interface for visualizing the complex anatom-ical layout of the hand, and the output of the system. Con-trols are provided for the clinical practitioner selectivelyvisualize and spread muscle while maintaining contextand to vary the actuation values of the musculo-tendonunits and their strength associated with their capacity togenerate force.


Winnie Tsang, Karan Singh, Eugene Fiume / Helping Hand: An Anatomically Accurate Inverse Dynamics Solution For Unconstrained Hand Motion3

DIP2

PIP2

MCP2

IP

MCP1

CMC

Distal Radiocarpal (wrist)

Metacarpal

ProximalPhalanx

MiddlePhalanx

DistalPhalanx

Ulna Radius

(a) Hand joint nomeclature (b) Plastinated handc©Gunther von Hagen.

Figure 2: Hand nomenclature and visualization.

2. Related Work

Anatomy and Biomechanics.Studies in the anatomy ofhand joints, tendons and muscles are the basis for build-ing biomechanical hand models. Computer hand models[AHS03, BY94, Moc96, ES03] are based on the anatomicalstructure of the hand consisting of a hierarchical arrange-ment of bones, as can be found in illustrated anatomy books[AL99]. Studies of musculo-tendon configurations providedegree of freedom and their limit for joints, and give insightinto joint interdependency. However this interdependency isstill poorly understood, and the prevailing view is that bothbiomechanical and neurological issues are at play.

Extensive studies have been conducted by Brand and Hol-lister [BH99] to describe hand function. They exploremuscle operation, joint co-ordination, and force transmis-sion via tendons. A thorough description of hand muscleswith information about average resting fiber length, ten-sion fraction, and moment arms can be found in [BH99].[NTH01, NTHF02] described different anatomically-basedmodels for physical and geometric reconstruction of mus-cles. This physical muscle representation incorporates Hill’sclassic three-element model [Zaj89], which mathematicallyrelates parameterized musculo-tendon forces to actual phys-iological measurements. [TSBNLF05] address the construc-tion of skeletal muscle geometry from the visible humandataset and its subsequent simulation modeling collision andcontact using a robust finite element method.

Delp et al. [DL00] developed computational tools to createmodels of musculoskeletal system that can be used in combi-nation with experimental studies to answer questions relatedto a variety of research and clinical applications. For exam-ple, their Software for Interactive Musculoskeletal Model-ing (SIMM) allows users construct computer models of avariety of musculoskeletal structures which consist of a setof rigid segments connected by joints. Given muscle activa-tions, a mathematical muscle model computes the force andmoments that each muscle generates, and the resulting joint

motions. Our work focuses on the high-quality anatomicalvisualization of the human hand using both forward and in-verse dynamics.

Recently, motion capture data has been used to compute ac-tivation levels of facial muscles [SNF05] using the musclesimulation techniques of [TSBNLF05]. Motion capture hasalso been datamined by a kinematic hand model [ES03] togenerate realistic hand postures that capture the sympatheticfinger motion to arbitrarily animated hands.

The most comprehensive computer graphical hand model todate is found in the work of [AHS03] and more recently[KM04]. [AHS03] present a human hand model with ananatomical structure suitable for real-time physical simula-tion of muscles, together with elastic skin properties. Theirapproach is focused on modeling the overall visual appear-ance of the hand by using pseudo-musculo-tendon unitswhich control bone rotation, and geometric muscles whichdeform the skin. Our work uses a similar hand and muscleforce model but is more comprehensive in modeling muscu-lotendon interdependencies: for example, the Extensor Dig-itorium (see Figure3) has a strong effect on the interdepen-dencies observed among different fingers. While hand neu-rophysiology is ill-understood, we also attempt to captureinterdependencies by aggregating related musculo-tendonunits. As our current focus is on the architecture, dynamicsand visualization of musculoskeletal models, we have notyet addressed skin tissue (e.g., [KJP02]), [KM04] presentsan approach to the construction of a geometric model of realhands based on the CT scans for use in animation and sim-ulation. We hope to employ such an approach to paramet-rically resize and calibrate our hand model to variations inmuscular capabilities of different hands.

Dynamics. Dynamics simulation is extensively usedin robotics, biomechanics and computer animation[AHS03, Bar96, BY94]. In this approach, the motion ofobjects, namely their position as a function of time, isthe outcome of a solution of a set of ordinary differentialequations for a discrete time step. Associated with dynamicsimulation is the issue of control. In biomechanics, the useof inverse dynamics to compute force from noninvasivemeasurements of body motions is an area of much inter-est. Most attempts to quantify muscle forces in humansare based on optimization [Cro78]. Static optimizationis the most commonly used method to estimate muscleforces during locomotion [TAD03]. [TAD03] investigatedthe problem of computing of muscle excitation patternsthat produce coordinated movements of muscle-actuateddynamic models. The authors introduced a new algorithmthat uses static optimization along with feedforward andfeedback controls to drive the kinematic trajectory of amusculoskeletal model toward a set of desired kinematics.Their computed muscle excitations were similar in timingto measured electromyographic patterns, thus improvingthe feasibility of using detailed musculoskeletal models



origin wrist (ED2-5)

insertion wrist/origin MCP5 (ED5)insertion wrist/origin MCP4 (ED4)insertion wrist/origin MCP3 (ED3)insertion wrist/origin MCP2 (ED2)

insertion MCP5/origin PIP5 (ED5)insertion MCP4/origin PIP5 (ED4)insertion MCP3/origin PIP3 (ED3)insertion MCP2/origin PIP2 (ED2)

insertion PIP5/origin DIP5 (ED5)insertion PIP4/origin DIP4 (ED4)insertion PIP3/origin DIP3 (ED3)insertion PIP2/origin DIP2 (ED2)

insertion DIP5(ED5)insertion DIP4 (ED4)insertion DIP2 (ED2)insertion DIP3 (ED3)

Figure 3: Extensor Digitorum forking into multiple tendonsspanning multiple joints.

to simulate and analyze movement. Static optimizationsolves a different optimization problem at each time instant,and is computationally inexpensive. Its main disadvantageis that the results depends on the accuracy of the datarecorded. Dynamic optimization is time dependent, andsolves a large, often expensive, optimization problem forone complete cycle of the movement. [AP01a] presented adynamic optimization solution for normal walking on levelground and showed that the predicted muscle forces andjoint contact forces of the dynamic and static solutions wereremarkably similar.

Visualization. Three-dimensional visualization can presentanatomical information in a form that is visually pleasingand more easily understandable. In [HBR∗92], volumetricinteractive atlases were generated from cross-sectional im-ages. Human anatomy information can be presented as vol-umetric data. Transparency, segmentation, and deformationare some of the techniques used to display the inside of thevolumetric volume. In [MTB03], deformations were usedfor browsing through data: the user can cut into, open up,spread apart, or peel away parts of the volume in real time,exposing the interior while still retaining surrounding con-text. This approach has the advantage of not removing po-tentially important surrounding contextual information.

3. Hand ModelWe have seen that there has been considerable researchinto the depiction of hand motion. However, there to dateare no computer-animated, biomechanically valid skeletalmusculo-tendon models that can support arbitrary joint inter-dependencies, forward and inverse dynamics, and aggregatemusculo-tendon strain computation. The components of our

hand model are: a joint hierarchy, a skeleton consisting of 29bone meshes, and a set of 41 musculo-tendon units attachedto this skeleton. We now elaborate on these components.

Joints. There are 16 joints in our hand model. Joints aremodeled as hinges that are capable of rotation about theirprincipal axes. Referring to Figure2a, there are 23 degreesof freedom in the joint system. Each finger has four, withtwo DOF for the MCP joint for flexion/extension and ad-duction/abduction, and one DOF each for the PIP and DIPof joints for flexion/extension. The thumb has five: one forthe MCP joint, one for the IP joint, and three for the CMCjoint. Finally there are two DOF for the rotation of the wrist.We introduced a extra degree of freedom for the CMC jointof the thumb because the two axes of rotations are not or-thogonal. The flexion/extension of the CMC joint occurs inthe trapezium, and its abduction/adduction occurs in the firstmetacarpal bone. See [BH99] for a deeper discussion.

Bones.There are 29 bone meshes in total in our hand model.The bones in the forearm and hand are arranged hierar-chically, as are most limbs in the body. As with [BY94],we group bones into 16 different links that join the 16joints in the articulated hand model (Figure2a). The sizesand masses of each link were gathered from biomechanicalsources [BH99, BY94] and measured from a 3D hand skele-ton model. The data for the bone are from averaged valuesobtained in medical literature [AL99, BY94].

Muscles.There are in total 41 hand and forearm musculo-tendon units in our hand model. To understand the dis-tinction between muscle and musculo-tendon unit, considerthe Flexor Digitorum Superficialis (FDS), Flexor DigitorumProfundus (FDP), and Extensor Digitorum (ED). These mus-cles originate near the proximal radiocarpal joint (elbow)and the tendons pass through the MCP and PIP joints ofthe digits 2-5 (index finger through to pinky). Each mus-cle thus containsfour musculo-tendon units correspondingto each tendon. The interdependency of related musculo-tendon units of each such muscle due to neuro-muscular con-trol of the hand produces sympathetic finger motion. This in-terdependency is ill-understood; it may vary among individ-uals and thus cannot be easily quantified. After consultationwith anatomists, we discovered that in general, the musculo-tendon units of FDS can be contracted more independentlythan FDP and ED, which are deep muscles. Thus, we cou-pled the four musculo-tendons units of FPD and ED so thatthey are contracted simultaneously, while the four musculo-tendons of FPS units can be contracted independently.

Biomechanics provides models to quantify musculo-tendonforce generation for locomotion. A basic model is the lin-ear spring-damper, which views muscles as spring-like, ex-hibiting linear forces in the direction determined by an originand an insertion point. However, a single spring-damper can-not capture multiple parameters taken from actual muscleand tendon anatomical measurements. Hill’s three-elementmodel provides a simple, parameterizable muscle repre-



Tens

ion

Tota

l ten

sion

Rest

ing

leng

th

Contractile tension

Passive

Stre

tch

Length

Figure 4: Blix curve

sentation onto which actual measurements can be mapped[Zaj89].

There are two ways for a muscle to exert tension: by ac-tive contraction due to nerve stimulus (active contraction),and by the muscle’s elastic behavior (passive contraction),which is independent of nerve stimulus. We refer to the time-varying neural control signal as thecontractionor actuationvalueof the muscle, denoted byc(t). In active contractionof a muscle, the highest tension is produced when the mus-cle is at its resting length. This relationship is summarizedin length-tension curves for passive contraction and activecontraction, known as theBlix curve as shown in Figure4.We adopt the quadratic curve fit used by [AHS03] to approx-imate the two length-tension curves, giving as a continuousfunction the magnitude of the force exerted by a muscle.

We define the current fiber length asl ∈ [0.5lo,1.5lo], withrest lengthlo. The contraction value isc∈ [0,1]. Let Fmaxbethe maximal isometric force that can be exerted by a mus-cle; it can be thought of as relative maximum strength of themuscle [BBT81]. The contraction force term correspondingto the contractile element in Hill’s model is

Fc(l) =

[1−4

(llo−1.1

)2]

Fmax . (1)

The stretch force term corresponds to Hill’s parallel element:

Fs(l) =

{2.77(l/lo−1)2Fmax l ≥ lo.

0 l < lo.(2)

The total force is thus

F(l) = cFc(l)+Fs(l). (3)

Hill’s three-element model only computes the magnitude ofthe muscle’s force, and not its direction. Force direction iscomputed along its line of action, which can be representedas a sequence of piecewise linear segments as it extendsthrough one or more joints via tendon. We estimated the ori-gin and insertion points for every muscle at every joint itcrosses. This introduces some possible inaccuracies. One isthat real insertion and origin attachment regions are areas,not points. This is acceptable when the real attachment re-gion is small, as in the thread-like tendons through the wristand fingers. The muscle origins on the forearm, however,

are larger. In principle, we should distribute the lines of ac-tion over the insertion area. However, there is no experimen-tal data that provides the distribution of muscle effort overan area of muscle attachment. A piecewise linear approxi-mation also neglects inter-muscle collision forces as adja-cent muscles exert force on one other. The pennation angle,which relates the orientation of muscle fibers to tendon tis-sue can also be used to approximate the force applied to thetendon by the muscle.

To estimate the location of origin and insertion attachmentpoints, we manually fit digitized muscle fibers and tendondata to our 3D skeleton model usingMaya. We estimatedthe origins and insertions to be the centers of the muscle ortendon attachments to the bones (see Figure3).

To address how forces displace bone and joints, let the pointat which the tendon attaches to the bone from the joint’scenter of rotation be thelever arm. The lever arms of themuscles as they cross the affected joints are derived from[BH99]. The total effect of a muscle at a joint is given byits torque, which is the cross product of the lever arm andthe linear force. The magnitude of the torque is the perpen-dicular distance between the axis of the joint and the tendonas it crosses the joint, called themomentarm. The momentarm determines the effectiveness of muscle tension at a joint.As a tendon crosses multiple joints, it may change directionafter crossing one joint on its way to another. However, thetension is the same along the length of the tendon, and isnot shared or divided between joints. At each joint, the mo-ment arm will be different, determining the relative effect ofa muscle at each joint.

4. Dynamic Simulation

We now discuss the equations of motion of our hand model.Joints have degrees of freedom only in orientation and notin position, since they cannot be translated unless they areskeletally dislocated. Let~Si be the state vector for a singlerigid body i at timet. Let~θi be its spatial rotation and let~ωibe its angular velocity. The state of jointi is represented by

~Si(t) =

(~θi(t)

~ωi(t)

). (4)

To compute the total torque at jointi, we sum the torquesfrom every muscle that crosses jointi. Let~r j denote the leverarm and~Fi, j denote the force of the musclej on joint i. Weintroduce a set of weights,~α = (α1,α2, . . . ,αM),αi ∈ [0,1],to weight the contribution to the total tension that a musclecan exert at timet. These weights model the effect of musclefatigue. For instance, a muscle with a weight of zero cannotexert tension, whereas a muscle with a weight of one exertstension normally. Ifm muscles cross jointi, then its totaltorque,~Ti at timet is given by

~Ti(t) =m

∑j=1

~r i, j ×α j (t)~Fi, j (t) . (5)



We defineF̂i, j to be a unit vector giving force direction, andlet~i i, j and~oi, j be respectively the insertion and origin pointsfor musclej at joint i. The force~Fi, j can be computed by

~Fi, j (t) = [c jFc, j (l j )+Fs, j (l j )]F̂i, j , (6)

where

F̂i, j =~i i, j −~oi, j

||~i i, j −~oi, j ||. (7)

Let I i be a 3×3 inertia tensor for jointi, and kµ be the coef-ficient of friction. The coefficient of friction is 0.015, takenfrom [AHS03], which they used in accordance to medicalliterature. The second-order ordinary differential equation ofmotion for joint i is given by

~Ti(t) = I i(t)ddt

~ωi(t)+kµ~ωi(t) . (8)

We solve the equations of motion using a stable integrationmethod, namely the implicit Euler method [PTVF02], for-mulating a system of equations in the form ofA~x=~b, where

~x =∆~ω,

A =I t

∆t+kµ ,

~b =~Tt −kµ~ωt .

(9)

We can solve the above for∆~ω by LU decomposition. Once∆~ω is found we can update the state vector as follows:

~ωt+1 = ~ωt +∆~ω, (10)

∆~θ = ~ωt+1∆t, (11)~θt+1 = ~θt +∆~θ . (12)

We also compute the new length of musclej as

l j = l j −n

∑i=1

∆~θ ·~r i, j (13)

where

l j =

{1.5· lo , l ≥ 1.5· lo0.5· lo , l ≤ 0.5· lo,

(14)

clamps muscle length to lie within the range 0.5· lo to 1.5· lo.

To computeI effectively, we need to look to the geometry ofthe hand. For the purposes of computing moments, all jointsbut the radiocarpal joint (wrist) can be approximated as asequence of rigid cylinders or rods of constant circular cross-section. This reduces the inertia tensor to a moment of inertiawith an analytic solution for each joint approximated in thisfashion [Pai02, AHS03]. For the wrist, we assume a constantinertia tensor and compute it once prior to simulation. Theinertia tensor is computed using the inertia tensor equationfor a particle system, by summing up the squared distances

(a) (b) (c) (d) (e)

Figure 5: Forward simulation sequence due to muscle con-traction (a)-(c). Same simulation with reduced muscle con-traction levels (d),(e).

of the bone mesh vertices to the rotation axis multiplied bythe mass of bone. Putting all this together, we achieve theforward dynamics simulation as depicted in Figure5.

5. Constraint-based Control

The technique of using energy constraints on parameter-ized models was introduced in [WFB87]. Constraints are ex-pressed asenergy functionson the model’s parameter space,which are non-negative functions with zeroes at the pointssatisfying the constraints. A scalar function over the param-eters is obtained by summing these energy functions, and theobjective is to find a set of model parameters that minimizethe scalar function.

In the forward model, the outputs are the rotation anglesand the angular velocities forn joints, and the inputs aremmuscle contraction values. Our control algorithm is to re-cover the contraction values given a sequence of desired ro-tation angles and angular velocities, which could originatefrom keyframed animation or motion-capture data. With therecovered contraction values, we can forward simulate themodel to produce animated joint movements that closelymatch target poses. Thus consider the forward simulation asa functionf that maps a set ofm contraction values to 3nrotation angles,~θi , and 3n angular velocities~ωi :

~y = f (~c) =

~θ1~ω1...

~θn~ωn

, (15)

where the contraction values form muscles is~c =(c1,c2, . . . ,cm), and the function output~y, is a 6n× 1 vec-tor, since~θi , ~ωi are each 3× 1 vectors. Let the targets be~t = (t1, t2, . . . , t6n) for then joints. We formulate theenergyfunctionor theobjective functionas

E( f (~c)) = ws12

Es(~y)+wc12

Ec(~c) (16)

This is a weighted sum with scalar weightsws andwc of thetermsEs andEc which respectively evaluate the motion, andthe controller. By minimizing the motion evaluation termEs,



we seek to minimize the distance to the desired goals. Min-imizing the controller evaluation termEc promotes a prefer-ence for contraction values with lower amplitudes. Mathe-matically, these energy function terms are expressed as

Es(~y) = ||~t− f (~c)||22 , (17)

Ec(~c) =12||~c||22 , (18)

where|| · ||2 denotes 2-norm of a vector. Thus we see thatEs constrains the states of the model by measuring the er-ror between the output of the simulator and the target.Ec

maximizes the efficiency of the controller by summing theamplitudes of the contraction values.

Since the energy function and its gradient are the weightedsum of its constraints, adding more constraints to the systemis simple. The energy function is solved by minimizing thesum of the nonnegative objective functions associated withevery goal:

E(~c) = ∑i

wiEi . (19)

Recall from Eq.15 that the forward dynamics operationf isa nonlinear function ofm muscle contraction values that areall in [0,1]. For optimization, we use L-BFGS-B, a limitedmemory, iterative algorithm for solving large nonlinear con-strained or unconstrained optimization problems [ZBLN97].

More than twenty tendons cross the wrist joint, making itdifficult to search interactively for a control solution that si-multaneously minimizes all joint errors. We thus first mini-mize the objective function by omitting the wrist joint. Oncea solution for all joints except the wrist is found, we invokethe equations of motion with the candidate joint contractionvalues to evaluate the resulting error in the wrist joint rel-ative to the target values. We then formulate the objectivefunction to minimize this error; however, we only include asdegrees of freedom a restricted set of seven muscles that onlyaffect the wrist joint. Specifically, these muscles areFlexorCarpi Radialis(FCR),Palmaris Longus(PL), Flexor CarpiUlnaris (FCU), Extensor Carpi Radialis Longus(ECRL),Extensor Carpi Radialis Brevis(ECRB),Extensor Carpi Ul-naris(ECU), andAbductor Pollicis Longus(APL), which actas wrist stabilizers. Finally, we combine the resulting sevenmuscle contraction values with the previously solved onesand forward simulate the system. The sequence of controlactivations performed by our nervous system on the musclescontrolling our wrist and fingers is still unknown. It is thuspossible that although our method of decoupling the wristjoint works well in practice, it may not be biologically valid.

This approach to the musculoskeletal hand control problemis the first we have seen in the literature, and it has someuseful properties. First, it recognizes that while the forwardsimulation problem is deterministic, the inverse problem ismore complicated: there can be many different control so-lutions to achieve the same motion. While this is true of

most inverse problems in animation, the solution space forour control system is of extremely high dimensionality. Ourtwo-step approach has to date given us very good results.Second, it is possible to create different affinities for mus-cle activations; as such our control method is able to searchfor different solutions in the case, for example, of musclefatigue, pathology or atrophy. Third, because our physicalmodel captures crosstalk across muscles and joints, the con-trol solutions will naturally exhibit this effect.

To search for different solutions, we used various clinicallymotivated heuristics forRepetitive Strain Injuries(R.S.I.)diagnosis to explore the solution space by simulating mus-cle fatigue by means of setting muscle weights and solvingagain for an alternative solution. The starting point for thesolutions in Figure6(c)-(f) is the same.

User-controlled.Musclo-tendon unit weights can be manip-ulated by the user to disable or damp the contribution of unitsto the solution. Figure6(a) and (b) shows animation curvesof the inverse solutions to the forward simulated motion offlexing the MCP index joint. Comparing the results of per-forming inverse simulation with all of the muscles enabled,and with only the active muscles plus a small subset enabled,the latter gives a more accurate match to the target motion.Top-usage.This approach makes use of a previous solutionto find the musculotendon units that are most used, by sum-ming up their contraction values throughout the entire ani-mation sequence. Thus the primary muscles will be assigneda smaller weight in the inverse simulation (see Figure6(d)).Thresholding.This approach also makes use of previous so-lutions to simulate fatigued and stretched or pulled musclesby searching for musculo-tendon units with their contractionvalues or lengths exceeding set thresholds. Thus, musculo-tendon units that are stretched, pulled or contracted past theirthresholds are assigned a smaller weight (see Figure6(e)).Random Combination. In this approach we randomly se-lect an user-defined number of musculotendon units and as-sign a smaller weight to them (see Figure6(f)).

6. VisualizationWe have implemented a variety of visualization techniquesto depict hand animation (see Figures1 and Figures7). Weillustrate muscle activity through the use of opacity, provid-ing a less obscured view of actively contracting muscles.Muscles also bulge under tension, which is visually eas-ier to perceive than variations in lengths, providing com-pelling feedback on muscle contraction. We visualize errorsbetween the inverse solution and target motion data throughghosting. The complex and compacted muscle architectureof the hand makes it difficult to visualize the deep structures.Adopting from [AL99], we categorized the forearm muscu-lotendon units into six groups: three groups for the anterioraspect and three groups for the posterior aspect of the fore-arm; those group are divided based on depth. Each groupcan be ”spread" by positioning the associated controller ob-jects. This ”spreading" is inspired from Von Hagen’s plasti-



(a) Animation curve of inverse solutionfor MCP2 using all muscles

(b) Animation curve of inverse solutionfor MCP2 using a superset of active mus-cles

(c) Best Fit Solution

(d) Solution without primary muscles (e) Solution without fatigued muscles (f) Randomly weighted solution

Figure 6: Exploring inverse solution space.

(a) Uncontractedmuscles spreadwithin context

(b) Muscle con-traction control-ling transparency

(c) Contractedmuscle control-ling bulging

(d) Ghostedskeleton indicat-ing simulationerror

Figure 7: Various techniques used to aid dynamic muscle visualization in a clinical application.

nation works (see Figure2b), which similar to peeling mus-cle from its attachment. Our goal through these visualizationtechniques is to present hand architecture and motion to clin-icians and animators in ways that illuminate hand function.

7. Implementation and Results

Our hand is composed of three modules: the forward simula-tor, the inverse simulator, and the muscle force model. all im-plemented within the commercial animation systemMaya.

The forward simulator takes the muscle activation values andweights as input, which are defined by the user by keyfram-ing the muscles’ attributes using theMaya animation in-terface. It performs implicit integration of the equation ofmotions described above for each user-defined frame step,starting and ending at the frames specified by user. We havefound a time step of to 24 frames per second allows for stableintegration. The forward simulator produces finger configu-rations for each frame step, and the resulting animation canbe viewed by advancing the time slider in the interface.



(a) Handrix animation (b) Simulation matching each frame of the finalkeyframed motion

(c) Simulation matching only the keyframedpostures

Figure 8: Simulating a hand keyframed using motion capture based IK postures

The inverse simulator takes finger configurations as input,which are joint rotations; as an option angular velocities canalso be used. It solves for muscle activation values, produc-ing finger configurations that conform to the input for eachframe step. The frame step is defined by the user, and shouldbe chosen according to the number of frames that separatethe keyframed target data. Sources of target data includekeyframed animation, motion captured data, or animationfrom the forward simulator.

The muscle’s force model defines the computation of theforce exerted by the muscle on each joint. The input to themodule is the activation value and the output is the result-ing torques at each affected joint. The modularity of our ar-chitecture allows for replacement of the muscle’s physicalmodel with a more sophisticated one.

7.1. Clinical Application

Helping Hand provides anatomists with tools for visualizingthe anatomical structure of hand muscles and function. Us-ing the forward simulation component, they can manipulateactivation values to obtain the resulting motion for contract-ing particular muscles. Using inverse simulation, anatomistscan examine which muscles are utilized to produce uncon-strained hand motion. The redundancy of muscles allowsdifferent sets of muscles to be used to give similar motion.This redundancy is essential since in the case of muscle fa-tigue or failure, other muscles can be used as temporary re-placement or reinforcement. Hand surgeons are also con-cerned with the inability of patients to move certain jointsdue to various pathologies. Anatomists can manipulate mus-cle weights to find optimal solutions over various groups ofmuscles. Figure5d,e shows the result by changing the mus-cle weights on a set of muscles with similar function.

We recorded a male and female subject performing variousAmerican sign language gestures as input to our inverse sim-ulation component. Using such data, we can validate andcalibrate our hand model by observing the errors in the in-verse solution. Figure1(e,f) shows the result of the inversesimulation fit to hand motion capture data. The errors forthe digits were quite small with moderate wrist error. Wrist

stabilization remains a challenging biomechanical problem:while qualitatively quite satisfactory, fully validating the re-sults requires an intrusive deep tissue scan which is unethicalfor human subjects.

7.2. Animation Application

Helping Hand can also be used to validate and improvethe quality of synthesized hand animation. Keyframed handposes were linearly interpolated producing hand animationwhich ignores hand dynamics, since biological motion isnonlinear in nature. Using this data as target motion data, weobtained an inverse solution that matches every frame of thekeyframed motion. This results in a jittery hand animationdue to the difficulty of the hand model to match the non-realistic, linear motion. We then performed another inversesimulation matching only the keyframed postures, produc-ing hand animation that is less robotic and more realisticthan the keyframed animation. The motion curves for theflexion/extension of MCP index joint using both approachesare shown in Figure8 for comparsion.

8. Conclusion and Future Work

The goal of this paper was to present a complete frameworkfor the anatomic modeling, simulation and visualization ofdynamic hand motion. Overall, our system takes a first steptoward the graphical understanding of hand function. Muchwork remains to be done. The representation, interaction andvisualization aspects of the musculoskeletal data are moremature than the algorithmic modules for hand simulation.The model requires greater refinement and validation againstclinical data. The control solution needs to be more flexible,and it may require revision should the clinical case for thetwo-step control solution (decoupling wrist stabilisation) beproved to be an erroneous assumption. The hand and forearmneed a good skin model. Our near-term aim is to work withclinical practitioners on the deployment of our hand modelin medical practice. There is also considerable room for op-timism that our hand anatomy can be evolved in reverse todevelop families of hand models and morphologies for ourancestors, given fossil evidence.



References

[AHS03] ALBRECHT I., HABER J., SEIDEL H.: Con-struction and animation of anatomically based humanhand models. InProceedings of SIGGRAPH Symposiumfor Computer Animation 2003(2003), vol. 22, ACM Press/ ACM SIGGRAPH, pp. 98–109.3, 5, 6

[AL99] AGUR A. M. R., LEE M.: Grant’s Altas ofAnatomy, 10 ed. Lippincott Williams & Wilkins, Balti-more, Maryland, USA, 1999.3, 4, 7

[AP01a] ANDERSONF. C., PANDY M. G.: Dynamic op-timization of human walking.Journal of BiomechanicalEngineering 123, 3 (2001), 381–390.4

[Bar96] BARAFF D.: Linear-time dynamics using la-grange multipliers. InProceedings of SIGGRAPH 1996(1996), vol. 22 ofComputer Graphics Proceedings, An-nual Conference Series, ACM Press / ACM SIGGRAPH,pp. 137–146.3

[BBT81] BRAND P., BEACH R., THOMPSON D.: Rel-ative tension and potential excursion of muscles in theforearm and hand.Journal of Hand Surgery 6, 3 (1981),209–219.5

[BH99] BRAND P., HOLLISTER A.: Clinical Mechanicsof the Hand., 3 ed. Mosby - Year Book, Inc., St. Louis,MO, 1999.3, 4, 5

[BY94] BIRYUKOVA E., YOUROVSKAYA V.: A model ofhand dynamics. InAdvances in the Biomechanics of Handand Wrist(1994), Plenum Press, New York, pp. 107–122.3, 4

[Cro78] CROWNINSHIELD R.: Use of optimization tech-niques to predict muscle forces. InTrans. of the ASME(May 1978), vol. 100, pp. 88–92.3

[DL00] DELP S. L., LOAN J. P.: A computational frame-work for simulation and analysis of human and animalmovement. vol. 2, pp. 46–55.3

[ES03] ELKOURA , G., AND SINGH, K. 2003. Handix:Animating the human hand. InIn Proceedings of the 2003ACM SIGGRAPH/ Eurographics Symposium on Com-puter animation, vol. 22, 110–119.3

[HBR∗92] HOHNE K., BOMANS M., RIEMER M.,SCHUBERT R., TIEDE U., LIERSE W.: A volume-basedanatomical atlas. IEEE Comput. Graph. Appl. 12, 4(1992), 73–78.4

[KJP02] KRY P., JAMES D., PAI D.: EigenSkin:Real time large deformation character skinning in hard-ware. InSCA ’02: Proceedings of the 2002 ACM SIG-GRAPH/Eurographics Symposium on Computer anima-tion (July 2002), pp. 153–159.3

[KM04] KURIHARA T., MIYATA N.: Modeling de-formable human hands from medical images. InSCA ’04:Proceedings of the 2004 ACM SIGGRAPH/Eurographicssymposium on Computer animation(2004), ACM Press,pp. 355–363.3

[MM00] MARZKE M., MARZKE R.: Evolution of the hu-man hand: approaches to acquiring, analysing and inter-preting the anatomical evidence.Journal of Anatomy 197(2000), 121–140.1

[Moc96] MOCCOZETL.: Hand Modeling and Animationfor Virtual Humans. PhD thesis, Univ. of Geneva, 96.3

[MTB03] MCGUFFIN M., TANCAU L., BALAKRISHNAN

R.: Using deformation for browsing volumetric data. InProceedings of IEEE Visualization(2003), IEEE Com-puter Society Press, pp. 401–408.4

[NTH01] NG-THOW-HING V.: Anatomically-Based Mod-els for Physical and Geometric Reconstruction of Humansand Other Animals. PhD thesis, Univ. of Toronto, 2001.3

[NTHF02] NG-THOW-HING V., FIUME E.: Application-Specific Muscle Representations. InProc. Graphics In-terface(May 2002), pp. 107–116.3

[Pai02] PAI D.: Strands: Interactive simulation of thinsolids using cosserat models.Proceedings of Eurograph-ics 2002, Computer Graphics Forum 21, 3 (September2002), 347–352.6

[PTVF02] PRESSW., TEUKOLSKY S., VETTERLING W.,FLANNERY B.: Numerical Recipes in C++, 2nd Edi-tion. The Press Syndicate of the University of Cambridge,Cambridge, UK, 2002.6

[SNF05] SIFAKIS E., NEVEROV I., FEDKIW R.: Au-tomatic determination of facial muscle activations fromsparse motion capture marker data.Proceedings of SIG-GRAPH 2005(2005), ACM Press.3

[TSBNLF05] TERAN J., SIFAKIS E., BLEMKER S., NG-THOW-HING V., LAU C., FEDKIW R.: Creating andSimulating Skeletal Muscle from the Visible Human DataSet. IEEE Transactions on Visualization and ComputerGraphics, (May/June 2005), vol. 11, 317–28.3

[TAD03] THELNE D. G., ANDERSONF. C., DELP S. L.:Generating dynamic simulations of movement using com-puted muscle control. vol. 36, pp. 321–28.3

[WFB87] WITKIN A., FLEISCHER K., BARR A.: En-ergy constraints on parameterized models. InSIGGRAPH’87: Proceedings of the 14th annual conference on Com-puter graphics and interactive techniques(New York, NY,USA, 1987), ACM Press, pp. 225–232.6

[Zaj89] ZAJAC F. E.: Muscle and tendon: properties, mod-els, scaling, and application to biomechanics and motorcontrol. Critical Reviews in Biomedical Engineering 17(1989), 359–411.3, 5

[ZBLN97] ZHU C., BYRD R. H., LU P., NOCEDAL J.:Algorithm 778: L-bfgs-b: Fortran subroutines for large-scale bound-constrained optimization.ACM Trans. Math.Softw. 23, 4 (1997), 550–560.7


Documents

Helping Hand: An Anatomically Accurate Inverse Dynamics ...karan/pdf/HandPaper_SCA.pdf · 2 Winnie Tsang, Karan Singh, Eugene Fiume / Helping Hand: An Anatomically Accurate Inverse