Deriving Motion Constraints in Finger Joints of Individualized Hand Model forManipulation by Data Glove

Takuya Funatomi Takuya Yamane Hirotane Ouchida Masaaki IiyamaMichihiko MinohKyoto University

Yoshida-Honmachi, Sakyo-ku, Kyoto, 606-8501, Japanfunatomi@media.kyoto-u.ac.jp

Abstract

In this paper, we propose a novel method of skeleton es-timation for the purpose of constructing and manipulatingindividualized hand models via a data glove. To recon-struct actual hand accurately, we derive motion constraintsin fully 6-DOF at the joints without assuming either a cen-ter of rotation or a joint axis. The constraints are derivedby regression analysis on the configuration of the finger seg-ments with respect to the sensor data. In order to acquirepairs of the configuration and the sensor data, we introducegraspable reference objects for reproducing postures withand without the data glove. To achieve accurate regressionon fewer samples, we introduce a regression model accord-ing to a detailed investigation using a fused motion capturesystem, which enabled us to perform optical motion captureand sensor data collection simultaneously. The effective-ness of the method is demonstrated through practical appli-cations involving grasped reference objects.

1. IntroductionTechniques for capturing the motion and posture of

the human hand have been well studied in the literature.One of the most reliable methods of capture involves aworn, sensor-lined device, commonly referred to as a “dataglove”. The data glove is equipped with strain gauges atfinger joints, allowing measure of the flexion of each fin-ger segment. Such devices are commonly used for dexter-ous interactions with virtual environments, and in many ap-plications, they relate directly to a virtual hand model thatis transformed based on sensor input. Although the size,length, and shape of the hand vary between individuals, theyuse a generic model for input to capture the hand motion.However, when we want to reconstruct the position of thefingertip precisely, the generic model is not enough accuratesince it does not reflect the individual variations.

Regression for deriving motion constraints

mo

tio

n p

ara

me

ter

Sensor data

postures with

reference object

Detailed investigation of motion constraints

Sensor data

mo

tio

n p

ara

me

ter

Motion parameter

Sensor data

Figure 1. The proposed scheme for manipulating individualizedposable model via a data glove. We derive the motion constraintsin the joints without assuming either center of rotation and jointaxis.

To address this deficiency, we aim to reconstruct a pos-able model of the individual hand and manipulate it, basedentirely on data glove input. The model will consist of the

1

shapes of finger segments and a skeletal structure to con-trol the posture. Given the many methods [1, 3, 5, 12, 16]which can be used to reconstruct shapes, this paper focusesmainly on reconstructing the skeletal structure, for use inmanipulating the finger shapes via the glove.

The main contribution of our method is that it naturallycombines sensor calibration and skeletal parameter estima-tion. In previous approaches, sensor data was converted intojoint angles, which were then used to control the posturebased on a link model at each joint, configured with skele-tal parameters. In our method, we reconstruct a skeletonthat is directly controlled by the sensor data.

To reconstruct an accurate skeleton for the shape model,we derive motion constraints for the joints without assum-ing either a center of rotation or a joint axis. In our scheme(see also Figure 1), we perform regression analysis of theconfiguration of the finger segments with respect to the sen-sor data. The skeletal parameters are represented by regres-sion coefficients. Experiments will show that this represen-tation can reproduce postures more accurately than modelsin which static “ball joints” are assumed.

2. Related works2.1. Building individualized hand model

As far as we know, no method has yet been proposedfor both reconstructing and manipulating an individualizedhand model based entirely on a given hand’s data glove in-puts. However, some methods [7, 10, 17] have been pro-posed for only reconstructing posable human hands specificto individuals. [17] captured the three dimensional shape ofan entire hand in a specific posture, and then manually posi-tioned a skeletal model consisting of rigid links and joints.They mainly focused on producing natural hand posturesbased on fewer user inputs, and not on reproducing the ac-tual posture. Hand shapes in postures other than the onecaptured depended greatly on the fixture of joint centers. [7]constructed posable hand models based on CT images of agiven hand in several poses. This method reconstructs theskin surface and bone shape from the CT images, and thenestimates joint centers by comparing bone shapes from thedifferent poses. [8] took a similar approach, but used boneshapes reconstructed from MR images of four hand poses.Although these in vivo approaches support investigation ofactual hand kinematics, they depend on impractical appli-cations of medical imaging technology, and on an overlysimplified ball joint model for deriving skeletal structuresfrom a limited number of captured postures.

To build an individualized hand model, [10] managed touse only a palmar side photo taken with an image scannerin conjunction with marker positions from an optical motioncapture (MoCap) system. This method estimated joint cen-ters based on regression analysis of the joint centers, marker

positions and certain hand dimensions derived from the MRimages of eight subjects. Based on these estimates, and fea-tures derived from the palmar side photo, the method de-forms a generic hand model [7] and associates this defor-mation with the optical markers. Although this model canbe used to reconstruct hand postures from MoCap data, itis not easily applied to building and manipulating a handmodel based on data glove input. Moreover, the authors in-dicated limitations in reproducing individual hand models,mainly due to the simplicity of the generic skeletal model.

2.2. Estimating skeleton modelSome work [9, 19] has demonstrated accurate estima-

tion of the skeletal structure of the hand based on MoCapdata. [2, 6] have estimated a skeleton for the entire humanbody. Other methods have estimated a skeleton from cap-tured range data [1, 3, 12, 18] and CG models [4, 14]. In away similar to these methods, our method must build con-straints for the motion of fingers in the individual hand, andassociate them with the data glove’s sensor data.

Aguiar et al. [4] estimated not only the joint positionsbut also the motion parameters for each frame. They foundthat errors in the registration of body segments sometimescaused the length of bone segments to vary slightly overtime. To address this, they enforced constant bone lengths,and inferred joint motion parameters for all joints at alltimes. Motion parameters relative to the reference framewere expressed as Euler angles.

Note that all of the above methods assume a linkage ar-ticulated by idealized spherical or axial joints, and therebyneglect the non-rigid motion that occurs naturally in hu-man limbs, due to local shifting or deformation of the skin.Although it is acceptable when simply estimating skele-tal structure, it is inappropriate when reproducing posture,especially in the human hand. As we traverse a skeletalhierarchy from wrist to fingertips, errors in estimation ateach joint accumulate. Although Mundermann et al. [11]regarded small amount of movement at the joint (a so-called “soft-joint”), they just proceeded this movement as apenalty for estimating idealized joint in least-squares terms.

3. Our approachTo reduce accumulated wrist-to-fingertip error, our

method does not assume idealized spherical or axial joints.Rather, it allows full 6-DOF rigid motion at each joint be-tween adjacent segments, and derives the appropriate con-straints at each joint from actual postures. In the data glove,several strain gauges are equipped at each joint according totheir degree of freedom. According to this design, we per-form regression analysis of the motion at each joint basedon the sensor data from the data glove. This simultaneouslyprovides us with the proper constraints on motion and theproper association between motion and sensor data.

In order to perform the regression, we need the pairs ofsensor data and motion parameters at each joint for a cer-tain set of postures. At each posture, the subject needs towear the data glove to acquire the sensor data. Meanwhile,the motion parameters are derived from the configurationof the finger segments, which are acquired from the processof building a segment-wise shape model from a sequenceof range data [3, 5, 12, 16]. To build a fully accurate shapemodel, we also need to capture hand postures directly, with-out the glove. In order to acquire the pair of both sensor dataand motion parameters in the same posture, we introduce areference object to be grasped, and reproduce the graspingposture with and without the data glove. As with the previ-ously mentioned methods using medical imaging [7, 8], it isnot practical to collect many samples. To achieve accurateregression with fewer samples, we introduce a regressionmodel for the motion constraints based on a detailed inves-tigation of the relationship between the sensor data and themotion parameters. For this investigation, we developed afused motion capture system, combining the data glove withoptical markers, as shown in Figure 2(a). The markers areplaced on a rigid plate along each finger segment. Withthis device and multiple calibrated cameras, we can gatherflexion and motion data simultaneously, and for a varietyof postures. This rich dataset is the basis for our detailedinvestigation of motion constraints.

We illustrate our scheme in Figure 1. First, we performour detailed investigation of the relationship between sensordata and motion parameters using our fused motion capturesystem (see Section 4 for a further explanation of our in-vestigative procedure), and construct a regression model ofthe motion constraints. Second, we use reference objects tocollect several pairs of sensor data and motion parametersfor a set of postures. We also sample the motion parame-ters as we build the posable model of the individual hand,so that the motion parameters and the individualized handmodel are consistent. Once the hand model is built, we col-lect samples of sensor data with the glove on, and derive theskeletal motion constraints using regression analysis, basedon the regression model of the motion constraints. We canthen estimate the configuration of the finger segments basedon the sensor data from the glove, thereby manipulating theindividualized hand model. We show this process in Section5.

4. Detailed investigation of motion constraintsin finger joints

In this section, we report our detailed investigation ofmotion constraints in the joints of the human hand.

4.1. A fused motion captureFinger segments in human hand First, we suppose thata human hand consists of 18 finger segments organized hi-

1

2

3

4

6

5

8

7

11

10

14

13

9 12 15

16

1817

(c) sensor allocation

1

23

4

5

6

7

8

910

1112

13 14

15 16

1718

192021

2223

24

25 26

27 28

29

3130

32

33 3435 36

37 38

39 40

41

4342

44

45 4647 48

49 50

51 52

53

5554

56

57 5859 60

61 62

63 64

65 66

67 68

69 7071 72

(d) marker allocation

(b) skeleton hierarchy

root

rubber band

rigid plate

marker

(a) fused motion capture system

1

32

4

6

5

7

9

8 11

13

12

14

16

15

17

10

Figure 2. A fused motion capture system, consisting of data gloveand optical motion capture.

erarchically, as illustrated in Figure 2(b) (circles and linesrepresent the segments and joints of the hand, respectively).The numbers in the figure indicate the IDs of the joints. Weassume that each segment is represented as a rigid part.

Sensor data of data glove We used the CyberGlove mo-tion capture glove system with 18-sensors. The sensor allo-cation and IDs for the device are shown in Figure 2(c). TheCyberGlove returns a vector of sensor data

θ = {θi|i = 1, · · · , 18} , (1)

where θi takes an integer value in the range [0, 255] cor-responding to the degree of bending in the ith sensor. Wecan capture θ at a maximum frequency of 150 Hz throughRS-232C.

Marker-based motion capture for the finger segmentsTo investigate the relationship between the sensor data andthe configuration of the finger segments, we need to capturethe configuration in synchronization with θ.

We build an apparatus for optical motion capture of thedata glove while it is in use. The observation area is a cubicspace measuring about 250 mm in each direction, and we set39 synchronized cameras approximately 1200 mm from thearea. With this system, we can capture synchronized images

(a) 10th joint (b) 11th joint (c) 12th joint (d) 17th joint

Figure 3. Our local coordinate system for each joint of ring finger.Blue and yellow spheres are the markers on the parent and childsegment, respectively.

at 7.5 fps. Camera calibration is performed according toZhang’s method [20] and the factorization method [15].

We place 1-mm-square fluorescent markers on every fin-ger segment, and observe them using the synchronized cam-eras under black light. We calculate the 3D positions of themarkers using triangulation. To calculate the rigid transfor-mations of the finger segments, we place four markers oneach, totaling 72 markers in all, as shown in Figure 2(d).Since the glove is made from an elastic fabric that slidesover the skin during movement, the non-rigid deformationon the fabric may be larger than that of the skin beneath. Toprevent this exaggeration, we attach a plastic board on eachfinger segment and then attach the markers to this board,ensuring rigidity within the segment. To force the board tomove with the segment, we fasten it to the segment withrubber bands.

4.2. Motion parameters at each jointWe expect that the sensor data for a given joint will re-

flect the degree of flexion of it, and estimate the motionparameters on this basis. We assume that each joint hasa parent segment and a child segment, based on the skele-tal hierarchy in Figure 2(b). The flexion will correspond tothe rigid transformation of the child segment relative to itsparent. Following this, we define the motion parameters ateach joint as follows.

First, we introduce a ‘base posture’, and a local coor-dinate system for each finger segment with respect to themarker position in the base posture. Although it is accept-able to set these arbitrarily, we adopt a relaxed natural pos-ture as the base posture, and set the origin of the local co-ordinate system at the center of the markers on the parentsegment for each joint. We present some examples in Fig-ure 3.

Second, we perform the marker-based registration withrespect to the parent segment. For other postures under ob-servation, we acquire the transformed markers ym of thechild segment using the following equation:

ym = Mjxm, m ∈ Cj (2)

Mj = arg minM

∑m∈Pj

‖bm − Mxm‖ (3)

where m is the marker index, Pj and Cj is a set of marker

ym

bm

qj, tj

(b) registration based on Cj

xm

bm yym

Mj

(a) registration based on Pj

Figure 4. The motion parameters are defined as the rigid transfor-mation of the child segment relative to its parent.

indices attached to the parent and child segments of the jthjoint, bm and xm are the position of m on the base postureand some other posture, and M ∈ SE(3) is a 4 × 4 matrixof rigid transformation, respectively. Here, Mj is the regis-tration transformation with respect to the parent segment ofjth joint. We illustrate this registration in Figure 4(a).

Third, we calculate the motion parameters of the childsegment relative to the parent segment. As our representa-tion, we adopt the quaternion qj and a translation vector tj .We then obtain the parameters that minimize the distancebetween bm and ym on the child segment:

(qj , tj) = arg minq,t

∑m∈Cj

‖bm − QTym‖ , (4)

where Q and T are the 4 × 4 matrix of rotation and trans-lation corresponding to q and t, respectively. We illustratethe rigid transformation of the child segment relative to itsparent in Figure 4(b).

4.3. Regression analysis of motion parameters onsensor data

We perform a regression analysis of qj and tj on thesensor data θ from the collected samples. For this, we usethe following model for motion constraints:

(qj(θ)tj(θ)

)=

∑k

wkj φk(θ), (5)

where qj(θ) and tj(θ) are the predicted values, φk(θ) is akernel function of θ for the regression (such as n-th orderpolynomial terms (θi

n), linear terms (θi), and a constantterm (1)), and wk

j is a (4 + 3) dimensional vector of theregression coefficient of φk(θ) for the motion parameters.

In practice, we can determine the coefficients of themodel from the collected samples of fewer postures if weinvolve reference objects in those postures. To do this accu-rately, we introduce a regression model of motion constraintas a set of kernel functions {φk(θ)} (see 4.5 for further de-tails).

4.4. Reconstruct hand posture from the sensor dataOnce we determine the regression coefficients wk

j , wecan predict the relative motion (qj , tj) at each joint basedon sensor data θ from the glove. In order to reconstruct theentire shape of the hand, we need to convert the relative mo-tion to the global configuration of segments. Hence, we tra-verse the skeletal hierarchy from the root to each segment,and multiply the relative transformations one by one.

As for the wrist, which is the root of the skeletal hierar-chy in our case, we simply treat that segment as fixed to theglobal coordinate system.

Mwrist = I (6)

Mwrist can be substituted for other global transformations,such as the output of another sensor attached to the wrist,according to actual use.

Thus, all finger segments are transformed recursively.

M cj = Mpj (Qj(θ) Tj(θ))−1

, (7)

where Mpj and M cj is the rigid transformation of the par-ent and child segment at joint j, Qj(θ) and Tj(θ) are the4 × 4 matrix of rotation and translation corresponding toqj(θ) and tj(θ), respectively. For example, the rigid trans-formation of the proximal phalange of ring finger is

M c12 = Mwrist (Q17(θ)T17(θ))−1

(Q12(θ)T12(θ))−1

.

(8)

4.5. Detailed investigationCapturing data Using our fused motion capture system,we can collect pairs of sensor data and motion parameters invarious postures. We instruct the subject to independentlyflex and extend his fingers to their limits, while simultane-ously capturing 3,068 samples of sensor data θ and multipleimages. We then calculate the motion parameters (qj , tj) ateach joint (j = 1, · · · , 17).

Variable selection from the sensor data Based on theallocation of strain gauges on the data glove, we manuallyassign a sensor ID to each joint for the regression analysisshown in Table 1. That is to say, we constrain the regres-sion coefficients for the kernel functions of the unselectedsensors θi to be 0.

Relationship between the sensor data and themotion pa-rameters In Figure 5, we plot the collected pairs of sensordata θ11 and motion parameters (q11, t11). Each dot corre-sponds to one sample in the collected data. We see thatwe can approximate their distribution as a quadratic func-tion of the sensor data. Thus, we prepare quadratic, linear,and constant terms as a set of kernel functions {φk(θ)},

Table 1. Assignment of the strain gauges for the joints.ID Sensor ID Sensor ID Sensor

1 3 7 8 13 142 2 8 8 14 143 1,4 9 7,9,12 15 13,154 6 10 11 16 175 6 11 11 17 186 4,5,9 12 10,12,15

-1

-0.5

0

0.5

1

0 50 100 150 200 250θ11

q0q1q2q3

q0(θ)q1(θ)q2(θ)q3(θ)

-20

-10

0

10

20

30

40

50

0 50 100 150 200 250θ11

txtytz

tx(θ)ty(θ)tz(θ)

Figure 5. The relationship between θ11 and (q11, t11), which de-scribes the motion in the middle phalange of the ring finger. Eachdot and curve represents the data sample and the regression model,respectively. This regression model is derived from all samples.

and perform regression analysis with them on all samples.We present the results of q11(θ11), t11(θ11) as the curvesshown in the respective figures. Intuitively, it seems thatthese curves approximate the distribution well. However,having assigned multiple variables to some joints, it is noteasy to visualize the approximations for all.

Comparing reproduction accuracy with the ball-jointsIn order to evaluate the regression quantitatively in all seg-ments, we use the regression models qj(θ) and tj(θ) topredict the configurations for the collected sensor data. Wetransform the markers from the base posture bm based onequation 7, then evaluate the error in terms of the markerposition. For each joint, we perform registration betweenthe predicted and observed markers relative to the parentsegment, and evaluate the distance between the respectivechild markers, as well as the residuals in equation 4.

In order to validate our approach, which derives the mo-tion constraints in the joints without assuming center of ro-tation, we compare it with the constraints assuming a balljoint. As many previous methods have referred, we esti-mate the center of rotation based on [6], and a the rotationparameters as well as we have done in 4.21. We plot theestimated motion parameters for the ball joint in Figure 6.Comparing to the Figure 5, the distribution of qj is similar,but slightly different.

1Different from equation 7, the rotation parameter is obtained fromqj = arg min

q

Pm∈Cj

‚‚‚bm − TjQT−1

j ym

‚‚‚ , where Tj represents a

translation matrix to the estimated center of rotation.

-1

-0.5

0

0.5

1

0 50 100 150 200 250θ11

q0q1q2q3

Figure 6. The relationship between θ11 and q11, which is motionin the middle phalange of ring finger, based on the ball joint.

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Resid

uals

(m

m)

Joint ID

quad-regression for 6DOF jointearth for 6DOF joint

quad-regression for ball jointearth for ball joint

Figure 7. Average distance between predicted and observed childmarker positions. This prediction is based on the regression modelderived from all samples.

Comparing with non-parametric approach

Using all samples We also try a non-parametric re-gression technique, multivariate adaptive regression splines,as a substitute for our regression using kernel functions. Forthis, we used the ‘earth’ package implemented in R [13]and adopted all of the sensor data as candidate explanatoryvariables. We expected that this approach would be lesssuccessful for regressions on fewer samples, but would ap-proach an upper bound of accuracy when using many sam-ples.

Figure 7 presents the residuals of the regression modelsfor each joint. Compared to the ball joint representation,our motion constraints achieved better accuracy in everyjoint, especially in the results of regression using ‘earth’.Generally, the phalanges closest to the wrist have largerresiduals, and the distal phalanges have smaller residuals.However, these distal phalanges will also have larger errorswhile traversing the skeletal tree based on equation 7. Com-pared to the regression using our kernel functions, ‘earth’achieves better results.

Using fewer samples Next, we evaluated a motionconstraint derived from fewer samples. We manually se-lected only 21 samples from the observed data, performedregression analysis on them, and compared the predicted

-1

-0.5

0

0.5

1

0 50 100 150 200 250θ11

q0q1q2q3

q0(θ)q1(θ)q2(θ)q3(θ)

-20

-10

0

10

20

30

40

50

0 50 100 150 200 250θ11

txtytz

tx(θ)ty(θ)tz(θ)

Figure 8. Regression results from 21 samples. Intuitively, the re-gression from fewer samples still approximates the full set of ob-served data well.

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17R

esid

uals

(m

m)

Joint ID

quad-regression for 6DOF jointearth for 6DOF joint

Figure 9. Average distance between child marker positions for theregression models of ‘earth’ and of our kernels.

configuration to the other samples. In Figure 8, the mark-ers represent the selected samples for the regression and thecurves represent the regression results. Because we selectedsamples covering all postures of independent flexion andextension, the samples are distributed non-uniformly withrespect to the sensor data. Even so, we can see that the re-gression results still approximate the distribution yielded bya large number of samples.

Figure 9 presents our quantitative comparison of our re-gression to that ‘earth’. Note that our kernels, using veryfew samples, still managed to match or exceed the accuracyof ‘earth’.

According to the analyses described above, we were ableto derive the motion constraints q(θ) and t(θ), based di-rectly on sensor data θ from 21 manually selected postures.Our motion constraints in 6-DOF at the joints achieved bet-ter accuracy than typical methods that assume a rigid balljoint. We also showed that our kernels achieved better pre-dictions based on fewer samples. This clearly indicates thesuccess of introducing a regression model for deriving mo-tion constraints.

5. Skeletal estimation for manipulating indi-vidualized hand model via data glove

In this section, we demonstrate a practical implementa-tion of our approach.

Figure 10. Twenty additional postures involving reference objects.

5.1. Collecting posture samples involving referenceobjects

We attached optical markers on the hand, and recon-structed the entire hand shape and marker positions for arelaxed natural posture. Using this as our base posture,we simply segmented the shape model into finger segmentsmanually. The resulting shape and marker positions areshown in Figure 1.

We also captured the marker positions while graspingeight reference objects in 20 total postures. These posturesare shown in Figure 10. In order to specify the posture sim-ply, we put five markers for fingertip placement on the refer-ence objects, and used different colored marker for differentpostures involving the same object. For each posture, andfor the base posture, we calculated the motion parameters,qj and tj , at the joints j.

Then, we used the same reference objects to collect thesensor data of the data glove. We captured the sensor data θ

for the same postures, and coupled with the motion param-eter acquired above.

5.2. Manipulating individualized hand model

For 21 pairs of (qj , tj) and θ, we performed regressionusing kernel functions {φk(θ)}, and determined the regres-sion coefficient wk

j . According to the regression model, wepredicted the motion parameters for any given sensor dataθ, and reconstructed the transformation M cj (from equa-tion 7) for each finger segment. We then transformed thehand shape model based on these segment transformations.

As a result, we could achieve direct manipulation of anindividualized hand model via the data glove. We illustratethis manipulation in Figure 11. Note that the individualizedhand model displayed on the screen accurately changes itsposture in real time according to the posture of the glovedhand. Because of equation 6, the wrist portion of the modelremains fixed on the screen.

Figure 11. Manipulating an individualized hand model via the dataglove.

0

1

2

3

4

5

6

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17R

esid

uals

(m

m)

Joint ID

using reference objectsusing fused motion capture

Figure 12. Average distance of marker position on the child seg-ments based on LOOCV in the actual implementation. For com-parison, we show the result for both the normal data glove andof our fused motion capture system (the same result as given inFigure 9).

5.3. Quantitative evaluationTo evaluate the accuracy of our method quantitatively,

we performed leave-one-out cross validation (LOOCV) forthe aforementioned 20 postures. We use 19 samples andthe base posture for regression, and evaluated it using theremaining posture. The average error in marker positionis shown in Figure 12. Note that the result is very simi-lar to the result when using the fused motion capture sys-tem. This implies that our approach involving reference ob-jects is valid for collecting posture samples with minimaltools. Comparing the two modes of collection, we note thatmetacarpals (c3, c16 and c17) show larger errors when us-ing the reference object. This will be due to the non-rigiddeformation of the segments.

6. ConclusionIn this paper, we proposed a novel method of skeletal

estimation for constructing and manipulating an individual-ized hand model via a data glove, without resort to idealizedlink structures. We were able to reconstruct arbitrary handpostures by appropriately configuring 18 finger segments,and to accurately imitate actual hand in real-time based onglove sensor data.

To achieve accurate reconstruction of the configuration,

we derived motion constraints for the joints of an individ-ual’s hand without assuming either a center of rotation ora joint axis. The motion constraints in fully 6-DOF at thejoints were investigated using a fused motion capture sys-tem, which enabled us to perform optical motion captureand glove sensor data collection simultaneously. Accord-ing to the investigation, we introduce a regression model toderive the constraints from fewer samples. In our practicalimplementation, we introduced graspable reference objectsfor reproducing postures with and without the data glove, inorder to acquire pairs of motion parameters and sensor datafor each of a number of postures.

We experimentally demonstrated that our motion con-straints achieved better accuracy than assuming the balljoint. We also demonstrated accurate manipulation of anindividualized hand model via the data glove.

As a part of our future works, we would like to adopt astate-of-the-art method for shape modeling, and to integrateit and our method into a single framework. We would alsolike to investigate further improvements in the accuracy ofour derived motion constraints.

Acknowledgment This work was supported by JSPSGrant-in-Aid for Young Scientists (B), 23700231.

References[1] D. Anguelov, D. Koller, H.-C. Pang, P. Srinivasan, and

S. Thrun. Recovering articulated object models from 3drange data. In Proceedings of the 20th conference on Uncer-tainty in artificial intelligence, UAI ’04, pages 18–26, 2004.2

[2] J. Cameron and J. Lasenby. Estimating human skeleton pa-rameters and configuration in real-time from markered op-tical motion capture. In F. Perales and R. Fisher, editors,Articulated Motion and Deformable Objects, volume 5098of Lecture Notes in Computer Science, pages 92–101. 2008.2

[3] W. Chang and M. Zwicker. Global registration of dynamicrange scans for articulated model reconstruction. ACMTrans. Graph., 30(3):26:1–26:15, 2011. 2, 3

[4] E. De Aguiar, C. Theobalt, S. Thrun, and H.-P. Seidel. Auto-matic conversion of mesh animations into skeleton-based an-imations. Computer Graphics Forum, 27(2):389–397, 2008.2

[5] T. Funatomi, H. Akuzawa, M. Iiyama, and M. Minoh.Pinhole-to-projection pyramid subtraction for reconstructingnon-rigid objects from range images. 2011 InternationalConference on 3D Imaging, Modeling, Processing, Visual-ization and Transmission, 0:254–261, 2012. 2, 3

[6] A. G. Kirk, J. F. O’Brien, and D. A. Forsyth. Skeletal pa-rameter estimation from optical motion capture data. In Pro-ceedings of the 2005 IEEE Computer Society Conference onComputer Vision and Pattern Recognition (CVPR’05) - Vol-ume 2, pages 782–788, 2005. 2, 5

[7] T. Kurihara and N. Miyata. Modeling deformable humanhands from medical images. In Proceedings of the 2004ACM SIGGRAPH/Eurographics symposium on Computeranimation, SCA ’04, pages 355–363, 2004. 2, 3

[8] N. Miyata, M. Kouch, M. Mochimaru, and T. Kurihara. Fin-ger joint kinematics from mr images. In Intelligent Robotsand Systems, 2005. (IROS 2005). 2005 IEEE/RSJ Interna-tional Conference on, pages 2750 – 2755, 2005. 2, 3

[9] N. Miyata, M. Kouchi, T. Kurihara, and M. Mochimaru.Modeling of human hand link structure from optical motioncapture data. In Intelligent Robots and Systems, 2004. (IROS2004). Proceedings. 2004 IEEE/RSJ International Confer-ence on, volume 3, pages 2129 – 2135 vol.3, 2004. 2

[10] N. Miyata, Y. Motoki, Y. Shimizu, and Y. Maeda. Individ-ual hand model to reconstruct behavior from motion capturedata. In Robotics and Automation (ICRA), 2011 IEEE Inter-national Conference on, pages 1951 –1956, 2011. 2

[11] L. Mundermann, S. Corazza, and T. Andriacchi. Accuratelymeasuring human movement using articulated icp with soft-joint constraints and a repository of articulated models. InComputer Vision and Pattern Recognition, 2007. CVPR ’07.IEEE Conference on, pages 1 –6, 2007. 2

[12] Y. Pekelny and C. Gotsman. Articulated object reconstruc-tion and markerless motion capture from depth video. Com-puter Graphics Forum, 27(2):399–408, 2008. 2, 3

[13] R Core Team. R: A Language and Environment for StatisticalComputing. R Foundation for Statistical Computing, Vienna,Austria, 2012. ISBN 3-900051-07-0. 6

[14] S. Schaefer and C. Yuksel. Example-based skeleton extrac-tion. In Proceedings of the fifth Eurographics symposium onGeometry processing, SGP ’07, pages 153–162, 2007. 2

[15] T. Ueshiba and F. Tomita. A factorization method for pro-jective and euclidean reconstruction from multiple perspec-tive views via iterative depth estimation. In Proceedings ofthe 5th European Conference on Computer Vision-Volume I- Volume I, ECCV ’98, pages 296–310, 1998. 4

[16] M. Wand, B. Adams, M. Ovsjanikov, A. Berner, M. Bokeloh,P. Jenke, L. Guibas, H.-P. Seidel, and A. Schilling. Effi-cient reconstruction of nonrigid shape and motion from real-time 3d scanner data. ACM Trans. Graph., 28(2):15:1–15:15,2009. 2, 3

[17] Y. Yasumuro, Q. Chen, and K. Chihara. Three-dimensionalmodeling of the human hand with motion constraints. Imageand Vision Computing, 17(2):149 – 156, 1999. 2

[18] Q. Zhang, X. Song, X. Shao, R. Shibasaki, and H. Zhao. Un-supervised skeleton extraction and motion capture from 3ddeformable matching. Neurocomput., 100:170–182, 2013. 2

[19] X. Zhang, S.-W. Lee, and P. Braido. Determining fin-ger segmental centers of rotation in flexionextension basedon surface marker measurement. Journal of Biomechanics,36(8):1097 – 1102, 2003. 2

[20] Z. Zhang. A flexible new technique for camera calibration.Pattern Analysis and Machine Intelligence, IEEE Transac-tions on, 22(11):1330–1334, Nov. 4