Dynamic touch-enabled virtual palpation

COMPUTER ANIMATION AND VIRTUAL WORLDS

Comp. Anim. Virtual Worlds 2007; 18: 339–348

Published online 2 July 2007 in Wiley InterScience

(www.interscience.wiley.com) DOI: 10.1002/cav.194* * * * * * * * * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Dynamic touch-enabled virtual palpation* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

By Hui Chen*, Wen Wu, Hanqiu Sun and Pheng-Ann Heng* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

Palpation is an important method of feeling with hands during a physical examination, in

which the doctor presses on the surface of the patient body to feel the organs or tissues

underneath. In current surgical simulation systems, the lack of an effective sense of touch is

still a major problem. In this paper, a dynamic touch-enabled virtual palpation model is

proposed. The palpation force sensing between the index finger and virtual tissues is

simulated through a body-based haptic interaction model. Both contact and frictional forces

are evaluated based on Hertz’s contact theory, and the press distribution within the contact

area is also specified. The non-linear viscoelastic behavior of typical tissues is mimicked via

a volumetric tetrahedral mass-spring system. Reaction during the palpation is restricted to a

local area to highly reduce the order of the dynamic equation of the entire system to guarantee

a fast working rate. Mechanical tests have been performed to evaluate the palpation force

perception and the realistic behavior of typical human tissues. Copyright# 2007 JohnWiley

& Sons, Ltd.

Received: 14 May 2007; Accepted: 15 May 2007

KEY WORDS: haptics; palpation; surgical simulation; virtual reality

Introduction

At present, the skills of junior surgeons are usually

trained by performing procedures on animals, cadavers,

and ultimately on actual patients under the supervision

of an experienced surgeon. However, how to handle a

wide variety of complications and unusual circum-

stances which may arise during a procedure is hard to

practice because that would entail putting patients often

at an unacceptable risk. Therefore, virtual-reality-based

surgical simulators may become a safe and feasible

alternative for enhancing traditional surgical training.

The ultimate goal is to allow surgical training, diagnosis,

and surgical procedures to be performed in cyberspace

as if carried out in the real world. Many research efforts

have been dedicated to this area to help medical

students learn human anatomy and practice the skills

in a virtual environment.1–5 Many simulation systems

allowing the interactive operation have been proposed,

such as a visual and haptic simulation system of bone

and mandibular surgery;1 a virtual training system on

bile duct exploration for searching gallstones during

*Correspondence to: H. Chen, Shenzhen Institute of AdvancedIntegration Technology, Chinese Academy of Sciences, TheChinese University of Hong Kong, Shenzhen, P.R. China.E-mail: [email protected]

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

Copyright # 2007 John Wiley & Sons, Ltd.

endoscopic cholecystectomy;2 an evaluation system of

novices’ performance in laryngoscopic procedures,

which can provide a guidance for improvement accord-

ing to quantitative comparison with expert skills;3 a

surgical training system for arthroscopic knee surgery;4

and a VR-based acupuncture training system in Chinese

medicine, which simulates needle manipulation.5 All

these works demonstrate the high potential of applying

virtual-reality-based surgical simulation systems to

enhance traditional practices in the surgical field.

Force feedback involved in virtual-reality-based

surgical simulation is not only made to improve the

realism of virtual environments, but also to provide

important diagnostic information through the sense of

touch. Palpation is an important method of feeling with

hands during a physical examination, in which the

doctor presses on the surface of the body to feel the

organs or tissues underneath. For example, it is an

important technique in cancer diagnosis for determining

the size, consistency, texture, location, and tenderness of

abnormal tissues. In this paper, a dynamic tou-

ch-enabled virtual palpation model is established with

the following novel features:

� N

* *

ovel palpation force simulation model:

A body-based haptic interaction model was pro-

posed to simulate the palpation force sensing between

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

* *

Co

H. CHEN ET AL.* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

the index finger and the virtual tissues. The contact

and frictional forces are modeled as the body contact

based on Hertz’s theory involving the physical attri-

butes: mass, volume, Poisson’s ratio and Young’s

modulus. The press distribution within the contact

area is also specified. Multi contacts can be detected

and the final integrated force is applied to the user via

the haptic interface.

� R
eal-time soft-tissue deformation:
A volumetric tetrahedral mass-spring system has

been constructed to achieve real-time soft-tissue

deformation during virtual palpation. The reaction

during the palpation is restricted to a local area to

highly reduce the order of the dynamic equation of the

whole system and guarantee a fast working rate.

Furthermore, time-dependent viscoelastic behavior,

creep, and relaxation of human tissues, are simulated

in our model.

The next section outlines related previous work. This

is followed by a haptic body-based palpation model we

have established. The subsequent section presents the

dynamic touch-enabled soft-object deformations in

reaction to the palpation and the next section shows

the implementation and experimental results of the

whole system. The conclusions are given in the last

section.

RelatedWork

In this section we give a brief overview of related

previous work in finger perception modeling and

soft-tissue deformation.

Finger PerceptionModeling

As addressed previously in this paper, palpation is an

important diagnosis technique, with use in cancer

diagnosis to find the size, consistency, texture, location,

and tenderness of abnormal tissues. Kaufman6 reported

the concept design of a VR prostate palpation system

using a PHANToM haptic interface. This small robotic

arm can provide forces to the user’s index finger in

response to interactions with the virtual anatomy of

interest. Burdea and his colleagues7,8 simulated palpa-

tion using the virtual index finger and prostate model in

a VR-based training system of direct rectal examination

for prostate cancer diagnosis. They also established a

VR-based palpation training system to search for tumor

beneath the surface via a haptic glove.9–11 The force

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

pyright # 2007 John Wiley & Sons, Ltd. 340

simulated was based on Hooke’s deformation law.

Barbagli et al. 12 compared four physical models for

rotational friction of soft finger contact. A 4DOF

point-based algorithm was presented and focused on

rotational friction at the contact. However, most

simulated palpation forces were reduced to point-based

interaction model with spring-damper linkage to

simulate the contact between one or more fingertips

and the virtual object. Some special haptic device,

Haptic Interface RObot (HIRO),13 was created and

applied in breast palpation simulation.

The contact problem between two elastic solids that

are pressed by an applied force was first solved by Hertz

in 188214 under several assumptions. Hertz’s theory yields

stresses, deformations, and the shape of the interface

formed at two contacting bodies. These quantities

depend on the force pushing them together, the elastic

properties of the two bodies, and the geometric shape at

the contact position. Several numerical methods15

solving Hertz type problems have been developed

employing similar properties of the solution. Three-

dimensional (3D) Hertz type contact problems for linear

and non-linear, elastic and non-elastic materials have

been similarly considered. Pawluk and Howe16,17

investigated the dynamic force and distributed pressure

response of the human finger pad based on Hertz’s

theory and developed a quasilinear viscoelastic model,

which successfully explained the observed measure-

ments. Chen and Sun18 established a body-based haptic

interaction model based on Hertz’ contact theory and

tested the force distribution on the finger pad.

Soft-Tissue Deformation

Modeling of soft-tissue deformation in virtual-

reality-based medical simulations is of great importance.

The goal is to allow virtual tissues responding to user’s

manipulations in a physically realistic manner as if

possessing the true mechanical behavior of real tissues.

Physically based methods19 are the most popular

representations for soft-tissue simulation. In physically

based methods, deformation of the object is described by

laws of physics. Once the physical properties of the

object, such as mass, forces and boundary conditions,

etc., are specified, the problem can be represented by

precise mathematical expressions in the form of partial

differential equations.

The mass-spring model is one popular model for

deformable organs in surgical simulation. It can give the

illusion of physical behavior and has been used widely

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


DOI: 10.1002/cav

Figure 1. Palpation force model between finger and object.

DYNAMIC TOUCH-ENABLED VIRTUAL PALPATION* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

and effectively. Typically, an object is modeled as a

collection of point masses connected by springs in a

lattice structure. The equations of motion for the entire

system are assembled from the motions of the mass

points in the lattice. Early works on the mass-spring

model related to human objects can be found in facial

animation.20 Due to the simplicity and low compu-

tational cost for interactive applications, mass-

spring-based human organs are used to realize the

endoscopic surgical training in virtual environments.21

The finite element method (FEM)22 is another choice for

researchers to construct physically accurate model in

surgical simulation. In FEM, the object is divided into a

set of elements jointed at discrete nodal points. The

continuity between the elements is guaranteed by

obeying the constraints at nodal points and element

boundaries. Advanced FEM simulations involved non-

linearity and anisotropy.23 A major drawback with FEM

is that it involves intensive computation, making

real-time interactive applications prohibitive. Improve-

ments have been made by using pre-computation and

static condensation methods, space and time adaptive

level of detail,24 or an accelerated hybrid condensation

method on the GPU.25

Haptic Body-based PalpationModel

When simulating the palpation of the virtual diagnosis

process, a combination of multi events, including tissue

deformation, pressing, and sliding, need to be con-

sidered. It ensures the capability of simulating any

possible situations in real palpation.

Basic Palpation Force Model

The virtual index finger is modeled with a triangle-mesh

surface object. The basic palpation force model between

the virtual index finger and the tissue is constructed as

follows (Figure 1),

~F ¼ ~Fc þ ~Fm þ ~Fa (1)

where ~Fc is the contact force between two solids based

on Hertz’s contact theory specified in Equation (2); ~Fa is

the ambient force in relation to the virtual finger, for

example, the gravity ~Fg of the virtual finger and other

compensation forces to balance the downward force of

stylus tip of the haptic device; ~Fm is the frictional force

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

Copyright # 2007 John Wiley & Sons, Ltd. 341

caused by the roughness of the tissue surface in relation

to the contact force and the gravity; and ~F is the

integrated palpation force applied to the haptic inter-

face.

When multi contacts are detected between the virtual

index finger and the interacted tissue, each palpation

force is evaluated using Equation (1), and the final

compound force is applied to the user through the haptic

interface.

ContinuumHertz’s Contact Load

Hertz’s contact theory yields stresses, deformation, and

the shape of the interface formed at two contacting

bodies. These quantities depend on elastic properties,

the object shape, the relative position of the two bodies at

the point of contact, and the force pushing them

together. Although original Hertz contact load is based

on the smooth (frictionless) contact, it can also be

developed to account for rough (frictional) surfaces.

Assuming h�R, and using the inverse of Hertz’s

contact theory14 based on solid bodies in contact, the

contact force ~Fc exerted on the tool by the elastic

deformation of the object is expressed below and shown

in Figure 2,

~Fc ¼ h32 4

3

1

l1 þ l2

R1R2

R1 þ R2

� �12

l1 ¼ 1 � v21

E1l2 ¼ 1 � v2

2

E2

(2)

where h is the penetration depth between two contact

bodies, vi (i¼ 1,2) is the Poisson’s ratio and Ei (i¼ 1, 2) is

the Young’s modulus that describe the elastic properties

of two contact bodies respectively.

Moreover, the distribution of the pressure over

the contact area is given by the radius of the contact

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


DOI: 10.1002/cav

Figure 2. Normal Hertz contact load on smooth surface (without frictional force), and frictional force distribution

on contact area.

Figure 3. Virtual finger representation.

H. CHEN ET AL.* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

circle and the expression of the pressure, which is

exerted on point j in the contact area. They are defined

as follows:

a ¼ ~F13c

34 ðl1 þ l2Þ

R1R2

R1 þ R2

� �13

~Pð~jÞ ¼ � 3~Fc2pa2

1 �~j�� a2

0@

1A

12

~P0 ¼ � 3~Fc2pa2

(3)

where j is measured from the center of the contact

region, and a is the radius of the contact area. ~P0 specifies

the contact pressure at the center of the contact area.

The frictional force on the contact area is determined

by the contact force and the gravity force, as follows:

~Fmg ¼ m~Fg ¼ mmg ~FmcðjÞ ¼ m~PðjÞ~Fmc ¼

R Rj<a

~FmcðjÞds ~Fm ¼ ~Fmg þ ~Fmc (4)

where m is the friction coefficient depending on the

roughness of the object, ~Fmg is the frictional force in

relation to the gravity of each body, ~Fmc is the frictional

force caused by the contact force. ~FmcðjÞ describes the

frictional force of the unit area to the locally exerted

pressure, shown in the right side of Figure 2. The

integration over the entire contact area is superposed to

the final ~Fmc. Finally ~Fm is the integrated dynamic

frictional force between the virtual index finger and the

tissue.

Discretized Approximation

In the simulator of Hertz’s contact load, the body-based

contact18 is applied during virtual finger-tissue palpa-

tion. An important step is to describe the flatness of the

tissue at the contact area dynamically, in which the mean

curvature is estimated discretely in real time.

Virtual Finger Representation. The virtual index

finger is modeled as a single-layer of sphere bodies

bounding the surface of the forefinger in advance. Each

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


sphere body siðmi; ri; vi;EiÞ in the layer has four

attributes. mi is the mass of sphere body si, ri is it’s

radius, vi is the Poisson’s ratio, and Ei is it’s Young’s

modulus (fundamental elastic constants reflecting the

stiffness of the forefinger).

Suppose r(x,y,z) is the density distribution in the

finger. It’s mass M and center of the gravity ðx; y; zÞ can

be acquired. The equivalent sphere body layer of the

virtual index finger is constructed in keeping the mass

consistency within the finger. Suppose n sphere bodies

are applied in finger simulation and ri is the density

distribution within each sphere body. The mass and the

radius attribute are constructed in the following way,

mi ¼M

n¼ 1

n

ZZZV

rðx; y; zÞdxdydz � ri4p

3r3i (5)

Utilizing the above equivalent transformation, the

index finger is modeled as the volume of sphere bodies

in advance and the tip of the haptic device is attached to

the center of gravity. In Figure 3, the left figure model

shows the simplest simulation with one sphere attached

to the finger tip, and the right one is simulated with eight

spheres to construct the volume of sphere bodies.

Tissue Flatness Detection. Taubin26 showed that

the symmetric matrix Mp at point p on a smooth surface

Mp ¼ 12p

R p

�pkpðTuÞTuTt

uduk1p ¼ 3m1

p �m2p k2

p ¼ 3m2p �m1

p(6)

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


DOI: 10.1002/cav

Figure 4. Tetrahedral mass-spring system.


has the equivalent eigenvectors to the principal direc-

tions {T1 T2} and the eigenvalues {m1p, m2

p}. Where {m1p,

m2p} are related to the principal curvatures {k1

p, k2p}

through a fixed homogenous linear transformation. As a

result, the mean curvature can be acquired by

kp ¼ ðk1p þ k2

pÞ=2.

An approximation of the above matrix on a discrete

mesh is also given by Taubin,

~Mp ¼ 12p

PvikiTiTt

i (7)

where ~Mp denotes the approximation of Mp at vertex p

through the combination of a finite set of directions Ti

and curvatures ki.vi is a discrete weight version of the

integration step and has the constraintP

vi ¼ 2p. The

two principal curvatures can be acquired by the eigen

analysis of matrix ~Mp.

Therefore, the estimation of the mean curvature at the

contact point p is transformed into the curvature voting

of the vertices within q-rings’ adjacent neighborhood

AdjðpÞ (where AdjðpÞ ¼ fvjDistðp; vÞq, Distðp; vÞ is the

shortest path connecting point p with point v). Each

vertex vi 2 AdjðpÞ has the curvature ki, along the

direction Ti,

ki ¼D#i

DSi; Ti ¼

~ti~ti�� ; ~ti ¼ pvi

�!� ðNtp pvi�!ÞNp (8)

where D#i is the change in the angle, and Dsi is the

shortest arc length fitting from vi to p. And D#i is

obtained by the following,

cosðD#iÞ ¼Nt

p ni!

ni!��

ni!¼ Nvi � ðPt

iNviÞPi Pi ¼ Np � Ti

(9)

where Nvi is the normal at vertex vi, and ni! is the

projection to the plane defined at point pwith the normal

Pi. Through collecting all voted curvatures, the discrete

matrix ~Mp in Equation (7) is obtained. The mean

curvature radius 1=kp in Equation (2) is evaluated as

the simulated radius at the contact area of the tissue

during virtual palpation.

Touch-enabled Soft-TissueDeformation

Human soft tissues are combinations of various proteins

and tissue fluid which exhibit complex biomechanical

characteristics. The study of biomechanics shows that

soft tissues are non-linear, time-dependent and history-

dependent viscoelastic materials. It is difficult to

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


precisely express the complex behavior of soft tissues

in computer-generated organs. Here, soft tissues are

modeled with viscoelastic behavior by a volumetric

tetrahedral mass-spring system with attention focused

on small deformations restricted into a local area.

Viscoelastic Behavior

Human organs in the study are constructed by

volumetric tetrahedral meshes. It is convenient to

extract multiple iso-surfaces among the different tissues.

Besides, the tetrahedral element can support modeling

of 3D organs with arbitrary shape.

The volumetric tetrahedral mass-spring system con-

sists of mass points and connected springs along the

edges (Figure 4). The Viogt rheological model in Figure 4

(left) is used to depict the time-dependent viscoelastic

behavior of tissues. The linear springs obey the Hook’s

law, whereas the viscous dampers generate a resistance

force proportional to the velocity. The dynamics of

points are governed by the Newton’s Second Law of

motion. The nodal displacement of the ith point (ui2R3)

due to an external force Fi is given by the following,

mi€ui þ di _ui þPj

sijð ~rijj j�lijÞ~rijj j ~rij

�� ¼ Fi (10)

where mi is the mass of the point i, di is the damping

constant of the same point, r*

ij is the vector distance

between point i and point j, lij is the rest length, and sij is

the stiffness of the spring connecting two mass points.

The right-hand term Fi is the sum of other external

forces.

The motion equations for the entire system are

assembled through concatenating the position vectors

of the N individual mass points into a single

3N-dimensional position vector U. Then the Lagrange’s

dynamics equation is satisfied,

M €U þD _U þ KU ¼ F (11)

where M, D, and K are the 3N� 3N mass, damping, and

stiffness matrices, respectively. M and D are diagonal

matrices and K is banded because it encodes spring

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


DOI: 10.1002/cav

H. CHEN ET AL.* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

forces which are functions of distances between

neighboring mass points only. The vector F is

a 3N-dimensonal vector representing the total external

forces acting on the mass control points.

Dynamics Equations Solver

If we regard the points U as a function related to the time

t in a pure physical system, the resulting time-varying

algebraic system (11) can be solved using iterative

methods. The derivatives are replaced with their

discrete approximations. It is natural to make use of

previous states to estimate the current state, namely the

backward differences method. We tested the exper-

imental results on a planar regular triangular mesh with

a force. The dynamics equation is solved by both the

backward difference and the central difference respect-

ively. Although the resulting surfaces have minimal

visual differences, based on the fact that the central

difference is the best approximation to a derivative, we

use the central differences instead, expressed as,

€UðtÞ ¼ Uðtþ DtÞ � 2UðtÞ þUðt� DtÞDt2

_UðtÞ ¼ Uðtþ DtÞ �Uðt� DtÞ2Dt

Initially we assume that Uð�DtÞ ¼ Uð0Þ, then we can

solve the time-varying algebraic system by recursively

computing Uððnþ 1ÞDtÞ from the linear equation,

ð2MþDDtÞUððnþ 1ÞDtÞ

¼ ð4M� 2Dt2KÞUðnDtÞ

þ ðDDt� 2MÞUððn� 1ÞDtÞ þ 2Dt2FðnDtÞ (12)

Hence, we can solve the above linear equation

efficiently with only a single LU-decomposition of the

coefficient matrix during the iterative procedure.

The linear equation has an order of N since obviously

all the points are involved in the system. In many cases, a

Tissue Mass density (kg/m3)

Finger 137028

Skin 110027

Muscle 104127

Bone 199027

Ligament 111027

Upper leg 102027

Table 1. Parame

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


few forces are applied locally and influence in the parts

far away from the contacted points can be ignored. We

can reduce the order of the linear system by approxi-

mately fixing the vertices far from the acting forces.

Suppose N0 is the number of points within the

influenced q-rings neighborhood of the contacted points,

then Equation (12) may be simplified from N-order to

N0-order.

Implementation andExperiment

An interactive touch-enabled soft-tissue discrimination

system has been developed under a Windows NT 4.0

platform on a Dell workstation PWS420 with a single

Pentium III 733MHz Processor, and 256M RAM. We

implemented the system with Visual Cþþ 6.0, using

OpenGL 1.3 for graphics rendering and GHOST SDK 3.0

library for force calculation. The haptic feedback device

is a PHANToM Desktop with 6 DOFs input and 3 DOFs

force feedback. Experiments were conducted to validate

the physical properties and evaluate the computation

performance of the proposed touch-enabled virtual

palpation model.

Experiment Setting

The experiments are set up to measure the simulated

palpation force just prior to transmission to the haptic

device. The produced biomechanical behavior of the

human tissues is also evaluated. Table 1 lists the physical

properties of the virtual finger pad and tested tissues.

The virtual index finger is simulated with the properties

of silicone rubber. The four typical tissue categories,

including skin, muscle, ligament, and bone, are

simulated. In addition, an upper leg portion is also

simulated as a typical limb organ, which consists of

various tissues of the human body.

Poisson’s ratio Young’smodulus (kPa)

0.5029 205030

0.3331 72.431

0.432 6(relaxed)32

0.333 1.7�107,34

0.4535 33.135

0.327 0.827

ter settings

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


DOI: 10.1002/cav

Figure 5. Simulated human tissue models.


Figure 5 shows the tested environment of the virtual

index finger and five tissue models as according to

Table 1. The left most image shows the cuboid

tetrahedral volume interacting via the virtual index

finger, which is used to test individual tissues such as

skin and muscle. To the right are models of the ligament,

the cortical bone, and the upper leg of human body,

respectively.

Soft-tissue Simulations

Figure 6 records the palpation force generated between

the virtual index finger and the touched tissues. The

Figure 6. Force evaluation between finger and tissues.

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


force is depicted as a regression via a natural logarithm

function. The forefinger tip is simulated with one sphere

of radius 11 mm attached to the haptic input. In Figure 6,

the top graph shows the palpation force in relation to the

indentation, and the bottom graph presents the force

computed in relation to the local geometry of the

contacted tissues. The force simulated for bone is

multiplied by a coefficient of 0.1 to translate into the

force reflection range of the PHANToM Desktop. Our

experimental results of virtual fingertip perception

show similar results to the finger pad interface proposed

by Pawluk and Howe16,17 in physics. Besides, the

simulated non-linear force curves are similar to the

typical load-deformation curves exhibited by corre-

sponding human tissues.

Figure 7 illustrates the time-dependent viscoelastic

mechanical properties of the simulated soft human

Figure 7. Simulated soft-tissue responses.

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


DOI: 10.1002/cav

Data model Data scale(nodes/surf.nodes/surf.triangles/tetrahedrons)

Collisiondetection

Forceevaluation

Deformationtime

Cube 140/90/120/432 0.12 0.25 0.08Head 456/320/420/1470 0.29 0.36 0.15Ligament 601/438/680/1900 0.37 0.43 0.21Upper leg 728/416/668/2550 0.31 0.57 0.29

Table 2. Time statistics (ms)

H. CHEN ET AL.* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

tissues. Experiments are tested with constant stress

applied in the beginning, and then withdrawn accord-

ingly. In the top plot, the time step used is 0.01 to

simulate the skin and the ligament tissues, and in the

lower plot, the time step used is 0.05 to simulate the

muscle and a portion of the upper leg. The left part of

the curve shows that the simulated creep is an increase

in strain under constant stress. And the right part of the

curve shows that relaxation is a decrease in stress under

constant strain. With higher elastic parameters in the

simulated skin and ligament, the relaxation process is

prolonged as it is more difficult to calm down to rest

stage. The experimental results depict similarity in

characters to that of simulated soft tissues in Reference

[27] and the non-linear load-deformation characteristics

are similar to the palpation force sensing via the virtual

index finger.

Time Evaluation

Table 2 records the time statistics during dynamic

touch-enabled virtual palpations. The virtual index

finger is modeled with a triangle mesh of 179 triangles.

The data scale of the tested models is listed in Table 2 by

the number of nodes in the entire model, the number of

nodes and triangles in the surface, and the tetrahedral

numbers of the whole model. For dynamic finger-tissue

collision detection, a free convenient ColDet (http://

photoneffect.com/coldet) 3D collision detection library

is modified to detect multi contacts simultaneously. The

working rate of this library is sufficient in considering

the complexity of our experiments; at most 6 ms for

collision detection between two objects with 3186 and

4812 polygons. The cost of force evaluation is mainly

contributed by the curvature evaluation of the contacted

tissues. Through restricting the deformation of the

whole object within a local area, the refreshing rate of

dynamic deformation during virtual palpation is very

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


fast. Our proposed dynamic touch-enabled virtual

palpation model can guarantee a high refreshing rate

of haptic interaction (1 kHz) and meet the requirements

of our visual update.

Conclusion

The proposed dynamic touch-enable virtual palpation

model is a suitable method to sense the force perception

and exhibit the typical soft-tissue behavior in the virtual

environments. A body-based force contact model based

on Hertz’ theory has been applied to model the

palpation force via the virtual index finger. It is more

natural to utilize the physical properties, Poisson’s ratio

and Young’s modulus, of different tissues directly to

simulate the palpation perception on the virtual index

finger. The non-linear viscoelastic behavior of human

tissues is simulated via a volumetric tetrahedral mass-

spring model, and high performance in computing the

deformation is acquired through considering the contact

influence in a local area. This can highly reduce the order

of the whole dynamic equation. Mechanical tests have

been performed to evaluate the feasibility of the

proposed model to sense and manipulate some typical

tissues. The results demonstrate similar characteristics

to the biomechanical behavior of human tissues in the

real world. The system is implemented on a low-cost PC

platform with a commercial haptic device, and may thus

be popularized in the medical learning. Current

PHANToM Desktop device applied does not have

torque feedback thus restrict the perceptual simulation

with rotation. The multi spheres index finger model can

achieve smoother force reflection18 though it requires

higher computational cost of soft-tissue response; future

work will include a means to obtain a balance between

the two aspects. Further more, our future work will also

focus on developing both more rigorous finite element

model of tissues behavior and the haptic force model to

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


DOI: 10.1002/cav


effectively distinguish different tissues by various

characters, such as size, haptic texture, and tenderness,

etc. The outmost of our goal is to incorporate the

proposed realistic haptic mode into a VR-based training

simulator.

ACKNOWLEDGEMENTS

The work described in this paper was supported by a grant

from the Research Grants Council of the Hong Kong Special

Administrative Region, China (Project No. CUHK4223/04E),

and UGC direct grant for research (No. 2050349) and National

Fundamental Research Grant of Science and Technology (973

Projeect: 2002CB312104).

References

1. Morris D, Sewell C, Barbagli F, Salisbury K, Blevins NH,Girod S. Visuohaptic simulation of bone surgery for train-ing and evaluation. IEEE Computer Graphics and Applications2006; 48–57.

2. Basdogan C, Ho CH, Srinivasan MA. Virtual environ-ments for medical training: graphical and haptic simulationof laparoscopic common bile duct exploration. IEEE/ASMETransactions on Mechatronics 2001; 6(3): 269–285.

3. Delson NJ, Koussa N, Hastings RH, Weinger MB.Quantifying expert vs. novice skill in vivo for developmentof a laryngoscopy simulator, In Proceedings of MedicineMeetsVirtual Reality; pp 45–51.

4. Heng PA, Cheng CY, Wong TT, et al. A virtual-realitytraining system for knee arthroscopic surgery. IEEE Trans-cations on Information Technology in Biomedicine 2004; 8(2):217–227.

5. Heng PA, Wong TT, Yang R, et al. Intelligent inferencingand haptic simulation for Chinese acupuncture learningand training. IEEE Transactions on Information Technology inBiomedicine 2006; 10(1): 28–41.

6. Kaufman D, Bell W. Teaching and assessing clinical skillsusing virtual reality. Studies in Health Technology and Infor-matics: Medicine Meets Virtual Reality 1997; 39: 467–472.

7. Burdea G, Patounakis G, Popescu V, Weiss RE. Virtualreality training for the diagnosis of prostate cancer,In Proceedings in IEEE International SymposiumVirtual Realityand Applications (VRAIS’98), Atlanta, GA, March, 1998; pp.190–197.

8. Burdea G, Patounakis G, Popescu V, Weiss RE. Virtualreality-based training for the diagnosis of prostate cancer.IEEE Trans Biomedical Engineering 1999; 46(10): 1253–1260.

9. Popescu V, Burdea G, Bouzit M. Virtual Reality SimulationModeling for a Haptic Glove, Computer Animation’99 Con-ference, Geneva, Switzerland, 26–28 May 1999; pp. 195–200.

10. Langrana N, Burdea G, Ladeji J, Dinsmore M. Humanperformance using virtual reality tumor palpation simu-lation. Computers Graphics 1997; 21(4): 451–458.

11. Dinsmore M, Langrana N, Burdea G, Ladeji J. VirtualReality Training Simulation for Palpation of Subsurface

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


Tumors, IEEE International Symposium on Virtual Realityand Applications (VRAIS’97), Albuquerque NM, March,1997; pp. 54–60.

12. Barbagli F, Frisoli A, Salisbury K, Bergamasco M.Simulating human fingers: a Soft Finger Proxy Modeland Algorithm, Haptics Symposium 2004, vol. 1; pp. 9–17.

13. Daniulaitis V, Alhalabi MO, Kawasaki H, Tanaka Y.Medical Palpation of Deformable Tissue using Physics-Based Model for Haptic Interface RObot (HIRO),In Proceedings of 2004 IEEE/RSJ International Conference onIntelligent Robots and System (IROS 2004), 2004, vol. 4; pp.3907–3911.

14. Landau LD, Lifshitz EM. Theory of Elasticity Vol. of Courseof Theoretical Physics. Pergamon Press: Oxford [Oxford-shire], New York 1986; 26–31.

15. Borodich FM, Galanov BA. Self-similar problems of elasticcontact for non-convex punches. Journal of Mechanics andPhysics of Solids 2002; 50(11): 2441–2461.

16. Pawluk D, Howe R. Dynamic lumped element response ofthe human fingerpad. Journal of Biomechanical Engineering1999; 121: 178–183.

17. Pawluk D, Howe R. Dynamic contact of the humanfingerpad against a flat surface. Journal of BiomechanicalEngineering 1999; 121: 605–611.

18. Chen H, Sun H. Body-based haptic interaction model fortouch-enabled environments. MIT Journal of PRESENCE:Teleoperators and Virtual Environments 2006; 15(2): 186–203.

19. Gibson SF, Mirtich B. A Survey of Deformable Modeling inComputer Graphics. Technical Report TR97-19, MitsubishiElectric Research Laboratory, Cambridge, MA, USA,November, 1997.

20. Waters K, Terzopoilous D. Modeling and animating facesusing scanned data. Visualization and Computer Animation1991; 2: 123–128.

21. Kuhnapfel U, Cakmak HK, Maab H. Endoscopic surgerytraining using virtual reality and deformable tissue simu-lation. Computers and Graphics 2000; 24(3): 671–682.

22. Bathe KJ. Finite Element Procedures. Prentice Hall: Engle-wood Cliffs, NJ, 1996.

23. Picinbono G, Delingette H, Ayache N. Non-linear andanisotropic elastic soft tissue models for medical simu-lation, Proceedings of IEEE International Conference on Roboticsand Automation, 2001; pp. 1370–1375.

24. Debunne G, Desbrun M, Cani MP, Barr A. Dynamicrealtime deformations using space & time adaptivesampling, Proceedings of ACM SIGGRAPH, 2001; pp. 31–36.

25. Wu W, Heng PA. A hybrid condensed finite elementmodel with GPU acceleration for interactive 3D soft tissuecutting. Computer Animation and Virtual Worlds (CAVW)Journal 2004; 15(3–4): 219–227.

26. Taubin G. Estimating the tensor of curvature of a surfacefrom a polyhedral approximation. In Proceedings of the FifthInternational Conference on Computer Vision, 1995; pp. 902–907.

27. Duck FA. Physical properties of tissue: a comprehensive refer-ence book. Academic Press: New York, 1990.

28. http://www.lib.umich.edu/dentlib/Dental_tables/Elas-mod.html

29. Yang X, Grosjean C, Tai Y. Design, fabrication, and testingof micromachined silicone rubber membrane valves. Jour-nal of Microelectromechanical Systems 1999; 8(4): 393–402.

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


DOI: 10.1002/cav

H. CHEN ET AL.* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

30. http://www.lib.umich.edu/dentlib/Dental_tables/Density.html

31. Atre A, Johnson WS. Analysis of the Effects of RivetingProcess Parameters on the Fatigue of Aircraft FuselageLap-Joints, 9th Joint FAA/DoD/NASA Aging Aircraft Confer-ence, 2006, Online: http://www.agingaircraftconferen-ce.org/all_files/22/22b/146_doc.pdf

32. Burton AC. Physiology and Biophysics of the Circulation: AnIntroductory Text. Year Book Medical Publishers: Chicago,1965.

33. Farah JW, Craig RG, Meroeh KA. Finite element analysisof three- and four-unit bridges. Journal of Oral Rehabilitation1989; 16: 603–611.

34. McBeil AR. Animal Mechanics. Blackwell Scientific: Oxford,1983.

35. Titze IR, Min YB, Alipour-Haghighi F. Stress-strainresponse of the human vocal ligament and its effecton F0 control, In Proceedings of 128th Acoustical Society ofAmerica Meeting, 5aSP—Speech Communications, Texas,Austin, November, 1994.

Authors’ biographies:

Hui Chen received her B.S. and M.S. degrees in Com-puter Science from Shandong University, P.R. China,and received Ph.D. in Computer Science from the Chi-nese University of Hong Kong, Hong Kong. She iscurrently an Assistant Professor in the Centre forHuman-Computer Interaction, Shenzhen Institute ofAdvanced Integration Technology, CAS/CUHK, SIAT,P.R. China. Her research interests include haptics, vir-tual reality, computer-assisted surgery, and computergraphics.

Wen Wu received her Ph.D. in the Department ofComputer Science and Engineering at the Chinese Uni-versity of Hong Kong. She is currently an Assistant

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


Professor in the Faculty of Information Technology atMacau University of Science and Technology. Herresearch interests include surgical simulation and com-puter graphics.

Hanqiu Sun received her Ph.D. in Computer Sciencefrom the University of Alberta, Canada. She is an Associ-ate Professor at the Department of Computer Scienceand Engineering, The Chinese University of Hong Kong.She has published more than 100 technical papers refer-eed in VR/CG journals and international conferences.Her current research interests include virtual and aug-mented reality, interactive graphics/animation, hyper-media, computer-assisted surgery, Internet-basednavigation, real-time rendering, and realistic hapticssimulation.

Pheng-Ann Heng received his B.Sc. degree from theNational University of Singapore, in 1985. He receivedM.Sc. (CS), M.Art. (Applied Math), and Ph.D. (CS)degrees from Indiana University, Bloomington, in1987, 1988, and 1992, respectively. He is currently aProfessor in the Department of Computer Science andEngineering, Chinese University of Hong Kong(CUHK), Shatin. In 1999, he set up the Virtual Reality,Visualization and Imaging Research Centre at CUHKand serves as the Director of the Centre. He is also theDirector of the CUHK Strategic Research Area in Com-puter-Assisted Medicine, established jointly by theFaculty of Engineering and the Faculty of Medicine in2000. His research interests include virtual reality appli-cations in medicine, scientific visualization, 3D medicalimaging, user interface, rendering and modeling, inter-active graphics, and animation.

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *


DOI: 10.1002/cav

Documents

Dynamic touch-enabled virtual palpation