CRANFIELD UNIVERSITY CRANFIELD HEALTH MSC BIOINFORMATICSmembres-timc.imag.fr/Emmanuel.Promayon/pdf/Reynier07.pdf · 2007. 8. 31. · Reynier Michael. MSc Bioinformatics GPU accelerated

CRANFIELD UNIVERSITY

CRANFIELD HEALTH

MSC BIOINFORMATICS

ACADEMIC YEAR 2006 – 2007

GPU accelerated deformable model for soft tissues and surgical

simulation

Reynier Michael

MSc BioinformaticsGPU accelerated deformable model for soft tissues and surgical simulation

ABSTRACT

Nowadays, surgeons have to deal with more and more complicated interventions. Model of human

soft tissues can help surgeons by providing further information on the deformation of specific

organ. The model developed in the Computer Assisted Medical Intervention laboratory in Grenoble

is modelling deformations of organs and interaction with needles.

In order to improve the processing time of this model, the Graphic Processing Unit (GPU) has been

used for its capacity to to parallel computation. These processors have already been used to

compute mathematical operations. Today, GPU is not only used to display images but also to

improve computation time. Programming GPUs needs other methods than usual CPU algorithm and

other tools like texture considered as an array in the graphic card.

This project describes the different step to use these specialized processors. Models' deformation

from four to more than two hundred particles have been tested in order to study the efficiency of the

graphic card.

I


ACKNOWLEDGEMENTS

First, I would like to thank all the team “ComputerAssisted Medical Interventions” from the

laboratory TIMC at Grenoble for welcoming me . I would also like to thank more especially the Dr

Emmanuel Promayon from the Laboratory TIMC for proposing this project to me, and for his help

and support.

I would also like to thank my supervisor Dr Conrad Bessant from Cranfield University for

all the advice that he gave me when I needed it.

II


TABLE OF CONTENTSABSTRACT............................................................................................................................................... ......I

ACKNOWLEDGEMENTS.................................................................................................. ........................II

LIST OF FIGURES.........................................................................................................................VI

Chapter 1 Introduction and Literature Review..............................................................................1

1.1 Modelling..................................................................................................................................2

1.1.1 Deformation modelling.....................................................................................................2

1.1.2 Springmass model............................................................................................................3

1.1.3 Shape memory model (phymul) .......................................................................................5

1.2 Graphic programming and GPU.............................................................................................10

1.2.1 Graphics pipeline............................................................................................................10

1.2.2 Graphics API (Application Programming Interfaces)....................................................12

1.2.3 Shader and Shading Language........................................................................................12

1.2.4 Cg....................................................................................................................................13

1.2.5 Why GPU?......................................................................................................................14

1.2.6 Vertex Vs Fragment Processors......................................................................................16

1.3 Related work in GPU based deformable model.....................................................................18

1.4 Project aim..............................................................................................................................19

III


Chapter 2 Materials and Methods.................................................................................................20

2.1 Hardware and Programming Languages used........................................................................20

2.1.1 Hardware.........................................................................................................................20

2.1.2 Programming language...................................................................................................21

2.2 Implementation of a particle system on the CPU...................................................................21

2.2.1 General evolution of the model.......................................................................................22

2.2.2 Initialisation of particles..................................................................................................24

2.2.3 Attractor position............................................................................................................26

2.2.4 Compute elastic forces....................................................................................................26

2.2.5 Compute reaction forces.................................................................................................27

2.2.6 Update Position...............................................................................................................27

2.2.7 Display particles..............................................................................................................27

2.3 Implementation of a particles system on the GPU.................................................................28

2.3.1 Texture as array ..............................................................................................................28

2.3.2 Data transfer....................................................................................................................29

2.3.3 Algorithm to process a fragment shader.........................................................................29

2.3.4 Particle neighbourhood access........................................................................................31

2.3.5 The shaders.....................................................................................................................34

2.4 Display Particles ....................................................................................................................40

IV


2.5 Measure of the processing time..............................................................................................41

Chapter 3 : Results..........................................................................................................................42

3.1 The tetrahedron.......................................................................................................................42

3.2 The regular cylinder................................................................................................................44

Chapter 4 Discussion.......................................................................................................................49

4.1 Discussion of results...............................................................................................................49

4.2 Difficulties and issues ............................................................................................................50

Chapter 5 Project conclusions........................................................................................................52

5.1 Further work...........................................................................................................................52

5.2 Conclusions.............................................................................................................................53

Bibliography......................................................................................................................................54

V


LIST OF FIGURES

Figure 1.1: Deformation of an organ with a needle.............................................................................2

Figure 1.2: springmass model.............................................................................................................3

Figure 1.3: Shape memory force F* applied to a particle P.................................................................6

Figure 1.4: Example of a particle with 2 neighbours...........................................................................7

Figure 1.5: Example of a particle with 3 neighbours...........................................................................7

Figure 1.6: General algorithm of the model.........................................................................................9

Figure 1.7: Example of coordinate texture associated with a cylinder..............................................10

Figure 1.8: The graphics pipeline from (Randima et al, 2006)..........................................................11

Figure 1.9: Graphics pipeline example from (Randima et al, 2006)..................................................12

Figure 1.10: The Programmable Graphics Pipeline from (Randima et al, 2006)..............................14

Figure 1.11: MassSpring sytem, updated with a scatter...................................................................17

Figure 1.12: MassSpring system with only gather operations..........................................................17

Figure 2.1: Flowchart representing the steps carried out by the model.............................................23

Figure 2.2: Flowchart of particle's initialisation.................................................................................25

Figure 2.3: Algorithm to compute the attractor of a particle.............................................................26

Figure 2.4: Algorithm to compute elastic force of a particle.............................................................26

Figure 2.5: Algorithm to redistribute forces.......................................................................................27

VI


Figure 2.6: Verlet integration to compute new positions...................................................................27

Figure 2.7: Representation of one pixel which store the position's particle.......................................28

Figure 2.8: Example of an addition of two textures...........................................................................30

Figure 2.9: Flowchart to compute a fragment shader on the graphics card.......................................31

Figure 2.10: Link between particle and their neighbours...................................................................33

Figure 2.11: Elastic force shader........................................................................................................34

Figure 2.12: Flowchart of the reaction force shader..........................................................................36

Figure 2.13: Positions update shader..................................................................................................37

Figure 2.14: Example of the content of the first pixel in the "IndexTexture"....................................38

Figure 2.15: Flowchart of the attractor shader...................................................................................39

Figure 2.16: Graphics pipeline describing the method "Renter to Vertex Array".............................41

Figure 3.1: Tetrahedron at the start of the simulation........................................................................43

Figure 3.2: Forced displacement of a particle....................................................................................43

Figure 3.3: Final position of the tetrahedron......................................................................................44

Figure 3.4: Initial position of the cylinder..........................................................................................45

Figure 3.5: Deformation applied on the cylinder...............................................................................46

Figure 3.6: Final positions..................................................................................................................47

VII


LIST OF TABLESTable 1.1: Comparison between CPU and GPU ...............................................................................15

Table 2.1: Notation in the algorithm..................................................................................................24

Table 3.1: Comparison of the processing time between the GPU and the CPU................................48

VIII


Chapter 1 Introduction and Literature Review

ComputerAssisted Medical Interventions (CAMI) consists in helping surgeons during their

interventions by using the latest technologies.

Nowadays it is possible to get a digital model of human organs and it can help surgeons to

define how they will operate and how this organs will react to the action of surgical tools.

As minimally invasive surgery become more and more common and the surgeon gestures

have to be more accurate, the CAMI laboratory team try to anticipate the different effect of

surgeons' gestures. The development of physical simulator of biological tissues is needed.

To create this digital model, we need to simulate tissue deformation on 3D models.

Deformable objects often use the massspring model (Montgomery et al., 2002). Another method

developed in TIMC and presented in (Marchal, 2006) uses the concept of a local shape memory.

This model tries to deal with complex interactions between a surgical tool and an organ as well as

the interactions between organs themselves (Marchal et al., 2006).

The simulation needs lots of computational resources and are particularly intensive for the

CPU (Central Process Unit). Recently, scientists started to use another hardware, the GPU

(Graphics Processing Unit),a specialized performing processor, more efficient than the CPU for

some specific task. Indeed, GPU is specialized in graphics task which allow to process matrix

operations in parallel. GPU is now use for a wide range of application and it has been recently

1


successfully used to directly compute matrix inversion and other mathematical operations (Latta,

2004), including soft tissue simulation for surgical simulator (Sørensen et al, 2006). General use of

the GPU is promoted by General Purpose Computation Using Graphics Hardware (GPGPU)

organisation.

1.1 Modelling

1.1.1 Deformation modellingModelling deformations can be used to simulate interaction between an instrument and an

organ (see Figure 1.1).

2

Figure 1.1: Deformation of an organ with a needle


1.1.2 Springmass modelThe springmass model is a physicallybased model for deformable model used in surgical

simulation when realtime behaviour is desired (see figure 1.2). It is a model which is represented

by 3D particles masses. These particles are connected through mechanical connections. These

connections are represented by linear springs.

Each particle i moves accordingly to the general equation of Newtonian motion :

m i Ẍ i=∑j=1

n j

F ij

Ẍ i is the acceleration of the considered particle (with the position X i ), m i its mass

3

Figure 1.2: springmass model


and the Fij the forces applied on the mass i by the n j neighbours.

The force applied on the particle due to springs is the elastic force. An elastic force applied

to the mass i neighbour of another mass j is :

F ij=k r∥X ij∥−lij .X

ij

∥Xij∥

k r representing the stiffness constant of the spring, lij the rest length of the spring

between the two mass and X ij the vector from the mass i to the mass j , ∥X ij∥ is the

current length of the spring. If two particles are connected by a spring, we considered the two

particles as neighbours.

In order to compute the dynamic evolution of the particle system, we need to use a standard

numerical integration scheme. One of the simplest is the Verlet integration scheme (Dummer,

2007). This method calculates the next position of each particle from its current and previous

positions and the acceleration vector.

X itdt=X idampingFactor∗X i

t−X it− dt ¨X i

tdt dt 2

The damping factor is usually close to 1.

4


1.1.3 Shape memory model (phymul) The phymul is a model developed in the CAMI laboratory team and use in the thesis of

(Marchal, 2006). This model also uses particles but with another calculation of forces applied on

particles. The aim of this model is to remove of all these springs which generally instabilities.

Connectivities are preserved but instead of getting as spring as connectivities, we get only one by

particle, calculated thanks to attractor link to the rest shape.

• Shape memory force

In this model, each particle has a shape attractor. The position P* of this attractor is the ideal

position of the particle (without any constraint). It means that when there is no deformation, P* and

P have the same coordinates. The force F* (shape memory force) is expressed as a distance

function between the position P of the particle and the one from its attractor P*.

F∗=k PP∗

k represents the weight of the force. The shape memory force F* is null when the particle

and its attractor are at the same position. In Figure 1.3, the particle has moved to position P. A

shape memory force F* is applied to the particle “attracting” it towards its shape attractor position

P*.

5


• Attractor computation

Considering P as the particle position and Ni (with i Є [1...n]) as the position of the n

particle's neighbours. The position P* of the attractor can be expressed according to the position of

the n neighbours.

In 2 dimension, we are interested in all the bases ( with i≠ j and i,j Є [1...n])

taken among the n neighbours. We can define :

P∗=Qi n

where Q is the projection of P along the normal n, with i representing the distance between Q

and P (see figure 1.4). Q can be calculated using the barycentric coordinate as:

Q=i∗N i j∗N j

6

Figure 1.3: Shape memory force F* applied to a particle P


with i j=1 . At each time step, P* can be considered as a measure of the discrete

deformation of triplet t.

In the example figure 1.4, the particle has 2 neighbours and thus only one base.

P∗=11 N 12

1 N 21 n

If there are more than 2 neighbours, there are more than one base and the parameters

has to be added.

For the base 1:

Q1=11 N1 2

1 N 2

7

Figure 1.4: Example of a particle with 2 neighbours

P

N1 N2Base 1

Beta1

Figure 1.5: Example of a particle with 3 neighbours

P

N1N2

N3

Base1Base2

Q1


For the base 2:

Q2=22 N 22

3 N 3

Finally :

Q=12

Q112

Q2=121

1 N 1212

2N 232 N 3

each neighbours has his parameter corresponding to :

1=11 2=2

122 3=3

2

For this example, we have also two , one for each base.

In 3 dimensions, bases are composed of three particles and create a triplet. Thus, all

calculation has one more coordinate but the method is the same as explained.

• General evolutionThere are different step of evolution of the model. First of all, we perform the computation

of the force (an Elastic Force in our case). Then, according to the law of ActionReaction (Newton's

3rd law), force is applied on all neighbours' particle. Finally, we calculate their new positions using

the Verlet Integration. The different steps of an iteration of the algorithm are presented in the Figure

1.6.

8


.

We will try to program this model on the CPU and the GPU (see chapter 2) and doing a

comparison. That is why, we will now introduce what the GPU is.

9

Figure 1.6: General algorithm of the model

For each particle ComputeElasticForce();RedistributeForce();ComputeNewPosition();

end


1.2 Graphic programming and GPU

1.2.1 Graphics pipeline 3D applications send many vertices to the GPU. Vertices are point which form geometric

primitives like polygons, lines and triangles. The common data attached to each vertex are its

coordinates, colour, texture coordinates and normal vector.

A texture is a 2D image composed of pixels (picture element). Texture coordinate are

associated with each vertex within a model, and indicate what point within the texture is attached to

that vertex ( Figure 1.7).

GPUs use several specialised internal hardware component in subroutine. These subroutines

are organised in pipeline.

“A pipeline is a sequence of stages operating in parallel and in a fixed order”(Randima et al,

2006). Figure 1.8 shows a typical graphics pipeline. The first step is the vertex transformation

10

Figure 1.7: Example of coordinate texture associated with a cylinder

2D Texture 3D Graphic


which transforms the vertex position into a screen position, generate texture coordinate and

determine the colour.

The next step is the primitive assembly. Transformed Vertices are assembled into geometric

primitives(triangles, line). Rasterization determines the set of pixels and fragments covered by a

geometric primitive. This step breaks up each primitive into pixelsized fragments for each pixel

that the primitive covers. Once it is done, each fragment gets a final colour in the stage “Fragment

Texturing and Coloring”. A fragment has different parameters such as colour, texture coordinates

and depth value.

Finally in the Raster Operations stage, fragments are eliminated if they are hidden thanks to

the depth testing. After the different raster operations, fragments become pixel.

Figure 1.9 shows the entire different step starting with several vertices. Triangles are filled

with fragments in the rasterizer and finally coloured at the end.

11

Figure 1.8: The graphics pipeline from (Randima et al, 2006)


The final stage is to store the entire pixels (actually the colour values for every pixel) in the

frame buffer. The video controller takes the data in the frame buffer and display them on the screen.

1.2.2 Graphics API (Application Programming Interfaces)Programming graphics card would be really difficult without passing through Graphic API.

The two leading API are OpenGL (Open Graphics Library) and Direct3D. We chose OpenGL

because it is compatible with both Windows and Linux. It is used to write applications that produce

3D computer graphics.

1.2.3 Shader and Shading LanguageA shader allows modifying vertices and fragments to create special effect. Shaders apply

their modification or calculation for every vertex or fragments and it is well suited to parallel

processing. There are several shading language which are HighLevel Shading Language (HLSL)

implemented by Microsoft Corporation, GL Shading Language (GLSL) created by the OpenGL

ARB consortium, RenderMan Shading Language by Pat Hanrahan (Pixar corporation) and finally C

12

Figure 1.9: Graphics pipeline example from (Randima et al, 2006)


for graphics (Cg) developed by NVIDIA corporation.

1.2.4 CgCg (C for Graphics) is a programming language, it is used for programming graphics

hardware and allow to program with a high level language. The Cg program is executed within the

GPU of a computer. Syntax is based on C language.

“Cg enables a specialized style of parallel processing” (Randima et al, 2006). A program

written in Cg is processed in parallel on the GPU and is very efficient. It can run with Linux and

Windows (compatible with OpenGL and DirectX) and can work with Vertex and Fragment.

Vertices and Fragments are programmable; these programs are called Vertex shader and

Pixel shader. Vertex shader will work mainly on the geometry of the object whereas pixel shader

will work with textures and colours of the object.

Figure 1.10 describes the graphic pipeline with the two programmable processor. The

Programmable Vertex Processor will run our Cg vertex program and the Programmable Fragment

Processor will run the fragment program.

13


The last step in the pipeline is the frame buffer. It is a buffer containing colour values of

every pixel.

1.2.5 Why GPU?

• AdvantagesThe main advantages of GPUs are their speed if we consider the comparison done by

(Harris, 2005).

14

Figure 1.10: The Programmable Graphics Pipeline from (Randima et al, 2006)


Table 1.1: Comparison between CPU and GPU

Processors Giga floating point operations per second Memory Bandwidth

3 GHz Pentium 4 6 GFLOPS 5.96 GB/sec

GPU Nvidia GeForce

FX 5900

20 GFLOPS 25.6 GB/sec

GPU Nvidia GeForce

6800 Ultra

40 GFLOPS 35.2 GB/sec

Moreover, GPU performance growth is higher than the CPU growth. Annual growth of GPU

is more than 2 whereas, according to Moore's law, CPU performance annual growth is 1.5. The

expansion of the video game market is the main a factor for this fast evolution of GPUs.

GPUs are now programmable and Highlevel languages are available (paragraph 3.5). GPUs

have a good precision with 32bit floating point throughout the pipeline.

• LimitationsGPU is a specialized processor initially and mainly designed for computer graphics

applications and not general applications. We need to reformulate algorithm as parallel tasks and

can not just copy a code from a CPU to a GPU. The architecture of these GPUs is rapidly evolving.

There is also the limitation of communication from GPU to CPU (data bandwidth) which is

really slow although communication should improve in the near future.

15


We can not access texture field in the pipeline. The only way to get texture fields back is to

transfer from the frame buffer to an array on the CPU. Output of the vertex processor can not be

directly access, vertex information needs to go until the end of the pipeline to be recovered.

Another limitation is that specific graphics API is needed to program GPU.

1.2.6 Vertex Vs Fragment ProcessorsModern GPUs have multiple vertex and fragment processors. The NVIDIA GeForce 6800

Ultra and the ATI Radeon X800 XT both have 6 vertex processors and 16 fragment processors

whereas the NVIDIA Quadro FX 3500 has 7 vertex and 20 fragment processors. This characteristic

is one of the main reason of the choice of the fragment processor for GPGPU applications. The

other reason is that the output of the fragment processors goes directly into memory whereas it is

not the case for vertex processors as it is mentioned in paragraph 1.2.5.

“Gather” and “scatter” are two words to know. A scatter operation is any memory write

operation from a computed address. Vertex processors are able to change the position of input

vertices, thus they are able to scatter (i.e. Scatter operation in C language : a[i]=x). A gather is

an indirect read operation, such as x=a[i].Vertex processors can not read information from vertex

and thus can not gather. Fragment processors can fetch data from textures but can not change

(write) the output location of a pixel. Fragment processors are thus able to gather but not capable to

scatter. We can work around this limitation as explained in (Pharr, 2005) section 32.3. This solution

16


explains how to convert a scatter operation into a gather operation. Indeed, considering a simple

MassSpring system in which we loop over each spring, we compute the force exerted by the spring

and add the force contribution to the masses connected to the spring (see Figure 1.11).

Converting the scatter in gather needs to introduce another loop (Figure 1.12 ).

This technique will be used in the chapter 2 to process forces in our model and explain the need to

use two shaders.

17

Figure 1.11: MassSpring sytem, updated with a scatter

For each springf = computedforce();mass_force[left] += f;mass_force [right] -= f;

Figure 1.12: MassSpring system with only gather operations

For each springf = computedforce();

For each massmass_force= f[left]-f[right];


1.3 Related work in GPU based deformable model

An overview of the many applications of GPGPU is available online (GPGPU) and in the

books in the GPU Gems series (Fernando, 2004)(Pharr, 2005)

(Green, 2007) has implemented a cloth simulation on a GPU. This demo is for MS

Windows and use two pass with fragment shaders to achieve the simulation. The first pass deal with

the cloth motion, it means that cloth vertices are updated to the new position by Verlet Integration.

The second pass check if the constraints are satisfied. This example handles collision with a sphere

and the floor.

In (Wang, 2005), another cloth simulation is done but this time with implementation on both

CPU and GPU. The cloth is modelled as a mesh hierarchy, with the coarsest mesh being the top

level. Position and collision of this top level mesh is calculated on the CPU. Mesh is sent onto the

GPU for refinement. Refinement consists in interpolating new point in the cloth to get an accurate

shape. This step can include more than one pass. The cloth is then rendered on graphics hardware.

18


1.4 Project aim

The main objective of this project is to study the possibility of using a GPU to compute the

model's deformation described in (Marchal, 2006)(see also paragraph 1.1.3) and implement the

solution. To reach this objective, several steps are needed :

● Understand GPU architectures and Application Programming Interface (API)

● Identify the different possibility of the GPU

● Find a way to implement the model on the GPU in order to get the best simulation rate

● Compare CPU and GPU performance in terms of speed

19


Chapter 2 Materials and Methods

This chapter describes methods used to implement the deformable model; specials

characteristics of the software design and implementation are discussed. The different

programming languages, as well as the hardware used are also detailed.

2.1 Hardware and Programming Languages used

All the different software was developed and tested using a desktop Computer (Dell)

running Linux Fedora core 6.

2.1.1 Hardware

The following hardware was on the computer:

● Graphic card :

○ Graphics Processor : Quadro FX 3500 (Nvidia)

○ Memory : 256 MB

○ Bus Type : PCI Express 16X

20


● CPU :

○ Processors : Two Dual Core Intel Xeon CPU 5140 @2.33GHz. (The CPU

implementation does not use any parallelization or thead, thus the comparison can

only consider the case of one Xeon CPU)

2.1.2 Programming languageThe different software and programming language used were :

● C++, compiler gcc 4.1.2

● Cg (C for graphics) used for the shade, compiler cgc 1.5

● Opengl 2.2, driver Nvidia 1.09762

● Kdevelop 3.4.1

2.2 Implementation of a particle system on the CPUIn order to compare the performance between the CPU and the GPU; the first thing to do

was to implement the Phymul model on the GPU. The first choice was to take a simple geometric

form which is a tetrahedron. It has four vertices which can be represented by four particles. A class

21


“Particle” was created to handle particles characteristic (positions in 3 dimensions, force, attractor

and bases).

2.2.1 General evolution of the modelThe program can be divided in several steps. First the creation of the particles and the

initialisation of all their parameters. As explained in paragraph 1.1.3, the distance between attractor

and particle positions correspond to the elastic force. When all the distance are null (or near to zero,

depending on the precision wanted), elastic forces are null and the model is at the rest position.

Thus, attractors and distance are computed in order to know if any forces are applied to the model.

If the model is not at the rest position, forces are computed and thus reaction forces. Finally, new

positions are calculated and OpenGL displays our particles on the screen.(see figure 2.1).

22


Steps of this flowchart are described in details in the next paragraphs. Algorithm will use the

same notation as indicated in the table 2.1.

23

Load particles

Initialisation

Compute elastic forces

Rest Position test

No

Yes

Update Positions

Display Particles

Compute attractor

Compute reaction forces

Figure 2.1: Flowchart representing the steps carried out by the model


Table 2.1: Notation in the algorithm

Notation Description

Bc Bases Count

dt Time step (in seconds)

Fni Force of the neighbour i

Fe Elastic Force

ni Normal of the base i

Ni Normal of the base i

Nc Neighbour Count

Np Neighbour Position

Pt Particle current position

Ptdt Particle position for the previous iteration

Pt+dt Particle position for next iteration

w Stiffness

i of the neighbour i

i of the neighbour i

2.2.2 Initialisation of particles

The initialisation needs to set different parameters. We have to consider all the possible

directions defined by the neighbour positions. Deformation for a particle is computed as the

24


barycentre of all possible discrete deformations (Marchal et al., 2006).

To achieve this initialisation, we need to get all the bases (triplet) which are composed of

three neighbour particles each (see section 1.1.3).

These parameters are initialized at the beginning of the algorithm : one consider the first

position to be the rest position. Once compute, these parameters will be used at each iteration to

compute the particle attractor.

The flowchart of the initialisation is described in the Figure 2.2.

25

Figure 2.2: Flowchart of particle's initialisation

Creation of all the bases

//NbNe = Number of neighbours//Ni=neighbour number ifor(i=0;i


2.2.3 Attractor positionAttractors are computed at each iteration. and are constants calculated only once

during the initialisation.

2.2.4 Compute elastic forcesWhen the new attractors and positions are calculated, we can compute the new forces

applied on each particles. The detailed algorithm is described in Figure 2.4.

26

Figure 2.3: Algorithm to compute the attractor of a particle

for (i=0;i


2.2.5 Compute reaction forcesEach force previously calculated is subtract to neighbour's forces. The detailed algorithm is

described in Figure 2.5.

2.2.6 Update PositionIn order to get the new particles' position, we need to apply forces on the particle. A

standard numerical integration schema is used to solve the system, which is the Verlet integration

(see section 1.1). The formula is presented in figure 2.6

2.2.7 Display particlesThis part uses vertex array in OpenGL. All the particles' position are sent in an array to

display a point corresponding to the particle.

27

Figure 2.5: Algorithm to redistribute forces

for (i=0;i


2.3 Implementation of a particles system on the GPU

For the GPU, the different steps of the algorithm are the same than for the CPU but have to

be done differently. The first two steps of the algorithm are still computed on the CPU (load and

initialisation of particles) because it is only done once, at the beginning of the simulation.

2.3.1 Texture as array The main difference to take in account in GPU programming is that a 2D array in the CPU

program will be replaced by a texture in a GPU program. Like 2D array, a texture can have

different size and has two dimensions. Because texture is composed of pixels and represent a

colour, pixels in textures are in the format RGBA (red, green, blue, alpha).Alpha is the value that

code transparency. It means that each pixel is made of four numbers or fields. All the fields in a

GPU texture are 32bit floatingpoint formats. Thus each pixel is stored on 128 bits.

On the CPU program, there are different array to store particles parameters like the

positions, forces or attractors. Each one of these arrays will become a 2D texture. All these

parameters are 3D vectors, so in order to simplified addressing problem, we will use one pixel per

vector. The field A representing the transparency will not be used (see figure 2.7).

28

Figure 2.7: Representation of one pixel which store the position's particle

x y

z


Accessing data in a texture depends of its “target”. There are two available texture target in

OpenGL, GL_TEXTURE_2D and GL_RECTANGLE. The first one uses normalized coordinates, it

means that coordinates are in the range [0,1]*[0,1]. GL_TEXTURE_RECTANGLE textures are

sampled with integer coordinates on [0,w]x[0,h] (with “w” the width and “h” the height in pixels).

For a GL_TEXTURE_2D texture of size (4*1), the second pixel would be at the coordinate (0,25 ;

0), the third at (0,5;0).

2.3.2 Data transferThe entire program was done in C++ with OpenGL. OpenGL is used to transfer of the data

from the array (on the CPU memory) to the texture (in the GPU memory). Another transfer from

the textures memory to the CPU memory is needed, in order to display the result on the screen.

2.3.3 Algorithm to process a fragment shaderFirst of all, we need to fill the input texture with our parameters. Parameter can be a single

float value but in most of the case several textures are sent to the graphics processor. We also have

to specify the output texture to get the result of our computation.

When textures are filled, the computation has to be triggered. Fragment shader is doing the

computation on each pixel drawn. Thus, the technique consist in drawing a quad covering all the

texture so that each pixel of the texture will be computed and rendered to the output texture. In the

29


figure 2.6, we can see a simple addition, each pixel from texture x is added to their corresponding

pixel in the second texture and the result is written to the output. Instead of a “for” loop like usual

program, the GPU is doing all additions in parallel.

Obviously, drawing a quad on the screen with result of an addition would not be really

useful. That is why, an offscreen buffer is used. Instead of putting results data in the Frame Buffer

or FB(see section 1.2.4), computed results will be transfer to this offscreen buffer called the Frame

Buffer Object(FBO) (Green, 05). Thus, FBO allow us to use an offscreen buffer as the target for

rendering operations such our particle's positions calculation. Before each computation, we have to

30

Figure 2.8: Example of an addition of two textures

4

1

87

2

5

3

6

9

13

10

1716

11

14

12

15

18

17

11

2523

13

19

15

21

27

Shader Programx+y

Texture x

Texture y

Output Texture


turn off the traditional FB and use the FBO.

The detailed algorithm to launch a fragment program can be seen in figure 2.9.

2.3.4 Particle neighbourhood accessThe entire particle movements depends of their neighbours. Neighbour's forces and

positions are needed to calculate attractor positions and forces for each particle. Each particle might

have a different number of neighbours and the problem is to access positions of these neighbours in

the “PositionTexture”.

31

Figure 2.9: Flowchart to compute a fragment shader on the graphics card

Fill input textures

Draw a quad to compute our shader

Disable the FBO

Enable the FBO foroffscreen rendering


To achieve this goal, two other textures are specifically designed to express the

neighborhood connectivity. These two textures (named “IndexTexture” and “NeigboursTexture”)

are the link between the particle and the information needed about its neighbourhood. In the first

“IndexTexture”, like in the “PositionTexture”, the information of the first particle is stored in the

first pixel and the second in the second, and so on.

We choose to put in the first field available (the red value) the number of neighbours

(remember that for each pixel there are four floatingpoint available, RGBA). In the second field,

we choose to put an index linking the “IndexTexture” to the “NeigboursTexture”.

The way to do this link is described in the Figure 2.10 below. Nci representing the number of

neighbour of the particle i and @NiPj representing the address of the neighbour i of the particle j.

32


33

Figure 2.10: Link between particle and their neighbours

Nc1 X Nc2 Nc3X+1 X+2 Nc4 X+4

Number of Neighbours of the particle 1

Address of the first neighbours in the neighbour's texture

P1 P2 P3

@N1P1

@N3P1

@N2P1

@N4P1

@N1P2 @N1P3@N2P2

@N3P2 @N3P3

@N2P3

@N4P3

@N5P3

X X+1 X+2 X+3 X+4 X+5

x

z

y x xy

z z

y x y

z

x

z

y

P1 P2 P3

Address of the third neighbours of the particle 1

Neighbours Texture

Index Texture

Position Texture


“IndexTexture” and “NeighboursTexture” are two constant textures built at the initialisation

of the simulation on the CPU.

2.3.5 The shadersIn the general flowchart of the model (see section 2.2.1), several operations has been

implemented on the GPU. Steps “Compute elastic force”, “Compute reaction forces”,

“Update positions” and “Compute attractor” has been implemented corresponding to four shaders.

• Elastic force shader :Elastic force consist in the subtraction of the attractor positions by the particle positions (see

figure 2.11).

34

Figure 2.11: Elastic force shader

Shader #1

Force Texture

Attractor Texture

Positions Texture


• Reaction forces shaderAt this stage, another shader is needed to compute the reaction force( see section 1.1.3). This

shader needs the “IndexTexture” and “NeighbourTexture” to reach each neighbour force. The

algorithm of this operation on the CPU (see section 2.2.5) needs a “scatter” operations. Thus, we

will adapt the algorithm to get a “gather” operation (see section 1.2.6). Instead of redistribute the

elastic force of the particle on its neighbours, all the neighbours forces will be redistribute (or

subtract) on the particle. The detailed flowchart is detailed in figure 2.12.

35


●

36

Figure 2.12: Flowchart of the reaction force shader

Index Texture

Neighbour Texture

Get coordinate of the first Neighbour in “NeighbourTexture”

Get coordinate of Neighbour i (@NiPj)

Force Texture

F=FFni

For (int i=0; i


● Positions update shader :To accomplish this task, two textures for old and new position are needed. This shader is

swapping between two output textures. This means that the role of the two textures are swapped

between old and new position (old position becoming new position and conversely). This technique

was used in (Göddeke, 2007) and is called the “PingPong technique” (see figure 2.13).

37

Figure 2.13: Positions update shader

Shader #3

Position Texture

Old Position Texture

Force Texture Old Position Texture


• Attractor shader :This shader calculates the evolution of the attractor position. The computation of the

attractors position is the most complicated task because it needs neighbours positions but also bases

positions to compute their normals. In 3D, bases are triangles formed by three neighbours of the

particles. Another texture called “BasesTexture” is used, the entire possible bases for each particle

is stored inside. One pixel in this texture stores the three neighbours and the corresponding to

the base (see section 1.1.3). We used the two fields reminded in the “IndexTexture” to store the

number of bases and the coordinate of the first base in “BasesTexture” called X' in the figure 2.14.

Finally a texture called “AlphaTexture” is used which stores all the parameters corresponding

for each neighbour.

Algorithm to compute attractors (see section 2.2.3) is composed of two parts : the

computation of parameters (with the neighbours position) and the computation of the

parameters (with the bases). The detailed flowchart is describe figure 2.15.

38

Figure 2.14: Example of the content of the first pixel in the "IndexTexture"

Nc1

Bc1

X

X'


39

Figure 2.15: Flowchart of the attractor shader

Compute alpha parameters

Compute beta parameters

Get coordinate of the first Neighbour in “NeighbourTexture”

For(int i=0; i


2.4 Display Particles When the computation is finished, it is needed to display the particle. For the CPU program,

the result is stored in an array and directly display thanks to an OpenGL function. Usually, each

vertex needs to call one OpenGL function. This means time, and so the simulation would be not

really efficient. In order to avoid this problem, vertex array is used. Vertex array needs only the

pointer to the vertex data and thus only one function is used and one transfer from the CPU memory

to the GPU memory.

In GPU program, the results are stored in a texture and displaying the texture would no be

very impressive (it will result in different colour equivalent to the position particles). If we use the

same method as for the CPU, we need to transfer results from the texture to the CPU memory and

use the vertex array to transfer another time between CPU and GPU. In order to avoid these

transfer, a method called “Rendertovertexarray” is used.

Thanks to the Pixel Buffer Object (PBO), results are transfer directly from the texture to this

buffer. With this technique, data stays resident on the GPU. Finally, the PBO is used as a vertex

buffer (see figure 2.16).

40


2.5 Measure of the processing time

The measure of time is achieved by the C++ library “Time.h”. This library allows to get the

time in millisecond when the function “clock()” is called. This function is called at the beginning of

the simulation and at the end of the simulation. The precision of the measure is 10 millisecond.

41

TextureFigure 2.16: Graphics pipeline describing the method "Renter to Vertex Array"

GPUFront End

PrimitiveAssembly

Rasterizationand

Interpolation

Raster Operations

ProgrammableVertex

Processor

ProgrammableFragmentProcessor

PBO


Chapter 3 : Results

This chapter shows the two examples used to compare performances. Result of time

processing and deformations of the two shapes are shown.

3.1 The tetrahedronIn order to validate the algorithm, only four particles were created. This particles describe a

tetrahedron.

In the figure 3.1, we can see the tetrahedron at the start of the simulation. The four dots in

green represent the position of the attractors which are at the same positions than the four particles.

Indeed when no forces or constraint are present, the system is at equilibrium. To see the deformable

and elastic reaction of the tetrahedron, we change the position of the top particle. This modification

generates a difference between the attractor and the particle position (see figure 3.2), which in turn

generates forces and displacements of the particles.

42


The system computes the forces and new positions until another position of stability is

reached. As this new equilibrium state is reached, the particles and their attractors are again at the

same position.(see figure 3.3).

43

Figure 3.1: Tetrahedron at the start of the simulation

Figure 3.2: Forced displacement of a particle


This demonstrates that the algorithm was working and the tetrahedron was evolving like a

real soft tissue. It was programmed on the CPU and the GPU, and we could compare performance.

The time calculated was the time of the calculation of the new positions and forces. The result of

the CPU program was under 10ms and for the GPU between 70 and 90ms.

3.2 The regular cylinder

In order to demonstrate the efficiency of GPU for a more complex model, the next step was

to program a particle system which could deal with more particles. In this case, 275 particles which

creates a cylinder (see figure 3.4). The position and connectivity of this object were locked by using

the PML library(Chabanas et al, 2004).This library is using an XML file containing the 275

44

Figure 3.3: Final position of the tetrahedron


positions particles and neighbours configuration for each particle. The attractor shader has not been

implemented in this program yet.

45

Figure 3.4: Initial position of the cylinder


Like in the previous program, a particle from the bottom of the cylinder is displaced. The

cylinder undergoes a deformation (see figure 3.5) and little by little returns to a normal shape

(figure 3.6).

46

Figure 3.5: Deformation applied on the cylinder


The table 3.1 presents the results of the time processing during the simulation.

47

Figure 3.6: Final positions


Table 3.1: Comparison of the processing time between the GPU and the CPU

Number of iteration GPU processing time

(ms)

CPU processing time

(ms)

CPU processing time

with attractor

calculation (ms)

1 10 0 0

10 10 10 10

100 30 10 40

200 80 10 50

300 120 10 90

650 220 10 180

1000 320 10 230

2000 650 40 580

3000 910 140 870

4000 1220 180 1110

5000 1560 210 1380

5851 230 1580

5890 1850

48


Chapter 4 Discussion

This chapter explains and interprets the different results. Difficulties and problems

encountered are discussed.

4.1 Discussion of resultsThese programs demonstrated that graphics card could be very efficient in some case but

inefficient in others. Indeed, with few particles, using the graphic card would not be really efficient.

We have to take in account that the CG code is compiled during the execution of the program and

for the first iteration, the time of execution is more important. With a small amount of particles, the

CPU is much faster.

When we take the second program, CPU program is processing each particle one by one

whereas the GPU program uses a texture of the size of the number of particles. It means that the

graphics processor does not need more time processing with four particles or two hundred.

GPU program is slower probably due to all the transfer of data between the graphic card

and the CPU. Indeed, in order to process attractor, the position particles are transfer from the GPU

49


to the CPU. If we compute attractors on the graphic card, time processing between the two

processors should decrease. Increase the number of particles should also be beneficial.

4.2 Difficulties and issues There were many difficulties. First of all, using graphics card for general computation

(GPGPU) is quite a new field and no ome in the laboratory was able to help me. Internet website or

forums are rarely technical and help hard to find that way.

The operating system was also issue. Indeed, most of the examples or tutorials are built for

windows. For instance, the code for the cloth simulation of Green (Green, 2007) is available but

this source was programmed for windows and did not work for Linux.

The two OpenGL extension, FBO and PBO, are recent extension and explanation are not

really frequent.

Another problem is the debugging. With usual programming language, we always have the

possibility to run the program step by step and see what the value of each variable is. Unfortunately,

for shaders, it is impossible to see what is happening in the graphics card. Input and output data are

the only thing that were possible to check. To check what I was doing in the different step of the

shader, I had to return the variables that I wanted to see. Obviously, it was not really quick nor very

easy to detect the different bugs.

50


The second program (with the cylinder) took a long time to develop, even if all the shaders

were already written, this program highlighted many bugs. With the first program and only four

particles, many problems were completely hidden. The second program with particles having more

than 4 neighbours shown special cases. Because each pixel stores four number in the four RGBA

fields, the information mapping was more difficult to do in the neighbours' texture.

This kind of programming is very specific and was completely new for me. It was really

difficult to understand and apply these entire new concepts, including the deformable model which

asks good skills in physics and geometry.

51


Chapter 5 Project conclusions

5.1 Further workThe next stages of this project would be to compute attractors calculation on the GPU to

improve the processing time. After this step other methods can be tried. Using a base texture needs

too much place in the texture. Another way to work around this issue would be to process all the

bases in the shader. Indeed, we can access each neighbour thanks to the index and neighbours

texture and thus using a loop in the shader to recreate all the possible bases at each iteration.

The program can be simplified using another “target” texture (see section 2.3.1). Using

GL_TEXTURE_RECTANGLE, it should be easier to deal with coordinate texture. Obviously,

using the entire capacity of the texture will be the next challenge. Only a few capacity is used at the

moment because only the first row is filled of data. Indeed, maximum size in our graphic card is up

to 4096*4096 pixels. Thus we can handle a deformable model of 1.677.216 particles with one

texture.

52


5.2 ConclusionsThe aim of this project was to understand GPU programming and to produce a feasibility

study. It was the first work in this field in the laboratory. One objective was to initiate and inform

people in the laboratory of the feasibility of using GPU programming in research project. Indeed,

this work can offer other possibilities for the other projects in the laboratory when processing time

is important.

Deformable models need to be fast enough to get a real time deformation so that they can be

used in the operating room. In a operating room, using clusters with many processors would not be

secure enouth, and the network connection would have to be very fast (3D interactive display) and

reliable. It would also unacceptably increase cost of an operation. Graphics cards are cheap (less

than 1000£) compare to clusters (starting at 50.000£) and would not need any dedicated space.

GPUs are efficient specialized processors, but they need a long time to understanding and

using their possibilities. I think that computation on GPU is faster for a particle system even if we

did not manage to finish all the computation of our model. Attractor shader computes normal and

the Cg language has a function to process it which is optimised. Finally, testing the model with

more particles would show advantages of this processors.

53


Bibliography• MONTGOMERY, BRUYNS, BROWN, SORKIN, MAZELLA, THONIER, TELLIER,

LERMAN, MENON. (2002). Spring: A General Framework for Collaborative, Realtime

Surgical Simulation. Medicine Meets virtual reality 11. pp. 2326.

• MARCHAL. (2006). Modélisation des tissus mous dans leur environnement pour l'aide aux

gestes médicochirurgicaux (in french).

• MARCHAL, PROMAYON, TROCCAZ. (2006). Simulating Prostate Surgical Procedures with a

Discrete Soft Tissue Model.In Eurographics Workshop in Virtual Reality Interactions and

Physical Simulations, VriPhys06, I. Navazo C. Mendoza (ed.)

• LATTA. (2004). Building a Million Particle System. Game Developers Conference.

http://www.2ld.de/gdc2004.

• SORENSEN, MOSEGAARD. (2006). An Introduction to GPU Accelerated Surgical Simulation.

Biomedical Simulation. Third International Symposium. Zurich,Switzerland. pp. 93104.

• GeneralPurpose Computation Using Graphics Hardware. http://gpgpu.org. (May 2006).

• DUMMER. A Simple TimeCorrected Verlet Integration Method.

http://www.gamedev.net/reference/programming/features/verlet/. (May 2007) .

• RANDIMA, KILGARD. (2006). The Cg Tutorial. .

• HARRIS. (2005). GPGPU: GeneralPurpose Computation on GPUs. I3D conference (the ACM

Symposium on Interactive 3D Graphics and Games).

54


• PHARR. (2005). GPU Gems 2. GPU Gems 2.AddisonWesleyPart IVVI.

• GeneralPurpose Computation Using Graphics Hardware. http://gpgpu.org. (May 2007).

• FERNANDO. (2004). Beyond Triangles.. GPU Gems.AddisonWesley.Part VI

• GREEN. Cloth Simulation on the GPU.

http://developer.nvidia.com/object/demo_cloth_simulation.html. (May 2007) .

• WANG. (2005). GPU Based Cloth Simulation on Moving Avatars.

• GREEN. (2005). The OpenGL Framebuffer Object Extension.Game Developers Conference

2005 (GDC 2005).

• GODDEKE. GPGPU : Basic Math Tutorial. http://www.mathematik.uni

dortmund.de/~goeddeke/gpgpu/tutorial.html. (August 2007) .

• CHABANAS, PROMAYON. (2004). Physical model language : Towards a unified

representation for continuous and discrete models;.In proceedings of International Symposium on

Medical Simulation

55

ABSTRACTACKNOWLEDGEMENTS

LIST OF FIGURES Chapter 1 Introduction and Literature Review 1.1 Modelling 1.1.1 Deformation modelling 1.1.2 Spring-mass model 1.1.3 Shape memory model (phymul)

1.2 Graphic programming and GPU 1.2.1 Graphics pipeline 1.2.2 Graphics API (Application Programming Interfaces) 1.2.3 Shader and Shading Language 1.2.4 Cg 1.2.5 Why GPU? 1.2.6 Vertex Vs Fragment Processors

1.3 Related work in GPU based deformable model 1.4 Project aim

Chapter 2 Materials and Methods 2.1 Hardware and Programming Languages used 2.1.1 Hardware 2.1.2 Programming language

2.2 Implementation of a particle system on the CPU 2.2.1 General evolution of the model 2.2.2 Initialisation of particles 2.2.3 Attractor position 2.2.4 Compute elastic forces 2.2.5 Compute reaction forces 2.2.6 Update Position 2.2.7 Display particles

2.3 Implementation of a particles system on the GPU 2.3.1 Texture as array 2.3.2 Data transfer 2.3.3 Algorithm to process a fragment shader 2.3.4 Particle neighbourhood access 2.3.5 The shaders

2.4 Display Particles 2.5 Measure of the processing time

Chapter 3 : Results 3.1 The tetrahedron 3.2 The regular cylinder

Chapter 4 Discussion 4.1 Discussion of results 4.2 Difficulties and issues

Chapter 5 Project conclusions 5.1 Further work 5.2 ConclusionsBibliography

Documents

CRANFIELD UNIVERSITY CRANFIELD HEALTH MSC BIOINFORMATICSmembres-timc.imag.fr/Emmanuel.Promayon/pdf/Reynier07.pdf · 2007. 8. 31. · Reynier Michael. MSc Bioinformatics GPU accelerated