Result presentation for honours project

Lee-Ying Wu BEng (Software)

Supervisor: Dr Greg Ruthenbeck

*Bleeding effects for Flinders Sinus Surgery Simulator

*Outline:

• Aim of the project

• Why build an endoscopic sinus surgery simulator?

• Why do we need a new bleeding effect?

• Tools

• Approaches and results

• Future work

*Aim of the project

The project aims to develop bleeding effects for the Flinders endoscopic sinus surgery simulator that effectively reproduce the behavior of blood in sinus surgery.

*Why build an endoscopic sinus surgery simulator?

Endoscopic sinus surgery (ESS)

*Why build an ESS simulator?

• Lack of opportunities to learn by operating on cadaver or real patients (Ooi and Witterick, 2010).

• Insufficient exposure to the number of cases to feel competent to perform the surgery.

• Simulator can provide realistic interactions with the simulated patients.

• Simulator allows learners to make mistakes.

• Residents specialised in the ear, nose and throat will be the primary users of the simulator.

Setup of the

Flinders ESS Simulator

Real ESS scene Simulated ESS scene

*Why build an ESS simulator?

*Why do we need bleeding effects?

• Make the simulator closer to the real surgery.

• Encourage the engagement between the user and simulator.

• Provide a mechanism to allow users to make mistakes which could result in immediate consequences.

• Require users to manage the surgical environment (rinsing, patties).

*Tools and Approaches

Tools

• DirectX 11 (Direct3D) oC++, Rendering/Graphics pipeline and

High Level Shader Language (HLSL)

• Parallel computation oGPU and CUDA (C-like language)

• IDE: Microsoft Visual Studio 2010

• C# and Java

*Tool

Rendering/Graphics pipeline

(http://rnd.azoft.com/fluid-dynamics-simulation-on-ios/)

*Tool

GPU and CUDA

(http://www.slashgear.com/moores-law-cpu-scaling-is-now-dead-claims-nvidia-vp-gpu-parallel-computing-is-the-future-3083858/)

(http://www.dmi.unict.it/~bilotta/gpgpu/ingv/)

GeForce Titan has 2688 cores!GPUs are great at parallel operations and vector arithmetic

*Approaches

• Texture-based blood simulation

• Bleeding effect simulation - particle system

*Scenes of endoscopic sinus surgery

*Texture-based blood simulation

Texture-based blood simulationDirect3D Initialisation

Create CUDA 2D texture structure

CUDA kernel computers pixel values of CUDA 2D texture

Blend the tissue texture with the CUDA 2D texture

Render the blended texture

Interoperate Direct3D with CUDA

*Result of the texture-based blood simulation

CUDA 2D texture Tissue texture minus CUDA 2D texture

*Scenes of endoscopic sinus surgery

*Bleeding effect simulation

Ble

edin

g e

ffec

t si

mu

lati

on

Direct3D Initialisation

Interoperate Direct3D with CUDA

[Yes?]

[No?]Render the same scene

Add the new particle to the vertex buffer

CUDA kernel updates the position and velocity of particles

Render to texture

CUDA kernel generates the trail and blur effects for the texture

Perform blending of the CUDA-processed texture with the tissue texture

Render the 2D model with the blended texture

Check if mouse is clicked

*Results of the bleeding effect simulation

With trail and blur effectsWithout trail and blur effects

*Future work

• Improve the blur effect for the simulation.

• Implement the technique of bump mapping (to enable blood to collect in crevices)

• Implement a 3D tissue model into the bleeding effect simulation.

*Conclusions

• The texture-based blood simulation is able to model the growing and radiant spreading of the blood pattern from the central cut point.

• The bleeding effect simulation shows the blood flow represented by the particle system when the user clicked the left-mouse button.

• This study will serve as a prototype to develop various techniques that best meet the bleeding effect requirement of the ESS simulator.

• Beyond the project, the most important future work is to integrate the bleeding effect simulation with the ESS simulator. After the integration, the optimisation needs to be carried out to meet the requirement of the refresh rate of the ESS simulator.

*Thank you!

Questions?

Ooi, E. H. & Witterick, I. J. 2010. Rhinologic Surgical Training. Otolaryngologic Clinics of North America, 43, 673-689.

*Reference

*Render to texture technique

Set a texture as the render target view

Bind the render target view to the output render pipeline

Copy the render-to-texture texture to the shader-resource texture

Reset the render target back to the back buffer

Render scene