The GATE-LAB system

Sorina Camarasu-Pop, Pierre Gueth, Tristan Glatard, Rafael Silva, David Sarrut

VIP Workshop

December 2012

2

GATE-LAB ?

What is GATE-LAB ? It is one module in VIP, dedicated to GATE

What is GATE ? A software for Monte-Carlo simulations in medical physics Particles tracking Both for imaging (PET, SPECT, X/proton-radiography, …) and

radiation therapy (dose computation) Based on the Geant4 library fro particles physic (CERN) Open source Developed by the OpenGate collaboration

http://www.opengatecollaboration.org Community of users (estimated to 1000 worldwide) First release of GATE in May 2004 ; 18 releases since 2 reference articles : [Phys Med Biol 2004 and 2011] (highly cited)

3

One example

GATE simulation of proton cancer treatment Goal: study the dose distribution inside patient data

Protontherapy treatment room CT patient data

4

One example

Proton beam (complex) Track every particles inside biological material Store deposited energy Around 108 particles to track + secondaries About 7 days computing time “brute-force”, reference

Proton beam• intensity, energy• position orientation

5

One example : results

Example of dose distribution• Overlaid on the CT data• Computed with GATE

• More than 1 month comp time• Done in few hours VIP/GateLab• Speed up = 290

6

GATE-LAB specificities

Why ? In general MC simulations can be very long (up to days) Access to grid or cluster need skills, not straightforward To provide easy and efficient access to Gate

GATE-LAB started before VIP Started since 2008 Test version 2011-2012 Open to public since Feb 2012 (beta)

Tight integration between GATE and GATE-LAB GATE code has been modified

to monitor the progress of the simulations to stop simulation on demand

VIP-GATE-LAB has been adapted To manage all GATE output types Very high performance computer

7

What is the GateLab ?

Design your simulations

Click to load macro file in web

browser

Monitor simulation

Click to download (merged) results

Computing (x 103 jobs)

GATE-LAB• Upload all files• Split simulation• Handle errors• Load balancing• Merge results

8

One example

From the GATE-LAB (VIP): Specific input parser Submission options: mode + GATE release Specific job monitoring Specific output merger

9

One example

Specific input parser Load the main macro file Parse the macro file to find all needed files Archive and zip together all files Upload files on the GATE-LAB server

10

Launch simulation screen

Simulation name

Estimated time

Gate release

Type of submission

Go !

11

Monitor simulation screen

Total # of particles

Current # of particles

12

One example

Specific output merger GATE simulations can have multiple results Each result is split into several files (nb of jobs) The “merger” gather all files and merge them according to the file

types

One of the most critical point Could be long Incremental merger in development

13

N jobs, p particles. Two modes

Static mode : Each job simulates p/N particles Dynamic mode : Jobs simulate particles until the system stops them

Static vs Dynamic mode

Static Dynamic

Two times faster

14

Submission modes

Which mode to use ? Dynamic : each job & events must be strictly equivalent If time is involved : use static mode

For radiation therapy applications : dynamic For PET/SPECT, one run : static For PET/SPECT, multiple run : not (yet) possible

Need to split according to time rather than Events, need refactoring

15

Limits of the GateLab

No scripting possible (launch one simulation at a time)

Difficult to manage large files Typically needed for “phase-space” Either as input or output

Difficulty for simulation involving time(with jobs not equivalent, not interruptible)

16

Results - GateLab

Currently, about 228 registered users

About 25 active user/months Active user = launch at least 3 simulations About 200 simulations per months (except in summer ...)

Global average speedup around 50 (increasing !) Max speedup of 350

(taking queuing & merging time into account)

17

Conclusion

GateLab allows easy and free high performance computation for Gate simulations

Work in progress New splitting procedure for PET simulations New more robust merger Improved interface

Feedback welcome !

http://gatelab.creatis.insa-lyon.fr

Acknowledgments : Sorina Camarasu, Tristan Glatard, Rafael Ferreira da Silva, Pierre Gueth, David Sarrut