cemracs_vivabrain_slides

../../../figures/logos/logoUDS../../../figures/logos/logoUJF../../../figures/logos/logoLNCMI../../../figures/logos/logoCMUC../../../figures/logos/logoUPMC

Feel++ Projects : VivaBrain

Céline Caldini-Queiros, Vincent Chabannes,Tarik Madini, Jussara Marandola, Christophe

Prud’homme, Marcela Szopos, RanineTarabay

CEMRACS’12

Marseille, August 23, 2012

C. Caldini-Queiros, V. Chabannes (CEMRACS’12) VIVABRAIN August 28 2012 1 / 23


Introduction

Objectives

Describe the problem :complex reality(multi-physics,multi-scale).Simplify.Develop a mathematicalmodel.Solve the mathematicalequations.Display results.



Introduction

Physical models and boundary conditions

Which forces induces blood flow in the cerebral arteries ?pressure dropgravityothers?

Related questionsFluid dynamic model : Newtonian vs Non-Newtonian?Fluid or Fluid-Structure model?Inflow : pressure, velocity profile ?Outflow : do nothing strategy ?How does surrounding tissues affect the blood flow ?



Introduction

Table Of Contents

1 Mathematical models and methods

2 HPC with feel++

3 Numerical results

4 Conclusion



Mathematical models and methods


2 HPC with feel++

3 Numerical results

4 Conclusion




Navier-Stokes

Modelling blood : Newtonian fluid with incompressible behaviour.

We use Stokes or Navier-Stokes equations ρ∂u∂t

+ ρ (u · ∇)u−∇ · σ = f∇ · u = 0+ Boundary conditions

with :stress tensor : σ = −pI+ τdeviatoric stress tensor : τ = 2µD

strain rate : D = 12

(∇u+ (∇u)T

)




Pressure drop

Impose the inlet pressureVariational formulation with weak Dirichlet condition :Find (u, p) ∈ H1(Ω)× L2(Ω) such as ∀(v, q) ∈ H1(Ω)× L2(Ω) :∫

Ω

(∇u : ∇v − p∇ · v +∇ · uq) +∫Γwall

(−∇u + pI)n · v

+

∫Γwall

(−∇v + qI)n · u +

∫Γwall

γ

hu · v =

∫Ω

f · v −

∫Γinlet

(pinletn) · v,

Variational formulation with strong Dirichlet condition :Find (u, p) ∈ H1Γwall

(Ω)× L2(Ω) such as ∀(v, q) ∈ H1Γwall(Ω)× L2(Ω) :∫

Ω

(∇u : ∇v − p∇ · v +∇ · uq) =∫Ω

f · v −

∫Γinlet

(pinletn) · v,




Inflow Dirichlet conditions

Create a 2D submesh with theinlet facesSolve a Laplacian on thesubmeshInterpolate the Laplaciansolution on the 3D mesh andfinally the inlet velocity is :u = un on Γinlet





∆u = f in Γinletu = O on ∂Γinlet









HPC with feel++


2 HPC with feel++

3 Numerical results

4 Conclusion



HPC with feel++

Feel++ [et al., ]Finite Element Embedded Library in C++ : a C++ library

FeaturesGeneralized Galerkin (cG, dG) methods in 1D, 2D and 3D with meshesfrom simplices and hypercubesDomain specific language embedded in C++ for variational formulationSeamless interpolation tool (space, toppological dimension and mesh)integrated in the variational languageConcepts/Classes mimic closely the mathematics which allows forgenericity and to handle complexity (Operators, function spaces,elements of function spaces ...)Seamless transition from 1D to 2D to 3D thanks to the variationallanguage (e.g. same code used either for 2D or 3D computations)

http://www.feelpp.org


http://www.feelpp.org


HPC with feel++

Example : Stokesfind (u, p) ∈ Xh = Xuh × X

ph such that :∫

Ω

∇u : ∇v−∫Ω

p∇ · v =

∫Ω

fv ∀v ∈ Xuh∫Ω

q∇ · u = 0 ∀q ∈ Xph

form2( _test=Xh, _trial=Xh, _matrix=D ) =integrate( _range=elements(mesh),

_expr=trace(trans(deft)*grad(v)) ) //∫Ω∇u : ∇v

-integrate( _range=elements(mesh),_expr=idt(p)*div(v) ) //−

∫Ωp∇ · v

+integrate( _range=elements(mesh),_expr=id(q)*divt(u) ); //

∫Ωq∇ · u

form1( _test=Xh, _vector=F ) =integrate( _range=elements(mesh),

_expr=f*id(v) ); //∫Ωfv



HPC with feel++

Feel++ //

Contruction of partitioned mesh : use Gmsh[Geuzaine and Remacle, 2009] with chaco or metis partitioners.

Degrees of freedom distribution :I Step 1 : build the local dof table with

ghost dofsI Step 2 : build the global dof table

without ghost dofsF the single global dof belongs to the process

of smallest rankF communication : update the id of

interprocess dofs on the global table

Algebraic parallel solver : Petsc libraries[Balay et al., 2011, Balay et al., 2010, Balay et al., 1997, Smith, 2006]



Numerical results


2 HPC with feel++

3 Numerical results

4 Conclusion



Numerical results

AortaPhysical model : StokesBoundary conditions :

I inlets : Dirichlet condition (u = g) computed with Laplacian subproblemI outlets : do nothing ( σn = 0 )I wall : no slip (u = 0)

Solutions strategies : Additive Schwarz Method with LU Preconditioner ineach block

(a) pressure (b) streamlines colored with velocity magni-tude



Numerical results

AortaP2P1

nDof : 322966nProc : 48Linear solve converged in 474 iterations (tolerance 1e−14)inflow : 0.0813213outflow : 0.0813301flow difference : 8.81439e− 06

P3P2nDof : 1101523nProc : 48Linear solve converged in 477 iterations (tolerance 1e−14)inflow : 0.0813643outflow : 0.0813649flow difference : 5.79816e− 07



Numerical results

Aneurysm on arteryPhysical model : Navier-StokesBoundary conditions :

I inlets : Pressure during 3msI outlets : do nothing ( σn = 0 )I wall : no slip (u = 0)

Solutions strategies : Additive Schwarz Method with LU Preconditioner ineach block and Newton method for nonlinear term.Statistic : nProc = 48, nDof : 1171496, number of iteration with linearsolver ≈ 50, time elapsed for each time step : 450 s

(c) pressure at t=0.0008 (d) pressure at t=0.0015



Numerical results

Cerebrovenous system

Figure : 1. Superior sagittal sinus ; 2.Right sinus ; 3. Confluence of sinuses ;4. Transverse sinus ; 5. Sigmoid sinus.

Cerebrovenous system : not wellexamined, only partiallyunderstood.

Anatomy : complexthree-dimensional structure, oftenasymetric, presenting considerablymore variable pattern than thearterial system.



Numerical results

Cerebrovenous system : Mesh generation

MRA image segmentation (N.Passat[Passat, 2005], LSIIT)Final mesh (O. Génevaux,LSIIT, S. Salmon, EA4535)



Numerical results

Cerebrovenous system : Mesh generation

MRA image segmentation (N.Passat[Passat, 2005], LSIIT)Final mesh (O. Génevaux,LSIIT, S. Salmon, EA4535)



Numerical results

Cerebrovenous systemPhysical model : StokesBoundary conditions :

I inlets : pressure ( σn = gn )I outlets : do nothing ( σn = 0 )

Solutions strategies : Additive Schwarz Method with ILU Preconditionerin each blocknDof : 1173594, number of iterations : 1184 (tolerance 1e−14)

(a) mesh partitioning (b) pressure

Figure : Partionining and pressure solution with 48 proc



Numerical results

Cerebrovenous system

Figure : Streamlines with 48 proc



Conclusion


2 HPC with feel++

3 Numerical results

4 Conclusion



Conclusion

ConclusionAchievement

Simple simulations in complex geometriesParallel computing experimentation with Feel++ on large cluster (thanksto MesoCentre Marseille)Inlet Dirichlet velocity (Laplacian subproblem)

PerspectiveVessels are endowed with valves (e.g. leg veins). What about cerebralveins ?One step further : modeling the whole circuit (coupling arterial andvenous system)What are the main differences between arteries and veins from amodeling standpoint ?

Boundary conditions : essential step to get realistic simulations



Conclusion

References IBalay, S., Brown, J., , Buschelman, K., Eijkhout, V., Gropp, W. D., Kaushik, D., Knepley, M. G., McInnes, L. C., Smith, B. F., and Zhang, H. (2010).

PETSc users manual.Technical Report ANL-95/11 - Revision 3.1, Argonne National Laboratory.

Balay, S., Brown, J., Buschelman, K., Gropp, W. D., Kaushik, D., Knepley, M. G., McInnes, L. C., Smith, B. F., and Zhang, H. (2011).

PETSc Web page.http://www.mcs.anl.gov/petsc.

Balay, S., Gropp, W. D., McInnes, L. C., and Smith, B. F. (1997).

Efficient management of parallelism in object oriented numerical software libraries.In Arge, E., Bruaset, A. M., and Langtangen, H. P., editors, Modern Software Tools in Scientific Computing, pages 163–202. Birkhäuser Press.

et al., C. P.

Feel++: The finite element embedded language (and library) in c++.http://www.feelpp.org.

Geuzaine, C. and Remacle, J.-F. (2009).

Gmsh: a three-dimensional finite element mesh generator with built-in pre-and post-processing facilities.International Journal for Numerical Methods in Engineering, 79(11):1309–1331.

Passat, N. (2005).

Contributiona la segmentation des réseaux vasculaires cérébraux obtenus en IRM. Intégration de connaissance anatomique pour le guidaged’outils de morphologie mathématique.PhD thesis, PhD thesis, Université Louis Pasteur de Strasbourg.

Smith, B. (2006).

Petsc introductory tutorial.


Documents

cemracs_vivabrain_slides