1

High performance Computing Applied to a Saltwater Intrusion Numerical Model

E. Canot IRISA/CNRS

J. Erhel IRISA/INRIA Rennes

C. de Dieuleveult IRISA/INRIA Rennes

2

Outline

Context Model and Test Cases

Coupled Model Test Cases : Henry and Elder

Parallel performances Presentation Results

Conclusion

3

Context

Effect of pumping in coastal aquifers

Necessity to predict the evolution of the water supply.

Context

4

Presentation of the software

Based on TVDV-2D software developped at IMFS in Strasbourg

Simulation of density driven coupled flow and transport in a porous media

Originally sequential

Coupled Model

5

Model

FLOW

TRANSPORT convection

dispersion

Coupled Model

),(*

)(_

),(

)(

),,,(*

)(

1*11

11

1*

11

1111

11

ndis

nn

nn

nnconv

nn

nnnnflow

nn

nn

DCbCD

vtensorDispersionD

CvbC

hVelocityv

hCCbh

CDensity

)(A

)(Α

dis

flow

6

Strong Coupling

Concentration

Timen

FLOW TRANSPORT

Velocity

Density

Time n+1

Coupled Model

7

Henry

Stable test case

Test Cases

8

Elder

Unstable test case

Test Cases

9

),D(Cb)*C(DA

)_tensor(vDispersionD

),C(vbC

)Velocity(hv

),h,C,C(ρb)*h(ρΑ

)Density(Cρ

n*dis

nndis

nn

nnconv

*

nn

nnnnflow

nnflow

nn

111

11

1

11

1111

11

First parallel version (v1)

Parallel performanc

es

Parallel linear

solvers

10

Parallel sparse solver

Use of MUMPS (« MUltifrontal Massively Parallel Solver »), a free package for solving linear systems of equations Ax=b

Adapted for sparse unsymmetric, and symmetric definite positive matrices.

Parallel performanc

es

11

Global parallelisation (v2) Partitioning mesh thanks to METIS, a free

package for partitioning graphs, meshes and for producing fill reducing orderings for sparse matrices.

Partitioning example with 5 parts Better data distribution.

Parallel performanc

es

12

Time step

number

1 processor 2 processorsIteration number

Time step

(in day)

Iteration number

Time step

(in day)1 10 1/4 10 1/4

2 to 7 12 1/4 12 1/48 26

(no convergence)

1/4 11 1/4

1/8 1/4

Results

Due to the convergence criterion

Elder (mesh 256x160)

Sensitivity to convergence criterion

13

Sensitivity to convergence criterion

with 1 processor

1,00E-14

1,00E-12

1,00E-10

1,00E-08

1,00E-06

1,00E-04

1,00E-02

1,00E+00

1,00E+02

1 4 7 10 13 16 19 22 25

error in flow

error in transport

convergence criterion

Results

with 2 processors

1,00E-14

1,00E-12

1,00E-10

1,00E-08

1,00E-06

1,00E-04

1,00E-02

1,00E+00

1,00E+02

1 4 7 10 13 16 19 22 25

Elder (mesh 256x160), time step number 8

14

0

100

200

300

400

500

600

700

1 2 4 8 16

Total wallclockMUMPS wallclock

Wall clock

(in seconds)

Number of processors

Results

Results for the first version (v1)

Henry (mesh 510x254)

15

0

100

200

300

400

500

600

700

1 2 4 8 16

v1 Total wallclockv1 MUMPSwall clockTotal wallclockMUMPS wallclock

Number of processors

Results

Results for the second version (v2)

Wall clock

(in seconds)

Henry (mesh 510x254)

16

Conclusion - Perspectives

3D geometry Adaptative mesh Improved coupling

Conclusion