1

A Domain Decomposition Analysis of a Nonlinear Magnetostatic Problem with　 100 Million Degrees of Freedom

H.KANAYAMA *, M.Ogino*, S.Sugimoto** and J.Zhao*

* Kyushu University* * The University of Tokyo

2

Contents Introduction

Backgrounds Objectives

DDM Applications to Magnetic Field Problems

Numerical Examples Conclusions

3

Backgrounds A large-scale complicated model

A　transformer　(Moriyasu,　S.,　2000)A　transformer　(Moriyasu,　S.,　2000)

Model offer by Japan AE Power Systems Corporation

and Fuji Electric Advanced Technology Co., Ltd.

4

Objectives Development of ADVENTURE_Magnetic

for analysis of large-scale magnetic field problems

Parallel computing Analysis of models with about 100 million

degrees of freedom (DOF)

5

Contents Introduction DDM Application to Magnetic Field

Problems Magnetostaic Problems DDM HDDM

Numerical Examples Conclusions

6

Non-Linear Magnetostatic Analysis

Formulation A method

Solution for non-linear equations Newton method

On the interface Conjugate Gradient (CG) method A simplified block diagonal scaling

In each subdomain The mixed formulation with the Lagrange multiplier p Skyline method with partial pivoting

7

Formulation

1 ： air or vacuum

2 ：magnetic material

E

N

2

1

2

[m/H]y reluctivit Magnetic:

vectornormalUnit :

0, 0

],[A/mdensity current Electric:

[Wb/m] potential vector Magnetic:

,0

,0

,0

,0

,

inA||rotν

inconstant

n

onnJinJdiv

J

A

onnA

onnArot

onnA

inAdiv

inJArotrot

N

N

N

E

8

Weak formulation A weak formulation is constructed by the

introduction of the Lagrange multiplier p:

,0,

,,,,

,anyforthat,such,Find

*

***

**

pgradA

AJApgradArotArot

QVpAQVpA

( . , . ) : the real valued L2-inner product.

EonnvLvrotLvV 0,;3232

EonqHqQ 0;1

9

Finite element approximationAh:Nedelec elements of simplex typeph ： Conventional piecewise linear tetrahedral

elements

D.O.F.D.O.F.

10

Finite element approximation

* *

* * *

*

Find , such that, for any ,

, , , ,

, 0,

h h h h h h h h

h h h h h

h h

A p V Q A p V Q

rot A rot A grad p A J A

A grad p

Vh , Qh: Finite element spaces corresponding to V and Q,

Ah : Finite element approximation of A by the

Nedelec elements of simplex type,

ph : Finite element approximation of p by the

conventional piecewise linear tetrahedral elements.

11

.A,J~

Arot,Arot *hh

*hhh

Finite element approximation

Elimination of the Lagrange multiplier ph

Correction of electric current density,IgradJJ

~hhh

12

Newton iteration Adoption of the Newton iteration to solve

the nonlinear equation

Solver for linear simultaneous equations

.,,~

,,

**

*1*1

hnh

nh

n

hh

hnh

nh

n

hnh

nh

ArotArotAA

AJ

ArotArotAA

ArotArot

13

DDM (Domain Decomposition Method)

I: corresponding to inner DOFB: corresponding to interface DOF

Domain decomposition

fuK

N

i

iB

iB

NI

I

B

NI

I

N

i

TiB

iBB

iB

TNIB

NB

TIBB

TNB

NIB

NII

TBIBII

fR

f

f

u

u

u

RKRKRKR

RKK

RKK

1

11

1

11

111

0

0

00

14

DDM (Domain Decomposition Method)

On the interface

In each subdomain B

TiB

iIB

iI

iI

iII uRKfuK )()()()(

N

i

iI

iII

TiIB

iB

iB

B

N

i

TiB

iIB

iII

TiIB

iBB

iB

fKKfR

uRKKKKR

1

1

†

†

gSuB

The interface problem

The subdomain problem

15

IDDM(Iterative Domain Decomposition Method)

;

;

;

byCompute

subdomaineach In

;Choose

00

1

00

000

00

0

0

rw

rRr

fuRKuKr

uRKfuK

u

u

N

i

iiB

iBB

TiB

iBB

iI

TiIB

i

BTi

Bi

IBi

Ii

Ii

II

iI

B

;end

;

;

;break,If

;

;

;

;

;

;

byCompute

subdomaineach In

,.....;1,0for

11

11

01

1

1

1

nnnn

nTnnTnn

n

nnnn

nnnB

nB

nTnnTnn

N

i

niiB

n

nTiB

iBB

niI

TiIB

ni

nTiB

iIB

niI

iII

niI

wrw

rrrr

rr

qrr

wuu

qwrr

qRq

wRKwKq

wRKwK

w

n

(a) (b)

16

The modification for the subdomain problem In step 0

In step n

byuCompute iI

0

00

0)()()(

0)(

0)(

)()(

)()(

BTi

Bi

IBi

Ii

I

iI

iII

TiII

iII

iII uRKf

u

u

KK

KK

pppp

p

iBB

TiB

iBB

iI

TiIB

i fuRKuKr 000

0

)()(

)(

)(

)()(

)()( nTiB

iIB

niI

niI

iII

TiII

iII

iII wRK

w

w

KK

KK

pppp

p

nTiB

iBB

niI

TiIB

ni wRKwKq

bywCompute niI

(a`)

(b`)

17

HDDM(Hierarchical Domain Decomposition Method)

Introduction of HDDM (Hierarchical Domain Decomposition Method) for computing in parallel environments

Single processor mode (S-mode) Parallel processor mode (P-mode) Hierarchical processor mode (H-mode)

18

Contents Introduction DDM Applications to Magnetic Field Problems Numerical Examples

TEAM Workshop Problem 20 Linear Magnetostatic Analysis Nonlinear Analysis of the model with 100 million DOF Checking for the accuracy

Conclusions

19

TEAM Workshop Problem 20

YokeSS400

Center pole

Coilpolyimide electric wire

|J| = 1,000 [A]

20

TEAM Workshop Problem 20

Wb/m0: iteration linear first for the valuesInitial

m/H100:polecenter yoke,

m/H1041:coil air,:yreluctivit Magnetic

A000,1:densitycurrent Electric

0

0

7

hA

J

21

TEAM Workshop Problem 20

Elements DOF Subdomains

Model 1 471,541 559,848 8×300

Model 2 952,845 1,125,501 8×600

Model 3 1,769,871 2,083,209 8×1,100

Model 4 9,326,492 10,945,318 56×830

Model 5 38,232,019 44,676,346 56×3,400

Model 6 86,570,893 100,818,053 56×7,730

22

Linear Magnetostatic Analysis(Computational conditions)

5

Solver: The Interface problem: CG method

Judge of convergence 1.0 10

A simplified block diagonal

scaling preconditioner

The subdomain problem:

The proposed method: Skyline method with partial pivoti

ng

The previous method: ICCG method

HDDM: P-mode

PC cluster: Intel Core2Duo E6600 Memory 8GB

The number of PCs: 4

23

Linear Magnetostatic Analysis(CPU time)

CPU time by the previous method

[s]

CPU time by the proposed method

[s]Ratio

(%)

Model 1 81.6 53.6 33.9

Model 2 230.7 153.3 33.6

Model 3 497.3 332.1 33.2

Ratio = ( | The previous method – The proposed method |/| The previous method | ) ×100

24

Linear Magnetostatic Analysis(Averaged memory)

Ave. memory

by the previous method [MB]

Ave. memory

by the proposed method [MB]

Times

Model 1 25.3 53.2 2.10

Model 2 56.3 107.4 1.91

Model 3 104.5 199.3 1.91

Times = ( |The proposed method |/| The previous method | )

25

Linear Magnetostatic Analysis(Iteration figure of the interface problem)

26

Nonlinear analysis of the model with 100 million DOF

-5

4

Solver: Non-linear :Newton method

Judge of convergence 1.0 10

The interface problem: CG method

Judge of convergence 1.0 10

A simplified block diagonal

scaling preconditioner

The subdom

ain problem: Skyline method with partial pivoting

HDDM: P-mode

PC cluster: Intel Core2Duo E6600 Memory 8GB

The number of PCs: 28

27

Nonlinear analysis of the model with 100 million DOF

Res

idua

l nor

ms

Iteration counts on the interface

Step 0Step 1Step 2

Step 1 Step 1Step 2 Step 2

Step 0 Step 0

Model 4 Model 5 Model 6

28

Nonlinear analysis of the model with 100 million DOF

Iteration counts

(Newton method)

CPU time

[s]

Memory per CPU

[MB]

Model 4 2 4,495 197

Model 5 2 15,777 812

Model 6 2 46,361 1,840

29

Checking for the accuracy

Measured Bz [T]Computed

Bz [T]

Relative error[%]

I II I II

Model 1

0.72 0.71

0.571 20.6 19.5

Model 2 0.609 15.4 14.2

Model 3 0.628 12.8 11.6

Model 4 0.669 7.1 8.5

Model 5 0.679 5.7 4.4

Model 6 0.681 5.4 4.1

30

Checking for the accuracy(VS. )Ⅰ

The average length of edge [m]

The

rel

ativ

e er

ror

9.8×1.0-4 1.9×1.0-3 3.9×1.0-3

Model 1

Model 2

Model 3

Model 4

Model 5Model 6

31

Checking for the accuracy (VS. )Ⅱ

The average length of edge [m]

The

rel

ativ

e er

ror

9.8×1.0-4 1.9×1.0-3 3.9×1.0-3

Model 1

Model 2

Model 3

Model 4

Model 5Model 6

32

Conclusions Improvement of ADVENTURE_Magnetic Demonstration of the possibility of large-

scale analysis in magnetic field problems

with over 100 million DOF Future work

Application of strong preconditioners Coupled analysis of magnetic field and other

phenomena (ex. solid, fluid …etc.)