9. Hardware-Aware Numerics continued · Similar to Peano curve, a grammar is used to strictly follow the SFC without ”jumps”. Grid structure is stored on a structure-stack which

Sierpinski SFC

9. Hardware-Aware Numerics continued

Towards simulation using Sierpinski SFC ...


Numerisches Programmieren, Hans-Joachim Bungartz page 1 of 35

http://www5.in.tum.de/wiki/index.php/Univ.-Prof._Dr._Hans-Joachim_Bungartz

Sierpinski SFC

9.1. Sierpinski SFC

Introduction

• Shallow Water simulation running on full adaptive triangle grid:

• Increase resolution (similar to increasing the sampling resolution) at specific areas(e. g. the wave-front) to increase accuracy.




Sierpinski SFC

Application

• Depth averaged height h and momentum u or velocity v for each triangle-element:

U = (h, ux, uy , b)T = (h, vxh, vyh, b)

T

• Shallow water equation:

F (U) =

~vh,~vvxh + 1

2gh2ex,

~vvyh + 12gh2ey

• Conservation law: dU

dt+∇ · F (U) = 0

• Applying Gaussian divergence theorem and discretization using basis functionsϕi: ∫

T

dU

dtϕidT︸︷︷︸

mass-term

−∫TF (U) · ∇ϕidT︸︷︷︸stiffness-term

+

∮TF (U)ϕi · ds︸︷︷︸

flux-term

= 0

• Discretization using explicit Euler-timestep for dUdt

:

Ui(t + ∆t)︸︷︷︸U for next timestep

= Ui(t)︸︷︷︸U of previous timestep

+c·( S(U(t))︸︷︷︸Locally evaluated data

+ F(U(t), U+(t))︸︷︷︸Term depending on edge-comm-data

)




Sierpinski SFC

• Difference between the simulation and Peano matrix-operations:– For Matrix-matrix multiplication the Peano SFC was used to optimize

cache-awareness across 3 matrices.– For our simulation we like to improve the cache efficiency by accessing

element-data stored for each triangle-element in a linear and thus compactlayout but also the cache-awareness when accessing data of adjacenttriangle elements.

• Questions:– How to store grid-structure?– How to express/use locality and how to store element-data?– How to improve cache-aware data exchange between triangle elements?




Sierpinski SFC

Grid structure (1/2)

• Create grid using top-down approach and recursive bisection.• Similar to Peano curve, a grammar is used to strictly follow the SFC without

”jumps”.

K H VV'H'

K'V'

K'H'

K' H' V'VH

KV

KH

• Grid structure is stored on a structure-stack which is read top-down during atraversal.

– 0: leaf element reached– 1: follow recursive definition

• Example:

|00011b1

10

00




Sierpinski SFC

Grid structure (2/2)

• Binary tree stored with 0’s and 1’s on a structure-stack• Recursive definition is used to run a grid-traversal using the Sierpinski SFC.

100| 1 100 100| 1 1100100 11000|

1 1 1100100 110010011000|

1 1 1101001100011010011000 11000|

1 1 11010011100001101010011000 11000|




Sierpinski SFC

Element-data

• Similar to the structure-stack, the element-data is stored on an element-datastack.

• During the traversal of a leaf element, the next stored element on this stack areaccessed / modified.

• Relation between number of items stored on structure-stack andelement-data-stack:

– Necessary to know size of memory to be allocated for structure-stack.– Starting with only a single triangle for the grid, the structure-stack is given by|0.

– Refining a grid-cell is expressed by replacing a 0 with 001Grid structure with 2 triangles is given by |001.

– Therefore there is always one more 0 stored to the structure-stack than thereare numbers of 1:

|number of 0’s on stack| = |number of 1’s on stack|+ 1

– ⇒ Required structure-stack size depending on number of fine-triangles isgiven by

|structure-stack| = |element-data| · 2− 1




Sierpinski SFC

Edge-based-communication (1/2)

• We still need to access the element-data of adjacent elements.• Instead of accessing adjacent element-data using pointers, we use a feature

provided by the 2D Sierpinski SFC:

Edge-based communication via stacks.• We annotate the edges of the recursive definition using ’old’, ’new’ and ’boundary’

types:

V'

H'bKnb

VK

H

K

no

n

n

b

b

n

nnn

n

b

b

b b

o

oo

• This type describes whether the triangle adjacent to the currently traversed one is– going to be accessed later on in the traversal (new) or– if it was already accessed (old)




Sierpinski SFC

Edge-based-communication (2/2)

• We further utilize 2 additional stacks for edge-communication:The left- and right-edge-communication stack.

• Depending on the edge type and the placement of the edge at the left or right sideof the SFC, edge-communication-data is pushed (new) or popped (old) from therespective edge-communication stack.

• Running a forward- and backward-traversal for edge-communication:– During a forward-traversal an additional edge-comm-buffer-stack is used

to store edge-comm-data stored by previously traversed adjacent triangles.– Running a backward-traversal (and thus reversing the old/new types for the

inner edges), also the edge-comm-data from the remaining adjacent trianglesis available.Finally the computation is executed since all edge-comm-data from theadjacent triangles is stored on the edge-comm-stacks and theedge-comm-buffer-stacks.

• Don’t worry :-)Example given on the next slides!




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

0001100100111

Structure StackForward/Backward

EdgeComm StackLeft/Right

EdgeBufferStack




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

00011001001



EdgeBufferStack

1




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

00011001001



EdgeBufferStack




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

0001100100



a bc

EdgeBufferStack

0




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

000110010



a bc

EdgeBufferStack

0




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

000110010



a bfd

EdgeBufferStack

c00




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

00011001



a bfd

EdgeBufferStack

c001




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

00011001



a bfd

EdgeBufferStack

c001




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

00011001



a bfd

EdgeBufferStack

c001




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

0001100



a bfhi

EdgeBufferStack

cd

00

01




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

000110



a bfhi

EdgeBufferStack

cd

00

01




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

00011



a bfhl

EdgeBufferStack

cdi

j00

01

00




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

00011



a bfhl

EdgeBufferStack

cdi

j00

01

01




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

00011



a bfhl

EdgeBufferStack

cdi

j00

01

011




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

00011



a bfhl

EdgeBufferStack

cdi

j00

01

011




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

00011



a bfhl

EdgeBufferStack

cdi

j00

01

011




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

0001



a bfhl

EdgeBufferStack

cdi

j00

01

011




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

000



a bfho

EdgeBufferStack

cdi

jm

l

00

01

01

01




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

00



a bfho

EdgeBufferStack

cdi

jm

l

00

01

01

01




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u



a bf

EdgeBufferStack

cdi

jm

lp

00

00

01

01

00

o

1h




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u



a bf

EdgeBufferStack

cdi

jm

lp

0 00

01

01

001

1

oh




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u



a bf

EdgeBufferStack

cdi

jm

lp

0 00

01

01

001

1

oh




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u

0



a bf

EdgeBufferStack

cdi

jm

lt

ho

s

p

00

01

01

001

1

0




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u



a bf

EdgeBufferStack

cdi

jm

lto

s

p

00

01

01

001

1

01

h




Sierpinski SFC

A

BCE

F

G

abc

de

fg

h

i

jkl

m

n D

o

pq r

st

u



a bf

EdgeBufferStack

cdi

jm

lto

s

p

00

01

01

001

1

011

h




Sierpinski SFC

Final slide

• Parallelization using massive tree splits/joins

• Parallelization using fractal borders

Node1 Node2 Node3




Documents

9. Hardware-Aware Numerics continued · Similar to Peano curve, a grammar is used to strictly follow the SFC without ”jumps”. Grid structure is stored on a structure-stack which