Using Automatic Differentiation for Adjoint CFD Code...

Using Automatic Differentiationfor Adjoint CFD Code Development

Mike Giles Devendra Ghate Mihai Duta

Oxford University Computing Laboratory

post-SAROD Workshop – p. 1/24

Overview

why discrete adjoint?

automatic differentiation

an airfoil code example

post-SAROD Workshop – p. 2/24

Discrete Adjoints

Adjoint methods are very efficient for obtaining thesensitivity of one output (objective function) to many inputs(design parameters)

There are two adjoint approaches:

continuous: construct adjoint PDE and then discretise

discrete: discretise original nonlinear PDE, thenlinearise and use its adjoint/transpose

Both approximate the true gradient of the output – lattergives the gradient of the discrete output but this consistencyis unnecessary for many optimisers.

post-SAROD Workshop – p. 3/24

Discrete Adjoints

I prefer discrete adjoints because:

clear prescriptive process for constructing the discreteadjoint equations and boundary conditions;

usually guaranteed to get same iterative convergencerate as original nonlinear code;

Automatic Differentiation can simplify the developmentof the adjoint CFD code.

post-SAROD Workshop – p. 4/24

Discrete Adjoints

Suppose an input α leads to an output J :

α −→ X −→ U −→ J.

Defining α, X, U , J to be derivatives with respect to α,

X =∂X

∂αα, U =

∂U

∂XX, J =

∂J

∂UU,

=⇒ J =∂J

∂U

∂U

∂X

∂X

∂αα,

Note this calculation goes forward:

α −→ X −→ U −→ J ,

post-SAROD Workshop – p. 5/24

Discrete Adjoints

Now, defining α,X,U, J to be derivatives of J with respectto α,X,U, J ,

αdef=

(∂J

∂α

)T

=

(∂J

∂X

∂X

∂α

)T

=

(∂X

∂α

)T

X,

and similarly X =

(∂U

∂X

)T

U, U =

(∂U

∂J

)T

J,

=⇒ α =

(∂X

∂α

)T (∂U

∂X

)T (∂U

∂J

)T

J.

Note this calculation goes backward:

α ←− X ←− U ←− J.post-SAROD Workshop – p. 6/24

Automatic Differentiation

AD views each floating point operation as a separate stage:

X0 X1. . . XN−1 XN

- - - - $?

∂J/∂XN

%�

∂X1

∂X0

∂X2

∂X1

∂XN

∂XN−1

? ? ?

� � � �X0 X1. . . XN−1 XN

Note requirement to store data on forward pass in order touse partial derivatives on reverse pass.

post-SAROD Workshop – p. 7/24

Automatic Differentiation

To create key parts of our linear and adjoint CFD codes,we use AD software called Tapenade:

developed at INRIA by Laurent Hascöet and ValeriePascual

uses source code transformation, takes a Fortransubroutine as input and generates a new Fortransubroutine as output

given a subroutine which computes f(u), it can createroutines to evaluate

f =∂f

∂uu (forward mode)

u =

(∂f

∂u

)T

f (reverse mode)

post-SAROD Workshop – p. 8/24

Discrete Adjoints

Steady discrete CFD equations

N(U,X) = 0.

are often solved by iteration

Un+1 = Un − P (Un, X)N(Un, X),

The linearised equations

L U + N = 0, L ≡∂N

∂U, N =

∂N

∂XX,

can then be solved by the iteration

Un+1 = Un − P(

L Un + N)

.

post-SAROD Workshop – p. 9/24

Discrete Adjoints

SinceU = −L−1N ,

the adjoint equation is

N = − (LT )−1U =⇒ LTN + U = 0,

which can be solved iteratively using

Nn+1

= Nn− P T

(

LT Nn

+ U)

.

Written paper explains that P T corresponds to sametime-marching algorithm in simple cases, and we useAD to get code to evaluate LTN and U (and alsoL U and N for linear code).

post-SAROD Workshop – p. 10/24

Airfoil Code

Very simple 2D airfoil code:

cell-centred, unstructured quadrilateral grid

inviscid fluxes plus simple numerical smoothing

simple predictor/corrector time-marching with localtimesteps

Starting with a nonlinear solver, we use AD to generate thekey bits of both linear and adjoint codes.

post-SAROD Workshop – p. 11/24

Airfoil Code

Fortran files:

airfoil.Fnonlinear code

air_lin.Flinear code

air_adj.Fadjoint code

testlinadj.Fvalidation code

input.Finput/output routines

routines.Fnonlinear routines

post-SAROD Workshop – p. 12/24

Airfoil Code

Nonlinear routines:

TIME_CELL:computes the local area/timestep for a single cell

FLUX_FACE:computes the flux through a single regular face

FLUX_WALL:computes the flux for a single airfoil wall face

LIFT_WALL:computes lift contribution from a single airfoil wall face

post-SAROD Workshop – p. 13/24

Airfoil Code#ifdef COMPLEX

subroutine Cflux_wall(x1,x2,q,res)#else

subroutine flux_wall(x1,x2,q,res)#endifcc compute momentum flux from an individual wall facec

implicit none#include "const.inc"c

integer nc#ifdef COMPLEX

complex*16#else

real*8#endif

& x1(2),x2(2),q(4),res(4),& dx,dy, ri,u,v,p

cdx = x1(1) - x2(1)dy = x1(2) - x2(2)

cri = 1.d0/q(1)u = ri*q(2)v = ri*q(3)p = gm1*(ri*q(4) - 0.5*(u**2+v**2))

cres(2) = res(2) - p*dyres(3) = res(3) + p*dx

creturnend post-SAROD Workshop – p. 14/24

Airfoil Code

Different AD-generated versions of wall flux routine:

FLUX_WALL(x1,x2,q,res)

FLUX_WALL_D(x1,x2,q,qd,res,resd)

FLUX_WALL_DX(x1,x1d,x2,x2d,q,qd,res,resd)

FLUX_WALL_B(x1,x2,q,qb,res,resb)

FLUX_WALL_BX(x1,x1b,x2,x2b,q,qb,res,resb)

post-SAROD Workshop – p. 15/24

Airfoil Code

Part of Makefile:flux_wall_d.o: routines.F

${GCC} -E -C -P routines.F > routines.f;${TPN} -forward \

-head flux_wall \-output flux_wall \-vars "q res" \-outvars "q res" \routines.f;

${FC} ${FFLAGS} -c flux_wall_d.f;/bin/rm routines.f flux_wall_d.f *.msg

flux_wall_dx.o: routines.F${GCC} -E -C -P routines.F > routines.f;${TPN} -forward \

-head flux_wall \-output flux_wall \-vars "x1 x2 q res" \-outvars "x1 x2 q res" \-difffuncname "_dx" \routines.f;

${FC} ${FFLAGS} -c flux_wall_dx.f;/bin/rm routines.f flux_wall_dx.f *.msg

post-SAROD Workshop – p. 16/24

Airfoil Code

Part of Makefile:flux_wall_b.o: routines.F

${GCC} -E -C -P routines.F > routines.f;${TPN} -backward \

-head flux_wall \-output flux_wall \-vars "q res" \-outvars "q res" \routines.f;

${FC} ${FFLAGS} -c flux_wall_b.f;/bin/rm routines.f flux_wall_b.f *.msg

flux_wall_bx.o: routines.F${GCC} -E -C -P routines.F > routines.f;${TPN} -backward \

-head flux_wall \-output flux_wall \-vars "x1 x2 q res" \-outvars "x1 x2 q res" \-difffuncname "_bx" \routines.f;

${FC} ${FFLAGS} -c flux_wall_bx.f;/bin/rm routines.f flux_wall_bx.f *.msg

post-SAROD Workshop – p. 17/24

Nonlinear Code

define grid and initialise flow field

begin predictor/corrector time-marching loop

loop over cells to calculate timestep

call TIME_CELL

loop over regular faces to calculate flux

call FLUX_FACE

loop over airfoil faces to calculate flux

call FLUX_WALL

loop over cells to update solution

end time-marching loop

calculate lift

loop over boundary faces

call LIFT_WALL

post-SAROD Workshop – p. 18/24

Linear Code

define grid and initialise flow fielddefine grid perturbation

loop over cells -- perturbed timestepcall TIME_CELL_DX

loop over regular faces -- perturbed fluxcall FLUX_FACE_DX

loop over airfoil faces -- perturbed fluxcall FLUX_WALL_DX

begin predictor/corrector time-marching looploop over cells to calculate timestep

call TIME_CELL_Dloop over regular faces to calculate flux

call FLUX_FACE_Dloop over airfoil faces to calculate flux

call FLUX_WALL_Dloop over cells to update solution

end time-marching loop

calculate liftloop over boundary faces -- perturbed lift

call LIFT_WALL_DX

post-SAROD Workshop – p. 19/24

Adjoint Code

define grid and initialise flow field

calculate adjoint lift sensitivityloop over boundary faces

call LIFT_WALL_BX

begin predictor/corrector time-marching looploop over airfoil faces -- adjoint flux

call FLUX_WALL_Bloop over regular faces -- adjoint flux

call FLUX_FACE_Bloop over cells -- adjoint timestep calc

call TIME_CELL_Bloop over cells to update solution

end time-marching loop

loop over airfoil faces -- adjoint fluxcall FLUX_WALL_BX

loop over regular faces -- adjoint fluxcall FLUX_FACE_BX

loop over cells -- adjoint timestep calccall TIME_CELL_BX

loop over nodes to evaluate lift sensitivity

post-SAROD Workshop – p. 20/24

Airfoil Code

Validation is very important – can be done at various levelsstarting with the consistency of the individual nonlinear,linear and adjoint routines.

The “complex variable trick” gives

limε→0

I {f(u+iεu)}

ε=

∂f

∂uu

so the linear code can be checked against the nonlinear.

Also, by taking different unit vectors u can get∂f

∂u, and

compare to(

∂f

∂u

)T

from adjoint routine.

post-SAROD Workshop – p. 21/24

Airfoil Code

The next level is iterative equivalence between the linearand adjoint codes.

If both codes are initialised to zero (U0 = N0

= 0) then thepaper explains that

(Nn)T N

︸ ︷︷ ︸

adjoint code

= UTUn

︸ ︷︷ ︸

linear code

i.e. after the same number of iterations n, both codesshould give the same value for the linear sensitivity of theoutput (lift) to one input (angle of attack).

post-SAROD Workshop – p. 22/24

Airfoil Code

Final check is between linear sensitivity and nonlinear finitedifference

10−10

10−8

10−6

10−4

10−2

100

10−8

10−6

10−4

10−2

∆ α

rela

tive

erro

r

O(∆α2) error due to finite difference step size ∆α

O(ε/∆α) error due to finite machine precision εpost-SAROD Workshop – p. 23/24

Conclusions

I think discrete adjoints are preferable to continuousadjoints for practical reasons

clear prescriptive process;same iterative convergence rate;AD can simplify code development;

AD tools are now quite mature, and I personallyrecommend Tapenade

validation is also very important and can be done at anumber of different levels to give confidence in theadjoint code

finally, there will be a “hands-on” tutorial at ADA onWednesday

post-SAROD Workshop – p. 24/24