Optimal Estimation of Bottom Friction Coefficient for Free-Surface Flow Models Yan Ding, Ph.D. Research Assistant Professor, National Center for Computational

Optimal Estimation of Bottom Friction Coefficient for Free-Surface

Flow Models

Yan Ding, Ph.D.

Research Assistant Professor, National Center for Computational Hydroscience and Engineering, The University of Mississippi, University, MS 38677

Presentation for ENGR 691-73, Numerical Optimization, Summer Session, 2007- 2008

Outline

• Introduction and Brief Review of Optimal Parameter Estimation

• Optimal Estimation of Manning’s Roughness Coefficient in Shallow Water Flows

• Variational Parameter Estimation of Manning’s Roughness Coefficient Using Adjoint Sensitivity Method

• Discussions

I. Introduction and Brief Review of Optimal Parameter Estimation

Parameters in Mathematical Models

• Physical Parameters: measurable describe physical properties and simple physical processes

have state equations which describe the physical process, e.g., =(T, P)

• Empirical Parameters: unmeasurable complicated physical processes, unmeasurable from the physical

point of view, e.g., Manning Roughness n,

without state equation

have to be determined from historical observations

Complexity of Manning’s Roughness Physical-process-based parameters

H109876543210

0 250 500m

1.0m/s

Depth (m)

Manning Roughness n = f(surface friction, bed form friction, wave, flow unsteadiness ,vegetation (?), etc) [1] , orn=n(x,y) : distributed parameters, spatially dependent

Main Channel

Flood Plain

Bridge Pier

Moyie River at Eastport, Idaho

• Theoretical Framework for estimation of the Manning’s roughness by using optimal theories (Optimization)

• Estimate the Manning’s roughness so as to minimize the discrepancy between computation and observation

• Compare the efficiency and accuracy of various minimization procedures (High Performance of Identification)

• Demonstrate practical parameter estimation to determine the Manning’s roughness for the CCHE1D and CCHE2D hydrodynamic models (Application)

Objectives of Presentation

General Analysis Tools Based on Optimal Theories

Performance Function

Optimal Theory

Minimization Procedure

e.g.,Weight Least-Square Method

1. Sensitivity Analysis 2. Maximum likelihood 3. Extended Kalman Filter

1. CG Method 2. Newton Method 3. LMQN (BFGS, LBFGS, etc) 4. Sakawa-Shindo Method 5. Linear Programming 6. Quadratic Programming

Trust Region Method

Ill-Posedness of the Problem of Parameter Estimation

The inverse problem for parameter estimation is often ill-posed. This ill-posedness is characterized by:

• Non-uniqueness (Identifiability)The concept of identifiability addresses the question whether it is at all possible to obtain unique solutions of the inverse problem for unknown parameters of interest in a mathematical model from data collected in the spatial and time domains

• Instability of the identified parametersInstability here means that small errors in data will cause serious errors in the identified parameters

History of Parameter Identification in Computational Hydroscience and Engineering

• 1980s-

Sakawa and Shindo (1980): a general optimal control algorithm

Bennett and McIntosh (1982) and Prevost and Salmon (1986) : early work of variational method for tidal flow

Heemink (1987) : EKF to flow identification (English channel)

Panchang and O’Brien (1989) : Adjoint parameter identification (API) for bottom friction in a tidal channel

Das and Lardner (1990, 1992) : bottom friction in a tidal channel

History of Parameter Identification in Computational Hydroscience and Engineering

• 1990s-

Yu and O’Brien (1991) : wind stress coefficient

Richardson and Panchang (1992) : API for eddy viscosity,

Zhou et al (1993): LMQN for meteorology

Kawahara and Goda (1993) : eddy viscosity by maximum likelihood

Lardner and Song (1995): eddy viscosity profile in a 3-d channel

Hayakawa and Kawahara (1996) : velocity and phase by Kalman filter

Atanov et al (1999): Roughness in open channel (de St. Venant equation)

Ding & Wang (2003): API for Manning’s roughness coefficient in channel network in the CCHE1D model

Ding & Wang (2004): Sensitivity-equation method for estimation of spatially-distributed Manning’s n in shallow water flow models (CCHE2D)

Reviews and Books about Parameter Identification

• Review

Ghil and Malanotte-Rizzoli (1991), Advances in Geophysics, vol.33 : Review about data assimilation in meteorology and oceanography,

Navon (1998), Dynamics of Atmosphere and Oceans, vol 27: Review about parameter identification, identifiability and nonuniqueness related to ill-poseness of parameter identification problem

• Books:

Malanotte-Rizzoli, P. (1996): Modern Approaches to Data Assimilation in Ocean Modelling, Elsevier Oceano. Series

Wunsch, C. (1996): The Ocean Circulation Inverse Problem, Cambridge University Press.

Nocedal & Wright (1999), Numerical Optimization, Springer, NY

Journals and Online Resources about Optimal Parameter Estimation

• Journals SIAM J. on Control and Optimization /Optimization (OOOOO) J. Acoust. Soc. (OOOO) J. Geophysical Research (OOOO) Water Resource Research (American Geophysical Union) (OOO) Monthly Weather Review (OOO) Tellus (OO) ASCE J. Hydr. Engrg. (OO)

• Web Resources SPIN Database Online ( http://scitation.aip.org/jhtml/scitation/search/)

American Institute of Physics, NY ISI Web of Science (http://scientific.thomson.com/products/wos/ )

EI (Engineering Village) http://daypass.engineeringvillage.com/

Output Error Measure: Performance Function J

• To evaluate the discrepancy between computation and observation, one can define a weighted least squares form as

0

1( ) ( ) ( )

2

ft obs T obs

tJ X X X W X X dt

where X = physical variables, the weight W is related to confidence in the observation data. The superscript ‘obs’ means measured data. t0 :

the starting time, tf : the final time during the period of parameter

identification

, , , ,1 1

1( ) ( ) ( )

2

M Nobs T obs

i j i j i j i j jj i

J X x x W x x t

or its discrete form:

Mathematical Framework for Optimal Parameter Estimation

• The parameter identification is to find the parameter n satisfying a dynamic system such that

• Local minimum theory[3] : Necessary Condition: If n* is the true value, then J(n*)=0; Sufficient Condition: If the Hessian matrix 2J(n*) is

positive definite, then n* is a strict local minimizer of f

0

1( ) min ( ) : ( ) ( ) ( )

2

ft obs T obs

tf n J X J X X X W X X dt

subject to ),,( tnXFt

X

Establish an Iterative Procedure to Search for Optimal Parameter Value: Sakawa-Shindo Method (1)

• Assumed that the computational domain =l

(l=1,2,..L) in which the Manning roughness is nk,

by adding a penalty term, the performance function can be modified as the following form,

0

*( ) ( ) ( 1) ( ) ( ) ( 1) ( )1( , ) ( ) ( ) ( )

2

ftk l k k T k k kl l l l lt

J X n J X n n C n n dt where (k) is iteration step in the Sakawa-Shindo procedure, C(k) is a diagonal weighting matrix at the (k)th step, the weight matrix is renewed at every step.

n1 nl

nL

Fig. Spatially-distributed parameters

• Correction Formulation of nk

Considering the first-order necessary condition

J(n)=0,

the parameter can be estimated from the previous value as follow,

0

( 1) ( ) ( ) 1( ) ( )f

Ttk k k

l l obstl

Xn n C W X X dt

n

Sakawa-Shindo Procedure (2)

• Change the weighting matrix C(l) :

If J(k+1)J(k), then C(k+1)=0.9C(k),otherwise C(k)=2.0C(k), continue to do next estimation of n.

Sensitivity Analysis

• Evaluate the gradient of performance function

the sensitivity matrix can be computed by 1. Influence Coefficient Method [2] : Parameter perturbation

trial-and-error 2. Sensitivity Equation Method: Forward computation Solve the sensitivity equations obtained by taking the partial

derivatives with respect to each parameter in the governing equations and all of boundary conditions

3. Adjoint Equation Method: Backward computation The sensitivity is evaluated by the adjoint variables governed

by the adjoint equations derived from the variational analysis.

ft

t

objT

dtXXWn

XnXJ

0

)(),(

Minimization Procedures for Unconstrained Optimization

Trust Region Methods: change searching directions and step size

• Sakawa-Shindo method[4]

considering the first order derivative of performance function only, stable in most of practical problems

Linear Search Approaches: directions are pre-determined, change step size

• Conjugate Gradient Methods (e.g. Fletcher-Reeves method) [5]

The convergence direction of minimization is considered as the gradient of performance function.

• Truncated Newton Methods or Hessian-free Newton methods

• Limited-Memory Quasi-Newton Method (LMQN) [3]

Newton-like method, applicable for large-scale computation, considering the second order derivative of performance function (the approximate Hessian matrix)

Unconstrained optimization methods

1k k k kn dn :k :kdStep length Search direction

1) Line search

2) Trust-Region algorithms

Searching in Parameter Space

n1

n2

.n* d1

Contour of J

1) Steepest descent (Cauchy, 1847)

( )k kd J n

2 1( ) ( )k k kd J n J n

2) Newton

3) Quasi-Newton (Broyden, 1965; and many others)

Methods for Unconstrained Optimization

( )k k kd H J n 2 1( )k kH J n

4) Conjugate Gradient Methods (1952)

Methods for Unconstrained Optimization (cont.)

1 1k k k kd g s

1k k ks n n

2 1( )k k kd J n g

k is known as the conjugate gradient parameter

5) Truncated Newton method (Dembo, et al, 1982)

2 ( )k k k kr J n d g

6) Trust Region methods

( )k kg J n

Step Length Computation

1) Armijo rule:

( ) ( ) ( )m m Tk k k k k k kf x f x d f x d

(0,1) (0,1/ 2)

2( ( ) ) /T

k k k kf x d d

2) Goldstein rule:

1 2( ) ( )T Tk k k k k k k k k kg d f x d f x g d

12 12

0 1

3) Wolfe conditions:

( ) ( ) Tk k k k k k kf x d f x g d

( )T Tk k k k k kf x d d g d

0 1 Implementations:

Shanno (1978)

Moré - Thuente (1992-1994)

( )T Tk k k k k kf x d d g d

Remarks:

2) In conjugate gradient methods the step lengths may differ from 1 in a very unpredictable manner. They can be larger or smaller than 1 depending on how the problem is scaled*.

1) The Newton method, the quasi-Newton or the limited memory quasi-Newton methods has the ability to accept unity step lengths along the iterations.

*N. Andrei, (2007) Acceleration of conjugate gradient algorithms for unconstrained optimization.(submitted JOTA)

Limited-Memory Quasi-Newton Method (LMQN) (Basic Concept 1)

Given the iteration of a line search method for parameter n

nk+1 = nk + kdk

k = the step length of line search sufficient decrease and curvature conditions

dk = the search direction (descent direction)

Bk = nn symmetric positive definite matrix

For the Steepest Descent Method: Bk = I

Newton’s Method: Bk= 2J(nk) Quasi-Newton Method:

Bk= an approximation of the Hessian 2J(nk)

)(1kkkk nJBd

n1

n2

.n* d1

Contour of J

L-BFGS (One of LMQN method) (1)

• Difficulties of Newton’s method in large-scale optimization problem: obtain the inverse Hessian matrix, because the Hessian is fully dense, or, the Hessian cannot be computed.• BFGS (Broyden, Fletcher, Goldfarb, and Shanno, 1970) Constructs the inverse Hessian approximation ,

Tkkkkk

Tkk ssVHVH 1

kkkkkk JJynns 11 ,

Tkkkk

kTk

k syIVsy

,1

1 kk BH

However, all of the vector pairs (sk, yk) have to be stored.

L-BFGS (2)

• Updating process of the Hessian by using the information from the last m Q-N iterations

Tkkk

kmkT

mkmkT

mkTkmk

kmkT

mkmkT

mkTkmk

kmkT

mkTkk

pp

vvppvv

vvppvv

vvHvvH

)()(

)()(

)()(

2ˆ1ˆ1ˆ2ˆ1ˆ

1ˆˆˆ1ˆˆ

ˆ0ˆ1

Where m is the number of Q-N updates supplied by the user, Generally [3] , 3 m 7 and }1,min{ˆ mkm

Only m vector pairs (si, yi),i=1, m, need to be stored

S e t t h e i n i t i a l n = ( n 1 , n 2 , … n L ) T

k = 0

S o l v e t h e i n i t i a l s t a t e v e c t o r X 0 S h a l l o w W a t e r E q u a t i o n s

C a l c u l a t i o n o f p e r f o r m a n c e f u n c t i o n J 0 , g r a d i e n t g 0 , a n d s e a r c h

d i r e c t i o n d 0

C a l c u l a t i o n o f )( 11 kk nJg S e n s i t i v i t y A n a l y s i s

| | g k + 1 | | m a x { 1 , | | n k + 1 | | }

C a l c u l a t i o n o f J k + 1

S t o p

Y e s

N o

C a l c u l a t i o n o f kkkk dnn 1 L i n e S e a r c h

S e n s i t i v i t y A n a l y s i s

C a l c u l a t i o n o f 111 kkk gHd U p d a t e H e s s i a n m a t r i x b y t h e r e c u r s i v e i t e r a t i o n

S h a l l o w W a t e r E q u a t i o n s

nlk

lk

lk

n

nnMax

)( 1

Y e s

Y e s

S o l v e t h e s t a t e v e c t o r X k + 1

L - B F G S

JkJ 1

Flow chart of parameter identification by using L-BFGS procedure

L-BFGS-B

• The purpose of the L-BFGS-B method is to minimize the performance function J(n) , i.e.,

min J(n),subject to the following simple bound constraint,

nmin n nmax,

where the vectors nmin and nmax mean lower and upper bounds on the Manning roughness.

• L-BFGS-B is an extension of the limited memory algorithm (L-BFGS) for unconstrained problem proposed in [6].• For the details about the limited memory algorithm for bound constrained optimization method, see [7].

Scaling Problems

• When the difference of the identified parameters is very large, the identification suffers from the poor scaling problem. lead to instability in the identification process

• Scaling : parameter transformation n/=Dn Gill et al [8] proposed the diagonal matrix D

But we found this approach could not do very well, even destabilize the identification process (L-BFGS). Therefore, we proposed the two approaches for forming the matrix D

)(2

)(1

nJ

nJDi

rms

i nJD

)(2

1

rms

i nJ

nJD

)(2

)(1

Scaling 1 Scaling 2

II. Optimal Estimation of Spatially-Distributed Manning’s Roughness Coefficient in Shallow

Water Flows

Application to Shallow Water Equations

0])[( ,

jjuht

irr

jijjieijiji u

h

uugnuuguu

t

u3/4

2

,,,,, )()(

• Governing Equations

where n is the Manning roughness coefficient, is to be identified.

in ,

Boundary and Initial Conditions

Given the boundary S=S1S2, the boundary conditions are

subject to

on S1,

on S2,

The initial conditions are

at t=t0,

ii uu ˆ

ijijjiei tnuut ˆ)( ,,

0ii uu

0

n1 nl

nL

Sensitivity Equations for Forward Computation

• The sensitivity equations of sensitivity vector

• can be derived, by taking the partial derivative with respect to each parameter in the governing equations.

0)())(())(( ,,,,

ik

iik

iii

ki

k

i

k nuh

n

uu

nn

uh

nt

,0)(

23/43/4

2

,

irrk

k

irr

k

j

ik

i

jje

ikj

i

k

j

k

i

jj

k

i

uh

uugn

n

u

h

uugn

n

u

xn

u

xxng

x

u

n

u

n

u

xu

n

u

t

k

k

ofout

within

0

1

,,2,1,,, Lkn

v

n

u

n kkk

Boundary and Initial Conditions

• Boundary Conditions

,0 1Sonn

u

k

i

20 Sonn

t

k

i

,00

k

i

n

u

• Initial Conditions

0

0

0 ttatnk

This becomes a well-posed problem for the identification of Manning Roughness in shallow water flows

Numerical Techniques and Program Properties

• Numerical method to solve the sensitivity equations: Efficient Element Method (CCHE2D)

• Structural modules for parameter identification are compatible with CCHE2D;

• The user can choose an appropriate minimization procedure (Sakawa-Shindo method, L-BFGS, or L-BFGS-B) (Multifunction)

• Easy to prepare the input and control data files

Data Flows for Parameter Identification

Model of Parameter Identification of

Manning Roughness in the CCHE2D

Input data for flow model

Observation data: Filename: *.obs

Initial control data for parameter : Filename: *.man

Control data of L-BFGS-B: Filename: *.lbfgsb

Output data for flow model

Results of identified parameter: Filename: *.para iterate.dat

Results of performance function: Filename: *.per

History output for each observed point: *.opt_his

Final sensitivity results: Filename: *_sen.plt

Input Data Output Data

Numerical Results (Verification)

• Strategy for verification

Flow Model (CCHE2D)

Specified n

Computed Flow Flied

Create observation data

Parameter Identification

Identified n*

n=n* ?

Initial Values of parameters : far from the true valuesVerify : identifiability, uniqueness of solution, stability of process

Observation Data and Identified Parameters(Open channel flow: N=3)

X (m)

Wa

ter

Ele

vatio

n(m

)

0 250 500 750 100010.00

10.01

10.02

10.03

10.04

10.05

n1=0.01n2=0.02

n3=0.03

0 100 200mObserved Point

n1=0.01 n2=0.02 n3=0.03

(a) Longitudinal profile of water elevation

(b) Velocity vectors and observed points

Cases for verification of parameter identification (Open channel flow)

CASE

No.

Identification Method Objective

Values of

Manning’s n

Initial

Values

of n

Initial Value

of Weighting

Matrix C

Total

Observed

Points*

Bound

Constraint

of n

1 Sakawa-Shindo 0.01, 0.02, 0.03 0.005 10.0 6 No

2 LBFGS 0.01, 0.02, 0.03 0.005 N/A 6 No

3 LBFGS + Scaling 1 0.01, 0.02, 0.03 0.005 N/A 6 No

4 LBFGS + Scaling 2 0.01, 0.02, 0.03 0.005 N/A 6 No

5 L-BFGS-B 0.01, 0.02, 0.03 0.005 N/A 6 [10-5, 1.0]

6 L-BFGS-B+ Transition 0.01, 0.02, 0.03 0.005 N/A 6 [10-5, 1.0]

*) The observed variables at each point have three, i.e., two velocity components and water elevation.

Number of Q-N updates in L-BFGS: m=5

Case 1 : Sakawa-Shindo Method

(a) Iterations of Manning’s n (b) Iterations of performance function

Iterations of Sakawa-Shindo loop

Ma

nn

ing

'sn

10 20 30 40 50 60 700

0.01

0.02

0.03

0.04

n1

n2

n3

Objective Valuen=0.03

n=0.02

n=0.01

Iterations of Sakawa-Shindo loop

Pe

rfo

rma

nce

fun

ctio

n

10 20 30 40 50 60 7010-8

10-7

10-6

10-5

10-4

10-3

10-2

Case 2 : L-BFGS Method

(a) Iterations of Manning’s n (b) Iterations of performance function

Iterations of L-BFGS

Ma

nn

ing

'sn

10 20 30 40 50 60

10-2

10-1

100

n1

n2

n3


n=0.02

n=0.01

Iterations of identification process

Pe

rfo

rma

nce

fun

ctio

n

10 20 30 40 50 60 7010-12

10-10

10-8

10-6

10-4

10-2

100

102

Sakawa-ShindoLBFGS without Scaling

Case 3 and Case 4: L-BFGS +Scaling (1)

(a) L-BFGS+Scaling 1 (b) L-BFGS+Scaling 2


Ma

nn

ing

'sn

10 20 30 40 50 60

10-2

10-1

100

n1

n2

n3


n=0.02

n=0.01


Ma

nn

ing

'sn

10 20 30 40 50 60

10-2

10-1

100

n1

n2

n3


n=0.02

n=0.01

Case 3 and Case 4: L-BFGS +Scaling (2)

(a) Comparison of J (b) Comparison of norm of error


Pe

rfo

rma

nce

fun

ctio

n

10 20 30 40 50 6010-12

10-10

10-8

10-6

10-4

10-2

100

102

L-BFGS without scalingScaling 1Scaling 2


No

rmo

fe

rro

r

10 20 30 40 50 60 7010-6

10-5

10-4

10-3

10-2

10-1

100

LBFGS without scalingLBFGS with Scaling 1LBFGS with Scaling 2

Norm of error = ||n -nobj||2

Case 5 and Case 6: L-BFGS-B

(a) L-BFGS-B (b) L-BFGS-B+Transition


Ma

nn

ing

'sn

10 20 30 40 50 60

10-2

10-1

100

n1

n2

n3


n=0.02

n=0.01


Ma

nn

ing

'sn

10 20 30 40 50 60

10-2

10-1

100

n1

n2

n3


n=0.02

n=0.01

Bound constraint: 0.00001 n 1.0

Excellent!!

Comparison of performance function

Iterations of Identification process

Pe

rfo

rma

nce

fun

ctio

n

10 20 30 40 50 60 7010-12

10-10

10-8

10-6

10-4

10-2

100

102

Sakawa-ShindoLBFGSLBFGS + Scaling 1LBFGS + Scaling 2LBFGS-BLBFGS-B+Transition

Comparison of norm error ||n -nobj||2

Iterations of identification process

No

rmo

fe

rro

r

10 20 30 40 50 60 7010-6

10-5

10-4

10-3

10-2

10-1

100

Sakawa-Shindo (Case 1)LBFGS (Case 2)LBFGS + Scaling 1 (Case 3)LBFGS + Scaling 2 (Case 4)LBFGS-B (Case 5)LBFGS-B+Transition (Case 6)

Comparison of norm of gradient

Iterations of performance function evaluation

No

rmo

fg

rad

ien

t

5 10 15 20 25 30 35 4010-7

10-6

10-5

10-4

10-3

10-2

10-1

100

L-BFGS + Scaling 1L-BFGS + Scaling 2L-BFGS-BL-BFGS-B+TRANSITION

2)(nJ

Summary of Verification for Open Channel Cases

Case

No.

Total

number of

function

evaluation

J10log Total

CPU

time (s)

Averaged

CPU time of

one iteration

(s)

2

objnn Identification

Method

1 67 -7.517 67053.6 1000.8 1.82610 -3 Sakawa-Shindo

2 52 -10.169 60239.5 1158.5 1.93910 -5 L-BFGS

3 32 -10.241 37122.6 1160.1 2.10410 -5 L-BFGS+Scaling 1

4 41 -9.966 47540.0 1159.5 7.55010 -5 L-BFGS+Scaling 2

5 30 -10.155 34832.8 1161.1 2.40210 -5 L-BFGS-B

6 31 -9.680 35983.7 1160.8 1.31110 -5 L-BFGS-B

+ Transition

Remark: The L-BFGS-B with bound constraints has excellent features for stabilizing the identification process and accelerating the convergence.

Application: East Fork River

n1

n2

n3

n4

n5

Partition of Manning’s n

L = 5

Length of study reach = 3.3km

Ma

nn

ing

'sn

5 10 150

0.05

0.1

0.15n5

Ma

nn

ing

'sn

5 10 150

0.05

0.1

0.15

n1

Ma

nn

ing

'sn

5 10 15-1

-0.8

-0.6

-0.4

-0.2

0

n2

Ma

nn

ing

'sn

5 10 150

0.05

0.1

0.15

n3

Ma

nn

ing

'sn

5 10 150

0.05

0.1

0.15

n4

History of Manning’s n at each river reach

Procedure: L-BFGS

Problem: n2 < 0(unphysical meaning)


Initial Values n1=0.05 n2=0.01 n3=0.02 n4=0.03 n5=0.08

Ma

nn

ing

'sn

5 10 15 200

0.2

0.4

0.6

0.8

1

n1

Ma

nn

ing

'sn

5 10 15 200

0.2

0.4

0.6

0.8

1

n3

Ma

nn

ing

'sn

5 10 15 200

0.2

0.4

0.6

0.8

1

n4

Ma

nn

ing

'sn

5 10 15 200

0.2

0.4

0.6

0.8

1

n5

Ma

nn

ing

'sn

5 10 15 200

0.005

0.01

0.015

0.02

n2

Iterations of L-BFGS-B

History of Manning’s n at each river reach

Procedure: L-BFGS-B

Bound constraint of n:0.00001 n 1.0

Initial Values n1=0.05 n2=0.01 n3=0.02 n4=0.03 n5=0.08

Comparison of Water Elevation

5

5.5

6

6.5

7

7.5

8

5 5.5 6 6.5 7 7.5 8

Observed Water Elevation (m)

Co

mp

ute

d W

ater

Ele

vati

on

(m

)Ideal Result

optimized Result

Procedure: L-BFGS-B

Comparison of Identified Parameters : East Fork River

Identification

Method

n1 n2 n3 n4 n5

Sakawa-

Shindo 0.05790 0.00001* 0.02144 0.03128 0.08249

L-BFGS 0.05615 - 0.00026** 0.02493 0.03283 0.08289

L-BFGS-B 0.05839 0.00489 0.02525 0.03995 0.08915

*) The parameter was forced to reach the lower bound of Manning’s n.**) The parameter is unphysical.

Remarks on Optimization Algorithms

• Sakawa-Shindo Method Stable in most of practical cases, ‘rigid’ feature of convergence

in multi-parameters identification, the solution may not be optimal or minimizer in some cases

• LMQN methods: Fast, effective for large-scale parameter identification • Scaling Technique for L-BFGS Necessary to stabilize the process of the identification process • L-BFGS-B: the excellent minimization procedure not only guarantee the demand of bound constraints, keep the

parameters from being unphysical,but also stabilize and speed up the identification process.

Questions? and Discussion?

III. Variational Parameter Estimation of Manning’s Roughness Coefficient Using

Adjoint Sensitivity Method

Outline

•Optimal Estimation of Manning Roughness Coefficient in channel network (Theory Framework)

Definition of Objective Function

Adjoint Sensitivity Analysis of 1D Saint Venant Equations

Parameter Identification Procedure•Numerical Techniques•Verification and Application•Conclusions and Further Topics

Theoretically, • Establish a theory framework for identification of the

Manning’s n so as to minimize the discrepancy between computation and observation using adjoint approach

• Find an efficient procedure (Ding et al 2003)

For Engineering Applications, • Enable optimal estimation of the parameter in natural

river which may be single or channel network• Give a distribution of the parameter along river• Provide an analysis tool to determine the Manning’s

roughness for the CCHE1D model (Watershed Model)

Objectives

de Saint Venant Equations- Dynamic Wave

01

qx

Q

t

AL

02 2

2

2

fgSx

Zg

A

Q

xA

Q

t

L

3/42

2 ||

RA

QQnS f

A=Cross-sectional Area; q=Side discharge;=correction factor; R=hydraulic radius n = Manning’s roughness identified in the study

where Q = discharge; Z=water stage;

Initial Conditions and Boundary Conditions

],0[),0,(

],0[),0,(

0

0

LxxAA

LxxQQ

t

t

I.C. (Base Flow)

B.C.s

],0[),,0(0

TttQQx

],0[),,( TttLZZLx

Upstream

Downstream

)(ZQLx

or (Stage-discharge rating curve)

(Hydrograph)

or open downstream boundary

Output Error Criterion- Objective Function J

To evaluate the discrepancy between computations and observations, a weighted least squares form is defined as

,)()(2

1),,(

0 0

22 T L obs

Zobs

Q dtdxZZWQQWnZQJ

where Q = discharge; Z=water stage; the weight W is related to confidence in the observation data. The superscript ‘obs’ means measured data; T=the period of parameter identification; L=length of channel

Mathematical Framework for Parameter Identification

• The parameter identification is to find the parameter n satisfying a dynamic system such that

where Q and Z are satisfied with the continuity equation and momentum equation, respectively (i.e., de Saint Venant Equations)

• Local minimum theory : Necessary Condition: If n* is the true value, then J(n*)=0; Sufficient Condition: If the Hessian matrix 2J(n*) is

positive definite, then n* is a strict local minimizer of f

)),,,(min()( nZQJnf

x

t

A B

CD

O L

T

Variational Analysis- To Obtain Adjoint Equations

Extended Objective Function

dxdtLLJJT L

QA 0 0 21

* )(

where A and Q are the Lagrangian multipliers

Fig. Solution domain

Necessary Condition (Stationary Condition)

0* JJ on the conditions that

0),(0),(

1

2{

ZQLZQL

Variation of Extended Objective Function

0

*

])([])[(

||2

)||21

(

)||)1(2

(

)(

23

2

2

0 0 2

0 0 22

0 0 3

3/2

3

2

2*

0 0

dtQA

QA

A

QdxQ

AA

A

Q

ndxdtnK

QgQ

QdxdtK

Qg

xA

Q

tAx

AdxdtnK

QQRg

xA

Q

tA

Q

xB

g

t

dxdtnn

fQ

Q

fδA

A

Z

Z

f

QAQQ

QA

T L Q

T L QQQA

T L QQQQA

T LJ

where

*;3

42;

*

*3/2

BB

RB

n

ARK Top Width

Adjoint Equations for the Full Nonlinear Saint Venant Equations

)()(||)1(2

*2*obsQobsZQQAA QQ

A

QWZZ

B

W

AK

QQg

xB

g

xA

Q

t

)(2

2obs

QQAQQ QQAW

K

QgA

xA

xA

Q

t

According to the extremum condition, all terms multiplied by A and Q can be set to zero, respectively, so as to obtain the equations of the two Lagrangian multipliers, i.e, adjoint equations

Variation of J with Respect to n

ndxdtnK

QgQ

n

fnJ

T L Q

0 0 2

||2)(

If the identified Manning’s roughness is a set of distributed parameter,i.e., n = (n1, n2, …, nN)T, we have the chain law

NN

nn

Jn

n

Jn

n

JJ

22

11

dxdtnK

QgQ

x

f

ni

J T x

x

Qi

i

0 2

1 ||2

where ni represent the Manning’s roughness in the reach Li [xi, xx+1]

Boundary Conditions - for Single Channel

x

t

A B

CD

O L

T

Fig. Solution domain

Considering the contour integral in J*, This term I needs to be zero.

0

])([])[(23

2

2

DACDBCAB

QAQQ

QA dtQA

QA

A

QdxQ

AA

A

QI

],0[,0),(

],0[,0),(

LxTx

LxTx

A

Q

Final Conditions

],0[,0),0( TttQ

],0[),,(),(2

TttLQ

AtL AQ

Upstream

Downstream

Backward Computation

Internal Boundary Conditions – for Channel Network

213

321

QQQ

ZZZ

I.B.C.s of Adjoint Equations

2

3

2*

1

3

2*

3

3

2*

A

QB

A

QB

A

QB QQQ

32

22

12

A

Q

A

Q

A

Q QA

QA

QA

I.B.C.s of Flow Model

Fig. Confluence

Numerical Techniques

])1()[1(])1([),( 111

1ni

ni

ni

nitx

ttt

tx ni

ni

ni

ni

1

11

1 )1(),(

xxx

tx ni

ni

ni

ni

111

1 )1(),(

1-D Time-Space Discretization (Preissmann, 1961)

Solver of the resulting linear algebraic equations (Pentadiagonal Matrix)

Double Sweep Algorithm based on the Gauss Elimination

where and are two weighting parameters in time and space, respectively;t=time increment; x=spatial length

Minimization Procedures

• Limited-Memory Quasi-Newton Method (LMQN) Newton-like method, applicable for large-scale computation,

considering the second order derivative of objective function (the approximate Hessian matrix) (Ding et al, 2002)

Algorithms:

BFGS (named after its inventors, Broyden, Fletcher, Goldfarb, and Shanno)

L-BFGS (unconstrained optimization)

L-BFGS-B (bound constrained optimization)

Data Flows for Parameter Identification

Model of Parameter Identification of

Manning Roughness in the CCHE1D

Input data for the CCHE1D, e.g., *.bc, *.bf

Observation data: Filename: *.obs

Initial control data for parameter : Filename: *.man

Control data of L-BFGS-B: Filename: *.lbf

Output data from the CCHE1D

Results of identified parameter: Filename: *.para iterate.dat

Results of Objective Function: Filename: *.per

History output for each observed point: *hispt.plt

Input Data Output Data

Numerical Results (Verification)

• Approach for Verification

Flow Model (CCHE1D)

Specified n*

Retrieving Numerical

Results

Parameter Identification

Identified n

n=n* ?

Observation Data

Initial Estimation of n0

Initial Estimations of Parameters : n0 << n*Verify : identifiability, uniqueness of solution, stability of identification process

Verification of Model ( Case 1)– A Hypothetic Single Channel

Time

Dis

char

ge

TpTd

Qp

Qb

+2.0m

+0.0m

20m

70m

1:2

1:1.

5

Channel

Hydrograph Cross-section

Parameter L x t QP Qb Tp Td Unit (km) (km) (min) (m3/s) (m3/s) (hour) (hour) Value 10.0 0.5 10.0 1.0(0.55**) 0.5 100.0 10.0 16.0 48.0

*: Time interval of observation = 10t**: This value is used for solving adjoint equations

Observation Station*

Verification of Model (Case 1) – Single Parameter in A Hypothetic Single Channel

nobj n0 nmin nmax 0.03 0.01 0.001 1.0

m WQ WZ 7 1.0 1.0


Ma

nn

ing

'sn

Re

lativ

ee

rro

ro

fn

10 20 3010-3

10-2

10-1

100

10-7

10-6

10-5

10-4

10-3

10-2

10-1

100

101

102

Manning's nRelative error of n

Objective value :n = 0.03

History of Identified Manning's n in A Hypothetical Single ChannelTable Control Parameters in the L-BFGS-B

nobj= objective n in channeln0 = initial estimationnmin = lower boundnmax = upper boundm = store size in the LBFGS-B

Relative Error = |n-nobj|/n

Verification of Model (Case 1) – Performance of Lagrangian Multipliers

Iteration Processes of Lagrangian Multipliers Q and A at the Observation Station

Time (hr)

0 10 20 30 40 50-2E-05

-1.5E-05

-1E-05

-5E-06

0

5E-06

1E-05

1.5E-05

2E-05

ITERATION= 4

Q

Time (hr)

0 10 20 30 40 50-2E-05

-1.5E-05

-1E-05

-5E-06

0

5E-06

1E-05

1.5E-05

2E-05

ITERATION= 6

Q

Time (hr)

0 10 20 30 40 50-0.0003

-0.0002

-0.0001

0

0.0001

0.0002

0.0003

ITERATION= 2

Q

Time (hr)

0 10 20 30 40 50-4E-06

-3E-06

-2E-06

-1E-06

0

1E-06

2E-06

3E-06

4E-06

ITERATION= 2

ATime (hr)

0 10 20 30 40 50-4E-07

-3E-07

-2E-07

-1E-07

0

1E-07

2E-07

3E-07

4E-07

ITERATION= 4

A

Time (hr)

0 10 20 30 40 50-4E-07

-3E-07

-2E-07

-1E-07

0

1E-07

2E-07

3E-07

4E-07ITERATION= 6

A

Q(t) A(t) Pulse Response

Converging

10t

Impulse response functions

Verification of Model (Case 2) – Distributed Parameters in A Hypothetic Single Channel

Observation Stations

n1=0.01 n

1=0.02 n1=0.03

n0=(0.005, 0.005, 0.005)

0.001 n 1.0

Initial Estimations:

Bound Constraints:

Objective n



Man

ning

'sn

Rel

ativ

eE

rror

ofn

0 10 20 30 40 50 60 7010-3

10-2

10-1

100

10-2

10-1

100

101

n1

n2

n3

Relative Error of n

Objective Valuen3obj=0.03

n2obj=0.02

n1obj=0.01

History of identified parameters (CASE: Multi-parameters) by L-BFGS-B

2

2obj

obj

n

nn



Obj

ectiv

efu

nctio

n

Nor

mof

grad

ient

10 20 30 40 50 6010-7

10-5

10-3

10-1

101

10-7

10-5

10-3

10-1

101Objective functionNorm of gradient

Iterative process of objective function and its gradient(CASE: Multi-parameters)

2)(nJ


Time (hr)

Wat

erS

tage

(m)

0 10 20 30 40 500

1

2

3

4

5

SimulationObservation

Time (hr)

Dis

char

ge(m

3 /s)

0 10 20 30 40 500

20

40

60

80

100 SimulationObservation

Comparisons of Discharge and Stage at An Observation Station

Verification of Model (Case 3) – Distributed Parameters in A Hypothetic Channel Network

Observation Station

Confluence

L3 =

13,000m

L2= 4,500m

L1

=4,

000m

1

2

3

Channel No.

A

Channel No.

QP (m3/s)

Qb (m3/s)

Tp (hour)

Td (hour)

1 50.0 2.0 16.0 48.0 2 50.0 2.0 16.0 48.0 3 60.0 6.0 16.0 48.0

nobj=(0.01, 0.02, 0.03)

n0=(0.005, 0.005, 0.005)

0.001 n 1.0


Iterations of L-BFGS-B

Man

ning

'sn

Rel

ativ

eE

rror

ofn

0 10 20 30 40 50 60 7010-3

10-2

10-1

100

10-3

10-2

10-1

100

101

n1

n2

n3

Relative Error of n

Objective Valuen3obj=0.03

n2obj=0.02

n1obj=0.01

History of identified parameters (CASE: Multi-parameters) by L-BFGS-B


Iterations of Function Evaluation

Obj

ectiv

eF

unct

ion

Nor

mof

Gra

dien

t

0 10 20 30 40 50 60 7010-6

10-5

10-4

10-3

10-2

10-1

100

101

102

103

10-6

10-5

10-4

10-3

10-2

10-1

100

101

102

103

Objective FunctionNorm of Gradient

Iterative process of performance function and its gradient(CASE 5 : Channel network)


Searching Processes

Time (hr)

Dis

char

ge(m

3 /s)

0 10 20 30 40 500

10

20

30

40

50

60

70

80

90

100

110

120

130

140

150 ITERATION= 1ITERATION= 2ITERATION= 3ITERATION= 4ITERATION= 5ITERATION= 6ITERATION= 7ITERATION= 20ITERATION= 40Observation at I = 45

Iterative process of hydrographs at station No.45 (St. A)

Simulation: dt=10minObservation: dt = 100min

Time (hr)

Wat

erS

tage

(m)

0 10 20 30 40 500

1

2

3

4

5

6

7

8

9

10 ITERATION= 1ITERATION= 2ITERATION= 3ITERATION= 4ITERATION= 5ITERATION= 6ITERATION= 7ITERATION= 20ITERATION= 40Observation at I = 45

Iterative process of hydrographs at station No.45 (St. A)

Simulation: dt=10minObservation: dt = 100min

Comparisons of Discharge and Stage at An Observation Station (Downstream)

Application (Natural River) – Distributed Parameters in East Fork River, Wyoming

INLET

OUTLET

St. A

Map of the Study Reach (3km), East Fork River, Wyomin

Application (East Fork River) – Error Criterion and Control Parameters

,)(2

1),,(

0 0

2 T L obs dtdxZZnZQJ

0.01 n 0.5

Total Number of Reaches: 83Total Number of the Manning’s Roughness: N= 83 Identify the profile of n along the riverThe Size of Memory in the L-BFGS-BL: m = 10CPU time in PC with 500MHz : nearly one hour

Days

Sta

ge(m

)

Dis

char

ge(m

3 /s)

0 10 20 300

1

2

3

4

5

6

7

8

9

10

0

5

10

15

20

25

30

35

40

45

50

Discharge at InletStage at Outlet

For Identification

Application (East Fork River) – Boundary Conditions at Inlet and Outlet

First Flood: Parameter Identification

The Other:Prediction

Application (East Fork River) – Identification Processes of Profiles of n along the river

Distance From Outlet (m)

Man

ning

sn

0 1000 2000 30000

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

ITERATION= 1ITERATION= 3ITERATION= 6ITERATION= 10ITERATION= 20Optimal Profile

nmin=0.01

0 200m

Application (East Fork River) – Objective Function and Norm of Its Gradient

Iterations

Obj

ecti

veF

unct

ion

Nor

mof

Gra

dien

t

10 20 30 40 50 6010-5

10-4

10-3

10-2

10-1

10-5

10-4

10-3

10-2

10-1

Objective FunctionNorm of Gradient

Application (East Fork River) – Predictions of Stages

Days

Wat

erS

tage

(m)

0 10 20 307

7.5

8

8.5

9

9.5Observation at InletSimulation at InletSimulation at St. AObservation at St. A

For Identification Prediction

Conclusions

• An effective parameter identification tool has been established, based on the adjoint sensitivity analysis for the full nonlinear de Saint Venant Equations.

• The derived internal boundary conditions enable the tool to identify the parameter in network channel.

• This tool can included engineers’ experience about the parameter n in terms of bound constraints.

• As a result, combining with the CCHE1D model, this tool can be utilized for identifications of n in natural channels and branched network rivers.

• The adjoint equations can be directly applied to optimal control for watershed management in the future.

Topics in the Future

• Apply to automatic calibrations in river flows;

• Enable the tool to handle other hydraulic structures, e.g. culvert, bridge cross, …, so as to make the applications widely.

• Further study the data requirements in the parameter identification for possible research direction in optimally selecting observation sites;

• Extend to establish optimal control procedures for flood control and/or water quality management;

• Extend to the 2-D problems in flood waves and tidal flows by using the CCHE2D

Variational Parameter Estimation in 2-D Shallow Water Equations

Governing Equations:

Continuity:

Momentum

0hU hU

Lt x y

0bx

u x

hu huu huvL F fhv gh

t x y x

0by

V y

hv huv hvvL F fhu gh

t x y y

f: the Coriolis parameter

Eddy Viscosity and Bottom Friction

2 ( )x t t

u u vF h h

x x y y x

2 ( )x t t

v u vF h h

y y x y x

| |bx bC u u

| |by bC u v

Cb= bottom friction coefficient

Performance Function

1( ) ( ) ( )

2obs T obs

xyt

J X X X W X X dxdydt X = physical variables, (u, v, η)

Variational Analysis- To Obtain Adjoint Equations

Extended Objective Function*( , ) ( )b u u v v

xyt

J X C J X L L L dxdydt where A and Q are the Lagrangian multipliers

Necessary Condition

0* JJ

on the conditions that

Lη = 0Lu = 0Lv = 0

Adjoint Equations

λη:

λu:

λv:

( )obsu vh hg W

t x y

2 2 2

2 2 ( )

2 2( )

| | | |

u u u v u u vt t

obsbv u v

h hu hv h ht x y x x x y y x

C u v uvfh h W u u

u u x

2 2 2

2 ( ) 2

2 2( )

| | | |

v u v v u v vt t

obsbu u v

h hu hv h ht y x y x y x y x

C uv u vfh h W v v

u u y

Transversality (Final) Conditions of Adjoint Variables

( , , ) 0fx y t

( , , ) 0u fx y t

( , , ) 0v fx y t

Variation of J with Respect to n

( ) | | ( )b u v bbxyt

fJ C u u v C dxdydt

C

If the identified bottom friction coefficient is a set of distributed parameter, i.e., Cb = (Cb1, Cb2, …, CbN)T, we have the chain law

1 21 2

b b bNb b bN

J J JJ C C C

C C C

0

| | ( )f

i

t

u vtbibi

J fu u v dxdydt

CC

where Cbi represent the bottom friction over a subdomain Ωi Ω

[1] Yen, B. C. (1991), Hydraulic resistance in open channel, In: Channel Flow Resistance: Centennial of Manning’s Formula, B.C. Yen, Ed., Water Resources Publications, Highland Ranch, Colorado, 1-135.

[2] Yeh, W. W. –G. (1986), Review of parameter identification procedures in groundwater hydrology: The inverse problem, Water Resour. Res., 22(2), 95-108.

[3] Nocedal J. and S. J. Wright (1999), Numerical optimization, Springer Verlag Series in Operation Research, Ed.: P. Glynn and S.M. Robinson, Springer-Verlag.

[4] Sakawa Y. and Y. Shindo (1980), On global convergence of an algorithm for optimal control, IEEE Transactions on Automatic Control, vol. AC-25, No.6, 1149-1153.

References

[5] Fletcher, R. (1987), Practical Methods of Optimization, John Wiley and Sons, New York.

[6] Liu D.C. and J. Nocedal (1989), On the Limited Memory Method for Large Scale Optimization, Mathematical Programming B, 45, 3, pp. 503-528.

[7] Byrd, R.H., P. Lu and J. Nocedal (1995), A Limited Memory Algorithm for Bound Constrained Optimization, SIAM Journal on Scientific and Statistical Computing, 16, 5, pp.1190-1208.

[8] Gill P.E., W. Murray, and M. H. Wright (1981), Practical optimization, Academic Press, pp346-353.

References (cont.)

References (Cont.)

Byrd, R.H., P. Lu and J. Nocedal (1995), A Limited Memory Algorithm for Bound Constrained Optimization, SIAM Journal on Scientific and Statistical Computing, 16, 5, pp.1190-1208.

Ding, Y., Jia, Y., and Wang, S. S. Y., (2002), Identification of the Manning's roughness in the CCHE2D model model with Sakawa-Shindo method and Limited-memory methods, submitted to J. Hydr. Engrg..

Liu D.C. and J. Nocedal (1989), On the Limited Memory Method for Large Scale Optimization, Mathematical Programming B, 45, 3, pp. 503-528.

Nocedal J. and S. J. Wright (1999), Numerical optimization, Springer Verlag Series in Operation Reasearchn, Ed.: P. Glynn and S.M. Robinson, Springer-Verlag.

Vieira, D.~A., and Wu, W.-M., (2002), CCHE1D version 3.0-model capabilities and applications, Technical Report No. NCCHE-TR-2002-05, NCCHE.

Yen, B. C. (1991), Hydraulic resistance in open channel, In: Channel Flow Resistance: Centennial of Manning's Formula, B.C. Yen, Ed., Water Resources Publications, Highland Ranch, Colorado, 1-135.

Documents

Optimal Estimation of Bottom Friction Coefficient for Free-Surface Flow Models Yan Ding, Ph.D. Research Assistant Professor, National Center for Computational