1

Estimating Empirical Unit Hydrographs (and More)

Using AB_OPT

LMRFC Calibration WorkshopMarch 10-13, 2009

2

What is AB_OPT?• Stands for Adjoint-Based OPTimizer (developed by DJ Seo for the VAR

project with WGRFC)• What does it do?

1. Given SAC parameters, estimates PX_ADJ and PE_ADJ such that the multi-year accumulation of simulated flow matches that of the observed flow• Similar to double mass analysis

2. Given PX_ADJ*, PE_ADJ* and SAC parameters, estimates empirical UHG such that the simulated flow best matches the observed over the multi-year simulation period

3. Given PX_ADJ*, PE_ADJ* and empirical UHG*, locally optimizes SAC parameters such that the simulated flow best matches the observed

4. Latest version estimates/optimizes hydrologic routing parameters• A prototype algorithm for layered 3-parameter Muskingum has

been developed• Need further work to incorporate into AB_OPT

3

What is AB_OPT? (cont.)• For 1, 2 and 4, AB_OPT uses the conjugate gradient method for

optimization– Gradients are evaluated by the adjoint code generated automatically

by Tapenade (http://tapenade.inria.fr:8080/tapenade/index.jsp) from the forward code

• For 3, AB_OPT uses Sequential Line Search (SLS, Kuzmin et al. 2007) for optimization

• The objective function is a linearly weighted combination of mean error, mean square error, error in the histogram (for 1,2,3), and maximum errors of under- and over-estimation (for 4) of hourly (if QIN is used), daily, 10-daily and monthly flows– This emulates manual calibration

• For details, see:– “Fast and Efficient Optimization of Hydrologic Model Parameters

Using a Priori Estimates and Stepwise Line Search by Kuzmin, Seo and Koren (J. Hydrology, Volume 353, Issues 1-2, 20 May 2008, Pages 109-128)

4

• AB_OPT is a critical tool for fast-track transition to lumped modeling at 1-hr timestep– RFC service needs– Consistency in time scale with distributed

modeling– Transparency in time scale with Site Specific

What is AB_OPT? (cont.)

5

1) Estimation of PXadj and PEadj

• Given the SAC parameters, AB_OPT solves for the adjustment factors such that;– the sum of the model-simulated streamflow is

approximately the same as the sum of the observed streamflow over the simulation period

• Only the long-term aggregate water balance matters in the calculation (ignoring loss due to deep percolation)

• Does not depend on the shape of the UH

6

2) Estimation of UH

• Given pxadj, peadj and the SAC parameters, AB_OPT minimizes the difference (in the least squares sense) between the observed and the simulated flows

• Does not guarantee mass balance• The solution generally has oscillations near the origin

and at the tail end of the UH (some adjustments generally necessary and are performed automatically)

7

3) Refinement of SAC parameters• Given, pxadj, peadj and UH, ab_opt locally optimizes the

SAC parameters• May or may not result in significant improvement

Obj

ectiv

e fu

nctio

n va

lue

Parameter value

Current position

Global minimum

Local minimum

8

Sequential Line Search (Kuzmin et al. 2007)

9

Input Data for AB_OPT

• MAPX and 1-hour streamflow time series (OH Datacard format)

• Initial SAC-SMA parameters in NWSRFS format

• Initial UHG (placeholder)

• AB_OPT input control file

10

BUFFALO/FLATWOODS 1 FLTT1 MAP FLTT1 INFW SUMS 1.0001.000 55.0 22.00.3000.0100.0200.015 00.000 150.0 2.90 125. 40.0 280.0.150.00400.2000.300 0.00 1.301.702.303.905.506.306.606.305.203.301.901.40 18.1 0.0 29. 0.0 148. 43.

FLTT1 AB_OPT SAC-SMA Input File

11

AB_OPT Input Control File (1/4)

##################################################### # directory path and filename for starting SAC deck # ##################################################### /fs/hsmb5/hydro/users/sheldons/AB_OPT/fltt1_sac_para ##################################################### # directory path and filename for starting UHG deck # ##################################################### /fs/hsmb5/hydro/users/sheldons/AB_OPT/fltt1_uh #################################################### # UHG flag: set to 0 if UHG is unknown, 1 if known # #################################################### 0 ############################################# # directory path and filename for precip ts # ############################################# /fs/hsmb/nwsrfs/calb/data/area_ts/oper/lmrfc/fltt1/fltt1.MAP01dy ####################################################################### # flag for choice of flow data if multihour timestep is used for SAC: # # set to 1 to use QIN, 0 to use QME # ####################################################################### 1 ###########################################

specifyinput

12

########################################### # directory path and filename for flow ts # ########################################### /fs/hsmb/nwsrfs/calb/data/area_ts/oper/lmrfc/fltt1/fltt1.qin01nm ################################################## # directory path and filename for final SAC deck # ################################################## /fs/hsmb5/hydro/users/sheldons/AB_OPT/output/fltt1_sac ################################################## # directory path and filename for final UHG deck # ################################################## /fs/hsmb5/hydro/users/sheldons/AB_OPT/output/fltt1_uh_final ################################################################## # directory path and filename for raw empirical UHG (in ts form) # ################################################################## /fs/hsmb5/hydro/users/sheldons/AB_OPT/output/fltt1_raw_emp_uh ####################################################################### # directory path and filename for adjusted empirical UHG (in ts form) # ####################################################################### /fs/hsmb5/hydro/users/sheldons/AB_OPT/output/fltt1_adj_emp_uh

AB_OPT Input Control File (2/4)

specifyinput

specifyoutput

13

###################################################################### # directory path and filename for S curve-derived multihour UHGs (in # # ts form) # ###################################################################### /fs/hsmb5/hydro/users/sheldons/AB_OPT/output/fltt1_uhs_Scurve ################################################################# # directory path and filename for simulated (large events only) # # and observed flow ts # ################################################################# /fs/hsmb5/hydro/users/sheldons/AB_OPT/output/fltt1_events_only_ts

# directory path and filename for simulated (all flows) and observed # # flow ts # ###################################################################### /fs/hsmb5/hydro/users/sheldons/AB_OPT/output/fltt1_all_hours_ts #################################################################### # beginning and ending times of analysis (CURRENTLY DOES NOT WORK) # #################################################################### 1990070101 2008010101

AB_OPT Input Control File (3/4)

specifyoutput

14

########################### # warm-up period in hours # ########################### 30000 ###################################################################### # number of hours to the left of the flow exceeding 100 cms; used in # # isolating large events # ###################################################################### 50 ####################################################################### # number of hours to the right of the flow exceeding 100 cms; used in # # isolating large events # ####################################################################### 150

AB_OPT Input Control File (4/4)

specifyoutput

15

• PXadj and PEadj values

• Raw and final empirical UHG

• Final optimized SAC parameters

• Various plots (use ‘R’ scripts to generate plots)

Output for AB_OPT

16

Initial Empirical UGH and Adjusted Empirical UHG For FLTT

Output from AB-OPT

17

Quantileplots

Output from AB-OPT

18

Output fromAB-OPT

19

Steps for Using AB_OPT

• Run AB_OPT

• Run ‘R’ scripts to generate plots

• Use Linux ‘display’ command to display plots