CHPS Calibration Features

CHPS Calibration Features

Ben Balk (AMEC) and Matthijs Lemans (Deltares USA)

December 3, 2013

2

CHPS Calibration Features Agenda

• Introduction• Broad overview of initial operating capabilities• Highlight OHD documents and templates

• FEWS demo – general calibration features• Focus on displays and general layout

• Configuration overview• OHD Calibration Configuration Guide

• FEWS demo – calibration run• OHD Calibration Reference Manual

• Discussion• Some ways to tailor configuration for RFC needs

• Summary

3

Introduction

• Overview of CHPS calibration features• Replicate MCP3/ICP capabilities (not OPT3)• Not identical, but comparable functionality• Initial release similar to GraphGen roll-out in previous release

• Transformations/Configurations• FEWS transformations (STATQME, WATERBAL, PEAKFLOW)• OHD – FEWS Open API/custom transformations

• OHD documents and templates• Calibration Reference Manual• Calibration Configuration Guide• Configuration templates/samples

4

Introduction

• Calibration workflow• Current workflows

• Preprocessor• Basin model

• • Line 45 or 48 in GA modules

• New calibration modules • WATBAL• STATQME• PEAKFLOW

• New report module

5

Introduction



• • Line 45 or 48 in GA modules



6

Introduction



• • Line 45 or 48 in GA modules



7

Introduction

• Import Data Cards• Historical MAP, MAT and QME data• Manual Forecast Display• Set cold state start to beginning of calibration period

• Running calibration workflow• Manual Forecast Display or Forecast tab• Cold state start set manually or default in Topology.xml

8

Introduction

• Run with Modifiers• Use Modifiers tab for Snow-17, SAC-SMA, Unit-HG• multipleModuleParameterModifier – only add to calb SA

• snow17 calibration• sacsma calibration

• Preserve ratio/difference between sub-basin parameters• Keeping track of modifiers

• New Modifiers tab in Forecast Management Display• Restore deleted modifiers from previous runs

• Export to updated ModuleParFile

9

Introduction

• Viewing data within FEWS• Data Viewer (Filters.xml)• Plots (DisplayGroups.xml)• ICP-like displays• Previous simulations

• HTML report

10

FEWS Demo – General Calibration Features

• Repeat and highlight initial overview• Use pre-loaded localDataStore

• NWRFC basin• Snow-17 and SAC-SMA• Two elevation zones

• Focus• Displays• General layout and functionality• Preview of live demo to follow Configuration presentation

11

Configuration Overview

• Calibrations should be done in separate SA (xxrfc_calb)• Calibration Configuration Guide

• Installing CHPS calibration components• Thorough step-by-step process for initial implementation• Nicely grouped by directories

• Assume basin or segment already configured in existing SA• Affected configuration files

• Nice summary image (page 6 – Config Guide)• Details new (green) and modified files (blue)• Details directory structure• First time installation vs. copied/edited files for new basin

• No direct upload to OC except:• ModuleParFiles• Refreshed ColdStateFiles

12

Configuration – Installing Components

• Sample templates included in OHD-CORE-CHPS-3.2.a• ConfigSAMPLE_rosn7_12Z (SAC-SMA)• ConfigSAMPLE_slyp1awkAPICONT_12Z (SNOW-17, API-CONT)• ConfigSAMPLE_wgcm8_00Z (SNOW-17, SAC-SMA, two elev zones)• ConfigTEMPLATE

• Use appropriate template to set up initial basin• Simple headwater configurations• Global search and replace of sample basin IDs• May need additional templates (e.g. routing, >2 sub-basins)• No scripts provided to convert NWSRFS decks to CHPS

• Time issues (local time vs. GMT)• OHD templates in local time – end of calendar day is 00Z• Appendix A: Use of Time Zones in FEWS

13

Configuration – Workflows

• WGCM8_Stats_Calibration• WorkflowFiles/calibration• Copy appropriate workflow• Edit for new basin• Edit forecast group if needed

• PreProcessing_UpdateStates• MergeMAP• MergeMAT• ImportDataCard

• Flow_UpdateStates• Basin models• Update GA modules

• Register new workflow• WorkflowDescriptors

14






15






16






17

Configuration – PreProcessing Modules

• MergeMAP_UpdateStates• ModuleConfigFiles/preprocessing• Variable

• Add MAP_cardfiles

• Transformation• Add simple expression OR• Add inputVariable to existing

merge hierarchy

• Edit forecast group name

• Repeat for MergeMAT• ImportDataCard module

• ModuleConfigFiles/import• Add QME timeSeriesSet

• <timeStep unit="day"/>

• Correct timeZoneOffset

18

Configuration – Flow_UpdateStates Modules

• Basin models• cd ModuleConfigFiles/xxxxx• rm –f ModuleConfigFiles/xxxxx/*Forecast.xml

• Update GA modules• Comment out loopTimeStep so model does not

loop daily

• Comment out <timeZone> in SNOW17, SACSMA

19

Configuration – Calibration Modules

• Add directory for new basin• ModuleConfigFiles/calibration/xxxxx• Copy calibration modules from similar basin (e.g. wgcm8) to xxxxx directory• Rename appropriately

• Edit all modules using global replace for new basin & catchments• Double-check for correct moduleInstanceId’s and locationId’s• WATBAL• STATQME• PEAKFLOW

• Copy Report_STATQME module• ModuleConfigFiles/calibration• Only needed for initial installation

• Register new modules• ModuleInstanceDescriptors

20

Configuration – Water Balance Modules

• Check WATBAL_SNOW17 modules for:• Correct FGroupId MergeMAT_UpdateStates reference

• MAT_6h variable

• Correct moduleInstanceId and locationId for RSEL parameter• RSEL_6h variable• RSEL_day variable

• TimeSeriesSets used in Displays• Skip if no SNOW17 model used in basin

21

Configuration – STATQME Modules

• Check STATQME_xxxxx_Calibration module for:• Correct moduleInstanceId reference for basin SQIN

• SQIN_6hr variable• Could be ADDSUB, LAG/K, UNITHG, etc.

• Proper module instances included in moduleInstanceSet• Zone_SQIN_6hr variable• Catchments_SQIN_UpdateStates

• Only needed for multiple sub-basins or elevation zones

22

Configuration – STATQME Modules

• STATQME_xxxxx_Calibration_StatEquations• Flow interval values

• Copy values from NWSRFS deck• Card 2; column 26-55 if it exists

• Add them to xml snippet below:

23

Configuration – Peakflow Modules

• PEAKFLOW_xxxxx_Selection_Calibration• Can set lower and upper limits for peak selections

• Can set gap period between peak selections

24

Configuration – where we stand now

• WGCM8_Stats_Calibration• Copy appropriate workflow• Edit for new basin

• PreProcessing_UpdateStates• MergeMAP, MergeMAT• ImportDataCard

• Flow_UpdateStates• Update GA modules

• Calibration modules• WATBAL• STATQME• PEAKFLOW

• Registered workflow• Registered modules

25

Configuration – What is needed for displays

• ModuleConfigFiles – OHD templates• Add time series in SNOW17 and SACSMA modules

• RegionConfigFiles – OHD templates• Parameters.xml• Filters.xml• ModuleInstanceSets.xml• LocationSets.xml• Qualifiers.xml• TimeSteps.xml• ThresholdValueSets.xml• Thresholds.xml• Topology.xml

• SystemConfigFiles – OHD templates• DisplayGroups.xml• TimeSeriesDisplayConfig.xml

• UnitConversionsFiles – OHD template• displayEnglishUnits.xml

26

• SNOW17 UpdateStates• PSFALL• PRAIN• SNTMP• NEGHS• LIQW• PQNET

• Add time series in SNOW17 and SACSMA modules• ModuleConfigFiles/xxxxx • <importTimeSeriesActivity>

• SACSMA UpdateStates• LZDEFR• DIR_RO• IMP_RO• INT_RO• PRI_RO• SUP_RO• SUR_RO

Display Configuration – ModuleConfigFiles

27

Display Configuration – RegionConfigFiles

• Parameters.xml• New parameters just added to SNOW17 and SACSMA UpdateStates modules• Add new parameters calculated within new calibration modules

• SQME.PCTBIAS, SQME.CC, SQME.LOG, etc.• May have to merge with existing Parameters.xml

• Some parameters may already be defined for your RFC• parameterGroup, unit, displayUnit• Flows to be displayed in CMSD like ICP

• Filters.xml• Copy appropriate template

• Default includes SNOW17• WithoutSNOW17• withAPICONT

28

• ModuleInstanceSets.xml

• Example time series in Filters.xml

• LocationSets.xml


29


• Qualifiers.xml• Self-explanatory; place at top of file

• TimeSteps.xml• Add wateryear, monthly, season and quarterly time steps• Start of aggregation period is exclusive; end is inclusive

30

• ThresholdValueSets.xml • Thresholds.xml


31


• Topology.xml• showRunApprovedForecastButton – true

• Usually disabled during segment forecast runs• coldState start• Node is triggered to run calibration workflow

32


• Topology.xml – alternative approach• Extra nodes to link to Plot Overview thumbnails• All nodes run the same calibration workflow• Use of filterId to link Forecast node to Data Viewer node

33

Display Configuration – SystemConfigFiles

• TimeSeriesDisplayConfig.xml• Selection of preferred display options for parameters

• preferredColor• lineStyle• markerStyle• Check schemas for options

• Can configure as you wish• Will be used in Data Viewer• Implications when viewing previous runs

34


• DisplayGroups.xml• Copy appropriate template

• Default includes SNOW17• WithoutSNOW17• withAPICONT

• Predetermined plots• Defined color schemes• ModuleInstanceSets• LocationSets

• Time series displays from calibration modules• ICP-like displays• Can configure to your needs

• Colors• relativeViewPeriods• Addition of plots

35


• DisplayGroups.xml• <nodeId> links to “node id” in Topology.xml (Forecasts tab)• <plotId> links to “plot id” template within DisplayGroups.xml• locationSet in <timeSeriesSet> filters on <locationId> in displays

36

Display Configuration – UnitConversionsFiles

• displayEnglishUnits.xml• Unit conversions added for calibration

• Most notably a log conversion for QME and SQME

37

Configuration – What is needed for modifiers

• RegionConfigFiles/ModifierTypes.xml• multipleModuleParameterModifier – only add to calb SA

• snow17 calibration• sacsma calibration

• Preserve ratio/difference between sub-basin parameters• Min/max values• Expiry time

38


• DisplayConfigFiles/ModifierDisplay.xml• Create shortcut buttons in Modifiers tab (optional)

39


• ModuleParFiles/xxxxx/SACSMA_*_UpdateStates.xml• Add RUNOFF_COMPONENT_INTERVAL to all SACSMA parameter files

• ColdStateFiles/xxxxx/• Copy updated SACSMA parameter files to ColdStateFiles directory• Refresh the zip file; do separately for each catchment (i.e. LWR/UPR)

• cd <ColdStateFiles_dir>/xxxxx• cp ../../ModuleParFiles/xxxxx/SACSMA_*_UpdateStates.xml params_previous.xml• zip –f “SACSMA_*_UpdateStates Default.zip” params_previous.xml

• * is a token; expand to your RFC nomenclature• rm params_previous.xml

• Skip above two bullet items if interval already exists• Use steps to refresh cold state parameters at end of calibration

40

Configuration – additional files for local time

• RegionConfigFiles/ModuleConfigProperties.xml• Calibrations run in local time assuming midnight is 00Z• STARTLOCALHOUROFDAY set to zero

• STARTLOCALHOUROFDAY is referenced in SNOW17 GA module

41

Configuration – HTML report

42

Configuration – What is needed for HTML report

• SystemConfigFiles/Explorer.xml• Add Calibration Report explorerTask just before closing </explorerTasks> tag

• sa_global.properties• Add contents below to set up your web browser

43

Configuration – What is needed for HTML report

• statqme_statistical_summary_template.html• mkdir ReportTemplateFiles in base Config directory• Copy provided file to ReportTemplateFiles directory• .html file can be customized

• Reports/statqme• mkdir –p Reports/statqme in base calb SA directory (xxrfc_calb)

• statqmeStyles.css• Copy provided file to Reports directory• .css file determines style of report; can be customized

• CoefficientSetsFiles/Area_Coefficients.xml• mkdir CoefficientSetsFiles in base Config directory• Copy provided template and edit appropriately• Basin area in km2• Needed to compute flow interval biases for report

44

Configuration – DataCards

• Copy MAT/MAP/QME datacards to backup directory• Suggest creating a backup cardfiles directory if one does not already exist• mkdir –p Import/backup/cardfiles in base calb SA (xxrfc_calb)

• Datacard preparation for calibration• MATs/MAPs

• Check column 35 in datacard header line for appropriate catchment name• e.g. WGCM8U, WGCM8L

• QME• At column 35 in datacard header line, change USGS id to FEWS location id

• e.g. WGCM8• No zeros are allowed in the months column, so a global search and replace needed

• e.g. :%s/ 01/ 1/g will replace all 01 with 1 for January• FogBugz Case 862

• Copy prepared datacards to Import/cardfiles directory for import

45

FEWS Demo – Calibration Run

• Start with empty localDataStore• NWRFC basin

• Import Data Cards• Historical MAP, MAT and QME data in Manual Forecast Display• Set cold state start to beginning of calibration period

• Running calibration workflow• Manual Forecast Display or Forecast tab• Cold state start set in Topology.xml• T0 set in sa_global.properties

• Run with Modifiers• Preserve ratio/difference between sub-basin parameters• Keeping track of modifiers; use of descriptions• Restore deleted modifiers from previous runs• Export to updated ModuleParFile

46

FEWS Demo – Calibration Run

• Viewing data within FEWS• Data Viewer (Filters.xml)• Plots (DisplayGroups.xml)• ICP-like displays• Previous simulations

• HTML report

• Show some tailored configurations with pre-loaded localDataStore• APRFC basin• Plot displays• Filters• Previous simulations

• Suggestion for new plot template in DisplayGroups• QME/SQME with no predefined color; will key off TimeSeriesDisplayConfig

• Overlap qualifier• Necessary if missing gaps in QME record

47

Forecast Management Display

• Recap from demo• Modifiers tab• Description – added through “Run options” or Manual Forecast Display• Show details modifier• Copy

48

Modifiers

• Recap from demo• Restored modifier from copy function in Forecast Management Display

• Magenta – needs to be rerun• Export button to drop moduleParFile to selected directory

• Next to Apply button

49

Overlap qualifier

• Overlap qualifier• Not all QME records will be complete• If QME record has gaps STATQME stats in report will not match ICP summary

• Both QME and SQME stats derived for available data• APRFC example (T0 10-01-2006, cold state start 10-01-1970):

• QME: 1978-1993: 2000-2006• SQME: 1970-2006

50

Overlap qualifier

• Overlap qualifier (continued)• Added additional variables and transformations with an overlap qualifier• Report overlap variables in HTML report

• Statistics match ICP statistical summary

51

Overlap qualifier – STATQME module update

• STATQME variables • STATQME transformations

52

Overlap qualifier – Report module update

• Report_STATQME.xml

With overlap

Without overlap

53

Water Balance displays

• Instantaneous (e.g. SWE) vs. accumulative (e.g. RAIM)• Average of 6h values not informative for accumulative parameters• Suggest aggregating over various time frames

54

View scroller

• SystemConfigFiles/TimeSeriesDisplayConfig.xml (optional)

55

Previous simulation display

• Create new plot display for QME and SQME (optional)• TimeSeriesDisplayConfig.xml handle color selection

56

Previous simulation display configuration

• SystemConfigFiles/DisplayGroups.xml• Add plot template and new display

57

Discussion topics

• Configuration templates are merely a starting point• Tailor configuration to RFC needs• Tailor configuration to personal preference• Create new displays

• e.g. APRFC – glacier mass balance; no need for outside spreadsheets

• May need additional templates• Routed basins

• Routing parameters adjusted directly in ModuleParFiles• Similar to editing MCP3 decks• Make backup copy first

• 3-or-more catchment basins• Others?

• Separate calibration SA’s for different forecast groups or users

58

Discussion topics – calibration team members

• Calibration module templates (NWRFC)• Savvy use of locationSets and moduleInstanceSets• Only one copy of modules used for all segments• Managing configuration changes more efficient and consistent• Multiple basin calibrations require edits to LocationSets and reloading config• Template concept is being implemented at Australia BOM – .csv files

• Previous simulation workflow (NWRFC)• Previous run time series displayed on some plots• Export-import to bring in previous run under different moduleInstanceId

• Streamlining calibration modules (CNRFC)• Deleting unused transformations (e.g. WATBAL)

59

Discussion topics – multiple basin calibrations

• Streamline configuration for efficiency• Trial period of building upon OHD templates• Prune back configuration to align with use• Delete or comment out unused transformations (e.g. WATBAL)• Comment out unused modules in Flow_UpdateStates workflow

• e.g. Adjust-Q, Stage-Q, Change-T• Extra topology node to run a slim workflow?• Select specific modules to run in Manual Forecast Display

• e.g. F12-4 to select UpdateStates and STATQME only• Manual Forecast Display not as useful as running from Forecasts tab

• Be careful of effect on mods

• Report_STATQME (individual vs. multiple)• Not basin specific (uses LocationSets and ModuleInstanceSets)• For multiple basins, report module will re-generate all summaries• Could add Report workflow and separate from individual calibration workflows

60

Summary

• New configuration for calibration but very similar to live config• Slimmed down• UpdateStates modules/workflows only

• Most ICP functions ported over• Prime opportunity to test

• Can lead to improvements and enhancements• Future enhancements depend on trial runs and feedback from RFCs• Use FogBugz to enter requests for fixes/enhancements

• Flexible• “everything is configurable”• Expect lots of customization at each RFC• Pros and cons; need configuration expertise

• Potential sharing of configuration ideas between RFCs

61

Summary – calibration to operations

• Initial implementation of calibration configuration (calb SA)• Calibrate• Export ModuleParFiles• Update/refresh ColdStateFiles parameters• Test ModuleParFiles in live SA• Upload ModuleParFiles/ColdStateFiles to MC02 via CM

• Test on devtest system

• Upload files to MC00 via CM• Files should then synch over to backup (MC01)

• For new calibrations not already in live system:• Uncomment sections of ModuleConfigFiles (GA modules)

• loopTimeStep• timeZoneOffset

• REMINDER – do NOT upload entire calb config to live system

Questions?

63

Questions?

Documents

CHPS Calibration Features