© Crown copyright Met Office Strategic Intervention A novel use of Ensembles in Forecast Guidance Ken Mylne (Ensemble Forecasting Applications Manager)

© Crown copyright Met Office

Strategic Intervention A novel use of Ensembles in Forecast GuidanceKen Mylne (Ensemble Forecasting Applications Manager) & Nick Grahame (Head of the Guidance Unit, Operations Centre)


Motivation – the Problem

• How to use forecasters’ guidance in automated production?

• Automated forecasts based on post-processed NWP verify best, but…

• No means for forecaster to correct rare forecast errors – busts

• How to edit or correct 5000 UK sites?

• Forecasters use Metmorph to produce modified guidance, but…

• Metmorph cannot feed automated 5000 sites

• One direct feed to BBC graphics


The Solution – “Change Once, Use Many”

• Edgemoor Agreement – forecasters and research met at Edgemoor Hotel and agreed new solution

• Strategic Intervention – allow forecaster to select an alternative NWP solution that will feed downstream outputs directly.

NWP Post ProcessingCustomer Specific Post Processing

Quality Check process

Forecaster Advice

Strategic Intervention

Output

PWS outputIntermediate State

using strategic Intervention

© Crown copyright Met Office NWP Refresher - Ensembles

The Edgemoor AgreementStrategic Intervention tool

• The SI tool will provide:• Allows the Chief Forecaster to select an alternative data

source to drive the production process

• Reduced usage of downstream intervention

• Priorities for usage:• Days 1-2

• Beyond day 2 should be probabilistic – no intervention

• Impact – large impact on customers

• 3 types of errors anticipated:

• Poor representation of trapped BL cloud/fog in winter or spring anticyclones

• Errors in precipitation intensity/distribution/type

• Synoptic scale errors and outliers compared to other models


SI Production Flow

Latest ObsForecaster Selects NWP guidance

Standard Production

(as now)

Intervened Production

Post-processing & Automated Production

UKV

Best-Data

Sites,

Charts

Website

Products

etc…

Probabilistic Production

MOGREPS members


MOGREPS-UK member

Best-Data

Sites,

Charts

Website

Products

etc…


UKV EPS

Ensemble Best-Data

Sites,

Charts

Website

Ensemble products

Impact model

etc…

UKV follows guidance

Change once, use many

Previous Deterministic NWP run

MOGREPS memberLatest Deterministic

NWP run

UKV


SI Project

• Phase 1 SI – allows forecaster to select the previous run of the deterministic NWP – delivered April 2011

• Phase 2 SI – will allow selection of a member of MOGREPS

• Controversial!

• Phase 1Previous Run

• Phase 2MOGREPS Member


Selection of Ensemble Members – a good idea?

• Ensembles designed for estimating the local forecast pdf

• Perturbed members intended to be used as a group, NOT as alternative scenarios

• Perturbed members expected to verify worse than control – factor √2

• Identification of the “preferred member” or the “best member” is very difficult:

• Best member locally at T+6 may not be best at T+24 or T+48

• Influences from upstream

• Different best members for different parameters and locations


SI Potential - Daily Skill scores Previous run & MOGREPS membersMSLP

25% members better than BaU


SI Potential - Daily Skill scores Previous run & MOGREPS members Temperature



SI Potential - Daily Skill scores Previous run & MOGREPS members Precipitation Ave(ETS for 6h precip>0.5, 1.0, 4.0mm)



Single day multi-variable plotCan be calculated for any member Different forecasts better for different variables


Potential for improvements

• MOGREPS often has >25% members better for certain variables

• Previous run occasionally better than BaU

• Little correlation between variables

• Less chance of there being a member which is best for everything

• Can we identify such a member if it exists?

• Likely that SI can only correct the worst element, with others being acceptable


Business Rules

• Using perturbed forecasts regularly would degrade the average verification

• Restrict SI selection to occasional use to correct severe errors

• Expect forecast to be degraded away from area targeted for correction

• Business Rules :• BUSTS - Forecaster believes latest NAE is seriously

in error having a major impact on UK customers.

• First 24h only


MOGREPS-UK and the UKV• UKV is 1.5km convective-scale

model • UK 36h forecast in Best Data

• MOGREPS-UK is new ensemble using 2.2km version of UKV

• Under SI Phase 2 forecaster will select alternative solution from MOGREPS-UK

• Replace UKV in Best Data

• Selection by:

• Synoptic errors – upstream error detection in analysis and selection from MOGREPS-EU

• Mesoscale errors – error detection in UKV and selection from MOGREPS-UK


Tools to aid selection

• Tools to compare imagery with model fields

• Very difficult with web-based tools

• Now developing pseudo-imagery from MOGREPS members to allow full comparison with imagery in forecaster workstations


SI Limitations

• SI is NOT a replacement for Metmorph

• Used rarely – major busts only

• Chosen solution will not correct all errors or fully represent guidance

• Only 12 members to choose from

• Quite likely worse than latest run AWAY from targeted area

• Aim is to reduce major high impact error to lesser lower impact errors


Conclusions and Discussion

• Strategic Intervention allows the forecaster to select an alternative source of NWP to drive the automated forecast production

• Change Once, Use Many

• Latest model is usually the best

• Overuse of SI would be damaging

• Strict Business Rules for rare severe errors

• Unconventional use of ensembles

• Use with care!

Documents

© Crown copyright Met Office Strategic Intervention A novel use of Ensembles in Forecast Guidance Ken Mylne (Ensemble Forecasting Applications Manager)