Upload
allison-ogrady
View
217
Download
1
Tags:
Embed Size (px)
Citation preview
© Crown copyright Page 1
Operational Use of ECMWF products at the Met Office: Current practice,
Verification and Ideas for the future
Tim Hewson 17th June 2005
© Crown copyright Page 2
Contents
Describe use of ECMWF output, in the Operations Centre at the Met Office (which provides guidance to the outfield), in 3 forecast categories:
1. ‘Short range’ (up to 36 hours)
2. ‘Medium range’ (36 hours to 6 days)
3. ‘Trend period’ (6 to 10 days)
4. Some verification ideas (severe weather)
© Crown copyright Page 3
1. Short Range
Relatively recent changes in ECMWF operational suite (2 operational runs per day, much reduced delay in forecast arrival time, etc) render ECMWF operational runs much more useful for short range forecasting than hitherto
Data time (DT) difference c.f. Met Office operational runs now averages 9 hours (previously it was 21 hours)
9 hours is comparable to the short period lead time gain of ECMWF over Met Office (10 hours), implying high potential for adding value to deterministic forecasts using the consensus (“poor man’s ensemble”) approach
Though not necessarily part of the ‘ECMWF remit’ if ECMWF model output can potentially improve our short range forecasts we will use it
Some use also made of ensemble data (though much less than in the medium range – due to reduced reliability of severe weather probabilities)
© Crown copyright Page 4
Operational Model Errors
1 5 days 10
RANK Best - EC UK FR..
NH rms MSLP errorvs Lead Time
Lead time gain = 10 hours
short range | medium range | trend period |
© Crown copyright Page 5
RAWMODOBS
Short Range Forecast Example
RAW
Cloud coverPpn ratePpn typeMslp
© Crown copyright Page 6
“What was it, specifically, about the good forecast that made it better?”
Helps highlight common model errors
Verification of Modifications
© Crown copyright Page 7
Scope for Improvement
Forecasters are usually able to improve upon raw (Met Office) model output, using different models, knowledge of systematic errors, comparison with current trends
Degree of improvement could potentially be increased by making more use of the high quality ECMWF operational run, which at present is under-utilised
WISH LIST! –
3 hourly data, T+0 to T+48
Instantaneous total ppn rates, plus cloud cover and mslp (same format?!)
Separate plots showing dynamic /convective rain and snow components
10m mean wind and likely gust strength
Sub areas – parts of Europe ?
Timely appearance on ECMWF web site is crucial (probably the most expedient route for making this data available)
© Crown copyright Page 8
Severe Weather
Table records ‘Significant Errors’, from perspective of hazardous weather, in Raw (Red) and Modified (Blue) forecasts (2.5 years of data)
A key forecasting target is a reduction in the number of blue boxes
This verification has highlighted warm air summer convection as one area where the forecaster is able to add little value, and where there can be major errors in the model
Potentially ECMWF oper runs could assist in this area, via the wish list data? (convective characteristics?)
Cold air convection is also a problem area, though one that the forecaster often addresses reasonably successfully. FGEW verification has highlighted this as a significant weakness in ECMWF output (snow in NE’lies, Feb/Mar 2005)
Low pressure centres and surface wind gusts in strong gradient regions are another sig problem
© Crown copyright Page 9
2. Medium Range
Use of ECMWF data - operational and ensemble runs - is more common in the medium range
Timeliness is a key issue for practical applications; recently the ECMWF web site has been used more often, because of timely updating
Issued forecast guidance has both deterministic and probabilistic components
© Crown copyright Page 10
Deterministic Component
Underlying Strategy: First ascertain the key meteorological feature(s),
Then incorporate different model and ensemble solutions accordingly, weighting as appropriate (on screen or on paper) to arrive at a consensus solution
© Crown copyright Page 11
Operational Model Errors
1 5 days 10
NH rms MSLP errorvs Lead Time
short range | medium range | trend period |
© Crown copyright Page 12
•Cold front is themain feature
•Consensus wouldmove GM front south,possibly hinting atwave development,as GM underdoeswaves, and as NCEPshows one.
Simple Example to Illustrate
© Crown copyright Page 13
Modifying a Selected Model Run
Use Field Modification tool (devised by Eddy Carroll) AFTER a model run has finished
Allows quick, interactive, dynamically consistent changes to be made to a set of 3-d fields from one model run, eg:
Move low or front Deepen/fill low Introduce low or wave…
Relies on modification vectors applied to PV distribution, followed by PV inversion with changed boundary conditions
Equivalent translation vectors applied to ppn and RH; simple boundary layer model used for surface winds
Temporal consistency achieved via time-linking (with ‘decay’ parameter)
Precipitation rate & type, winds etc can also be adjusted directly and time-linked
© Crown copyright Page 14
Field Modification Example – moving a low with slight deepening
© Crown copyright Page 15
Resulting fields (takes ~2 seconds)
© Crown copyright Page 16
Initial Fields
© Crown copyright Page 17
500mb ht and 100-500mb Thickness - before
© Crown copyright Page 18
500mb ht and 100-500mb Thickness - after
© Crown copyright Page 19
Objective Verification
© Crown copyright Page 20
Subjective Verification
Met Office Global model, ECMWF Oper and issued Modified forecasts are compared, primarily for mslp, but also for fronts and thickness, over NW Europe, using 12Z data times, for T+48,72, 96 and 120h
The overall ranking is (best first): Modified >> ECMWF >> UKMO
This is despite the objectively calculated lead time gain of ECMWF over UKMO (10-16 hours) being generally greater than it is for Modified over UKMO (0-12 hours)
Objective schemes can penalise good forecasts of cyclogenesis (especially rms errors)
A new ‘more discriminating’ subjective verification scheme will be introduced this year
© Crown copyright Page 21
Forecaster Impact
Modification Time
© Crown copyright Page 22
Probabilistic component
Consists of a headline summary of probabilities of severe weather, in a number of categories (introduced relatively recently)
Minima and maxima for sites are issued with upper and lower bounds. Probabilities are given for rainfall total exceedance. Starting point is calibrated ECMWF ensemble output.
‘Alternative scenarios’ can be issued to highlight uncertainties
Probabilistic components appear to be used far less by customers than the deterministic component. Somewhat disappointing – education required, but may take a long time.
© Crown copyright Page 23
3. Trend Forecast
Consists of an issued worded forecast, with expected temperature anomalies, and considerable discussion of uncertainties, highlighting possible severe weather
Based primarily on ECMWF data, but with some input from other models and ensembles, notably from NCEP
Customer base for this forecast has been dwindling (perhaps they watch Swedish TV)
© Crown copyright Page 24
Trend Period – Subjective Verification
ECMWF - OPERATIONAL vs ENSEMBLE - DAYS 6 TO 10 (2004)
0102030405060708090
100
Operational Days 6 and7
Ensemble Days 6 and 7 Operational Days 8 to 10 Ensemble Days 8 to 10
A = good/useful forecasts. B = indeterminate. C = poor or misleading forecasts
A
B
C
Based primarily on mslp, over NW Europe, ECMWF only Ensemble mean rated a little more useful than Oper run, but not by much Only 1 in 5 forecasts for days 6 and 7 were considered useful, and 1 in 10 forecasts for days 8, 9 and10 Scores have not changed a great deal over the years
© Crown copyright Page 25
Extras for ‘Wish List’
Meteograms to include overlapping 24-hour rainfall totals (but still in 6 hour blocks)
Total cloud cover – is this ‘altitude weighted’, or is 8 oktas cirrus considered ‘cloudy’? Weighting would be preferable
Postage stamps showing estimated surface gusts, with colour-shading for high values
Cluster ensemble means for mslp, thickness, annotated with percentages of members
© Crown copyright Page 26
4. Verification – some ideas
Conceptually there should be a ‘deterministic limit’ for predicting a pre-defined meteorological event
Simply defined this could be the point in lead time beyond which forecasts concerning that event are more likely, on average, to be wrong than right
Defined in this way, this provides some guidance on when to shift the emphasis, in forecasts for particular events, towards the probabilistic
For rare events at least, correct null forecasts – ie the majority - can be ignored as not relevant
© Crown copyright Page 27
Verification ideas (contd)
The ‘deterministic limit’ for the event in question is then simply the lead time at which, over a suitably large forecast sample, hits equals the sum of misses and false alarms (or CSI = 0.5)
misses + false alarms
hits
number
‘deterministic limit’
lead time
dc
baFc
Ob
a/(b+c) = 1
© Crown copyright Page 28
Event Examples (numbers are very crude estimates)
Tornado within 2km radius (deterministic limit ~ 5 mins)
Snow falling at a point (~5 hours)Rain falling at a point (~18 hours)Gale force gusts at a point (~6 hours)Gale force gusts within a UK county (~24 hours)Rainfall >15mm in 3 hours somewhere in a UK county
(2 hours)Cyclonic surface pressure pattern at a point (~120
hours)Atmospheric front within 200km of a point (~60 hours)Day with maximum above 30C in London (~96 hours)
© Crown copyright Page 29
Benefits
Potential to provide a meaningful measure of what to expect from, and therefore what to put into, a forecast. Too many forecast elements are deterministic.
It is something that the public, other customers (and auditors!) could potentially relate to
The equivalent, from an idealised, reliable ensemble prediction system, would be the lead time at which the average probability, for hindcasts of observed past events, fell to 50% (?)
As always extreme events would be more difficult to represent (though hindcasts from re-analyses are becoming increasingly tractable)
Facility also to measure forecast improvements, compare systems, assess forecaster performance
© Crown copyright Page 30
Summary
The Met Office makes extensive use of ECMWF products for forecasts from T+48 to T+240, and is increasingly using the operational run as an input to short range forecasts
Verification indicates that the forecaster is adding value in many areas, in part using the poor man’s ensemble approach, though some weaknesses remain
Various enhancements to ECMWF web-based output have been suggested
Disappointingly, the customer base for probabilistic forecasts is currently limited
More guidance on what we can and cannot forecast deterministically is required
A new measure of ‘deterministic limit’ has been tentatively proposed
© Crown copyright Page 31
References
• Carroll, Meteorological Applications, 1997, for field modification description
• Carroll and Hewson, Weather and Forecasting, 2005 (out shortly) for ops centre practice and verification
© Crown copyright Page 32
Accreditation
WAFCWorld Area Forecast Centre