Meteorology 415Meteorology 415
October 2010October 2010
Numerical GuidanceNumerical Guidance
Skill dependent on:Skill dependent on: Initial ConditionsInitial Conditions ResolutionResolution Model PhysicsModel Physics ParameterizationParameterization Computational PowerComputational Power Boundary valuesBoundary values
Statistical GuidanceStatistical Guidance
There are several techniques that may be used beside multiple There are several techniques that may be used beside multiple linear regression to generate statistical relationships between linear regression to generate statistical relationships between predictor and predictand. These includepredictor and predictand. These include
Kalman FiltersKalman Filters
Neural NetworksNeural Networks
Artificial IntelligenceArtificial Intelligence
Fuzzy Logic Fuzzy Logic
Remember that Poor Model Performance translates into Poor Remember that Poor Model Performance translates into Poor MOS GuidanceMOS Guidance
Numerical GuidanceNumerical Guidance
Perfect Prog TechniquePerfect Prog Technique
The development of a Perfect-Prog can The development of a Perfect-Prog can make use of the NCEP reanalysis make use of the NCEP reanalysis database which provides long and database which provides long and stable time series of atmospheric stable time series of atmospheric analyses, which are necessary for analyses, which are necessary for building reliable regression building reliable regression equations equations based solelybased solely on past on past observations.observations.
OutlineOutline
MOS BasicsMOS BasicsWhat is MOS? What is MOS? MOS PropertiesMOS PropertiesPredictand DefinitionsPredictand DefinitionsEquation DevelopmentEquation DevelopmentGuidance post-processingGuidance post-processingMOS IssuesMOS Issues
Future WorkFuture WorkNew Packages - UMOS (Canadian)New Packages - UMOS (Canadian)Gridded MOSGridded MOS
Model Output Statistics (MOS)Model Output Statistics (MOS)MOS relates observations of the MOS relates observations of the weather element to be predicted weather element to be predicted ((PREDICTANDSPREDICTANDS) to appropriate variables ) to appropriate variables ((PREDICTORSPREDICTORS) via a statistical method) via a statistical method
Predictors can include:Predictors can include: NWP model output interpolated to NWP model output interpolated to observing siteobserving site Prior observationsPrior observations Geoclimatic data – terrain, normals, Geoclimatic data – terrain, normals, lat/lon, etc.lat/lon, etc.Current statistical method: Multiple Current statistical method: Multiple Linear Regression (forward selection)Linear Regression (forward selection)
MOS PropertiesMOS Properties Mathematically simple, yet powerful Mathematically simple, yet powerful techniquetechnique Produces probability forecasts from a Produces probability forecasts from a single run of the underlying NWP model single run of the underlying NWP model outputoutput Can use other mathematical approaches Can use other mathematical approaches such as logistic regression or neural such as logistic regression or neural networksnetworks Can develop guidance for elements Can develop guidance for elements not directly forecast by models; e.g. not directly forecast by models; e.g. thunderstormsthunderstorms
MOS Guidance ProductionMOS Guidance Production1.1. Model runs on NCEP IBM mainframeModel runs on NCEP IBM mainframe
2.2. Predictors are collected from model Predictors are collected from model fields, “constants” files, etc.fields, “constants” files, etc.
3.3. Equations are evaluatedEquations are evaluated
4.4. Guidance is post-processedGuidance is post-processed Checks are made for meteorological and Checks are made for meteorological and
statistical consistencystatistical consistency Categorical forecasts are generatedCategorical forecasts are generated
5.5. Final products disseminated to the Final products disseminated to the worldworld
MOS GuidanceMOS Guidance GFS (MAV) 4 times daily GFS (MAV) 4 times daily (00,06,12,18Z)(00,06,12,18Z) GFS Ext. (MEX) once daily (00Z)GFS Ext. (MEX) once daily (00Z) Eta/NAM (MET) 2 times daily Eta/NAM (MET) 2 times daily (00,12Z)(00,12Z) Variety of formats: text bulletins, Variety of formats: text bulletins, GRIB and BUFR messages, GRIB and BUFR messages, graphics … graphics …
Samples of EachSamples of Each
GFS (MAV)GFS (MAV)
Samples of EachSamples of Each
GFS EXT (MEX)GFS EXT (MEX)
Samples of EachSamples of Each
NAM (MET)NAM (MET)
Statistical GuidanceStatistical Guidance
Most common approachMost common approach
Statistical ApproachStatistical Approach
Statistical GuidanceStatistical Guidance
Statistical GuidanceStatistical Guidance
Predictand StrategiesPredictand Strategies
Predictands always come from Predictands always come from meteorological data and a variety of meteorological data and a variety of sources:sources: Point observations (ASOS, AWOS, Co-Point observations (ASOS, AWOS, Co-op sites)op sites) Satellite data (e.g., SCP data)Satellite data (e.g., SCP data) Lightning data (NLDN)Lightning data (NLDN) Radar data (WSR-88D)Radar data (WSR-88D)It is very important to quality control It is very important to quality control predictands before performing a predictands before performing a regression analysisregression analysis……
Predictand StrategiesPredictand Strategies
1.(Quasi-)Continuous Predictands: best 1.(Quasi-)Continuous Predictands: best for variables with a relatively smooth for variables with a relatively smooth distributiondistribution
Temperature, dew point, wind (u and v Temperature, dew point, wind (u and v components, wind speed)components, wind speed)
Quasi-continuous because temperature, dew point Quasi-continuous because temperature, dew point available usually only to the nearest degree C, available usually only to the nearest degree C, wind direction to the nearest 10 degrees, wind wind direction to the nearest 10 degrees, wind speed to the nearest m/s.speed to the nearest m/s.
2. Categorical Predictands: 2. Categorical Predictands: observations are reported as categoriesobservations are reported as categories
Sky Cover (CLR, FEW, SCT, BKN, OVC)Sky Cover (CLR, FEW, SCT, BKN, OVC)
Predictand StrategiesPredictand Strategies3. “Transformed” Predictands: predictand values 3. “Transformed” Predictands: predictand values have been changed from their original valueshave been changed from their original values
Categorize (quasi-)continuous observations such as ceiling Categorize (quasi-)continuous observations such as ceiling heightheight
Binary predictands such as PoP (precip amount Binary predictands such as PoP (precip amount >> 0.01”) 0.01”) Non-numeric observations can also be categorized or Non-numeric observations can also be categorized or
“binned”, like obstruction to vision (FOG, HAZE, MIST, “binned”, like obstruction to vision (FOG, HAZE, MIST, Blowing, none)Blowing, none)
Operational requirements (e.g., average sky cover/P-type Operational requirements (e.g., average sky cover/P-type over a time period, or getting 24-h precip amounts from 6-h over a time period, or getting 24-h precip amounts from 6-h precip obs)precip obs)
4. Conditional Predictands: predictand is 4. Conditional Predictands: predictand is conditional upon another event occurringconditional upon another event occurring
PQPF: Conditional on PoPPQPF: Conditional on PoP PTYPE: Conditional on precipitation occurringPTYPE: Conditional on precipitation occurring
MOS PredictandsMOS PredictandsTemperatureTemperature
Spot temperature (every 3 h)Spot temperature (every 3 h) Spot dew point (every 3 h)Spot dew point (every 3 h) Daytime maximum temperature [0700 – 1900 LST] Daytime maximum temperature [0700 – 1900 LST]
(every 24 h)(every 24 h) Nighttime minimum temperature [1900 – 0800 LST] Nighttime minimum temperature [1900 – 0800 LST]
(every 24 h)(every 24 h)
WindWind U- and V- wind components (every 3 h)U- and V- wind components (every 3 h) Wind speed (every 3 h)Wind speed (every 3 h)
Sky CoverSky Cover Clear, few, scattered, broken, overcast [binary/MECE] Clear, few, scattered, broken, overcast [binary/MECE]
(every 3 h)(every 3 h)
MOS PredictandsMOS Predictands
PoP/QPFPoP/QPF PoP: accumulation of 0.01” of liquid-equivalent PoP: accumulation of 0.01” of liquid-equivalent
precipitation in a {6/12/24} h period [binary]precipitation in a {6/12/24} h period [binary] QPF: accumulation of QPF: accumulation of
{0.10”/0.25”/0.50”/1.00”/2.00”*} CONDITIONAL on {0.10”/0.25”/0.50”/1.00”/2.00”*} CONDITIONAL on accumulation of 0.01” [binary/conditional]accumulation of 0.01” [binary/conditional]
6 h and 12 h guidance every 6 h; 24 h guidance 6 h and 12 h guidance every 6 h; 24 h guidance every 12 hevery 12 h
2.00” category not available for 6 h guidance2.00” category not available for 6 h guidance
ThunderstormsThunderstorms 1+ lightning strike in gridbox (size varies) [binary]1+ lightning strike in gridbox (size varies) [binary]
SevereSevere 1+ severe weather report (define) in gridbox [binary]1+ severe weather report (define) in gridbox [binary]
MOS PredictandsMOS PredictandsCeiling HeightCeiling Height
CH < 200 ft, 200-400 ft, 500-900 ft, 1000-1900 CH < 200 ft, 200-400 ft, 500-900 ft, 1000-1900 ft, 2000-3000 ft, 3100-6500 ft, 6600-12000 ft, > ft, 2000-3000 ft, 3100-6500 ft, 6600-12000 ft, > 12000 ft [binary/MECE- Mutually Exclusive and 12000 ft [binary/MECE- Mutually Exclusive and Collectively ExhaustivCollectively Exhaustivee]]
VisibilityVisibility Visibility < ½ mile, < 1 mile, < 2 miles, < 3 miles, Visibility < ½ mile, < 1 mile, < 2 miles, < 3 miles,
<< 5 miles, 5 miles, << 6 miles [binary] 6 miles [binary]
Obstruction to VisionObstruction to Vision Observed fog (fog w/ vis < 5/8 mi), mist (fog w/ Observed fog (fog w/ vis < 5/8 mi), mist (fog w/
vis vis >> 5/8 mi), haze (includes smoke and dust), 5/8 mi), haze (includes smoke and dust), blowing phenomena, or none [binary]blowing phenomena, or none [binary]
MOS PredictandsMOS Predictands
Precipitation TypePrecipitation Type Pure snow (S); freezing rain/drizzle, ice pellets, Pure snow (S); freezing rain/drizzle, ice pellets,
or anything mixed with these (Z); pure or anything mixed with these (Z); pure rain/drizzle or rain mixed with snow (R)rain/drizzle or rain mixed with snow (R)
Conditional on precipitation occurringConditional on precipitation occurring
Precipitation Characteristics (PoPC)Precipitation Characteristics (PoPC) Observed drizzle, steady precip, or showery Observed drizzle, steady precip, or showery
precipprecip Conditional on precipitation occurringConditional on precipitation occurring
Precipitation Occurrence (PoPO)Precipitation Occurrence (PoPO) Observed precipitation Observed precipitation on the houron the hour – does NOT – does NOT
have to accumulatehave to accumulate
StratificationStratification
Goal: To achieve maximum homogeneity in Goal: To achieve maximum homogeneity in the developmental datasets, while keeping the developmental datasets, while keeping their size large enough for a stable their size large enough for a stable regressionregression
MOS equations are developed for two seasonal MOS equations are developed for two seasonal stratifications:stratifications: COOL SEASON: October 1 – March 31COOL SEASON: October 1 – March 31 WARM SEASON: April 1 – September 30WARM SEASON: April 1 – September 30**EXCEPT Thunderstorms 3 seasonsEXCEPT Thunderstorms 3 seasons
(Oct. 16 – Mar. 15, Mar. 16 – Jun. 30, July 1 – Oct. (Oct. 16 – Mar. 15, Mar. 16 – Jun. 30, July 1 – Oct. 15)15)
Pooling DataPooling Data Generally, this means Generally, this means REGIONALIZATION: collecting nearby REGIONALIZATION: collecting nearby stations with similar climatologiesstations with similar climatologies Particularly important for forecasting Particularly important for forecasting RARE EVENTS:RARE EVENTS:QPF Probability of 2”+ in 12 hoursQPF Probability of 2”+ in 12 hoursCeiling Height < 200 ft; Visibility < ½ mileCeiling Height < 200 ft; Visibility < ½ mile
Regionalization allows for guidance Regionalization allows for guidance to be produced at sites with poor, to be produced at sites with poor, unreliable, or non-existent observation unreliable, or non-existent observation systemssystems All MOS equations are regional except All MOS equations are regional except
temperature and windtemperature and wind
Example of RegionsExample of Regions
GFS MOS GFS MOS PoP/QPF PoP/QPF Region Map,Region Map,Cool SeasonCool Season
• Note that Note that each element each element has its own has its own regions, regions, which usually which usually differ by differ by seasonseason
TransformTransformPoint Binary PredictorPoint Binary Predictor
FCST: F24 MEAN RH PREDICTOR CUTOFF = 70%INTERPOLATE; STATION RH ≥ 70% , SET BINARY = 1;
BINARY = 0, OTHERWISE
96 86 89 94
87 73 76 90
76 60 69 92
64 54 68 93RH ≥70% ; BINARY AT KBHM = 1
•KBHM
(71%)
TransformTransformGrid Binary PredictorGrid Binary Predictor
FCST: F24 MEAN RH PREDICTOR CUTOFF = 70%WHERE RH ≥ 70% , SET GRIDPOINT VALUE = 1, OTHERWISE = 0
INTERPOLATE TO STATIONS
1 1 1 1
1 1 1 1
1 0 0 1
0 0 0 10 < VALUE AT KBHM < 1
•KBHM
(0.21)
Binary PredictandsBinary Predictands
If the predictand is BINARY, MOS If the predictand is BINARY, MOS equations yield estimates of event equations yield estimates of event PROBABILITIES…PROBABILITIES…MOS Probabilities are:MOS Probabilities are: Unbiased – the average of the probabilities over a Unbiased – the average of the probabilities over a
period of time equals the long-term relative frequency of period of time equals the long-term relative frequency of the eventthe event
Reliable – conditionally (“piece-wise”) unbiased over Reliable – conditionally (“piece-wise”) unbiased over the range of probabilitiesthe range of probabilities
Reflective of predictability of the event – range of Reflective of predictability of the event – range of probabilities narrows and approaches relative frequency probabilities narrows and approaches relative frequency of event as predictability decreases, for example, with of event as predictability decreases, for example, with increasing projections or with rare eventsincreasing projections or with rare events
Post-Processing MOS GuidancePost-Processing MOS Guidance
Meteorological consistencies – Meteorological consistencies – SOME checksSOME checks T T >> Td; min T Td; min T << T T << max T; dir = 0 if wind speed = 0 max T; dir = 0 if wind speed = 0 BUT no checks between PTYPE and T, between PoP BUT no checks between PTYPE and T, between PoP
and sky coverand sky cover
Statistical consistencies – again, SOME checksStatistical consistencies – again, SOME checks Conditional probabilities made unconditionalConditional probabilities made unconditional Truncation (no probabilities < 0, > 1)Truncation (no probabilities < 0, > 1) Normalization (for MECE elements like sky cover)Normalization (for MECE elements like sky cover) Monotonicity enforced (for elements like QPF)Monotonicity enforced (for elements like QPF) BUT temporal coherence is only partially checkedBUT temporal coherence is only partially checked
Generation of “best categories”Generation of “best categories”
Unconditional Probabilities from Unconditional Probabilities from ConditionalConditional
If event B is conditioned upon If event B is conditioned upon A occurring:A occurring: Prob(B|A)=Prob(B)/Prob(A) Prob(B|A)=Prob(B)/Prob(A) Prob(B) = Prob(A) × Prob(B|A) Prob(B) = Prob(A) × Prob(B|A)
Example: Example:
Let A = event of Let A = event of >> .01 in., .01 in., and B = event of and B = event of >> .25 in., then .25 in., then if:if:Prob (A) = .70, and Prob (A) = .70, and Prob (B|A) = .35, Prob (B|A) = .35,
U
A
B
then then Prob (B) = .70 × .35 = .245Prob (B) = .70 × .35 = .245
Truncating ProbabilitiesTruncating Probabilities
0 0 << Prob (A) Prob (A) << 1.0 1.0Applied to PoP’s and Applied to PoP’s and thunderstorm thunderstorm probabilitiesprobabilitiesIf Prob(A) < 0, ProbIf Prob(A) < 0, Probadjadj (A)=0 (A)=0 If Prob(A) > 1, ProbIf Prob(A) > 1, Probadjadj (A)=1.(A)=1.
Normalizing MECE ProbabilitiesNormalizing MECE ProbabilitiesSum of probabilities for Sum of probabilities for exclusive and exhaustive exclusive and exhaustive categories must equal categories must equal 1.01.0If Prob (A) < 0, then If Prob (A) < 0, then sum of Prob (B) and sum of Prob (B) and Prob (C) = D, and is > Prob (C) = D, and is > 1.0.1.0.Set: ProbSet: Probadjadj (A) = 0, (A) = 0,
ProbProbadjadj (B) = Prob (B) / (B) = Prob (B) /
D,D,ProbProbadjadj (C) = Prob (C) / (C) = Prob (C) /
DD
Monotonic Categorical Monotonic Categorical ProbabilitiesProbabilities
If event B is a subset of If event B is a subset of event A, then:event A, then: Prob (B) should be Prob (B) should be << Prob (A).Prob (A).Example: B is Example: B is >> 0.25 in; A 0.25 in; A is is >> 0.10 in 0.10 inThen, if Prob (B) > Prob Then, if Prob (B) > Prob (A)(A)set Probset Probadjadj (B) = Prob (A). (B) = Prob (A).Now, if event C is a subset Now, if event C is a subset of event B, e.g., C is of event B, e.g., C is >> 0.50 0.50 in, and if Prob (C) > Prob in, and if Prob (C) > Prob (B),(B),set Probset Probadjadj (C) = Prob (B) (C) = Prob (B)
Temporal Coherence of Temporal Coherence of ProbabilitiesProbabilities
Event A is Event A is >> 0.01 in. 0.01 in. occurring from 12Z-18Zoccurring from 12Z-18ZEvent B is Event B is >> 0.01 in. 0.01 in. occurring from 18Z-00Zoccurring from 18Z-00Z A A B is B is >> 0.01 in. 0.01 in. occurring from 12Z-00Zoccurring from 12Z-00ZThen P(AThen P(AB) = P(A) + B) = P(A) + P(B) – P(AP(B) – P(AB)B)
Thus, P(AThus, P(AB) should B) should be:be:<< P(A) + P(B) and P(A) + P(B) and>> maximum of P(A), maximum of P(A), P(B)P(B)
A BC
0
20
40
60
80
0.01" 0.10" 0.25" 0.50" 1.00" 2.00"PRECIPITATION AMOUNT EQUAL TO OR EXCEEDING
FORECAST
THRESHOLD
MOS Best Category SelectionMOS Best Category SelectionP
RO
BA
BIL
ITY
(%
)
THRESHOLD
EXCEEDED?
TO MOS GUIDANCE MESSAGES
An example with QPF…An example with QPF…
Other Possible Post-ProcessingOther Possible Post-Processing
Computing the Expected ValueComputing the Expected Value used for estimating precipitation amountused for estimating precipitation amount
Fitting probabilities with a distributionFitting probabilities with a distribution Weibull distribution used to estimate median or Weibull distribution used to estimate median or
other percentiles of precipitation amountother percentiles of precipitation amount
Reconciling meteorological Reconciling meteorological inconsistenciesinconsistencies
Not always straightforward or easy to doNot always straightforward or easy to do Inconsistencies are minimized somewhat by use Inconsistencies are minimized somewhat by use
of NWP model in development and application of of NWP model in development and application of forecast equationsforecast equations
MOS Weaknesses / IssuesMOS Weaknesses / Issues
MOS can have trouble with some local MOS can have trouble with some local effects (e.g., cold air damming along effects (e.g., cold air damming along Appalachians, and some other terrain-Appalachians, and some other terrain-induced phenomena)induced phenomena) MOS can have trouble if conditions are MOS can have trouble if conditions are highly unusual, and thus not sampled highly unusual, and thus not sampled adequately in the training sampleadequately in the training sample But, MOS can and has predicted record highs & But, MOS can and has predicted record highs &
lowslows
• MOS typically does not pick up on MOS typically does not pick up on mesoscale-forced featuresmesoscale-forced features
MOS Weaknesses / IssuesMOS Weaknesses / Issues Like the models, MOS has problems with Like the models, MOS has problems with QPF in the warm season (particularly QPF in the warm season (particularly convection near sea breeze fronts along the convection near sea breeze fronts along the Gulf and Atlantic coasts)Gulf and Atlantic coasts) Model changes can impact MOS skillModel changes can impact MOS skill MOS tends toward climatology at extended MOS tends toward climatology at extended projections – due to degraded model accuracy projections – due to degraded model accuracy CHECK THE MODEL…MOS will correct CHECK THE MODEL…MOS will correct many systematic biases, but will not “fix” a many systematic biases, but will not “fix” a bad forecast. GIGO (garbage in, garbage bad forecast. GIGO (garbage in, garbage out).out).
Why do we need Gridded MOS?Why do we need Gridded MOS?
Because forecasters Because forecasters have tohave toproduce products produce products like this for the like this for the NDFD…NDFD…
Traditional MOS GraphicsTraditional MOS Graphics
This is This is better, but better, but still lacks still lacks most of most of the detail the detail in the in the Western Western U.S.U.S.
ObjectivesObjectives
Produce MOS guidance on high-Produce MOS guidance on high-resolution grid (2.5 to 5 km resolution grid (2.5 to 5 km spacing)spacing) Generate guidance with sufficient Generate guidance with sufficient detail for forecast initialization at detail for forecast initialization at WFOsWFOs Generate guidance with a level Generate guidance with a level of accuracy comparable to that of of accuracy comparable to that of the station-oriented guidancethe station-oriented guidance
BCDG AnalysisBCDG Analysis
Method of successive correctionsMethod of successive corrections Land/water gridpoints treated Land/water gridpoints treated differentlydifferently Elevation (“lapse rate”) Elevation (“lapse rate”) adjustmentadjustment
MOS Max Temperature ForecastMOS Max Temperature Forecast
NDFD Max Temperature ForecastNDFD Max Temperature Forecast
Statistical GuidanceStatistical Guidance UMOS – Updateable MOSUMOS – Updateable MOS
Updating concept As described by Ross (1992), a UMOS system is intended to facilitate the rapid and frequent updating of a large number
of MOS equations from a linear statistical model, either MLR or MDA. Both of these techniques use the sums-of-squares-and-cross-products matrix (SSCP), or components of it. The
idea of the updating is to do part of the necessary recalculation of coefficients in near–real time by updating the SSCP matrix, and storing the data in that form rather than as
raw observations.
Weather and ForecastingArticle: pp. 206–222 | Abstract | PDF (560K)The Canadian Updateable Model Output Statistics (UMOS) System: Design and Development TestsLaurence J. Wilson and Marcel ValléeMeteorological Service of Canada, Dorval, Quebec, Canada(Manuscript received April 19, 2001, in final form October 12, 2001)
Statistical GuidanceStatistical Guidance
UMOS – Updateable MOSUMOS – Updateable MOS
Future of Gridded MOSFuture of Gridded MOS
Evaluation (objective & Evaluation (objective & subjective)subjective) Expansion (area & elements)Expansion (area & elements) Improvement –make Improvement –make UpdateableUpdateable Use of remote-sensing Use of remote-sensing observationsobservations