NOAA’s National Weather Service Tools To Access the NDFD

NOAA’s NOAA’s National Weather ServiceNational Weather ServiceTools To Access the NDFDTools To Access the NDFD

Arthur TaylorArthur TaylorRS Information Systems / Meteorological Development Laboratory RS Information Systems / Meteorological Development Laboratory

National Digital Forecast Database Technical WorkshopNational Digital Forecast Database Technical Workshop

August 13, 2003August 13, 2003

NDFD Technical Workshop 2003NDFD Technical Workshop 2003

NDFD: How do I use it?NDFD: How do I use it?

• Since NDFD uses the WMO’s GRIB2 format, which is relatively Since NDFD uses the WMO’s GRIB2 format, which is relatively new, NWS is providing a decoder library. The current versions new, NWS is providing a decoder library. The current versions of the library are located at:of the library are located at:– http://www.nws.noaa.gov/mdl/iwt/grib2/decoder.htmhttp://www.nws.noaa.gov/mdl/iwt/grib2/decoder.htm

– http://www.nws.noaa.gov/mdl/iwt/grib2/encoder.htmhttp://www.nws.noaa.gov/mdl/iwt/grib2/encoder.htm

• In addition, NWS is providing a driver “degrib” (aka In addition, NWS is providing a driver “degrib” (aka “NDFD_GRIB2Decoder”) for the library. Degrib’s primary “NDFD_GRIB2Decoder”) for the library. Degrib’s primary purposes are:purposes are:– Allow users to convert GRIB2 to other formats for use with various Allow users to convert GRIB2 to other formats for use with various

Geographic Information Systems (GIS), without need for a programmer.Geographic Information Systems (GIS), without need for a programmer.

– Provide an example of how to call the decoder librariesProvide an example of how to call the decoder libraries

– Enable some form of access to the data without requiring other packagesEnable some form of access to the data without requiring other packages• Generate ImagesGenerate Images

• Probe a give lat / lon location for all the relevant dataProbe a give lat / lon location for all the relevant data


Degrib: InstallationDegrib: Installation

•Degrib can be downloaded from:Degrib can be downloaded from:– http://www.nws.noaa.gov/mdl/NDFD_GRIB2Decoder/register.htmhttp://www.nws.noaa.gov/mdl/NDFD_GRIB2Decoder/register.htm

• On the download page, MS-Windows users should get “ndfd-On the download page, MS-Windows users should get “ndfd-demo.exe”, which contains:demo.exe”, which contains:

• An installation wizardAn installation wizard

• Compiled codeCompiled code

• Source code for the libraries, and the “degrib” and “tkdegrib” programs Source code for the libraries, and the “degrib” and “tkdegrib” programs (superImageGen and htmlmaker source code are not provided)(superImageGen and htmlmaker source code are not provided)

• DocumentationDocumentation

• Unix users should get “degrib-unix.tar.gz”, which contains:Unix users should get “degrib-unix.tar.gz”, which contains:• Source code for the libraries and the “degrib” and “tkdegrib” programsSource code for the libraries and the “degrib” and “tkdegrib” programs

• Documentation.Documentation.

• To compile the code you need: To compile the code you need: • FORTRAN (g77 should work) : for the decoder libraryFORTRAN (g77 should work) : for the decoder library

• C (gcc should work) : for the “drivers” and projection libraryC (gcc should work) : for the “drivers” and projection library

• Optional : Tcl/Tk (free from Optional : Tcl/Tk (free from http://scriptics.comhttp://scriptics.com) : for the Graphical User Interface) : for the Graphical User Interface


Degrib: Data downloadDegrib: Data download

Step 1: Download some data.Step 1: Download some data.• The GUI version of degrib, The GUI version of degrib,

“tkdegrib”, provides a “tkdegrib”, provides a convenient interface to get convenient interface to get NDFD either via ftp or http.NDFD either via ftp or http.– Highlight the desired sectorHighlight the desired sector

– Click on either “Download by ftp” Click on either “Download by ftp” or “Download by http”or “Download by http”

• Alternatively, you can use Alternatively, you can use various UNIX tools to get the various UNIX tools to get the data directly from:data directly from:– http://weather.noaa.gov/pub/SL.uhttp://weather.noaa.gov/pub/SL.u

s008001/ST.expr/DF.gr2/DC.ndfs008001/ST.expr/DF.gr2/DC.ndfd/d/

– ftp://tgftp.nws.noaa.gov/SL.us008ftp://tgftp.nws.noaa.gov/SL.us008001/ST.expr/DF.gr2/DC.ndfd/001/ST.expr/DF.gr2/DC.ndfd/


Degrib: ConvertDegrib: Convert

Currently degrib can convert Currently degrib can convert to the following formats:to the following formats:• ESRI Point .shp filesESRI Point .shp files

• ESRI Polygons .shp filesESRI Polygons .shp files

• GrADS lat / lon gridsGrADS lat / lon grids

• GrADS Projected gridsGrADS Projected grids

• .flt files for use with ESRI .flt files for use with ESRI Spatial AnalystSpatial Analyst

• .flt files for use with other .flt files for use with other programs that can read NxM programs that can read NxM 4 byte floats.4 byte floats.

In addition, degrib creates In addition, degrib creates a .txt file which contains the a .txt file which contains the meta data that was in the meta data that was in the GRIB2 message.GRIB2 message.


Degrib: Convert to .shp fileDegrib: Convert to .shp file

To convert to .shp format:To convert to .shp format:– Double click on the GRIB2 file Double click on the GRIB2 file

(top pane)(top pane)

– Select message to convert Select message to convert (middle pane)(middle pane)

– Choose an “output file name”Choose an “output file name”

– Click on “Generate .shp files”Click on “Generate .shp files”

• ““Polygon .shp” creates Polygon .shp” creates better images in ESRI and is better images in ESRI and is easier to manipulate, but it is easier to manipulate, but it is larger than the “Point .shp”larger than the “Point .shp”

• ““Include Missing Values” Include Missing Values” allows you to study the allows you to study the entire grid, or limit the result entire grid, or limit the result to where data existsto where data exists


CONUS MaxT in ArcViewCONUS MaxT in ArcView


Polygon vs Point, DC areaPolygon vs Point, DC area


Degrib: Convert to .flt fileDegrib: Convert to .flt file

To convert to .flt file (for use To convert to .flt file (for use with GrADS or ESRI Spatial with GrADS or ESRI Spatial Analyst):Analyst):

– Double click on the GRIB2 file Double click on the GRIB2 file (top pane)(top pane)

– Select message to convert Select message to convert (middle pane)(middle pane)

– Choose an “output file name”Choose an “output file name”– Click on “Generate .flt files”Click on “Generate .flt files”

• ““GrADS .ctl file”: creates a GrADS .ctl file”: creates a control file which GrADS can control file which GrADS can use to access the .flt fileuse to access the .flt file

• ““Interpolate”: bi-linear Interpolate”: bi-linear interpolates to a lat / lon interpolates to a lat / lon coverage grid.coverage grid.

• ““M.S.B. first”: creates the .flt M.S.B. first”: creates the .flt files in “Big Endian” format.files in “Big Endian” format.


CONUS MaxT in GrADSCONUS MaxT in GrADS


NDFD, GRIB2 & WeatherNDFD, GRIB2 & Weather

To encode Weather (Wx), NDFD disseminates “ugly To encode Weather (Wx), NDFD disseminates “ugly strings”.strings”.• Advantage: A flexible format that can encode a description of Advantage: A flexible format that can encode a description of

weather such as “Chance of thunderstorms and chance of weather such as “Chance of thunderstorms and chance of heavy rain showers”heavy rain showers”

Difficulty: An “ugly string” is not a number, so it is Difficulty: An “ugly string” is not a number, so it is challenging to store in GRIB2.challenging to store in GRIB2.• Solution: Use GRIB2 section 2 to provide an ASCII look up table, Solution: Use GRIB2 section 2 to provide an ASCII look up table,

and store the numeric values in the regular GRIB2and store the numeric values in the regular GRIB2– Note: each weather grid, for each forecast projection, has a different Note: each weather grid, for each forecast projection, has a different

ASCII look up table, so without section 2 it has no meaning.ASCII look up table, so without section 2 it has no meaning.

• Some Questions:Some Questions:– What does “Chc:T:<NoInten>:<NoVis>:^Chc:RW:+:<NoVis>:” mean, and What does “Chc:T:<NoInten>:<NoVis>:^Chc:RW:+:<NoVis>:” mean, and

how, for example, is an Emergency Manager supposed to know that?how, for example, is an Emergency Manager supposed to know that?– What can be done for the .flt file? What can be done for the .flt file?


Weather & .shp filesWeather & .shp files

Degrib looks up the value, and parses it Degrib looks up the value, and parses it when it saves Wx to a .shp filewhen it saves Wx to a .shp file• Wx-Index: Ugly string index used in the Wx-Index: Ugly string index used in the

GRIB2 message.GRIB2 message.• Visibility: The lowest visibility in the 5 Visibility: The lowest visibility in the 5

“ugly string words” (in statute miles).“ugly string words” (in statute miles).• NDFD wxCode: An encoding of the ugly NDFD wxCode: An encoding of the ugly

string using the first wx-type, intensity, and string using the first wx-type, intensity, and coverage, and the second wx-type.coverage, and the second wx-type.

• Weather 1: English translation of the “ugly Weather 1: English translation of the “ugly word”word”

• Wx-Inten 1: A number code for the wx-type Wx-Inten 1: A number code for the wx-type and wx-intensityand wx-intensity

• Cover 1: A number code for the coverageCover 1: A number code for the coverage• Hazard 1: A number code for hazards.Hazard 1: A number code for hazards.

– There can be 5 hazards per word, there are 9 There can be 5 hazards per word, there are 9 types of hazards. So each hazard is given a types of hazards. So each hazard is given a digit, the digits are sorted. Result: “00045” = 45 digit, the digits are sorted. Result: “00045” = 45 for “Damaging Wind (4) and “Small Hail (5)”for “Damaging Wind (4) and “Small Hail (5)”


Weather Codes:Weather Codes:

0 No Weather

1-5 Smoke (no Intensity, very light, light, moderate, heavy)

6-10 Blowing Dust (no Intensity, very light, light, moderate, heavy)

11-15 Blowing Snow (no Intensity, very light, light, moderate, heavy)

16-20 Haze (no Intensity, very light, light, moderate, heavy)

21-25 Fog (no Intensity, very light, light, moderate, heavy)

26-30 Drizzle (no Intensity, very light, light, moderate, heavy)

31-35 Rain (no Intensity, very light, light, moderate, heavy)

36-40 Rain Showers (no Intensity, very light, light, moderate, heavy)

41-45 Hail (no Intensity, very light, light, moderate, heavy)

46-50 Frost (no Intensity, very light, light, moderate, heavy)

51-55 Freezing Drizzle (no Intensity, very light, light, moderate, heavy)

56-60 Freezing Rain (no Intensity, very light, light, moderate, heavy)

61-65 Ice Pellets (no Intensity, very light, light, moderate, heavy)

66-70 Snow (no Intensity, very light, light, moderate, heavy)

71-75 Snow Showers (no Intensity, very light, light, moderate, heavy)

76-80 Thunderstorms (no Intensity, very light, light, moderate, heavy)

0 No Coverage

1 Isolated

2 Scattered

3 Numerous

4 Widespread

5 Occasional

6 Slight Chance of

7 Chance of

8 Likely

9 Definite

10 Patchy

11 Areas of

0 No Hazards

1 Frequent Lightning

2 Gusty Winds

3 Heavy Rain

4 Damaging Wind

5 Small Hail

6 Large Hail

7 Outlying Areas

8 On Bridges and Overpasses

““Wx-Inten” TableWx-Inten” Table

““Hazards” TableHazards” Table

““Coverage” TableCoverage” Table


Example: Weather in ArcViewExample: Weather in ArcView

In “/degrib/ArcView/”, In “/degrib/ArcView/”, there is a weather.avl there is a weather.avl which was applied to which was applied to the “Wx-Inten 1” to the “Wx-Inten 1” to create this image.create this image.• Problem: Chance of Problem: Chance of

thunderstorms (T), thunderstorms (T), and severe T have the and severe T have the same color.same color.

• Intent: get people Intent: get people started.started.

• If you have a If you have a better .avl, and want better .avl, and want to share, let me know.to share, let me know.


Weather & .flt “-SimpleWx”Weather & .flt “-SimpleWx”0 <NoWx> 20 R/IP (Rain / Ice Pellets) Low 40 R/ZR (Rain / Freezing Rain) High

1 L (Drizzle) Low Probability 21 S/IP (Snow / Ice Pellets) Low 41 ZR- (light Freezing Rain) High

2 R- (light Rain) Low 22 R/S (Rain / Snow) High Probability 42 ZR (Freezing Rain) High

3 R (Rain) Low 23 RW/SW (Rain / Snow Showers) High 43 ZR+ (heavy Freezing Rain) High

4 R+ (heavy Rain) Low 24 R/IP (Rain / Ice Pellets) High 44 R/ZR (Rain / Freezing Rain) High

5 R/T (Rain & Thunder) Low 25 S/IP (Snow / Ice Pellets) High 45 ZR/IP (Freezing Rain / Ice Pellets) High

6 RW (Rain Showers) Low 26 IP- (light Ice) Low 46 SW (Snow Showers) Low Probability

7 RW/T (Rain Showers & Thunder) Low

27 IP (Ice) Low 47 S- (light Snow) Low

8 T (Thunderstorms) Low 28 IP+ (heavy Ice) Low 48 S (Snow) Low

9 L (Drizzle) High Probability 29 ZL (Freezing Drizzle) Low 49 S+ (heavy Snow) Low

10 R- (light Rain) High 30 R/ZL (Rain / Freezing Drizzle) Low 50 SW (Snow Showers) High Probability

11 R (Rain) High 31 ZR- (light Freezing Rain) Low 51 S- (light Snow) High

12 R+ (heavy Rain) High 32 ZR (Freezing Rain) Low 52 S (Snow) High

13 R/T (Rain & Thunder) High 33 ZR+ (heavy Freezing Rain) Low 53 S+ (heavy Snow) High

14 RW (Rain Showers) High 34 R/ZR (Rain / Freezing Rain) Low 54 Fog

15 RW/T (Rain Showers & Thunder) High

35 ZR/IP (Freezing Rain / Ice Pellets) Low

55 Haze

16 T (Thunderstorms) High 36 IP- (light Ice) High 56 Smoke

17 Severe Thunder storms 37 IP (Ice) High 57 Blowing Snow

18 R/S (Rain / Snow) Low Probability

38 IP+ (heavy Ice) High 58 Blowing Dust

19 RW/SW (Rain / Snow Showers) Low

39 ZL (Freezing Drizzle) High


Example: .flt “-SimpleWx”Example: .flt “-SimpleWx”


NDFD CONUS Map ProjectionNDFD CONUS Map Projection

For CONUS (continental U.S.), NDFD uses a Lambert Conformal For CONUS (continental U.S.), NDFD uses a Lambert Conformal Conic Map Projection:Conic Map Projection:• Tangent latitutude 25, orientation longitude -95, mesh latitude 25, mesh Tangent latitutude 25, orientation longitude -95, mesh latitude 25, mesh

size 5079.406 m.size 5079.406 m.

• In ESRI ArcMap that would be:In ESRI ArcMap that would be:– 1st Parallel = 21st Parallel = 2ndnd Parallel = 25 degrees north (tangent latitude) Parallel = 25 degrees north (tangent latitude)

– Lon of center of projection = -95 degrees (orientation longitude) Lon of center of projection = -95 degrees (orientation longitude)

– Lat of center of projection = 25 degrees (tangent latitude) Lat of center of projection = 25 degrees (tangent latitude)

– False easting = 0 m, False northing = 0 m, X Shift = 0, Y Shift = 0 False easting = 0 m, False northing = 0 m, X Shift = 0, Y Shift = 0

In addition, NDFD uses a spherical earth with radius 6371.2 kmIn addition, NDFD uses a spherical earth with radius 6371.2 km– Note: 6367.47 km is currently in the GRIB2 messages, but NDFD used the same Note: 6367.47 km is currently in the GRIB2 messages, but NDFD used the same

methods that NCEP used, which depended on the 6371.2 km sphere.methods that NCEP used, which depended on the 6371.2 km sphere.

To assist ESRI ArcView 3.x, degrib creates a .ave (ArcView Avenue To assist ESRI ArcView 3.x, degrib creates a .ave (ArcView Avenue script) to set the projection and radius correctly.script) to set the projection and radius correctly.


Un-projected vs ProjectedUn-projected vs Projected


When Earth Radii go Bad.When Earth Radii go Bad.

GOOD:GOOD:Point used 6371.2 kmPoint used 6371.2 km

Polygon used 6371.2 kmPolygon used 6371.2 km

BAD:BAD:Point used 6371.2 kmPoint used 6371.2 km

Polygon used 6367.47 kmPolygon used 6367.47 km


Degrib: Using superImageGenDegrib: Using superImageGen

To generate images using To generate images using superImageGensuperImageGen

– Select the “Download” tabSelect the “Download” tab

– Highlight the sector in questionHighlight the sector in question

– Download the dataDownload the data

– Click on “Generate Images”Click on “Generate Images”

• This results in the following:This results in the following:– Tkdegrib generates a set of Tkdegrib generates a set of

“.mosaic” files“.mosaic” files

– ““superImageGen” generates superImageGen” generates the corresponding .png imagesthe corresponding .png images

– htmlmaker generates a set of htmlmaker generates a set of web pages to browse the .png web pages to browse the .png filesfiles

– IE is called to view the web IE is called to view the web pages.pages.


CONUS MaxT using superImageGenCONUS MaxT using superImageGen


Degrib: Probe pointDegrib: Probe point

This is for people who know their lat/lon and just want This is for people who know their lat/lon and just want text output. To probe a point from a command prompt text output. To probe a point from a command prompt (not available yet in “tkdegrib”) :(not available yet in “tkdegrib”) :

– C:\ndfd\degrib1.2\bin\degrib maxt.bin –P –pnt 38.99,-77.03C:\ndfd\degrib1.2\bin\degrib maxt.bin –P –pnt 38.99,-77.03

– C:\ndfd\degrib1.2\bin\degrib maxt.bin –P –pntFile point.txtC:\ndfd\degrib1.2\bin\degrib maxt.bin –P –pntFile point.txt

• The first example probes just 1 point, while the second reads a The first example probes just 1 point, while the second reads a file which is of the form: Station ID, lat, lon per line. The second file which is of the form: Station ID, lat, lon per line. The second form allows for numerous stations.form allows for numerous stations.

• By default, probe chooses the “nearest” grid cell, but using the By default, probe chooses the “nearest” grid cell, but using the “-Interp” option, causes it to perform bi-linear interpolation.“-Interp” option, causes it to perform bi-linear interpolation.

Everything previously discussed can be done using Everything previously discussed can be done using degrib, except data download, so the degrib step can degrib, except data download, so the degrib step can be automated. See (./degrib1.2/docs/degrib.txt for help)be automated. See (./degrib1.2/docs/degrib.txt for help)


Probe point (MaxT, MinT, PoP)Probe point (MaxT, MinT, PoP)

element, unit, refTime, validTime, (38.993600,-element, unit, refTime, validTime, (38.993600,-77.022400)77.022400)

MaxT, [F], 200307212300, 200307230000, 86.882MaxT, [F], 200307212300, 200307230000, 86.882

MaxT, [F], 200307212300, 200307240000, 82.375MaxT, [F], 200307212300, 200307240000, 82.375

MaxT, [F], 200307212300, 200307250000, 85.233MaxT, [F], 200307212300, 200307250000, 85.233

MaxT, [F], 200307212300, 200307260000, 87.753MaxT, [F], 200307212300, 200307260000, 87.753

MaxT, [F], 200307212300, 200307270000, 89.395MaxT, [F], 200307212300, 200307270000, 89.395

MaxT, [F], 200307212300, 200307280000, 87.775MaxT, [F], 200307212300, 200307280000, 87.775

MaxT, [F], 200307212300, 200307290000, 86.800MaxT, [F], 200307212300, 200307290000, 86.800

element, unit, refTime, validTime, (38.993600,-element, unit, refTime, validTime, (38.993600,-77.022400)77.022400)

MinT, [F], 200307212300, 200307221200, 72.625MinT, [F], 200307212300, 200307221200, 72.625

MinT, [F], 200307212300, 200307231200, 73.165MinT, [F], 200307212300, 200307231200, 73.165

MinT, [F], 200307212300, 200307241200, 69.025MinT, [F], 200307212300, 200307241200, 69.025

MinT, [F], 200307212300, 200307251200, 65.965MinT, [F], 200307212300, 200307251200, 65.965

MinT, [F], 200307212300, 200307261200, 67.945MinT, [F], 200307212300, 200307261200, 67.945

MinT, [F], 200307212300, 200307271200, 69.745MinT, [F], 200307212300, 200307271200, 69.745

MinT, [F], 200307212300, 200307281200, 69.025MinT, [F], 200307212300, 200307281200, 69.025

element, unit, refTime, validTime, (38.993600,-element, unit, refTime, validTime, (38.993600,-77.022400)77.022400)PoP12, [%], 200307212300, 200307220000, 39.000PoP12, [%], 200307212300, 200307220000, 39.000PoP12, [%], 200307212300, 200307221200, 39.000PoP12, [%], 200307212300, 200307221200, 39.000PoP12, [%], 200307212300, 200307230000, 50.000PoP12, [%], 200307212300, 200307230000, 50.000PoP12, [%], 200307212300, 200307231200, 50.000PoP12, [%], 200307212300, 200307231200, 50.000PoP12, [%], 200307212300, 200307240000, 50.000PoP12, [%], 200307212300, 200307240000, 50.000PoP12, [%], 200307212300, 200307241200, 50.000PoP12, [%], 200307212300, 200307241200, 50.000PoP12, [%], 200307212300, 200307250000, 20.000PoP12, [%], 200307212300, 200307250000, 20.000PoP12, [%], 200307212300, 200307251200, 20.000PoP12, [%], 200307212300, 200307251200, 20.000PoP12, [%], 200307212300, 200307260000, 14.000PoP12, [%], 200307212300, 200307260000, 14.000PoP12, [%], 200307212300, 200307261200, 5.000PoP12, [%], 200307212300, 200307261200, 5.000PoP12, [%], 200307212300, 200307270000, 5.000PoP12, [%], 200307212300, 200307270000, 5.000PoP12, [%], 200307212300, 200307271200, 11.000PoP12, [%], 200307212300, 200307271200, 11.000PoP12, [%], 200307212300, 200307280000, 11.000PoP12, [%], 200307212300, 200307280000, 11.000PoP12, [%], 200307212300, 200307281200, 21.000PoP12, [%], 200307212300, 200307281200, 21.000PoP12, [%], 200307212300, 200307290000, 21.000PoP12, [%], 200307212300, 200307290000, 21.000


Degrib: Future?Degrib: Future?

Maintain the program by keeping up with NDFD as more Maintain the program by keeping up with NDFD as more variables / sectors are introduced.variables / sectors are introduced.

– In order to inform users of updates, degrib has an “announcement mailing In order to inform users of updates, degrib has an “announcement mailing list”, which currently has 119 people on it.list”, which currently has 119 people on it.

Improvements: Based on user feedback, current Improvements: Based on user feedback, current thoughts are:thoughts are:

– Create merged polygons. Instead of a polygon per cell, join similar Create merged polygons. Instead of a polygon per cell, join similar values together to form large polygons.values together to form large polygons.

– Create some form of graphical probe point capabilityCreate some form of graphical probe point capability– Create some form of un-compressed database so that a CGI callable Create some form of un-compressed database so that a CGI callable

probe program can be used to access the data quicklyprobe program can be used to access the data quickly– Add some way for UNIX users (without GrADS) to plot the imagesAdd some way for UNIX users (without GrADS) to plot the images– Add some way for users to zoom in on a mapAdd some way for users to zoom in on a map– Interface to more file formatsInterface to more file formats

Documents

NOAA’s National Weather Service Tools To Access the NDFD