NDBC Web Data Guide

NDBC Web Data Guide

National Data Buoy Center (NDBC)

Stennis Space Center, Mississippi 39529-6000

October 2015

NDBC Web Data Guide

Table of Contents 1. General Guidance .................................................................................................................................. 1

2. Realtime Data ........................................................................................................................................ 1

3. Latest Observations File........................................................................................................................ 2

4. Historical Data & Climatic Summaries................................................................................................. 2

5. Active Station List ................................................................................................................................ 3

6. Metadata in XML .................................................................................................................................. 3

7. NDBC Data Directory ........................................................................................................................... 4

8. DIF/SOS Services ................................................................................................................................. 4

9. DODS Services ..................................................................................................................................... 5

10. Tropical Atmosphere Ocean (TAO) Data Download Page ............................................................... 6

11. High Frequency (HF) Radar Data Access ......................................................................................... 6

12. DART Data Access ........................................................................................................................... 6

13. Observation Widget .......................................................................................................................... 7

14. BuoyCAM Images ............................................................................................................................ 8

15. Observations by Latitude/Longitude ................................................................................................. 9

16. Ship Observations ............................................................................................................................. 9

17. Historical Station – User Defined Conditional Search ..................................................................... 9

18. Feedback ........................................................................................................................................... 9

Appendix A – File Extension Mapping to Data Types ............................................................................. A-1

Appendix B – Subdirectories of www.ndbc.noaa.gov/data ....................................................................... B-1

NDBC Web Data Guide

Version 1.0 1

1. General Guidance

This document provides a high level over view of data available on the NDBC web site

(http://www.ndbc.noaa.gov/) to support data consumers. The goal is to assist the user in

determining the best option for obtaining their required data while minimizing the potential

load on the NDBC web farm. We recommend that you review the entire content of this

document prior to deciding on your implementation strategy, because NDBC supports

numerous methods to obtain data from our web site. We highly discourage you from

developing unique processes (i.e. screen scrape mobile page) to obtain our data via other

methods since future development may have a negative impact on your process. We also ask

that you consider the frequency of the data updates and limit your data request to conserve

our bandwidth and ability to support other users. If after reviewing this document you still

have concerns in regards to the best method please contact us via the web master e-mail

webmaster.ndbc@noaa.gov and we can provide additional guidance.

NDBC currently has FTP services that are available; however, our future plans are to

discontinue these services, therefore, we will not address them in this document. We

strongly recommend that you avoid the use of FTP services when using the NDBC web site.

The majority of data posted to the NDBC web server are stored in ascii files that may be

downloaded via HTTP, wget, or Perl/LWP (among other utilities). HTTP is the preferred

protocol.

2. Realtime Data

The Realtime directory http://www.ndbc.noaa.gov/data/realtime2/ contains the current (last

45 days) data. The term Realtime refers to the version of the data. In general, Realtime data

are the data that have undergone automated quality control checks as they were received in

real time and released on the Global Telecommunications System (GTS). The files are named

station_id.datatype.

See appendix A for a listing of the data type extensions and the associated data.

As an example, the following files would apply to station 41002,

the standard meteorological data is in 41002.txt

the continuous winds data is in 41002.cwind

the spectral wave summary data is in 41002.spec

the raw spectral wave data is in 41002.data_spec

etc.

NDBC Web Data Guide

Version 1.0 2

In building file names all alphabetic characters must be in upper case. For instance the C-

MAN (land) station FPSN7 – Frying Pan Shoals, NC must be entered in uppercase, e.g.,

FPSN7.txt.

Not all of these data types are available for each station; however, each station has at least

one data type.

See Measurement Descriptions and Units for more information on the units of measure used

on the NDBC web site. Samples of each file type are available at the following URL:

http://www.ndbc.noaa.gov/mods.shtml

You can also navigate to this data by selecting a station page and then scrolling down to the

bottom and selecting the “Real Time Data” link.

3. Latest Observations File

The latest observation file is available at:

http://www.ndbc.noaa.gov/data/latest_obs/latest_obs.txt

This file contains essentially the same data elements as the standard meteorological data file,

however, instead of having the observations from a single station; this file has the most

recent observation (provided that the observation is less than two hours old) from all stations

hosted on the NDBC web site. Since this file has multiple stations it also contains the

position information (latitude and longitude) for each station. The file is relatively small, less

than 100KB, and is updated approximately every 5 minutes, so it would be a good source of

data if you are interested in meteorological observations from multiple stations.

4. Historical Data & Climatic Summaries

The historical files are available at:

http://www.ndbc.noaa.gov/station_history.php?station=42040

Where the 42040 represents the station that you are interested in.

You can also navigate to this page by selecting a station page and then scrolling down to the

bottom and selecting the “Historical Data & Climatic Summaries” link.

The returned page will have four main sections:

Quality Controlled data for the current year – by months

Historical Data – by calendar year

NDBC Web Data Guide

Version 1.0 3

Search historical meteorological data for observations that meet your threshold

conditions

Climatic summary table and plots – by parameter. For details about this information

please review the description available at:

http://www.ndbc.noaa.gov/climatedesc.shtml

5. Active Station List

The Active Station List is available at http://www.ndbc.noaa.gov/activestations.xml

This file provides metadata in regards to the current deployment for all active stations on

the NDBC web site. The metadata includes station ID, latitude, longitude, station name,

station owner, program to which the station belongs, and type of data reported as detailed

below:

met: indicates whether the station has reported meteorological data in the past eight

hours (y/n).

currents: indicates whether the station has reported water current data in the past

eight hours (y/n).

waterquality: indicates whether the station has reported ocean chemistry data in the

past eight hours (y/n).

dart: indicates whether the station has reported water column height/tsunami data in

the past 24 hours (y/n).

This file is refreshed every five minutes as needed. Note: The main activity that drives

changes are: a service visit, establishment of a new station, or changes in the type of data

received (i.e. sensor/station failure) therefore, minimal updates would be expected in a 24

hour period.

Note, the TAO entries do not include the data type attributes (met, currents, water quality and

dart) but do include a seq attribute for syncing access to the TAO web site. The TAO array is

the climate stations in the equatorial Pacific.

6. Metadata in XML

The metadata in XML is available at:

http://www.ndbc.noaa.gov/metadata/stationmetadata.xml

The supporting XML schema can be found at:

http://www.ndbc.noaa.gov/metadata/stationmetadata.xsd

NDBC Web Data Guide

Version 1.0 4

This file contains the historical metadata back to 2000 for all stations on the NDBC

website. Limited metadata is available for non-NDBC stations. The file is generated once

daily at midnight U.S. Central Time (05:00 UTC during daylight saving time or 06:00 UTC

during standard time).

Note: this file is fairly new and there were some inconsistencies in our older metadata, so this

file is not 100% accurate, however, it is the best representation of the station history from our

perspective.

7. NDBC Data Directory

The most important files from this directory have already been covered in depth in previous

sections; however, this section gives you a broad overview of all the files available under the

data directory available at: http://www.ndbc.noaa.gov/data/

See Appendix B for a listing of the subdirectories under the Data directory.

8. DIF/SOS Services

The NDBC Sensor Observation Service (SOS) is an implementation of the Open Geospatial

Consortium (OGC) SOS 1.0 specification that supports the Core Operations Profile. An

overview is available at: http://sdf.ndbc.noaa.gov/sos/. An interactive demonstration is

available at: http://sdf.ndbc.noaa.gov/. The actual service is available at:

http://sdf.ndbc.noaa.gov/sos/server.php . This service provides an API for accessing

observation data for the following measurements:

Air Temperature

Conductivity

Currents

Salinity

Sea Level Pressure

Water Level

Water Temperature

Waves

Winds

These data may be returned in the following formats:

comma-separated values (CSV)

tab-separated values (TSV)

KML/JSON (experimental)

OGC Sensor Web Enablement (SWE) 1.0 using the IOOS template

NDBC also supports a GML response, but its use is discouraged as it will be discontinued in

a future update. SOS is intended to support machine to machine interface, without human

NDBC Web Data Guide

Version 1.0 5

intervention. One of the limitations of the current NDBC implementation is that you can only

request one data type per request. However, you can make multiple calls to obtain all

observations. This lack of multiple observed properties and higher complexity queries are a

limitation to its current use. However, if you are interested in a single or limited data types

(i.e. water level/Tsunami) then SOS maybe a good fit.

9. DODS Services

The NDBC Distributed Oceanographic Data System (DODS) makes netCDF files available

to the science community and general public via our website. It uses the Open Source

Project for a Network Data Access Protocol (OPeNDAP) software for data providers to share

data with each other and the end users. This service is available at the following URL:

http://dods.ndbc.noaa.gov/

The following provides a high level overview of the directory structure/data types available

via the NDBC DODS server:

All NDBC Data

NDBC Data

adcp – Acoustic Doppler Current Profiler Data contains depth, direction and

speed

adcp2 –Acoustic Doppler Current Profiler Data with additional QC

information primarily from Gulf of Mexico oil rigs.

cwind – Continuous Winds data

dart – Deep-ocean Assessment and Reporting of Tsunamis data

eval – DO NOT USE - Unofficial data

mbcurr – Marsh-McBirney Current Measurements data

ocean – Oceanographic data

oceansites – see OceanSITES Data directory below.

pwind – Peak Winds data

stdmet – Standard Meteorological data

swden – Spectral Wave Density data with Spectral Wave Direction data

tao-adcp - see “NDBC TAO Data” directory below.

tao-ctd - see “NDBC TAO Data” directory below.

wlevel – Water Level data (tide) measured from a non-DART NDBC station

NDBC TAO Buoy Data – The subdirectories are named for each station, based on the

latitude and longitude. For example the station at equator and 110W is named

T0N110W, the station at 2N 110W is named T2N110W. The “T” in the first position

of the file name designates that it is a TAO site.

NDBC TAO CTD Data – contains the TAO CTD data collected on servicing cruises

organized by cruise id. The cruise id consist of a two letter ship id, a two digit fiscal

NDBC Web Data Guide

Version 1.0 6

year, and a sequence number for each cruise performed in support of TAO for that

year.

OceanSITES Data – provides access to OceanSITES data, including TAO. Note the

TAO data is the same data as available under the ‘NDBC TAO Data” directory

above.

10. Tropical Atmosphere Ocean (TAO) Data Download Page

NDBC recently modified the TAO Data Download page to make it more user friendly. The

new TAO Data Download is available at:

http://tao.ndbc.noaa.gov/tao/data_download/search_map.shtml

This data download page provides numerous options for the end user to select: stations, data

types, start dates, end dates, temporal resolution, file format, etc. This download process is

limited to only TAO data from the equatorial Pacific; you cannot get other data hosted on the

NDBC web site via this process.

11. High Frequency (HF) Radar Data Access

NDBC has the most recent 72 hours of HF radar data available via DODS/THREDDS. This

service is available at: http://sdf.ndbc.noaa.gov/thredds/catalog.html

In addition to the DODS server HF radar data is available from the main HF radar page at:

http://hfradar.ndbc.noaa.gov/. If you scroll down to the bottom of the page the last hyperlink

“Tabular Format” will provide observation data in a tab delimited file for the current area

displayed on the map. If you scroll to the bottom of the second page there is a link to

“Download results as text.”

12. DART Data Access

You can retrieve and display a plain text listing of NDBC's historical DART data that

matches the station ID, starting date, and ending date that you specify. Specify these by

modifying the arguments in this URL -

http://www.ndbc.noaa.gov/dart_data.php?station=46401&startmonth=&startday=&startyear=

&endmonth=&endday=&endyear=.

The arguments for the station ID and starting and ending dates are specified as follows:

station= station ID;

startmonth = starting month (numeric, 1-12);

startday = starting day (numeric, 1-31);

startyear = starting year (numeric);

endmonth = ending month (numeric, 1-12);

NDBC Web Data Guide

Version 1.0 7

endday = ending day (numeric, 1-31); and

endyear = ending year (numeric).

These arguments may be entered in any order, but all must be given valid values.

For example; to retrieve data from station 46401 for January 4, 2004 through January 10,

2004; use the following URL:

http://www.ndbc.noaa.gov/dart_data.php?station=46401&startmonth=1&startday=4&startye

ar=2004&endmonth=1&endday=10&endyear=2004

This will return all water column heights on the NDBC web site for station 46401 from

00:00:00 UTC on January 4, 2004 through 23:59:59 UTC on January 10, 2004, inclusive.

The output will display the parameters followed by the data. The data includes:

date/time of the observation - formatted as "YYYY MM DD hh mm ss" or "2004 01 04

01 30 00" for 01:30:00 UTC on January 4, 2004.

measurement Type - 1 = 15 minute measurement; 2 = 1 minute measurement; 3 = 15

second measurement.

water column HEIGHT - The data are recorded in pounds per square inch absolute

(PSIA), but are displayed in meters of water after applying a constant 670.0 mm of

water/PSIA conversion factor.

A sample format follows:

station 46401

startyear 2004

startmonth 01

startday 18

endyear 2004

endmonth 01

endday 22

YYYY MM DD hh mm ss T HEIGHT (meters)

2004 01 22 15 00 00 1 5526.870

2004 01 22 14 45 00 1 5526.883

2004 01 22 14 30 00 1 5526.896

2004 01 22 14 15 00 1 5526.907

2004 01 22 14 00 00 1 5526.915

2004 01 22 13 45 00 1 5526.917

2004 01 22 13 30 00 1 5526.914

13. Observation Widget

NDBC Web Data Guide

Version 1.0 8

If you host a web page and are interested in providing the most recent observation from a

limited number of NDBC hosted stations, NDBC provides an Observation Widget that

allows you to easily integrate NDBC observations into your web page.

Additional Information is available at: http://www.ndbc.noaa.gov/widgets/

Note: This technique only works for web pages in which the user has the ability to paste

HTML content.

14. BuoyCAM Images

NDBC operates BuoyCAMs at several stations. These BuoyCAMs typically take photos only

during daylight hours.

To view the most recent BuoyCAM image from an NDBC station, use this URL:

http://www.ndbc.noaa.gov/buoycam.php?station=xxxxx

where xxxxx is the desired station ID. To see which stations are currently reporting

BuoyCAM images, check the BuoyCAMs map.

If the server encounters any difficulties in processing your request, you will receive one of

these error messages:

No station specified

Modify your URL to use the station parameter to specify a valid station with a

BuoyCAM (station=xxxxx where xxxxx is the station ID). Look at the BuoyCAMs

map to see which stations have a BuoyCAM.

Invalid station specified

The station on the URL is not a valid station. Review the BuoyCAMs map to see

which stations have a BuoyCAM.

This station has no BuoyCAM

The station on the URL is valid but has no BuoyCAM installed. Look at the

BuoyCAMs map to see which stations have a BuoyCAM.

BuoyCAM photo for this station is older than 16 hours

The BuoyCAM on the specified station has not reported in the past 16 hours, hence

there is no image to display.

Unable to access BuoyCAMs at this time

There is an issue preventing the BuoyCAM process from functioning properly.

Recommend waiting at least 30 minutes and trying again, if the problem persist

NDBC Web Data Guide

Version 1.0 9

contact the NDBC webmaster with the URL used and the date/time the error was

received.

15. Observations by Latitude/Longitude

On the left navigation bar, there is an option labeled as Obs Search. This option will allow

you to perform a Radial Search or a Box Search. The Radial Search will return all

observations within a user specified distance from a user specified latitude and longitude.

The Box search will return all observations within a user defined box, based on the latitudes

and longitudes entered by the user. This feature is available at the following URL:

http://www.ndbc.noaa.gov/os.shtml

Currently this option only provides the response to the screen and is not available for

downloading.

16. Ship Observations

On the left navigation bar, there is an option labeled as Ship Obs Report, this option will

allow you to see all ship observations for the current hour back through the last twelve hours.

This feature is available at the following URL: http://www.ndbc.noaa.gov/ship_obs.php

17. Historical Station – User Defined Conditional Search

This feature allows the user to select a station and then perform a historical (past years)

search for a user defined condition. The search is limited to a specific station and a specific

year, but the condition is user defined based on the parameters measured at the station. For

instance a user could determine when in 2013 station 46047 experienced wave heights

greater than 5 meters. This feature is available at the following URL:

http://www.ndbc.noaa.gov/histsearch.php

18. Feedback

If after reviewing this document you have any comments or suggestions to improve this

document please contact us via the web master e-mail webmaster.ndbc@noaa.gov and use

the subject “NDBC Web Data Guide.”

NDBC Web Data Guide

Version 1.0 A-1

Appendix A – File Extension Mapping to Data Types

File Extension Data Type

txt standard meteorological data drift meteorological data from drifting buoys and

limited moored buoy data mainly from

international partners cwind continuous winds data (10 minute average)

spec spectral wave summaries

data_spec raw spectral wave data swdir spectral wave data (alpha1)

Swdir2 spectral wave data (alpha2)

Swr1 spectral wave data (r1) Swr2 spectral wave data (r2) adcp Acoustic Doppler Current Profiler

ocean oceanographic data

tide tide data

srad solar radiation data

dart water column height

supl supplemental measurements data rain hourly rain data

NDBC Web Data Guide

Version 1.0 B-1

Appendix B – Subdirectories of www.ndbc.noaa.gov/data

Directory Name Contents Recommended

Use

5day2 Observations for the last 5 days from all active

stations. The file extensions are the same as

shown in Appendix A.

DAB Forecast The NWS forecast for the region that includes

station. Supports the Dial A Buoy application.

Do Not Use

Forecasts The NWS forecast for the region that includes

station. Supports the web site.

Recommend the NWS Point Forecast.

Do Not Use

adcp Acoustic Doppler Current Profile by station for

the current calendar year by month with only

high level range checks applied.

adcp2 Acoustic Doppler Current Profile by station for

the current calendar year by month with full

RDI quality checks applied.

adcp raw data Acoustic Doppler Current Profile, raw binary

files by year by station.

climatic Climatic Summaries for each station. Dates

vary by station; however, none has been

updated since 2009.

cwind Continuous Winds by station for the current

calendar year by month.

dart Water level height by station for the current

calendar year by month.

dart deployment history 2009 data for station 43412 and 43413 Do Not Use

dart deployment realtime Water level height by station each file may

contain multiple years

derived2 Derived meteorological data (i.e. chill, heat,

icing, 10 and 20 meter wind speeds) by station

for the past 45 days.

glider Contains 3 subdirectories active, historical,

dives. Active – contain the GTS formatted

KKYY message for active stations. Historical –

provides the historical GTS formatted KKYY

message by station id. Dives – provides the kmz

file for active gliders.

gzip Provides GZIP software. Do Not Use

historical Provides historical data (previous calendar

years) for all directory types listed in this table.

NDBC Web Data Guide

Version 1.0 B-2

Directory Name Contents Recommended

Use

hourly2 Provides ADCP, continuous winds,

oceanographic data, spectral wave summary,

solar radiation, supplemental measurements,

and standard meteorological measurements for

hours 0-23 (UTC) for the current date. Note:

directories for hours not reached will contain

observation data for the previous day. Each file

contains observations from all stations.

l adcp Provides non-BOEM (oil rig) ADCP data by

station for the prior month. Updates around

mid month for the previous month.

l adcp2 Provides BOEM (oil rig) ADCP data by station

for the prior month. Updates around mid month

for the previous month.

l cwind Provides continuous wind data by station for

the prior month. Updates around mid month for

the previous month.

l dart Provides water height data by station for the

prior month. Updates around mid month for the

previous month.

l mbcur No content is available

l ocean Provides ocean data by station for the prior

month. Updates around mid month for the

previous month.

l srad Provides solar radiation data by station for the

prior month. Updates around mid month for the

previous month.

l stdmet Provides standard meteorological data by

station for the prior month. Updates around

mid month for the previous month.

l supl Provides supplemental data by station for the

prior month. Updates around mid month for the

previous month.

l swden Provides spectral wave density by station for

the prior month. Updates around mid month for

the previous month.

l swdir Provides spectral wave data (alpha1) by station

for the prior month. Updates around mid month

for the previous month.

l swdir2 Provides spectral wave data (alpha2) by station

for the prior month. Updates around mid month

for the previous month.

NDBC Web Data Guide

Version 1.0 B-3

Directory Name Contents Recommended

Use

l swr1 Provides spectral wave data (r1) by station for

the prior month. Updates around mid month for

the previous month.

l swr2 Provides spectral wave data (r2) by station for

the prior month. Updates around mid month for

the previous month.

l wlevel Provides water level (tide) by station for the

prior month. Excludes DART/Tsunami

stations. Updates around mid month for the

previous month.

latest obs See section 3 of this document.

In addition there are files for each station with

the latest observation in both a text and an RSS

format.

Good source of

latest

meteorological

observation for

multiple stations

mbcurr No content is available

ocean Ocean data by station for the current calendar

year by month.

rain Hourly rain data by station for the current

calendar year by month.

rain10 10 minute rain data by station for the current

calendar year by month.

rain24 Daily rain data by station for the current

calendar year by month.

realtime2 See section 2 of this document Good source of

observation data

for the last 45

days

sar2 Search and rescue support data by station

seakeepers Do Not Use

ship_obs Old ship observations – no longer used Do Not Use

special Special report on Station 42040 during

Hurricane Ivan in 2004

Do Not Use

srad Solar radiation data by station for the current

calendar year by month.

stations Contains several files that provide metadata on

buoy and CMAN site and sensor elevation and

station owner and limited metadata. The files

are updated daily.

Metadata

stdmet Standard meteorological data by station for the

current calendar year by month.

supl Supplemental data by station for the current

calendar year by month.

NDBC Web Data Guide

Version 1.0 B-4

Directory Name Contents Recommended

Use

swden Spectral wave density data by station for the

current calendar year by month.

swdir Spectral wave data (alpha1) by station for the

current calendar year by month.

swdir2 Spectral wave data (alpha2) by station for the

current calendar year by month.

swr1 Spectral wave data (r1) by station for the

current calendar year by month.

swr2 Spectral wave data (r2) by station for the

current calendar year by month.

wlevel Water level (tide) data by station for the current

calendar year by month. Excludes

DART/Tsunami stations.