Slide 1

AMRC Meteorological Data Distribution System Part II: Data Services

Antarctic Meteorological Research Center (AMRC), Space Science and Engineering Center (SSEC), University of Wisconsin-Madison, Madison, WI

Zach Uttech, Matthew Lazzara, K. Elena Willmot and Linda M Keller

1Presentation OutlineOverview of AMRCGoals of AMRC and efforts to improve efficiency of data distribution

New AMRC WebsiteImproved organization

User friendly

Current Data Services

Future Data Services

Summary

2

Overview of AMRCAMRC aims to provide real-time and archived data and observations to the publicMultiple data sourcesObservational dataNWP data

Old methods of data distributionFTP (File Transfer Protocol) site Old website

A new method of data distributionDeveloping a new website portalImproved organizationIncreased efficiencyAnd more

Up to now, our data distribution was accomplished through our FTP (File Transfer Protocol) website and our old AMRC website Because of the quantity and variety of data that we provide, it became necessary to develop a new website with improved organization, and increased efficiency for users 3New AMRC Website

Commonly used informationAWS Network tab

Data & Imagery tab

Recent news posted

4AWS Network TabDynamic (click-able) AWS MapAMRC project distributes data for ~63 AWS stations

Serves as an accessible central location for archived AWS data

Sidebar Contains supporting AWS information

5AWS Network TabDynamic (click-able) AWS Map

How to select an individual AWS

Beneficial feature

Simultaneous access to data of selected AWS and the dynamic map

Zooming and panning capabilities

To select an individual AWS, click a region or an individual dot, this highlights the desired station and the selected stations name, picture, and metadata are displayed underneath the dynamic map

allows users to navigate from one station to another with ease6AWS Network TabAWS Ferrell Station Selection Example

General layout

Metdata data loads next to picture

Additional data will be organized into tables

Compress region and station lists into pull-down tabs

7AWS Network TabSidebarWhat makes up an AWS?AWS diagramAWS MapsData Books/Field Reports

WI AWS 2B weather station is designed to work for several years unattended.

Annual data book filled with our monthly summary data, field reports, and lists with the status of our stations during the given year

8Data & Imagery Tab

AMRCs Archived Data

(FTP site and more)The thumbnail organizational layout of this page is a substantial improvement over the old AMRC website, which was simply an aggregation of text bullet points

The content within these categories is dynamically displayed and easily configurable, to keep the maintenance of the site to a minimum9Satellite Imagery and Composite Thumbnail

Constructed from geostationary and polar orbiting satellitesExamplesIR, WV, pseudo color, hourly animations

Data & Imagery Tab

IRWV

10Data & Imagery TabModels and Forecasts Thumbnail

Computer model data (ECMWF and GFS) overlayed with IR and WV composites (new!)

Mid-level data heights, RH, and VV

Surface data isobars, isotherms, streamlines

500mbHeights

MSLP11Data & Imagery TabData Fusion ThumbnailComposite AnimationsSatellite derived motion vectorsIce concentrationsControl speed, zoom, and refresh image

Additional compositesSST with IRAqua and Terra motion vectors

Satellite Derived Motion Vectors

Ice ConcentrationsThe different colored wind vectors signify wind speeds at different levels in the atmosphere

12

Data & Imagery TabAntarctic Surface ObservationsCurrent and archived data

Upper-Air ObservationsRecent and archivedRadiosonde data, Skew-t diagrams, and hodographs

Aircraft ObservationsCurrent meteorological aircraft reports

Satellite NavigationsAmerican and international polar orbiting satellite tracks

USAP StationsHistorical surface observation databaseSouth Pole, McMurdo, and Palmer

AMRCs Archived DataFtp site and more

13Current Data Services5 Data ServicesFTP serverStores archival data with minimal real-time datasetsWont be eliminated

Rsync ServiceUpdates requested data automatically; saves user timer

Antarctic-IDD (Internet Data Distribution) Local Data ManagerData sharing network based on Unidatas LDM softwareFocus on real-time data

Abstract Data Distribution Environment (ADDE)Contains both real-time and archived data

E-mail distributionEverything we have can be distributed via email

Goal provide self-serviceable data to reduce the effort to fill data requests!

Still plan to have the FTP, so those you like to use will still be able to.14Antarctic-IDD (Internet Data Distribution)

Columbia Scientific Balloon Facility (CSBF) and Alfred Wegener Institute (AWI).15ADDE (Abstract Data Distribution Environment)AMRC.SSEC.WISC.EDUAWS.SSEC.WISC.EDUADDE Group AWSAll Real-time datasetsExamplesHourly compositesAWSModel outputSatellite windsMoreADDE Group ARCHIVEAll ADDE ready archive data

ADDE Group AMRCAll Real-time datasetsExamplesThree hourly compositesAWSModel outputMoreBecoming obsoleteADDE Group ARCHIVEWill be discontinued

Future Data ServicesRAMADDA (Repository for Archiving, Managing and Accessing Diverse Data)

Hierarchical collection of folders and entries (accessed by public via URL)

Primary advantages Store any data type Make data easily accessible to users

Built by NSF funded Unidata Program

Besides reconstructing the AMRC website to make data more accessible, efforts have also included experimenting with RAMADDA

17Future Data ServicesRAMADDA (Repository for Archiving, Managing and Accessing Diverse Data)

Testing capabilities as an experimental effortEasy to load and manage new/old data entries

Long-term goalMost of AMRCs archived data ingested into RAMADDA

18Future Data ServicesRAMADDA

LayoutAdd descriptions about a data entry and the time range it coversData type examplesxls file Mt Flem and Capte Hr file Pegasustxt file FTPMcIDAS Area FileURLsView TabAllows for instant visualization of dataAnimations and McIDAS area files

Graph ViewThis page on the right shows the general layout of a RAMADDA account homepage

Graph view which provides a more visual representation of the data file organization hierarchy 19GIS (Geographic Information Systems) software and extensions used for AMRC products

ArcGIS

PostGIS

PostgreSQL

ZigGIS

QuantumGIS

Future Data Services

Other data services we are beginning to use for AMRC products

Schematic - how different layers of attributes in a GIS are geographically referenced, which can be in a raster (grid cell) or vector (point, lines, or polygon) form20Definitions for GIS Software and ExtensionsGIS computer system capable of capturing, storing, analyzing, and displaying georeferenced information

ArcGIS suite of GIS software products produced by the Environmental Systems Research Inst (ESRI)

QuantumGIS (QGIS) and ArcGIS interact with PostGIS-enabled databases for data management, visualization, and cartographic mapping

ZigGIS provides access specifically to PostGIS databases (extension)

Future Data ServicesArcGIS ZigGIS PostGISArcGIS requires an extension to connect with PostGIS; ArcGIS has excellent cartographic tools for mapping

QuantumGIS provides a superior interface for managing and editing spatial data in PostGIS; connection between ArcGIS and PostGIS 21Definitions for GIS Software and Extensions

PostgreSQL powerful, open-source, relational database systemAccessed by users indirectly through an external application rather than by application programming stored within the database (back-end database)

PostGIS spatially enables the PostgreSQL serverOpen source add-on, gives PostgreSQL the ability to store and manipulate spatial data used in GIS and Mapping

Future Data ServicesOpen source developed as a public collaboration and freely available. A back-end database is a database that is accessed by users indirectly through an external application rather than by application programming stored within the database itselfDBMS data and relationships among the data is stored in form of tables.

22Future Data ServicesApplications of the GIS software

We use PostgreSQL, PostGIS, and ZigGIS to

Add tables which contain data that is not spatially referenced

Create a database that updates the AWS network map in real-time

23Future Data ServicesApplications of the GIS software

We use ArcGIS and qGIS in day to day work

Beginning to manage information about the ~120 (AMRC + International) active Antarctic AWS using PostGIS

Using the zigGIS (extension to ArcGIS) to

Generate maps of AWS directly from the database

Map any station-related (non-spatial) data through relation joins in the database

Relation join - merging twoattribute tables using their common item24Future Data ServicesApplications of the GIS software

Server applications such as Geoserver and Mapserver can generate web mapping/feature services from the same PostGIS-enabled database

Allows us toBuild near real-time web mapping interfaces that access the most current data automatically (from the PostGIS database)

Develop a PHP 5/Apache web interface to allow group editing/management of data about the AWS network (in the PostGIS database)

25SummaryCurrent data servicesAntarctic-IDDContinue to expand the number of participants in the networkADDEAMRC group has more capabilities than AWS serverAWS group becoming obsolete

Future data servicesNew AMRC websiteDynamic AWS mapSatellite composite and meteorological variable (NWP) overlays (new)Improved organization over old websiteGIS software/extensionsUpdate AWS network in real-timeAllow for group editing

AcknowledgementsThanks to the Office of Polar Programs at the National Science Foundation grant No ANT-0838834 for its support of the AMRC project The authors also wish to thank SSEC and AMRC members for their contributions to the new AMRC websiteReferencesLazzara, MA, G Langbauer, KW Manning, R Redinger, MW Seefeldt, R Vehorn, and T Yoksas, 2006 Antarctic Internet Data Distribution (Antarctic-IDD) System 22nd International Conference on Interactive Information Processing Systems for Meteorology, Oceanography, and Hydrology Atlanta, GA American Meteorological Society

Seefeldt, MW, T Yoksas, and MA Lazzara, 2009a Real-time and archived Antarctic meteorological data via a synergy of interactive processing tools Antarctic Meteorological Observing Modeling and Forecasting Workshop 14-15 July 2009 Charleston, SC

Seefeldt, MW, T Yoksas, and MA Lazzara, 2009b The distribution, retrieval, and visualization of real-time Antarctic numerical weather prediction, satellite and observational data 10th Conference on Polar Meteorology and Oceanography, May 18-21, 2009 Madison, WI

Yoksas, T, J Caron, J McWhirter, M Richards, M Seefeldt, KW Manning, JG Powers, D Steinhoff, and MA Lazzara, 2009 Using Unidatas RAMADDA and THREDDS Services to Support Two Disparate Projects AGU 2009 Joint Assembly The Meeting of the Americas, May 24-27, 2009, Toronto, ON, Canada

www.postgis.orgwww.postgresql.orgwww.qgis.orgwww.unidataucar.edu/software/ramadda

28