Data Discovery Tools and ServicesPart A

A Panel Presentation:

Pat Liggett, Ron Weaver, Sean Hardman, Ben Holt

(with help from Dave Gallaher, NSIDC)

October 15, 16 2008

Burbank Airport Marriott

User Case Scenario: Scientific and comprehensive data view/access/news for polar regions. Starting point was NSIDC ‘Arctic Sea Ice News and Analysis’ page.

Sea ice extentPresent Daily-Anomaly

Data - AMSR-E, SSM/I, QuikSCAT

Arctic Data Collection

°Region

°Time

°Ice extent

°Sea surface temperature

°Ocean winds

°Ocean color

°Sea ice freeboard

°Sea ice motion

°Clouds

°Albedo

°Atmospheric fields

Etc.

Note: Could extend to land, ice sheets etc

Sea Surface TemperaturePresent

Data - AVHRR, Modis, GHRSST

Ocean WindsPresent

Ex.Monthly Climatology

Data - QuikSCAT, AMSR-E, SSM/I

Ocean ColorPresent Ex. Annual (monthly)

Anomaly

Data - Modis, SeaWiFS

Additional products -Sea ice freeboard,Sea ice age, etc

Additional products- Sea ice motion,Clouds, Atmospheric fields,etc

Ex. Annual (monthly)Anomaly

B. Holt-JPL

User Scenario: Arctic IceNASA Views the Polar Region

04/21/23 2008 UWG3

Joint Implementation Scenario

Example of a Dual Purpose Interface

QuickTime™ and a decompressor

are needed to see this picture.

July 16-17, 2008 PO.DAAC Review, NASA HQ6

Backup Material

• Create a simple to use interface(s) for Cryospheric and Physical Create a simple to use interface(s) for Cryospheric and Physical Oceanographic dataOceanographic data

• Get more data distributed to more users Get more data distributed to more users

• Build the 80% solution for experts to novicesBuild the 80% solution for experts to novices

• Get the user just the data they need, in the format they want…Get the user just the data they need, in the format they want…quickly. Accelerate the use of data.quickly. Accelerate the use of data.

• Allow users to display and interact with the dataAllow users to display and interact with the data

• Create a standard & flexible framework for new applicationsCreate a standard & flexible framework for new applications

• Build a system that allows the user to analyze the data, at the Build a system that allows the user to analyze the data, at the server, so they don’t need to move as much dataserver, so they don’t need to move as much data

• Allow the system to be expanded by users or other machines

The Objective

1.1. User describes what they wantUser describes what they want

2.2. User selects What data (product)User selects What data (product)

3.3. User selects Where (location)User selects Where (location)

4.4. User selects When (time interval)User selects When (time interval)

5.5. User selects filters (if any) User selects filters (if any) Then:Then:

6.6. If the number of data files is small the user If the number of data files is small the user is given thumbnailsis given thumbnails

7.7. User can view the thumbnailUser can view the thumbnail

8.8. The user can analyze the dataThe user can analyze the data

9.9. User can select the output formatUser can select the output format

Quick search, retrieval, display & deliveryQuick search, retrieval, display & delivery

In the In the order order the the user user wants:wants:

Implementation Scenario of Arctic Ice Scenario at PODAAC and NSIDC

• Extend Sea Ice Extent Map to be accessible through both PODAAC and NSIDC. • Single map, accessible from PODAAC and NSIDC. • Map interface is modified and maintained by NSIDC based on shared requirements from PODAAC and NSIDC

• Provide local and remote access to data products identified by region selected via an interactive table.• PODAAC middle ware developed to allow access to products at both sites. Technology is operational from both sites and

is modified and maintained by PODAAC based on shared requirements from PODAAC and NSIDC.

• Provide Level 2 and Level 3 subsetting capability for all available products for the area of interest and based upon requirements for interoperability and data display and distribution. • PODAAC develops and extends L2, L3 subsetters for PODAAC products. • NSIDC develops and extends L2, L3 subsetters for NSIDC products.

• Provide all products in a compatible form and format for display and further tool input. • PODAAC and NSIDC provide all products in an agreed form and format for display and further tool input and distribution.

• Provide visualization capability for all selected products.• PODAAC and/or NSIDC develops visualization capability for product overlay to Ice Extent map based on agreed upon formats and

requirements.

• Provide DataCasting availability at both sites for notification of products• JPL developed Datacasting will be ported and configured for NSIDC by PODAAC.

• Provide a single User Interface accessible from both sites that integrates the Sea Ice Extent Map, InteractiveTable of Products, visualization capability, format conversion, datacasting and product delivery.• Interface is developed and maintained by NSIDC based upon requirements from both PODAAC and NSIDC.

• Provide machine to machine accessible products from both NSIDC and PODAAC• NSIDC and PODAAC select a single point of access for user selected products. • FTP directory specific to the user selection located at one or the other site.

July 16-17, 2008 PO.DAAC Review, NASA HQ9

Target Capabilities

• Query & Display MapQuery & Display Map - Query and display data in the same widow so the the context of the data relative to the region of interest can be seen along with the thumbnail view.

• Value FilteringValue Filtering - Filtering of search results by parameter values, e.g. minimum, maximum, distribution, etc.

• PeriodicPeriodic - The user may want the same time-series period for a study, for example, September of the past 10 years.

• Multi-Data SelectionMulti-Data Selection - Allow multiple product selections• Region SearchRegion Search - Using geographic terms for spatial

specification, e.g. Greenland, Antarctica, Alaska etc.

Target Capabilities - Continued

• Display data in Polar coordinatesDisplay data in Polar coordinates - such as Polar Stereo• Linked List & MapLinked List & Map - Dynamically link the top listed granule

to the top displayed thumbnail • Analyze, Plot & AnimateAnalyze, Plot & Animate - Give the user the chance to work with the

data.• On demand subsettingOn demand subsetting - select just the data in the region of interest.• Dual data displayDual data display - Allow comparisons by time, product and location.• Multi-format outputMulti-format output - Give it to the user in the format they need.• SubscriptionSubscription - Allow users to get new data as available.• User modelingUser modeling - Allow users to build their own processes, models

and plug-ins to extend the system. (machine to machine)

Capabilities - Comparison

System WIST CDC Giovanni DADDI NEO RangeView Arctic RIMSData Catalog

Explorer

Agency NASA NOAANASA

GoddardNSIDC NASA

University of Arizona

University of New Hamp.

Alaska Ocean Observing

Query & Map Display

N N N N Y Y Y Y

Polar View Y N N Y N N Y N

Value Filter Y Y Y/N N N N Y N

Periodic Y/N Y N Y Y/N N Y N

Multi-Data Selection

Y Y Y/N Y Y Y Y Y

Regions N N N N Y Y Y Y

Linked List & Map

N N N N N Y Y Y/N

Analyze, Plot &

AnimateN Y Y N Y Y/N Y Y/N

On Demand Subsetting

N N Y Y N N N N

Dual Displ

ayN N N N N Y N N

Multi-format Output

N N N N Y N N Y/N

Subscription

Y/N N N N N N N N

User Modeling

N Y/N N N N N N N

Operational Scenario

• A user at either PODAAC or NSIDC selects a tool for searching for and selecting data. The tool, by all appearances, is identical at either site. Only the banners are specific to the data center. The tool provides the ability to select a region of interest anywhere on the globe. The user is able to enter a time or time range for the area of interest. After entering this information, a list of available products in the area of interest and time at both PODAAC and NSIDC (at a minimum) is provided. This list of products allows the user to select products from the list for display and/or download. As products are selected, a display of the region of interest is updated with a visualization of the product overlaying the region. The updated display is also annotated with the information from the data related to time, location, name of product and information pertinant to the data (speed, temperature, etc). Products can be overlayed upon each other, or can be displayed individually on the area of interest. The user can download a copy of the image currently displayed which will include the annotation information. In addition, the user can request data products for download in one of several available formats. The selectable formats shall include but not be limited to: HDF, NetCDF . The user can request to be notified of new products from this list as they become available in this region. The user can also request notification for any other products, not in the list but within the region when they become available at any of the data locations, PODAAC and NSIDC (at a minimum). The user can request that these products be available as input into other tools or services provided by the data locations, PODAAC and NSIDC (at a minimum) for further manipulation, display or analysis.

July 16-17, 2008 PO.DAAC Review, NASA HQ13

July 16-17, 2008 PO.DAAC Review, NASA HQ14

Functional Requirements

• Functional Requirements for a single tool providing both PODAAC or NSIDC data

• JPL’s Requirements List• Support user request for information and/or products within a specified area and time from a range

of products resident locally and remotely. • Provide ability to select area of interest by location and time.• Provide ability to select data available in area of interest by location and time from multiple

measurements both local and remote.• Provide data center identification of data location .• Provide visualization of selected products as an overlay within specified area of interest.• Provide overlay visualization capability of multiple selected products within specified area of

interest.• Provide annotation of the visualized overlays which includes not less than name of product, time of

acquisition, lat/lon, measurement value range and type (speed, height, temperature).• Provide acquisition of annotated image products displayed within area of interest. • Provide acquisition of selected data products displayed within area of interest.• Provide acquired data products in selectable data format of at least HDF and NetCDF• Provide notification of new products as they become available within a specified period of time.• Provide notification of new products as they become available within the specified area of interest.• Provide machine to machine interface for acquisition of products via FTP. • Provide data linkage for selected products to other available tools and services at local and remote

data center. • Provide access to tool described above from either PODAAC or NSIDC. • Provide data center unique product emphasis such as product table list order, or area of interest

emphasis (masking, etc) .

July 16-17, 2008 PO.DAAC Review, NASA HQ15

Functional Requirements

• Functional Requirements for a single tool providing both PODAAC or NSIDC data• NSIDC’s Requirements List

• A Web-accessible system to provide NSIDC's users the ability to preview and analyze key data sets in our collection. The interface will be straightforward, in the spirit of the CDC reanalysis web site, but will provide "something better" which is TBD. The system will initially focus on the Greenland area, but will not be limited to only work in that region

• Performance and Design Objectives 1. system response times will depend on action performed:

1. 5-second response time for display results, or in-progress indicator2. ??-second response time for data delivery (this should probably be a rate, as in seconds per file ordered, or

seconds per GB of data ordered)

2. the user will be able to access the system with a common web browser, with no plug-ins or special software for the user to download; Ajax functionality is specifically in-scope; the current list of supported browsers is

1. Windows: IE 7.x; Firefox 3.2. Macintosh/Linux: Firefox 3.x

3. system will be built with open source (C) and/or Government-funded technologies (C) (W says "preferably open source")

4. the user experience will be consistent; users will not be required to to know where the data are coming from, and will not be required to learn this in order to use the system.

5. to maximize reuse and minimize maintenance costs, there will be a standard user interface built for each kind of information (spatial, temporal, etc). Each of these interfaces may have different ways for the user to specify input, (e.g. specifying a date using a calendar widget vs. a text box, but there will not be 2 implementations of a calendar widget)

6. the system will incorporate reasonable security measures, for example, using ssl for at least user authentication