Upload
truongcong
View
215
Download
0
Embed Size (px)
Citation preview
11/28/2014
1 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Giovanni-4:
The Next Generation
C Lynnes, M Hegde, C Smit, D DaSilva, K Bryant, P. Zhao, Z. Liu, S. Shen,
A. Savtchenko, W. Teng, J. Wei, J. Acker
https://ntrs.nasa.gov/search.jsp?R=20160005868 2018-09-08T00:55:37+00:00Z
11/28/2014
2 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Outline
• Overview of Giovanni
• New and Notable Features
• Discussion?
11/28/2014
3 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Giovanni and “Big Data”
WRITE CODE
Select Data
Find
Download
Learn format
Read
Summarize / Analyze
Analyze
Quality Filter
Subset
Publish
Derive Conclusions
Visualize
Exploratory Data Analysis
Main Analysis
Phase
11/28/2014
4 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Why So Difficult?
WRITE CODE
Select Data
Find
Download
Learn format
Read
Summarize / Analyze
Analyze
Quality Filter
Subset
Publish
Derive Conclusions
Visualize
Exploratory Data Analysis
Main Analysis
Phase
FTP Data Products up to 1 GB Volume
Summarize data over long time periods, large areas
Volume
11/28/2014
5 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Why So Difficult?
WRITE CODE
Select Data
Find
Download
Learn format
Read
Summarize / Analyze
Analyze
Quality Filter
Subset
Publish
Derive Conclusions
Visualize
Exploratory Data Analysis
Main Analysis
Phase
2000+ datasets Variety
HDF4, HDF5, HDF-EOS2, HDF-EOS5, netCDF, GRIB, binary
Variety
Files up to 800+ variables Variety
Locate, read, use geolocation info Variety
11/28/2014
6 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Remote Sensing Science: Why So Difficult?
WRITE CODE
Select Data
Find
Download
Learn format
Read
Summarize / Analyze
Analyze
Quality Filter
Subset
Publish
Derive Conclusions
Visualize
Exploratory Data Analysis
Main Analysis
Phase
FTP Data Products up to 1 GB
HDF4, HDF5, HDF-EOS2, HDF-EOS5, netCDF, GRIB, binary
Files up to 800+ variables
More studying, coding
Locate, read, use geolocation info
Volume
Variety
Variety
Veracity
Variety
2000+ datasets Variety
11/28/2014
7 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Giovanni provides (relatively) rapid exploration of datasets
WRITE CODE
Select Data
Find
Download
Learn format
Read
Summarize / Analyze
Analyze
Quality Filter
Subset
Publish
Derive Conclusions
Visualize
Exploratory Data Analysis
Main Analysis
Phase
Giovanni provides Server-Side, Quick-Start
Exploratory Data Analysis:
no coding necessary
no downloads necessary
Web-based Services
Gio
vann
i
Find
Fetch
Reformat
Filter
Regrid
Summarize
Visualize
11/28/2014
8 Goddard Earth Sciences
Data Information Services Center
GES – DISC
What’s New?
• Giovanni-4, relative to Giovanni-3 – Omnibus Giovanni Portal
– Federated Giovanni
– Variable selection interface
– Sharable/reproducible URLs
• New in Giovanni-4 – More comparison functions
– Vector Plots
– Seasonal Analysis
– Shapefile-based Area Averages
11/28/2014
9 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Omnibus Portal
• All variables in one portal
– Compare across instruments and disciplines
– Caveat: Multi-temporal comparisons not yet supported
• However: Requires more navigational aids
11/28/2014
10 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Federated Giovanni will further increase the availability of data in Giovanni
• Ongoing now: Funded by NASA-ACCESS: 2014-2016
• Eventual release of source and Virtual Machine to the community
MODAPS
Giovanni
GDAAC
Giovanni
LP DAAC
Giovanni
OBPG
Giovanni
PO DAAC
Giovanni
11/28/2014
11 Goddard Earth Sciences
Data Information Services Center
GES – DISC
MODIS NDVI from LP DAAC
11/28/2014
12 Goddard Earth Sciences
Data Information Services Center
GES – DISC
SST from PO.DAAC
11/28/2014
13 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Giovanni-4 User Interface
Sortable columns
Search with Autosuggest
Faceted Browse
11/28/2014
14 Goddard Earth Sciences
Data Information Services Center
GES – DISC
11/28/2014
15 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Keyword search on precipitation
11/28/2014
16 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Reset with button or by clearing search
11/28/2014
17 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Selecting a Discipline “Facet”
11/28/2014
18 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Selecting a Discipline “Facet”
16
Number of matching variables: 16 of 219
11/28/2014
19 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Narrow by selecting more facets
11/28/2014
20 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Selecting variables leads to hints on date range
11/28/2014
21 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Selecting variables leads to hints on date range
Valid Range: 1997-12-31 to 2014-08-31
11/28/2014
22 Goddard Earth Sciences
Data Information Services Center
GES – DISC
You can also sort on columns
11/28/2014
23 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Spatial and temporal resolution facets are also available
11/28/2014
24 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Giovanni-4 selections are encoded in the URL Making them Sharable and Reproducible
http://giovanni.gsfc.nasa.gov/giovanni/#
service=INTERACTIVE_MAP&
starttime=2001-01-01T00:00:00Z&
endtime=2001-01-31T23:59:59Z&
bbox=-124.4531,24.2578,-68.2031,52.3828&
data=MOD08_D3_051_Optical_Depth_Land_And_Ocean_Mean&
variableFacets=dataFieldDiscipline%3AAerosols%3B
Service
Time Range
Bounding Box
Data Variable
11/28/2014
25 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Service
Time Range
Bounding Box
Data Variable
Giovanni-4 URLs are Sharable and Reproducible
11/28/2014
26 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Data comparison services to compare similar datasets
• Correlation Map
• Difference of Time-Averaged Maps
• Time Averaged Scatterplot
• Area Averaged Scatterplot
• Time Series of Differences of Area Averages
11/28/2014
27 Goddard Earth Sciences
Data Information Services Center
GES – DISC
2010-2011
Correlation Map: TRMM Rainfall Versions 6 vs. 7
11/28/2014
28 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Difference of Time-Averaged Maps
• LPRM from X-band is more affected by vegetation than by C. This difference map shows that C and X don't differ that much when the surface is bare to sparsely vegetated, but do differ when the surface is more densely vegetated and soil moisture retrieval becomes problematic. (Courtesy Bill Teng)
11/28/2014
29 Goddard Earth Sciences
Data Information Services Center
GES – DISC
MODIS NDVI from LP DAAC
11/28/2014
30 Goddard Earth Sciences
Data Information Services Center
GES – DISC
New Services
• Vector Plots
• Seasonal Analysis
– User-defined Climatology
– Interannual Time Series
• Shapefile-based Area Average Time Series
11/28/2014
31 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Vector Plots
• Available for some wind-related variables
– More coming soon…
• Time Averaged Map service only
• Computes averages of U and V, then vector magnitude + direction
11/28/2014
32 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Vector Plots (Time-Averaged Map)
Wind Stress
Dec 1997 (El Nino)
Dec 2000 (La Nina)
Courtesy Chung-Lin Shie
11/28/2014
33 Goddard Earth Sciences
Data Information Services Center
GES – DISC
User-Defined Climatology
Available for Monthly data Aggregate by Month or Meteorological Season (DJF, MAM, JJA, SON)
11/28/2014
34 Goddard Earth Sciences
Data Information Services Center
GES – DISC
User-Defined Climatology
Click Here to Repeat
March, 2001-2013
September, 2001-2013
MODIS Monthly AOD
Comparison of aerosol regimes in March vs. Sept. over 2001-2013. (Courtesy Suhung Shen)
11/28/2014
35 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Seasonal Analysis: Interannual Time Series
• Available for Monthly data
• Plot the area average for <month> for the years <start> to <end>
11/28/2014
36 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Interannual Time Series Example
Strong summer (July) warming, on the order of 0.5K/dec during the past decade, is observed by AIRS over the permafrost of the Tundra region. (Courtesy Andrey Savtchenko)
Click here to repeat
11/28/2014
37 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Shapefile Support
• Area-Averaged Time Series
• Coming soon…time-averaged map, histogram
• Limitations: does not work across dateline
11/28/2014
38 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Shapefile-based Area Averaged Time Series
Compute average over area, weighted by % coverage by shape and cos(lat)
11/28/2014
39 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Shapefile-based Area Averaged Time Series
TRMM Rainfall for Oct. 2012, showing Hurricane Sandy landfall. (Courtesy Zhong Liu)
11/28/2014
40 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Shapefile based Computation of MODIS-Terra Monthly AOD
U.S.
China
Courtesy Jennifer Wei
11/28/2014
41 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Giovanni 4 Performance
0
500
1000
1500
2000
0 500 1000 1500 2000 2500 3000
Ela
pse
d S
eco
nd
s
Days of Data
Time Averaged Map
G4
G3
0
2000
4000
6000
8000
0 100 200 300 400 500 600 700 800
Elap
sed
Se
con
ds
Days of Data
Area-averaged Time Series
G4
G3
11/28/2014
42 Goddard Earth Sciences
Data Information Services Center
GES – DISC
DISCUSS…
11/28/2014
43 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Features Still on the TO DO list
Pre-computed Climatology (up next) OMI Level 2G Level 2 data support Cross sections (Z-Lat, Z-Lon, Z-Time) Time series of area statistics Zonal mean Multi-variable plots (overlays) Web Coverage Service interface MyGiovanni: user-contributed color palettes, shapefiles,
data, algorithms Geometric Mean Units conversion More download options (KMZ, more ASCII, batch)
Your input can help us prioritize them
11/28/2014
44 Goddard Earth Sciences
Data Information Services Center
GES – DISC
Giovanni was designed for exploratory data analysis…
WRITE CODE
Select Data
Find
Download
Learn format
Read
Summarize / Analyze
Analyze
Quality Filter
Subset
Publish
Derive Conclusions
Visualize
Exploratory Data Analysis
Main Analysis
Phase
Web-based Services
Gio
vann
i Find
Fetch
Reformat
Filter
Regrid
Summarize
Visualize
1. …Should Giovanni evolve to support the Main Analysis Phase better? 2. What modifications would enhance “scientific rigor”?