Upload
tucker
View
39
Download
0
Tags:
Embed Size (px)
DESCRIPTION
EPIC Tools for in-situ data collections. Donald W. Denbo, NOAA/PMEL Willa H. Zhu, NOAA/PMEL/JISAO John Osborne, OceanAtlas Software Christopher Moore, NOAA/PMEL/JISAO. NOAA Coastal Storms Initiative Meeting May 6-8, 2003. EPIC Components. Data Archive and Retrieval - PowerPoint PPT Presentation
Citation preview
EPIC ToolsEPIC Toolsfor for in-situin-situ data collections data collections
Donald W. Denbo, NOAA/PMELWilla H. Zhu, NOAA/PMEL/JISAO
John Osborne, OceanAtlas SoftwareChristopher Moore, NOAA/PMEL/JISAO
NOAA Coastal Storms Initiative MeetingNOAA Coastal Storms Initiative MeetingMay 6-8, 2003May 6-8, 2003
EPIC ComponentsEPIC Components Data Archive and RetrievalData Archive and Retrieval
Multiple file formats supported with “epslib”Multiple file formats supported with “epslib” Metadata database supports subsetting of large data collectionsMetadata database supports subsetting of large data collections
““Classic EPIC” Classic EPIC” Unix programs for desktop display, analysisUnix programs for desktop display, analysis Gateways to Matlab, PPLUS, Ferret, IDL, etc.Gateways to Matlab, PPLUS, Ferret, IDL, etc.
EPIC Web BrowserEPIC Web Browser Search, graph, list, download from the WebSearch, graph, list, download from the Web Toolkit for building custom web applicationsToolkit for building custom web applications
Java applicationsJava applications Java OceanAtlas - Sophisticated browse and data explorationJava OceanAtlas - Sophisticated browse and data exploration Climate Data Portal - distributed data access (CORBA/OPeNDAP)Climate Data Portal - distributed data access (CORBA/OPeNDAP) OceanShare - Network data collaboration toolOceanShare - Network data collaboration tool ncBrowse - Generalized browse tool for data/attributes (local files, ncBrowse - Generalized browse tool for data/attributes (local files,
network files, OPeNDAP files)network files, OPeNDAP files)
EPIC Archive and RetreivalEPIC Archive and Retreivalfor Classic EPIC, Web, and Java Applicationsfor Classic EPIC, Web, and Java Applications
Data collections may include thousands Data collections may include thousands or even millions of profiles or time seriesor even millions of profiles or time series
Typically the researcher will use EPIC Typically the researcher will use EPIC tools to:tools to: Select data by lat, lon, time, keyword, etc.Select data by lat, lon, time, keyword, etc. Refine the initial data selection Refine the initial data selection Display and/or analyze the selected dataDisplay and/or analyze the selected data
Pacific
NODC
Alaska
Meta DataData
File Collections
Data LoaderUtility
Selecteddatasets pointers
to data files
Data SearchLat, long, timekeyword, etc
creates
DataSearch
andSelection
“Classic EPIC”
programs
MatlabFerret
IDLPPLUS
EPIC Web
Java OceanAtlasClimate Data Portal
OceanSharencBrowse
MySQLdatabase
EPIC Data Management EPIC Data Management StrategyStrategy
EPIC data usageEPIC data usage
EPIC Data File Input/OutputEPIC Data File Input/Output
Applications programs
API
File Independent Layer
FDL
Existing Planned
Data file formatData file format Multiple data file formats Multiple data file formats
are supportedare supported New formats are readily New formats are readily
addedadded
Applications software Applications software Independent of file formatIndependent of file format API for most API for most
programming languagesprogramming languages
Gateways Gateways to Matlab and other to Matlab and other
graphics/analysis graphics/analysis packagespackages
EPS Library
Classic EPICClassic EPICSuite of Unix programs for the desktopSuite of Unix programs for the desktop
Over 100 EPIC programs are available for:Over 100 EPIC programs are available for: Data selection, metadata and file editing...Data selection, metadata and file editing... GraphicsGraphics
Time series, multivariable plots, section plots...Time series, multivariable plots, section plots...
Parameter calculationParameter calculation Dynamic height, geostrophic velocity...Dynamic height, geostrophic velocity...
Statistical, spectral, tidal harmonic analyses...Statistical, spectral, tidal harmonic analyses...
EPIC’s modularity allows a researcher to EPIC’s modularity allows a researcher to easily add new analysis techniques or use easily add new analysis techniques or use a favorite graphics or analysis program.a favorite graphics or analysis program.
EPIC Web BrowserEPIC Web Browser
EPIC Web BrowserEPIC Web BrowserData selection, graphics, downloadData selection, graphics, download
EPIC Web Browser CapabilitiesEPIC Web Browser Capabilities
PlotsPlots User customized plots for profiles and User customized plots for profiles and
time series time series (T, S, Sigma-t, Ox) vs Pressure/Depth(T, S, Sigma-t, Ox) vs Pressure/Depth Temperature vs SalinityTemperature vs Salinity One or two user-selected Properties vs One or two user-selected Properties vs
Pressure/DepthPressure/Depth Contoured section plotsContoured section plots
List/download data, metadataList/download data, metadata New plots and functions readily includedNew plots and functions readily included
EPIC tools for the Web EPIC tools for the Web for building customized web access to datafor building customized web access to data
EPIC Web ToolkitEPIC Web Toolkit Interface design Backend graphics Map data selector
Example applicationsExample applications EPIC Web Browser (general) TAO (El Nino buoys) Unaami (Arctic time series) OSCAR (Satellite ocean
currents) Multibeam data
(hydrography) http://www.epic.noaa.gov
EPIC Java Tools and ApplicationsEPIC Java Tools and Applications
Distributed data accessDistributed data accessand and
Advanced data browsing Advanced data browsing
Java ToolsJava Tools
EPIC Java EPIC Java ApplicationsApplications
for locating and selecting for locating and selecting from distributed data from distributed data
collectionscollections
ChallengeChallenge
Locating and selecting individual Locating and selecting individual observations from large, distributed observations from large, distributed collections of collections of in situin situ data data observationsobservations Examples: Individual oceanic or
atmospheric profiles, time series, buoy tracks, etc.
EPIC Java EPIC Java ApplicationsApplications
Some solutionsSome solutions
Tools and ApplicationsTools and Applications LDAP Directory Services NdEdit for subsetting large in-situ collections Interactive Java graphics Intensive exploration of multiple data types OPeNDAP (DODS) clients and servers for in-situ data Access from Desktop and Web
Assembled from OpenSource componentsAssembled from OpenSource components http://www.epic.noaa.gov/epic/software/ep_java.htm
EPIC Java EPIC Java ApplicationsApplications
LDAP Directory ServiceLDAP Directory ServiceFinding distributed data collectionsFinding distributed data collections
Search on geographic region, observed quantities or Search on geographic region, observed quantities or keywordkeyword Automatically populated from data serversAutomatically populated from data servers Metadata instantly known to LDAP directoriesMetadata instantly known to LDAP directories Access control for experimental data productsAccess control for experimental data products
Open SourceOpen Source Standards-basedStandards-based Platform independent Platform independent Scalable, extensible Scalable, extensible Supports desktop or web clientSupports desktop or web client
EPIC Java EPIC Java ApplicationsApplications
X-Y map view
Y-TY-T time range
NdEdiNdEditt
Investigate space-time distribution
Select data in any View
Explore Data Locations and Subset Data Collections
Interactive, linked windows
View any 2D sliice xy, xt, xz, yt, yz, yt Select
time rangeSee data in range
X-Y map view
Y-TY-T time range
EPIC Java EPIC Java ApplicationsApplications
Section Tool - select stations along a depth contour
NdEdiNdEditt
EPIC Java EPIC Java ApplicationsApplications
Selected stations are in a narrow time band
NdEdiNdEditt
EPIC Java EPIC Java ApplicationsApplications
Polygon tool – select an oblique section line
See time range of stations
NdEdiNdEditt
EPIC Java EPIC Java ApplicationsApplications
NdEdiNdEditt
NdEdit is an interactive Java toolNdEdit is an interactive Java tool NdEdit is being re-engineered for the WebNdEdit is being re-engineered for the Web
EPIC Java EPIC Java ApplicationsApplications
SGT: SGT: A Scientific Graphics ToolkitA Scientific Graphics Toolkit Java toolkit for scientific graphicsJava toolkit for scientific graphics
On-line documentation, examples, tutorials, streaming On-line documentation, examples, tutorials, streaming videovideo
JavaBeans in next releaseJavaBeans in next release FeaturesFeatures
Contour, line and vector plots, Contour, line and vector plots, Axes user settable or automatically scaledAxes user settable or automatically scaled multiple Time Axis formats multiple Time Axis formats mouse-movable, customizable Labels, line keys, iconsmouse-movable, customizable Labels, line keys, icons Legend generation automaticLegend generation automatic
Widely used, freely availableWidely used, freely available scientific and commercial applications use SGTscientific and commercial applications use SGT 14,782 downloads by 5,151 unique sites from 70 14,782 downloads by 5,151 unique sites from 70
countriescountries http://www.epic.noaa.gov/java/sgt/http://www.epic.noaa.gov/java/sgt/EPIC Java EPIC Java
ApplicationsApplications
Java ApplicationsJava Applications
for the Desktop and the for the Desktop and the NetworkNetwork
EPIC Java EPIC Java ApplicationsApplications
ncBrowsencBrowseJava tool for data/metadata browseJava tool for data/metadata browse
NetCDF and full OPeNDAP supportNetCDF and full OPeNDAP support Desktop Java application Axis mapping, parametric plots, flexible mapping of
independent variables to axes Line, vector and contour plots Animations, 3D graphics via VisAD
Popular, freely availablePopular, freely available 8,187 downloads by 3,415 unique sites in 59 countries OPeNDAP (DODS) client, Unidata netCDF client, WOCE
data CD, etc. http://www.epic.noaa.gov/java/ncBrowse/
EPIC Java EPIC Java ApplicationsApplications
Java OceanAtlasJava OceanAtlasfor browsing ocean datafor browsing ocean data
Intensive Java desktop browsing of ocean dataIntensive Java desktop browsing of ocean data Property-property plots with up to 7 x axes, waterfall diagrams, Property-property plots with up to 7 x axes, waterfall diagrams,
Contour plots/residual contour plots, Map plots with optional Contour plots/residual contour plots, Map plots with optional "bubble" isosurface plots, Linked browsing of all plot types, Built-in "bubble" isosurface plots, Linked browsing of all plot types, Built-in and custom calculations, NdEdit for data location/selectionand custom calculations, NdEdit for data location/selection
Reads WOCE, EPIC, OPeNDAP netCDF data, some ARGO formatsReads WOCE, EPIC, OPeNDAP netCDF data, some ARGO formats Extended for wide range of data typesExtended for wide range of data types
Profile, time series, drifting buoy, gridded analyses, model outputsProfile, time series, drifting buoy, gridded analyses, model outputs Widely used, freely availableWidely used, freely available
http://www.epic.noaa.gov/epic/software/JavaJOA.htmhttp://www.epic.noaa.gov/epic/software/JavaJOA.htm http://odf.ucsd.edu/joa/jsindex.htmlhttp://odf.ucsd.edu/joa/jsindex.html
EPIC Java EPIC Java ApplicationsApplications
Climate Data PortalClimate Data PortalDistributed ocean observationsDistributed ocean observations
Major Major in situin situ data data collections:collections: TAO El Nino buoy data
(realtime/PMEL), Global sea level data (realtime/U of Hi), GTSPP (NODC), PMEL hydrographic data and time series collection, WOCE ADCP (U of Hi), IPRC/APDRC in situ data, US Coastal Ocean Time Series database (NODC)
OPeNDAP: in-situ data server and clientOPeNDAP: in-situ data server and client LDAP directory services NdEdit for data selection/subsetting Data Servers, Clients (Java and Web) – focus on in
situ data http://www.epic.noaa.gov/cdp/cdpjava.htmEPIC Java EPIC Java
ApplicationsApplications
OceanShareOceanShareNetwork collaboration toolNetwork collaboration tool
Collaboration featuresCollaboration features Local, remote data file access and Java graphicsLocal, remote data file access and Java graphics Standard collaboration tool features (annotation, Standard collaboration tool features (annotation,
whiteboard, etc) whiteboard, etc) Secure LDAP Document RepositorySecure LDAP Document Repository
Proposals, papers in progress, data, imagesProposals, papers in progress, data, images Access control by document or by folderAccess control by document or by folder Accessible from web and desktop Accessible from web and desktop
OPeNDAP compatible soonOPeNDAP compatible soon http://www.epic.noaa.gov/collab/http://www.epic.noaa.gov/collab/
EPIC Java EPIC Java ApplicationsApplications
EPIC Java Tools and Applications EPIC Java Tools and Applications
Modular Web and Open Source Modular Web and Open Source componentscomponents
Java interactivityJava interactivity Low cost development/maintenance Low cost development/maintenance OpenSource, standards-basedOpenSource, standards-based Scalable, extensibleScalable, extensible OPeNDAP supported or plannedOPeNDAP supported or planned
http://www.epic.noaa.govhttp://www.epic.noaa.gov http://www.epic.noaa.gov/epic/software/http://www.epic.noaa.gov/epic/software/
ep_java.htmep_java.htm