View
45
Download
5
Tags:
Embed Size (px)
DESCRIPTION
Adding value to data. Presentation by Peter Isaac at ESA conference workshop 1 October 2014
Citation preview
Adding Value to Data
A Rake’s Progress
Contents
• Anecdotes from A Rake’s Progress• The TERN/OzFlux facility• The OzFlux data path• Non-generic things about OzFlux• Generic things about OzFlux• Conclusions
TOGA-COARE
OASIS ‘94 and ‘95
The Savanna Project
The TERN/OzFlux Facility
Knowledge of ecosystem exchange of carbon, water & energy.
Surface fluxesRadiation
MeteorologySoil properties
Ecosystem dynamicsSpatial and temporal
dynamicsContinental & global
budgets
Vegetation typeLeaf area indexGross primary
productSoil moistureHyperspectral
OzFluxFlux tower network
AusCoverRemote sensing
eMASTLand surface models
Site characteristicsBiomass
Soil carbon & nutrientsLeaf-level photosynthesis
AusPlots, SupersitesIntensive field campaigns
The OzFlux Site Data Pathu’, v’, w’, T’
q’,c’WS, WD, T, RH
Fsd, Fsu, Fld, Flu
Fg, Sws, Tsoil
Rain
10 Hz
0.1 Hz
Fast (10 Hz raw)
Slow (30 minute average)
CR3000data
logger
Slow(30
minute average)
Fast (10 Hz)(optional with
ethernet modem)
Modmaxmodem
CF card
OzFlux Data Path
Towerdata
OzFluxQC
L3 data
L4 data
L5 data
L6 data
THREDDS server
OzFlux portal
OAI-PMH server
Community developed Python
scripts
WWW
netCDF files
netCDF files
netCDF files
PI institution
NeCTAR/RDSI
OzFlux Data Portal
• Publicly accessible archive of data• Ability to audit data provided by TERN funded
sites• Publication of data via RDA and TDDP– Make the data easy to find
• Repository of self-documenting data files– netCDF files conforming to CF Metadata
Conventions
Non-generic things about OzFlux
• Data is very homogeneous– Standard suite of instruments– Standard data collection and processing– High level of automation
• Existed prior to TERN with a long history of collaboration
• Lucky breaks– Standard data processing and portal developed before
TERN– Choice of data format compatible with public access to
data
Generic things about OzFlux
• Prior to TERN– Individual solutions to data curation– No license applied to data– Data difficult to discover
• Constituency– Community of rugged individualists
Messages for Others
• Clear vision of what data curation is achievable by your community– Accept outside help
• Solutions need to fit the community– Voluntary compliance is better than enforcement– Guiding is better than directing
• Staged solutions to help community accept change– Herding cats
Communication
• Email lists• ASN/OzFlux newsletter• Use of collaborative resources– Trello for scheduling workflow– GitHub for software updates– CloudStor+ for providing data to site Pis– ResearchGate project
What happens when communication fails?
• Establishment of 2 data paths:– Arose through misunderstanding and a lack of
formality in process– Compromise allowed co-existence as “research”
and “operational” systems– OzFlux will need to make sure boundaries
between the 2 systems remain clear inside and outside OzFlux
– OzFlux will need to make sure maintaining 2 systems does not adversely affect resources
Conclusions
• Scientists can be lead to water and they will drink if;– They can see there is something in it for them.– Change is managed, staged and suits the
community.