Upload
nathan-boone
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
Wouldn’t it be cool if…
…at the press of a button, we could
• calculate Wedderburn number and other physical lake characteristics
• smooth buoy data to specific scales
• isolate patterns by scale
…we could mine non-traditional data sources to help understand our lakes
Lake Mendota, Wisconsin
Beach monitoring network
Manual data
MERIS
3D Simulation
Vega Data Model
• Value oriented structure • Store data from any
number of sites• Highly optimized
‘Values’ table• Query Times < 1 sec• GLEON central ~30
million values
StreamsStreams
Buoy data
…combine multiple data sources to simulate lakes
…we could work in teams to produce science that transcends site boundaries
CDI-Type II: Collaborative Research: New knowledge from the GLEON
PIsPaul Hanson, UWMiron Livny, UW CSAnHai Doan, UW CSChin Wu, UW CEEKen Chiu, SUNY-B CSMatt Hipsey, UWAFang-Pang Lin, NCHC
Many GLEON collaborators!
Lauri Arvola University of Helsinki, Lammi Biological Station, Finland
Thorsten Blenckner, Institute of Ecology & Evolution, Sweden
Evelyn Gaiser Florida International University
David Hamilton University of Waikato, New Zealand
Zhengwen Liu Nanjing Institute of Geography and Limnology, China
Diane McKnight University of Colorado, Boulder
David da Motta Marques, Universidad Federal do Rio Grande do Sul, Brazil
Kirsti Sorsa Public Health Madison, Wisconsin
Peter Staehr University of Copenhagen, Denmark
transform ecological sensor networks from data collectors to knowledge
generators through integration of the people, data, and cyberinfrastructure of
lake sensor networks.
CDI: Cyber-enabled Discovery and Innovation
Human interface
Virtual private server
Modeling
1
2
3
4
CDI
Each site has a POP that saves sensor data to text file.
Each site has a POP that saves sensor data to text file.
Web services for data accessWeb services
for data access Vega data model on mySQL
Vega data model on mySQL
20+ observatories
50+ sensing platforms
>100 million records
Projected > 1 billion by 2012
1 2 3 4 5 6 7log 10
8 9
GLEON Numbers(September 2009)
169 members from 25 countries
CDI
• Uses existing GLEON infrastructure• Open to all interested scientists• It’s a way of doing science• Starts at the data repository • Implement existing technologies• Develop some new technologies
Lake Observatory
+ I T Development
GLEONObservational Data
Repositories
Query and display observational data
dbBadgerSoftware suite
Streaming data
Web, e.g.,dbBadger
Mendota buoyLSPA
New to this proposal
Model suite
Existing
22
XY
Z
10-4
10-3
10-2
10-1
100
100
105
1010
Frequency (Hz)
Power Spectrum
chlorophyll
phycocyanin
dissolved oxygen
hourdayweek
Po
we
r
11
33
Multi-dimensional virtual data
Total Chl
Mendota group:CFL, CEE, SSEC and others
CDI Team:Wisconsin, NY, UWA,
NCHC, GLEON
Some CDI Activities – 1st year
• QA/QC Sensor network data• Implement basic signal processing• Incorporate manually sampled data• Workshops to calibrate nD models• Run nD models• Web display of lake data
Get Involved!(Thur, 10:15 break)
Frequency
Sp
atia
l ext
en
t
Minute Hour Day Month Season
2
1
Me
ters
Eco
syst
em
3
4
Model input
Model input
Circle size data quality
Lake Mendota, Wisconsin
Target scale of model
Target scale of model
Sensor network data
Sensor network data
Unstructured data on singular events from watershed
Unstructured data on singular events from watershed
Historical data
Historical data
Unstructured data from life-guards and city of Madison
Unstructured data from life-guards and city of Madison
Unstructured data
Raw sensor data level 1
Other data 1 Other data 2Sensor data level 2,3
Model(filters, transforms, etc.)
Standard data-model interface file (e.g., NetCDF)
Standard data-model interface file (e.g., NetCDF)
Model(process)
Model(QA)
Standard data-model interface file (e.g., NetCDF)
Environment: Condor on clusterWorkflow: DagMan
Model(QA)
Transfer protocols?
Transfer protocols?
Data structure?
Data structure?
Data structure?
Data structure?
PCB model coupling?
PCB model coupling?
Algorithms?Algorithms?
Visualization?Visualization?
Virtual private server
END
Opinions About Technology Solutions
• Best long-term solution is unknowable– Tools to move data rapidly to shareable state– Are short-term needs at odds with long-term
solutions?• Solutions for all ecologists
– Most ecologists aren’t funded to create technology – Simplicity, autonomy, compatibility– Technology transfer?
• Who wants to adapt another’s system? • Outsource, partner, federate
• Culture of experimentation and change– Must try solutions from outside science– Social networks as science networks?– Look to current graduate students
GLEON: an international grassroots network of people, data, and lake
observatories
ActivitiesShare experience, expertise, and dataCatalyze joint projects Develop toolsConduct multi-site trainingCreate opportunities for studentsMeet and communicate regularly
Briefly…
• GLEON as an organization• Current technology – from sensors to
ecologists• CDI – data to knowledge
• Points not covered: grassroots approach, decision making and timing; controlled vocabulary; metadata; the science of GLEON
Vega Data Model
• Value oriented structure
• Store data from any number of sites
• Highly optimized ‘Values’ table
• Query Times < 1 sec• GLEON central ~30
million values
Streams
POP
Text file
Ziggy,state,
metadata
Virtual Private Server, Ubuntu Linux
FTP (push)
XML fileXML file XML file
Ziggy
Vega, global db,mySQL POP
XML file
(http pull)
Any db
Site-specific
Local db Buoy system
Ope
n so
urce
Pro
prie
tary
e.g.,Logger-
Net
Buoy system
e.g.,Logger-
Net
Webservice
1
2
3 1