Use of www to achieve environmental data
Benjamin Pfeil
Bjerknes Centre for Climate Research / University of Bergen
Ways how to get data...
• Often data shows a snapshot of the environment at that time/space
• Sampling can be very expensive (average of over 900.000 NOK for one data set for bio-, geoscience - including costs for expeditions, laboratories, etc)
• Therefore is data very valueable for future scientific work and has to be archived and made available
Why do we need data?
• Verification of research results
• Comparison of results
• Indication of trends
• Model input
• Remote sensing
• Etc.
Some facts about data in the scientific community
• Scientific instruments and computer simulations create large amount of data
• Due to new measurement (and better precision) are the data volumes doubling each year
• Scientific data has to be archived according to ”Good scientific practise in research and scholarship” (European Science Foundation 2000)
0
5
10
15
20
25
30
1970 1980 1990 2000 2010
Publications
Data
?
Global increase in publications in empirical sciences
36. Data are produced at all stages in experimental research and in scholarship. Data sets are an important resource, which enable later verification of scientific interpretations and conclusions. They may also be the starting point for further studies. It is vital, therefore, that all primary and secondary data are stored in a secure and accessible form.
Good scientific practice in research and scholarship European Science Foundation (ESF), 2000
37. Institutions may pay particular attention to documenting and archiving original research and scholarship data. Several codes of good practice recommend a minimum period of 10 years, longer in the case of especially significant or sensitive data. National or regional discipline-based archives should be considered where there are practical or other problems in storing data at the institution where the research was conducted.
Data accumulation, handling, and storage
4. Scientific advances rely on full and open access to data. Both science and the public are well served by a system of scholarly research and communication with minimal constraints on the availability of data for further analysis. The tradition of full and open access to data has led to breakthroughs in scientific understanding, as well as to later economic and public policy benefits. The idea that an individual or organization can control access to or claim ownership of the facts of nature is foreign to science.
Principles for dissemination of scientific data (International Council for Science/CODATA)
5. The interests of database owners must be balanced with society’s need for open exchange of ideas. Given the substantial investment in data collection and its importance to society, it is equally important that data are used to the maximum extent possible. Data that were collected for a variety of purposes may be useful to science. Legal foundations and societal attitudes should foster a balance between individual rights to data and the public good of shared data.
International Council for Science (ICSU)
• Founded in 1931 to promote international scientific activity in the different branches of science and its application for the benefit of humanity
• One of the oldest non-governmental organizations
• More than 135 nations adhere to it• ISCU established the World Data Center
system in the 1950sSource: www.iscu.org
World Data Center system
Mission Statement of the World Data Center System
• Data constitute the raw material of scientific understanding. The World Data Center system works to guarantee access to solar, geophysical and related environmental data. It serves the whole scientific community by assembling, scrutinizing, organizing and disseminating data and information
Network of ICSU WDCs
•Nuclear RadiationTokyo, Japan
WDC Co-ordination OfficesWashington DC, USABeijing, China
•MeteorologyAsheville NC, USABeijing, ChinaObninsk, Russia
•OceaographyObninsk, RussiaSilver Spring MD, USATianjin, China
•PaleoclimatologyBoulder CO, USA
•Marine Geology and GeophysicsBoulder CO, USAMoscow, Russia
•Remotely Sensed Land DataSioux Falls SD, USA
•Renewable Resources and EnvironmentBeijing, China
•Recent Crustal MovementsOndrejov, Czech Republic
•AirglowMitaka,Japan
•AstronomyBeijing, China
•Atmospheric Trace GasesOak Ridge TN, USA
•AuroraTokyo, Japan
•Cosmic RaysToyokawa, Japan
•GeologyBeijing, China
•Human Interactions in the EnvironmentPalisades NY, USA
•IonosphereTokyo, Japan
•Earth TidesBrussels, Belgium
•GeomagnetismCopenhagen, DenmarkEdinburgh, UKKyoto, JapanColaba, India
•GlaciologyBoulder CO, USACambridge, UKLanzhou, China
•Marine Environmental SciencesBremen, Germany
•Rotation of the EarthObninsk, RussiaWashington DC, USA
•Satellite InformationGreenbelt MD, USA
•Rockets and SatellitesObninsk, Russia
•SeismologyDenver CO, USABeijing, China
•Solar Radio EmissionNagano, Japan
•Space ScienceBeijing, China
•Space Science SatellitesKanagawa, Japan
•Solar ActivityMeudon, France
•SoilsWageningen, The Netherlands
•Sunspot IndexBrussels, Belgium
•Solar Terrestrial PhysicsBoulder CO, USADidcot Oxon, UKMoscow, RussiaHaymarket, Australia
•Solid Earth GeophysicsBeijing, ChinaBoulder CO, USAMoscow, Russia
Where and how do you get data?
• Ok, now you have been (hopefully) listening for some time, but how can you have access to environmental data?
• You have 15-20 minutes in order to find environmental data using the internet
Good luck
What are scientific data and how can they be structured?
Age model [kyr]
What are data ?
0.0900.0900.1000.150
:::
Age, error [kyr]
1.32
6.73
::
Sed rate [cm/kyrs]
:3.6
1.670.050.100.15
:21.2121.70
:
Age, dated [kyr]
Depth [m]
0.0150.0750.0750.620
:3.3103.355
:
Dated material
G. inflataG. bulloides:::N. pachyderma sin.N. pachyderma sin.
:
:::::
DataSet title: VogelsangE et al 2001/Age control of sediment core V23-81
Reference: Broecker, WS et al (1988): Preliminary estimates for the radiocarbon age of deepwater … Bond, GC et al (1993): Correlations between climate records …, Nature, 365: 143-147 Sarnthein, M; Winn, K; Jung, S J A; Duplessy, J C; Labeyrie, L D … (1994): Changes in east…
Project: Glacial Atlantic Mapping and Prediction (GLAMAP2000)
Event: V23-81 * LATITUDE: 54.2500 * LONGITUDE: -16.8300 * ELEVATION: -2393.0 * DATETIME: 18 Oct 1966 00:00:00 * GEAR: Piston corer, unspec. * CAMPAIGN: V23
Parameter: Age, dated - Age, dated [kyr] * … METHOD: AMS 14C dating. Broecker et al. 1988. … Dated material - Age, dated material * PI: Sarnthein Sed rate - Sedimentation rate [cm/kyrs] * PI: Sarnthein * METHOD: calculated
PI: Sarnthein, Michael, e-mail: [email protected]
Source: PANGAEA - DataSet ID: 59872
Data details: http://www.pangaea.de/Cores/Age/V23-81.pdf
Metadata – describing your data
Principal investigator(s) (PI), Project(s)who
what
where
when
how
Data types, Parameter [unit]
Methods
Spatial coverage -> geographical positions
Temporal coverage ->
Title, Identifier (DOI)
Reference(s)
Quantities
Sampling event, Campaign, Location
Ratio
Ordinal
Nominal
Level of scale
Interval
e.g. Kelvin scale { 15.456; -3.2; 760; 0 }
e.g. Fahrenheit scale
Semiquantitative, rank-ordered, intervals may not be equal
e.g. { male; female } { red; green; blue }
Quantitative, zero included
Quantitative, no zero, equal intervals (addition, subtraction), but no proportions
e.g. { first; second; third } { rare; frequent; abundant }
Qualitative, no ordering implied
numerical data
text data
pictures
tertiary data
secondary data
primary data
interpretations, aggregated data (e.g. timeslices)
raw data (e.g. counts, d18O)
calculated from raw data (e.g. paleotemperatures)
Technical
Processing level
Classification schemes
SSTMG/CA
SSTformam
SSTalkenone
What are geocodes?
LATITUDE (decimal degrees)LONGITUDE (or degree, minute, second)UTM (Universal Transverse Mercator)
DEPTH, sediment [m]DEPTH, ice/snow [m] DEPTH, water [m b.s.l.]ALTITUDE [m a.s.l.]ELEVATION [m a.s.l.]ORDINAL NUMBER eg. Tree ringDISTANCE [cm]
DATE/TIMEAGE [kyr BP]
Spatial
Temporal
OceanShelf
LandLake
Ice
Depth in sediment
Corals (distance)
Altitude / Elevation
Depth in water
Outcrops (depth, distance,ordinal number)
Depth in sediment
Depth in ice
Depth in water
Depth in ice
Geocodes – the third dimension
Trees (ordinal number)
Warves (ordinal number)
Geocodes – temporal
GEOLOGIC AGE
TreesWarves
DATE/TIME Calendars & timezones
chronography
Corals
relative age datingbio- / lithostratigraphy
Absolute age datingradiometric time scale
nominal ages absolute ages
Ways to archive data
File systems
Relational databases (RDBs)
Technical data organisation
advantage: fast & cheap archiving procedure (on a short run)disadvantage: low consistency of data
disadvantage: work intensive archiving procedure, needs high degree of data organization usage for mass data is limited
advantage: high consistency of data, low costs for data curation, good retrieval qualities
Mixed
Relational database -> geocoded data & metadata
File system -> mass data (geophysical data, pictures, films)
How to make data available to science?
Possible problems in retrieving data from the net
• Version conflicts (data is archived in many data centres – in different stages e.g. raw data, quality controlled, etc.)
• Bad documented metadata and data (methods, units, unclear parameter definitions, etc)
• Just metadata is available online – data has to be requested• Naming of cruises varies in many countries > hard to identify same
cruises• Date formats (mm/dd/yyyy; yy/mm/dd; dd/mm/yyyy etc)• Ways to report the position (Lat/Long, UTM)• Different export formats (plain text, xml, netCDF, etc)• Different entities (one data set = data from one cruise or data from one
station or data from one)• Data set is too large to be downloaded (e.g. model data)
Result: Can take a lot of time to create large homogenic data collections!
(Some) important WDCs for environmental data
• WDC for Atmospheric Trace Gases Carbon Dioxide Information Analysis Center USA
• WDC for Climate Model and Data Max-Planck-Institute for Meteorology GERMANY
• WDC for Glaciology, Boulder University of Colorado USA• WDC for Marine Environmental Sciences Center for
Marine Environmental Sciences (MARUM) GERMANY• WDC for Marine Geology & Geophysics, Boulder USA• WDC for Oceanography, Silver Spring USA
Remember that WDC is a status!
There are many national and international data centres as well which are no WDC e.g.
ICES – International Council for the Exploration of the Sea, Denmark
BODC – British Oceanographic Data Centre, UKBADC – British Atmospheric Data Centre, UKNODC – National Oceanographic Data Center,
USANMD - Norsk marint datasenter, Norway
World Data Center for Marine Environmental Sciences (WDC-MARE) at University of Bremen, Germany
• is aimed at collecting, scrutinizing, and disseminating data related to global change in the fields of environmental oceanography, marine geology, paleoceanography, and marine biology. It focuses on georeferenced data using the information system PANGAEA. The WDC stores and handles numeric, string, and image data. Users can retrieve data through the Internet via different gateways.
• offers data management services, in particular project data management and data publication. It maintains an inventory of site and sampling locations for all related fields. It provides hosting and mirroring of electronic journals and serves software products for analyzing, visualization, and transformation of data.
How to access dat via WDC-MAREhttp://www.wdc-mare.org/ or http://www.pangaea.de/
• Data is available via www using the search engine PangaVista www.pangaea.de/PangaVista
• use it like • E.g. Search by parameter, scientist, region,
project, research vessel, institute, etc
You can either view the data online
Or dowload the data
Nice, but what else can I do with the data
Since all data at WDC-MARE is archived in a relational database it can be easily converted to other formats like:
Ocean Data View
ArcGIS
PanPlot (Open Source plotting software)
Pan2Application – converter for data from WDC-MARE
Ocean Data Viewhttp://odv.awi-bremerhaven.de/
Ocean Data View (ODV) is a software package for the interactive exploration, analysis and visualization of oceanographic and other geo-referenced profile or sequence data.
PanPlothttp://www.pangaea.de/Software/PanPlot/
Networking between different data holders is essential
The user can use one website in order to find metadata and data that is archived in many different data centres
Global Change Master Directory
Gives access to metadata, but can be hard to find the data
Gives access to metadata and links directly to the data set
Thanks for listening!
Questions?Comments?