Upload
rachel-haynes
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
Virtual Geophysics LaboratoryExploiting the Cloud and Empowering Geophysicists
Ryan Fraser, Terry Rankine, Lesley Wyborn, Joshua Vote, Ben Evans.Presented by Joshua Vote
March 2013
CSIRO | MINERALS DOWN UNDER FLAGSHIP
Virtual Geophysics Laboratory | Joshua Vote
Rough Computational Science WorkflowData +
2 |
Methods + Resources
The current state of the data problem...Common Platform
MarineEnvironment, Water
Groundwater GeologyGeophysics
Virtual Geophysics Laboratory | Joshua Vote3 |
Virtual Geophysics Laboratory | Joshua Vote4 |
ChemicalReactions
Coupled Processes
Rock Mechanics
Fluiddynamics
DeformationSurface
ProcessesFluidFlow
SocietalNeed
NumericalBasis
Modelling Codes
ModellingServices
Base Scientific Concepts
Finite Volumes
Inundation
Natural HazardsNatural Hazards
Resources & Mining
Resources & Mining
SustainableEnergy
SustainableEnergy GeodynamicsGeodynamics
TsunamiStorm Surge
Finite Element
Solver(Fastflo)
Reactive Transport
Predictive ore deposition
Mine Waste Disposal
Finite Element Solver (Finley)
Scripting environment for Finite element simulations
(E-Script)
Mine design
Earthquake
MantleConvection
FiniteElements
Chemistry(Log K)
Geodynamics Modelling code
Underworld
Finite Element
Solver(StGFEM)
ESysCrustal
Coupled Mechanics & Reactive Transport
Particle in Cell
Slabsubduction
MantleConvection
Code(CitcomS)
FiniteDifference
Visual-isation
(gLucifer)
SurfaceModelingPackage
(SPModel)
Chemistry(Gibbs)
Particle
Vulnerability Modelling
Interpolation Mesh
Recipes for solving a problem
Processing
Virtual Geophysics Laboratory | Joshua Vote5 |
Virtual Geophysics Laboratory | Joshua Vote6 |
Introducing The Virtual Geophysics Laboratory
Data discovery
Virtual Geophysics Laboratory | Joshua Vote7 |
Layers discovered via remote registries
Layers consist of numerous remote data services
Data discovery
Virtual Geophysics Laboratory | Joshua Vote8 |
Some data services support subsetting
Some data services support reformattinge.g. CSV, NetCDF, GeoTIFF
Data discovery
Virtual Geophysics Laboratory | Joshua Vote9 |
Some data is only registered with flat files
What just happened?
Virtual Geophysics Laboratory | Joshua Vote10 |
Data processing
Virtual Geophysics Laboratory | Joshua Vote11 |
A variety of different scientific codes are already available in the form of “Toolboxes”
Flexibility in what computing resources to utilise
Data processing
Virtual Geophysics Laboratory | Joshua Vote12 |
Further input files can be uploaded.
Input files are passed directly into the cloud
Data processing
Virtual Geophysics Laboratory | Joshua Vote13 |
The steps so far have been building an environment to run a processing script
...or build from existing templates
Either write your own...
What just happened?
Virtual Geophysics Laboratory | Joshua Vote14 |
Processing script/ small input files uploaded
Start processing
Download big data sets
Perform data processing
Download Job Script/user input files
Upload processing results
Managing results - provenance
Presentation title | Presenter name15 |
All of a job’s outputs are also accessible
Each job has a lifecycle that can be managed
Virtual Geophysics Laboratory | Joshua Vote15 |
A job’s console log can be inspected
Managing results - provenance
Virtual Geophysics Laboratory | Joshua Vote16 |
Successful jobs can have their entire process captured in a ISO 19115 ‘provenance record’
Each provenance record tracks all inputs, outputs, processing scripts and other metadata.... Spatial bounds...
Input/output data...
What just happened?
Virtual Geophysics Laboratory | Joshua Vote17 |
What’s the processing status?
What are the job input/outputs?
Publish the job’s process and results
Cloud storage will persist the final artefacts
Still under construction
Virtual Geophysics Laboratory | Joshua Vote18 |
Courtesy - www.textfiles.com
What’s left?
• A Common Processing Services Platform • Analytical Spatial Information Services Stack (ASISSt)
• Virtual Laboratory client toolkit
• VGL Improvements• More data and models• HPC Integration
• Your code here!
Virtual Geophysics Laboratory | Joshua Vote19 |
AnalyticalASISSt
+
Thank you
Joshua VoteSoftware Developer
e [email protected] vgl.auscope.org
VGL is Powered by:
VGL acknowledges funding from the NeCTAR project http://www.nectar.org.au NeCTAR is an Australian Government project conducted as part of the Super Science initiative and financed by the Education Investment Fund.
VGL is a collaboration between:
VGL is supported by: University of Queensland, Monash University and the Australian National University