Click here to load reader

OGCE Workflow and LEAD Overview Suresh Marru, Marlon Pierce September 2009

Embed Size (px)

DESCRIPTION

Suresh Marru LEAD Pioneering Technology

Citation preview

OGCE Workflow and LEAD Overview Suresh Marru, Marlon Pierce September 2009 Suresh Marru The LEAD Gateway Suresh Marru LEAD Pioneering Technology Suresh Marru Streaming Observations Storms Forming Forecast Model Data Mining Refine forecast grid Instrument Steering LEAD Dynamic Adaptive Infrastructure Suresh Marru LEAD Scientists and Educational Interactions Developers Researcherss End Users Lowering the barrier for using complex end-to-end weather technologies Democratize Empower Facilitate Suresh Marru Analyze & Predict Research & Reproducibility Education & Outreach Discover &Visualize Suresh Marru Gateways/E-Science Community TG GIG Gateways (LEAD, GridChem, ) TeraGrid User Portal Generalize, Harden, Build Test Open Grid Computing Environments Suresh Marru Middleware Services Compute & Data Resources User Interactions Computational Cloud GFac Services Local Lab Resources Other ClientsXBaya GUI XBaya Core Web Portal Computational Grids Event Bus Workflow Engine (ODE) XRegistry XMCCat Metadata Catalog Flexible Layered Service Oriented Architecture Suresh Marru Application Wrapper Framework Scientific Applications are wrapped into web services by filling in a web- form. The Application Factory generates a web service for each application with I/O interfaces. Registers WSDL for the service with a registry Each service generates a stream of notifications that log the service actions back to the XMCCat Metadata Catalog, user monitoring, and provenance tracking toolscc Application Factory Run program & publish events App Service Suresh Marru Application Wrapper Architecture Suresh Marru Service Monitoring via Events The service output is a stream of events I am running your request I have started to move your input files I have all the files I am running your application The application is finished I am moving the output to you file space I am done These are automatically generated by the service using a distributed event system (WS-Eventing / WS-Notification) Topic based pub-sub system with a well known channel Application Service Instance Application Service Instance Notification Channel Notification Channel listener publisher x x x x Suresh Marru Workflow Suite Architecture Suresh Marru Composition and Monitoring Abstract DAG Model BPEL 1.1 BPEL 2.0 SCUFL Python Dynamic Enactor/Interpreter Jython Based Enactor GPEL Engine Apache ODE Engine Taverna Python Runtime Message Bus Interoperable Xbaya Architecture Suresh Marru 14 OVP/ RST/ MIG Fault Tolerance/ Recovery Service Application Performance Models Resource Reliability Models NWS, MDS BQP Deadline & Success Probability Portal BPEL Workflow Engine Application Service Notification Service Simple Recovery Architecture Suresh Marru Workflow Composer Suresh Marru Example: Assimilation Workflow Suresh Marru Domain webs of large genomes Input list of amino acid sequences Identify all known domains Construct webs Parallel processing Capability-type computing Ensemble-type processing (minimal network reqs) Capacity-type computing Science Independence Example Suresh Marru Biology Gateway Suresh Marru Polar Grid Example Suresh Marru LEAD Lower entry barrier to using weather analysis tools Improve detection, analysis & prediction of mesoscale weather Motif-Network Transformation of sequenced genomes to domain-space Cyber-Infrastructure Evaluation Performance evaluation of future supercomputer architectures ADAM Algorithms for feature extraction, data normalization, classification and normalization GridChem Molecular Chemistry Grid helping researchers run chemistry applications on Grid Environment Example Gateways using OGCE Workflow Suite Suresh Marru Live Demo & Questions?