Upload
imogen-horton
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
Ocean Observatories Initiative
Common Execution Infrastructure (CEI)
Overview
Michael Meisinger
September 29, 2009
OOI CI Kick-Off Meeting, Sept 9-11, 20092
Outline
• Subsystem Architecture Overview• Scope of Release 1• Selected Components
– Elastic Computing– Global Network Strategy
3OOI CI Kick-Off Meeting
Sept 9-11, 2009
Common Execution Infrastructure
• Provides the execution and deployment environment for CI services and user processes
Common Operating Infrastructure
Data Management
(Science)
Sensing & Acquisition
Data Management (Information Distribution)
Analysis & Synthesis
Identity Management
State Management
GovernanceFramework
Resource Management
Planning & Prosecution
Exchange
Service Framework
Presentation Framework
Common Execution
Infrastructure
Deployment Scenario
4OOI CI Kick-Off Meeting
Sept 9-11, 2009
Scope of Release 1• Virtual Process Execution Management
– Definition of target execution environment independent processes
– Support of multiple cloud execution environments– Generation of target specific executables
(images)• Elastic Computing
– Elastic scheduling, provisioning, execution– Monitoring and management
• Resource Management– Compute Nodes and Target Environments– Process Repository
OOI CI Kick-Off Meeting, Sept 9-11, 20095
Elastic Provisioning Example
6OOI CI Kick-Off Meeting
Sept 9-11, 2009
• Imperative to separate mechanism from (smart, specialized) policy
Cloud Provisioning Architecture
7OOI CI Kick-Off Meeting
Sept 9-11, 2009
CEI Functional Components
Common Execution Infrastructure
Process Execution & Resource Provisioning
Service Agreement Proposal
Service Agreement Proposal
Status
Fault Analysis
Processing Plan
Deployable UnitDeployable
Type
Execution Environment
Adapter
Fault Monitor
ProvisionerComputation
SchedulerComputation
Controller
OperationalUnit
Execution Engine
Deployable Unit,Control
Status,Fault
Process Definition,Control
Status,Fault
Deployable Type
Deployable Type Repository
Data Management (Information
Distribution) SN
Exchange
Process Definition Repository
Process Definition
Processing Service Requestor
Processing Plan
Operational Unit
Operational Unit
Process Execution & Resource Provisioning
Fault MonitorProvisioner
Execution Resource
ContextualizerExecution
Monitor & Control
Deployable Unit
Operational Unit Status
Operational Unit Status
Control Commands
Service Agreement Proposal
Registration
Resource & Context
Description
ExecutionContext
Service Agreement Proposal
Computation Controller
Execution Engine
Process Execution & Resource Provisioning
Execution Engine
Processing Plan
Fault MonitorComputation
SchedulerComputation
Controller
Execution Engine Dispatcher
Execution Engine Processor
Execution EngineFault Monitor
Service AgreementProposal,
Process Definition
Process Definition
Processing Status
Processing Status
Fault Analysis
Service Agreement Proposal
Example: Data Exchange
Deployment Concepts
12OOI CI Kick-Off Meeting
Sept 9-11, 2009
Operational Architecture
DeploymentArchitecture
ImplementationArchitecture
Execution Engines
13OOI CI Kick-Off Meeting
Sept 9-11, 2009
Deployment - Execution
14OOI CI Kick-Off Meeting
Sept 9-11, 2009
Cloud Provisioning Basics
15OOI CI Kick-Off Meeting
Sept 9-11, 2009
Cloud Instance Startup
16OOI CI Kick-Off Meeting
Sept 9-11, 2009
Observe–Decide–Act Pattern
17OOI CI Kick-Off Meeting
Sept 9-11, 2009
Network Logical Deployment Model
18OOI CI Kick-Off Meeting
Sept 9-11, 2009
Component Deployment Model
19OOI CI Kick-Off Meeting
Sept 9-11, 2009
Hardware Deployment Model
20OOI CI Kick-Off Meeting
Sept 9-11, 2009
Network Architecture
22OOI CI Kick-Off Meeting
Sept 9-11, 2009
Network Deployment
Academic Global Gigabyte Network
Academic National Gigabit Network
OOI Operational LocationsMicrosoft
AmazonWest
AmazonEast
Woods Hole
Seattle
Washington DC
San Diego
Portland
Corvallis
Management Point
Acquisition Point
Distribution Point
Execution Point
Technology Mapping
• Cloud Execution Sites– AWS/EC2, Science Clouds (Nimbus)– GoGrid, FlexiScale (optional)
• National Grid Resource Interface– Condor for high throughput jobs– Teragrid Gateways
Functional Component Technology TRLGrid Scheduler Condor 9Cloud Computing Environment Amazon EC2 9Cloud Computing Environment Nimbus 8VM Packaging & Distribution Elastic Server (CohesiveFT) 8Virtualization Layer Software Individual virtualization vendors, e.g., Amazon AMI)9
OOI CI Kick-Off Meeting, Sept 9-11, 200927
Thanks !
Cloud Computing Strategy• Requirements
– a secure, scalable, fault-tolerant network and computing infrastructure that spans the OOI marine instrument networks to the national (academic & commercial) computing/data facilities
– a participation model that allows individual research teams to integrate the computing environments at their intuitions into this infrastructure
• Strategy– provide an effective development and operational process that allow the
OOI members and its operational staff to transparently use and effectively manage the OOI specific computing resources as a coherent cloud resource
• Definition– style of computing where ICT capabilities are provided “as a Service”, – allowing users to access technology-enabled services without
knowledge of, expertise with, or control over the technology infrastructure that supports them
Cloud Strategy Benefits• Attributes
– Effective coupling of resource acquisition to user demand– Streamlining the preparation, deployment and management of
observatory applications– Removing the management and ownership of physical
computing resources from the direct concern of the Observatory• Results
– Eliminate centralized capital assets associated with networking storage, and computing as well as associated staffing
– Continual transparent tracking of the technology advancements– Ability to respond to changes in usage demand in minutes and
changes in market costs within weeks
Architectural Concerns• Operational
– Enterprise of Applications, Services and the deployed Modules to support the Enterprise
• Deployment (concern of CEI)
– Secure, Scalable, Self-healing Execution Platform
• Implementation (concern of CEI)
– Configuration framework to compose and condition specific classes deployable execution units (Capability Containers)
• Functional– Decomposition of the system based on functional and
construction concerns
CEI Components• 40 min: Elastic scaling and provisioning
– Load graph: Start story. From there to CPE drawings or DX drawing. System scales according to load.
– Domain models development, deployment, operation architecture OV7 overview, execution engine OV7
– CPE use case• 20 min Network: Global site selection (Matthew)
– Deployment architecture– Detailed network diagrams (Brian)– National backbone infrastructure– GLIF (Tom) for network environment we’re deploying into.– Story: National DIF framework using Solace technology, link to
COI/Messaging. Higher density computing per enclosure, diversity of deployment. Possibility for inner core messaging architecture (very efficient on the backbone); commodity equipment
OOI CI Kick-Off Meeting, Sept 9-11, 200931