Upload
ivandatasynapse
View
4.933
Download
0
Embed Size (px)
Citation preview
1
A Technical Overview
Leveraging Amazon Web Services with DataSynapse GridServer
2
Overview
• On Wednesday February 11th, DataSynapse announced support for Amazon EC2 and Amazon Web Services external cloud platform
- Customers will be able to run unlimited number of DataSynapse engines in the EC2 cloud
• As part of this announcement, DataSynapse is launching a beta program for Federator 1.5
- This new release of the Federator grid management software makes integration with external clouds seamless
- Beta program members can extend there current DataSynapse environment into the Amazon EC2 cloud, free for all of 2009
• The objective of this strategy is to accelerate the adoption of external cloud computing models for mission critical business applications and lower the operating expenses for grid computing
3
GridServer and AWS
Technical Overview
4
• High performance service execution
• Modern, open and extensible
• Multi-platform, multi-language
• Self-managing:- HA/Failover- Deployment and
provisioning- Monitoring
• Large platform ecosystem of premier ISVs
GridServer – Service Execution Environment
Grid Clients
DataSynapseManager
5
Federator – Grid Engine Management
DataSynapseFederator
Federator UI• Policy based re-distribution of
resources between resource pools
• Centralized resource management
GridServer 4.2 (U16) GridServer 5 (SP2) EC2 Engines
• Enables running additional engine pools in the Amazon EC2 cloud
6
Federator - Integrating with Amazon WS
Grid Client Federator
DataSynapseManager
On Premise In the Cloud
Federator 1.5 extends GridServer seamlessly onto the Amazon cloud
- Save all your Amazon configuration information
- Create policies for starting / stopping cloud based engines
- Create secure connections to cloud based engeines
7
Activate Amazon EC2 Account
• Create an Amazon AWS account- Web-based sign up, with credit card
• Sign up for DataSynapse paid AMI- Retain your Activation Key
• Gather your Amazon keys- Access Key ID- Generate public and private keys
8
Setting Up Federator
• Install Federator software
• Configure Federator for your GridServer Manager
• Define a Virtual Resource Manager- Basic information necessary to
interact with EC2 (e.g., AWS login information)
• Define a Virtual Resource Group- More dynamic configuration data, information
specified in VRG closely tied to how customerswill actually use virtual nodes
• Define Policy- Assign Virtual Resource Group to target Resource Manager
9
Starting Up the Proxy Service
• Federator uses AWS interface to create a cloud-side proxy based on Virtual Resource Manager Settings
On Premise
On AWS
• On premise proxy, installed as a service, will initiate secure cloud connections
AWSS3 Storage
ProxyService AMI
• AMI (Amazon Machine Image) for the Proxy is stored in S3, DataSynapse incurs the costs of this storage, not the client
AWS EC2Instance
Proxy Service
AWS Interfaces
FederatorDataSynapse
Manager
Proxy Service
10
Proxy Service
• Once proxy is running, Federator instructs the GridServer Manager to establish an SSH tunnel connection to the proxy
Federator
On Premise
On AWS
DataSynapseManager
Proxy Service
Secure SSHTunnel
Secure connection point between on premise and cloud based resources
Proxy Service
• Proxy model enables secure connectivity between GridServer Manager on premise and Amazon EC2 based DataSynapse resources
11
OpenVPN Alternative
• Amazon recommended model for extending your network into the cloud
• Ethernet bridging enables cloud resources to securely access on premise resources
• DataSynapse proxy service is used for resource caching
Federator
On Premise
On AWS
DataSynapseManager
Secure VPN connection
VPN Server
VPN Bridge Client
Caching Service
12
DataSynapse AMIs
• Standardized machine images- Common engine configuration - Supporting services
• Operating Systems- Windows 2003- Fedora - OpenSUSE
• AMI sets stored in each Region- Windows 2003 unavailable in EMEA
• Users may build customized AMIs:- Must be based on default AMIs - Storage and associated costs of
custom AMIs are the responsibility of the client
AWSS3 Storage
AWS EC2Instance
DS EngineWindows
2003 (64bit)
Fedora (32bit)
OpenSUSE (64bit)
Fedora (64bit)
OpenSUSE (32bit)
13
DataSynapse Engines on EC2
When activation policy executes:
• Federator launches specified # of engine AMI instances
• Federator configures each engine to use the Proxy
Activating Engines on Amazon EC2 Cloud
On Premise
On AWS
DS Manager
Proxy Service
Secure SSHTunnel
DS Base AMI
DS S3 Account
Custom AMI
Client S3 Account
Proxy Service
Federator
Activation Policy
Cloud based engines do not count against
GridServer license counts
14
• Proxy synchronizes grid library resources with the Manager
• Grid libraries are then cached on the Proxy side for distribution to EC2-based Engines
Uploading Grid Libraries
Federator
On Premise
On AWS
DataSynapseManager
Proxy Service
Secure SSHTunnel
GridLibs
GridLibs
Proxy Service
15
• When an Engine comes online they will first check with the proxy for any GridLibs
• Updates to engines are done within the EC2 environment, there are no additional costs associated with this
Configuring Engines on Amazon EC2 Cloud
Federator
On Premise
On AWS
Proxy Service
Secure SSHTunnel
GridLibs (cached)
Engines Instances
Proxy Service
DataSynapseManager
16
Engines Instances
Submitting Work to Cloud
Federator
On Premise
On AWS
DataSynapseManager
Proxy Service
Secure SSHTunnel
• Clients submit work to the GridServer Manager as before, no change required
• Engines then request work from the GridServer Manager via the Proxy
• Engines will receive start/stop service commands directly from the GridServer Manager
Grid ClientProxy Service
17
Use Cases
• Dev and Test Grids:
- GridServer environments are large – some customers running 10k+ engines
- Difficult to duplicate the scale of environment for testing
- GridServer and EC2 provides a low cost solution
• Disaster Recover Grids:
- Again with large environments – it is difficult to duplicate for DR function
- GridServer and EC2 provides a low-cost, pay only when needed option
• “Flex-Service”:
- If workloads are not meeting desired SLAs – it is expensive and time consuming to add more
compute capacity to crowded, expensive corporate datacenters
- GridServer and EC2 provides a low-cost, solution for ad-hoc workload surges
18
Beta Program Overview
19
Federator 1.5 – Beta Program
• Program runs from April ‘09 through November ’09
• Registration will close on March 4
• Space is limited
• $25K registration fee
20
Program Benefits
• Free use of DataSynapse Engines Running on Amazon EC2*
• No limit on DataSynapse Engines Running on Amazon EC2
• DataSynapse Engines Running on Amazon EC2 don’t count against existing license caps
• Access to Federator 1.5 Beta, with periodic updates
• Discounts on GA version of Federator 1.5
*Amazon Infrastructure Charges Still Apply
21
Pricing Post-Beta
• No limit on DataSynapse Engines Running on Amazon EC2
• DataSynapse Engines Running on Amazon EC2 don’t count against existing license caps
• Engine charges are $0.10 per hour, plus Amazon infrastructure charges*
*Dependent on Amazon Instance Selected
22
Q & A