22
1 A Technical Overview Leveraging Amazon Web Services with DataSynapse GridServer

DataSynapse and Amazon EC2 Technical Overview

Embed Size (px)

Citation preview

Page 1: DataSynapse and Amazon EC2 Technical Overview

1

A Technical Overview

Leveraging Amazon Web Services with DataSynapse GridServer

Page 2: DataSynapse and Amazon EC2 Technical Overview

2

Overview

• On Wednesday February 11th, DataSynapse announced support for Amazon EC2 and Amazon Web Services external cloud platform

- Customers will be able to run unlimited number of DataSynapse engines in the EC2 cloud

• As part of this announcement, DataSynapse is launching a beta program for Federator 1.5

- This new release of the Federator grid management software makes integration with external clouds seamless

- Beta program members can extend there current DataSynapse environment into the Amazon EC2 cloud, free for all of 2009

• The objective of this strategy is to accelerate the adoption of external cloud computing models for mission critical business applications and lower the operating expenses for grid computing

Page 3: DataSynapse and Amazon EC2 Technical Overview

3

GridServer and AWS

Technical Overview

Page 4: DataSynapse and Amazon EC2 Technical Overview

4

• High performance service execution

• Modern, open and extensible

• Multi-platform, multi-language

• Self-managing:- HA/Failover- Deployment and

provisioning- Monitoring

• Large platform ecosystem of premier ISVs

GridServer – Service Execution Environment

Grid Clients

DataSynapseManager

Page 5: DataSynapse and Amazon EC2 Technical Overview

5

Federator – Grid Engine Management

DataSynapseFederator

Federator UI• Policy based re-distribution of

resources between resource pools

• Centralized resource management

GridServer 4.2 (U16) GridServer 5 (SP2) EC2 Engines

• Enables running additional engine pools in the Amazon EC2 cloud

Page 6: DataSynapse and Amazon EC2 Technical Overview

6

Federator - Integrating with Amazon WS

Grid Client Federator

DataSynapseManager

On Premise In the Cloud

Federator 1.5 extends GridServer seamlessly onto the Amazon cloud

- Save all your Amazon configuration information

- Create policies for starting / stopping cloud based engines

- Create secure connections to cloud based engeines

Page 7: DataSynapse and Amazon EC2 Technical Overview

7

Activate Amazon EC2 Account

• Create an Amazon AWS account- Web-based sign up, with credit card

• Sign up for DataSynapse paid AMI- Retain your Activation Key

• Gather your Amazon keys- Access Key ID- Generate public and private keys

Page 8: DataSynapse and Amazon EC2 Technical Overview

8

Setting Up Federator

• Install Federator software

• Configure Federator for your GridServer Manager

• Define a Virtual Resource Manager- Basic information necessary to

interact with EC2 (e.g., AWS login information)

• Define a Virtual Resource Group- More dynamic configuration data, information

specified in VRG closely tied to how customerswill actually use virtual nodes

• Define Policy- Assign Virtual Resource Group to target Resource Manager

Page 9: DataSynapse and Amazon EC2 Technical Overview

9

Starting Up the Proxy Service

• Federator uses AWS interface to create a cloud-side proxy based on Virtual Resource Manager Settings

On Premise

On AWS

• On premise proxy, installed as a service, will initiate secure cloud connections

AWSS3 Storage

ProxyService AMI

• AMI (Amazon Machine Image) for the Proxy is stored in S3, DataSynapse incurs the costs of this storage, not the client

AWS EC2Instance

Proxy Service

AWS Interfaces

FederatorDataSynapse

Manager

Proxy Service

Page 10: DataSynapse and Amazon EC2 Technical Overview

10

Proxy Service

• Once proxy is running, Federator instructs the GridServer Manager to establish an SSH tunnel connection to the proxy

Federator

On Premise

On AWS

DataSynapseManager

Proxy Service

Secure SSHTunnel

Secure connection point between on premise and cloud based resources

Proxy Service

• Proxy model enables secure connectivity between GridServer Manager on premise and Amazon EC2 based DataSynapse resources

Page 11: DataSynapse and Amazon EC2 Technical Overview

11

OpenVPN Alternative

• Amazon recommended model for extending your network into the cloud

• Ethernet bridging enables cloud resources to securely access on premise resources

• DataSynapse proxy service is used for resource caching

Federator

On Premise

On AWS

DataSynapseManager

Secure VPN connection

VPN Server

VPN Bridge Client

Caching Service

Page 12: DataSynapse and Amazon EC2 Technical Overview

12

DataSynapse AMIs

• Standardized machine images- Common engine configuration - Supporting services

• Operating Systems- Windows 2003- Fedora - OpenSUSE

• AMI sets stored in each Region- Windows 2003 unavailable in EMEA

• Users may build customized AMIs:- Must be based on default AMIs - Storage and associated costs of

custom AMIs are the responsibility of the client

AWSS3 Storage

AWS EC2Instance

DS EngineWindows

2003 (64bit)

Fedora (32bit)

OpenSUSE (64bit)

Fedora (64bit)

OpenSUSE (32bit)

Page 13: DataSynapse and Amazon EC2 Technical Overview

13

DataSynapse Engines on EC2

When activation policy executes:

• Federator launches specified # of engine AMI instances

• Federator configures each engine to use the Proxy

Activating Engines on Amazon EC2 Cloud

On Premise

On AWS

DS Manager

Proxy Service

Secure SSHTunnel

DS Base AMI

DS S3 Account

Custom AMI

Client S3 Account

Proxy Service

Federator

Activation Policy

Cloud based engines do not count against

GridServer license counts

Page 14: DataSynapse and Amazon EC2 Technical Overview

14

• Proxy synchronizes grid library resources with the Manager

• Grid libraries are then cached on the Proxy side for distribution to EC2-based Engines

Uploading Grid Libraries

Federator

On Premise

On AWS

DataSynapseManager

Proxy Service

Secure SSHTunnel

GridLibs

GridLibs

Proxy Service

Page 15: DataSynapse and Amazon EC2 Technical Overview

15

• When an Engine comes online they will first check with the proxy for any GridLibs

• Updates to engines are done within the EC2 environment, there are no additional costs associated with this

Configuring Engines on Amazon EC2 Cloud

Federator

On Premise

On AWS

Proxy Service

Secure SSHTunnel

GridLibs (cached)

Engines Instances

Proxy Service

DataSynapseManager

Page 16: DataSynapse and Amazon EC2 Technical Overview

16

Engines Instances

Submitting Work to Cloud

Federator

On Premise

On AWS

DataSynapseManager

Proxy Service

Secure SSHTunnel

• Clients submit work to the GridServer Manager as before, no change required

• Engines then request work from the GridServer Manager via the Proxy

• Engines will receive start/stop service commands directly from the GridServer Manager

Grid ClientProxy Service

Page 17: DataSynapse and Amazon EC2 Technical Overview

17

Use Cases

• Dev and Test Grids:

- GridServer environments are large – some customers running 10k+ engines

- Difficult to duplicate the scale of environment for testing

- GridServer and EC2 provides a low cost solution

• Disaster Recover Grids:

- Again with large environments – it is difficult to duplicate for DR function

- GridServer and EC2 provides a low-cost, pay only when needed option

• “Flex-Service”:

- If workloads are not meeting desired SLAs – it is expensive and time consuming to add more

compute capacity to crowded, expensive corporate datacenters

- GridServer and EC2 provides a low-cost, solution for ad-hoc workload surges

Page 18: DataSynapse and Amazon EC2 Technical Overview

18

Beta Program Overview

Page 19: DataSynapse and Amazon EC2 Technical Overview

19

Federator 1.5 – Beta Program

• Program runs from April ‘09 through November ’09

• Registration will close on March 4

• Space is limited

• $25K registration fee

Page 20: DataSynapse and Amazon EC2 Technical Overview

20

Program Benefits

• Free use of DataSynapse Engines Running on Amazon EC2*

• No limit on DataSynapse Engines Running on Amazon EC2

• DataSynapse Engines Running on Amazon EC2 don’t count against existing license caps

• Access to Federator 1.5 Beta, with periodic updates

• Discounts on GA version of Federator 1.5

*Amazon Infrastructure Charges Still Apply

Page 21: DataSynapse and Amazon EC2 Technical Overview

21

Pricing Post-Beta

• No limit on DataSynapse Engines Running on Amazon EC2

• DataSynapse Engines Running on Amazon EC2 don’t count against existing license caps

• Engine charges are $0.10 per hour, plus Amazon infrastructure charges*

*Dependent on Amazon Instance Selected

Page 22: DataSynapse and Amazon EC2 Technical Overview

22

Q & A