ArcGIS for Server in the Amazon Cloud - Recent …...ArcGIS for Server in the Amazon Cloud, 2015...

ArcGIS for Server in the Amazon CloudTom Shippee, Sridhar Karra

Before we begin…

• Who are we?- Tom Shippee – Tech Lead Training Services- Sridhar Karra – Tech Lead Esri Managed Cloud Services

• Who are you?- No Amazon experience looking to learn more?- How many current Amazon Web Services (AWS) users?- How many ArcGIS Server on AWS users?- How many attended this session last year?

Agenda

• Cloud computing overview

• Ways to deploy ArcGIS Server on AWS- Cloud Builder: Quick & Easy wizard driven application- Amazon Console: For those familiar with AWS administration- CloudFormation: Scripted flexibility and repeatability

• Key take aways- Deployment architectures- AWS tips and tricks

Cloud computing Overview & Context

Types of cloud environment

IT Infrastructure

ArcGIS Server o Amazon Web Serviceso Microsoft Azure

IaaSInfrastructure as a

Service

Application Development

ArcGIS REST services

PaaSPlatform as a Service

Increasing User Responsibility

Business Processes

ArcGIS Online

SaaSSoftware as a Service

Benefits of IaaS cloud computing

• No physical infrastructure

• Pay for what you need, when you need it

• On demand scaling

Time

Web service demand

$$$

$

$$$$$$

$$$$

$$

What are Amazon Web Services?

Key EC2 terminology

• AMI: Amazon machine image

• EC2 Instance: machine rental service

• Security group: firewall access rules

• ELB: elastic load balancer

• Auto-scaling: automatically add/remove machines

• Region & Availability Zone: where your system runs

Esri AMIs

N

Security Group (firewall)

Auto-Scaling

EC2Instances

AWS Region (e.g., Oregon)

Availability zone-2

Availability zone-1

AWS AMIs

ArcGIS Server AMIs in Amazon Web Services

Ubuntu + ArcGIS Server + PostgreSQL

Windows Server + ArcGIS Server + SQL Server

Windows Server + ArcGIS Server + SQL Express

Esri AMI My Services Application Custom AMI

Licensing

• Amazon- Virtual hardware

- Operating system

- Enterprise database

• Esri- ArcGIS Server

- ArcGIS Desktop

Cloud BuilderAWS Beginner

DemoCloud Builder

Who should deploy Cloud Builder?

• New to IaaS cloud computing

• Need to deploy VERY quickly

• Desire to minimize hands-on AWS administration

• Wish to leverage multi-node ArcGIS Server site

AmazonWeb Services

Amazon ConsoleAWS Administrator

Who should deploy using the Amazon Console?

• Experienced AWS administrators

• Want more control over AWS services

• Flexibility to deploy various architectures

Amazon CloudFormationScripted cloud deployment

DemoCloud Formation

Who should deploy using Amazon CloudFormation?

• Developers or Advanced AWS Administrators- Desire maximum flexibility- Require repeatable consistency across multiple deployments- Control and track changes to your infrastructure

• Large enterprise deployments (dev, staging, pre-production, production)

• Fully supported platform by Esri

Comparing AWS deployments

Cloud Builder AWS Console CloudFormation

Typical user AWS beginner Experienced AWS admin AWS Admin or DevOps

Architecture Single ArcGIS Server site Architectural flexibility Architectural flexibility

Strengths Easy to set up & maintain

Very fast deployments

Leverage a multi-node site

Black box view of AWS

Leverage & integrate wide variety of AWS components

Complete freedom to customize AWS solutions

Stronger fault tolerance

Repeatable deployments

Flexibility to refine scripts for different uses

Leverages a powerful AWS-base API

Challenges Limited AWS flexibility Requires AWS knowledge

More planning to deploy

Managed via AWS only

Requires scripting/coding knowledge

Longer learning curve

More planning to deploy

Deployment Architectures

AWS Security Group

ELB

Site Site

AZ 1 AZ 2(Multi-AZ deployment)

Multi-site siloed

ELB Site

Cloud Builder single site

...

Deployment Architectures

NSite

ELB

Site Site Site

AWS Security Group

SQL “HA” RDS

ELB

RDS

N

AZ 1 AZ N

AZ 1 AZ N

AZ 1 AZ N Multi-AZ deployment

Tips and tricks

• Large Datasets (TB) – Imagery or large cache- AWS import/export - Back up data before shipping the drives

• Sizing AWS instances- CPU and RAM- Auto scaling

• EBS Drives- Max 1 TB- Use striping or mirroring – if needed

• Custom counters- System reboots, RAM usage, disk usage, process monitors, CPU thresholds

• Termination protection- Enable it (production systems)

• Detailed monitoring- Enable it (production systems)

Tips and Ttricks• Auto-scaling

- Plan for the Worst- Pick the right instance type when AS - launch script- Min, max limit for auto-scaling- Time needed to launch a new fully functional instance- Termination Policy: Yes or no?

- AGS / IIS logs – usage logs?

- Licensing- Check with your account manager.

• Alerts and notifications- System counters- Cloud Watch

• Remember this is cloud and not your traditional data center.- Scale up both vertically and horizontally when needed- Keep cost low

• Monitor.. monitor.. monitor..

Site-Host vs. Siloed Architecture

• Site-Host Approach:- Use Cloud Builder- Acceptable downtime window- Caching Jobs- DR: AGS Backups, Cloud Builder Backups, Cloud Builder Templates- Publish Once

• Siloed Approach:- Cloud Builder or AWS Console or Cloud Formation- True SLA’s or SLA’s with penalties (HA) or Multi Region deployments- Fine grain control over AWS (EIP, ELB’s, Security Groups, Route 53, etc)- DR: AGS Backups, AMI’s, Templates (Cloud Builder or Cloud Formation)- Publish to each AGS Server in your deployment

Tools

• System Monitor:- Free Tool by Esri PS- Key Performance Indicators (KPI), System Counters, DB Counters, AGS Statistics,

etc.- http://www.arcgis.com/home/item.html?id=848f48b0f88e4de7a036377197453efe

• System Designer:- Free Tool by Esri PS- Design and Capacity Planning- http://www.arcgis.com/home/item.html?id=8ff490eef2794f428bde25b561226bda

• AWS CloudWatch, AWS SES, AWS SNS, etc.• PowerShell Scripts to perform custom counters:

- http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/mon-scripts-powershell.html

Reference Materials

• What is ArcGIS Server on AWS? http://server.arcgis.com/en/server/latest/cloud/amazon/what-is-arcgis-server-on-aws.htm

• ArcGIS CloudBuilder introduction: http://server.arcgis.com/en/server/latest/cloud/amazon/overview-cloud-builder.htm

• ArcGIS Server CloudFormation template examples: http://server.arcgis.com/en/server/latest/cloud/amazon/example-cloudformation-template-for-amazon-vpc.htm

• Amazon CloudFormation documentation: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/gettingstarted.templatebasics.html

• RDS Instances: http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_VPC.html

• http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_SQLServer.html

• Getting Started with AWS: http://aws.amazon.com/getting-started/?nc2=h_l2_cc

• AWS FAQS: http://aws.amazon.com/faqs

Surveys: Please rate this session…

• http://www.esri.com/RateMyDevSummitSession- Search: “ArcGIS Server in the Amazon Cloud”- 10 seconds- Comments really welcome