Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
The Cloud's Cutting Edge: ArcGIS for Server Use Cases for Amazon Web
Services David Cordes David McGuire
Jim Herries Sridhar Karra
Jim Herries
Atlas Maps
Atlas sample application
• The Esri Thematic Atlas Sample App is a configurable web application that uses a collection of intelligent web maps with text, graphics, and images to talk about our world.
Lots of maps needed, quickly
Key requirements
• Supply dozens of maps to an online atlas application • Stand up dozens of map services in two weeks • Enable team of map publishers • Option to iterate quickly on cartography, data, popups
CloudBuilder delivered value to the team instantly
• Data lead – knows the data but not CloudBuilder
• Cartographers on team knew color and cartography but not the data and not CloudBuilder
• We all knew ArcGIS server and publishing workflows
• None of us wanted to manage a server in a production situation
CB
743 items 16 apps
Lessons Learned
• Same day startup of server was a huge factor in project success • Teaching the team publishing methods was easy • Cache local, copy up FTP, now tile packages • Hundreds of cached services for the U.S. to 72k scale • 6 hours to cache the US down to 72 k • Use a checklist to save hundreds of hours’ time • Whatever data you have, you’re going to want additional attributes • Your corporate IT network staff are your friends (DNS changes, ports, etc) • Serve dynamic until popups and carto finalized • It’s just another server… • …but we don’t have to worry about it
An ongoing resource • http://atlasmaps.esri.com/arcgis/rest/
David McGuire
Going viral
Cloud Formation Demo
Crisis Thursday Saturday Friday
Servers Down
Traditional Trouble Shooting
• Reboot Servers? • Spread Services Across Local Machines? • Order New Equipment?
Panic!
Transition to the Cloud
Deployment
1 hr 15 mins
30 mins
15 mins
To Stage
To Generate An AMI
To Deploy Live Servers
ELB
ArcGIS Servers
Enterprise Geodatabase
Server
Handle Traffic
What We Learned
Start in the Cloud
Design for the Cloud
Plan Ahead
Avoid internal, static machines
Architecture should be cloud focused
Avoid emergencies
Aggregated Live Feeds…
Frequently Changing Data
How Can I Deploy and Scale?
Cloud Servers
Sync
Sync
Internal Processing
Server
S3 Bucket
Amazon
ELB
ArcGIS Servers
ELB
Web Application
Servers
Hurricane PIM
Hurricane PIM
• Over 1 Million hits in 1 day on our web map • Over 7 Million dynamic map requests in 1 day
Next Phase
Linux
Auto Scaling
Cloud Front
Linux instances are cheaper and boot faster
Avoid manual load management
Take advantage of global content distribution
Complication
Internal Machine
TMAppsEventsS3 S3 Bucket
Production Security Group
US East Region
Availability Zone B Availability Zone C
m1.small proxy server
m1.small proxy server
Internet
Elastic Load Balancing
Amazon CloudFront Distribution
Auto Scaling
VPC Subnet 1 VPC Subnet 2
Resources
• Esri Sample Templates - http://esriurl.com/EsriAWSTemplate
• Amazon Sample Templates - http://esriurl.com/AWSTemplate
• Aggregated Live Feeds - http://esriurl.com/LiveFeeds
- Make sure the “Show ArcGIS Desktop Content” box is checked
Sridhar Karra
Managed Services
Agenda
• Introduction • Program Overview • Customer Deployments
- Architecture - Lessons
• Questions
Program Overview
Extending the ArcGIS platform…
Esri Managed Services Cloud based GIS infrastructure support
REPORTING ARCHIVE
REDUNDANCY PERFORMANCE TESTING
HIGH AVAILABILITY
BANDWIDTH
MONITORING
HARDWARE CHANGE MANAGEMENT
DEPLOYMENT SECURITY
DATA MANAGEMENT
INFRASTRUCTURE
SYSTEM DESIGN SOFTWARE
BACKUP
SCALABILITY STORAGE
DISASTER RECOVERY
CACHING AGOL INTEGRATION
Experienced, Secure, Reliable, Scalable
• Rapid Deployment • Reduced cost of ownership
• Access to Enterprise GIS Expertise • Scalable Resources
• Platforms: - Public Cloud: AWS, Azure, Verizon, IBM - Private Cloud: Esri, Customer Private Cloud, CGI
• FedRAMP / FISMA Moderate Security Offering • ArcGIS Desktop in the cloud support
Road Ahead
Esri Managed Services Cloud based GIS infrastructure support
Esri Managed Services Cloud based GIS infrastructure support
Requirements Proposal / ROM
Staging
Contract
Verification
Questionnaire On Call Sizing / Arch. Design
Production Verification
Prod. Images
Monitoring Alerts / Notifications
Esri Managed Services Cloud based GIS infrastructure support
Requirements Proposal / ROM Contract Questionnaire On Call Sizing / Arch. Design
User Load
Application
Users Count; Peak (Yes / No)
GIS Services (Dynamic, Image, GP, Feature, Cached)
Security
Availability
SSL / AGS Tokens / Data at Rest / Encryption
SLA – 95%, 99% or 99.9%; Monitoring (24/7)
Infrastructure Public Cloud or Private Cloud; DB/Web/App Servers
Esri Managed Services Cloud based GIS infrastructure support
Requirements Proposal / ROM Contract Questionnaire On Call Sizing / Arch. Design
Benchmark
System Designer
PassMark; SystemCPU (AWS, Azure, Private Cloud)
Models; Designing; Capacity Planning
System Test
System Monitor
GIS Services; Web Applications; Service Times
System; GIS; DB; Process; Alerts / Notifications
MXDPerfStat Fix MXD issues (if any)
• http://www.arcgis.com • owner: EnterpriseImp
Esri Managed Services Cloud based GIS infrastructure support
Staging Verification Prod. Images
Deployment
Test
Update
MXD’s (MXDPerfStat); GIS Services (System Test)
Create Snapshot
Esri Managed Services Cloud based GIS infrastructure support
Production Verification Monitoring Alerts / Notifications
Snapshot
Prod. Servers
System
Application (URL)
DB
Process
Emails
24/7 Support
Many successful deployments…
Customer Deployments
Oregon Imagery Explorer
• Cached and dynamic image services • JS Application with Clip/Zip/Ship • SLA – 95%
• Search, download, use large imagery datasets • Simple, easy to use web viewer
Oregon Imagery Explorer (Lessons)
• Deployed in Amazon Web Services (AWS) • 40+ GIS Services (Image, Cache, GP, etc)
- (Min, Max) SOC instances vs. RAM - Paging Files / Swap space
• Raster/Cache data: 22 TB (Approx.) - Ship drives to AWS using Import / Export - Backup before shipping drives (New!!)
• Server Specs: 1 x Standard Extra Large Server - 4 Cores and 15 GB RAM - EBS Volumes: 23 drives
- Max EBS Volume is: 1 TB ONLY
- Windows Stripe (Server OS / Software RAID)
• CloudWatch (Monitor Disk Usage, Custom Counters (service stop, system reboot), CPU Alerts/Notifications)
Iberdrola USA Outage Viewer
• JS Application • Server Auto-Scaling • Data Update Automation
• High Availability • SLA 99.9% • Geographic Redundancy
Iberdrola USA Architecture and Workflow
Load Balancer Load Balancer
Outage Database
Amazon EC2 West
Amazon S3
Amazon EC2 East Primary / Auto-Scaled Cold Backup (DR Site)
Updates Every 15 Minutes
Iberdrola USA Outage Viewer (Lessons)
• Deployed in Amazon Web Services (AWS) • Points to Remember when Auto-Scaling:
- Plan for the Worst - Pick the “right” Instance Type when AS - “Launch” script - Contractual Cap: Max limit for Auto-Scaling - Time needed to launch a new “fully functional” instance - Termination Policy: Yes or No?
- IIS Logs – Usage logs?
- Licensing ? - Check with your Account Manager.
• Alerts and Notifications - System Counters - Data Update Scripts (ArcPy) - CloudWatch (Monitor Disk Usage, Custom Counters (service stop, system reboot), CPU
Alerts/Notifications)
Hurricane Sandy (Usage Statistics – New York)
Hurricane Sandy (Usage Statistics – New York)
Hurricane Sandy (Usage Statistics – Maine)
Hurricane Sandy (Usage Statistics – Maine)
Hurricane Sandy – Outages (New York)
2/10/2014 -11:30 am
Peak Sandy Hours
Heat Map – Fun !!
Maine – October 29
Maine – October 30
Maine – October 31
Maine – November 1
Maine – November 2
Survey
• http://www.esri.com/events/devsummit/session-rater • Enter: “Extending Services in ArcGIS for Server” • 10 seconds • Comments really welcome
FYI: Slides for this presentation are available at: http://1drv.ms/1fnkpTK