Embed Size (px)
Citation preview
Online Webinar Google Cloud Platform Cost Optimization
Feb 24th, 2016
First Things first…
• Questions are always welcome
• There are no bad questions
• We’ll demo as much as we can
• Things will break, be patient
Before we get started..• $500 Credit for Google Cloud Platform• https://goo.gl/GuPXUt
• $100 Credit for Spotinst• https://spotinst.com/signup?GCP_Webinar
Amiram ShacharFounder & CEO
Spotinst
Grace MollisonSenior Solution Architect Google Cloud Platform
Vadim SoloveyCTO
DoIT International
@AmiramShachar @LiranPolak @vadimsolovey
Liran PolakCo-Founder & Chief Architect
Spotinst
@grapesfrog
Agenda• The Basics: Spotinst on Google Cloud Platform
• Preemptible VMs Introduction - Vadim Solovey
• Elastigroup for GCE Fundamentals.
• Demo: Spotinst, Elastigroup for GCE
• Q&As, tips and tricks by Grace Mollison (Google)
The ultimate platform for running and managing clusters in the cloud.
Spotinst provides a framework for reliable, highly available and cost efficient use of AWS Spot instances and GCP Preemptible VMs, with typical savings of 70-90% while ensuring full availability.
“ We carry spare capacity in our datacenters for a variety of reasons.
Preemptible VMs fill this spare capacity, but let us reclaim it if needed, helping us optimize our datacenter utilization … … … The tradeoff is that Preemptible VMs are limited to a 24 hour runtime, and will sometimes be preempted (shut
down) earlier than that. “
Spare capacity is wasted capacity
PVM. What Are They?
Just Like Regular VMs (the ones we sell today...)Same configurations, guest OSes, performance, tools, etc.Anybody who dares can buy it
More Cost EffectiveFlat pricing: ~70% off regular VM ratesUS n1-standard-1 price: $0.015
Reasonable, Predictable Limitations24h uptime limitPre-termination notice (30s) to allow for clean shutdownMany non-realtime workloads ideally suited for Preemptible VMs
Preemptible VMs: Typical Use Cases
Log processing, Analytics, Big Data (Hadoop, MapReduce, etc.)• Typical use case would use Preemptible for “worker” nodes and normal VMs for data nodes
that need to be reliable• Google Dataproc have built-in PVM integration• More integrations are in baking- Container Engine, Dataflow and Managed VMs
Media transcoding• Wide usage of PVM for this type of a use case
Financial modeling and simulation• Monte Carlo simulations• Back testing• Suitability depends on specific implementation
Preemptible VMs: Typical Use Cases
Rendering• Several VFX clients are using or are interested.• Viability depends on the length of the render tasks and how they manage their nodes (if they
can recover from a preemption)
Continuous integration - automated build and testing• Great use case. If a test or build fails, just restart it and move on.• Chrome Clusterfuzz using ~6,500 PVMs (8,000 cores) for 24x7 randomized security testing.
Double the number of tests they were running before, less than half the cost• Golang (build bots) using PVMs
Genomics• DNA Sequencing, resampling and reintegration.
What actually happens at preemption?
1. Cloud decides a machine must be preempted 2. 30 second notice is sent to the VM
a. Same mechanism as “VM stop” b. Most modern guest OSes will catch this signal and perform basic cleanup
steps (sync disks, etc.) c. Normal shutdown script mechanism can be used to catch this notice for
application specific cleanups (but they only get 30s) 3. Hard power off is sent to the VM if not finished after 30s
Note: Just like a normal user-initiated shutdown, the VM remains as a resource in GCE (consuming CPU / PD / IP quota) until deleted by the user...
Elastigroup for GCE Fundamentals
Preemptible VMs Capacity - Zone A
n1-highcpu-2 n1-highcpu-4 n1-highcpu-8 n1-highcpu-16
Total Capacity Used Capacity
Preemptible
Preemptible VMs Capacity - Zone B
n1-highcpu-2 n1-highcpu-4 n1-highcpu-8 n1-highcpu-16
Total Capacity Used Capacity
Preemptible
Multi Zone & Multi Instance TypeElastigroup discovers Google’s available capacity
For efficient and highly-available utilization of Google’s spare
capacity
n1-highcpu-2 n1-highcpu-4 n1-highcpu-8 n1-highcpu-16Total Capacity Used Capacity
Available Capacity
Auto Scaling & Load Balancer Support Scaling based on CPU / Network with LB and connection draining
To ensure smooth workloads without impacting customers’
experience.
Regular VMs Preemptible VMs
Powerful VM replacementSeveral hours before a PVM is likely to expire,
Elastigroup provisions a new PVM replacement.
Preemptible VMs Minimum Capacity
Hybrid ClustersHybrid clusters of Prmemptbile and Sustained use discounts
Easy to use system that provides stability and reliability.
Demo: Elastigroup for GCE
Grace MollisonSenior Solution Architect Google Cloud Platform
@grapesfrog
Ask the expert
Don’t forget your credit!• $500 Credit for Google Cloud Platform• https://goo.gl/GuPXUt
• $100 Credit for Spotinst• https://spotinst.com/signup?GCP_Webinar
Thank You
