Upload
haeseung-lee
View
122
Download
4
Embed Size (px)
Citation preview
Introduction
CDNetworks inc. Product Manager / Haeseung Lee
before we start…
•Pros & Cons of Cloud Computing?
Pros of Cloud Computing• say ‘goodbye’ to costly systems
• access from innumerable options
• software expense
• cooked food
• lower traditional servers’ cost
• data centralization
• data recovery
• sharing capabilities
• cloud security
• free cloud storage
• instantly test
Cost Saving Reliability Manageability Strategic Edge = Edge Computing
Cons of Cloud Computing
• net connections
• low bandwidth
• affected quality
• security issues
• non-negotiable agreements
• cost comparison
• no hard drive
• lack of full support
• incompatiblity
• lack of insight into your network
• minimal flexibility
Downtime Security and Privacy Vulnerability to attack Limited control and flexibility Platform Dependency(aka ‘Lock in’) Cloud Computing costs
Cloud Vendors? = Our Competitors!
first runners…
Public Cloud Adoption
30©2016 RightScale, Inc. All rights reserved. This work by RightScale is licensed under a Creative Commons Attribution 4.0 International License.
This year, we looked at IaaS and PaaS for Google and Azure separately as well as combined. The
separate IaaS and PaaS numbers reveal how each service is doing on its own, while the combined
numbers show the number of unique respondents that use any Azure (IaaS or PaaS or both) or any
Google (IaaS or PaaS or both) service. Because many users leverage both IaaS and PaaS offerings
from these vendors, the combined adoption is less than the sum of the adoption numbers for each
individual service.
Whereas 17 percent are using Azure IaaS and 13 percent are using Azure PaaS, 20 percent are using
either or both Azure offerings. 7 percent are using Google App Engine and 6 percent are using
Google IaaS, while 10 percent are using either or both of Google IaaS/PaaS.
Private Cloud Adoption 2016 vs. 2015
Private Cloud Adoption
35©2016 RightScale, Inc. All rights reserved. This work by RightScale is licensed under a Creative Commons Attribution 4.0 International License.
Private cloud adoption grows across all providers.The 2016 State of the Cloud Survey reveals that adoption of private cloud is growing across
all providers. Across all sizes of organizations, 44 percent percent of respondents leverage vSphere
environments as private clouds. OpenStack and VMware vCloud Suite tied for second at 19 percent.
In its first year on the survey, bare-metal clouds are being used by 15 percent of respondents.
29 percent of respondents are experimenting with or planning to use OpenStack, and 27 percent
are planning to use Microsoft Azure Pack.
In comparison to last year, there were noticeable increases in adoption rates of every private cloud
technology. VMware vSphere/vCenter went up eleven percent (33 to 44 percent). OpenStack and
VMware vCloud Suite both increased six percent (13 to 19 percent).
MS Azure Service
Why choose Google Cloud Platform?
• enables developers to build, test and deploy applications on Google’s highly-scalable, secure, and reliable infrastructure.
• computing, storage, big data/machine learning, and application services for web, mobile, analytics, and backend solutions.
Google’s Infrastructure
. Google has been building for the past 15 years: Massive, Powerful infrastructures such as datacenter and high-speed fiber optic networks.
. Data Centers / Backbone / 70+ edge POPs in 33 Countries / edge caching platform
Cloud Regions and Zones
• Regions • Regions are specific geographical locations where you can run your
resources • are collections of zones • Regional resources are available to resources in any zones in the region
• Zones • Isolated locations within a region • Zonal resources are only available in that zone
• Multi-regional resources - below are one or more multi-regional deployment in addition to any regional deployment • Google App Engine and its feature • Google Cloud Database • Google Cloud Storage • Google BigQuery
Innovative, Customer-Friendly Pricing
• Sub-Hour billing • Sustained-use discounts
• automatically reward users who run virtual machines for over 25% of any calendar month
• Compute Engine custom machine types • pay only for the resources you need for your application
• online pricing calculator
©Google Inc. or its affiliates. All rights reserved. Do not distribute. 12
Image by Connie Zhou
NextFully automated
NowUser managed, user
configured, user maintained
1st wave:Colocation
2nd wave:Virtualized datacenters
3rd wave:A global,
elastic cloud
The Future of Cloud Computing
Notes: The concept of cloud computing began with colocation. Instead of operating your own data center, you rented space in a colocation facility. This was the first wave of outsourcing IT. With colocation, the transfer of ownership was minimal - you still owned the machines and you maintained them. Traditionally, colocation is not thought of as cloud computing, but it did begin the process of transferring IT infrastructure out of your organization.
Today, cloud computing involves virtualized datacenters - virtual machines and APIs. Virtualization provides elasticity. You automate infrastructure procurement instead of purchasing hardware. With virtualization you still maintain the infrastructure. It is still a user-controlled/user-configured environment. This is the same as an on-premises datacenter, but now, the hardware is in a different location. Virtualization does provide a number of benefits: your development teams can move faster and you can turn capital expenses into operating expenses.
The next wave of cloud computing is a fully automated, elastic cloud. This involves a move from user-maintained infrastructure to automated services. In a fully automated environment, developers do not think about individual machines. The service automatically provisions and configures the infrastructure used to run your applications. Google is uniquely positioned to
©Google Inc. or its affiliates. All rights reserved. Do not distribute.
IaaS and PaaS
14
PaaSPreset run-times
Java, Go, PHP, Python...Focus is application logic
Pay for what you useLess management overhead
IaaSRaw compute, storage and
networkMore granular control
Pay for what you allocateMore management overhead
Towardsmanaged
infrastructure(DevOps)
Towardsmanaged services(NoOps)
Compute Engine App Engine
Notes: This slide highlights the shift in cloud computing. Virtualized datacenters brought you infrastructure as a service (IaaS) and platform as a service (PaaS) offerings. At Google, we have services that provide these options. As cloud computing has evolved, the momentum has shifted toward managed infrastructure (DevOps) and managed services (NoOps). As the course progresses, you see how the Google Cloud Platform products and services are positioned to propel you into the next wave of cloud computing - a fully automated, elastic cloud.
©Google Inc. or its affiliates. All rights reserved. Do not distribute.
Google Cloud Platform
15
BigQuery
Big Data
Pub/Sub Dataflow Dataproc Datalab
Compute
Compute EngineApp Engine Container Engine
Storage
Cloud Storage
Cloud SQL Cloud Datastore
Bigtable
Machine Learning
Speech APIMachine Learning
Translate APIVision API
Notes: Google Cloud Platform’s products and services can be broadly categorized as Compute, Storage, Big Data, Machine Learning, Networking, and Operations/Tools. This slide highlights many of the services that form the core of the Google Cloud Platform. Throughout this course, you explore the Cloud Platform products and services in lectures and in the hands-on labs. For now, it is helpful to see the breadth of the Cloud Platform offerings at a glance.
For more information, see: https://cloud.google.com/products/
Google Cloud Platform Console• https://console.gloud.google.com
Projects
• All GCP services are associated with a project that is used to: • Track resources and quota ranges • Enable billing • Manage permissions and credentials • Enable services and APIs
• Projects use three identifying attributes: • Project Name • Project Number • Project ID - aka Application ID
• Interact with projects using the Cloud Console or the Cloud Resource Manager APIAlpha
Project Permissions - primitive roles
Owner Billing AdministratorViewerEditor
Invite members Remove members Can delete project includes Editor rights
Deploy applications Modify code Configure services includes Viewer rights
Read-only access Manage billing Add administrators Remove administrators
A project can have multiple owners, editors, viewers and billing administrator. - Primitive roles -above - curated roles - new IAM roles that give finer-grained access control than the primitive roles
©Google Inc. or its affiliates. All rights reserved. Do not distribute.
Identity and Access Management
8
Who can do what on which resource
IAM roles - Curated Roles
©Google Inc. or its affiliates. All rights reserved. Do not distribute.
IAM Roles - Curated Roles
9
IAM
✔ compute.instances.delete✔ compute.instances.get✔ compute.instances.list✔ compute.instances.setMachineType✔ compute.instances.start✔ compute.instances.stop . . .
List of Permissions
InstanceAdminRole
project_a
GoogleGroup
Notes:The “can do what” part is defined by an IAM role. An IAM role is a collection of permissions. Most of the time to do any meaningful operations you need more than 1 permission. For example to manage instances in a project, you need to create, delete, start, stop and change an instance. So the permissions are grouped together into a role to make it easier to manage.
To give a user the desired permissions, you grant a role to the user on a resource. In this example we are granting a group of users the InstanceAdmin role on project a so the user can manage instances in the project. Whenever possible, it is a best practice to use groups. You should also strictly control the ability to change policies and group memberships which will allow additional users to gain access to resources.
For a complete list of roles by product, see: https://cloud.google.com/iam/docs/#supported_cloud_platform_services
Service Accounts and IAM
• Service Accounts authentication with keys • google manage keys, key rotation for
Compute engine and App engine
• Can assign an IAM role to the service account
• Can also assign serviceAccountActor role to users/groups
©Google Inc. or its affiliates. All rights reserved. Do not distribute.
Service Accounts and IAM
14
● Service accounts authenticate with keys
○ Google manages keys, key rotation for Compute Engine and App Engine
● Can assign an IAM role to the service account
● Can also assign ServiceAccountActor role to users/groups
Service Account InstanceAdmin Role Compute Instances
Service Account
Identity IAM Role Resource
User/Group ServiceAccountActor Role
Notes:Users require a username and password to authenticate. Apps use a key. One or more keys can be generated for each IAM service account. Keys are sensitive and need to be carefully managed because they give you access to resources. When you run applications on Compute Engine or App Engine, Google manages the keys for you and automatically rotates them. You never have the risk of losing/exposing your key. When you run apps elsewhere, you can generate and download the keys to use in your code. Keep them safe and rotate them.
A service account is both an identity and a resource. A service account is used as an identity for your application to authenticate; for example, a Compute Engine VM running as a service account. To give the VM access to the necessary resources, you need to grant the relevant IAM roles to the service account. At the same time, you need to control who can create VMs with the service account so random VMs cannot assume the identity. Here, the service account is the resource to be permissioned. You assign the ServiceAccountActor role to the users you trust to use the service account.
Google App Engine and Google Cloud Datastore
©Google Inc. or its affiliates. All rights reserved. Do not distribute. 3
Google Cloud Platform
NetworkingCompute Big Data Machine Learning Storage Operations and
Tools
Compute Engine
Container Engine
App Engine
Notes: App Engine is one of several Google Cloud Platform compute options for running your applications.
What is Google App Engine
• A platform(platform as a service) for building scalable web applications and mobile backends
• App Engine makes deployment, maintenance, and scalability easy so you can focus on innovation
©Google Inc. or its affiliates. All rights reserved. Do not distribute. 6
700 million
“App Engine enabled us to focus on developing the application. We wouldn’t have gotten here without the ease of development that App Engine gave us.”Bobby Murphy, CTO
Snapchat sends
photos and videos each day Google App Enginescaled seamlesslyduring growth to
millions of users
Small team is ableto innovate quickly
and expandglobally
Snapchat
Notes:● Today, Snapchat’s photo messaging app is among the top 10 most
downloaded mobile apps of any type for both Android and iOS.● They launched in 2011, and just two years later, had tens of millions of
users across the world sending hundreds of millions of photos and videos each day.
● Before the era of Cloud computing, supporting this level of growth with a developer and operations team of just a few people would have been impossible.
● With Google App Engine, Snapchat doesn’t have to worry about managing the underlying infrastructure. Cloud Platform allows their app to scale with minimal effort from their team.
● But it’s not just about scale. App Engine also allows Snapchat to quickly release new features or versions of the application - a difficult task when your app is constantly in use by millions of people.
Read more about Snapchat here: https://cloudplatform.googleblog.com/2016/03/Snapchat-shares-security-best-practices-for-running-on-GCP-practices.html.
App Engine - Best Practice
App Engine Standard Environment
• Managed runtimes for specific versions of Java, Python, PHP & Go
• Autoscale workloads to meet demand • Free daily quota, usage based pricing • SDKs for development, testing and deployment • Need to conform to sandbox constraints:
• no writing to local file system
• request timeouts at 60 seconds
• Limit on 3rd-party software installations
©Google Inc. or its affiliates. All rights reserved. Do not distribute.
Example App Engine Standard Workflow - Web Applications
10
Project
App EngineApp Servers
Application instances
Application instances
Application instances
App Engine can access a variety of services using dedicated APIs1 Develop & test the web
application locally
2Use the SDK to deploy to App Engine
3App Engine automatically scales & reliably serves your web application
Task queues
Scheduled tasks
Search
Memcache
Logs
Notes:There are many services available to your applications running in the App Engine Standard environment. For the most up-to-date information on App Engine services, see: https://cloud.google.com/appengine/docs/about-the-standard-environment#index_of_features.
Below are details on some of the more commonly used services available to your App Engine Standard applications.
Users APIApp Engine Standard environment applications can authenticate users using Google Accounts or accounts on your own Google Apps domains. An application can detect whether the current user has signed in, and can redirect the user to the appropriate sign-in page to sign in or, if your app uses Google Accounts authentication, create a new account. While a user is signed in to the application, the app can access the user's email address. The app can also detect whether the current user is an administrator, making it easy to implement admin-only areas of the app.
Modules APIModules are used to factor large applications into logical components that can share stateful services and communicate in a secure fashion. An app that
App Engine Flexible EnvironmentBeta
• Build, deploy containerized apps with a click • Standard runtimes - Python, Java, Go, Node.js
- with no sandbox constraints • Custom runtime support for any languages
that support HTTP requests • During beta pricing based on Compute Engine
usage • Local development relies on Docker • Standard runtimes can access App Engine
services: Datastore, Memcache, task queues, logging, users, and so on.
©Google Inc. or its affiliates. All rights reserved. Do not distribute.
App Engine Standard vs Flexible Environment
Standard Environment Flexible Environment
Instance startup Milliseconds Minutes
SSH access No Yes (not default)
Scaling Manual, basic, automatic Manual, automatic
Write to local disk No Yes (ephemeral)
Support for 3rd party binaries
No Yes
Network access Via App Engine services Yes
Customizable stack No Yes
18
Notes: You can run an App Engine application in two environments, the standard environment and the flexible environment. You can use both environments in your application at the same time if you structure your application using the microservices architecture.
Flexible environment versus Compute Engine
While flexible environment runs modules in Compute Engine VM instances, it differs from Compute Engine in the following ways:● Flexible environment VM instances are restarted on a weekly basis.
During restarts Google's management services will apply any necessary operating system and security updates.
● You always have root access to Compute Engine VM instances. SSH access to VM instances in the flexible environment is disabled by default. If you choose, you can enable root access to your app's VM instances.
● Flexible environment VM instances are automatically located by geographical region according to the settings in your project. Google’s management services will ensure that all the VM instances for a project are co-located for optimal performance.
Google Cloud Endpoints
• Build your own API running on App Engine Standard
• Expose your API using a RESTful interface • Includes support for OAuth 2.0
authorization • Generate client libraries • Support Java and Python server-side
code • Includes App Engine features
• scaling
• denial of service protection
• high availability
• Support iOS, Android, and JavaScript Clients
©Google Inc. or its affiliates. All rights reserved. Do not distribute. 23
Google Cloud Platform
Compute Machine Learning Storage Operations and
Tools
Cloud Storage
Cloud SQL
CloudDatastore
CloudBigtable
Networking Big Data
Notes: Cloud Datastore is a highly-scalable NoSQL database for your applications. Cloud Datastore automatically handles sharding and replication, providing you with a highly available and durable database that scales automatically to handle your applications' load. Cloud Datastore provides a myriad of capabilities such as ACID transactions, SQL-like queries, indexes and much more.
Google Cloud Datastore
• Database designed for application backends
• NoSQL store for billions of rows • Schemaless access, no need to think
about underlying data structure • Local development tools • Automatic scaling and fully managed • Built-in redundancy • Supports ACID transactions • Includes a free daily quotas • Access from anywhere through a
RESTful interface
©Google Inc. or its affiliates. All rights reserved. Do not distribute. 3
Google Cloud Platform
Compute Machine Learning Storage Operations and
Tools
Cloud Storage
Cloud SQL
CloudDatastore
CloudBigtable
Networking Big Data
Notes:Google Cloud Platform has many storage options that satisfy nearly every customer use case. Cloud Datastore was discussed previously. In this module, we turn our attention to the remaining core storage options: Google Cloud Storage, Google Cloud SQL, and Google Cloud Bigtable. First, is Cloud Storage.
Google Cloud Storage
• High performance, internet-scale, immutable BLOB(binary large object) storage
• Not a file system(but can be accessed as one via 3rd-party tools such as Cloud Storage Fuse)
• Simple administration and does not require capacity mgmt
• Data encryption in-flight and at rest
• All storage classes accessed through the same APIs
Cloud Storage Classes
©Google Inc. or its affiliates. All rights reserved. Do not distribute.
Cloud Storage Classes
6
Standard Storage
DRA Storage
Nearline Storage
Standard Storage provides the highest durability, availability and performance with low latency and is ideal for use with website content distribution and video streaming
Durable Reduced Availability Storage offers the same durability as Standard Storage but with a lower availability SLA at a reduced cost
Nearline Storageoffers low-cost, highly durable storage service for data archiving, online backup, and disaster recovery, without having to wait hours or days to retrieve or access your data
Each option comes with detailed pricing
Notes: Google Cloud Storage offers three types of storage: Standard Storage, Durable Reduced Availability (DRA) Storage, and Nearline Storage with different availability, latency, and throughput characteristics. All storage classes offer the same high level of durability.
Cloud Storage Integration
©Google Inc. or its affiliates. All rights reserved. Do not distribute.
Cloud Storage Integration
11
Import and exporttables
Object storage, logs, Datastore backups
Startup scripts, images and general object storage
Import and export tables
App Engine
Compute Engine
Cloud Storage
Cloud SQL
BigQuery
Notes:Google Cloud Storage is tightly integrated with many of the Google Cloud Platform products and services. Cloud Storage is often the ingestion point for data being moved into the cloud, and is frequently the long-term storage location for data.
Integration between Cloud Storage and other services includes using Cloud Storage to:● Store tables imported into BigQuery and to store tables exported from
BigQuery● Store App Engine logs, Cloud Datastore backups, and objects used by
App Engine applications (such as images)● Store instance startup scripts, Compute Engine images, and objects
used by Compute Engine applications● Store tables imported into Cloud SQL and to store tables exported from
Cloud SQL
For more information on Cloud Storage integration with other Google Cloud Platform services, see: https://cloud.google.com/storage/docs/google-integration.
DEMO 1. Getting Started with GCP 2. Deploying Applications using Google App Engine and Cloud Datastore
Google Cloud SQL
• Google-managed MySQL relational database in the cloud
• Pay-per-Use model • RESTful API for management • Affordability and performance • Good Security • Vertical scaling(read and write) • Horizontal scaling(read) • Seamless integration with App
Engine and Compute Engine
©Google Inc. or its affiliates. All rights reserved. Do not distribute. 26
Managed backups
Google security
Automatic replication
Familiar Flexible pricing
Cloud SQL Features
Notes:FamiliarBuild and deploy for the cloud faster because Cloud SQL offers standard MySQL databases. Use standard connection drivers and built-in migration tools to get started quickly. Cloud SQL supports most MySQL statements and functions:● Stored procedures● Triggers● Views
Cloud SQL does not support:● User-defined functions● MySQL-esque replication● Statements and functions related to files and plugins
Flexible pricingFor First Generation instances, Google offers two billing plans: Packages and Per Use. Which plan you should select depends on how your database will be used, but generally it is more economical to use the Package plan if your instance is used for more than 450 hours each month. If your instance is used sporadically, you can reduce costs by using the Per-use plan and setting your activation policy to ON DEMAND. Learn more.
©Google Inc. or its affiliates. All rights reserved. Do not distribute. 28
Cloud SQL Integration
Cloud SQL can be used with App Engine using standard drivers like Connector/J for Java or MySQLdb for Python.
App Engine applications are authorized to access Cloud SQL, and the instance can be configured to follow one application.
Compute Engine instances can be authorized to access Cloud SQL instances using an external IP address.
Cloud SQL instances can be configured with a preferred zone - to stay close to the Compute Engine infrastructure.
Cloud SQL can be used with external applications and clients by authorizing IP addresses or networks using CIDR notation.
Standard tools like MySQL Workbench can be used to administer databases. External read replicas can be configured.
Externalservice
Google Cloud SQL second generationBeta
• Same features as first generation with higher performance, storage capacity at lower cost • up to 7x throughput and 20x storage
capacity of first generation instances
• less expensive than first generation for most use cases
©Google Inc. or its affiliates. All rights reserved. Do not distribute. 32
Comparing Storage Options (1 of 2)
NoSQLBLOB SQL
Cloud Storage
Cloud Bigtable
Cloud Datastore
Cloud SQL
Good for:Structured and unstructured binary or object data
Good for:Getting started, App Engine applications
Good for:“Flat” data, Heavy read/write, events, analytical data
Good for:Web frameworks,existing applications
Use cases:Images, large media files, backups
Use cases:User profiles,product catalog
Use cases:AdTech, Financial and IoT data
Use cases:User credentials, customer orders
Notes:Google Cloud Platform delivers various storage service offerings which remove much of the burden of building and managing storage and infrastructure. Like our other cloud services, cloud storage will free you to focus on doing what you do best and differentiating at the application or service layer.
Our storage offerings range from SQL, NoSQL, Blob and Block storage depending on what you are trying to do, and it's easy to mix and match. ● Cloud SQL gives you fully managed MySQL so you have relational DB
and a more traditional approach to queries. ● Cloud Datastore provides a nearly infinitely scalable, schemaless
solution. ● If you want a disk you can mount Persistent Disk as a block store that
can be used by Compute Engine.● For just pure data and blobs, Cloud Storage can deliver what you need. ● Google Cloud Bigtable offers companies a fast, fully managed, infinitely
scalable NoSQL database service ideal for web, mobile and IoT applications.
©Google Inc. or its affiliates. All rights reserved. Do not distribute.
Comparing Storage Options (2 of 2)
33
Cloud Datastore Cloud Storage Cloud SQL (1st Generation)
Bigtable
Storage type NoSQL, document
Object (BLOB) store
Relational SQL NoSQL, wide-column
Overall capacity Terabytes + Petabytes + up to 500 GB Petabytes +
Unit size 1 megabyte / entity
5 TB / object Standard MySQL limits
Recommended - Individual values: ~10 MB per cellAll values per row: ~100 MB
Transactions Yes No Yes No
Complex queries No No Yes No
Notes:● Cloud Bigtable is not a relational database; it does not support SQL
queries or joins, nor does it support multi-row transactions. Also, it is not a good solution for small amounts of data (< 1 TB).
● If you need full SQL support for an online transaction processing (OLTP) system, consider Google Cloud SQL.
● If you need interactive querying in an online analytical processing (OLAP) system, consider Google BigQuery.
● If you need to store immutable blobs larger than 10 MB, such as large images or movies, consider Google Cloud Storage.
● If you need to store highly structured objects, or if you require support for ACID transactions and SQL-like queries, consider Cloud Datastore.
Google Container Engine Google Compute Engine
What is a Container?
• Virtualization at the operating system layer
• Separates operating system from application code and dependencies
• Isolated individual processes
• Popular implementations include Docker and rkt
Why use Containers?
• Support consistency across development, testing, and production environments
• Loose coupling between application and operating system layers
• Much simpler to migrate workloads between on-premises and cloud environments
• Support agile development and operations ©Google Inc. or its affiliates. All rights reserved. Do not distribute.
● Virtualization at the operating system layer
● Separates operating system from application code and dependencies
● Isolates individual processes
● Popular implementations include Docker and rkt
3
What is a Container?
Operating system
Shared libraries
ContainerCode and libraries
ContainerCode and libraries
ContainerCode and libraries
Virtual Machine
Kubernetes(aka ‘k8s’)
• Open Source container cluster orchestration system • automates deployment,
scaling, and operations for container clusters
• Based on Google’s experience over 10+ years
• Built for a multi-cloud world: • public, private, hybrid
Features of Kubernetes
• Workload portability • run in many environments, across cloud providers
• implementation is open and modular
• Rolling updates • upgrade application with zero downtime
• Autoscaling • automatically adapt to changes in workload
• Persistent storage • abstracts details of how storage is provided from how it is consumed
• Multi-Zone clusters • run a single cluster in multiple zones
• Alpha on Google Cloud Platform
• Load balancing • external IP address routed traffic to correct port
Competitors of k8s
©Google Inc. or its affiliates. All rights reserved. Do not distribute. 12
Google Cloud Platform
NetworkingCompute Big Data Machine Learning Storage Operations and
Tools
Compute Engine
Container Engine
App Engine
Notes: Container Engine is one of several Google Cloud Platform compute options for running your applications.
©Google Inc. or its affiliates. All rights reserved. Do not distribute.
IaaS and PaaS
14
PaaSPreset run-times
Java, Go, PHP, Python...Focus is application logic
Pay for what you useLess management overhead
IaaSRaw compute, storage and
networkMore granular control
Pay for what you allocateMore management overhead
Towardsmanaged
infrastructure(DevOps)
Towardsmanaged services(NoOps)
Compute Engine App Engine
Notes: This slide highlights the shift in cloud computing. Virtualized datacenters brought you infrastructure as a service (IaaS) and platform as a service (PaaS) offerings. At Google, we have services that provide these options. As cloud computing has evolved, the momentum has shifted toward managed infrastructure (DevOps) and managed services (NoOps). As the course progresses, you see how the Google Cloud Platform products and services are positioned to propel you into the next wave of cloud computing - a fully automated, elastic cloud.
Google Container Engine(1/2)
• Fully managed cluster management and orchestration system for running containers • based on Kubernetes
• uses Compute Engine instances and resources
• Complimentary services: • Google Cloud Container Builder Beta -
Create Docker container images from app code in Google Cloud Storage
• Google Container Registry - Secure, Private Docker image storage
Google Container Engine(2/2)
• Uses a declarative syntax to manage applications • declare desired application configuration,
Container Engine implements, manages
• Decouples operational, development concerns
• Manage and maintains • logging, health management, monitoring
• Easily update Kubernetes versions as they are released
Container Engine=k8s :: concepts
- Nodes: node is working machine in a k8s cluster, and in Google Container engine, the machine is always a Computer Engine Instance.
- Pods: Pod is a group of one or more containers, shared storage, and configuration data relating to those containers. It is common for production applications running in k8s to include multiple, relatively tightly-coupled containers in a single pod.
- Replication Controllers: Replication Controller works to ensure that the requested number of pod replicas are always available and running at a given time. automatically adds or remove pods as required to maintain a desired state.
- Services: defines a logical set of pods and a way to access them using an IP address and port number pair.
©Google Inc. or its affiliates. All rights reserved. Do not distribute.
Comparing Compute Options
9
Compute Engine
Container Engine
App Engine Standard
App Engine Flexible
Language support
Any Any Java, Python, Go & PHP
Any
Service model IaaS Hybrid PaaS PaaS
Primary use case
General computing workloads
Container-based workloads
Web and mobile applications
Web and mobile applications, container-based workloads
©Google Inc. or its affiliates. All rights reserved. Do not distribute. 3
Google Cloud Platform
NetworkingCompute Big Data Machine Learning Storage Operations and
Tools
Compute Engine
Container Engine
App Engine
Notes: Compute Engine is one of several Google Cloud Platform compute options for running your applications.
Google Compute Engine(1/2)
• Run large-scale workloads on virtual machines hosted on Google’s infrastructure
• Robust networking features • default, custom networks
• firewall rules
• regional HTTP(S) load balancing
• network load balancing
• subnetworks
• High CPU, high memory, standard and shared-core machine types
Google Compute Engine(2/2)
• Persistent disks • standard, SSD, local SSD
• snapshots
• Resize disks, migrate instances with no downtime
• Instance metadata and startup scripts
• Advanced APIs for auto-scaling and instance group management
• Innovative pricing • per-minute billing, sustained use discounts • preemptive instances
• high throughput to storage at no extra cost
• custom mashine types - only pay for the hardware you need
DEMO 1. Deploying Applications using Google Container Engine 2. Deploying Applications using Google Compute Engine
Google Cloud Networking Google Cloud Operations and Tools Google Big Data Google Machine Learning(“TensorFlow”) Amazon Web Services case Messaging system (such as ‘Kafka’, etc.) Object Storage, File system introduction …
goes to next part.
END
References
. Google Cloud Platform Fundamentals: https://sites.google.com/a/google.com/cloud-platform-training/cloud-platform-training/cp100-v2 . Google Developers - codelab: https://codelabs.developers.google.com/?cat=Cloud
©Google Inc. or its affiliates. All rights reserved. Do not distribute. 13
Google Cloud Platform
Compute Machine Learning Storage Operations and
Tools
Cloud Storage
Cloud SQL
CloudDatastore
CloudBigtable
Networking Big Data
Notes:The next storage option on Google Cloud Platform is Google Cloud Bigtable.
Google Cloud Bigtable
• Fully managed, NoSQL, wide-column database service for large workload application - TB to PB
• Integrated • accessed using HBase API
• native compatibility with big data, Hadoop ecosystem
• Protected • replicated storage
• data encryption in-flight and at rest
• Role-based ACLs
• Proven • drives major applications such as Google
Analytics and Gmail
©Google Inc. or its affiliates. All rights reserved. Do not distribute. 19
Bigtable Access Patterns
Cloud Bigtable
Application APIData can be read from and written to Cloud Bigtable through a data service layer like: Managed VMs, the HBase REST Server, a Java Server using the HBase client. Typically this will be to serve data to applications, dashboards and data services.
StreamingData can be streamed in (written event by event) through a variety of popular stream processing frameworks like: Cloud Dataflow Streaming, Spark Streaming, Storm.
Batch ProcessingData can be read from and written to Cloud Bigtable through batch processes like: Hadoop MapReduce, Dataflow, Spark. Often, summarized or newly calculated data is written back to Cloud Bigtable or to a downstream database.
©Google Inc. or its affiliates. All rights reserved. Do not distribute. 20
Cloud Bigtable Integration
Cloud Bigtable
Google Cloud DataflowUse Cloud Dataflow connector for Bigtable for batch and streaming operations in pipelines
Google Cloud DataprocUse Bigtable HBase client to integrate Hadoop jobs with Cloud Dataproc
On-premises, cloud-based HadoopUse Bigtable HBase client to integrate with Hadoop clusters
Externalservice
Notes: Dataflow Connector for Cloud BigtableThe Cloud Dataflow connector for Cloud Bigtable makes it possible to use Cloud Bigtable in a Cloud Dataflow pipeline. You can use the connector for both batch and streaming operations. The connector is written in Java and is built on the HBase client for Cloud Bigtable. You can find the connector on GitHub in the repository GoogleCloudPlatform/cloud-bigtable-client.
For more information on the Cloud Dataflow connector, see the full API documentation.
Cloud Bigtable and the HBase APIOne way to communicate with Cloud Bigtable is through the Cloud Bigtable HBase client, which is a customized version of the standard Apache HBase client. The Cloud Bigtable HBase client allows you to write Java applications that communicate with Cloud Bigtable through the open-source HBase API. Source code for the client is available in the GitHub repository GoogleCloudPlatform/cloud-bigtable-client.
When you use the Cloud Bigtable HBase client, you don't need to run the HBase server. You are simply using a modified version of the HBase client interface. Google runs and maintains the servers for the Cloud Bigtable
©Google Inc. or its affiliates. All rights reserved. Do not distribute. 23
Google Cloud Platform
Compute Machine Learning Storage Operations and
Tools
Cloud Storage
Cloud SQL
CloudDatastore
CloudBigtable
Networking Big Data
Notes:The next storage option on Google Cloud Platform is Google Cloud SQL.