Upload
boyd-hemphill
View
4.549
Download
1
Embed Size (px)
DESCRIPTION
StackEngine describes the problems of running containerized applications in production and the solutions it is offering.
Citation preview
Managing Docker Applications in Production
Boyd E. Hemphill - @behemphi Mike Raab - @stackengine
History - Boyd
Oracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged), Evangelist
History - Boyd
Oracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged)
Started Austin DevOps In 2012
History - Boyd
Oracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged)
Started Austin DevOps In 2012
Co-Organizer of DevOps Days Austin
History - Boyd
Oracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged)
Started Austin DevOps In 2012
Co-Organizer of DevOps Days Austin
Became involved in Docker Austin in August.
History - BoydOracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged)
Started Austin DevOps In 2012
Co-Organizer of DevOps Days Austin
Became involved in Docker Austin in August.
Working on the first Container Days Austin for late Feb
History - Boyd
Oracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged)
Started Austin DevOps In 2012
Co-Organizer of DevOps Days Austin
Became involved in Docker Austin in August.
Working with Docker for the first Docker Days Austin
Keep an eye on @stackengine to see if we can make it happen!
History -StackEngine
Started in May 2014 to address challenges facing Docker adopters
History -StackEngine
Started in May 2014 to address challenges facing Docker adopters
Funding announcement October
History -StackEngine
Started in May 2014 to address challenges facing Docker adopters
Funding announcement October
Currently in private beta
History -StackEngine
Started in May 2014 to address challenges facing Docker adopters
Funding announcement October
Currently in private beta
Common Ground
What is The Goal of Your
Company?
What is The Goal of Your
Company?Make Money!
So … What is DevOps?
Is DevOps a Process?
Is it an intersection of overlapping concerns?
Is DevOps a Culture?
DevOps:
DevOps is the way in which a technology organization embeds itself in a business to the benefit of that business.
Presentation Prelude
Describe the problems StackEngine sees in using containers in production.
The Ideal Containerized Application
Netflix (unicorn)Service Oriented Architecture
Benefits
Feature velocity - Increase developer productivity using containers to model application topology
Benefits
Feature velocity - Increase developer productivity using containers to model application topology
Innovation velocity - Decreased build and test times make more time for trying that next great idea!
BenefitsFeature velocity - Increase developer productivity using containers to model application topology
Innovation velocity - Decreased build and test times make more time for trying that next great idea!
Density - Reduce Costs
Benefits
Feature velocity - Increase developer productivity using containers to model application topology
Innovation velocity - Decreased build and test times make more time for trying that next great idea!
Density - Reduce Costs
Separation of Concerns - “Code to an interface” moves to the infrastructure level
Benefits
Taken together a company that can get to the end state has a significant and lasting competitive advantage
Today’s Docker Reality
Concerns
This is still very early days for Docker
Concerns
This is still very early days for Docker
It is even earlier for Rocket
Concerns
This is still very early days for Docker
It is even earlier for Rocket
Tools to support containerized applications are immature or do not exist
Concerns
This is still very early days for Docker
It is even earlier for Rocket
Tools to support containerized applications are immature or do not exist
Service oriented architecture is hard
ConcernsThis is still very early days for Docker
It is even earlier for Rocket
Tools to support containerized applications are immature or do not exist
Service oriented architecture is hard
Increasing developer throughput is only a local optima, thus a false economy
Concerns
Docker adoption is the first major DevOps initiative not driven by Ops.
Concerns
Docker adoption is the first major DevOps initiative not driven by Ops.
Laundry Ops: http://stackengine.com/laundryops-practical-devops-at-home/
SDLC
Developers are moving forward
SDLC
Developers are moving forward
QA sees production modeling, large test grids, more effective integration testing
SDLC
Developers are moving forward
QA sees production modeling, large test grids, more effective integration testing
Build engineers see parallelized build grids for faster feedback and better build quality
xkcd … er … SDLCOperations panics - no tools or instrumentation
SysAdminDon’t want this guy panicking
Information Architecture
Problem Description
Information Architecture
Problem Description
StackEngine’s work on that problem today
Information Architecture
Problem Description
StackEngine’s work on that problem today
StackEngine’s planned work on that problem for the future
Problem #1 The Final Mile
The Final Mile
Developer uses containers
The Final Mile
Developer uses containers
System is built and images are produced
The Final Mile
Developer uses containers
System is built and images are produced
Images stored in a private registry
The Final Mile
Developer uses containers
System is built and images are produced
Images stored in a private registry
Now we need to deploy to a QA environment
The Final Mile
So we build a manual solution depending on human behavior
The Final Mile
So we build a manual solution depending on human behavior
Someone builds a tool
Crossing the Final Mile - Today
A Simple Solution to Deployment
Deploy My Stuff - TodayCreate a component to represent your registry
Deploy My Stuff - TodayCreate a component to represent the image you would like to run
Deploy My Stuff - TodayDistribute that image and run it on the hosts in your Mesh.
Deploy My Stuff - TodayMagic happens.
Deploy My Stuff - TodayLife is good
Crossing the Final Mile - Future
Heterogeneous, Policy Driven Deployment
Deploy My Stuff - Future
Support other container types
Deploy My Stuff - Future
Support other container types
Policy Based Deployment
Deploy My Stuff - FutureWe should put another bunny cloud over there …
Problem #2 Container Sprawl
Container Sprawl
First there was VMware and suddenly a large corps departments could easily spin up environments
Container Sprawl
First there was VMware and suddenly a large corps departments could easily spin up environments
Then there was the cloud and developers could completely circumvent ops
Container SprawlFirst there was VMware and suddenly a large corps departments could easily spin up environments
Then there was the cloud and developers could complete circumvent Ops
Now there is Docker and Developers are creating numerous abstractions that are not managed like traditional VMs
Container SprawlFirst there was VMware and suddenly a large corps departments could easily spin up environments
Then there was the cloud and developers could complete circumvent Ops
Now there is Docker and Developers are creating numerous abstractions that are not managed like traditional VMs
Rocket potentially segments the Market and means different performance, security, runtime and management characteristics
Container Sprawl
Containers, Dockerfiles, and Images on VMs
Container Sprawl
Containers, Dockerfiles, and Images on VMs
VMs on physical hardware located in availability zones
Container Sprawl
Containers, Dockerfiles, and Images on VMs
VMs on physical hardware hardware located in availability zones
AZs in data centers in geographic regions.
My Head HurtsContainer Sprawl
Artifact Inventory - Today
A drillable and searchable view of infrastructure
Where’s My Stuff - TodayHosts
Where’s My Stuff - TodayImages
Where’s My Stuff - TodayContainers
Where’s My Stuff - TodayFilters and Build Actions
Where’s My Stuff - TodaySearch
Artifact Inventory - Future
Geographically aware physical topology
Where’s My Stuff - Future
Geographically drillable physical topologies
Where’s My Stuff - Future
Geographically drillable physical topologies
Policy based inventory levels (pull not push)
Where’s My Stuff - FutureHey, why are there two purple stripes?
Problem #3 Telemetry
Telemetry
What does monitoring a containerized app look like?
Telemetry
What does monitoring a containerized app look like?
http://goo.gl/VkAQo4
Telemetry
What does monitoring a containerized app look like?
http://goo.gl/VkAQo4
What does log aggregation look like?
Telemetry
What does monitoring a containerized app look like?
http://goo.gl/VkAQo4
What does log aggregation look like?
http://goo.gl/uNPe9P
Decision Dimensions - TodayHost level metrics to help you manage
running containers.
What’s my stuff doing - Today
I have this much stuff.
What’s my stuff doing - Today
My hosts are not yet stuffed.
What’s my stuff doing - Today
The stuffing in a host is as expected.
What’s my stuff doing - Today
A bit of the stuffing is doing what it should.
What’s my stuff doing - Today
What is the stuff in this image?
What’s my stuff doing - Today
How do I visualize the stuff in an image?
Decision Dimensions - Future
There is never too much information about run time.
What’s My Stuff Doing - Future
Improve Host Metrics
What’s My Stuff Doing - Future
Improve Host Metrics
Improve Image Metrics
What’s My Stuff Doing - Future
Improve Host Metrics
Improve Image Metrics
Add Container Metrics
What’s My Stuff Doing - Future
Ahh … look … the bunny on the left is happy!
Problem #4 Centralized Control Plane
Centralized Control Plane
The ability to act upon a system from a single point of entry
Centralized Control Plane
The ability to act upon a system from a single point of entry
Actions should happen next to the information causing the action
Centralized Control Plane
The ability to act upon a system from a single point of entry
Actions should happen next to the information causing the action
Actions should happen within the context of inventory
Centralized Control Plane
The ability to act upon a system from a single point of entry
Actions should happen next to the information causing the action
Actions should happen within the context of inventory
UI, API, CLI
Actions - TodayPreventative, automatic, deterministic
Make Stuff do Stuff - Today
Manage Images
Make Stuff do Stuff - Today
Manage Containers
Actions - FutureInitialization, Taxonomy, Corrective
Make Stuff do Stuff - Future
Policy based actions by the Controller
Make Stuff do Stuff - Future
Policy based actions by the Controller
Powerful Control Plane Taxonomy (Tagging)
Make Stuff do Stuff - Future
Policy based actions by the Controller
Powerful Control Plane Taxonomy (Tagging)
Ants not Cattle
Make Stuff do Stuff - Future
Blow really hard and make the bunnies kiss.
Problem #5 Heterogeneous Service
Providers
Heterogeneous Service Providers
Containers live on services (IaaS, PaaS)
Heterogeneous Service Providers
Containers live on services (IaaS, PaaS)
Images come from different places (Docker Hub, Quay.io)
Heterogeneous Service ProvidersContainers live on services (IaaS, PaaS)
Images come from different places (Docker Hub, Quay.io)
Within an IaaS service provider are other services (Databases, load balancers)
Heterogeneous Service ProvidersContainers live on services (IaaS, PaaS)
Images come from different places (Docker Hub, Quay.io)
Within an IaaS service provider are other services (Databases, load balancers)
Persistent Data stores present their own unique problem as a service
Components - Today
Abstractions for Control Plane Operation
Building Apps with My StuffComponents are infrastructure service abstractions
Building Apps with My StuffComponents can be arranged in logical ways to represent
physical infrastructure.
Building Apps with My Stuff
Start the db
Building Apps with My Stuff
Start the web server
Building Apps with My Stuff
Run the application server
Building Apps with My Stuff
Run the Cache
Components - Future
More and varied components
Building Apps with My Stuff - Future
More component types is like having more varied Lego bricks
Building Apps with My Stuff - Future
More component types is like having more varied Lego bricks
Pluggable component architecture
Building Apps with My StuffWhat else could we make with bunnies and rainbows?
Roadmap
Problem #6 Service Discovery
Service Discovery
Allows multiple deployments of the same service on the same host.
Service Discovery
Allows multiple deployments of the same service on the same host.
Containers register themselves upon startup and are found by tags rather than URI’s, fixed ports, etc.
Service Discovery
Allows multiple deployments of the same service on the same host.
Containers register themselves upon startup and are found by tags rather than URI’s, fixed ports, etc.
We favor something like consul with a search interface like the chef server.
Service DiscoveryAllows multiple deployments of the same service on the same host.
Containers register themselves upon startup and are found by tags rather than URI’s, fixed ports, etc.
We favor something like consul with a search interface like the chef server.
A step towards cattle containers instead of pet containers
Service DiscoveryAllows multiple deployments of the same service on the same host.
Containers register themselves upon startup and are found by tags rather than URI’s, fixed ports, etc.
We favor something like consul with a search interface like the chef server.
A step towards cattle containers instead of pet containers
Ultimate goal is “ants not cattle”
Roadmap
User Management
Roadmap
User Management
Addition of new Components
Roadmap
User Management
Addition of new Components
Better telemetry
Roadmap
User Management
Addition of new Components
Better telemetry
Scheduling
Roadmap
User Management
Addition of new Components
Better telemetry
Scheduling
A/B Deployment support
StackEngine is the StuffIt’s easier to dream with StackEngine.
Live Product DemoAl Gore and his gremlins willing
Boyd E. Hemphill - @behemphi Mike Raab - @stackengine
Thank You for Your Time and Comments.