Learning the Alphabet – A/B, CD…

BrettTimpermanDeveloper Lead, Kroger@BrettTimperman

10 102013:

10 102016:

10 10

10 10

10 10

10 10

10 10

10

10

*This slide needs better graphics

Contentious DeliveryA Tale of Dev and Ops

Contentious Delivery

Deploy my code faster!

Your crap code crashes!

* Placeholder graphics

Contentious Delivery

Features in prod right

NOW!

99.99999%uptime!

* Placeholder graphics

_

Contentious Delivery

Prod broke, WHAT

CHANGED?!

* Placeholder graphics

_

Contentious Delivery

I haven’t committed in 6

months.

* Placeholder graphics

My code is flawless. 10 lines.2,000 unit tests.

Contentious Delivery* Placeholder graphics

Let’s write functional tests so this never

happens again.

What are all these red things?

42 TESTS FAILED *coding*

codecodecode

Collaborate!

Easier said than done, but also:

• Microservices• Docker!

• Monitoring• Docker!

• Communication• Not yet Dockerized

and…

Lessons learned

How can we deployfaster and safer?

Faster and SaferA DevOps Compromise

Pull requests!

All commits subject to approval.

• Applications• Base Docker images• Deploy scripts• Reverse proxy configuration

Open source model

Faster

Safer

- +

+-

Regression’s a dirty job but someone’s got to do it

What test cases can’t be covered with a lower level unit or contract test?

Automation is required for true Continuous Delivery.

Reduce manual regression tests

Faster

Safer

- +

+-

http://www,martinfowler.com/bliki/TestPyramid.html

Move functional to unit tests

Faster

Safer

Mike Cohn’s Test Pyramid

- +

+-

Echo – coming to open source soon!

Technical diagram of Echo system:

• Kafka• ElasticSearch• Kibana• Thrift• Custom apps

Custom Monitoring System

Faster

Safer

- +

+-

Echo demo

$ cat docker-compose.yml$ docker-compose up

Docker Trusted Registry and Universal Control Plane

Docker with additional manageability including:

• Trusted CI build and push to DTR• RBAC for running containers on UCP• Auditing

Docker Datacenter

Faster

Safer

Colorful Delivery

• Two versions of app deployed• Docker!

• Dark test non-prod color• If (failed):

• Remove non-prod color• Else:

• Swap non-prod color to prod• Profit

Blue/Green deploys

Faster

Safer

- +

+-

Blue/green demo

Deploy new version to UCP Run tests Switch color

Code walkthrough See new version in prod Rollback

• Consul vs. Interlock• Registration by apps vs.

Registrator

• Docker open source vs. Datacenter• Admin UI vs. Authz plugins

Tech stack considerations

Continuous Delivery and Experimentation (CDE)A/B testing and Canary deploys

Faster

Safer

NGiNX

User1

User2

Echo

A B

- +

+-

• Marriages and relationships?• Stock trades?• Startups?• Job candidates?• Conference presentations?• Anything and everything?

Real life A/BWhat if we could A/B test…

A/B demo

Show button clicking site Monitor button click metrics

What color do you think will generate the most

clicks?

A/B demo

Configure and deploy app with new color as B route Code walkthrough Demonstrate UCP RBAC with routing and app stacks

Audience vote Monitor performance metrics in real time

Go to [demo URL] on your mobile device and

click away!

A/B demo

Determine winner and scale down losing route

• A/B, always better• CD, cautious distribution• CDE, carefully doing experiments• DTR, docked trusty register• UCP, ultimate container platform• RBAC, read books about cats

Your new alphabetThere will be a quiz

• Open source all code• https://github.com/timperman

• Automated scaling• Automated rollback• Blog posts

Next steps

Thank you!