Upload
shiva-narayanaswamy
View
158
Download
0
Embed Size (px)
Citation preview
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Shiva Narayanaswamy – Solution Architect, AWSColin Panisset – Global Infrastructure Lead, REA
April 2016
Application Delivery PatternsOwn your destiny. It is a Pipeline dream
Technical 401
Agenda: Application Delivery Patterns
• Why?• What?• How?• The REA Journey• What the…?• X Commandments• What next?
Mission Statement
Deliver better features to customers, quickly, securely, more reliably and cheaper.
http://www.space.com/images/i/000/034/954/original/blue-origin-test-fires-new-rocket-engine-3.jpg
Deliver “Hello World”package main
import “fmt”
func main() {
fmt.Printf(“Hello, World.\n”)
}
#!/path/to/python
print “Hello World!”
using System;
using System.Collections.Generic;
using System.Text;
namespace ConsoleApplication1 {
class Program {
static void Main(string[] args) {
Console.WriteLine("Hello, world!");
Console.ReadLine()
}
}
}
#!/path/to/ruby
puts "Hello World!";
var http = require('http');
http.createServer(function (request, response) {
response.writeHead(200, {'Content-Type': 'text/plain'});
response.end('Hello World\n');
}).listen(8080);
Development LifecycleREPOSITORY
PM SYSTEM
CI SYSTEM
DEVELOPER
LOOKUPTASKS
SUBMITCODE
SCHEDULE BUILD
AUTOMATIC BUILD
BUILD RESULTS
BUILD ARTEFACTS
CLOUDFORMATION,PUPPET,
CHEF,ANSIBLE.
CONTAINERS,AMIs
EXE, MSI, RPM, DEB,
ZIP, TAR
Delivery LifecycleREPOSITORY
PM SYSTEM
DEPLOYMENT SYSTEM
DEVELOPER
LOOKUPTASKS
SUBMITCODE
SCHEDULE BUILD
BUILD / ARTEFACTS
BUILD RESULTS
CI SYSTEM
ENVIRONMENTS
SYSTEST
PRE-PROD
PRODUCTIONDEPLOYMENT RESULTS
DEPLOYMENT CONFIG
TRIGGER A PUSH / PULL
Delivery LifecycleREPOSITORY
PM SYSTEM
DEPLOYMENT SYSTEM
DEVELOPER
LOOKUPTASKS
SUBMITCODE
SCHEDULE BUILD
BUILD / ARTEFACTS
BUILD RESULTS
CI SYSTEM
ENVIRONMENTS
SYSTEST
PRE-PROD
PRODUCTIONDEPLOYMENT RESULTS
DEPLOYMENT CONFIG
ATOMIC UNIT OF DELIVERY
§ FUNCTIONAL CODE§ UNIT TESTS§ API§ BUILD CONFIG§ RUNTIME CONFIG§ INFRASTRUCTURE CODE§ DELIVERY PIPELINE
PUSH / PULL
PUSH / PULL
Delivery LifecycleREPOSITORY
PM SYSTEM
DEPLOYMENT SYSTEM
DEVELOPER
LOOKUPTASKS
SUBMITCODE
SCHEDULE BUILD
BUILD / ARTEFACTS
BUILD RESULTS
CI SYSTEM
ENVIRONMENTS
SYSTEST
PRE-PROD
PRODUCTIONDEPLOYMENT RESULTS
DEPLOYMENT CONFIG
ATOMIC UNIT OF DEPLOYMENT
§ DEPLOYABLE ARTEFACT§ API§ TEST SCRIPTS§ BUILD CONFIG§ RUNTIME CONFIG§ INFRASTRUCTURE CODE§ DELIVERY PIPELINE§ ENVIRONMENT CONFIG§ SECURITY CONFIG§ MONITORING CONFIG§ BACKUP CONFIG§ AVAILABILITY CONFIG§ SLA CONFIG
Application Deployment Patterns
Arbitrary Snowflakes Periodic deployments Blue Green Deployments
Canary Deployment Dark Launch / Feature Toggle
Environment Promotion
Orchestrate Delivery with Pipelines
A pipeline models your release process as a series of stages that promote changes along a set of environments into the hands of your customers.
A pipeline is a model of your standard procedure for deploying software.
Pipeline
Stage
Action
Pipeline Run
Source change • starts a run; and• creates an artifact to be used by other actions.
Manual trigger
Stage transition
Parallel Actions
Pipeline Concepts – AWS CodePipeline
Characteristics of a Good Pipeline
Fast Feedback Validation IdempotentSecureDesired State
Consistency
Roll Forward API Driven Visualization ExtensibleAs Code
Pipeline Design
Blockers
Simple Integral Security
Metrics Driven
Chained
Andon Cord Process Events
Loosely Coupled
Corollary to Conways Law : Your pipelines design will be a copy of your organisation's communication structure.
Pipeline metrics
• Number of Builds• Number of Deployments• Average Time For Changes to Reach Production• Average Time From First Pipeline Stage to Each Stage• Number of Changes Reaching Production• Average Build Time
Let us Deliver a Polyglot BeastPacker/Puppet/Chef scripts
Foundation SOE pipeline
Foundation SOE AMI/Container
Nginx SOE pipeline
Java SOE pipeline
Tomcat SOE pipeline
Developer SOE pipeline
Nginx SOE Java SOE Tomcat SOE Developer Workstation
Nginx SOE Java SOE Tomcat SOE HAProxy SOE
Let us Deliver a Polyglot Beast
IIS SOE
Application 1 on EC2
Application 2 on ECS
Application 3 on Beanstalk
REA's Application Delivery Journey
Colin PanissetGlobal Infrastructure Lead
Context
Opportunities
• Not a nuclear facility• Move fast, fail fast &
cheap• Fail fast & cheap = learn
fast & cheap• Tight feedback loops =
higher quality
History – 2010 and before
• All tin, all the time• Co-tenant apps to improve
utilisation• Coupled monoliths• Inconsistent environments• Infrequent deployments,
expensive discovery
History – Moving to AWS
• Early 2011: one dev acct.• Early 2012: prod & staging
accounts• Environment consistency• (Trusted) Dev access to
staging and prod!• Early microservices, layer 3
coupling:• microliths
History – Managing the Blast Radius
• IAM Users means persistent credentials
• Useful if you build components outside AWS
• Credential management is hard
• Federated identity, EC2 Instance Roles, build inside AWS
History – Isolation vs Overhead
A few big accounts:• IAM policy management
overhead• Blast radius• Tragedy of the commons
So: many small accounts
Evolution of Delivery Artifacts
• Code blobs/tarballs via Puppet or manual
• Dependencies (RPM/DEB)
• Externalised config• Fully resolved artifacts
(Docker, AMI)
Evolution of Delivery Patterns
• Pets: “Chuck it over the wall”
• Push via SSH/pull via agent
• Cattle, not pets: re-provision to deploy• Blue/Green• Rolling update
Challenges with Deployments across Multiple Accounts
• Layer 3 coupling• Decouple along
boundaries of data responsibility, SLA
• Managing secrets and keys
• Consistency across environments
Operational Challenges & Resolutions in Multiple Accounts
• Push vs Pull models of deployment: • where's the trust?• signaling patterns
• Manual intervention for artifact promotion
• Monitoring and alerting
Demo
Who wants a demo of a real REA pipeline?
Metrics – How will You Know when you've Succeeded?
• Time To Deliver (commit -> live)
• Deployments per X• 95%-ile Deltas Delivered
(lower is better)• Exclude human factors
…
Key Lessons Learned
• Multiple accounts over managing fine-grained IAM policies
• Decouple across boundaries of responsibility or SLA, not org structure
• Trust your tests
Key Lessons Learned (more)
• Deploy fully-resolved artifacts
• Decouple deployment tooling from apps
• Put your delivery teams on pager, give them the power to respond
• Keep metrics!
Thank You!
Colin PanissetGlobal Infrastructure Lead
Application Delivery from Outer Space
Cost Aware Serverless
Shared Experience Observer PatternThrottles
Location Aware
Some Tools
AWS Codepipeline BuildKite
Commandments for Application Delivery
I. Acknowledge time. Version control everythingII. Be the master of your dependenciesIII. Externalising configurations shall set you freeIV. Don’t be a prisoner of stateV. Loosely coupleVI. Audits that don’t kill you, make you strongerVII. Everyone sees everything, all the timeVIII. Measure successIX. Continuous practiceX. Own your Destiny, end to end
How do I start?
THINK BIGSTART SMALLACT NOW
AWS Training & Certification
Intro Videos & Labs Free videos and labs to help you learn to work with 30+ AWS services
– in minutes!
Training ClassesIn-person and online
courses to build technical skills –
taught by accredited AWS instructors
Online Labs Practice working with AWS services in live
environment –Learn how related
services work together
AWS CertificationValidate technical
skills and expertise –identify qualified IT talent or show you
are AWS cloud ready
Learn more: aws.amazon.com/training
Your Training Next Steps:
ü Visit the AWS Training & Certification pod to discuss your training plan & AWS Summit training offer
ü Register & attend AWS instructor led training
ü Get CertifiedAWS Certified? Visit the AWS Summit Certification Lounge to pick up your swag
Learn more: aws.amazon.com/training
Title Only
Thank You!