Embed Size (px)
Citation preview
AWS SDKDevOps AutomationSachin Dole
About me
▪ AWS Certified Solutions Architect
▪ Independent AWS Consultant – Genvega Inc.
▪ Trelair.com – now defunct startup
▪ www.cricclubs.com – ½ million+ page views per month
▪ Backup & Archival on S3 – project for law firm
▪ EC2 automation for image processing
▪ Available for AWS consulting – find me: www.genvega.com
About CricClubs (My customer)
30th Jan
2016@2015 CRICCLUBS.COM | www.cricclubs,.com | CONFIDENTIAL 3
Started in 2008
Geographic Usage20+ Countries
600+ Leagues 100,000+ Players
HeadquartersMilwaukee, WI,
USA
Features50+ features on
Website and iOS, Android, Windows
Mobile Apps
Premium Support24/7 via E-mail, Phone andTicketing Systems
Data CentersHigh Availability with servers across 12 Data Centers
4 million pageviews/month
About you?
▪ Have day to day AWS experience?
▪ Have personal projects on AWS?
▪ Want a good introduction to DevOps & AWS?
Console access
▪ https://tvarit.signin.aws.amazon.com/console
▪ ijugdemo
▪ devops123
▪ Read only access to all services
Topics
▪ DevOps for Trelair
▪ Lessons learnt
▪ Tvarit.io & AWS Java SDK
▪ Questions?
Trelair.com
System Architecture
Clients
ELB
Wildfly on EC2
S3 & Postgres
External
Services
Building Trelair
Commit
Build
FTP
Deploy
Test
Backup & Release
1 to 8 person-hrs
DML Disk Compute DB App server Backups
Commit
Build
FTP
Deploy
Test
Release
Enterprises?
1 to 8 person years
DML Disk Compute DB App server Backups
Dev
EA
Test Ops
EA
Trelair Automation
GitCommit• Teamcity
• JBossDMR
Auto-Deploy
SeleniumTest &
Release
DevOps + 0 person hoursCustom Maven Plugin
Used AWS Java SDK
AMI DML/DDL DNS Snapshots ScalingAuto
Automated Flow
A. Commit Code (the only manual step)
B. CI build
C. DB DML patches are applied to replica
D. Deploy code to candidate server
E. UI tests run
F. DML patches applied to production
G. Candidate server is added to cluster
H. Old servers are terminated
AWS API
us - eastAWS Java
SDK
JBoss DMR Wildfly AS
Provision
Deploy
Lessons
▪ Automation is easy – mindset is difficult
▪ Better to automate in small increments – big bang is a waste
▪ Automate everything – partial automation is overhead
▪ Release frequently – rely on automation
▪ Use selenium automation (e.g. saucelabs.com)
▪ Rely on AWS products
Tvarit
▪ http://www.tvarit.io, https://github.com/sdole/tvarit-maven
▪ Trelair automation on steroids, opensource
▪ Automated provisioning, deployment and scaling
▪ Alpha release available in Maven Central
▪ Free support through GitHub issues
▪ Working α version & a deep roadmap
Console access
▪ https://tvarit.signin.aws.amazon.com/console
▪ ijugdemo
▪ devops123
▪ Read only access to all services
Tvarit – demo
▪ CloudFormation templates
▪ Maven goals
▪ VPC, IAM console
▪ EC2: LC, ASG, ELB
▪ S3 bucket
▪ Cloudwatch monitoring, flow logs
▪ Code walkthrough
API structure
▪ Many flavors – Java, C++, JS, Ruby, Python, CLI etc.
▪ APIs are RESTful
▪ Deep documentation
API Structure
AmazonEc2Client ec2 =
new AmazonEc2Client(new BasicAwsCredentials());
SomeEc2Request req = new SomeEc2Request();
req.withSomething(thing1).withSomething(attr1);
SomeEc2Result result = ec2.submitRequest(req);
inspectWithYourLogic(result);
Automation Security - Individuals
IAMus - east
us - west
Access
Keys
groups
users
roles
Authorized
Automation Security - Services
IAMOther
services
Other
services
ec2
lambda
S3
Assumed
RoleAuthorized
AWS Automation Features
▪ Autoscaling
▪ Monitoring
▪ Alarms
▪ Cloudwatch Events
▪ Lambda Functions
▪ S3 Events, SQS and SNS events
▪ SWF
▪ CloudFormation
Tvarit Target Architecture
Clients
WAF
ELB
EC2
ELB
EC2
Lambda
Java SDKAutoscaling
Cloudwatch
Zookeeper
S3, RDS
S3, RDS
Cloudformation
ELB
Router
App 1
Version 1
App 2
Version 1
Other concepts:Blue-Green Deployment
http://martinfowler.com/bliki/BlueGreenDeployment.html
Other concepts:Canary Release
http://martinfowler.com/bliki/CanaryRelease.html
Revisit Tvarit Target Architecture
Clients
WAF
ELB
EC2
ELB
EC2
Lambda
Java SDKAutoscaling
Cloudwatch
Zookeeper
S3, RDS
S3, RDS
Cloudformation
ELB
Router
App 1
Version 1
App 2
Version 1
Tvarit Release concept (future)
ELB
ELB
Router
App Versions
Router: routes customers based on Http Headers to correct app version
Step Back – what is (not) DevOps?
Is…
Infrastructure as code
Not…
Auto-scale, failover, monitoring
Continuous delivery
Dedicated teams, Dev, Test, Ops
Forms, Emails, Release plans
Infrastructure Architects/EA
Soft side of DevOps
▪ Cloud-native vs Lift ‘n shift
▪ CALMS
▪ C – Culture
▪ A – Automation
▪ L – Lean
▪ M – Measurement
▪ S – Sharing
▪ Automation as Kaizen – not a single project
Costs of AWS Automation
▪ In most cases – FREE
▪ Priced as usual: Compute, Storage, Monitoring, Network
▪ Monitoring is a must
▪ CPU, Memory, Network, Disk…
▪ Health checks, failed hardware, traffic spikes
▪ Audit trail – API actions, modifications
▪ Configuration changes
How to improve Tvarit
▪ Open an Issue on GitHub:
“We use a customized version of tomcat – can you make the app server container configurable so we can use our own?”
Fork it on GitHub and DIY
Email me
Other things
▪ Chef, Puppet, Ansible, SaltStack
▪ AWS OpsWorks, Beanstalk
▪ AWS CodeCommit, CodeDeploy
How to find me
@sdoledotcom
https://linkedin.com/in/sdole
http://genvega.com/blog/
https://github.com/sdole/
ΩThank you
