Cloudsolutionday 2016: DevOps workflow with Docker on AWS

DevOps workflow with Docker on AWS

Nov 19 2016

@Wayarmy – Quanpc@topica.edu.vn Topica Edumall SysAdmin

About me

• @wayarmy • Interested: Cloud Computing, AWS, Docker,

IaaS, SaaS, PaaS, Apache Stack, Elastic Stack, Hashi Stack, Ruby on Rails, Linux, Unix…

• Github: http://github.com/wayarmy • Sysadmin, SysOps, DevOps, HumanOps

1. DevOps

2. CI - CD 3. System Design with Docker on AWS 4. CI - CD workflow with Docker5. Q&A

Techtalk AWS and Docker

DEVOPS

• DevOps is the practice of operations and development engineers participating together in the entire service lifecycle, from design through the development process to production support. https://theagileadmin.com

DevOps requirement

• DevOps Skills • Program Language, Application design... • Knowhow about system design • Experience with SaaS, PaaS, IaaS... • Knowhow about Cloud Computing,

Virtualization. • Knowhow about concept of CI - CD

1. DevOps 2. CI - CD 3. System Design with Docker on AWS 4. CI - CD workflow with Docker5. Q&A

Techtalk AWS and Docker

Continous Intergration

• Continuous integration (CI) is the practice of merging all developer working copies to a shared mainline several times a day. - Wiki Pedia -

• Implement: • Source code managerment • Automation testing • Automation QA • Automation releasing

Continous Delivery

• Continuous delivery (CD) is a software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released at any time

• - Wiki Pedia -

Target

• Optimize released-time of Production. • Optimize Dev workflow. • Decrease number of tester - Optimize business

benefit.

1. DevOps 2. CI - CD 3. System Design with Docker on AWS

4. CI - CD workflow with Docker5. Q&A

Techtalk AWS and Docker

Difficulty of DevOps in System

• So hard to get experience with physical, network • So difficult to know all SaaS, PaaS, IaaS... • Everything with Dev is code, everything with

developer will be moved to code • So difficult to maintain software=> System need "write once, run forever"

Topica Edumall (old) System Design

AWS EC2 Feature

• Autoscale • High Availability • Secure

Edumall Infrastructure System Design on AWS

Benifit

• High availability system • Easy with scale and everything will be automated • Auto healing • Save much money ( save about 40% with 100%

running application on AWS )

Difficult with DevOps

• Server is not transparent with Dev • Deploy with Command line, get error when

deploy • Application environment is not isolate • Deploying is not automation • Hardware is still get high, cannot optimize

Difficult with DevOps

• Server is not transparent with Dev • Deploy with Command line, get error when deploy • Application environment is not isolate • Deploying is not automation • Hardware is still get high, cannot optimize

=> Need a system with isolate environment and orchestration tools for platform

Docker

• Docker containers wrap a piece of software in a complete filesystem that contains everything needed to run: code, runtime, system tools, system libraries – anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment.

• Build – Ship – Run • Build once, run any where

Kubernetes

• Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.

• “Write once, run forever” • Build Auto-scale and self-healing system

Final Design

Some numbers

• Video content transfer: 1,5 TB (/1 day) • Static Content (image, js, html, css): 100GB • Total Request: ~ 5 mil • Total Ram: ~ 2TB • Total CPU: ~400vCpus • Unique Visitor: ~50k • Total User: ~20m • Maximum Unique Visitors: ~5k • EC2 Start: ~ 5 (M4 2xLarge)

Techtalk AWS and Docker

1. DevOps 2. CI - CD 3. System Design with Docker on AWS 4. CI - CD workflow with Docker 5. Q&A

Target

• Server is transparent with Dev • PaaS • Everything is Automation • Easy with operation • Document as Code

Element

• Source code managerment: Gitlab • CI builder: Gitlab multi runner • Platform: Docker • Docker registry: Private registry store on AWS S3, ECR • Provisioning: Gitlab runner / Ansible (Near future)

CI-CD workflow

Developer push a new commit to gitlab

Gitlab Trigger Gitlab Runner fetch and pull repo

Gitlab-runner build Docker image from Dockerfile

Excute test on all target With all environment

Delivery images to Docker hub Or Private registry

Deploy to K8s Cluster

Sumary

• DevOps workflow with AWS IaaS, Kuberntes, Gitlab... • Isolate application environment with Docker • Increase team productivity • Decrease the number of member in Developer Team,

SysAdmin Team, increase the benefit • Write one, run any where and forever!!!!!!=> Don't care about the system, take care of your health

Techtalk AWS and Docker

1. DevOps 2. CI - CD 3. System Design with Docker on AWS 4. CI - CD workflow with Docker 5. Q&A

Thank You

Meetup: www.meetup.com/AWS-vietnam FB: www.facebook.com/groups/amazonwebservicevietnamJoin Slack: https://aws-vn.herokuapp.com/