12 Factor App Methodology

12 Factor App Method-ologyLaeshin Park

laeshiny@gmail.com

Table of Content

• About Me• What is 12 Factor App• Summary• 12 Factor App Check List

About Me

• Interesting in Cloud Native• This kind of methodology is new to me

What is 12 Factor App?

• Methodology for building software-as-a-service in the Cloud

Who wrote?

• http://12factor.net• Manifesto written around 2012• by Adam Wiggins (Heroku, co-founder)

For Who?

• Any developer building applications which run as a service.• Ops engineers who deploy or manage such applications.

What is good?

• Minimize time and cost for new developers joining the project.• Offer maximum portability between execution environ-

ments.• Enable continuous deployment for maximum agility.• Obviate the need for servers and systems administration.• Can scale up without significant changes to tooling, archi-

tecture, or development practices.

How?

• Use declarative formats for setup automation.• Have a clean contract with the underlying operating sys-

tem.• Minimize divergence between development and produc-

tion.

How?

• I. Codebase• II Dependencies• III. Config• IV. Backing services• V. Build, release,

run• VI. Processes

• VII. Port binding• VIII. Concurrency• IX. Disposability• X. Dev/prod parity• XI. Logs• XII. Admin pro-

cesses

I. Codebase

• One codebase tracked in revision control, many deploys• VCS(Version Control System): subversion, git, ...

I. Codebase

I. Codebase

• Demo• Git• https://github.com/laeshiny/12factorapp

II. Dependencies

• Explicitly declare and isolate dependencies• Never rely on implicit existence of system-wide packages• Do not rely on the implicit existence of any system tools• Example: curl

II. Dependencies

• Python• Virtualenv

• Ruby• Gemfile

• Demo

How to handle Dependen-cies

III. Config

• Resource handles to the database, Memcached, and other backing services• Credentials to external services such as Amazon S3 or Twit-

ter• Per-deploy values such as the canonical hostname for the

deploy• Everything that is likely to vary between deploys (staging,

production, developer environments, etc)

What is con-fig?

III. Config

• “config” does not include internal application config• Use config files which are not checked into revision control• You can make your project open source• Stores config in environment variables!!!!

How to handle config

III. Config

•Demo• python code

IV. Backing services

• Any service the app consumes over the network as part of its normal operation.

What is Backing ser-vices?

IV. Backing services

• locally-managed services,• datastores : MySQL or CouchDB• messaging/queueing systems : RabbitMQ or Beanstalkd• SMTP services : Postfix• caching systems : Memcached or Redis

• third parties managed• SMTP services : Postmark• metrics-gathering services : New Relic or Loggly• binary asset services : Amazon S3• API-accessible consumer services : Twitter, Google Maps, or Last.fm.

What is Backing ser-vices?

IV. Backing services

• Makes no distinction between local and third party services• Treat backing services as attached resources

How to handle Backing ser-vice

IV. Backing services

API Server as interface for using Backend service

AppsAPI

Server

https://10.10.10.10/resource/1

SELECT * FROM re-sourceSELECT * FROM re-source2

{“data1”: 123}{“data2”: “abc”}

{“data1”: 123, “data2”: “abc”}

DNS

Lookup apiserver.com10.10.10.10

Get https://apiserver.com/resource/1

V. Build, release, run

• Build : code repo + vendors dependencies + binaries and assets• Release : Build output + config• Run : Launching set of the app’s processes against a selected re-

lease

V. Build, release, run

• Use strict separation between the build, release, and run stage

How to handle Build, release, run

V. Build, release, run

VI. Processes

• Execute the app as one or more stateless processes• Processes are stateless and share-nothing

VI. Processes

• Never assumes that anything cached in memory or on disk will be available on a future request or job• Any data that needs to persist must be stored in a stateful backing

service, typically a database• Web systems should never be used or relied upon ““sticky ses-

sions”• Session state data is a good candidate for a datastore that offers

time-expiration, such as Memcached or Redis

How to handle Processes

VII. Port binding

• Export services via port binding• One app can become the backing service for another app

VII. Port binding

• Ex)• http://main-portal.com/users/laeshiny >> http://main-

portal.com:5001/users/laeshiny• http://main-portal.com/config/app1 >> http://main-

portal.com:5002/config/app1

How to handle Port binding

VIII. Concurrency

• Scale out via the process model• The application must also be able to span multiple processes running

on multiple physical machines

VIII. Concurrency

VIII. Concurrency

• Should never daemonize or write PID files.• Rely on the operating system’s process manager

• manage output streams• respond to crashed processes• handle user-initiated restarts and shutdowns

• ex) Upstart, Foreman, Systemd• Demo

How to handle Concurrency

IX. Disposability

• Be disposable, meaning they can be started or stopped at a mo-ment’s notice

• Maximize robustness with fast startup and graceful shutdown

• This facilitates fast elastic scaling, rapid deployment of code or con-fig changes, and robustness of production deploys

IX. Disposability

• Processes should strive to minimize startup time• Processes shut down gracefully when they receive a SIGTERM signal

from the process manager• A web process ceases to listen on the service port (thereby refusing any new

requests), allowing any current requests to finish, and then exiting• A worker process returns the current job to the work queue

• Processes should also be robust against sudden death, in the case of a failure in the underlying hardware

How to handle Disposability

X. Dev/prod parity

• Keep development, staging, and production as similar as possible• developer resists the urge to use different backing services between

development and productionTraditional app Twelve-factor app

Time between deploys Weeks Hours

Code authors vs code deployers Different people Same people

Dev vs production environments Divergent As similar as possible

XI. Logs

• Logs are the stream of aggregated, time-ordered events collected from the output streams of all running processes and backing ser-vices• Logs provide visibility into the behavior of a running app• Treat logs as event streams

XI. Logs

• Never concerns itself with routing or storage of its output stream• In staging or production deploys, each process’ stream will be cap-

tured by the execution environment, collated together with all other streams from the app, and routed to one or more final destinations for viewing and long-term archival• Demo

• Fluentd

How to handle Logs

XII. Admin processes

• Run admin/management tasks as one-off processes• One-off admin processes should be run in an identical environment

as the regular long-running processes of the app• The same dependency isolation techniques should be used on all

process types• Strongly favors languages which provide a REPL shell out of the box,

and which make it easy to run one-off scripts

Summary

• Scalable• Easy Management

• I. Codebase• II Dependencies• III. Config• IV. Backing services• V. Build, release, run• VI. Processes

• VII. Port binding• VIII. Concurrency• IX. Disposability• X. Dev/prod parity• XI. Logs• XII. Admin processes

12 Factors App Check List

Use VCS(subversion, git, ....)Execution environment is isolatedIt is easy to access Backend ServiceBuild, Stage, Run environment is speratedUse Process Manager to manage the applicationSupport short startup time and graceful shutdownDev environment is identical to prod environmentCollect logs in the datastore by the another app.

Want to talk more

Thank you for your attention