View
1.138
Download
0
Category
Preview:
Citation preview
Continuous Integration at CartoDB
JUAN IGNACIO SNCHEZ
CAS 2015 Basic level talk
Were hiring! www.cartodb.com/jobs
First things first
YOU
http://www.cartodb.com/jobs
LETS GET TO KNOW EACH OTHER!
Who has done a software deployment, or release, this year?
Who has done a software deployment, or release, during the last 6 months?
Who has done a software deployment, or release, during the last month?
Who has done a software deployment, or release, during the last week?
Who has done a software deployment, or release, during the last day?
15 / week peak: 7
Tools?
Team!
Team!
Process!
(just a little)
Continuous deployment is more of a deal
with the devil than magic ;-)
Continuous deployment is
more of a deal with the devil
than magic ;-)
15 / week peak: 7
Breaking CartoDB up
Components
1. Editor
2. Platform
1. SQL API
2. Maps API
3. CartoDB.js
Products
1. SaaS
2. On-premise
3. Open Source
Product Team
1. Editor: 8
2. Data services: 5
3. Web: 6
4. APIs: 2
5. Systems: 5
The GOALS in the deal
Reduce risk
SLA-own
ed
Reduce risk
impact x chance
Add value sooner
Roadmap
7 big* features /
milestone (2 weeks)
Deliver features
22 PRs each week
15 / week peak: 7
users
support
users
Summary, agile basics
(quick) feedback
(less) cost
HOW
Integration Delivery Deployment
Continuous<
Continuous Integration[CI.2] Continuous Integration is a software
development practice where members of a
team integrate their work frequently,
usually each person integrates at least daily -
leading to multiple integrations per day.
Each integration is verified by an
automated build (including test) to detect integration errors as quickly as possible.
XP (1999), Booch method (1991)
Continuous Delivery
[CD.1] software engineering approach in
which teams keep producing valuable
software in short cycles and ensure that the
software can be reliably released at any
time.
[CD.2] You build software in such a way that
the software can be released to production
at any time
Continuous Deployment
[CD.1*] software engineering approach in
which teams keep producing valuable
software in short cycles and ensure that the
software can be is reliably released at any
time.
[CD.2*] You build software in such a way that
the software can be is released to
production at any time
Continuous Deployment at CartoDB: a deal with the devil?
JUAN IGNACIO SNCHEZ
CAS 2015 Basic level talk
[CI.1]
1. Maintain a code repository
2. Automate the build
3. Make the build self-testing
4. Everyone commits to the
baseline every day
5. Every commit (to baseline)
should be built
6. Keep the build fast
7. Test in a clone of the
production environment
8. Make it easy to get the latest
deliverables
9. Everyone can see the results of
the latest build
10. Automate deployment
Best practices
The CartoDB Way
[CI.1]
1. Maintain a code repository
2. Automate the build
3. Make the build self-testing
4. Everyone commits to the baseline
every day
5. Every commit (to baseline)
should be built
6. Keep the build fast
7. Test in a clone of the
production environment
8. Make it easy to get the latest
deliverables
9. Everyone can see the results of
the latest build
10. Automate deployment
Best practices
Step 0: hiring
1. Initiative
2. Courage
3. Teamwork
4. Problem solving
Step 1: onboarding
1. Big picture
2. Empowerment:
deploy from the very
beginning, senior
or junior
Step 2: plan
1. Incremental
2. Iterative
3. Accept tradeoffs
Step 3: code
1. Take issue
2. Branch master
3. Code + tests
4. Staging testing
Step 4: pull request
1. Integration tests
2. Code style
3. Code review
Step 5: merge
& deploy
Step 6: evil devil
#devops
Rollbar
Kibana
The devil is in the detail
Feature flags
[FF.1] The basic idea is to have a
configuration file that defines a bunch of
toggles for various features you have
pending. The running application then uses
these toggles in order to decide whether or
not to show the new feature. - Fowler
Feature flags
Feature flags
Canary Releases[CR.1] Canary release is a technique to
reduce the risk of introducing a new
software version in production by slowly
rolling out the change to a small subset of
users before rolling it out to the entire
infrastructure and making it available to
everybody.
Quick rollbackIntegration issues
Scaling issues
Corner cases
1
2
3
Quick rollback Drawback
Migrations in its own PR
Previous PR for preparing ORM (ActiveRecord)
Post-mortem Affected components Timing What happened
5W Actions
Smoke tests Manual testing On-premise releases Tag
Wrapping it Up
Costs
Riskier Have plan B
backups
rollbacks soft deletions
Heavy
instrumentalized servers
fast tests real time alerting
Costs
Unexpected DB locks
Integration issues
Benefits
Cheaper Early detection &
diagnosis No release day
panic Small and easy
reverts.
Benefits
Accelerated TTM Building the Right
Product Test in actual
environment Visible progress
Is Continuous Delivery suitable for you?
Is Continuous Delivery suitable for you?
\_()_/
Questions?
Thank you!
Juan Ignacio Sanchez Backend Engineer at CartoDB
juanignaciosl@cartodb.com @juanignaciosl
References
Continuous Integration
[CI.1] Wikipedia: https://en.wikipedia.org/wiki/Continuous_integration
[CI.2] Fowler: http://www.martinfowler.com/articles/continuousIntegration.html
[CI.3] Duvall: Continuous Integration. Improving Software Quality and Reducing Risk
http://martinfowler.com/books/duvall.html
Continuous Delivery
[CD.1] Wikipedia: https://en.wikipedia.org/wiki/Continuous_delivery
[CD.2] Fowler: http://martinfowler.com/bliki/ContinuousDelivery.html
[CD.3] Agile Alliance: http://guide.agilealliance.org/guide/cd.html
XP
[XP.1] eXtreme Programming: https://en.wikipedia.org/wiki/Extreme_programming
Feature Flags
[FF.1] FeatureToggle - Martin Fowler: http://martinfowler.com/bliki/FeatureToggle.html
Canary Releases
[CR.1] CanaryRelease - Danilo Sato: http://martinfowler.com/bliki/CanaryRelease.html
Misc
[M.1] Booch method: https://en.wikipedia.org/wiki/Booch_method
[M.2] One Hacker Way - Erik Meijer: https://vimeo.com/110554082
[M.3] https://medium.com/gutefrage-net-engineering/how-to-escape-from-the-
deployment-and-integration-hell-5c55e1628137#.o7d0n3a5h
https://medium.com/gutefrage-net-engineering/how-to-escape-from-the-deployment-and-integration-hell-5c55e1628137#.o7d0n3a5h
Recommended