Upload
spring-by-pivotal
View
635
Download
6
Embed Size (px)
Citation preview
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
No Outage Database Development with
Spring Boot and Liquibase
Alan Barrington-Hughes Pavithra Ramaswamy
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Agenda
• Goal
• We'll tie together several concepts to perform a no-outage deployment which includes database DDL changes
• Topics
• Blue Green / Canary deployments
• Agile database development concepts
• Introduction to liquibase
• Embedding liquibase db changes within spring boot
• Live demo
2
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Blue Green deployments - central tenets
• You'll have two distinct versions of the codebase infrastructure fronted by a router
• One version is customer facing live (Blue), the other dormant (Green)
• Green acts a stage, into which you deploy the next release
• Upon the warm & fuzzy sign off, the router is instructed to re-direct traffic to Green
• Blue then becomes the dormant (hot standby) system and Green is live
• The following release uses Blue as stage and customer facing line (Green)
• And so on
• Gives a rapid mechanism to roll back
3
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Canary deployments - central tenets
• You'll have instances of the codebase infrastructure fronted by a router
• All systems are running in parallel
• Rather than a complete cut over, only a small portion of the instances are upgraded with new code
• As you gain confidence in the new release more instances are upgraded and receive more traffic until the old version becomes dormant
4
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Database considerations
• Databases can often be a challenge with these technique. Particularly when you need to change the schema to support a new version of the software.
• The trick is to apply a database refactoring that changes the schema to support both the new and old version of the application
• So first, deploy that canary/ green build, check everything is working fine so you have a rollback point, then roll out the new version of the application.
• And when the upgrade has bedded down remove the database support for the old version
5
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Agile database development concepts
• Large feature releases are high risk and more likely for deployment outage
• Small (more frequent) feature release are lower risk and more manageable to deploy without an outage
• Developers are used to the concept of refactoring code when "code smells", databases smell also and are ripe for refactoring
• A large change should be broken down and completed via several small changes
6
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Agile database development refactorings• Structural changes
• Renaming a column
7
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Agile database development refactorings
• Structural changes
• Renaming a column
• Why?
• Increases the readability of the schema / needed to conform to the enterprise naming conventions
8
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Agile database development refactorings
• Structural changes
• Renaming a column
• Why?
• Increases the readability of the schema / needed to conform to the enterprise naming conventions
• How?
• introduce new column
• migrate original data
• introduce synchronization trigger
• (after a deprecation period)
• drop original column & trigger
9
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Agile database development refactorings
• Structural changes
• Moving a column from one table to another
10
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Agile database development refactorings
• Structural changes
• Moving a column from one table to another
• Why?
• Increase normalization to reduce data redundancy
• Denormalization to reduce common joins
11
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Agile database development refactorings
• Structural changes
• Moving a column from one table to another
• Why?
• Increase normalization to reduce data redundancy
• Denormalization to reduce common joins
• How?
• identify insertion & deletion rules
• introduce new column
• migrate data
• introduce triggers
• (after a deprecation period)
• drop original column & triggers
12
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Agile database development refactorings• Structural changes
• Splitting a multipurpose column into several separate columns
13
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Agile database development refactorings• Structural changes
• Splitting a multipurpose column into several separate columns
• Why?
• Increased data granularity ("name" -> "fName", "mName", "lName")
• Overloaded status field
14
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Agile database development refactorings• Structural changes
• Splitting a multipurpose column into several separate columns
• Why?
• Increased data granularity ("name" -> "fName", "mName", "lName")
• Overloaded status field
• How?
• introduce new columns
• migrate data (may need transforming stored proc)
• introduce triggers
• refactor application code
• (after a deprecation period) drop original column and triggers
15
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Agile database development - lessons learned
• Smaller changes are easier to apply
• Uniquely identify refactorings
• Use triggers over views for data synchronization
• Put database assets under change control
16
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Introduction to liquibase - fundamentals
• essentially a scripting language
• implemented and executed in java
• controls the modifications to the schema of a database
• manages/ records all changes to your database
• no change is forgotten
• ability to roll back a release (and re-roll forward)
• out of the box support for multiple DBs (mysql / postgresql / oracle /mssql / derby / hsqldb / h2
• extensible for other databases
17
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Introduction to liquibase - what why how
• snapshot comparison
• change set qualification
• verification
• atomic application
• ensures singleton upgrades
• changes recorded
18
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Introduction to liquibase - changes
• Changelogs (collection of change sets can span many files)
• Changesets (As few changes as possible per change set)
• Changes (Refactorings such as add column/ create table)
• Think small additive changes to the database
• Coded in YAML, JSON or XML
• Liquibase translates change to vendor specific SQL
• Liquibase can automatically rollback sequences of changes
19
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Introduction to liquibase - functionality
• Applying all DDL changes to the schema
• createTable / dropTable / createProcedure / addPrimaryKey / addIndex /
addLookupTable / mergeColumns /sql blocks
• Managing views / stored procedures / triggers
• manage static / reference data
• manage test data
20
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Embedding liquibase within spring boot
21
// This the code required to enable liquibase in spring boot:
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Embedding liquibase within spring boot
22
// This the code required to enable liquibase in spring boot:
// There are however, a few steps of configuration a) You need to add “liquibase-core” dependency.
<dependency> <groupId>org.liquibase</groupId> <artifactId>liquibase-core</artifactId> <version>3.4.1</version></dependency>
b) Put your first change log in “db/changelog/db.changelog-master.yaml”
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Embedding liquibase within spring boot
23
// It goes without saying that a database migration tool also needs a database to migrate:
spring: datasource: driverClassName: org.postgresql.Driver username: demo password: "demo_pass" url: jdbc:postgresql://${PG_PORT_5432_TCP_ADDR}:${PG_PORT_5432_TCP_PORT}/sboot_demo
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Live demo - goals
24
• Two spring boot nodes sharing the same database fronted by ngnix
• Goal is to move a "email" column from the current "user" table into a new table
• Accomplished without the app becoming unavailable.
• Start with initial version v1.0 that creates our tables & app code
• Deploy our new v1.1 code, introducing new column, migrating data, keeping
data in sync using triggers.
• v1.0 and v1.1 are running in parallel and co-existing with the new DB schema
• Once all nodes are on v1.1 remove old column and drop triggers
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Live demo
25
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Footnotes / links
• http://martinfowler.com/bliki/BlueGreenDeployment.html
• http://martinfowler.com/bliki/CanaryRelease.html
• http://www.liquibase.org
• Refactoring Databases ISBN-13: 978-0-321-77451-4
• https://github.com/abhsc/sboot_liquibase_demo
26
Unless o therw ise ind ica ted , these s l ides are © 2013-2016 P ivo ta l So f tware , Inc . and l i censed under a Creat ive Commons At t r ibu t ion-NonCommerc ia l l i cense: h t tp : / / c rea t ivecommons.org / l i censes /by-nc /3 .0 /
Thank you! (and please rate this session)
• Alan Barrington-Hughes
• https://www.linkedin.com/in/alanbarringtonhughes
• @abhPremier
• https://github.com/abhsc
27
• Pavithra Ramaswamy
• https://www.linkedin.com/in/pramaswamy
• @pavi2master
• https://github.com/pavi2master
Premier, Inc. (NASDAQ: PINC) is a healthcare performance improvement alliance of approximately 3,600 U.S. hospitals and 120,000 other providers. Our mission is simple: To improve the health of communities.
We Are Hiring @PremierHATalent