[ThoughtWorks] Perforce for Release Engineers

1  

Perforce for Release Engineers

Logan McGrath Developer Consultant ThoughtWorks

2  

Managing Configuration How can configuration be managed as an application goes from DEV through PROD environments?

3  

Movement of Configuration through a Pipeline

Email server host configuration

4  

Common Configuration Strategies

•  All configuration stored in single file •  Specific configuration deployed with application •  Configuration stored in relational database •  Etc...

Each strategy has problems!

5  

Single Configuration File

Application is built containing the configuration file or is co-deployed with the file

6  

Configuration File Per Environment

The application is built with an environment-specific configuration file or co-deployed with one.

7  

Configuration Database Per Environment

The deployment uses a database to store configuration for the respective environment.

8  

Using Perforce for Configuration Why and how Perforce beats all other options

9  

Why is Perforce the better choice for application configuration?

•  Change audit •  Permissions and access control •  Maintained divergence •  Live configuration

•  Support for feature toggles http://martinfowler.com/bliki/FeatureToggle.html

10  

How would application configuration with Perforce work?

•  Applications fetch their own configuration over HTTP/REST •  Or config pushed to them

•  Configuration stored in plain-text, newline-delimited formats that are human-readable •  XML •  JSON

11  

App-Config-App in Action

•  Making a change •  Committing the change •  Merging the change

from DEV to QA •  User permissions

between environments

12  

App-Config-App in Action

•  Making a change •  Committing the change •  Merging the change

from DEV to QA •  User permissions

between environments

13  

App-Config-App in Action

•  Making a change •  Committing the change •  Merging the change

from DEV to QA •  User permissions

between environments

14  

App-Config-App in Action

•  Making a change •  Committing the change •  Merging the change

from DEV to QA •  User permissions

between environments

15  

App-Config-App in Action

•  Making a change •  Committing the change •  Merging the change

from DEV to QA •  User permissions

between environments

16  

Applications Using the App-Config-App

Obtaining runtime configuration from the App-Config-App and a demonstration

17  

Application Usage

Runtime configuration of an application using the App-Config-App

18  

Strategies for Propagating Changes

•  Applications poll against config MD5 hash for changes

•  App-Config-App notifies applications of changes •  App-Config-App publishes ATOM feed,

applications poll for updates then query the changes

19  

Scaling the App-Config-App

Horizontal scaling of foo.com

20  

The Internals of the App-Config-App Overview of architecture and technologies used

21  

Tech Choices in Our Proof of Concept

•  HTML5 + AngularJS •  Sinatra (Ruby)

•  REST interfaces •  Pretty-printed JSON

•  Perforce backend - of course!

22  

Why JSON and not XML?

•  JavaScript: JSON’s natural habitat •  Transparent serialization and deserialization •  Supported everywhere

•  Natively in .NET, Ruby, Python, and PHP •  Supported in Java through libraries like Jackson

23  

Architecture

•  Roughly 100 lines of JavaScript on the front-end using AngularJS

•  Roughly 400 lines of Ruby on the back-end using Sinatra

•  Ruby drives all interaction with Perforce server and working copy using the p4 command line client

AngularJS  

Sinatra  

Perforce  Working  Copy  

24  

Architecture

•  A different branch is created in Perforce for each environment the application runs in.

•  The application fetches configuration stored in Perforce through the App-Config-App

25  

Directory Structure

•  dev/ •  stack_configuration.json - the configuration •  stack_configuration.html - the form to manipulate it •  stack_configuration.js - additional javascript

•  qa/ •  uat/ •  stage/ •  prod/

26  

App-Config-App Source

27  

28  

29  

30  

31  

32  

Alternative Technologies

•  Client-side technology alternatives to AngularJS •  Backbone, Knockout (and 20 others)

•  Server-side technology alternatives to Sinatra •  Java •  .NET •  Rails •  PHP •  Python

•  Generally, any technology that supports the following capabilities will work: •  HTTP, HTTPS •  JSON over REST •  Driving interaction with Perforce using p4 on the command line

33  

Recap

•  AngularJS •  Easy client-side

•  Sinatra •  Easy server-side

•  P4 command line client •  Easy integration with P4D

34  

Questions? Logan McGrath lmcgrath@thoughtworks.com www.thoughtworks.com www.loganmcgrath.com