AGILE PLAYBOOK FOR SOFTWARE TEAMS

Enterprise Drupal Edition

THE SOFTWARE CRAFTSMANSoftware Craftsman n. A man who practices the software craft.

1. “software craft”A craftsman believes that software is a craft. This is important because not everyone believes this. A craftsman takes pride in his work an

strives to do the best job he can. He believes that writing good software requires skill and careful attention. That software is not something

that can be manufactured nor can it be delivered faster by merely adding more bodies.

2. “practices”A craftsman practices his craft, always striving to become more skillful, to produce better software.

There are traditionally 3 stages of craftsmanship:

1. Apprentice

2. Journeyman

3. Master

No matter which stage one may be in, as long as he practices software as a craft, he is a craftsman.

3. “A man”Technically the term “craftsman” is gender specific. Women are just as capable of software craftsmanship. Indeed, I’d like to see more software

craftswomen out there. In an effort not to alienate anyone we should use the term “software craftsperson” more liberally.

APPLICATION DEVELOPMENT: MANIFESTO

We are continually uncovering better ways of developingsoftware by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items onthe RIGHT, we value the items on the LEFT more.

WE FOLLOW THESE 12 PRINCIPLESOur highest priority is to satisfy the customer

through early and continuous delivery of valuable software.

Welcome changing requirements, even late in development. Agile processes harness change

for the customer's competitive advantage.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference

to the shorter timescale.

Business people and developers must work together daily throughout the project.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

The most efficient and effective method of conveying information to and within a

development team is face-to-face conversation.

Working software is the primary measure of progress.

Agile processes promote sustainable development. The sponsors, developers, and users should be able to

maintain a constant pace indefinitely.

Continuous attention to technical excellence and good design enhances agility.

Simplicity--the art of maximizing the amount of work not done--is essential.

The best architectures, requirements, and designs emerge from self-organizing teams.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its

behavior accordingly.

OUR OPEN SOURCE MENU FOR SUCCESS

Our Development Group:• Is part of a global community with a common vision• Is a pool of talented and motivated development professionals • Is a mix of open source and commercial• Follows an organized, light weight software development approach• Has a clear leadership structure• Excels at communication• Is a business model for success• Is Agile

SCRUM: A LIGHT WEIGHT DEVELOPMENT PROCESS

Based on Agile Programming methodology and Getting Real, our Scrum process is:

• High intensity design, test, implementation cycle

• Supported with web-enabled tools• Automated testing integrated with the

software development cycle• Short product delivery iterations to

provide constant business value

OUR DELIVERY MODEL:A “CONTINUOUS” AGILE PROCESS

We will compresses the standard Analyze, Design, Implement, Test cycles into a Continuous Agile Process of Design, Develop, Implement and where Testing is always being done.

Requirements

Design

Code

Test

Requirements

Design

Code

Test

Requirements

Design

Code

Test

Requirements

Design

Code

Test

Reqs. Design Code

… Test …

Agile (Continuous)

Reqs Design Code

… Test …

Reqs. Design Code

… Test …

Reqs. Design Code

… Test …

Reqs. Design Code

… Test …

Waterfall Iterative

OUR DELIVERY MODEL: SEQUENTIAL VS. OVERLAPPING DEVELOPMENT

Requirements Design Code Test

...Our team does a little of everything all

the time

Rather than doing all of one thing at a

time...

Sequ

entia

lO

verla

ppin

g

OUR AGILE DEVELOPMENT MODEL

Behavior Driven Development (BDD)

MAPPING AGILE BDD

OUR AGILE PRACTICE

OUR AGILE PRACTICE

CODE MOVES FORWARD,CONTENT MOVES BACKWARD

Functionality and configuration is developed in Development Environment

Functionality and configuration is tested in QA Environment

Content is developed and previewed in Staging (UAT) Environment

Production contains the Staging (UAT) and Live Content and Code states

Whi le the content and the software lifecycles will be de-c o u p l e d , t h e y a r e a l s o interdependent. Developers need realistic content to develop and test code with, and the content relies on the code to define and display itself.

The standard approach for managing these interdependencies is “code forward, content backward.” Content and configuration is developed in the Development environment and tested in the QA environment. When it is ready, it is deployed to the Live environment for UAT and Production instances. Content is developed and previewed in the Production instance that contains the Staging (or preview/UAT) and Live content states. Periodically, content should be published backwards to the Development and QA instances so that testing can be realistic as possible.

Continuous Integration

Dev QA Live

TEST. DEVELOP. TEST. REVIEW. DEPLOY.

Test Automation

DeployDeploy Deploy

Please note the complete lack of developers in this picture...only Hudson, or similar automated build service, can deploy code, directly from the code revision control system.

(UAT/Prod)

Envi

ronm

ent

Code Quality Control Source CodeCode Revision Control

Build & Release

SOURCE CODE MANAGEMENT: DISTRIBUTED VERSION CONTROL SYSTEMS (DVCS)

In a DVCS (such as Git, Mercurial, Bazaar

or Darcs), clients don’t just check out the

latest snapshot of the files: they fully

mirror the repository. Thus if any server

d i e s , a n d t h e s e s y s t e m s w e r e

collaborating via it, any of the client

repositories can be copied back up to the

server to restore it. Every checkout is really

a full backup of all the data.

Git Basics:•Snapshots, Not Differences•Nearly Every Operation Is Local•Git has integrity (checksum everything)•Git Generally Only Adds Data•Operates in three states (working, staging, local)

http://progit.org/book/

DVCS BRANCH PATTERN: FEATURE BRANCHING

DVCS: FEATURE BRANCHING EXAMPLE

GIT WORKFLOW

CONTINUOUS INTEGRATION WORKFLOW

Start Finish

RELEASE EARLY, RELEASE OFTEN…RELEASE QUALITY.

Although our web developers will be tempted to keep their code under wraps until it is perfect, this process will encourage them to release their code as soon as it passes its intended tests.

The longer the code is visible to the community (other developers, team leads, testers, managers, clients, etc.) the better integrated it will be.

All developers agree to keep their software defect free.

Although everyone is encouraged to submit their code early, the code must compile and

pass nightly tests

A continuous build process sends e-mails to developers who check in code that does not

compile

More effectively, the development group enforces the commitment though peer

pressure

UNIT TESTS + ACCEPTANCE TESTSBuilding The Code Right Automated unit tests are white-box tests that

describe and verify very low-level behavior. Especially when working test-first, developers rely on such tests to make sure that the design is sound, and that there are no defects. Such tests are built by and largely for developers, and can be independent of functionality.

Our web team uses SimpleTest to perform test-driven development. However, if our TDD effort is still hampered by a broken requirements process, we can easily end up building robust, lean, well-factored code with no business value.

Building The Right Code Automated tests are about building the right

code in the first place: the code with the most business value. While developers may be involved in writing such tests, they are not just for our developers. Tests are especially for customers, business analysts, testing/quality professionals, support staff, management, and all other stakeholders who need evidence that the project is producing solid return on investment, every step of the way.

AGILE TOOLS

Engineering• Revision control• Story Mgmt (& Issue Tracking)• Continuous Builds• Data Quality (Schema Analysis)• Code Quality (Code Analysis)

Documentation and Product Management• Project Management (& Issue Tracking)• Community Communication & Collaboration• Technical Documentation• Product Documentation (End User)

Collaboration & Communication • Group Wiki• Live Chat• Video Chat• File Sharing

APPLICATION DEVELOPMENT STACK

Application Development Stack (ADS): Architectural Patterns

ADS: LAMP PATTERN

ADS LAMP PATTERN: DRUPAL STACK

CentOS/RHEL Linux

Apache SolrMySQL

Apache

Zend PHP + APC

MemCached

Pressflow Drupal

Varnish

Through several rounds of benchmarking internally as well as external benchmarks performed by industry leaders, we have found that our previous web application stack for Drupal requires significant hardening for large scale implementations. In light of this, architectural analysis and our commitment to open source, Drupal, LAMP and of course, growth, the below grid outlines our standard Drupal Stack,

hardened by the enterprise scale and commercially supported PHP stack, Zend Server.

PHP WEB APPLICATION SERVER: ZEND

Zend Server is a complete, enterprise-ready Web Application Server for running and managing PHP applications that require a high level of reliability, performance and security on Linux, Windows or IBM i.

Business-Grade PHPAn up-to-date, tested and supported PHP stack ensures high reliability, enhances security and

increases staff productivity

Deploy with confidence

A complete and consistent environment used in development, testing and production eliminates

many of the problems encountered during deployment

Rapid response to problems

Advanced application monitoring and diagnostics enable early

problem detection and quick root cause analysis

Optimal application performance

Built-in optimization and acceleration ensures high

performance and low resource utilization

Highlights: Enhance PHP Application Reliability and Security

Ensure Successful DeploymentsDetect Problems Before the Phone Rings

Quickly Pinpoint Root Cause of Problems With Code Tracing

Boost Application PerformanceJob Queue: Offload Execution of Long-running PHP Scripts

APPLICATION PERFORMANCE MONITORING

What is Application Performance Monitoring?

Application performance monitoring (APM)

detects slow response times, poor throughput,

user errors, and other performance problems, and

then guides us to diagnose the cause. It also

provides powerful features for tuning and

optimizing our apps such as capacity planning and

scalability reporting.

APM helps to ensure the health and availability of

our production and lab PHP web apps — such as

Drupal. We can monitor overall application

response time and throughput including

memcached, database and external services. We

can go deeper into individual web transactions

and database calls to MySQL and all the way

down to SQL statements. Zend Server

(commercial) + New Relic RPM provides this to

us, on demand.

+

CONFIGURATION MANAGEMENTTaking configuration management to the automation level allows us to create a consistent environment spec for our applications, albeit a lab, test, or even production environment. We utilize Chef, an open source systems integration framework, to bring the benefits of configuration management to our entire web infrastructure.

The result is a fully automated infrastructure: when a new server comes on line, the only thing we have to do is tell Chef what role it should play in the architecture.

IMPROVING QOS THROUGH A SIMPLE SERVICE ORIENTATED

ARCHITECTURE (SSOA)

Service Creation & Mgmt• Expose end-points, legacy interfaces, and objects as services• Host these services as a lightweight service container

Service Mediation• Completely separate business logic from messaging• Shield services from message formats and protocols• Enable location-independent service calls and provides protocol bridging

Message Routing• Route messages based on content and rules• Filter, aggregate and re-sequence messages

Data Transformation• Exchange data across applications with varying data formats• Manipulate message payload, including encryption, compression and encoding transformations• Format messages across heterogeneous transport protocols data types

To provide a more agile, autonomous, scalable, integrated and quality driven system for Enterprise Architectures and their applications and clients to operate within, the application stack used for web enabled applications requiring services from back

office systems, such as legacy systems, should consume systems as common services.

TYING THE KNOTS: AGILE TOOLS

We bring our Agile Practice, Development Model, and of course...People to tie this knot. And by people, we mean everyone involved in the product development life cycle, be it agile developers, business and product managers, scrum

masters (or project managers), analysts or users.

We bring this all together by using several tools to provide us integrated agile project management, issue tracking, wiki collaboration, source code analysis and reviews, code version control to improve release planning, team communication and

customer feedback gathering.

Atlassian JIRA + GreenHopper for agile project managementGreenHopper adds agile project management to any JIRA project with one click. GreenHopper is the perfect tool for managing our product and sprint backlogs, planning sprints and tracking our team through the entire release process.

Atlassian Confluence for agile collaborationConfluence is designed to help us as, agile developers, to plan requirements, collaborate on changes and display metrics.

Cloudbees Nectar or Atlassian Bamboo for continuous integrationAs agile developers we use continuous integration to get the most from our unit tests. We use CI to get instantaneous feedback on the impact of each commit and monitor build metrics.

Sonar Crucible for code quality and coverageSonar integrates peer code review and code coverage directly into our agile developer’s work habits. We use Sonar to get timely feedback on our code, maintain quality metrics, assesses the impact of our tests and notifies us of test coverage gaps before it is too late.

Best of breed