October 26, 2011Boston, MA

Migration Practice

Wednesday, October 26, 2011

Moshe Weitzman

• 1st core patch in 2001

• 1st USA developer

• #1 contributor on Ohloh. Rank 11 on CTR

• Drush, Devel, Organic Groups,

• groups.drupal.org founder

Wednesday, October 26, 2011

Mike Ryan

• Lead engineer

• Drupal dev since 2005

• Pathauto module

• Migrate module

• CTR score of 8.

Wednesday, October 26, 2011

Clients

Wednesday, October 26, 2011

Data migration fears

High risk

Lots of manual work

Always goes over budget

Full of gotchas and bad surprises

Not a reusable investment

Lots of downtime during cutover

I’ll get fired if we fail

Wednesday, October 26, 2011

Methodology

✓ Client is heavily engaged from the beginning.

✓ Client has web app for monitoring progress.

✓ Show early results, and frequent progress.

✓ Completely scripted. No manual steps. No surprises.

✓ Migrate module is already battle tested.

✓ Incremental migrations. Drupal lags by only 5 minutes.

Goal: ConfidenceWednesday, October 26, 2011

Migrate Module• A Drush application written by Cyrve.

• Only enterprise grade migration tool for Drupal.

• Robust - many site changes don’t affect migration

• Fully code driven. No config in DB. Easy to deploy.

• UI for clients and developers to collaborate.

• Shows not yet dispositioned source data.

• Shows progress for any un-progress import.

Wednesday, October 26, 2011

Import Flow

id name

11 sue

22 fred

44 alan

source drupal

11 33

22 44

Fetch

source drupal

name uname

email mail

Map Munge Save Log

<xml>

Wednesday, October 26, 2011

Intimate with data sources

• DB: mysql, postgres, mssql, oracle

• Formats: csv, xml, json

• Local sources or remote web services

• See Migrate/plugins for example code

Wednesday, October 26, 2011

Build Drupal content types and fields

• Forum post

• User profile

• Community (an Organic group)

• Tutorial

• Reference document

• Announcement

• Image, Video, Audio

Wednesday, October 26, 2011

Write migration classes

• A class for each import. The guts of the code.

• Defines a fetch SQL query (or media file path, ...)

• Mapping source column to Drupal field

• Dependency declaration (Y must run before X)

• Hooks for data massaging

• Hooks for referenced item creation

• Kick off any post-import routines

Wednesday, October 26, 2011

Import and rollback• Can choose one migration or all.

• Can limit import to N records or specific records.

• Maintains a map between source and destination IDs.

• Error reporting and logging.

• Facilitates the import/rollback/code/re-import loop.

• Reports throughput and % complete.

• XHProf integration for speed and memory optimization.

• Developer or sysadmin can perform imports.

Wednesday, October 26, 2011

Go Live

Client has been reviewing Drupal site for weeks.

Drupal site gets tweaked and improved until ...

Client is satisfied - point load balancer to Drupal.

No major data migration. We are always synched.

Champagne.

Wednesday, October 26, 2011

Success with Acquia

✓ We have never failed. Nobody gets fired ☺

✓ Extremely experienced in Drupal and Migration.

✓ We achieve high confidence and comfort via:

✓ Highly repeatable process.

✓ Data is constantly synched. Easy to evaluate new site.

✓ Client is continously involved and informed.

Wednesday, October 26, 2011