A pain free (sort A pain free (sort of) of)

approach to the approach to the Migraine (aka Migraine (aka

Migrate)Migrate) module. module.

2

Dennis SolisDennis Solis

Solis Media GroupSolis Media Group Over 20 years of application software Over 20 years of application software

development.development. Two years of Drupal developmentTwo years of Drupal development Procedural php coderProcedural php coder Learning OO phpLearning OO php Do not understand the magic of OODo not understand the magic of OO Migraine module helpedMigraine module helped

3

My AssignmentMy Assignment

Migrate OA Beta to OA 1.4 on Drupal 6. Migrate OA Beta to OA 1.4 on Drupal 6.

4

Data Migration Data Migration Options Options

Move data by handMove data by hand Feeds and Feeds TamperFeeds and Feeds Tamper Migrate moduleMigrate module

5

Smoothing the Move Smoothing the Move http://denver2012.drupal.org/program/http://denver2012.drupal.org/program/

sessions/migration-smoothing-movesessions/migration-smoothing-move Analyze – understand the legacy dataAnalyze – understand the legacy data Map – Line up source fields with Map – Line up source fields with

destination fieldsdestination fields Iterate – try, try, again – until it worksIterate – try, try, again – until it works Congratulations – you doneCongratulations – you done

6

Feeds Feeds No coding except for special No coding except for special

conditionsconditions Setup / execution is done via a UISetup / execution is done via a UI LOT easier to setup and useLOT easier to setup and use Comes with a mapping screen.Comes with a mapping screen. It is limitedIt is limited

Receiving module needs to support feeds.Receiving module needs to support feeds. Does not support all fields but you can Does not support all fields but you can

write code to handle the fields.write code to handle the fields.

7

Feeds Feeds Works greatWorks great Limited by lack of field handlersLimited by lack of field handlers It is a bit slowIt is a bit slow

Especially for LOTS of dataEspecially for LOTS of data Does not work well ifDoes not work well if

Different contents types that reference Different contents types that reference the other.the other.

8

Feeds TamperFeeds Tamper Manipulate the data before Manipulate the data before

processingprocessing Works with FeedsWorks with Feeds

9

Migrate ModuleMigrate Module Import data / contents into Drupal 6, 7, Import data / contents into Drupal 6, 7,

and 8and 8 Migrate for Drupal 6 and 7 are about Migrate for Drupal 6 and 7 are about

95% compatible95% compatible Drupal 6 – uses dbtng and autoloadDrupal 6 – uses dbtng and autoload Mapping is done in codeMapping is done in code Very flexibleVery flexible Much FasterMuch Faster

Migraine is here to stay.Migraine is here to stay. It is in Drupal 8 core It is in Drupal 8 core

10

MigrateMigrate

Migrate 2.0Migrate 2.0 New Migrate 7.2-x-dev - – released 10-01-New Migrate 7.2-x-dev - – released 10-01-

20132013 Migrate 2.0 for D6 requires the autoload and Migrate 2.0 for D6 requires the autoload and

dbtng modules.dbtng modules. Migrate d 2 dMigrate d 2 d

New Migrate 7.2-x-dev – released 10-01-2013New Migrate 7.2-x-dev – released 10-01-2013 Migrate Extra – Sub-module within Migrate Extra – Sub-module within

MigrateMigrate

11

Migrate D 2 DMigrate D 2 D

Use this to migrate from 6 to 7 or 8Use this to migrate from 6 to 7 or 8 All the benefits and pain of the All the benefits and pain of the

migrate modulemigrate module Provides class specific to older Provides class specific to older

versions of Drupal.versions of Drupal. It does a lot of the work that I had to do It does a lot of the work that I had to do

manuallymanually I could not use because I went D6 to D6.I could not use because I went D6 to D6.

12

Migrate OGMigrate OG

OOnly use if you are using Migrate 1.0

The Organic Group migration support for Migrate 2.0 has been moved into Organic Groups itself.

13

Drupalcon Denver Drupalcon Denver Andrew Morton stated the migrate Andrew Morton stated the migrate

module:module: Two weeks of banging his head Two weeks of banging his head Tricky to get started withTricky to get started with Steep learning curveSteep learning curve Inscrutable Inscrutable ComplexComplex

14

Learning CliffLearning Cliff Open AtriumOpen Atrium Open Atrium Data Open Atrium Data

StructureStructure Migrate ModuleMigrate Module OO Php OO Php

15

My Gifts to You My Gifts to You How to connect to multiple databasesHow to connect to multiple databases Easily find the D6 source data structureEasily find the D6 source data structure How to user the Migration UIHow to user the Migration UI How to write migration codeHow to write migration code

16

Ease the pain Ease the pain

phpMyAdmin phpMyAdmin Devel moduleDevel module Migrate UIMigrate UI Beer and Wine Beer and Wine

examplesexamples DrushDrush

17

phpMyAdmin phpMyAdmin How to read the keys to the tablesHow to read the keys to the tables Click on the index tabClick on the index tab Database search for dataDatabase search for data

18

Open Atrium StructureOpen Atrium Structure

19

OA Data StructureOA Data Structure

TaxonomyTaxonomy User and User ProfileUser and User Profile Organic Group nodesOrganic Group nodes BooksBooks Content TypesContent Types CommentsComments Case Tracker Data Structure

20

Tools you will needTools you will need Football helmetFootball helmet A LOT of aspirinsA LOT of aspirins PatiencePatience TenacityTenacity A little bit of

knowledge A little OO A little migration

code

21

Devel ModuleDevel Module

DevelDevel Devel node accesDevel node acces

22

Migrate ModuleMigrate Module

Migrate UIMigrate UI Migrate Migrate Migrate ExtrasMigrate Extras

23

Other ModulesOther Modules

dbtng – back ports many of the Drupal 7 database functions back to D 6

autoload – helper module to autoload classes under PHP 5

24

OO definitionsOO definitions

An object is a self-contained component that contains properties and methods needed to make a certain type of data useful. For example, in a project management

application, you would have a: Status object Cost object Client object among others.

25

OO definitionsOO definitions

An object’s properties are what it knows. (This of this as variables related to the object.)

Its methods are what it can do. (This if this as functions or subroutines.)

26

OO definitionsOO definitions

A class is a blueprint or template or set of instructions to build a specific type of object.

Every object is built from a class. Each class should be designed and programmed to accomplish one, and only one, thing.

27

OO definitionsOO definitions

An instance is a specific object built from a specific class. It is assigned to a reference variable that is used to access all of the instance's properties and methods. When you make a new instance the process is called instantiation and is typically done using the new keyword.

28

OO definitionsOO definitions

A class is like a recipe for chocolate cake. The recipe itself is not a cake. You can't eat the recipe (or at least wouldn't want to).

If you correctly do what the recipe tells you to do (instantiate it) then you have an edible cake.

That edible cake is an instance of the chocolate cake class.

29

OO conceptsOO concepts

A class is like a recipe for chocolate cake. The recipe itself is not a cake. You can't eat the recipe (or at least wouldn't want to).

If you correctly do what the recipe tells you to do (instantiate it) then you have an edible cake.

That edible cake is an instance of the chocolate cake class.

30

Your Migrate ModuleYour Migrate Module

Migration should be in a SEPARATE module So you can turn it off once you are

done. Isolate it if you are going to be

doing ongoing date based updates.

31

Migrate supports Migrate supports

Support multi-field or composite keys. Tracks the source id and it’s final

destination id and you can access this information during your migration.

Translate from the old key to the new key.

Can do a roll back. Can re-do a migration and update those

articles in place. Do “date” based ongoing migrations.

32

Migration class Migration class Configure all of your pieces for the

migration Selects the source data and map it

to the destination data.

33

Migration class Migration class prepareRow(), prepare(),

complete() Allow you to manipulate the data or skip

records Update other rows while migrating data

Writes out the destination data.

34

Migration class Migration class

35

Source dataSource data Source data can be in SQL, CSV, XML,

JSON format Need to have definition of these files. I used the SQL format.

36

Field HandlersField Handlers A field definition within Drupal is an

array structure Converts simple string data into the

Drupal array structure

37

Destination HandlersDestination Handlers comments.inc entity.inc fields.inc file.inc node.inc path.inc table.inc table_copy.inc term.inc user.inc

38

DestinationDestination Migrate will write the Destination

nodes for you (node, user, other supported entities, or SQL rows).

Create one destination record for each source record

39

Order is criticalOrder is critical Must migrate user accounts before Must migrate user accounts before

you can migrate user profiles.you can migrate user profiles. Know you dataKnow you data Create a list of the sequence in Create a list of the sequence in

which the data is to be converted.which the data is to be converted.

40

Migration ClassMigration Class Everything is done in code You write a “base class” from which

you then extend in your code. In this base class, you can define things about this migration.

Everything you want to do is an extension of an existing migrate class.

41

Migration ClassMigration Class Inherit and extend these classes You have to write a small module to

“extend” the classes. Documentation is provided in the

form of code examples. beer.inc wire.inc

42

Multiple DB in D6 Multiple DB in D6

Database select from within current database:

* $db_url = 'pgsql://username:password@localhost/databasename';

*/

*// - das - $db_url = 'mysqli://root@localhost/mis_project';

$db_url['default'] = 'mysqli://root@localhost/mis_project';

$db_url['legacy'] = 'mysqli://root@localhost/open_atrium_legacy';

$db_prefix = '';

 

Database select from within current database:

$query = db_select('users_legacy', 'ul');

43

Multiple DB in D7 Multiple DB in D7 Define the databaseDefine the database

Database::addConnectionInfo('jnn_xoops', 'default', array(

'driver' => 'mysql',

'database' => 'jnn_xoops',

'username' => 'root',

'password' => 'root',

'host' => 'localhost',

'prefix' => '',

));

Define the queryDefine the query$query = Database::getConnection('default', 'jnn_xoops')

->select('xoops_users', 'xu')

->fields('xu', array('uid', 'name', 'uname', 'email'));

44

My ModuleMy Module Structure of my migration module

was placed in sites/all/modules/custom.

My module name is: migrate_mc_oa migrate_mc_oa.info migrate_mc_oa.install migrate_mc_oa.module migrate_mc_oa.inc

45

My ModuleMy Module Structure of my migration module

migrate_mc_oa_user.inc migrate_mc_oa_node_og.inc migrate_mc_oa_book.inc migrate_mc_oa_node_ct.inc migrate_mc_oc_comment.inc

46

My ModuleMy Module

47

Let’s see the codeLet’s see the codeclass McOaBasicMigration extends McOaMigration {

public function __construct() {

parent::__construct();

$this->description = t('Descriptions');

$this->dependencies = array('Other_Migration_Dependencies');

$this->map = new MigrateSQLMap(....));

 

$query=db_select('comments_legacy', 'cl');

or

$query = Database::getConnection('default', 'legacy')

->select('comments', 'cl');

$this->source = new MigrateSourceSQL($query);

 

$this->destination = new MigrateDestinationComment('comment_case_tracker_case');

 

$this->addFieldMapping('pid', 'pid');

$this->addFieldMapping('nid', 'nid')

->sourceMigration('McOaCaseCt');

 

// Unmapped destination fields

$this->addUnmigratedDestinations(array('fname', 'lname'));

48

Drush Migrate CmdsDrush Migrate Cmds https://drupal.org/node/1561820https://drupal.org/node/1561820 drush migrate-import Article drush migrate-import Article --

limit="100 items" drush migrate-import --all=User drush migrate-import –rollback –all drush migrate-import –update –group =

[grp_name]