Migrate Module for Drupal: Intro For Beginners

This blog post is the first of a small series of posts discussing the Migrate module, typical use cases and tips when using Migrate.

Most websites or E-commerce websites require us to take data from a legacy source and move it into Drupal. Eg, make our old content into Drupalized content fully integrated as Nodes, Users, Files or Fields.

The tool I use most for this job is the Migrate module. This module is not expressly for beginners in PHP or Drupal. If you want something with a simple UI and interaction to import content into Drupal I recommend you look at the Feeds module. If you require flexibility, reproducible test runs while you fine-tune your import or a complex long-running migration workflow then Migrate is for you!

What is a Migration, What is Migrate?

A Migration, or a Data Migration described by Wikipedia is:

Data migration is the process of transferring data between storage types, formats, or computer systems. Data migration is usually performed programmatically to achieve an automated migration, freeing up human resources from tedious tasks. It is required when organizations or individuals change computer systems or upgrade to new systems, or when systems merge (such as when the organizations that use them undergo a merger or takeover).

To achieve an effective data migration procedure, data on the old system is mapped to the new system providing a design for data extraction and data loading. The design relates old data formats to the new system’s formats and requirements. Programmatic data migration may involve many phases but it minimally includes data extraction where data is read from the old system and data loading where data is written to the new system.

Migrate is a Drupal module that provides a Framework to define and codify a complete Migration of Data between disparate data formats and systems. If that description sounds vague — Migrate lets you write PHP classes that let you take Sources of data and define their Destination and any number of sequential, related steps to retrieve, massage and move the data from point A to point B.

Migrate is a general purpose migration tool and is by no means a Drupal-only tool. You could for example define a migration that for a set of Javascript based templates and Ajax framework performs the following:

  • create a migration from a list of legacy HTML files in a directory

  • process each file and create a JSON file summarizing the HTML contents, author, etc.

  • These JSON files could be served or used via Javascript or JS framework as needed

By default Migrate comes with a wealth of pre-built classes to interact with Drupal Core components such as Fields, Entities, Nodes, Taxonomy Terms and Files. The additional Migrate Extras module contains 3rd party contrib support for other popular Drupal data (such as GeoEncoded Fields).

I’ve migrated data many times between systems (Joomla to Drupal, Mambo to Drupal) or sometimes between databases (SQLite to MySQL, Access to MySQL). In general the steps to moving a piece of data involve:

  • load original source item A

  • check it, sanitize it, and verify its OK as-is

  • figure out the destination location of this piece of data (a row in some new database schema, a specific folder for some image gallery)

  • move it, perform any extraneous cleanup

Any migration typically involves the above steps. The computer language doesn’t matter and the systems involved don’t matter. A simple ad-hoc pseudo code PHP script to do this might look like:

# connect to legacy ane new DB

$legacy = new PDO($db = new PDO("pgsql:dbname=legacy;host=localhost", "username", "password" );

$new = new PDO($db = new PDO("pgsql:dbname=new;host=localhost", "username", "password" );

# get old data
foreach ($legacy->query("Select * From data") as $row) {
 # iterate and migrate the data to new location.
 # note we change the legacy data a little by capitalizing words.
 $new->exec("INSERT INTO awesome_data SET bar=:bar", array(':bar' => ucwords($row['name']));

echo "all done!";

In subsequent posts will look at how to define a simple migration, and some variations using Migrate.

Looking for quality Drupal Web Hosting? Look no further than Arvixe Web Hosting!

Tags: , , , , , , , , , | Posted under Drupal | RSS 2.0

Author Spotlight

David Gurba

I am a web programmer currently employed at UCSB. I have been developing web applications professionally for 8+ years now. For the last 5 years I’ve been actively developing websites primarily in PHP using Drupal. I have experience using LAMP and developing data driven websites for clients in aviation, higher education and e-commerce. If you’d like to contact me I can be reached at david.gurba@arvixe.com

Leave a Reply

Your email address will not be published. Required fields are marked *