Upgrading Drupal 6 to 7 using a Bash script

David G - DrupalRecently I upgraded a website from Drupal 6 to Drupal 7. Here I’ll be outlining my approach and the tools I used in this process. I believe a similar approach is usable in many cases for a Drupal 7 to Drupal 8 upgrade path. I have been trying to develop more Drupal solutions using Code over Configuration in my last few projects.

What is Code Over Configuration?

The goal of this approach of website development is to leverage scripting tools and the APIs available in the Drupal ecosystem to perform a site upgrade from Drupal 6 to Drupal 7. That means if database updates must occur, do this task using Code and not manually live on the website. If Drupal modules or 3rd party javascript libraries are needed by our new site, then fetch them programmatically. If new database settings or cleanup must occur in the database then write SQL logic to update your schema and data. The benefits of using Code over manaully performing all these tasks are:

  • The process is repeatable. As you build a new site incrementally adding new components and completing features of the site you can add the steps needed for the new functionality and by re-running the whole process to test the complete build process you assure you haven’t broken the project, or it’s parts.
  • Code can be versioned in a Version Control System such as Git. The script I wrote I was able to track changes to it and my overall progress by using Git and tickets in a project management system.
  • Code can have comments and documention for future eyes. If I ever leave my current position and my successor adopts the project — he may wonder how I accomplished some task or feature within the website. The build process is simply an incremental set of directions to build out the site. The answer is in there!
  • The process is repeatable … oh wait I said that; well in this case I mean the script or approach I’ve built out is largely repeatable for many Drupal 6 to Drupal 7 upgrades — or I wager Drupal 7 to Drupal 8 upgrades. Once you’ve built a particular type of wheel in the past — no point in inventing it again in the future so we can use a skeleton of this script, or workflow, as a starting off point for future similar projects.

The Project Setup and Script Skeleton

For my project I have the following project folder structure:

Project Root Directory
 |- dump.tar.gz
 |- d7-plain.tar.gz [generated by build process]
 |- update-to-d7.sh
 |- /docs
 |- /src
 |- /snapshot
 |- /dest_www
 |- .git

The folders hold the following content:

  • docs: project documentation, designer mockups, 3rd party library files usually in Zip format, brief research notes or implementation gotchas I encounter.
  • snapshot: A full webroot of the existing Live Drupal 6 site.
  • dest_www: The destination webroot for the generated/built/upgraded Drupal 7 site. The Drush command site-upgrade is pointed to this location.
  • src: A directory containing custom sourcecode of modules and themes, standalone php scripts to be invoked via Drush.
  • The project root directory contains dump.tar.gz and update-to-d7.sh. These files are a Drush Archive Dump of the snapsot of the Drupal 6 website and the update-to-d7.sh Bash script is our script to update the website.

The contents I place into a Git repository is usually: docs, src and update-to-d7.sh. I assume backups exist of the live Drupal 6 site elsewhere and the actual Drupal 7 site is built from our script so I consider it output that is not tracked in this Git repository.

The Update Script (Layout and Excerpts)

My update script is in total just shy of 500 lines of code. While this may seem scary it really follows a basic structure:

# ... create other project variables, src, destination, etc.

if [ "$FAST_BUILD_MODE" = "slow" ]; then
    echo "Taking Drupal 6 sitedump and migrating up fully to D7+addons ..."
    drush arr complit.tar.gz --destination=/var/www/2013/complit/snapshot/www/ --debug --overwrite;

    # Remove un-needed or problematic D6 Modules before upgrade.

    # ...
    drush @complit.dev     pm-disable rules -y;
    drush @complit.dev     pm-uninstall rules -y;

  # The D6 WYSIWYG didn't upgrade properly it seems, remove and later reinstall 2.x-dev.
  # 2.x-dev needed because current 2.2 release from 2012 doesn't read version values of CKEditor library correctly.

  # 11-17-2014 kill views_bulk_operations and xmlsitemap.

  # Update all D6 modules/site to current Stable releases.
    drush @complit.dev    up --yes;

  # Upgrade to D7.
    drush @complit.dev    site-upgrade @complit.dest --auto --create-db;
  # Overwrite temp path for local dev work.
    drush @complit.dest   vget file_temporary_path
    drush @complit.dest   vset file_temporary_path "/tmp"
  # Save copy of "Plain" Upgraded Drupal 7 site.
    drush @complit.dest arb comparative-literature,default --destination=${SELF}/d7-plain.tar.gz --overwrite
    echo "Fast build mode -- restore from D7 archive-dump of previous build process."
    sudo chown -R dgurba:www-data $DESTROOT
    drush arr ${SELF}/d7-plain.tar.gz --overwrite --destination=$DESTROOT

# Enable D7 required modules for site features. For example:
drush @complit.dest dl configuration -y;
drush @complit.dest en configuration configuration_ui -y;

# Lots lots more D7 modules downloaded and enabled ...

# Download 3rd party JS libraries ... for example:

# Backbone.js dependancy.
wget https://github.com/jashkenas/backbone/archive/1.0.0.zip -O ${DESTROOT}/sites/all/libraries/backbone-temp.zip;
unzip ${DESTROOT}/sites/all/libraries/backbone-temp.zip -d ${DESTROOT}/sites/all/libraries/;
rm ${DESTROOT}/sites/all/libraries/backbone-temp.zip;
mv ${DESTROOT}/sites/all/libraries/backbone-1.0.0 ${DESTROOT}/sites/all/libraries/backbone;

# Run Drush scripts to fix random bugs. For example:
echo " *** Updating Theme Settings *** "
cat - > /tmp/theme-settings.php <<"EOF-PHP"
 module_load_include('inc', 'system', 'system.admin');
 global $theme_key;
 // Nicked strait from commons_install(), but uses $theme_key,
 // which we hope is the active theme.
 foreach (array('adaptivetheme', $theme_key) as $theme_name) {
 $form_state = form_state_defaults();
 $form_state['build_info']['args'][0] = $theme_name;
 $form_state['values'] = array();
 drupal_form_submit('system_theme_settings', $form_state);
 drupal_set_message(t('Cached CSS files rebuilt for !themename', array('!themename' => $theme_name)), 'status');
drush @complit.dest php-script theme-settings.php --script-path=/tmp

# Import D7 Configuration Files. For example:
echo "Copying config dir contents to live site."
cp -R ${SELF}/config/* $SUBSITE/files/config/
sudo bash /var/www/fix-drupal-permissions.sh /var/www/2013/complit/snapshot/dest_www/ dgurba
drush @complit.dest cc drush
drush @complit.dest config-sync -v --source=$SUBSITE/files/config

I’ve tried really really hard to just provide the skeleton of this script here and leave out its bulk. Important key pieces or processes of this script are:

  • The build script takes a parameter of simply fast to say “don’t re-build Drupal 6 to Drupal 7 from the base Drupal 6 dumpfile”. If fast is passed to the script then a d7-plain.tar.gz dumpfile is expanded as the starting point of the build process.

This is because my Drupal 6 is stable, sure it’s slowly getting new content but it’s functionality is set in stone. An upgrade to D7 now, or 5 months from now yields the same starting point from a Drupal site configuration standpoint. I can focus on developing the Drupal 7 features after simply upgrading the D6 site once! (It took me a few days of running this script  fully to realize this :/). So using the fast mode lets me debug and build out my Drupal7 version of the site quickly … when everything is finally done I run the slow version of the complete update 1 last time from beginning to end — ideally.

  • I use Drush Aliases to point to the D6 snapshot of the site and the D7 generated version of the site!

Heaven forbid typing the full path for everything!

  • When I fetch remote 3rd party libraries I assure that 1) the version number is compatible with Drupal and any modules that need it and 2) I explicitly declare the version of the dependent library (Backbone 1.0.0).

This helps the script not break from day to day and vendor libraries are updated regularly.

  • If needed I create simple PHP code invoked by Drush to perform cleanup and administrative tasks. If the task is huge I write the PHP code in-line in Bash and save it to a file in my systems /tmp directory to be processed. I simply like having 1 update script and not a million little dependent files.

Simple Drush commands could look like following if they fit on 1 or 2 lines:

drush @complit.dest ev “require_once DRUPAL_ROOT.’/includes/actions.inc’; actions_synchronize(actions_list(), TRUE);”

The above code removes orphaned Actions from modules that defined actions at one point and were the module was uninstalled but the Action wasn’t properly removed from the system.

Again the full script is much bigger. But has those basic worksteps and the self imposed guidelines I used to develop the script.

Note: Those readers that have used Continuous Integration (CI) processes may think this is close to or able to be used via CI. It probably can but being a small shop we’ve not had the time yet to invest in adopting a CI app/platform.

Some Pitfalls I Encountered

Everything isn’t all fun under the sun. A headache I usually have with this approach is that there is some Content I need to generate in the update. For example sometimes my projects use the HeaderImage or the Menu Reference module to place a graphic throughout 1 section of the website (such as the url personnel/*). Some of this content I can manage if I’m using Display Suite and creating a new Footer for example using custom ctools content. But, other times the content is not easily created. Some people have said try to use Migrate from a CSV file to make this custom content. I use Migrate alot ! … but it seems like alot of coding for 10 pieces of content and other site flare.

Looking for quality web hosting? Look no further than Arvixe Web Hosting!

Tags: , , , , | Posted under Drupal, Drush | RSS 2.0

Author Spotlight

David Gurba

I am a web programmer currently employed at UCSB. I have been developing web applications professionally for 8+ years now. For the last 5 years I’ve been actively developing websites primarily in PHP using Drupal. I have experience using LAMP and developing data driven websites for clients in aviation, higher education and e-commerce. If you’d like to contact me I can be reached at david.gurba@arvixe.com

Leave a Reply

Your email address will not be published. Required fields are marked *