Guide

Learn to Migrate to Drupal for Drupal 8, 9, and 10

Learn to Migrate to Drupal

What will you learn

  • Planning a migration
  • Preparing a Drupal-to-Drupal migration
  • Importing custom data
  • Upgrading a site from Drupal 6 or 7
  • Understanding a migration YAML file
  • Extracting data from custom data sources
  • Writing custom plugins for data manipulation

Overview

Whether you're updating from Drupal 6 or Drupal 7, or importing data from some other source, you need to know about the migrate system in Drupal core. This guide provides a comprehensive look at the features of the Drupal migrate API for both Drupal-to-Drupal updates, and migrating from any external datastore into Drupal.

This guide is for anyone who wants to:

  • Update to the latest version Drupal from Drupal 7 or earlier
  • Import data from external datasources including MySQL, CSV, JSON, or XML into Drupal 9 or 10
  • Provide an update path from Drupal 6 or 7 for a contributed module

Where should I start?

If you're new to the Migrate API, start from the beginning and get an overview of how the system works and how all the pieces fit together. This is essential background information for anyone wanting to migrate data into Drupal. Additionally, part of performing any migration is planning and preparation. Migrations involve preparing and analyzing your source data, building a new website that data can be migrated into, and lots of testing, rolling back, and testing again, in order to get everything right.

Migrating to Drupal Essentials

More information

Whether you're updating from Drupal 6 or Drupal 7, or importing data from some other source, you need to know about the migrate system in the latest version of Drupal. This tutorial provides an overview and links to additional tutorials where you can learn more about how all the individual parts work.

By the end of this tutorial you should have a better understanding of what the migration system is capable of and know where to find more information about how to use it.

More information

To follow along with the rest of the migration tutorials you'll want to make sure you understand the following concepts and terms as they relate to Drupal migrations.

In this tutorial, we'll take look at the basic components of a migration and familiarize ourselves with some of the concepts and terminology needed to understand how the system works. We'll cover:

  • What is a migration?
  • Migration templates
  • The extract, transform, load process
  • Destinations and sources
  • Additional Drupalisms

By the end of this tutorial you should be able to identify the various components that a migration is composed of, and explain at a basic level what each is responsible for.

More information

Every migration is unique, which means there are loads of great examples you can review and learn from. We'll keep this tutorial up-to-date with our favorites and let you know how we think these examples might help.

More information

In Drupal, there are 3 modules in core related to migration that you'll want to know about. These modules can help you import data into Drupal from disparate sources, or upgrade from a previous version of Drupal.

In this tutorial we'll look at what each of these core migration modules do, and talk about when you'll need to use them.

More information

There's a whole ecosystem of contributed modules that build on the Migrate API in Drupal core. They do things like provide Drush commands for working with migrations, add new sources (CSV, JSON, etc.), add new destinations, provide code examples, and fill in other gaps. Think of these as the tools of the trade you’ll use to do your work. In most cases you’ll use these to do the migration, but then once the migration is complete you can remove them from your project.

Then there are the contributed modules that add features to your site. Like Flag, Paragraphs, or Webform. These modules often contain migration-related code that is intended to help make it easier to handle data specific to these modules. There is code in the Flag module, for example, that can help with knowing how to extract flagging records from Drupal 7, and for transforming that data into the format the module expects it to be in for Drupal 10. You might end up having to tweak it a bit, but at least you’re not starting from scratch.

In this tutorial we'll:

  • Look at some of the most commonly used toolset modules.
  • Explain what you can expect to find in standard contributed modules related to migrations.

By the end of this tutorial you should have a better sense of the various tools available to you for authoring a migration.

Update or migrate?

The Drupal core migration system is used for both importing data from custom sources, and for migrating content and configuration from previous versions of Drupal into Drupal 9 or 10 -- effectively updating your site from one major version to another. Depending on which of these two tasks you're performing you'll use the tools in different ways.

Updating from Drupal 7 or earlier

If you're migrating from old versions of Drupal, start with our Prepare for a Drupal-to-Drupal Migration tutorial. From there you'll learn how to perform a basic update using either the built in UI or the contributed Drush commands, as well as some common best practices to help things go as smoothly as possible. If you're looking to customize your Drupal-to-Drupal migration a bit, continue through the tutorials on writing custom migrations, and then we'll go in-depth on how to use those skills to tailor the Drupal-to-Drupal update process to your specific use-case.

Drupal-to-Drupal Migrations (Drupal 7 Upgrade)

More information

Before you begin a Drupal 6 or 7 (source) to Drupal 9 or 10 (destination) migration there are a number of things you should consider. Taking the time to plan your migration will help to ensure that you're successful. In this tutorial we'll take a high-level look at:

  • Evaluating your existing Drupal 6/7 site for migration feasibility
  • Preparing your source Drupal 6/7 site for a migration
  • Preparing the destination Drupal site you're migrating to

By the end of this tutorial you should be ready to start assessing the feasibility of performing a successful migration, and begin making a migration plan.

More information

Because current versions of Drupal are so much different from older versions like Drupal 7 upgrading to the latest version requires creating a new site using the latest version of Drupal and then migrating your old site's content and configuration into it.

There is no one right way to tackle a Drupal-to-Drupal migration. Instead, it's like walking down a path and coming to a fork in the trail and then choosing a direction over and over and over. Since every site is different, every path to a finished migration will be different, too. I know nobody wants to hear it, but every migration is its own unique adventure. Every successful migration will require its own custom code, weird shell scripts, and detailed lists of the exact order of things. But that doesn't mean there's no plan to follow.

A successful migration charts a course through the maze, while leveraging existing tools and experience to help find the shortest route and the right path at each fork--with minimal backtracking due to wrong choices.

In this tutorial we'll:

  • Look at what makes a Drupal-to-Drupal migration (a major version upgrade) so tricky, and how to think about performing one
  • Define 3 high-level approaches to performing a Drupal-to-Drupal migration
  • Get a better idea of what the migration work entails, so you go into it with a proper mental model.

By the end of this tutorial you should be able to explain the different approaches to performing a Drupal-to-Drupal migration in broad strokes, and have a better picture of the work that will be involved.

More information

Before you can migrate your Drupal 7 site to the latest version of Drupal you'll need to be able to build the features that make up the current site. Part of this is evaluating all the modules you've got installed, figuring out what you're using them for, and if there's a version that's compatible with the latest version of Drupal along with a migration path.

I usually make a spreadsheet for this. But any list of the modules you’re currently using that allows you to keep track of how you plan to update them will work. You also want to keep track of where you are in the process of figuring that all out. Because it’s likely you’ll have some modules for which the path is clear, and others where it’s pretty murky and requires more in-depth research to find a path forward. Having a list means you can break that up into tasks, and ensure you’re not missing something. It'll also help you define when your migration is done as well as any final quality assurance (QA) tasks.

In this tutorial we'll:

  • Start a list of the modules that make up our current site.
  • Point to some tools that can help speed up the process of evaluating a module's readiness.
  • Provide a set of questions that you can ask about each module you're using as part of your planning process.

By the end of this tutorial you should have a list of all the modules you're currently using, and some tools you can use to help you figure out how to move forward with each one.

More information

When preparing to migrate from an older version of Drupal (Drupal 7 or previous) to the latest version of Drupal (Drupal 8 or later) you'll need to determine if the contributed modules you use are ready to go. The Upgrade Status module can give you a list of all the modules installed on your site, and information about the availability of a latest Drupal version. It's not perfect, and will still require some manual research for some modules, but it is a great start towards helping you plan for your migration.

In this tutorial we'll:

  • Install the Upgrade Status module and use it to generate a status report
  • Review the generated report
  • Talk about using the Contrib Tracker project, and other methods for getting more details about the status of upgrades for any module

By the end of this tutorial you should be able to evaluate all the installed modules on your existing Drupal site and determine the status of a module release that is compatible with the latest version of Drupal.

More information

It's not uncommon to patch, or customize, contributed modules during the lifetime of a site. It's important to know if you've done so when planning for a migration since you'll want to be sure that you don't lose any customizations when you start using an updated version of a module.

In this tutorial we will walk through using the Hacked! module to check for any alterations to your Drupal code base.

More information

If you want to be able to say you're done with your Drupal-to-Drupal migration, first you have to be able to define "done". And part of that is doing a content analysis and inventory. Performing a content analysis and inventory will help you ensure that you don't miss any fields or important records. It also gives you an opportunity to spend some time thinking about the overall information architecture of your site. You're already going to be doing a lot of work to migrate your content, so deciding to shuffle things a bit now might not add any significant extra time. Additionally, the latest Drupal (as of Drupal 10) is a different platform than either Drupal 6 or 7, and as such, there are some new best practices and new ways of doing things that might not have been available before.

In this tutorial we'll:

  • Provide a set of questions you can ask yourself about the content of your current site to kick-start your analysis.
  • Give an example of how we create a content type inventory in a spreadsheet and use that to help define "done" for our projects.

By the end of this tutorial you should be able to get started analyzing the content and content types of your existing site in order to start planning for your Drupal-to-Drupal migration.

More information

The Drupal-to-Drupal migration system is still a work in progress. As such, there are a few things that simply don't work, and a few others that still have kinks to be ironed out. In this tutorial, we'll look at some of the common hang-ups that we've encountered, the status of resolving those issues, and what your options are in the meantime.

More information

The Migrate Drupal UI module allows you to execute a migration from older versions of Drupal to the latest version of Drupal. In this tutorial we'll:

  • Run a full Drupal 6 to latest-Drupal (Drupal 8 or 9) Migration from the UI
  • Explore the user interface as it exists
  • Understand how we can deal with the output from our migration
More information

In this tutorial we will run a site migration using Drush, and understand how to deal with any failures that occur.

More information

Drupal core provides support for migrating data from Drupal 6 or 7. The templates provided in core migrate your data in a very specific way. They attempt to copy things verbatim whenever possible. However, you may wish to alter this for your migrations. In this tutorial we are going to explore the various ways that you can alter the existing Drupal-to-Drupal migrations to meet your needs.

More information

Why use Migrate Upgrade and Migrate Plus? The short answer is: because it'll save you from having to type out a bunch of YAML by scaffolding a migration for you.

The contributed Migrate Plus and Migrate Upgrade modules are commonly used to aid in the process of performing a Drupal-to-Drupal migration. The combination of the two makes it easier to customize the configuration of individual migrations on a per-project basis by allowing you to edit each individual migrations configuration using the same mechanics that you would to write a custom migration.

It's not the only approach to making these changes, but for many people (including us), it's the most efficient way.

In this tutorial we'll:

  • Get a more in-depth look at the role of the Migrate Plus and Migrate Upgrade modules
  • Discuss the use cases for using them as part of a Drupal-to-Drupal migration

By the end of this tutorial you should be able to explain what both the Migrate Plus and Migrate Upgrade modules are used for.

More information

One method of creating a custom Drupal-to-Drupal migration involves using the Migrate Upgrade module to generate a set of migration configuration entities that you can use as a starting point. Even if you're not going to use the generated output in the end, this is still an informative exercise as it allows you to see examples of various migration paths.

In this tutorial we'll:

  • Use the Migrate Upgrade module's Drush commands to import Drupal core's migration templates
  • Examine the generated configuration entities
  • Use the Migrate Tools module's Drush commands to view a list of the individual migrations that make up our Drupal-to-Drupal migration

By the end of this tutorial, you should be able to use the Migrate Upgrade module to generate the migrations that Drupal core would use to migrate content, list those migrations, and inspect them individually.

More information

If you want to modify the Drupal-to-Drupal migrations created by Migrate Upgrade you'll need to export the Migrate Plus configuration entities, convert the ones you want to customize to standard migration .yml files, and put them into a custom module. Then, you can make edits the YAML definition of the migration, and keep your customizations in Git.

We recommend creating a new module to house the code that makes up your custom Drupal-to-Drupal migration.

In this tutorial we'll:

  • Create a new module
  • Export the migration configuration entities generated by drush migrate-upgrade as YAML files
  • Copy the files that represent the migrations we're interested in into our new module
  • Customize the copied files

By the end of this tutorial you should have a new module that contains the starting migration YAML files for your custom migration.

More information

As part of creating a custom Drupal-to-Drupal migration we want to limit the set of users that are migrated from our source site into our destination Drupal site. In this tutorial we'll:

  • Extend the existing source plugin
  • Alter the query that's used to select users from our source site
  • Update our user migration to use the new source plugin

By the end of this tutorial you should be able to override the core source plugins used when migrating from prior versions of Drupal in order to gain more control over exactly what is migrated.

Categories
Drupal 8, 9, and 10
More information

Every row returned, from every source, during the execution of a migration is passed through hook_migrate_prepare_row() and hook_migrate_MIGRATION_ID_prepare_row(). Using these two hooks we can augment our migration in a variety of different ways.

In this tutorial we'll:

  • Discuss the use case for hook_migrate_prepare_row()
  • Implement hook_migrate_prepare_row() and use it to skip all but a select list of fields during the field migration
  • Implement hook_migrate_MIGRATION_ID_prepare_row() and use it to skip all but a select list of node types

By the end of this tutorial you should have a better understanding of when hook_migrate_prepare_row() might be useful when writing your own migration, as well as how to skip rows in a migration based on conditional logic.

Importing data from external sources

If you're importing data from a non-Drupal datastore, start with Set up Migrate Demo Site and Source Data . We'll walk through the process of connecting the migrate system to an external data source, writing custom migration paths, using custom process plugins to transform data during import, and best practices for executing your custom migrations. We'll primarily look at using an external MySQL database as our data source, but the techniques learned will apply to any data source. We'll also discuss how to extract data from CSV, JSON, and XML sources.

The Drupal migration system is currently a work-in-progress; there are still some kinks, and best practices continue to evolve. As with all of our Drupal material, we're committed to keeping these tutorials up-to-date as work progresses. We'll keep an eye on the changes that are happening both in core and with the main contributed modules and make adjustments as needed.

External Data and Custom Plugins

More information

Before we can learn to write a custom migration, we need some sample data and a destination site for that data.

In this tutorial we'll obtain some source data to work with and configure our Drupal destination site by creating the necessary content types and fields to accommodate the source data. Then we'll look at the data that we'll be importing and start to formulate a migration plan.

By the end of this tutorial you'll have some source data and an empty but configured destination Drupal site ready for data import.

Categories
Drupal 8, 9, and 10
More information

Source plugins extract data from a source and return it as a set of rows representing an individual item to import and some additional information about the properties that make up that row.

Anyone writing a custom migration, or module developers who want to provide a migration path from their Drupal 6 or 7 module, will need to work with source plugins.

In this tutorial we'll talk about the role that source plugins fulfill and how they work. By the end of this tutorial you should be able to determine whether or not you need to write a source plugin for your migration.

Categories
Drupal 8, 9, and 10
More information

This tutorial covers writing a custom source plugin that imports data from a MySQL database into Drupal nodes. After completing this tutorial you should understand how to write your own custom source plugin that can:

  • Extract data from an SQL source
  • Describe the various fields in the source data to the Migrate API for mapping
  • Provide unique IDs for each row of imported data

By the end of this tutorial you should be able write a custom source plugin that uses an SQL data store as well as have a foundation for writing source plugins that extract data from any source that PHP can read.

Categories
Drupal 8, 9, and 10
More information

Process plugins manipulate data during the transform phase of the ETL process while the data is being moved from the source to the destination. Drupal core provides a handful of common process plugins that can be used to perform the majority of data transformation tasks. If you need some functionality beyond what is already provided you can write your own custom process plugins.

In this tutorial we'll:

  • Examine the role that process plugins fullfil
  • Understand the processing pipeline
  • List the existing process plugins in Drupal core and what each one does
  • Better understand when you might need to write your own process plugin

By the end of this tutorial you should be able to explain what process plugins do, and understand how you'll make use of them in your own migration.

Categories
Drupal 8, 9, and 10
More information

This tutorial covers writing a custom process plugin that you can use to manipulate the value of any field during the process (or transform) phase of a migration. Process plugins take an individual field value provided by a source plugin, and perform transformations on that data before passing it along to the load phase.

In this tutorial we'll write a process plugin that can either uppercase an entire string or the first letter of each word in the string depending on configuration.

By the end of this tutorial you should know how to start writing your own process plugins.

Categories
Drupal 8, 9, and 10
More information

Destination plugins handle the load phase of the ETL (Extract, Transform, Load) process and are responsible for saving new data as Drupal content or configuration.

In this tutorial, we'll:

  • Examine the role that destination plugins fulfill
  • Learn about existing destination plugins
  • Better understand when you might need to write your own destination plugin

By the end of this tutorial, you should be able to explain what destination plugins does and understand how you'll make use of them in your own migration.

Categories
Drupal 8, 9, and 10
More information

Migration plugins are the glue that binds a source, a destination, and multiple process plugins together to move data from one place to another. Often referred to as migration templates, or simply migrations, migration plugins are YAML files that explain to Drupal where to get the data, how to process it, and where to save the resulting object.

Source, process, and destination plugins do the heavy lifting in each phase of the ETL process in a custom migration. We need to choose which plugins we want to use for each phase, as well as map fields from our source data to fields at our destination. A migration YAML file glues it all together and gives it a unique name that we can use to run it.

In this tutorial we'll:

  • Determine what information we're going to move, and where we're going to move it to
  • Install Migrate Plus and Migrate Tools which we'll use to run our custom migration
  • Write a custom migration plugin (configuration) YAML file that will work with Migrate Tools

By the end of this tutorial you should be able to write a custom migration YAML file and understand how to choose the source, destination, and process plugins that will do the work.

Categories
Drupal 8, 9, and 10
More information

As of right now, the most reliable way to run custom migrations is using Drush. Depending on the version of Drush you're using you may also need the Migrate Tools module. In this tutorial we'll walk through using Drush to run a custom migration, as well as the other commands that can be used to manage the execution of migrations.

By the end of this tutorial you should know how to run your custom migrations.

More information

When running migrations you can use the highwater_mark source plugin configuration option to influence which rows are considered for import on subsequent migration runs. This allows you to do things like only look at new rows added to a large dataset. Or to reimport records that have changed since the last time the migration was run. The term, highwater mark, comes from water line marks found on structures in areas where water level changes are common. In running migrations, you can think of a highwater mark as a line that denotes how far the migration has progressed, and saying, "from now one, we only care about data created after this line".

Another common use case for highwater marks is when you're importing a large dataset and the system runs out of resources. Usually this will look like a migration failing because it timed out, or the process ran out of memory. A highwater mark should allow you to pickup from where you left off.

In this tutorial we'll:

  • Define what a highwater mark is, and how you can use them to limit the rows considered for importing each time a migration is executed.
  • Demonstrate how highwater marks can be used to reimport source records that have been modified since the previous time the migration was executed.
  • Introduce the track_changes option.

By the end of this tutorial you should be able to define what a highwater_mark is and how to use them to speed up the import of large datasets or force the migration to reimport records when the source data is changed.

You should also be aware of the track_changes feature which is a slower, but more dynamic, method of checking for changes in source data and reimporting records when a change is found.

More information

If you need to write a migration that is capable of being executed multiple times and picking up changes to previously imported data, you can use the track_changes configuration option of most source plugins. This will tell the migration to keep a hash of each source row, and on subsequent runs of the same migration it will compare the previous hash to the current one. If they don't match this means the source data has changed, and Drupal will reimport that record and update the existing destination record with new data.

Using track_changes differs from calling drush migrate:import --update in that using --update will force every record to be re-imported regardless of whether the source data has changed or not.

In this tutorial we'll:

  • Learn how change tracking works to detect changes in your source data
  • Use the track_changes option in a migration

Note that progress of a migration can also be tracked using highwater marks if the source data has something like a last_updated timestamp column. Using highwater marks in this case is likely more efficient than using track_changes. It's a good idea to understand how both features work, and then choose the appropriate one for each migration.

“Drupalize.Me has trained thousands of Drupalistas. Their video lessons help to address our constant need for more Drupal talent. Drupal needs training like this to support its growth.”
Dries Buytaert
Drupal Founder and Project Lead
Photo of Dries Buytaert
“They’re easy to understand. They’re thorough. They’re funny. They’re always entertaining, and it makes it easy to learn parts of Drupal and how you can integrate it into your workflow and learn more and have a good time doing it.”
Roger Carr
Drupalize.Me Member
Photo of Roger Carr
“The mission of the Drupal Association is to foster and support the Drupal software project, the community, and its growth. Drupal education, like that provided by Drupalize.Me, is important to this mission.”
Megan Sanicki
Former Exec. Director, Drupal Assoc.
Photo of Megan Sanicki

FAQs

Am I required to sign a contract?
No. You can purchase a membership and/or cancel any time. Drupalize.Me is a pay-as-you-go service.

Can I preview tutorials before joining?
Yes! Just navigate to our tutorial library. Our free tutorials are labeled with a green "FREE" tag.

Can I watch videos on my mobile device?
Yes! Drupalize.Me is a responsive site and can be accessed in the browser on any mobile device. More FAQs