One of the problems that Drush solves for developers is the automation and optimization of routine tasks. Drush commands attempt to speed up workflows and tasks that developers and site maintainers would otherwise have to do manually through the UI, or run one-by-one via the command line. One of those tasks is the process of deploying changes to a Drupal application from one environment to another.
The typical Drupal deployment process consists of repeatable steps such as importing configuration changes, applying database updates, and clearing the cache. Drush comes with the handy drush deploy command that allows you to automate the execution of all of these tasks post code deployment.
In this tutorial we'll:
- Learn about the
drush deploycommand - Discuss when you would use the
deploycommand
By the end of this tutorial you'll know how to use the drush deploy command in conjunction with other useful deployment-related commands to help automate the task of deploying changes to a Drupal site's configuration and code.
Code generators are great productivity boosters that allow generating scaffolds for common development tasks in Drupal. One of the most common use cases for generators is scaffolding the code required for a custom entity type. Custom entities require many files and complicated annotations in order to function properly. There is a lot of boilerplate code that is more-or-less the same for every entity type. Creating all the files is repetitive, time-consuming, and prone to human error. Generators can help automate this task and make creating your own custom entity types quicker.
In this tutorial we'll:
- Learn how to generate the code for a custom entity with Drush
- Learn about the options that generators provide for custom entities
By the end of this tutorial you should know how to generate custom entities with Drush.
Some Drush commands return a lot of information -- lists of modules, generators, and status reports, for example. It might be hard to find a property you need in the small command window output. Luckily, the output of Drush commands can be piped to other commands, used as a source for imports, settings for CI, and other DevOps tasks.
In order to accommodate all these different use cases, Drush comes with a formatting system that allows you to format and filter output to meet your needs. This system allows to you specify what fields you want returned when the output contains more than one field. It also allows Drush command output to be formatted as JSON, XML, raw PHP, a table, and more.
In this tutorial we'll:
- Learn how to specify the output format -- and what formats are available
- Limit the fields that are returned in a report
- Learn to filter the output to only the data we're interested in
By the end of this tutorial you'll know how to format a Drush command's output to fit your needs.
If you want to change the way an existing Drush command works, you use hooks. Hooks are useful for altering command parameters, options, attributes data, and adding custom logic during particular stages of the command execution process. Drush hooks are conceptually similar to Drupal hooks.
Hooks are methods on a Drush command class with an attribute indicating what hook is being implemented, and thus when the code should be invoked. The code in the methods is executed during specific stages of the command cycle. Developers can use core hooks -- predefined methods that come with Drush core -- or declare a custom hook that other commands can use.
In this tutorial we'll:
- Explore the different core Drush hooks
- Learn which hooks are called at what stage of the command cycle
- View example hook attributes and method implementations
By the end of this tutorial you'll know what types of core hooks are available to you and when they are called.
Adding logging and error handling are an important part of authoring Drush commands. Logs allow developers to get timely feedback from a command and inform users about potential alterations and flags, events to monitor, and the progress of long-running commands. Correct error handling allows for clean exits, meaningful error descriptions, and provides a path forward for developers to fix the errors and accomplish their goals.
In this tutorial we'll:
- Explore the different types of logging messages Drush commands can output
- Learn how to handle errors from within a Drush command
By the end of this tutorial you'll know what types of log messages you can use; how to log success messages, errors, or debug statements from a custom Drush command; and how to handle errors and exceptions.
When the logic of a command depends on user input, it's useful to set up an interactive questionnaire inside the command code. This allows you to provide the user with more context about the input they're providing, and ensure that you collect all the necessary values. This is especially useful when the command uses a pre-defined list of options and the values require memorization. An example of this is the drush cache-clear command that comes with Drush core. It requires an argument indicating which cache to clear, which you can specify at the command line; however, if you invoke the command with no arguments it will present you with a list of cache bins to choose from and a UI for selecting one.
Drush 9+ can access the Input/Output (I/O) object via the $this->io() method. This object -- an instance of \Drush\Style\DrushStyle -- holds information about user-provided input, and utilities for manipulating that input. To ask a user a question, use an io() object in the command callback method. It can take over the execution flow of the command as needed to stop and gather additional input. The I/O system has various methods for asking confirmation or choice questions such as confirm() and select().
In addition to prompting for input, the I/O object can be used to provide other styling to the command, like progress bars.
In this tutorial we'll:
- Learn how to prompt the user for additional input
- Process the user's answer as part of the command execution flow
By the end of this tutorial you should understand how to prompt a user for additional input for a custom Drush command.
While Drush empowers all Drupal users with its commands, it's even more powerful when used in combination with scripting solutions such as Composer and Bash. Scripts can be used to power post-deployment tasks like importing new configuration or clearing the cache, as part of CI processes to sync a database from one environment to another, to run background processes on the server such as imports and migrations, search indexing, running cron, and much more. If you want to write Bash (or any other scripts) that interact with a Drupal site, then Drush is the tool for you.
In this tutorial we'll:
- Learn how to use Drush commands within Composer and Bash scripts
- Learn how to chain multiple Drush commands together in a script
By the end of this tutorial you'll know how to use Drush as part of a script that automates common or tedious tasks.
When you manage many Drupal websites, you may perform repetitive tasks that are common across all of your sites. In our experience, this usually relates to having a personal preference for how certain tasks are accomplished. For example, maybe you like to make backups of the database and files in a specific way before testing upgrades, or you have a set of scripts for running scans of core web vitals. Although these tasks can be bundled into a custom module, it could be useful to create a site-wide Drush command instead. Site-wide commands can be installed with Composer, managed in a separate Git repository, and act as a project dependency. This way they are easy to maintain through a separate upstream. Changes to this code will be reflected on all the sites where it's used.
In this tutorial we'll:
- Declare a custom site-wide Drush command
- Demonstrate how to use Composer to manage a package that contains a Drush command
By the end of this tutorial you'll be able to create a site-wide Drush command and manage the code with Git and Composer.
The Migrate module itself contains some excellent examples of data migrations implementing the various APIs provided by the module and serves as the canonical documentation for how to write a migration. In this lesson we'll take a look at the beer and wine import examples provided with the migrate module as a way to familiarize ourselves with the concepts discussed earlier and to be able to see the code that makes up a basic migration before attempting to write our own. In practice these examples serve as a great starting point and can often times be copy/pasted and adjusted for your own needs.
Additional resources
Before we can write our own custom migration we need to construct the site that we're going to import data into and of course we need some source data to import. In this lesson we'll obtain some source data to work with and configure our Drupal site by installing a couple contributed modules, and creating the content types and fields necessary for our information architecture. During this process we'll also be taking a look at the baseball player and team data that we'll be importing and familiarizing ourselves a bit more with the tables and columns in our source database.
Note: The Lahman database structure has changed since this video was recorded, and the latest files provided by Lahman don't match what is used in the video. When following along with these examples you'll either need to use the 2012 source data, or make a few adjustments to field/table names throughout the source code as you follow along so that they match the current structure of the Lahman data.
Additional resources
Running your own migration requires a bit of setup and boilerplate code, Drupal needs to know where to find the source data, and the Migrate module needs to be provided with some basic information about our custom code. In this lesson we'll look at setting up Drupal to be able to connect to multiple databases, and then create the skeleton of a simple module which will house our custom migration code and an implementation of hook_migrate_api(). Finally we'll create a base class from which we can begin writing our own custom migrations, and talk about why this is a good way to start organizing our migration.
In this lesson we're going to write our first custom data migration and start importing some of the player data in to Drupal. The primary concepts covered in this lesson are the creation of a migration class following the pattern necessary for Migrate module to be able to discover our code and then dealing with defining source and destination objects so that the Migrate module will know where to read data from and where to write data to. Finally we'll add a simple single field mapping where we map the player's name to the node.title field allowing us to run the migration for the first time and import some real data.
Additional resources
In this lesson we'll continue to add field mappings to the basic migration class created in the previous lesson. We'll see how to add more information about the available source fields. We'll map more of the player source data fields to their equivalent destination fields and learn about some of the many ways that fields can be mapped. Finally we'll also cover mapping sub-fields which allows us to import data for things like the text format of a node's body field or the alt column in an image field. Information that's contained within a meta field.
Additional resources
In this lesson we're going to finish mapping the fields for the player migration and learn about how to deal with source data that requires some additional massaging before being saved to the destination. We'll learn about the use of field mapping callback functions and the migration's prepareRow method as possible spots to perform additional logic during a migration. Then we'll use these techniques to combine our player first and last name fields together for the node title field, deal with our birth and death date fields by concatenating the three source columns together into a single date string, and finally add some additional information to the notes field during import that will allow us to track imported records in the future.
Note: This lesson was recorded using the 7.x-2.6 version of the date module, however the 7.x-2.7 version is now out which contains some changes to the module's integration with the migrate module. The biggest change being that the date_migrate module is no longer required and has been deprecated. You can read more about the changes here: https://drupal.org/node/2034231
Additional resources
Although not required when writing your own migration, the Migrate module provides ways for us to decorate our migrations with additional information making it easier to keep track of who is working on what, what needs to get done, and related issues. We've already seen some of this in previous lessons with with the Do Not Migrate option for fields and ability to provide field mapping descriptions. In this lesson we'll take a look at how we can make use of the additional tools to do things like; Show the name, contact info, and roles of individuals working on the migration, pose questions to other team members via the migration UI, and link individual field mappings to related tickets in our bug tracking software. Making it easier for team members who don't want to be involved with the code to help move our migration along.
Additional resources
In this lesson we're going to take a look at creating relationships between two Drupal nodes during a migration. In our case we've got player and team nodes, and each player node has an entity reference to a team node which we need to populate during our migration. In order for this to work we need to ensure that the team node has already been created so that we know the unique node.nid to use in the entity reference field for the player.
To accomplish this we're going to write a migration for team data and ensure that it is run prior to our player migration being run. Then we're going to make use of the mapping between source and destination rows that the Migrate module is tracking for teams so that during a player migration we can lookup the corresponding team node's nid and make use of it.
This lesson is a short one but it covers an important topic, multi-value fields. Almost any field in Drupal can be configured to support more than one value being entered for a single field. Our teams entityreference field is a good example of this, a player could have played for one or more team over the course of their career. This lesson will look at two different ways to map multiple values for a single field.
First we'll look at doing it in a callback method where we perform an additional query and then use the values returned by that query. And second we'll look at using the field mapping's separator method to take a column in a source row that has multiple values separated by a comma and import them as individual field values.
The infamous causality dilemma of the chicken and the egg examines which of the two came first constantly battling with the fact that you need one to produce the other. It's a vicious circle. In this lesson we're going to explore this dilemma in the context of data migrations. Imagine a scenario where you've got an article node type that has a reference field for similar articles which you need to populate with the node ID of the similar articles. During a migration when the article is being imported the article that is being referenced may or may not exist already. If it doesn't exist already how do we know what ID we need to put into the reference field?
One option would be to solve this problem using multiple passes. A first pass that goes through and creates all the articles, and a second that comes back through and updates the similar articles field. Though what happens if the similar articles field is required? You wouldn't be able to save the article without a value in that field the first time around? So you see how this quickly becomes another example of the chicken or the egg problem?
Lucky for us the migrate module has a solution to this called stub migrations. A process that allows creating a stub or a shell for the referenced but not yet created article so that we can use it's unique ID, then when that article is encountered in the migration it will update the stub rather than create a new article.
Additional resources
So far all the data we've been migrating has come from a MySQL database but Migrate supports a number of other data sources which we can access by using a different source migration class. In this lesson we'll look at the source migration classes that are available with the migrate module and talk about what each one could be used for. Then we'll implement a migration that imports data from a CSV file since that's another common way of receiving source data.
For your convenience, we've included a copy of the sample data in the companion files. This is a duplicate of the data from lesson Set Up Migrate Demo Site and Source Data. If you've been following the lessons in order, you should not need to download the sample data set again.
Additional resources
Migrate Source Class documentation
Up to this point we've focused focused on creating nodes as the result of our migrations. The Migrate module however supports a number of different destinations that we can use when importing data. In this lesson we'll take a look at the destination classes that the Migrate module provides for us and talk about what each one is used for and where to find more information and examples of using them. Then we'll implement a migration that imports data as vocabulary terms using the MigrateDestinationTerm class.