Automated Testing in Drupal 7 with SimpleTest

Testing, and quality assurance, are an important part of maintaining high-quality software. Together they ensure that the code we write functions as it is intended to, and that changes made later on do not introduce any regressions into our application. Its tedious work and generally involves coming up with a list of tasks that prove a specific feature is working, and then repeating the list of tasks every time a change is made.

Automated testing is the process of writing code to reduce the number of things we need to do manually when testing our software. That list you created to ensure your software is working correctly is likely something that can be easily automated with a testing framework like SimpleTest. Doing so can dramatically reduce the amount of time it takes to perform QA, and improve the overall stability of our applications.

The SimpleTest testing framework was introduced into Drupal core early on in the Drupal 7 development cycle. Since then it has had a profound impact on the way our community develops Drupal. Core now contains a whole suite of tests that cover somewhere in the neighborhood of 75% of the softwares functionality. Every time a new features is introduced, or a bug is fixed, this battery of tests confirms that the fix to the date formatting function didn't inadvertently break something else.

In this series we'll learn to write our own automated tests for Drupal 7 using SimpleTests. We'll walk through enabling SimpleTest, and running one or more test cases with both the SimpleTest UI, and with drush. Then we'll cover some of the related terminology like the difference between functional tests and unit tests. As well as discuss why testing is an important part of development and worth the investment.

Then we'll walk through writing a test case and a set of assertions in orders to verify that various features of our site are working. We'll use functional tests based on the DrupalWebTestCase class, which simulates a browser navigating our site and clicking on links, and filling out forms. And the DrupalUnitTestCase class, which allows us to write super fast unit tests for the business logic contained within our code.

After watching this series you should be able to:

• Install the SimpleTest module and run the tests included with core.
• Define and know the difference between functional tests, unit tests, test cases, and assertions.
• Create a new test case so that SimpleTest can discover and run your tests.
• Write assertions that test the validity of your application in a given state.
• Use the test browser to click on links and navigate to pages on a site.
• Fill out and submit forms and AJAX form elements using the test case browser.
• Handle file and image uploads in test cases.
• Write unit tests, and understand how the workflow for running unit tests differs from that of functional tests.
• Use the SimpleTest 7.x-2.x module from contrib to test existing websites.

This series assumes that you have basic understanding of Object Oriented PHP, and are familiar with Drupal module development. If you need to brush up on your module development watch our series on Drupal 7 Module Development. And, while not required to run tests, we do make use of drush in this series so knowing how to run drush commands is helpful.

Writing tests can be a challenging. But it doesn't have to be. And the benefits of doing so are huge. Especially as our web applications become more complex, as our teams grow, and as the multitude of ways in which people interact with our web applications changes constantly. Spending some time learning how to write tests will not only make your applications more reliable, it'll also make you a better developer.

Additional resources

• Drupal 7 Module Development series
• Introduction to Drush

Developers write tests for a variety of different reasons and understanding why, and how, writing tests can improve the quality of our code and our sanity helps motivate ourselves to write tests. The tests we write usually take one of two forms. Functional tests are used to test an applications behavior. When I click this link does it take me to the about page? Unit tests are used to verify the logic within a small segment, or unit, of code. When I call this function that is supposed to return a link with these specific parameters does it return the correct data? There are a ton of reasons that investing time and resources into writing automated tests is a big win in the long run and in this lesson we'll cover a bunch of them, including:

• Eliminate repetitive tasks.
• Improve overall stability of an application.
• Ensure critical paths & edge cases function.
• Reduce (not eliminate) the number of tests that need to be performed manually.

Types of Tests

There are are two main types of tests and we're going to be writing. Functional tests, and Unit tests.

Functional tests, also known as integration tests, test the functionality of a particular aspect of an application by simulating normal interaction with that system. In the case of web software this usually refers to simulating a user navigating and interacting with the website as if they where using their browser, keyboard, and mouse.

Unit testing is the art and practice of taking a small portion of code, a unit, and subjecting it to programmatic tests to prove its correctness. For example, testing a function that performs unit conversion by comparing a known input with an anticipated result.

UPDATE: This video references qa.drupal.org which is no longer in use. On Friday, October 23rd 2015 - qa.drupal.org received and processed it's last
test. The qa.drupal.org testbots have now been superseded by DrupalCI - and test results are now displayed directly on Drupal.org. While the mechanics are different now, the end result is still all patches to Drupal core are automatically tested.

SimpleTest is the tool that Drupal 7 core uses for discovering, and running all of it's tests. SimpleTest is also the name of the system that is used to write tests that can be run by the SimpleTest module. Understanding how SimpleTest locates the code that makes up our tests, and then executes that code, is an important part of understanding how to write tests. In this lesson we'll be discussing the pieces that make up the SimpleTest toolkit, some related terminology, and some best practices for dealing with SimpleTests in Drupal.

Methodology

Some best practices to keep in mind when working with SimpleTest and writing tests.

• Tests should always be run in a separate environment from your production site.
• Each test case sets up a completely new Drupal env, as such your test case needs to prepare the new environment as appropriate for your tests. Things like enabling any necessary modules, creating dummy content, and users etc. For this reason, it's best to have tests that require a similar setup be part of the same test case for better performance.
• After a test case is run the environment is torn down and started fresh for the next test case. This ensures there is no contamination between test cases and that each test can count on knowing exactly what to expect from the environment it's running in.
• SimpleTest is not always ideal for doing functional testing of your existing site. Drupalize.Me for example, we want to test that our About page is visible. But that's content, and doesn't live in code anywhere so creating it at install time can be tricky: We'll cover this scenario in more depth later.
• Always test both sides of the problem. If your site only allows authenticated users to make comments, and you write a test to make sure authenticated users can comment you'll also want to make sure you write a test that verifies that anon. users can not comment. If you're checking that a value exists under specific conditions, also verify that it doesn't exist when those conditions are not met

Terminology

SimpleTest: The original name of the module used for testing in Drupal. Though it's now called just "Testing", the terms are used interchangeably. Based on the PHP SimpleTest library.

Test Case: A set of conditions under which the tester will determine whether the system being tested satisfies requirements and works correctly. A Test case defines the environment in which a test or set of tests will run. Including preconditions such as what modules are enabled, what content exists, etc. It is assumed that these same preconditions are true for all individual tests within a test case.

In Drupal, all test cases are classes that extend the DrupalWebTestCase, or the DrupalUnitTestCase class.

Test (Test method): Each individual test checks the functionality of one discreet thing. Example, if our test case is "User cancellation" there are many variations of cancellation that we need to test. Cancel the account and keep their content, cancel the account and delete their content, don't allow cancellation of the account with UID 1, etc. Each of these individual tests is checking one piece of functionality with the test case.

Assertion: The smallest unit of a test. An assertion is used to check wether or not a given condition is true in the current context.

Additional resources

Before we get into writing our own tests we need to enable the SimpleTest module, and while we're at it we may as well talk about the various configuration options it has. And then why not run a few of the tests that come with Drupal core as a way to learn how to discover and run all the tests, a group of tests, or even an individual test case.

The SimpleTest module comes with Drupal 7 core, but is not enabled by default. So we can start by enabling the module. On the administration page for enabling modules the module is listed as "Testing" instead of SimpleTest which can be a bit confusing. The the UI you'll almost always see the name "Testing", but in the code, documentation, and at the command line it's generally referred to as SimpleTest.

Once the module is enabled there are just a few new permissions and configuration options to cover. The new 'run tests' permission, and a couple of settings related to enabling verbose output for debugging of tests. Make sure you check them all out.

Tests can be run in a variety of different ways, either from your browser using the Testing module's UI, or at the command line with drush. We'll cover both of them in this lesson. And in doing so we'll look at how to run a group of tests, or an individual test case, and the output that's generated by running tests.

I've also got a module (which you can download from this page) that contains a couple of failing tests. This way we can run a test case that fails, and see what the output looks like from a failure. SimpleTest outputs some helpful debugging information, and with the verbose option enabled will even save the generated HTML of each page the SimpleTest browser saw so that we can review it after the test run is complete.

If you're not familiar with drush checkout our series on using drush.

By the end of this lesson you should be able to enable and configure the simple test module and run some tests.

Example drush commands

Run all tests in the "Block" group:

drush -l http://localhost/demos/simpletest-7x/docroot test-run Block

Run just the "BlockHashTestCase" test case:

drush -l http://localhost/demos/simpletest-7x/docroot test-run BlockHashTestCase

Note: as of Drush 7 the drush test-run command is no longer part available. See https://github.com/drush-ops/drush/issues/599.

You can either, use Drush 6, or use the scripts/run-tests.sh scrip that comes with Drupal core. See the documentation for the run-tests.sh script for more information.

In this lesson we'll write a hello world test that navigates to the front page of a standard Drupal installation and verifies the existence of the text, "No front page content has been created yet.". This will allow us to walk through creating a .test file, adding it to our modules .info file, as well as cover the basics of extending DrupalWebTestCase to write our own test case. For this lesson we'll use the helloworld sample module as a starting point and expand it to add a very basic test. The goal being to gain a better understanding of how the SimpleTest module discovers, and runs our tests. Then in future lessons we can expand on our test case to add more robust features.

All SimpleTest tests are an extension of either the DrupalWebTestCase (we'll focus on this one), or the DrupalUnitTestCase base classes. There are two pieces to the discover process. Writing a new class that extends one of the base classes, and then declaring the file that contains this new class in your module's .info file so that the file and it's code are indexed and inserted into the registry.

Create a new file inside the helloworld module's directory: tests/helloworld.test. The convention is to name files that contain test classes with a .test extension. Similar to how module file, despite being purely PHP, are named with a .module extension.

Then we'll add a basic skeleton test class like so:

class HelloworldTests extends DrupalWebTestCase {
  /**
   * Metadata about our test case.
   */
  public static function getInfo() {
    return array(
      'name' => 'Hello World',
      'description' => 'Tests for the Hello World module.',
      'group' => 'Hello World Group',
    );
  }
}

Our class includes a getInfo() method, it's required, and it returns a simple associative array with meta-data about our test. This information is used by the SimpleTest UI and other test runners to provide human readable information about our tests and to do things like group like tests together. When you view the lists of tests in the SimpleTest UI, you're seeing the 'name', and 'description' provided by your test classes' getInfo() method.

All tests should have a setUp() method as well. This method is called by the test runner after Drupal has been installed, but before our tests are run. It gives us the chance to perform any additional configuration on the environment within which the tests are running. In our case, we need to enable the helloworld module since it's not part of the default install profile. We can do that by delegating to the parent classes' setUp() method and passing an array of modules we want enabled. That might look something like this:

/**
 * Perform any setup tasks for our test case.
 */
public function setUp() {
  parent::setUp(array('helloworld'));
}

Finally, we need to provide at least one method whose name starts with "test". This method, and all others whose name starts with test will be called in succession by the test runner and are expected to contain the assertions that make up our individual test.

public function testHelloWorld() {
  $this->drupalGet('helloworld');
  $this->assertText('Hello World. Welcome to Drupal.', 'The page content is present.');
}

This test will navigate to the url /helloworld, and then verify that the text "Hello World. Welcome to Drupal.", is displayed on the page.

In order for our test to show up in the SimpleTest UI we need to add it to the registry. We can do so by modifying the helloworld.info file and adding the line files[] = tests/helloworld.test to the file. Then clear the cache and Drupal should discover your new test case and allow you to run the tests either through the SimpleTest UI or with drush.

Additional resources

DrupalWebTestCase documentation
Written tutorial based on this video

Assertions are like the answer sheet a teacher uses when grading a multiple choice test. When we write tests our primary objective is to check, or assert, that the state of some system, or the value of some variable, in the specified context matches our expectations. Like saying, "I expect that when I break this cookie in half I'll see chocolate pieces.". If you break the cookie and they are not there you might then decide that this particular cookie does not pass the chocolate chip cookie test. And flag it for review.

There are a lot of different types of assertions that we can make when using SimpleTest and this lesson attempts to demystify what each of the different types of assertions does. There is some really good documentation about the various types of assertions and some examples of using them in the Assertions documentation page on Drupal.org.

Not all of the assertions in SimpleTest are listed there though. For that, your best bet is to look at the API documentation for DrupalWebTestCase, and use the filter field at the top of the table to limit the list to only methods that start with the keyword "assert". This will give you a complete, and up-to-date list of all the assertions.

Basic assertion types reference:

Checking the value of something

• assertTrue
• assertFalse
• assertNull
• assertNotNull
• assertEqual
• assertNotEqual
• assertIdentical
• assertNotIdentical

Checking for the presence of something

• assertPattern
• assertNoPattern
• assertRaw
• assertNoRaw
• assertText
• assertNoText
• assertTitle
• assertNoTitle
• assertUniqueText
• assertNoUniqueText
• assertLink
• assertNoLink
• assertResponse

Checking form elements

• assertFieldById
• assertNoFieldById
• assertFieldByName
• assertNoFieldByName
• assertFieldChecked
• assertNoFieldChecked
• assertOptionSelected
• assertNoOptionSelected

Additional resources

Drupal.org Assertion Examples / Documentation

If we're going to do functional testing of the various pages in our application we'll need to be able to navigate to the pages in question. The DrupalWebTestCase's built in browser provides us with two different ways to navigate our site. In this lesson we'll explore using the DrupalWebTestCase'::drupalGet(), and DrupalWebTestCase::clickLink() functions in order to navigate throughout our application. The former is useful when we know the exact URL of the page we want to navigate to and the latter is useful when we don't necessarily know the URL but we know which link to follow to get there.

I like to think of DrupalWebTestCase::drupalGet() as typing an address into the URL bar in my browser. I know the page I want to navigate to so take me right to it. DrupalWebTestCase::drupalGet() can be used inside a test case like so:

public function testHelloWorld() {
  $this->drupalGet('helloworld');
}

This points the SimpleTest internal browser to the helloworld page on our site, and loads the HTML content of that page into the buffer so we can perform assertions on its content.

DrupalWebTestCase::clickLink() on the other hand is more like looking at the content of the page, reading through the links in the navigation menu and then deciding that you would like to click on the one labeled "Hello World". You don't know what address it's going to take you to, but that's okay because what matters is that you end up on the page. DrupalWebTestCase::clickLink() operates by scanning the SimpleTest browsers buffer of the current page's HTML content for any link whose label matches the value provided.

public function testHelloWorld() {
  $this->clickLink('Hello World');
}

This would have the effect of clicking on the following link:

<a href="helloworld">Hello World</a>

Or really, just extracting the value of the href attribute from that a tag and feeding it to DrupalWebTestCase::drupalGet().

I find the best reference for these helper methods is located in the Drupal.org handbook. https://www.drupal.org/node/265762

Additional resources

SimpleTest helper method reference

DrupalWebTestCase API documenation

Much of the functionality that web based applications provide is in the form of an HTML form. Enter some data into the various fields, press the submit button, and see what happens. The DrupalWebTestCase provides a handful of methods to aid in interacting with forms. In this lesson we'll take a look at filling out various types of form fields, submitting forms, and validating the result.

We'll start by navigating to a page that contains a form and simply verifying that the appropriate fields are found on the page.

$this->drupalGet('helloworld/form');
$this->assertFieldByXpath("//form[@id='helloworld-cake-form']//input[@name='name']", '', 'The name field is present.');
$this->assertFieldByName('choice', 'cake', 'The choice field is present.');

This code checks for the input name="name" field, and the select name="choice" field, and ensures they are present on the page at hellworld/form.

Then we'll use the DrupalWebTestCase::drupalPost() method to fill in, and submit, a form. Here's an example:

$data = array(
  'name' => '',
  'choice' => 'chicken',
);
$this->drupalPost('helloworld/form', $data, 'Submit');
$this->assertText('Your name field is required.', 'Name field is required.');

This code fills in the form on the hellworld/form page, and then clicks the submit button using the DrupalWebTestCase::drupalPost() method. The first argument is the URL at which the form lives, the location to which the HTTP POST request will be sent. The second argument is an associative array of values for the form fields. The keys are the names of the input elements, and the values are the information we would like to enter into those fields.

In this lesson we also make use of the $this->assertFieldByXPath() method, which allows us to select a form element on the page using an XPath selector. In our case, we need to do this because there are actually two input element on the page with their name attribute set to "name", and we need to make sure we're testing the correct one. If you're not familiar with XPath it's worth taking some time to review the syntax since chances are you're going to end up using it at some point when working with SimpleTest.

For more information about these helper methods checkout the API Functions handbook page.

Additional resources

SimpleTest helper methods

XPath documentation

Typically our application is going to require testing of functionality that is restricted to only users with a certain set of permissions. Like creating a new node for example. In order for the DrupalWebTestCase browser to test these pages we need to first log in to our site. In order to log in we need to have a username and password that we can use to login. In this lesson we'll look at using DrupalWebTestCase::drupalCreateUser() and DrupalWebTestCase::drupalLogin() in order to gain authenticated access to our application and then access and submit a restricted form.

Here's an example of creating a user with the 'administer site configuration` permissions:

$new_user = $this->drupalCreateUser(array('administer site configuration'));

And then logging in as that user:

$new_user = $this->drupalLogin($new_user);

After that the SimpleTest browser will be logged in as that user and all requests that it makes will be authenticated. If you want to return to an anonymous user session you can use the DrupalWebTestCase::drupalLogout() method.

For more information about these helper methods check out the API Functions documentation in the Drupal.org handbook.

Additional resources

SimpleTest helper method documentation

DrupalWebTestCase::drupalLogin()

DrupalWebTestCase::drupalCreateUser()

There's another set of really useful helper methods that we haven't looked at yet which are geared towards creating dummy content for testing. Including simple tasks like generating random strings, or random names of a specified length, and more complex things like creating nodes, or even new content types. In this lesson we'll take a lot at the various helper methods in DrupalWebTestCase that assist in the creation of dummy content by creating various pieces of random content and then testing functionality based on that content.

In order to demonstrate these tools lets look at writing some tests for the core statistics module which amongst other things tracks page views for content. In order to perform these tests we're going to need a piece of content that we can view, and then view again to ensure that the statistics are properly updating.

Creating random strings can be done with DrupalWebTestCase::randomName() and DrupalWebTestCase::randomString(). Both methods take an argument that allows you to set the number of characters that should be returned, or you can just leave out to get the default. The difference between the two is that DrupalWebTestCase::randomName() will only use alphanumeric characters and the returned string will always start with a letter. This is useful for things like generating random usernames or email addresses, or other things that don't allow for special characters.

The DrupalWebTestCase::drupalCreateNode() helper can be used to create nodes. While you could navigate to and fill out the form using SimpleTest this is much quicker and requires far less code. Here's an example of creating a node:

$title = $this->randomName();
$settings = array(
 'type' => 'page',
 'promote' => 1,
 'title' => $title,
 'body' => array(LANGUAGE_NONE => array(
   array(
     'value' => 'Copy goes here',
     'format' => filter_default_format(),
   ),
 )),
);
$node = $this->drupalCreateNode($settings);

This will create a new page node with a random title and the string "Copy goes here" in the body field. The array of values passed to DrupalWebTestCase::drupalCreateNode() should mirror the structure of a node object. If you're unsure what that structure is one easy way to figure it out is to use node_load() to retrieve an existing node and inspect the returned object.

Additional resources

One element of interacting with a web application that we haven't looked at yet is uploading files. How does the DrupalWebTestCase browser attach a file to the the image field when filling out an article form? In this lesson we'll look at the helpers that DrupalWebTestCase has for handling file interactions including how to use the sample files provided in core, attaching files to a file upload input element, and submitting a form with files attached. In order to demonstrate this we'll be writing a test that creates a new article node with an image attached by filling out the article node creation form and submitting it.

You can get a list of the dummy/test files that come with SimpleTest using the DrupalWebTestCase::drupalGetTestFiles() method. Whenever possible I recommend making use of the provided files.

$files = $this->drupalGetTestFiles('image');
$this->verbose(print_r($files, TRUE));

You'll need to tell SimpleTest what type of files you're interested in, and it'll then return an array of the available files of that type. Valid types are: 'binary', 'html', 'image', 'javascript', 'php', 'sql', 'text'. Check out the DrupalWebTestCase::drupalGetTestFiles documentation for more information.

Once you've got the list of files you can attach files to a form element using DrupalWebTestCase::drupalPost() by specifying the full path to the file as the value for the file or image field in the $data array.

$images = $this->drupalGetTestFiles('image');
$image_realpath = drupal_realpath($images[0]->uri);

$edit = array(
  'title' => 'Test Article With Image',
  'files[field_image_und_0]' => $image_realpath,
);

If SimpleTest doesn't contain a suitable file you can use your own by including the file with your module, and then using the full path to that file in your tests.

$custom_file = drupal_get_path('module', 'helloworld') . '/tests/test.csv';
$realpath = drupal_realpath($custom_file);

This lesson also covers the use of DrupalWebTestCase::drupalPostAJAX(), which allows for the simulation of an AJAX request using the SimpleTest browser. The primary difference between it and the drupalPost method is you need to specify the name of the element that triggers the AJAX request.

$this->drupalPostAJAX('node/add/article', $edit, 'field_image_und_0_upload_button');

Additional resources

Documentation about SimpleTest file uploads

DrupalWebTestCase::drupalGetTestFiles()

Due to the tightly coupled nature of much of Drupal 7's code most of the test suite is based around functional tests and not unit tests. However, there are a handful of unit tests in core as well as the DrupalUnitTestCase class that we can extend to write our own unit tests. Since unit tests are an important part of a complete test suite lets take a look at how we can extend the DrupalUnitTestCase class and add some unit tests to our module. While functional tests do a good job of telling us that something is broken and we should fix it they often are not very helpful for locating the exact problem. Unit tests on the other hand are much more specific and can generally tell us both that something is wrong, and what is wrong.

Unit tests don't have access to the database or the file system since they are not run in the context of a fully bootstrapped Drupal environment. Note that calling any Drupal functions which attempt to access the database will result in an exception being thrown and cause your tests to fail. This includes functions like watchdog(), and module_implements(). However, because Drupal doesn't have to be bootstrapped, and we don't have to create an entirely new environment, unit tests are a whole lot faster to execute than functional tests.

In this lesson we're going to take a look at writing a unit test for a function in our module that is used to convert a length of time in seconds to a string like 1 hour 6 minutes. We'll start by looking at the code that does the conversion, and then come up with a list of known inputs and their expected outputs. Finally, we'll write a unit test by adding a new .test file that contains our unit tests and bombarding our conversion function to make sure it's working properly.

Additional resources

DrupalUnitTestCase()

So far all the tests that we've written are testing the functionality of our module in the context of a fresh Drupal install. However, in the real world we're also going to want to have tests that test the content of our application. As well as other things that it's unrealistic to assume we'll also have the time and resources to write a complete installation routine to replicate. Instead, it would be nice if we could test against a clone of our site rather than a from scratch installation of Drupal.

The SimpleTest module included with Drupal 7 core doesn't support this feature. However, the 7.x-2.x version of the module in contrib has a DrupalCloneTestCase class that we can extend instead of the usual DrupalWebTestCase that operates on a clone of the database from our existing site. Allowing us to test things like, "Does the about page exist.", and other mission critical features of our site. In this lesson we'll take a look at installing the SimpleTest module from contrib, and writing some new tests using the 2.x version of the API in order to test an existing website.

Note that after you install the 2.x version of the SimpleTest module none of the tests we've written so far, or any of the tests from core will be in the list of available tests to run. This is because each tests needs to explicitly declare that it is compatible with the 2.x version of the test suite. This is done in the .info file of the module that provides the tests.

Add this to your .info file:

testing_api = 2.x

As of the time that this video was recorded you also need to apply the patch in this SimpleTest issue. The current version of the patch to use is https://www.drupal.org/files/issues/983266-10-simletest-clone.patch. Without it the DrupalCloneTestCase setup method won't properly clone the tables from your existing site. You should always use whatever the latest working patch is from that issue.

Here's how I patched it:

curl -O https://www.drupal.org/files/issues/983266-10-simletest-clone.patch
patch -p1 < 983266-10-simletest-clone.patch

Additional resources

SimpleTest 2.x module

Patched with patch from issue #983266