Speed Up CasperJS Tests by Skipping Unnecessary HTTP Resources

Image

You know all those JavaScript tracking codes that get added to the footer of every page on your site? Google Analytics is the classical example, but there are tons of others out there. They are slowing your pages down. In most cases it's probably not that big of a deal. If used correctly, the performance hit seen by the end-user should be pretty minimal. We're usually talking about milliseconds. But, when you're using a tool like CasperJS to perform testing on your site, it's not uncommon for a single test run to visit tens, or even hundreds of pages—and have to load that JS tracking code on every one. Those milliseconds start to add up, and can increase the amount of time it takes for your test suite to run.

Generally, when your browser encounters one of those JavaScript tracking codes the embedded JavaScript either directly references, or ultimately ends up including, a .js file hosted on the providers server. This requires that your browser make and extra HTTP request to retrieve that file. More time spent downloading more data. That, when you're running tests, you probably really don't need. In fact, it's unlikely that you want your test suite mucking up your analytics data or your customer insights information. So instead, let's teach CasperJS to just skip those external files altogether when running tests.

Find the scripts you don't need

The first thing you'll need to do is figure out what script(s) it is that your test-suite is loading but doesn't need. The easiest way to do this is to view your site as a regular user would (since CasperJS is just simulating regular users anyway) and use your browser's web inspector to view the network requests that it's making when you view a page.

• In Chrome, open the Developer Tools (CMD - Option - I) and switch to the "Network" tab
• Navigate to the page you want to test, example: https://drupalize.me/videos
• Browse through the list of resources loaded, look for those of type "script" as a good starting point, hover over an item to see its full path
• If the resources comes from an external server, and you don't need that script, make note of the URI of that resource
• Repeat until you've found the URI's for all the resources you want to skip

Image

In our example we'll skip https://www.googleadservices.com/pagead/conversion.js, and https://googleads.g.doubleclick.net/pagead/viewthroughconversion/1004527117/?random=1438379014341&cv=7....

Stop the requests before they happen

CasperJS is a wrapper around the headless webkit browser PhantomJS. When you instruct CasperJS to navigate to a page it does so in the PhantomJS browser, which loads the page you asked it to, and then any linked resources. Just like Chrome did above. We're going to intercept those resource requests before they happen, compare the URI of the requested resource to our blacklist, and if it matches simply instruct PhantomJS to skip that resource.

Use the casper.options.onResourceRequested configuration option to bind a callback function to the PhantomJS onResourceRequested action. Whatever function you bind to this action will be triggered once for every resource requested by PhantomJS just before making the actual HTTP request. Note that when proxying though CasperJS like this in order to bind to a PhantomJS callback CasperJS will also pass the current Casper instance as the first argument to your function. All remaining arguments are those passed along from PhantomJS.

Example:

casper.options.onResourceRequested = function(casper, requestData, request) ...

For our purposes, the two arguments passed in from PhantomJS are useful. The requestData argument is an object containing meta-data about the request being made. Including the URL. requestData.url. And the second argument is a network request object with the various methods that allow us to control the request, including request.abort(). So now all you have to do is compare the URL of the request in the requestData object to your blacklisted URLs from above, and if you get a match call request.abort() which will instruct PhantomJS to skip that request.

In my experience I've found it useful to use substring matches for blacklisted URLs instead of trying to match the whole thing because often times the URLs contain query parameters that are unique for every request. The code below shows how we're handling this.

/**
 * Add a listener for the phantomjs resource request.
 *
 * This allows us to abort requests for external resources that we don't need
 * like Google adwords tracking.
 */
casper.options.onResourceRequested = function(casper, requestData, request) {
  // If any of these strings are found in the requested resource's URL, skip
  // this request. These are not required for running tests.
  var skip = [
    'googleads.g.doubleclick.net',
    'cm.g.doubleclick.net',
    'www.googleadservices.com'
  ];

  skip.forEach(function(needle) {
    if (requestData.url.indexOf(needle) > 0) {
      request.abort();
    }
  })
};

Load your code when CasperJS runs

The last thing we need to do is make sure that this code is included somewhere where CasperJS will encounter it while running our test suite. You've got two options here. Either, just include the code at the top of your homepage-test.js file and it'll be loaded when you run that test like so:

casperjs test homepage-test.js

Or, if you've got a lot of test files, add this to a file named casperjs-options.js and include it when running your other tests.

casperjs test --include="path/to/casperjs-option.js" homepage-test.js

An aside about images

Images are another common culprit here. In most cases your CasperJS tests aren't going to need to load the images for every page. Luckily, PhantomJS makes it really easy to skip the HTTP request for image resources with just a simple setting.

casper.pageSettings.loadImages = false;

Conclusion

When you're running a test suite that is navigating through your site just as fast as it can load each page excluding resources from the page that don't need to be loaded can speed things up. Using an onResourceRequested callback we can detect requests for resources we don't need and abort those requests before they happen. The improvements might not be huge, but every little bit counts.

For more about using CasperJS check out this post on Using a Remote Debugger with CasperJS.

Interested in other topics related to automated testing, like SimpleTest? I've put together a series of video tutorials on Automated Testing in Drupal 7 with SimpleTest.