Performance and Scalability for Drupal Sites

No one likes to wait for a slow site to load. Not me, not you, and definitely not search engines. And the effect of site load times on things like SEO, user bounce rates, purchase intent, and overall satisfaction are only going to become more pronounced over time.

Drupal is a modern web framework that is capable of serving millions of users. But every site is unique, and while Drupal tries hard to be fast out of the box, you'll need to develop a performance profile, caching strategy, and scaling plan that are specific to your use case in order to be truly blazing fast.

Drupal site performance depends on multiple components, from hardware setup and caching system configuration to contributed modules, front-end page weight, and CDNs. Experienced Drupal developers looking to optimize their applications know where to start looking for potential savings. They can manipulate settings and combinations of these components to achieve the desired results. Our goal with this set of tutorials is to help explain the process and provide you with the insight that comes with experience.

In this tutorial we'll:

• Introduce high-level performance concepts for Drupal that we'll then cover in more detail elsewhere
• Provide an overview of the main Drupal performance components.

By the end of this tutorial, you should understand what components around your Drupal application are responsible for site performance.

Drupal core is built with performance and scalability in mind. It is Fast by Default. But performance is often a by-product of your specific application, and depending on how you're using Drupal, you can further optimize your site using contributed modules. These modules range from debugging utilities to cache-related modules.

It's worthwhile to have a general idea of what's available in the contributed module space. And, when you need to address your site's unique performance needs, it helps if you already know about existing solutions.

In this tutorial we'll:

• Look at a few popular contributed modules that improve Drupal's performance
• Learn about the benefits these modules may provide to your site
• Provide tips on how to configure these modules

By the end of this tutorial you should be able to list some popular performance related Drupal modules and describe their use case.

Performance profiling allows you to see an overview of how your Drupal application stacks up against your users' needs and business requirements. A good profile will help you understand where the performance bottlenecks are and where you should focus your efforts in order to achieve the best results when optimizing your application.

There are many profiling tools available to help you analyze your Drupal site's performance. Some are free -- like the browser’s built-in development tools, the Lighthouse Chrome extension, and XHProf. Some are paid -- like New Relic, Blackfire, and other profiling SaaS solutions.

In this tutorial we'll:

• Outline the general concepts and goals of performance profiling
• List some available profiling tools and their features

By the end of this tutorial you should be able to describe what performance profiling is, and list the tools commonly used to establish a performance profile for a Drupal site.

Over the history of the Internet, the single Page Speed metric evolved into various parameters that influence user experience. These metrics are commonly referred to as Core Web Vitals. Together, they paint a comprehensive picture of the performance of your site from an end user's perspective. These metrics are considered by Google and other search engines when assigning SEO scores.

Knowing what these metrics are, and what they are intended to measure, is an important part of creating a performance profile for your site. This knowledge can also help you find solutions for common performance issues.

In this tutorial, we'll:

• Define the metrics that make up the Core Web Vitals
• Point to additional resources where you can learn more about each metric

By the end of this tutorial, you should know which performance metrics are considered Core Web Vitals and what aspects of site performance they cover.

Lighthouse is an open source, automated tool for analyzing your site's performance. Lighthouse is built-in to the Google Chrome browser. When auditing a page, Lighthouse runs various tests against the page and then reports how well the page did across a broad spectrum of metrics. While Lighthouse doesn't improve the performance of a Drupal site itself, it helps to establish a performance profile and point towards areas that could be improved.

Lighthouse requires the use of Google Chrome. Other browsers include their own performance auditing tools. While the exact usage of each tool varies, the end result is the same: a report that can be interpreted to suggest where to focus your performance-tuning efforts.

In this tutorial, we'll:

• Learn how to run Lighthouse tests against a Drupal site
• Interpret the results of the report generated by Lighthouse
• Provide guidance on next steps to take to address the performance issues Lighthouse finds in our Drupal site

By the end of this tutorial you should be able to use Lighthouse to profile a Drupal site, interpret the results, and know where to start on making improvements.

WebPageTest (webpagetest.org) is a free open source resource that runs performance tests on a site, provides educational reports about what it finds, and suggests optimizations you can make. The tests performed via the WebPageTest interface include Lighthouse tests, performance-specific tests, Core Web Vitals, visual comparisons, and traceroute tests. The tool also allows saving a history of tests if you sign up for a free account. This tool won't make your site faster on its own, but it will give you some good ideas about where to focus your efforts.

In this tutorial we'll:

• Learn how to run performance tests via the WebPageTest web interface
• Learn how to read and interpret the results

By the end of this tutorial, you should know how to use the WebPageTest online interface to analyze a Drupal site's performance.

The performance optimization settings and modules provided by Drupal core are intended to work for the broadest possible set of use cases. From an administrator's perspective they provide minimal configuration options, and are designed to just work by being enabled. But behind that simplicity are some powerful features that will help speed up any Drupal-powered application.

The core Dynamic Page Cache and Internal Page Cache modules are designed to provide a base cache setup for any site. These modules are responsible for the static page cache, dynamic page cache, and lazy loading optimizations.

For developers, Drupal provides a complete and well-designed Cache API. You can, and should, integrate it into your custom code. This integration includes defining the cacheability of any content your module outputs so that Drupal can be smart about how that affects how and when a page that incorporates the output can be cached -- as well as storing and retrieving the results of complex or long-running operations. The API also helps with setting appropriate HTTP headers for the responses Drupal generates for each request so that the user's browser and other layers in the stack can appropriately cache the output.

The entire system is flexible, and there are many contributed modules that can aid in making the default caching system even faster for specific use cases.

In this tutorial, we'll:

• Learn about the caching-related modules in Drupal core
• Review the Drupal core performance settings and recommended values

By the end of this tutorial you should be familiar with the Drupal core modules responsible for caching, their settings, and recommended values.

Drupal has robust Cache API, and various caching layers (both internal and external to Drupal), that work together to decrease application load and boost performance. Drupal's APIs allow developers to declare the cacheability of data. How long can this be stored before it becomes stale? And under what conditions should it be invalidated? Drupal uses that information during the process of building a page to cache as much of the work it does as is possible so that it won't need to do it again. Additionally, Drupal bubbles up the cacheability data from everything required to build a page into HTTP response headers that caching layers external to Drupal can also use to cache the rendered HTML.

When these APIs are combined (and used appropriately), Drupal can be extremely fast for both anonymous and authenticated traffic. But doing so requires understanding the various caching layers, their roles, and their interconnections.

In this tutorial, we'll:

• Review the caching layers and systems behind them
• Learn about components of the Drupal cache system

By the end of this tutorial, you should have a broad understanding of the Drupal caching system, its layers, and a better understanding of where in the stack you should look to optimize for different scenarios.

Drupal site performance relies heavily on caching. Optimal caching (and invalidation) requires that each page is rendered with the correct cacheable metadata. This metadata allows for intelligent caching -- but when something isn't working correctly, it can be tricky to figure out where exactly the metadata was generated from.

When debugging Drupal cache issues, you're usually trying to answer 1 of 2 primary questions:

• Why is this cached? If the information gets stale, why isn’t it updated?
• Why is this not cached? And why is our cache hit rate low?

The Drupal cache system consists of many layers, each of which may contribute to the problem. This tutorial focuses on debugging the Drupal application cache layer, and strategies for debugging Varnish. Given that most external to Drupal layers rely on the use of HTTP headers for caching, you should be able to use similar techniques to those used for debugging Varnish.

In this tutorial, we'll:

• Learn strategies for debugging the Drupal application cache and render cache
• Share strategies for debugging low hit rates when using Varnish

By the end of this tutorial, you should know how to enable and use various cache debugging mechanisms in Drupal to help identify problems in your site performance and resolve them.

There's no magical set of right tools to use to monitor a Drupal site's performance and health. While thinking about performance monitoring, you need to optimize your approach depending on the number of applications you manage, their complexity, business needs, and the skill-set of your team. Based on these factors, you may choose to use one of the core or contributed modules, go with third-party solutions and services, or some combination of both.

Drupal core comes with a couple of modules that allow you to monitor the health and performance of the site including Syslog, Database Logging, and the status reports provided by the System module. There are also numerous community-contributed modules, a sampling of which we'll cover here.

In this tutorial, we'll:

• List some contributed modules that are commonly used for monitoring a Drupal site
• Provide an overview of what each module does

By the end of this tutorial you should be able to list a few contributed modules that might be useful for monitoring your Drupal application and define what each one does.

Pantheon uses its Object Cache and Global CDN services to help improve the performance of your Drupal sites. Using them requires a few steps to enable the services and configure your Drupal site to use them effectively.

Many performance optimizations are hosting-provider-specific. This tutorial covers some Pantheon-specific methods for increasing your Drupal site's performance. It's only relevant if your site is hosted on Pantheon. For other common providers, and generic performance tuning tips, refer to other tutorials in this course.

In this tutorial we'll:

• Learn how to configure and use Pantheon's Object Cache to speed up Drupal caching
• Learn about the use case for Pantheon's Advanced Page Cache module
• Install and configure the Advanced Page Cache module for better Drupal caching of sites hosted on Pantheon

By the end of this tutorial you should be able to explain when and why you would want to use Pantheon's Object Cache and Advanced Page Cache modules, and how they can help speed up Drupal sites hosted on Pantheon.

Note: This tutorial is specific to Drupal sites hosted on the Acquia platform and covers integrating its features to improve performance.

The Acquia platform includes Memcache, Varnish, and Content Delivery Network (CDN) integration. In order for these to be as effective as possible, they should be configured and tuned for your specific use case. This tutorial provides an introduction to these utilities and common configuration. For more detail, you should consult the Acquia documentation.

In this tutorial, we'll:

• Learn what caching utilities are included in the Acquia platform
• Set up and tune different parts of Acquia's application caching level including Memcache and Varnish

By the end of this tutorial, you'll know what application-level caching options exist on Acquia's platform. And how to configure it, and your Drupal application, for better performance.

Content Delivery Networks (CDNs) play an important role in making a Drupal-powered site fast and secure. The distributed nature of CDNs allows serving web assets such as HTML files, JavaScript, CSS, and media assets through servers located in close geographical proximity to the users, thereby reducing the physical distance data has to travel between the user and the server, and improving performance.

In addition to providing a performance boost, CDNs may also act as a firewall and protect sites from common attacks such as DDoS. The popularity of CDNs has been growing over the past few years, and integrating with them has also gotten easier. Most Drupal web operation platforms, such as Acquia and Pantheon, offer integrations with CDNs out-of-the-box. Even if your hosting platform doesn't provide a CDN, you can always set up your Drupal site to use one.

In this tutorial we'll:

• Define what a CDN is and what it can offer for your site
• Learn about popular CDNs used with Drupal sites
• Review some contributed modules that you can use to help integrate your Drupal site with a CDN

By the end of this tutorial you should be able to define what a CDN is, list CDNs with Drupal integrations, and describe the steps you will need to take to set up your site to work with a CDN.

Server scaling is the process of adding more resources (CPU, memory, disk space, etc.) to a server (or servers) to help with performance. This might be a single server, or a cluster of different machines. When we talk about server scaling, think more about the resources and less about the specific hardware. Modern servers may not always resemble a physical machine that you can open up and insert additional RAM into. But the theory is the same: more memory means the server can handle more concurrent requests.

Sometimes your Drupal site is optimized, but traffic is still high and takes most of the server’s resources. In order to sustain that load you'll need to scale your server up.

Sometimes you don’t have the resources or expertise to implement caching optimizations, refactoring code, and modifying slow queries -- all of which would improve Drupal's performance. In these cases, you may consider server scaling.

Server scaling can be done in two ways: horizontally or vertically.

In this tutorial we'll:

• Learn what server scaling is
• Discuss examples of both horizontal and vertical server scaling
• Talk about when to choose horizontal versus vertical scaling strategies

By the end of this tutorial, you should understand the concept of server scaling and how it applies to a Drupal application.

Sites evolve over time. We're constantly adding and removing modules, modifying content, authoring custom plugins, and changing design elements. All of these changes impact our application's performance -- some more so than others. But if you're not measuring it, you can't know when your site inadvertently gets slower or by how much.

If you are responsible for a site's performance, it might be good to look into benchmarking it and establishing a performance budget early on, then monitor it on an ongoing basis. Many tools, paid and free, allow measuring key web performance indicators and backend code and server performance.

One-time measurements can be useful for immediate debugging, or when figuring out if that big new feature is going to have a negative impact on performance. But for long-term projects, it's helpful to have known baseline values and an established performance budget to see whether your performance improves or declines over time with every new feature.

Establishing the baseline (performance budget) and comparing future measurements is called site performance benchmarking.

In this tutorial, we'll:

• Learn the basics concepts of benchmarking
• Learn a benchmarking process and best practices
• List some commonly used tools for benchmarking Drupal

By the end of this tutorial, you should understand the concept of a performance budget, know when to benchmark your site, and list some tools available to help.

Apache Bench (ab) is a tool that comes with the commonly used Apache HTTP server. It is designed to give you an impression of how your current Apache installation performs. It will work for any HTTP server, not just Apache. Apache Bench shows you how many requests per second your server can serve. This metric is in part a measure of how long it takes Drupal (PHP) to process the request and create a response. While there are other things that the HTTP server does too, executing PHP is by far the most expensive when serving Drupal pages.

Therefore, Apache Bench helps profile your PHP code for new features, patches after their application, and PHP libraries used on the site. You can quickly compare before and after metrics as an indicator of the scale of the impact a change has.

In this tutorial, we'll:

• Learn how to run the Apache Bench tool on our local environment
• Learn to interpret the result of the tests

By the end of this tutorial, you should know how to benchmark and profile your local Drupal installation using Apache Bench (ab).

New Relic is a monitoring service that provides insights into your application stack from front-end performance to the server and infrastructure metrics. New Relic uses a combination of aggregating server logs, and pre-built (or custom) monitors to track the metrics that are most important to your application. The collected data can be organized into custom dashboards, and alerts can be set up and issued per customizable conditions.

In this tutorial, we'll:

• Learn about different New Relic modules and their purpose
• Review some default dashboard components and reports
• Discuss how to use the information in New Relic to understand the health of your Drupal application

By the end of this tutorial, you should understand the basics of using New Relic and the insights it offers to monitor and improve the performance of your Drupal site.

When your site is experiencing performance issues, one way to pinpoint the cause is to use profiling tools. Before you can fix the issue you have to be able to identify what's causing it. All profiling tools do roughly the same thing: they tell you what code is called during the request and how much time is spent executing it. This helps to identify the slowest code and dig deeper into the cause. Once the cause is determined you can start figuring out how to optimize the code.

For this tutorial, we’ll use New Relic as a profiling tool, but you can apply a similar methodology using the profiling tool of your choice.

In this tutorial, we'll:

• Learn how to identify and analyze slow transactions
• Look at common things to check for while profiling
• Cover some questions you should ask when looking at profiling data to help track down the slow code

By the end of this tutorial, you should know how to profile a Drupal site (specifically with New Relic) to find performance bottlenecks.