Drupal 7 Token API

Tokens are simple strings of text that serve as placeholders for a dynamic value. This is such a common task for modules that, rather than have every module developer reinvent the wheel, Drupal's token API allows for this kind of placeholder replacement using a unified syntax and a set of hooks and functions, which we'll cover throughout the series.

This video gives an overview of the six lessons in this series, from background to a sample installation of modules we’ll need, implementing tokens using hooks, and looking at the relationships between core and token modules.

After watching this series, you should have a firm grasp on the Drupal token API and be able to use it in your own custom modules in order to provide your users with static placeholder tokens that can be replaced with a dynamically calculated value.

Additional resources

• Drupal 7 Module Development Series
• Contrib Token Module

There just a couple of components that make up the token system in Drupal 7, which is nice because it means fewer new things we have to learn. But, it's important to understand what those components are and some of the fundamentals of how tokens work in order to make the most of Drupal's Token API. In this lesson we'll take a look the use case for tokens, talk about token types, and the concept of global tokens vs. those that require some additional context in order to have their dynamic values calculated.

What are tokens?

Tokens are specially formatted chunks of text that serve as placeholders for a dynamically generated value. Here's a really simple example: You want to display a welcome message to user's of your site, and you want it to be personalized so how about you add their name and instead of just saying "Welcome", you can say "Welcome, Joe". In order to avoid having to hard-code a welcome string for every single user of the site it would be nice to dynamically generate the string. So you use a one like the following, "Welcome, [current-user:name]".

The [current-user:name] here is what Drupal refers to as a token. A string of static text that will be located and replaced with a dynamic value. This token is made up of a few parts, inside of the mandatory square brackets that signify that this is a token. The first part current-user: in this case is the token type.

Token types are used to group like things together into a namespace. User's for example have name, mail, and last-login properties. Nodes have title, nid, and author properties. Token type also plays an important role in determining what tokens are available in what context. User tokens for example might be available when sending an email to a specific user but node type tokens might be irrelevant in this use case. We need to be able to tell end user's what types of tokens can be used in what context so that they don't use a node token (which has no value) in a user context.

The second part, after the colon (:) is the token itself. This signifies what value will be substituted into the string containing the token. :name in this case indicates we want the current user's username. This also brings up another important point. Global vs. contextual tokens. Some tokens like this one [system:date] can be calculated without any additional information. Simply call the PHP date function and you've got a value. This token [current-user:name], on the other hand, requires knowledge about the currently logged-in user in order to be able to determine that user's name. This token requires additional context in order to be useful.

Additional resources

Handbook documentation for tokens (drupal.org)

Before we can get started creating our own tokens we'll need a basic Drupal site setup and module to start from. This lesson will walk through the prerequisites for the rest of the series including having Drupal installed, downloading and installing the databasics module, creation of some dummy content, and a quick tour of what the module does.

In order to follow along in this series you'll want to have a copy of Drupal 7 installed. If you need a refresher on installing Drupal check out this series on getting up and running with Drupal 7.

Once you've got Drupal 7 installed you'll want to download and install a copy of the databasics module that was created as part of the Drupal 7 module development series. We'll use this as a starting point for adding custom tokens so that we have some new data to play with. You can grab a copy of the files attached to this page.

Finally, we'll create some dummy content on our site and walk through what the databasics module does and talk about the problem we're trying to solve.

The databasics module has a string of text that's displayed at the bottom of each node that displays some dynamic information. We would like this text to be configurable by an administrator, so we'll add some new tokens and then update both the form where text is entered and the code that displays the text to allow for use of the newly created tokens.

Additional resources

• Drupal 7 module development series
• Drupal 7 download

The first of two steps required to add new tokens to Drupal 7 is implementing hook_token_info() in order to give Drupal a list of the placeholders your module provides. Placeholders can be grouped together by creating a new token type and placing new tokens under that grouping or by adding new tokens to an existing token type provided by another module like the node module. Token types can be defined as global meaning their value can be calculated without any additional context, or as 'needs-data' tokens that require additional information about the context in which they are being used in order to calculate their value. For example, a [node:title] token needs to know which node is being referenced, whereas a [system:date] token doesn't need any additional reference to calculate the current date. In this lesson we will be adding the following token types:

• [databasics-totals] - For tokens that can be calculated without any additional context.
• [databasics-page] - For tokens that require additional context in order to be rendered. In this case, they need to know which page is currently being viewed and are thus only useful in the context of viewing a page.

And these tokens:

• [databasics-totals:count] - Total page views for the entire site.
• [databasics-page:view-count] - Number of times the current page has been viewed by the current user..
• [databasics-page:last-viewed] - Date the current page was last viewed..

We use hook_token_info() to provide Drupal with information about available placeholder tokens and their types. Here's an example hook_token_info() implementation:

/**
 * Implements hook_token_info().
 */
function databasics_token_info() {
  $info = array();

  $info['types'] = array(
    'databasics-totals' => array(
      'name' => t('Databasics totals'),
      'description' => t('Global databasics tokens.'),
    ),
    // [databasics-page:]
    'databasics-page' => array(
      'name' => t('Databasics'),
      'description' => t('Tokens for databasics page counts.'),
      'needs-data' => array('databasics_record'),
    ),
  );

  $info['tokens'] = array(
    'databasics-totals' => array(
      // [databasics-totals:count]
      'count' => array(
        'name' => t('Total page views'),
        'description' => t('Total page views for entire site.'),
      ),
    ),
    // Page specific tokens.
    'databasics-page' => array(
      // Add a token for the view count.
      // [databasics-page:view-count]
      'view-count' => array(
        'name' => t('View count'),
        'description' => t('Number of times the page has been viewed by the current user.'),
      ),
      // [databasics-page:last-viewed]
      'last-viewed' => array(
        'name' => t('Last viewed'),
        'description' => t('Date the page was last viewed.'),
      ),
    ),
  );

  // [node:view-count], [node:view-count:last-viewed]
  $info['tokens']['node'] = array(
    'view-count' => array(
      'name' => t('View count'),
      'description' => t('Number of times the current node has been viewed by the current user.'),
      'type' => 'databasics-page', 
    ),
  );

  return $info;
}

Additional resources

• hook_token_info()

Note: In the video, around 9:30, we added at the :uid argument to the query, but did not add the corresponding portion to the WHERE clause of the query. The final query should look like the one in the code sample below.

Now that we've got placeholder tokens that we can enter into a string of text we need to provide the actual values that should be used to replace those placeholders. In this lesson we'll implement hook_tokens() in order to provide the Drupal Token API with the values that correspond to the placeholders our module provides.

Implementations of hook_tokens() are called once for every token type that's in scope for the current string that's being processed. So, one for all user tokens, once for all node tokens, and once for all global tokens. Each time the hook is passed an array which contains all the tokens of that specific type that where found in the string being processed as well as any additional contextual information such as the current $user or $node object.

hook_tokens() is expected to return an array of values for each of the tokens in question that your module is responsible for. In our case, since we added the [databasics-page:*] and [databasics-totals:*] tokens we're responsible for calculating and returning their value whenever they are requested.

We'll also look at using the token_find_with_prefix() function which will allow us to detect and provide values for any chained tokens. Like for example [node:view-count:last-viewed].

Example:

/**
 * Implements hook_tokens().
 */
function databasics_tokens($type, $tokens, array $data = array(), array $options = array()) {
  $replacements = array();
 
  if ($type == 'databasics-totals') {
    foreach($tokens as $name => $original) {
      switch($name) {
        case 'count':
          $count = db_query('SELECT SUM(view_count) FROM {databasics}')->fetchField();
          $replacements[$original] = $count;
          break;
      }
    }
  }
 
  if ($type == 'databasics-page' && !empty($data['databasics_record'])) {
    $record = $data['databasics_record'];
 
    foreach ($tokens as $name => $original) {
      switch ($name) {
        case 'view-count':
          $replacements[$original] = $record->view_count;
          break;
 
        case 'last-viewed':
          $replacements[$original] = $record->last_viewed;
          break;
      }
    }
  }
 
  if ($type == 'node' && isset($tokens['view-count']) && !empty($data['node'])) {
    $node = $data['node'];
    global $user;
    $count = db_query('SELECT SUM(view_count) FROM {databasics} WHERE nid = :nid AND uid = :uid', array(':nid' => $node->nid, ':uid' => $user->uid))->fetchField();
    $replacements[$tokens['view-count']] = $count;
 
    // [node:view-count:last-viewed]
    if ($count_tokens = token_find_with_prefix($tokens, 'view-count')) {
      $record = databasics_get_record($node->nid, $user->uid);
      $replacements += token_generate('databasics-page', $count_tokens, array('databasics_record' => $record), $options);
    }
  }
 
  return $replacements;
}

Additional resources

• hook_tokens()
• token_find_with_prefix()

With a list of placeholders and a way to retrieve the values for those placeholders we can now bring it all together and use the token_replace() function to locate placeholders in a string of text and replace them with their dynamically generated counterparts. We'll use token_replace() to process the $message variable displayed on nodes by the databasics module so that we can use our new tokens. Then we'll look at passing contextual data like the current $node to the token_replace() function so the code that does the actual value calculation can have all the information it needs to do so.

The token_replace() function will take a string of text like the following: "Welcome, [current-user:name]", and perform the replacement of the token with it's value resulting in something like "Welcome, Joe".

Additional resources

• token_replace()

The Token API built into Drupal 7 core provides the ability to add placeholder tokens and replace them with dynamic values. But it's missing some critical features. A UI for allowing end users to browse the list of available tokens for example. And core only provides some very basic tokens, missing support for things like field values. In this lesson we'll look at additional functionality provided by the token module from contrib and how we can make use if it to provide both a UI for browsing available tokens as well as better validation of form elements with tokens in them.

First we'll download and install the token module from contrib and look at some of the features it provides us with without having to write any code.

Then, we'll look at using theme_token_tree_link() to provide our end users with a user interface for browsing the tokens available in a given context. theme_token_tree_link() also allows us to specify the token types that should be displayed so we can limit the list to just those that are relevant.

Finally, we'll look at using the token_element_validate() function as an extra element validator on our Form API element where we allow users to enter strings of text that contain tokens. This validator will provide valuation of the tokens entered into the form element, for example: if the valid token types are 'node' which means they can only use 'node' tokens in that field. This makes sure that the tokens you enter into a field will work in that context.

Additional resources

• Token module
• Token module documentation