There just a couple of components that make up the token system in Drupal 7, which is nice because it means fewer new things we have to learn. But, it's important to understand what those components are and some of the fundamentals of how tokens work in order to make the most of Drupal's Token API. In this lesson we'll take a look the use case for tokens, talk about token types, and the concept of global tokens vs. those that require some additional context in order to have their dynamic values calculated.
What are tokens?
Tokens are specially formatted chunks of text that serve as placeholders for a dynamically generated value. Here's a really simple example: You want to display a welcome message to user's of your site, and you want it to be personalized so how about you add their name and instead of just saying "Welcome", you can say "Welcome, Joe". In order to avoid having to hard-code a welcome string for every single user of the site it would be nice to dynamically generate the string. So you use a one like the following, "Welcome, [current-user:name]".
The [current-user:name] here is what Drupal refers to as a token. A string of static text that will be located and replaced with a dynamic value. This token is made up of a few parts, inside of the mandatory square brackets that signify that this is a token. The first part current-user: in this case is the token type.
Token types are used to group like things together into a namespace. User's for example have name, mail, and last-login properties. Nodes have title, nid, and author properties. Token type also plays an important role in determining what tokens are available in what context. User tokens for example might be available when sending an email to a specific user but node type tokens might be irrelevant in this use case. We need to be able to tell end user's what types of tokens can be used in what context so that they don't use a node token (which has no value) in a user context.
The second part, after the colon (:) is the token itself. This signifies what value will be substituted into the string containing the token. :name in this case indicates we want the current user's username. This also brings up another important point. Global vs. contextual tokens. Some tokens like this one [system:date] can be calculated without any additional information. Simply call the PHP date function and you've got a value. This token [current-user:name], on the other hand, requires knowledge about the currently logged-in user in order to be able to determine that user's name. This token requires additional context in order to be useful.
Additional resources
Before we can get started creating our own tokens we'll need a basic Drupal site setup and module to start from. This lesson will walk through the prerequisites for the rest of the series including having Drupal installed, downloading and installing the databasics module, creation of some dummy content, and a quick tour of what the module does.
In order to follow along in this series you'll want to have a copy of Drupal 7 installed. If you need a refresher on installing Drupal check out this series on getting up and running with Drupal 7.
Once you've got Drupal 7 installed you'll want to download and install a copy of the databasics module that was created as part of the Drupal 7 module development series. We'll use this as a starting point for adding custom tokens so that we have some new data to play with. You can grab a copy of the files attached to this page.
Finally, we'll create some dummy content on our site and walk through what the databasics module does and talk about the problem we're trying to solve.
The databasics module has a string of text that's displayed at the bottom of each node that displays some dynamic information. We would like this text to be configurable by an administrator, so we'll add some new tokens and then update both the form where text is entered and the code that displays the text to allow for use of the newly created tokens.
Additional resources
The first of two steps required to add new tokens to Drupal 7 is implementing hook_token_info() in order to give Drupal a list of the placeholders your module provides. Placeholders can be grouped together by creating a new token type and placing new tokens under that grouping or by adding new tokens to an existing token type provided by another module like the node module. Token types can be defined as global meaning their value can be calculated without any additional context, or as 'needs-data' tokens that require additional information about the context in which they are being used in order to calculate their value. For example, a [node:title] token needs to know which node is being referenced, whereas a [system:date] token doesn't need any additional reference to calculate the current date. In this lesson we will be adding the following token types:
[databasics-totals]- For tokens that can be calculated without any additional context.[databasics-page]- For tokens that require additional context in order to be rendered. In this case, they need to know which page is currently being viewed and are thus only useful in the context of viewing a page.
And these tokens:
[databasics-totals:count]- Total page views for the entire site.[databasics-page:view-count]- Number of times the current page has been viewed by the current user..[databasics-page:last-viewed]- Date the current page was last viewed..
We use hook_token_info() to provide Drupal with information about available placeholder tokens and their types. Here's an example hook_token_info() implementation:
/**
* Implements hook_token_info().
*/
function databasics_token_info() {
$info = array();
$info['types'] = array(
'databasics-totals' => array(
'name' => t('Databasics totals'),
'description' => t('Global databasics tokens.'),
),
// [databasics-page:]
'databasics-page' => array(
'name' => t('Databasics'),
'description' => t('Tokens for databasics page counts.'),
'needs-data' => array('databasics_record'),
),
);
$info['tokens'] = array(
'databasics-totals' => array(
// [databasics-totals:count]
'count' => array(
'name' => t('Total page views'),
'description' => t('Total page views for entire site.'),
),
),
// Page specific tokens.
'databasics-page' => array(
// Add a token for the view count.
// [databasics-page:view-count]
'view-count' => array(
'name' => t('View count'),
'description' => t('Number of times the page has been viewed by the current user.'),
),
// [databasics-page:last-viewed]
'last-viewed' => array(
'name' => t('Last viewed'),
'description' => t('Date the page was last viewed.'),
),
),
);
// [node:view-count], [node:view-count:last-viewed]
$info['tokens']['node'] = array(
'view-count' => array(
'name' => t('View count'),
'description' => t('Number of times the current node has been viewed by the current user.'),
'type' => 'databasics-page',
),
);
return $info;
}
Additional resources
Note: In the video, around 9:30, we added at the :uid argument to the query, but did not add the corresponding portion to the WHERE clause of the query. The final query should look like the one in the code sample below.
Now that we've got placeholder tokens that we can enter into a string of text we need to provide the actual values that should be used to replace those placeholders. In this lesson we'll implement hook_tokens() in order to provide the Drupal Token API with the values that correspond to the placeholders our module provides.
Implementations of hook_tokens() are called once for every token type that's in scope for the current string that's being processed. So, one for all user tokens, once for all node tokens, and once for all global tokens. Each time the hook is passed an array which contains all the tokens of that specific type that where found in the string being processed as well as any additional contextual information such as the current $user or $node object.
hook_tokens() is expected to return an array of values for each of the tokens in question that your module is responsible for. In our case, since we added the [databasics-page:*] and [databasics-totals:*] tokens we're responsible for calculating and returning their value whenever they are requested.
We'll also look at using the token_find_with_prefix() function which will allow us to detect and provide values for any chained tokens. Like for example [node:view-count:last-viewed].
Example:
/**
* Implements hook_tokens().
*/
function databasics_tokens($type, $tokens, array $data = array(), array $options = array()) {
$replacements = array();
if ($type == 'databasics-totals') {
foreach($tokens as $name => $original) {
switch($name) {
case 'count':
$count = db_query('SELECT SUM(view_count) FROM {databasics}')->fetchField();
$replacements[$original] = $count;
break;
}
}
}
if ($type == 'databasics-page' && !empty($data['databasics_record'])) {
$record = $data['databasics_record'];
foreach ($tokens as $name => $original) {
switch ($name) {
case 'view-count':
$replacements[$original] = $record->view_count;
break;
case 'last-viewed':
$replacements[$original] = $record->last_viewed;
break;
}
}
}
if ($type == 'node' && isset($tokens['view-count']) && !empty($data['node'])) {
$node = $data['node'];
global $user;
$count = db_query('SELECT SUM(view_count) FROM {databasics} WHERE nid = :nid AND uid = :uid', array(':nid' => $node->nid, ':uid' => $user->uid))->fetchField();
$replacements[$tokens['view-count']] = $count;
// [node:view-count:last-viewed]
if ($count_tokens = token_find_with_prefix($tokens, 'view-count')) {
$record = databasics_get_record($node->nid, $user->uid);
$replacements += token_generate('databasics-page', $count_tokens, array('databasics_record' => $record), $options);
}
}
return $replacements;
}
Additional resources
With a list of placeholders and a way to retrieve the values for those placeholders we can now bring it all together and use the token_replace() function to locate placeholders in a string of text and replace them with their dynamically generated counterparts. We'll use token_replace() to process the $message variable displayed on nodes by the databasics module so that we can use our new tokens. Then we'll look at passing contextual data like the current $node to the token_replace() function so the code that does the actual value calculation can have all the information it needs to do so.
The token_replace() function will take a string of text like the following: "Welcome, [current-user:name]", and perform the replacement of the token with it's value resulting in something like "Welcome, Joe".
Additional resources
The Token API built into Drupal 7 core provides the ability to add placeholder tokens and replace them with dynamic values. But it's missing some critical features. A UI for allowing end users to browse the list of available tokens for example. And core only provides some very basic tokens, missing support for things like field values. In this lesson we'll look at additional functionality provided by the token module from contrib and how we can make use if it to provide both a UI for browsing available tokens as well as better validation of form elements with tokens in them.
First we'll download and install the token module from contrib and look at some of the features it provides us with without having to write any code.
Then, we'll look at using theme_token_tree_link() to provide our end users with a user interface for browsing the tokens available in a given context. theme_token_tree_link() also allows us to specify the token types that should be displayed so we can limit the list to just those that are relevant.
Finally, we'll look at using the token_element_validate() function as an extra element validator on our Form API element where we allow users to enter strings of text that contain tokens. This validator will provide valuation of the tokens entered into the form element, for example: if the valid token types are 'node' which means they can only use 'node' tokens in that field. This makes sure that the tokens you enter into a field will work in that context.
Additional resources
Web services, APIs, and structured data are all the rage right now, and with good reason. As more and more internet enabled devices start wanting to make use of the data in our websites, we need to give them some way to interact with that data that isn't point and click in a browser. Enter the Drupal 7 services module.
In this series Joe Shindelar will walk through the services module and teach you how to create a RESTful web services API that exposes Drupal's internal data like nodes and users as JSON or XML. This paves the way for a huge variety of clients using a number of programming languages to access the data contained within your Drupal site, from native mobile applications and partner websites to internet enabled refrigerators.
The first few lessons in the series are focused more on theory. They explain the various terms that are used both in Drupal and the wider web development sphere to describe web services and all their components. Joe describes REST itself and explains why it's a good fit, and discusses Drupal 7's content model and the ways that it lends itself nicely to serializing data into various formats via the services module in order to be consumed in structured and meaningful ways.
Throughout the series we'll see a couple of examples of how to first enable Drupal to return requested data elements as JSON and then subsequently how to properly format an HTTP request to retrieve those elements. Joe will demonstrate making requests to the API via both cURL and the Chrome REST Console. This demonstration will help students to learn about both the Drupal configuration that is required and the way in which any third party applications can request data from Drupal via concrete examples.
With the basics of creating an endpoint and making simple requests out of the way, Joe will demonstrate how to enable authentication via the services module and then use both cURL and the REST Console to explain the authentication handshake: a somewhat complex exchange of a username and password for a session authentication token that takes place in HTTP requests and headers whenever you want to perform an action via the API that requires an authenticated user. This will enable to you create third party applications that can access Drupal as a specific user, allowing for greater personalization.
After that, Joe looks at ways to integrate the lists that site administrators create using the views module with services in order to output their contents as structured data, as well as ways to use views to create entirely new custom services resources, followed by an example of leveraging the power of views exposed filters via services.
Finally, for those scenarios where you simply can't point and click your way to an answer, Joe will teach you how to implement your own custom services resources for both saving and retrieving data from within Drupal, demonstrating the basic knowledge required to allow you to use services to solve all of your own custom API needs.
Things you'll be able to do after completion of this series:
- Understand the basic principles of both web services and RESTful APIs.
- Explain why Drupal is a good fit for web applications that want to allow third party clients to access their data.
- Create a RESTful web services API using the services module for Drupal 7.
- Retrieve data from your Drupal site in both JSON and XML format.
- Create new nodes in Drupal via web services.
- Make authenticated requests to Drupal via web serivces.
- Integrate the views module with services to retrieve views data in various formats.
- Write your own module providing custom services data and actions.
- Create meaningful documentation that will enable others to take advantage of your newly created service.
This series provides information about the services module for both intermediate and advanced Drupal users. Since this module makes use of basic Drupal site building skills and custom views, users should be familiar with those topics. Also, while not required, knowledge of running commands via the command line will be helpful. Finally, if you want to write your own custom services resources you'll need to understand basic module development.
Additional resources
This lesson covers what an API is and why you might want to build one. Joe also explains the basic concepts behind any API and provides real world examples of existing APIs that you're probably familiar with as well as use cases for when and why you might want to build your own.
In this lesson Joe shows you where to locate the services module and it's documentation. Then walks through installing the services module and confirming that it's working. Followed by a quick overview of the module's codebase and a general overview of what the services module provides.
Additional resources
The API that we're going to be building through this series is a REST based API so we need to understand the basic tenants of RESTful web services. In this lesson Joe gives a short presentation explaining the basic terminology, workflow, and tennants of REST based web services.
Additional resources
This lesson walks through defining the URL that will serve as the endpoint for your API as well as all the basic configuration for your new services endpoint with the exception of resource configuration which is covered in the following lesson.
Additional resources
The services module exposes most of Drupal core's data as resources and provides CRUD operations and additional actions for each. In this lesson Joe will look at the available resources, talk about the various CRUD operations and actions and then enable the node resource and demonstrate retrieving node data as JSON from the services endpoint.
Additional resources
JSON Formatter - Chrome plugin to display results of a JSON request in a nicer way.
While cURL may not be the simplest way to interact with a REST API it is the most ubiquitous and the one that is most often referenced in documentation through the web. So to ensure that students have at least a baseline of understanding, and will know how to read the documentation in the future, in this lesson Joe takes a quick look at how to test our API with cURL by retrieving, and creating data.
Example Commands:
`curl http://localhost/demos/services-7x/docroot/api/v1/node/1`
`curl --data '{"title":"hello world!","type":"page"}' --header "Content-Type:application/json" http://localhost/demos/services-7x/docroot/api/v1/node`
Additional resources
Instead of using the CLI to test our API we can use the powerful Chrome REST Console plugin instead. This provides a nice GUI and makes it a little easier to both test and understand what is going on. In this lesson Joe looks at installing the plugin and making basic requests to our API using the REST Console interface.
Additional resources
The Chrome plugin demonstrated in this video is no longer available but there are many alternatives available. Use Postman or search for "REST client" to find tooling options.
In this lesson Joe will demontrate using additional parameters and arguments to refine the returned data when making requests from our API. As well as walking through the code that shows where the various parameters are defined and how you can find what options are available. Finally, we install the services tools module which provides some additional documentation.
Additional resources
Services implements resources for all of core's basic data types but sometimes the information returned in the response is either to much or to little and we want to make changes to the response data. In this lesson Joe will look at how we can write a simple custom module that implments basic hooks provided by the services module to allow us to alter the response returned by a request to the node resource.
Most of the actions we want to perform and data we want to retrieve from an API is likely restricted to authenticated users. This lesson will outline the process of making an authenticated request and walk the user through the configuration that is required for our services endpoint in order to start allowing session based authentication.
In this lesson Joe walks through making authenticated requests to our API with cURL. Although cURL can be a bit verbose when making authenticated requests it serves as a good way to talk about all the headers that are a required and a lowest common denominator for how you could accomplish authentication in just about any language or application.
Example commands
curl http://services-demo.lan/api/v1/user/login -d '{"name":"admin","pass":"admin"}' -H "Content-type: application/json" -H "Accept: application/json"
curl http://services-demo.lan/api/v1/system/connect -X POST
curl http://services-demo.lan/api/v1/system/connect -H "Cookie: SESS60f8c5b86739b7e326223b4ef35867b2=A86XHGJWlnDcMOGcArbOT-qHrsIi5P2NrcoNTXwWluw" -H "X-CSRF-Token: T77haXwD7JKOJsBlKP3p3kLbjQO96bQWvGJAE1_PUZM" -X POST
curl http://services-demo.lan/api/v1/user/login -X POST -H "Content-type: application/json" -H "Accept: application/json"
In this lesson Joe walks through making authenticated requests using the Chrome REST Console Plugin
Additional resources
The Chrome plugin demonstrated in this video is no longer available but there are many alternatives available. Use Postman or search for "REST client" to find tooling options.
All the power of views made available to your API. Really, what else is there to say? In this lesson Joe walks through installation and basic configuration of the services views module which provides the ability to expose views as resources via the services module.