There are a couple of configuration options available when configuring a Search API index that we haven't looked at yet: adding additional fields, and using boost values to increase the relevance of a keyword when found in a specific field.
Solr allows you to index any number of additional fields, so we'll add a species and genus field to our index. This is one of the reasons using Search API to interface with Solr is so great. Through it's use of the Entity API, the Search API module has a deep understanding of all the content types on your site and the fields that are attached to them, without you having to write any code, or do anything other than configure things in the UI.
One of the benefits of creating your own search index is that you know your data better than anyone, and you know what people are hoping to find in your content. Solr allows you to configure a boosting value that can be used to increase the relevancy of keywords found depending on where in the data it's located. For example, when someone searches for a keyword we can probably assume that if the keyword is in the page title that the keyword is worth more relevancy points than if the keyword is found in the page body. With boosting we can affect the relevancy ranking of results and help our users more quickly find what they are looking for.
By the end of this tutorial you should be able to add additional fields to your Solr index so their content is available for searching, as well as assign a relevancy boosting value when keywords are found in specific fields.
The Search API module supports a handful of data alterations and processors; additional operations that can be performed on a document before it's indexed or during the display of search results. While Solr actually handles the majority of these for us already, this tutorial will look at the available options, talk about what each one does, and explain which ones are still relevant when using Solr as a backend.
Looking at data alterations in the Search API module also raises an important point about security. By default, Search API doesn't care about your content's access control settings. In order to prevent people from seeing results for their searches that contain data they shouldn't have access to we need to make sure we account for that in our configuration.
Here's a good list of the currently available data alterations and processors, though it's worth noting that not all of them are available for all search backends. Also, as we'll see, not all of them are recommended when using Solr even if they are available. Solr's tokenizer for example is much more full featured than the Search API tokenizer, so when using Solr as a backend it's best to keep the Search API tokenizer turned off and let Solr do its thing.
By the end of this lesson you should be able to use data alterations and processors to filter out specific content types from your Solr index and to highlight keywords found when displaying search results. You'll also be able to explain why some alterations and processors are better left off so that Solr can handle those tasks directly.
Additional resources
Being able to display search results using the Views module provides a huge amount of flexibility with respect to what is listed, what it looks like, and more. In this tutorial we'll look at using the Search API Views module, included in the Search API project, to create a view that allows users to search our Solr index and display the results as a table, or really, in any other way that Views can display content. We'll also cover some special considerations regarding access control and entity relationships that we need to keep in mind when using Views to display search results.
The biggest difference between creating a view that lists a bunch of nodes, and one that displays search results is that you need to use your Search API index as the base table from which you're building your view. Then, by default, the view only has access to the fields that are in the Solr index. This allows you to build the entire view without having to query the database. Or you can use the Views module's ability to define relationships to other buckets of content to query the database and pull in additional information. There's a huge amount of flexibility.
When building views from the Solr index you can optionally expose one or more filters. Essentially creating a form that allows someone to construct a search query. This can be as simple as exposing a keyword text field, or as complex as you would like to get. We'll look at using exposed filters to create a form that users can perform a search with and create a more complete search experience. We'll also look at how you can move those exposed filters into a block that can be displayed on the home page of our site, allowing us to replace the functionality provided to the Drupal core Search module with Views and Solr.
By the end of this tutorial you should be able to create a view that displays search results using the Search API Views module.
Additional resources
Using facets allows users of your search application to further narrow the results returned from a keyword search by selecting one or more attributes of the returned content and saying either show me only these, or show me everything but these. In this tutorial we'll take a look at some examples of faceted searching in practice, and then we'll use the Facet API module to expose facets for our genus and species fields.
One of the most common uses of facets is on e-commerce sites like Zappos.com that have huge collections of products that users can browse through, and narrow down, to focus in on exactly the pair of shoes they are after. In this example facets allow you do to things like narrow the results returned from your initial keyword search to just shoes for men, which are brown, size 10.5, and on sale. You can can also see faceting in action any time you perform a search on our site.
We'll use facets to allow users of the fish finder application to limit the results returned to just those of a specific species or genus. In doing so we'll also look at the options available for determining how facets should be displayed, whether or not we should show a facet that has zero documents in our result set, and how to combine multiple facets together into a single query using either AND or OR logic.
By the end of this tutorial you should be able to use the Facet API module in conjunction with Search API in order to provide facets that your users can use to further narrow and refine their search results.
Additional resources
Depending on the data that is being searched, some shorter general words, like "a", "the", or "is" can adversely effect search result relevancy. Consider the word "the", which in a standard description of a fish in our database could easily appear hundreds of times or more. When a search is performed, part of the algorithm that calculates the relevancy of any document in the index is to count the number of times a word appears in the text being searched. The more often it appears, the more relevant the document. Words like "the" however often have little to no real bearing on a document's actual relevancy. These words should instead be excluded from the ranking algorithm.
Stop words can also serve another purpose. You can filter out words that are so common in a particular set of data that the system can't handle them in a useful way. For example, consider the word "fish" in our dataset. It's probably very common. With only 500 fish being indexed it's not really going to make much difference, but what if we were indexing five million fish, and each one had the word "fish" in the description even just five times? That's 25 million occurrences of the word "fish". Eventually we might start to hit the upper limit of what Solr can handle. The word "fish" in this case is probably also not very useful in a search query. You're browsing a fish database. Are you really likely to search for the query fish and expect any meaningful results? Likely it would instead return every result. It would be like going to Drupal.org and searching for the word "drupal" and expecting to get something useful. Not going to happen.
Solr has the ability to read in a list of stop words, or words that should be ignored during indexing, so that those words do not clutter your index and are removed from influencing result relevancy. In this tutorial we'll take a look at configuring stop words for Solr.
First, we'll use the Solr web UI to see the most common terms in our index for the body field. Then, based on that list, and the list of common stop words provided by the Solr team, we'll configure our stopwords.txt file. Finally, we'll re-index all the content of our site so that it makes use of the new stop words configuration and re-examine the most common terms noting that our stop words no longer appear in the list.
By the end of this tutorial you should be able to use the Solr web UI to get a list of the most common terms in your index, and know how to add terms to Solr's stopwords.txt file to prevent them from showing up in your index.
Additional resources
Solr provides the option to configure synonyms for use during both indexing and querying of textual data. A synonym is a word or phrase that means exactly or nearly the same thing as another word or phrase in the same language. For example, shut is a synonym of close. Synonyms, if not accounted for, can cause a dilution of search result relevancy when searching for a keywords that have lots of variations in your index.
Consider for example the words, "ipod", "i-pod", and "i pod". It's pretty easy to imagine a scenario in which the content of our site could contain all three variations of the word. When someone searches though they are likely just going to search for one, but expect results for all three. In order to not break those expectations we need to make sure we account for this scenario. Another example from the the Drupal world would be the terms "CMI" and "configuration management". Chances are if you search for one you would be happy to see results for the other.
In this tutorial we'll look at using the synonyms.txt file that is part of our Solr configuration in order to account for synonyms in our data. Of course the exact words you use will depend on the content of your site, but we can at least cover how they work and how to configure them.
By the end of this tutorial you should be able to configure Solr to be aware of synonyms in your data in order to improve the quality of your search results.
Additional resources
One of the benefits of building our own search application is that we have ultimate control over the ranking of items. Combined with our superior knowledge of our own content we can use this to ensure that when someone searches for a specific keyword we bubble our best content for that term to the top of the list, regardless of whatever Solr might rank it based on its internal algorithms. This is commonly referred to as promoted, or sponsored, results; the artificial boosting of a particular document to the top of the result list for a specific query.
A similar, but not exactly the same, example would be sponsored results on Google searches, where you can pay to have your page listed at the top of the results for a specific keyword or set of keywords. We are going to be doing all of this except for the part where we let people pay to promote results, though you could certainly build that part on your own if you need that.
Solr uses a configuration file named elevate.xml, in conjunction with a processor, to elevate results at the time a query is performed. We can promote specific documents in our Solr index by figuring out the unique Solr ID for a document and then adding it to the elevate.xml file along with some information about a query, or queries, this document should be promoted for.
In this tutorial we'll learn how to find a Solr document's unique ID, and then configure Solr to use an elevate.xml file that will promote the "How to Use the Fish Finder" page to the top of the results when someone searches for the term "fish". This configuration is all within the Solr application itself and doesn't really rely on Drupal in anyway. As such, the material in this tutorial should be applicable to your Solr search applications even if you're not building them with Drupal.
By the end of this lesson you should be able to configure promoted documents in your own Solr-based search application.
Additional resources
In this series, Getting Started with Responsive Web Design in Drupal, we'll take an old Drupal theme based on a 960 pixel grid, and convert it, step by step, using just good ole CSS and HTML, to be more fluid, more flexible, more responsive than ever.
Our case study is the Anytown Farmers Markets. Anytown Farmers Markets has an existing web site that uses a theme based on a 960 pixel grid. It works great on larger screens, pretty well on iPads, but on an iPhone, the text is really small and you have to pinch and zoom and horizontally scroll to get around the screen.
Our goal will be to transition the site from a fixed width two column desktop site to a fluid and flexible site whose layout, images, and type gracefully transform at practically any size screen to provide a user-friendly experience where our site's content can be enjoyed by users browsing with a more diverse set of devices.
Throughout this series, you'll learn how to use, configure, and customize the style of a Drupal contributed module that provides a responsive, mobile-friendly main menu. We'll tackle images, tables, and slideshows and explore some select solutions for making these traditionally rigid elements flex with a fluid container. In the process of converting this theme to be responsive, you'll learn to tackle some real-world, sometimes messy and often times not-so-clear-cut problems and potential solutions.
To take advantage of this series, you'll want to be comfortable with HTML and CSS and the basics of setting up a theme in Drupal 7. You don't need to know Sass or any advanced theming. This series will help you understand common problems encountered in responsive web design and how to solve them in the context of a Drupal 7 theme.
Additional resources
In the process of transitioning their site to be more mobile friendly, The Anytown Farmers Market web team needs to take a closer look at what content they currently have on their site and how they may want to prioritize and deliver that content differently, given the broader audience base they are now targeting.
As we adapt content and navigation regions to stack, we need to decide on a priority system for each page template. We'll take the home page as an example and discuss how each component and region should be prioritized when the content is stacked into one column.
Additional resources
The original 960 Robots theme is based on a 960 pixel grid. It has container and grid classes with widths declared in pixels—an absolute metric. In this lesson, we will take a first step in implementing a responsive design: converting absolute units to relative ones. We'll begin this process by editing the 960.css file, which contains the width declarations for our container and grid classes that make up our site's layout. Using the target ÷ context = result formula, we'll convert widths declared in pixels first to ems, then to percentages, with a little nudge of the decimal point over two places.
With our layout's container and grid classes using relative widths declared in percentages, we'll be on well on our way to making our site more fluid and flexible.
Additional resources
https://github.com/DrupalizeMe/demo-rwd-7x (Checkout branch 03-fluid-layout)
We're making progress in making our theme more flexible and fluid. Next, we’ll address our site's typography and implement relative font-sizing. This will help ensure that our text is more legible without the need to pinch and zoom. To do this, we’ll convert our font-sizes declared in pixels to ems, using the target ÷ context = result formula. Break out those calculators!
Additional resources
https://github.com/DrupalizeMe/demo-rwd-7x (Checkout branch 04-responsive-typography)
As we've been converting our font-sizes to ems, we've noticed other properties that need updating, including margin and padding. In order to correctly apply our target ÷ context = result formula, we need to know the appropriate value for "context." Converting padding presents us with a new context, different than that of a margin. In this lesson, we'll learn how to determine the appropriate context and convert padding and margin pixels to relative units.
Additional resources
https://github.com/DrupalizeMe/demo-rwd-7x (Checkout branch 05-responsive-padding-margins)
So far, we've adapted our layout to be flexible and fluid using relative units like percentages and ems, instead of absolute ones, like pixels. The problem is that smaller devices will often use resolutions much greater than the actual size of the screen size or viewport. So while our relative sizing is technically working, it's not going to be terribly useful or make our content more readable and accessible until we add an important tag to the head of our html template file: the viewport meta tag.
By adding the viewport meta tag to the head of our html template file, we're letting the browser know that we want the viewport size to be the size of the device, not some huge resolution more suitable for a 17" monitor, for example. This is a small but critical step in implementing a responsive design. Without it, our hard work of converting to relative units just isn't going to pay off.
In this lesson, you'll learn how to add a viewport meta tag to the head of all html pages using a special Drupal template file that we'll create. Then we'll step back and admire the results using our mobile emulation tools.
Additional resources
https://github.com/DrupalizeMe/demo-rwd-7x (Checkout branch 06-viewport-meta)
Written tutorial based on this video
While it's not forbidden to use absolute positioning in responsive designs, the way that absolute positioning was utilized in the original theme is less than ideal as we work on moving theme toward flexibility, fluidity, and responsiveness.
In this lesson, we'll focus on putting elements in the header back into the natural flow of the document. We'll refactor some HTML and CSS, removing absolute positioning declarations and change the source order of some elements in our page template file.
Restoring the natural order and flow of the document will make things easier for us down the road as we adapt our components to stack into one-column in small screens.
Additional resources
https://github.com/DrupalizeMe/demo-rwd-7x (checkout branch 07-08-flow-media-queries)
In this series, we’ll start out by adapting the Anytown Farmers Market website layout for display on a small mobile screen. The current layout is a stacked 2-column layout with a full-width header, primary content column, and two sidebars. We'll simplify and update the wide layout to use just one main content column plus a right sidebar that will stack under the content region on smaller viewports. We'll use the breakpoints that we identified in the previous lesson to implement width-based media queries and trigger the appropriate layout — stacked for narrow screens and 2-column for wider viewports.
Additional resources
https://github.com/DrupalizeMe/demo-rwd-7x (checkout branch 07-08-flow-media-queries)
Responsive Menus module enables a site builder to install and configure a responsive menu with very little CSS required. We'll walk through the configuration of the module using the media query we built in the previous lesson. We'll also add some CSS to hide our desktop navigation when the mobile menu is shown.
Additional resources
Responsive Menus module (Drupal.org)
https://github.com/DrupalizeMe/demo-rwd-7x (checkout branch 09-responsive-menu-contrib-module)
There are quite a few images on the site that aren’t scaling very well. They’re either way too small or way too big and break the layout. For now, we just want ensure that our images don’t overflow beyond the layout container and that they are viewable at a comfortable size across all viewport sizes. While we're at it, we can make sure all HTML media objects stay safely contained in a flexible container, especially when sizing down.
Additional resources
https://github.com/DrupalizeMe/demo-rwd-7x (checkout branch 10-max-width)
Our image gallery plugin has long been known to be barely navigable on a small screen. Let’s fix our images and replace our photo gallery plugin with a responsive image slider that will work on small to large screens. We'll swap out Views Slideshow in favor of FlexSlider in this lesson.
Additional resources
https://github.com/DrupalizeMe/demo-rwd-7x (checkout branch 11-flexslider)
In this tutorial, you will learn how to get a Drupalize.Me tutorial demo site up and running using Pantheon. You'll learn about the various components that make up the Drupalize.Me demo site downloads and how each part should be imported. By the end of this lesson, you'll know how to create a Drupalize.Me demo site on a free Pantheon Dev instance so that you can follow along with the trainer in the Drupalize.Me video tutorial.
Additional resources
In this tutorial, you will learn how to use Acquia Dev Desktop 2 to get a Drupalize.Me tutorial demo site up and running. You will learn how to import a Drupal codebase and database containing a finished site for an individual tutorial on Drupalize.Me. This will enable you to walk through the lesson and see what was accomplished on the site during the lesson.