In our experience, unfortunately, site search engines often let the team down. Instead of being governed by the company’s customer service policy, many site search engines exhibit incompatible personality traits and behaviours, because of the way it has been configured and designed (or implemented on the default settings). While some site search pages are friendly and informative, with navigatable results, too often we find they are:

• Unhelpful (returning unrelated results)
• Uncaring (no results found)
• Out on an extended lunch break (search function coming soon).

In the offline world, would we continue to employ a customer service staff member who exhibited these negative behaviours? No!

Yet a number of companies are keeping this very active staff member (aka the underperforming search engine) on the payroll and, worse, doing nothing to monitor or optimise it.

Identify your search engine’s personality

Role play your search process out loud with your team. Have a team member play the role of the customer visiting your site (use your internal search keyword reports for inspiration) and another member entering the request into the search engine and verbally communicating the results.

It is important not to add any further information to the interaction except for the results the search engine provides. Also, don’t offer further assistance or prompting unless the search engine offers it. At each “show-stopper” event – such as no result, non-targeted results, or a broken link (to name a few) – yell at the top of your voice, “Next customer please!”

Write down words that describe the interaction and the number of customers you sent away, if any.

So, is your search engine with or against you?

Identify your desired search engine behaviours

Now, would you respond to the customer the same way offline and/or if you were following your customer service standards?

For example, on a hot day, a customer walks into your store and asks for ice-cream, but you don’t carry ice-creams. How would you respond to this enquiry? Would you tell the customer that “no results were found”? A more helpful response would be “Sorry, we don’t sell ice-creams.” If you are an astute and caring business person, you would also add, “but we do have fresh frozen yoghurt bars that are very popular on hot days.” What are the chances of the customer buying frozen yoghurt? Some – probably better than the chance of them walking out empty handed.

Is there a match between the real and desired performance of your search engine? If so, congratulations! If not, your search engine is need of a personality makeover.

Using your customer service standards as a guide, configure your hardest-working member of the customer service team (who else will work 24/7?). Soon your site search engine will be serving customers in line with the customer service qualities your company admires.

For further information, download Search Analytics: A Guide to Analyzing and Optimizing Website Search Engines.

With the right site search analysis strategy, you can:

• Discover the needs, motivations and behaviours of users by assessing what they are searching for
• Measure how well the search results lead to desired actions
• Devise techniques for optimising the search function.

Search analytics reveals shortcomings of a website which are not easily identifiable by other analytics techniques, and is equally suitable for intranets, extranets, portals and public websites.

Benefits

• Fully leverage your investment in search engine technology. Using search analytics techniques, you can configure your search engine for optimum search results. A well-functioning search engine makes content more accessible to users and helps improve their experience with your website.
• Understand what is really happening on your website. Users explicitly state what they are looking for when they conduct searches. Without needing much interpretation, their needs, motivations, and current trends can be filtered out to align closely with your website.
• Guide users to high value content. By classifying your website content with users’ choice of vocabulary, you can make most out of user interest by guiding them to high value content items.
• Prioritise development activities. Search analytics reveal your customers’ focus of interest and trends, missing and not well-classified content from your website. Equipped with this knowledge, you can prioritise your product and service development, and website enhancements.

How to do it

At Bienalto, we recommend a three-phase approach to site search analytics:

• Step 1: Review of your website. Conduct a brief review of your website to understand its purpose and target audiences as well as how the search function is integrated.
• Step 2: Analyse search log files. Using your search log files, perform a variety of analyses such as top search terms, lowest hits search terms, single term analysis, analysis of search errors, search context analysis and search analysis by user segments.
• Step 3: Report findings & determine actionable items. In a workshop environment, determine a set of action items to optimise the search function.

To be effective, search analytics should be conducted on an-ongoing basis (i.e. monthly).

According to a research report from Forrester, 90% of firms rate search as `very important’ or `extremely important’, but 52% do not actually measure its effectiveness. This means that website owners don’t know if people can find what they are looking for when they use the search function.

As online content continues to grow, and as users become more experienced in their interactions with websites, search functionality has become more popular as a way of navigating a site. A retailer who participated in the Forrester research reported that one-third of its traffic is found on the search results page.

Does this ring true with your experience as a user in visiting websites? How often do you locate the position of the keyword search function, and then start searching?

As search functionality becomes increasingly commonplace on websites, it’s more important to measure its effectiveness. By doing so, you will:

• Discover the needs of users by assessing what they are searching for
• Learn users’ choice of vocabulary when searching for content on your site
• Measure how the search results lead to desired actions
• Replicate users’ search activity using the same keywords to identify the difference between what they see and what you want them to see.

How to measure site search

Basically, measuring the effectiveness of site search functionality involves three activities.

1. Define what you want people to see
This is a strategic decision and will be different from one site to another. Suffice it to say that you can limit, select, categorise, prioritise, and personalise the search results.

For example, a retail site can stop the display of a press release about their new range of dairy products when a shopper enters the keyword “dairy” (limiting the search results). The same retailer might offer the shopper both higher-priced and lower-priced options in a featured search results section, while listing everything else underneath (selecting). Again, the same retailer might list the subcategories of dairy products in the search results, as opposed to listing the individual products so that the shopper can drill down themselves (categorising). Alternatively, the search results might list products in order of price (prioritising). Finally, placing the brands that the shopper has purchased before at the top of the list is another option (personalising).

2. Monitor the search functionality and outcomes (Search Term Analysis)

The second activity requires capturing the keywords used in the searches and monitoring the outcomes of the searches, also referred to as Search Term Analysis. Recording the keywords and establishing a mechanism to assess the performance of the site against these keywords is essential for measuring the effectiveness of onsite search functionality. In the Forrester research mentioned above, one of the participant companies performed the search for the top keyword manually, and discovered that the right answer for the top query came in at number 47 on the search results. For the top keywords in particular, performing manual searches yourself is a valuable process.

3. Aligning the search functionality

You will occasionally hear site operators complaining that customers use the “wrong” keywords in their searches. This is an unwarranted complaint, because organisations should speak the same language as their customers in the first place, and they should be taking the opportunity to learn their customers’ terminology from the search keywords.

There are two other things to consider when analysing keywords. First, some of them are seasonal and can expire at any given time – for example, if a popular event has drawn to a close. Second, you should produce metatags from the top keywords for search engine registrations. It is very likely that other people will use the same keywords to search for your site.

Conclusion

If the path analysis shows that a lot of people leave the site after they view the search results, you must investigate why. On the one hand, it might mean that your search results show sufficient information to meet your visitor’s needs. Conversely, it might indicate that the results are not good enough to entice the user to drill deeper in the site. In the second situation, it could be either that the information returned does not accurately convey the meaning and value of the content kept on the site, or that the content itself is unsatisfactory to the visitors and does not encourage them to look any further.

If the search cannot return any results, you might consider recommending alternatives – which might help you to retain the user. These could include:

• An advanced search functionality to try again
• Displaying an alternative navigation mechanism (such as the site map)
• Asking the user’s permission to email him or her later when the site has developed content on the search keyword.

Search functionality is significant for many websites. If you are the operator of one of these sites, your can choose to build it in-house, or you can use a commercially developed program. Regardless of your choice, you should ensure that the activities performed on the search functionality are measured for effectiveness.

This is an excerpt from Measuring the Success of Your Website.

To start with, two examples are provided below to give you a sense of the sort of discoveries that you should expect to make as a result of search keyword analysis.

On an Australian Federal Government website, we have discovered that one of the most frequently searched terms was not even in the Department’s portfolio, hence the website offered no relevant content. Many visitors, faced with unsuccessful search results, would send emails to the customer service department, who would have to deal with irrelevant enquiries. Our solution was simple – a new page informing the user that this Department does not deal with this matter, and providing a link to the correct department.

On a major professional services firm’s website, the analysis revealed that a great majority of people searched for office contact details. The fix was simple: include office phone numbers on the page footers.

Without the aid of a commercial product, I perform the search keyword analysis by uploading the search log files into a database (such as MS Access).

For analysis to be effective, collect search log files for at least a week, preferably for longer. Most commercial search engine products such as ISYS, Verity K2, Ultraseek and Autonomy produce log files in a similar fashion to web servers. They log the user’s IP address, timestamp of the search, search term used and the number of hits in search results.

After uploading the search log files into a table created in your choice of database with the same field definitions, you can analyse them in the following ways.

Analysis of top search terms
Using the query capabilities of your database, create a report that lists the top 50 terms along with the number of hits in their search results. Although it is advisable to go through each one of these terms by repeating the search by yourself to see how satisfied you are with the results, you should focus on very high and very low hit search terms first. Very low hit search terms may indicate shortage or even the lack of relevant content, while very high hit search terms may bring far too many search results than what the user desires.

Analysis of lowest hit search terms
To pick up the low hit search terms that were frequently used but missed by the top search terms analysis, query your database table by grouping search terms in order of occurrence and sorting it to identify the lowest 20 hit search terms out of a larger group of top search terms (say, 200). Again, go through these search terms individually – checking what your visitors are looking for and how well your website responds.

You will notice that many of the terms are OK, but some terms will need an improvement on their search results. Various problems may present themselves here. For example, an important page that should be displayed is not picked up by your search engine due to the lack of or infrequent of use of the search keyword on the page content.

Single keyword analysis
You will notice that many of terms used to search your website are composite words or phrases. With the reports discussed above, we looked at how well your search engine responded to exact terms. And these analyses may be skewed due to slight differences in terms. So it may be useful to decompose search terms into single words and analyse frequently occurring words. To decompose the search terms, I simply upload the log files into a temporary table with space as a field delimiter for the search terms – eliminating the need for writing a script program to do this.

Search context analysis
For more advanced analysis, you may want to understand the context of the searches through classification of search terms. This analysis would reveal the concentration of searches by relevant segments of the website and may assist you in determining where to focus your new content development. Considering the size of log files, it could be a major undertaking to map each search term to a context area. But remember, website analysis is not an exact science. A simple approach with a relatively high degree of accuracy would suffice in most instances.

Going back to the classification discussion, visually examine the search terms to see if you can spot frequent occurrence of keywords used, create a new field in your table for context, and make global updates to your database table. For instance, words such as careers, hr, human resources, employment, positions, vacancies, appointments, etc can all be grouped under employment.

Analysis of search term errors
As part of the classification exercise described in Search Context Analysis, it is also advisable to classify errors so that you can device techniques to minimise them. The nature of errors might be various but common types are hitting the search button without entering a keyword, misspellings, mistakes in using notations such as + and -. After examination of frequently occurring errors you may figure out some ways of dealing with them. For instance, modifying the search help text, associating frequently misspelled words to their content items, etc.

Analysis of search terms by user segments
Another analysis may involve examining search terms by major user segments. For instance, the searches conducted by internal users, business partners, major customers due to their familiarity with your services and website may differ significantly from that of a prospect, a job seeker or a researcher. Based on their IP addresses, well known user segments might be easily isolated to conduct this analysis. Service priorities placed on user segments may make you favour certain groups – for example, customers or business partners – and you may want to optimise your search engine performance for these segments first.

The effort pays off

As you can see, search term analysis requires a reasonable amount of effort. The first three techniques can be performed relatively quickly, but you should allow more time for the last three. In most cases, a few days would be enough for a comprehensive analysis.

The frequency with which you repeat the analysis really depends on the outcomes of your first analysis and the role your website plays in your business. If you discovered shortcomings and actioned them, you should repeat the analysis to determine how effective the changes were. Although for most businesses a bi-monthly or quarterly analysis would suffice, you should do it more frequently if you are interested in trends in search usage, if your website is an integral part of your business, or if it integrates with your core processes such as sales and fulfilment.

Considering a large proportion of online users resort to keyword search at some stage during their visits, without search keyword analysis, your website analysis would be incomplete. Findings from search keyword analysis will help you improve the online customer experience.