Best practice KPI reporting for the online channel
by Lars Ammitzboell - Head of Data and Analytics, Bienalto Consulting
With so much data on hand, the online channel is still struggling to figure out an effective way to report its performance. Many organisations are now exploring Key Performance Indicators (KPIs) for reporting due to their simplicity, widespread recognition and powerful communication.
A good KPI should be easy to understand, states the current level of performance, sets direction and prompts action.
However, while reporting on raw data - such as number of visitors, page impressions, number of exits per page and so forth - might be useful for operational purposes, it will not impact strategic decision making.
A major concern for web analytics professionals should be how best to use the data to impact decision-making, and then how to make sure the data is easily understood throughout the organisation.
KPI reports for the online channel should embed the following best practices to improve relevance and enable better decision making:
- The data the reports are based on must be accurate, and free of bias. KPI reports that rely on inaccurate data sets are not only invalid but also dangerous as they may cause the wrong decisions to be made.
- KPIs need to be tailored to the objectives of the audience group, and be industry specific.
- Data captured via the online channel is rarely enough to deliver the information required by management. Web data should be amalgamated with customer data.
- Web data requires timely analysis and interpretation - data is captured instantly and reports must support immediate action.
Each of these best practices is described below.
Accurate data and free of bias
It should go without saying that KPI reports cannot be based on inaccurate or biased data, but it is not uncommon to see that campaign tracking has been setup incorrectly and results are being misinterpreted.
Similarly, marketers often only report on high level results by campaign and channel, but are not able to explain how the responses of one channel (eg online) were influenced by another (eg DM).
Common lapses occur due to:
- Inaccurate tracking
- Overlap between channels (online, DM, email, above the line)
- Overlap between ads on multiple sites
- Not knowing which individual components of a campaign worked (price, text, headline, colour, placement, etc)
- Uplift provided by paid search vs natural search
Most inaccurate campaign data can be avoided by designing campaigns up front using techniques such as Experimental Design or at least A/B testing; then analysing the results using multivariate analysis. Without this, campaign results often just rely on high level reporting and estimation.
Yet, according to Bienalto’s Australian Web Analytics Survey 2007, very few companies or agencies conduct multivariate campaign analysis.
Tailored KPIs to match specific objectives
KPIs need to be defined specifically for the job role of the receiver - for instance, executives would require different KPIs than operational personnel.
The crucial first step is to define your reporting audiences. These may be senior management, but are also likely to include marketing, sales, finance and product managers. Online managers also need the right reports to function in their role.
Next comes the identification of what each one of these reporting audiences needs to know, what the information will be used for and how often it is required.
This has to be considered in the context of the specific business objectives for each audience group - what information does the audience need in terms of reaching their business objective.
Remember, the information requirements of management are rarely met by generic reports. It is, for example, little use for a bank’s Retention Manager to know how many customers use online banking. But it will be a relevant KPI to track the percentage of accounts that have active Bill Pay.
Similarly, for a Credit Card Product Manager it is useful to know what the cost per response was for the most recent online campaign, but this KPI needs to be tracked alongside KPIs for approval rate, churn rate, and profitability per card.
This links to the requirement of reporting on online data as well as offline data.
Encompassing offline and online data
There can be a disconnect between website data and the customer data in CRM systems. By keeping website data separate to offline data, you get a pile of tactical reports that are difficult to interpret in terms of impact on business.
The online channel should be integrated into the overall marketing strategy. A company might, for example, email customers who have undertaken certain actions on a website, or compare the web data to transactional data.
Typically we differentiate between three types of data:
- Behavioural - the user’s behaviour on the web, described via click-stream analysis
- Profile data - typically demographic or descriptive data supplied by the user and collected via log-in or surveys, but can also be third party data appended at segment level
- Transactional data - offline transactional data eg banking data, insurance data, purchase history, etc.
Some companies only report on behavioural web data in their KPI reports. Most companies should be able to append some degree of profiling to the web behavioural data. Few companies are able to overlay their transactional data to their web data.
Web analytics is linked with customer intelligence. Particularly with the current focus on event based marketing (EBM), it’s more important to use online and offline data within a data mining environment. This will identify triggers or changes in behaviour that suggest a certain demand. Analytics will determine the best offer which can then be communicated to prospects and customers through both the online and offline channels.
Best practice suggests an integrated multi-channel approach by which the same offer is communicated to the customer consistently across multiple channels.
A multi-channel approach means marketers use different campaign channels in combination and are aware of their impacts on each other. For example: What is the overlap between those being targeted for DM and email and those that respond via web? What measurable lift does a DM campaign have on leads captured via the web? Did the DM campaign impact the behaviour of the high demand segments, or would they have responded via the web anyway?
Timely analysis, interpretation and action
By its very nature, the web channel allow for data to be captured instantly and actions to be applied immediately. It’s surprising that many marketers only see high level reports on a weekly or fortnightly basis. Online ads, search engine marketing (SEM), email campaigns and online user behaviour can all be tracked instantly, and actions can quickly be made to improve results.
For instance, a KPI might indicate that the click-through rate for an email campaign is down by 20 per cent compared to other campaigns that are tested in parallel. Using multivariate analysis, the analyst should be able to explain what components are not working for this particular campaign, and changes can be made immediately. No good reporting a week later that the campaign did not work!
Analysis should be done early and throughout - not at the end. Exposing the customers to a campaign and posthumously finding out that it did not work is not good enough.
Similar rule applies to site traffic. A small design change to a site might have a detrimental effect to that site’s conversion rate. These things need to be picked up fast or they can be very costly.
KPIs should also avoid simply reporting on “What is happening” without analysing “Why”. It is a fair assumption that a senior manager will ask that question.
Conclusion
KPI reports translate analytical data into metrics required to track performance against business objectives.
Most companies are aware of the importance of using KPIs to track their online performance. Many, though, still rely on generic high-level reporting not supported by analysis. These companies run the risk of not knowing the reason for changes in KPIs - or more seriously - their KPI reports might be telling the wrong story.
The need to set KPIs that communicate information in the context of specific business objectives and that are aimed at specific audience groups cannot be overstated.
Good KPI reports should show how marketing is impacting the bottom line, what is and what is not working, and they should be presented in a manner that supports timely decision making.