Categories
KPIs Web Analytics

Measuring content effectiveness

Recently I began work on a small analytics project. I wanted to establish an a KPI measurement of our site content effectiveness. The plan was to develop a scorecard that could be leveraged on a regular basis by our communications staff (the content writers) to help them determine what pages and content needed to be fixed. I wanted to be able to use this scorecard to set priorities without having to do a ton of additional analysis (something like a number that yelled ‘fix me’ at a glance) and wanted to be able to monitor pages over time to see if the changes being made were in fact resolving problems.

Robbin at LunaMetrics agreed to help (and was a fantastic sounding board throughout this project).

We began by looking at a number of options for measurement. A number of metrics and KPIs came to mind: Ratio page views to visits; Exit ratio; Bounce rate and Percent one page visits. Here’s an explanation of each of the KPIs I’ve mentioned in more detail:

Ratio page views to visits
Formula: page views of a particular page / visits to the same page
A high ratio, especially on deep pages, seemed to indicate that the visitor was stuck and using the back button.

Exit ratio
Formula: exit visits / visits to the same page
While we might have a high number of visits for a page (meaning that we were getting the traffic there), this KPI would highlight which pages were causing the user to leave when they couldn’t find the information they were looking for.

Bounce rate
Formula: single access visits / entry visits to the same page
A measure of how many people leave without viewing any other pages. A low number tells you that the page is effective in moving the user deeper into the site, a higher bounce rate, the less effective a page is at keeping the user engaged. (Note: HBX, unlike Google Analytics, doesn’t display a bounce rate metric like Google Analytics does, so you have to calculate this in Excel).

Percent one page visits
Formula: single access visits / visits to the same page
Jim Novo noted this one in a Google Conversion University article. It’s the percentage of visitors bouncing off the site (like ‘plexiglas’ says Jim). It’s and usually is tied to global navigation issues and should trend down over time as you make changes to the site or copy. Robbin took the Bounce Rate and Percent One Page Visits KPIs and charted them and they correlated nicely, so using either KPI would work for us.

Looking at each of this KPIs on a per page basis told us we had some problems. Each by themselves was partially effective, but missing bits of information. So we set out to create a KPI mashup. This became known as the Steif Mashup Content KPI. Here’s what we came up with:

Visits * Bounce Rate / Time Spent on Page in Seconds

We liked this. Visits were a good indicator of the traffic volume and page importance to users, Time Spent on Page was a good indicator of the engagement of the user with the page content when they got there and Bounce rate was a good indicator of whether the page itself was working or not.

  • A high visit page with a high bounce rate needed to be fixed ASAP. People were going here and we were losing them
  • A low visit page with a high bounce rate, could be attended to when priority allowed
  • A High visit page with a low bounce rate we could leave for now and come back to tweak later
  • A low visit page with a low bounce rate we’d likely not even look at.

Using HBX Report Builder I created an Excel worksheet that pulled in the page path and the metrics related to that page path. For those who have never done this using Report Builder, here’s what you need to do.

  • For the page path select ‘Most Requested Pages’ and bring in the page name and page path. If you set a filter you can bring in only pages that meet certain criteria (e.g. I used this to bring in only pages for the ‘Banking’ section of our site – which you can set either within the Report Builder interface or reference to a cell in your Excel spreadsheet)
  • Adding dependent requests off the page name, I grabbed each of the other values (visits, single access, entries, exits and time spent on page) and added them to their own columns.
  • Since HBX reports time spent on page as D:HH:MM:SS you need to break this apart to get a value for the time overall in seconds. That’s easy using the MID function in Excel (cell G3 in the example is the time spent on page cell):
    (MID(G3,6,2)*60)+(MID(G3,9,2)

Then doing the calculations for each KPI was just a matter of referencing the data pulled down into Excel.

Problem was that this still wasn’t giving me an “at-a-glance” view of what pages were working or not. For example for a high traffic sick page, the resulting number was higher than that of a low traffic sick page, even if the bounce rate was the same for both pages. So setting priority at a glance was difficult. So we tried changing the mashup KPI taking out the ‘visits’ and ended up with this:

Bounce Rate / Time Spent on Page in Seconds

This was starting to look better. Now if a page had a higher “score” the more this page needed attention.

  • A high bounce rate page with high read time could mean that the content was missing some information the visitor was looking for
  • A low bounce rate page with a high read time meant that the content was likely working.
  • A high bounce rate page with a low read time meant we were losing the visitor quickly. Content wasn’t working or providing the necessary information.
  • A low bounce rate page with a low read time wasn’t getting used much, but was meeting the needs when it was accessed.

Now with this new KPI we’re starting to prioritize content changes that need to be made. As an overlay on this we’ve also segmented these KPIs looking at how the values change based on the traffic source, new vs. returning visitor, member vs. non-member, etc. Where we have pages with lite traffic volumes, we can extend it out to cover a longer period (e.g. last quarter instead of the last month).

So what do you think? Have we tackled this the right way? How do you measure your site’s content effectiveness?

Categories
KPIs Search Analytics

Are you watching the ‘broken’ parts of your site?

In web analytics, we all can get a bit too focussed on how our/the site is doing in relation to the KPIs that you measure against. But are you watching the ‘broken’ parts of your site?

I do, and here’s some suggestions on things to look for:

Errors

Most analytics programs gather error reports. They typically they are configured to report out on 404 errors (a page not found) and list the URL or page name requested and often the referring link/URL where the request originated. You can track other error types easily as well. For example, in HBX, all you need to do is pass a different value to the hbx.mlc variable in the page error code (e.g. /error/404 or /error/500) to distinguish between the types. You could even write code to pass custom variables such as the time the error occured.

You can also eliminate a lot of these errors by running a link check on your site occassionally. The free W3C link checker is an option and there are also many tools on the market that do the same or better.

Failed Keywords

Looking at your ‘failed keyword’ reports (keywords/phrases typed in your site search engine that did not return a result) can be quite interesting. As Hurol Inan writes in Search Analytics – A Guide to Analyzing and Optimizing Website Search Engines “the nature of keyword errors might be diverse, but common types include hitting the search button without entering a keyword, misspellings and mistakes in using notations such as + and -” and “examining frequently occurring errors, [can help you] figure out ways of dealing with them; for instance, modifying the search help text or associating frequently misspelled words to their content items etc.”

Indexing of error pages

Another worthwhile option is to look at your favourite search engine and see how many error pages are being indexed. if you know how you title your error pages, you can do this using the allintitle: and site: search options. For example, looking at WebSideStory.com we can see that they have 174 ‘404 error pages’ listed in Google. No reason for that, and easy to clean up. Saves page rank headaches as well.

Funnels

Tracking funnels and forms can not only help highlight abandonment points but may also flag server-side issues causing users to not complete. Take a bit of time too to ensure that your forms are being tracked correctly. For example, a couple weeks ago I noticed a discrepancy between the actual number of form submissions and those being tracked in my analytics tool. A bit of investigation and problem was resolved.

What are some of the ‘broken’ parts you are watching? What are you doing to make your site experience the best it can be?

Categories
KPIs Search Analytics Web Analytics

Learn more with Hurol Inan

I just picked up Hurol Inan’s two books Measuring the Success of your Website and Search Analytics. Hurol offers them in either PDF or print versions and has a sweet deal if you buy both online. I’ve been a big Hurol fan for sometime now and these books just reinforce the fact that he is one of the great contributors to a better web.

Categories
KPIs

Functionalism

Semphonic has published an interesting whitepaper on web analysis from a functionalist perspective. You can download the white paper here in PDF format. I haven’t had a ton of time to drill through it but I like the concepts and it is a unique approach with some definite value.

Their functionlist approach “breaks up a website into pieces then assigns one or more functions to each piece — then each piece is measured using KPIs. Their are four steps: 1) Classification; 2) Measurement Protocol; 3) Adaptation; 4) Communication. They provide a few examples of some of the KPIs they use with this approach in the whitepaper.

Eric Peterson covers off a similar, though different method in Web Analytics Demystified where you group content by business objective and then leverage these groupings for deeper analysis. Both approaches have their own value.

Like many of these advanced types of analysis, expect to spend some time with an analyst determining which pages fall into which buckets and then doing a bunch of page configuration or report building so that you’re able to capture data points at this level.

For me, something blending functionalist and the Eisenberg’s Persuasive Architecture would be ideal. Who knows, maybe we’ll end up there.

p.s. When I took my initial glance through it I kept being reminded of the book Submit Now: Designing Persuasive Websites by Andrew Chak. He uses a similar concept in explaining effective ways to construct a site and that there are various types of users that you need to support when building sites, making content, etc. If you haven’t read the book, check it out.

Categories
KPIs

Mini KPI scorecards (a.k.a adhoc reports)

I’m re-tooling my scorecard to better monitor both week-over-week and month-over-month. I’ve been using a scorecard for the past 10 months since moving to HBX. The current scorecard was working fine but it was time for some tweaks given our new strategy rollout.

Recently we made some site changes to IA, modified our “pillar” pages (banking, loans and credit, investments and insurance) to see if we could get visitor to get into the depths of the site more efficiently and introduced a new tabbed page model to make it easier to browse and sort complex product categories. All of these changes were driven entirely by analysis of our web metrics rather than sheer intuition and have been measured since launch to monitor effectiveness. At a aggregate level I can tell you the changes have been well received.

What I have started is creating separate scorecards to monitor specific site changes. This makes it easier to share externally with our design agency, team, etc. so they can see the effects of certain design changes. What’s great about this is that it allows you to specifically track your changes and introduce other KPIs that you may not be monitoring normally in your main KPI scorecard.

One of my favorite KPIs is stickiness. As Eric Peterson points out in The Big Book of KPIs, stickiness is “one of the most important marketing KPIs” and is a great indicator of the liklihood that your pages will keep people on the site. One of the things I like about this KPI is that it is made from easily accessible data, even easier to explain when you show the results to someone and simple to monitor over any period of time.

For those not aware of this KPI, the stickiness of a page is measured as:

1.00 – (single access page views of the page / entry page views of the same page)

If you can’t grab page views, visits will do as long as you maintain a one-to-one relationship and use visits throughout (thanks to Eric for confirming this on his web analytics group on Yahoo!).

The closer to one, the stickier the page is. You can also treat the ratio as a percentage — the percentage that the user will at least see one more page. Layer on top some segmentation and you can look at the conversion rate for particular segments traversing each page, referrers and their affect on stickiness (which could also point to traffic quality).

Using WebSideStory’s HBX Excel plug-in, Report Builder, you can quickly and easily build out a report and gather the necessary data without having to spend time in various interfaces grabbing numbers.

I’d be interested in hearing how you measure and monitor your site success/failure and leverage for optimization.