KPIs Web Analytics

Measuring content effectiveness

Recently I began work on a small analytics project. I wanted to establish an a KPI measurement of our site content effectiveness. The plan was to develop a scorecard that could be leveraged on a regular basis by our communications staff (the content writers) to help them determine what pages and content needed to be fixed. […]

Recently I began work on a small analytics project. I wanted to establish an a KPI measurement of our site content effectiveness. The plan was to develop a scorecard that could be leveraged on a regular basis by our communications staff (the content writers) to help them determine what pages and content needed to be fixed. I wanted to be able to use this scorecard to set priorities without having to do a ton of additional analysis (something like a number that yelled ‘fix me’ at a glance) and wanted to be able to monitor pages over time to see if the changes being made were in fact resolving problems.

Robbin at LunaMetrics agreed to help (and was a fantastic sounding board throughout this project).

We began by looking at a number of options for measurement. A number of metrics and KPIs came to mind: Ratio page views to visits; Exit ratio; Bounce rate and Percent one page visits. Here’s an explanation of each of the KPIs I’ve mentioned in more detail:

Ratio page views to visits
Formula: page views of a particular page / visits to the same page
A high ratio, especially on deep pages, seemed to indicate that the visitor was stuck and using the back button.

Exit ratio
Formula: exit visits / visits to the same page
While we might have a high number of visits for a page (meaning that we were getting the traffic there), this KPI would highlight which pages were causing the user to leave when they couldn’t find the information they were looking for.

Bounce rate
Formula: single access visits / entry visits to the same page
A measure of how many people leave without viewing any other pages. A low number tells you that the page is effective in moving the user deeper into the site, a higher bounce rate, the less effective a page is at keeping the user engaged. (Note: HBX, unlike Google Analytics, doesn’t display a bounce rate metric like Google Analytics does, so you have to calculate this in Excel).

Percent one page visits
Formula: single access visits / visits to the same page
Jim Novo noted this one in a Google Conversion University article. It’s the percentage of visitors bouncing off the site (like ‘plexiglas’ says Jim). It’s and usually is tied to global navigation issues and should trend down over time as you make changes to the site or copy. Robbin took the Bounce Rate and Percent One Page Visits KPIs and charted them and they correlated nicely, so using either KPI would work for us.

Looking at each of this KPIs on a per page basis told us we had some problems. Each by themselves was partially effective, but missing bits of information. So we set out to create a KPI mashup. This became known as the Steif Mashup Content KPI. Here’s what we came up with:

Visits * Bounce Rate / Time Spent on Page in Seconds

We liked this. Visits were a good indicator of the traffic volume and page importance to users, Time Spent on Page was a good indicator of the engagement of the user with the page content when they got there and Bounce rate was a good indicator of whether the page itself was working or not.

  • A high visit page with a high bounce rate needed to be fixed ASAP. People were going here and we were losing them
  • A low visit page with a high bounce rate, could be attended to when priority allowed
  • A High visit page with a low bounce rate we could leave for now and come back to tweak later
  • A low visit page with a low bounce rate we’d likely not even look at.

Using HBX Report Builder I created an Excel worksheet that pulled in the page path and the metrics related to that page path. For those who have never done this using Report Builder, here’s what you need to do.

  • For the page path select ‘Most Requested Pages’ and bring in the page name and page path. If you set a filter you can bring in only pages that meet certain criteria (e.g. I used this to bring in only pages for the ‘Banking’ section of our site – which you can set either within the Report Builder interface or reference to a cell in your Excel spreadsheet)
  • Adding dependent requests off the page name, I grabbed each of the other values (visits, single access, entries, exits and time spent on page) and added them to their own columns.
  • Since HBX reports time spent on page as D:HH:MM:SS you need to break this apart to get a value for the time overall in seconds. That’s easy using the MID function in Excel (cell G3 in the example is the time spent on page cell):

Then doing the calculations for each KPI was just a matter of referencing the data pulled down into Excel.

Problem was that this still wasn’t giving me an “at-a-glance” view of what pages were working or not. For example for a high traffic sick page, the resulting number was higher than that of a low traffic sick page, even if the bounce rate was the same for both pages. So setting priority at a glance was difficult. So we tried changing the mashup KPI taking out the ‘visits’ and ended up with this:

Bounce Rate / Time Spent on Page in Seconds

This was starting to look better. Now if a page had a higher “score” the more this page needed attention.

  • A high bounce rate page with high read time could mean that the content was missing some information the visitor was looking for
  • A low bounce rate page with a high read time meant that the content was likely working.
  • A high bounce rate page with a low read time meant we were losing the visitor quickly. Content wasn’t working or providing the necessary information.
  • A low bounce rate page with a low read time wasn’t getting used much, but was meeting the needs when it was accessed.

Now with this new KPI we’re starting to prioritize content changes that need to be made. As an overlay on this we’ve also segmented these KPIs looking at how the values change based on the traffic source, new vs. returning visitor, member vs. non-member, etc. Where we have pages with lite traffic volumes, we can extend it out to cover a longer period (e.g. last quarter instead of the last month).

So what do you think? Have we tackled this the right way? How do you measure your site’s content effectiveness?

5 replies on “Measuring content effectiveness”

FWIW? My at a glance view for content pages is from Eric’s books of Hacks: #58. Specifically: “ Ratio of page entries to exits”.

Set with a simple Red/Green flipper. Pages = “1.0” Green. Can obviously put an “Orange” (1.0 -> 1.3 for example) in there if you need a maybe.

If’s but’s and maybe’s around the use of this one.
Even so, I find it extremely useful to get a quick glace at what is vs isn’t working. Been extremely accurate for me – both personally and @ work. Both being content sites.
Even works very nicely on pages that I regard as “one hit wonders”. ie If you see nothing else, see this one page. 1:1.

Works very effectively if you have Page A as a landing page and generic page B as the obvious exit.

“Bounce Rate / Time Spent on Page in Seconds”
I’d need to see some numbers first as a qualifier – but wouldn’t this be better off inverted (1/##)?
Why? We (as people 🙂 ) tend to associate high numbers with Good Things(tm), not bad – so playing more to the obvious psychology?
You may find some sort of inbuilt fudge factor useful to smooth numbers out to account for quick pages to read vs slow pages.

My gut feel tho, any metric dependent on “Time on Page” has an inbuilt (IMHO fatally flawed) assumption that people are *actually* looking at the page.
I’ve worked for many years where I will open 5-10 pages off search results and gradually work my way through them. Possibly the last one may have been “open” for 10-20 minutes, or longer, before I actually read anything off it.
I’d argue this behaviour is more prevalent on content sites vs shopping style. Happy to be proven wrong.
And I could be weird. It’s been noted before. 😉

“Exit Ratio”?
You could also suggest that an end user found exactly what they were looking for and didn’t need to go any further. ie 1:1 again. That could be a sign of perfect success. Perhaps not for “your” site, but could be for “mine”; if you ken.


– Steve

Thanks for taking the time to comment Steve.

I like the suggestion from Eric’s Hacks — forgot about that one. Will have to take a look at it again. Have you found it to work well regardless of the traffic level of the page? Does the effectiveness change if trended over longer periods of time?

Either methods work for me for the Bounce Rate / Time Spent on Pages. Not sure the viewing of pages over time long periods of time is all that common on the site I’m measuring, but worth deeper eval. Doubtful most users are viewing this way. Both Jim Novo and Brian Eisenberg would say that high time visits are typically reflective of a more engaged user. That was our thinking in including it.

Agree with Exit Ratio. You can easily end up with a one-to-one. We had examples like this with our RRSP content. We ranked high in search engines for the information. People came to our site. Got the answer and left. Not a problem with the page really from a content perspective but it was from an inability to move the reader towards some further engagement or conversion.

Thanks again,

– Scott

Have you found it to work well regardless of the traffic level of the page?

Strictly? No. Generally? Yes.
Why “No”? 🙂 If & When Exits == 0. Division by Zero is a no no. 🙂
I fudge around that by finding the max “ratio” from all other ratios from all pages and substituting that instead. Yeah it’s a nasty fudgy hack…
As with any metric – really low numbers can throw things screwy. I find > 30 entries is probably the useful limit. Any less is too “noisy”. Big numbers are no problem – tho you can get an averaging effect which swamps detail.
I also base this off the “Entry Pages” so entry == 0 is not an issue.

I haven’t (yet) tried trending it over a period of time. Be interesting to see if it changed anything…

I suspect it would be wise to segment before generating the ratio, to really drill down, but for general KPI work is probably a Good Start(tm).

High Time visits? I’d agree with the sentiment. But is an “engaged user” an agreeable goal? On work’s site – no. Or rather, not really. Get ’em in, and show them the right exit for their needs asap. Anything more is wasting their time.
If we can solve their need in two pages that’s perfect; Any more suggests we’ve failed somewhere.
Slight hyperbole. Well a lot actually, but you get the idea. 🙂
Ideally we’d solve their need in one page, but legals won’t let us….

Ditto on my personal site to a large degree – single page – no engagement wanted or even desirable. I would rather see greater engagement, via discussion, on the mailing list I host, than the website.
Which leads me to (re) think that I probably need to do more to make the list more obvious. Thanks! 🙂

– Steve

my first reaction is that using bounce rate and time spent are more measures of page effectiveness in totality as opposed to content effectiveness…I think it could be more useful if combined with some other measures of content utilization such as a weighted average of clicks. Something that incorporates total possible clicks versus actual clicks. That way your looking at two distinct groups, those who bounced as well as those who used the content and assessing them together.

Benry, I for one applaud your efforts in development of a customized KPI. It says to me that you are passionate about your work and have talent to burn. I think bounce rate can mean many things depending on where it is, where they came from and the offering of interest before them.

It can be difficult to contextualize an idea as dynamic as traffic analysis and KPI’s.


Comments are closed.