Categories
KPIs

Functionalism

Semphonic has published an interesting whitepaper on web analysis from a functionalist perspective. You can download the white paper here in PDF format. I haven’t had a ton of time to drill through it but I like the concepts and it is a unique approach with some definite value.

Their functionlist approach “breaks up a website into pieces then assigns one or more functions to each piece — then each piece is measured using KPIs. Their are four steps: 1) Classification; 2) Measurement Protocol; 3) Adaptation; 4) Communication. They provide a few examples of some of the KPIs they use with this approach in the whitepaper.

Eric Peterson covers off a similar, though different method in Web Analytics Demystified where you group content by business objective and then leverage these groupings for deeper analysis. Both approaches have their own value.

Like many of these advanced types of analysis, expect to spend some time with an analyst determining which pages fall into which buckets and then doing a bunch of page configuration or report building so that you’re able to capture data points at this level.

For me, something blending functionalist and the Eisenberg’s Persuasive Architecture would be ideal. Who knows, maybe we’ll end up there.

p.s. When I took my initial glance through it I kept being reminded of the book Submit Now: Designing Persuasive Websites by Andrew Chak. He uses a similar concept in explaining effective ways to construct a site and that there are various types of users that you need to support when building sites, making content, etc. If you haven’t read the book, check it out.

Categories
Analytics Vendors

Choosing your analytics vendor

Everyone always asks me who we use and why we chose WebSideStory (WSS) over other vendors.

My answer is simple. We chose WSS over other vendors because they took the time and made the effort to let us see, use and understand their product. How innovative! (note: some other vendors did the same to varying degrees, but the vast majority didn’t).

Needs assessment

When we started evalutating tools, we first did a needs assessment. Look at what metrics you might need now to support your organizational strategies, business unit needs, scalability, etc. What do you want out of this? What does your boss want out of it? What other clients do they have? What clients are in your vertical or horizontal? Who in your local area is using their tool?

Connect with potential vendors

Try to find tools that can meet the needs you’ve defined. Contact vendors and ask them to demo their product. This will be your first indication of the type of company your going to be working with and how they respond to their customers. A particular peeve of mine with the enterprise market is that many have forgotten that all clients are important. Coremetrics, Omniture both couldn’t be bothered to followup on calls. Fireclick, WebTrends and WSS did (funny, but about 6 months after moving to HBX all the guys that didn’t call, started making calls asking if we’d be interested in evaluating their tools. Good luck with that sales tactic).

Test the software

Get the vendor to provide a no-conditions trial period, with a live account pulling in stats using your actual site and their page code. Get the manuals for installation and configuration so you can go deep and truly understand what will be required to configure. Do they support first or third party cookies? Shoot for at least 30 to 60 days to see how their tool reported, how page code worked and what effort ‘tagging’ pages you needed to go through to get the metrics we were seeking.

Get a price estimate

Remarkably many vendors won’t provide this without a deep committment. If they can’t, say thank-you for their time. If they can, don’t just take the first price they provide. Negotiate. Many have wiggle room to adjust their pricing, and provide payment plans that can meet your budget. When pushed or when facing the idea of losing a client many suddenly come to bat.

Be sure to check costs carefully. Most vendors price on a per page view basis. What and how you’re tracking can greatly influence this figure. Think tracking units and take into consideration the items you’re planning on measuring (flash, pop-ups, frames, etc.).

Think about your needs now and later. Will you need segmentation? Do you need a method to create reports outside of the tool (such as HBX’s Report Builder plug-in for Excel). What type of support does the vendor offer? Some provide fantastic support during a setup period (30 – 60 days), then push you over to a support desk for any questions after that — others provide one-to-one account management with people who will get to know your site and your business goals.

Find out what else is included

HBX had all kinds of things we hadn’t even considered — like report builder, the ability to tie into our data warehouse, consulting services, other tools like ATOMZ/WSS Search, their digital marketing university, etc. By taking a few services at once you might be able to get a price break as a package.

That said, here’s what the first 8 months on HBX have been like:

  • Out of the gate they assigned us to a Account Manager and Implementation Engineer. These become our go to people when we have questions. They also make sure we have everything setup correctly.
  • They provide in-depth training on their product to make sure we understood how to use the tools effectively. Free webinars over WebEx that you can attend as frequently as you like.
  • Digital Marketing University was a great information source and a chance to meet other folks and some of the pros at WSS.
  • They continue to be available and take the time to address our questions.

It hasn’t been all perfect though:

  • They assigned us a new account rep and implementation engineer who we had to get back up to speed on our site and business goals
  • They sometimes don’t take the time to dig deep enough when we’re probing for answers to questions
  • They haven’t formalized their model with their other business lines like Search, so we have to deal with different people over there. Sometimes people who don’t know a lot about HBX.

That’s my journey. Hope your selection goes well.

Categories
KPIs

Mini KPI scorecards (a.k.a adhoc reports)

I’m re-tooling my scorecard to better monitor both week-over-week and month-over-month. I’ve been using a scorecard for the past 10 months since moving to HBX. The current scorecard was working fine but it was time for some tweaks given our new strategy rollout.

Recently we made some site changes to IA, modified our “pillar” pages (banking, loans and credit, investments and insurance) to see if we could get visitor to get into the depths of the site more efficiently and introduced a new tabbed page model to make it easier to browse and sort complex product categories. All of these changes were driven entirely by analysis of our web metrics rather than sheer intuition and have been measured since launch to monitor effectiveness. At a aggregate level I can tell you the changes have been well received.

What I have started is creating separate scorecards to monitor specific site changes. This makes it easier to share externally with our design agency, team, etc. so they can see the effects of certain design changes. What’s great about this is that it allows you to specifically track your changes and introduce other KPIs that you may not be monitoring normally in your main KPI scorecard.

One of my favorite KPIs is stickiness. As Eric Peterson points out in The Big Book of KPIs, stickiness is “one of the most important marketing KPIs” and is a great indicator of the liklihood that your pages will keep people on the site. One of the things I like about this KPI is that it is made from easily accessible data, even easier to explain when you show the results to someone and simple to monitor over any period of time.

For those not aware of this KPI, the stickiness of a page is measured as:

1.00 – (single access page views of the page / entry page views of the same page)

If you can’t grab page views, visits will do as long as you maintain a one-to-one relationship and use visits throughout (thanks to Eric for confirming this on his web analytics group on Yahoo!).

The closer to one, the stickier the page is. You can also treat the ratio as a percentage — the percentage that the user will at least see one more page. Layer on top some segmentation and you can look at the conversion rate for particular segments traversing each page, referrers and their affect on stickiness (which could also point to traffic quality).

Using WebSideStory’s HBX Excel plug-in, Report Builder, you can quickly and easily build out a report and gather the necessary data without having to spend time in various interfaces grabbing numbers.

I’d be interested in hearing how you measure and monitor your site success/failure and leverage for optimization.

Categories
Search Analytics

Scent from internal search terms

In the the case of our site the vast majority of searches do come after people visit lower level pages in the site. What we can infer is that many users are trying to find what they are looking for by clicking on categories and links and that only when the scent dries up they turn to search. When they do turn to search, the queries are often the trigger words that they were seeking in the page content — in other words they are using search to create their own links because the information or links on the page failed them.

What I’ve done is configured my HBX page code used on our internal search template so that it populates one of the custom search variables (you can have up to 4 custom variables in HBX for Internal Search) so that it captures the referring URL. Then I created a Report Builder report in Excel that captures the:

  • The keyword
  • Referring URL for the keyword
  • The count

This report tells me what people are searching for, where the scent dried up and the number of times this is happening. Very helpful for trendspotting and for flagging content and IA changes required.

I also have another report that captures:

  • the Keyword
  • Link ID (the link they clicked)
  • Link Position

This report tells me how well my search results are matching to keywords typed. It’s usually pretty easy to see if they found the “correct” page or the page that will get them the information they were looking for.

I also have a third report that captures the Failed searches. Flags potential content gaps and also often provides some good laughs (you’d be surprised what people sometimes look for).

Note: I came across this idea at UIE Brain Sparks and have expanded on it somewhat. Not a new idea really — the Eisenberg’s have been talking about “scent” for ages.