Archive for 'Key Performance Indicators'

This morning the fine folks at Nedstat in Holland published a white paper that Michiel Berger and I co-wrote titled Measuring Multimedia Content in a Web 2.0 World.  This free white paper explores the emerging direct measurement model for multimedia content by examining several common business cases for deploying video and provides a new set of definitions and key performance indicators (KPIs) designed to help companies effectively track their investment in video based content.

The timing is somewhat ironic because Judah has been writing a fair amount about Video Analytics over in his blog — I guess great minds think alike!

While video measurement has been around for awhile, the new social media certainly increases the complexity associated with determining the efficacy of video from a business perspective.  The folks at Nedstat are committed to helping their customers resolve these issues, and are generously making our white paper available without registration requirements.

You can read the press release about the paper’s availability or download your own copy right away.

(Last week I published PART I of this post which you should read first if you haven’t already done so.)

STEP FOUR: DETERMINE YOUR KEY PERFORMANCE INDICATORS AND CRITICAL REPORTS

You’re probably thinking “shouldn’t we have done this after we defined our business objectives and activities?” Conventional wisdom would probably say you should, but in my experience if you don’t have a clear process for leveraging those key performance indicators (KPIs) and critical reports, you may end up with one of three things:

1. A huge report of 40 KPIs distributed across the organization that few people are likely to read and even fewer likely to act upon
2. No KPIs distributed at all, and the expectation that everyone will simply “log in” and get the information on their own
3. Well-defined and clearly articulated KPIs distributed hierarchically throughout the organization (because hey maybe you read a great book on the subject at some point)

The problem is that only the third possibility will deeply benefit your organization. I know that some people talk about hundreds of internal users who really get web analytics and all make superb decisions with the data, but this is very much the exception, not the rule. Remember, in our Web Analytics Demystified Spring Survey 69 percent of respondents said that they did not believe the majority of people using web analytics data in their organization actually understood that data.

It is far better for your analytics hub, as mandated by their executive sponsor in agreement with his or her peers throughout the organization, work directly with the individual spokes to ensure that appropriate KPIs are defined and the basis for those measures is clear. The hub then follows-up with appropriate explanation about the measures, including training on the reports and data that forms the basis of the indicators.

Your critical reports are directly tied to your key performance indicators (which remember are tied directly to your business objectives.) If you belong to the marketing organization than your KPIs will be measures like “Campaign Response Rate”, “Campaign Conversion Rate” and “Campaign Cost per Click”. Obviously as these KPIs change, appropriate tactical resources in the marketing spoke will review campaign response, conversion, and cost reports in your analytics application.

Your KPIs and critical reports will differ dramatically depending on what department you work for and where in that department you work — remember that the best practice for key performance indicator distribution is to deliver the specifically and hierarchically. Most attempts that I have seen to send “everything to everybody” have failed (often miserably).

STEP 5: DETERMINE HOW YOU’LL DELIVER ANALYSIS

Once you know what your KPIs and critical reports will look like, the next step is to determine how you’ll produce and deliver analysis. Let’s assume for a moment that you’ve got a hub-and-spoke model in place and the hub is receiving regular requests for more information, insights, and recommendations. The question then becomes “how will you deliver those insights and recommendations?”

As I said last week, there is no one “right” way to communicate about web analytics data but there are many, many wrong ways. The central challenge when delivering analysis stems from the fact that so few people really understand what web analytics terms mean, what the limitations of the technology are, and what is possible and impossible to report on. But it’s not like you can just give up and ignore the confusion, so what’s a great analyst to do?

The answer is “work harder, and think outside the box” (to use an overused term). While reports and raw data are best delivered using the Bottom Line Up Front (BLUF) method, analysis really needs to be more engaging. Remember: when you deliver analysis, what you really need to do is to convince the listeners that they need to take some action. To do this you absolutely have to be engaging.

Things that have worked for clients of mine in the past include:

• Well-delivered presentations, given IN PERSON, not just sent via email in hopes that people will review and understand
• Well-written documents, followed by a meeting to make sure that everyone READ the document and is on the same page
• Short summary documents, written up like a newsletter or newspaper article, designed to get people to attend a meeting or presentation

Since we’re in a Web 2.0 world, and since many of you are increasingly comfortable using new technology, a few other things you may want to consider include:

• An internal analysis Wiki that people can subscribe to and participate in. The Wiki is a good idea because it allows you to capture the conversation in a searchable format
• A regular analysis podcast, providing an update on past analysis and summarizing the data currently being reviewed
• A analysis video or vidcast, created with tools like TechSmith Camasis that allow you to easily blend images, live screen capture (useful when showing people live data in your analytics application), and annotation

The advantage the final two ideas confer is their ability to be downloaded to an MP3 player like the iPod or iPhone. If you have busy executives, you might be better able to reach them if you give them something to watch on the airplane or listen to on the drive home.

Keep in mind that none of these “Web 2.0″ strategies should replace well-written, well-presented analysis, delivered in person whenever possible and making specific recommendations for changes (including a testing plan when possible!)

STEP 6: PUT IT ALL TOGETHER!

Assuming you’ve completed the previous five steps, you now have a functional web analytics organization, one capable of delivering relevant reports and producing actionable analysis. Now the challenge is to stop spending all of your time generating reports and start delivering analysis!

Unfortunately, for many organizations this is really, really difficult. Even when there are dedicated resources — people specifically hired to do web “analytics” (not web “reporting”) — far too many bright folks end us spending all of their time churning out reports. Even worse, these reports often go unread, unused, and unnoticed despite the real and opportunity costs associated with generating them.

To be really, really successful with web analytics you have to train the organization to stop looking for reports and start asking for analysis, insights, and recommendations. While every situation is different, ask yourself how closely your organization follows these steps:

1. Automated KPI reports arrive, highlighting a potential problem associated with a core business objective
2. Line of business analytics resources consult critical reports directly looking for a reasonable explanation
3. Failing a reasonable explanation, business resources request analysis resources from the analytics hub
4. Analytics hub double-checks LOB’s cursory analysis, confirming the need for deeper exploration
5. Analytics hub prioritizes analysis with the business based on pre-agreed criteria
6. Analysis is delivered back to the business along with recommendations and a testing plan
7. Recommendations are reviewed by the business, test plan is agreed upon
8. Tests are run, results are socialized as follow-up to the original analysis
9. Incremental value of change is recorded to help calculate web analytics return on investment

Individual departments are still getting their reports, but they’re generating them by themselves. Senior managers have an appropriate view into the metrics, and their own resources to evaluate observed changes. Those resources have a way to get help when help is needed. Help (the hub) isn’t bogged down generating ad hoc reports all the time and is able to focus on high-value priorities. People produce analysis and make recommendations. Recommendations are tested. Optimization happens.

Kinda brings a tear to your eye, doesn’t it?

I know there are a hundred other things that come up in the line of business for any of you who are working practitioners, but having a clear communication strategy is the first step towards whittling that list down to something reasonable and, more importantly, valuable to your organization. Defining your business objectives, clarifying ownership and organization structures, establishing KPIs and critical reports, and knowing what your analysis output will actually look like is fundamental.

Defining your web analytics communication strategy will let the data work for you, not make you work for the data. It will help you move from making purely tactical decisions and start using web analytics strategically as part of your entire business. Over time you’ll find that a clear strategy, no surprise, helps the entire organization better understand web analytics in general and the value your investment can provide. And perhaps most importantly, a clear strategy will cut down on the volume of under-used, unused, and ignored reports traveling across your network.

If you’re interested in defining a web analytics communication strategy in your organization, I’d love to talk to you. If you don’t need help, I’m still happy to provide encouragement. If I can help you, great. If I can’t help you, I bet I know somebody who can!

Judah’s recent post titled “what does your web analytics team look like” reminded me of something that has been on my mind a lot since I presented my Web Analytics: A Day a Month webcast for the American Marketing Association last month. As I travel the world talking about web analytics to companies of all shapes and sizes, one thing I’m struck by is the number of differences in how companies approach sharing web analytic data and information.

It’s not as if there is any one “right” way to communicate about web analytics, but it is clear that there are many, many wrong ways to do it. But rather than dwell on wrongness, I prefer to focus on rightness so here are a few thoughts on developing a clear strategy for communicating web analytics.

This post may seem pretty basic to many of you, but if it does I would encourage you to ask yourself these questions:

• What decisions are web analytics driving in your organization?
• Are those decisions largely tactical or are they truly strategic?
• Do you feel like most people in your organization understand web data?
• Are you producing reports that are going under-used, unused, or are flat out being ignored?

If you are less than impressed with your responses I would encourage you to read on. I’m not saying you’ll necessarily learn anything new, but maybe you’ll read something that you think your boss should hear.

STEP ONE: DEFINE YOUR BUSINESS OBJECTIVES

I know, I know, you’ve heard me say this before. I’ve been saying this since 2002 but I’m going to keep on saying it since it bears repeating. By clearly defining your business objectives you get two things done:

1. You remind everyone in your organization why you have a web site and why those of you who work online come to work every day.
2. You build a framework against which you will define the core activities and interactions that are worth measuring and communicating

The second point is important: You cannot measure everything effectively and efficiently — you have to have some basis for deciding what to measure and what to report. I have seen any number of companies work hard to collect “all possible data” only to realize that few people are actually asking for that data and even fewer are doing anything with it.

When you define your business objectives and get consensus on what is most important to your online business, the measurable activities that you will be communicating across the organization become clear. Suddenly rather than struggling to measure every aspect of every page across every segment you’re able to focus on critical measures in critical paths in your most important visitor segments.

I covered all of this in Web Analytics Demystified what seems like years ago and again in Web Site Measurement Hacks (which you can now purchase direct from my site, had I mentioned that?) but again it is worth repeating. And while it is far less common now that I will ask companies about their business objectives and get conflicting opinions, many companies have still not gone through the process of clearly documenting these objectives and the associated activities to serve as basis for their measurement efforts.

STEP TWO: DETERMINE WHO OWNS ANALYTICS AT YOUR COMPANY

One of the biggest problems I see in web analytics today is a lack of clarity regarding ownership of analytics inside the organization. On this point I will be as clear as possible:

The owner of web analytics in your company NEEDS to be someone senior enough to ensure that analysis is being produced and used!

I spend an awful lot of time as a consultant talking about ownership and structure in analytics. Your executive sponsor needs to be closely connected to web analytics and have a clear understanding of the value and opportunity measurement provides. If this is not the case, you may spend an awful lot of time producing reports that go unread and analysis that goes unused.

I suspect that my fellow blogger Daniel Shields can attest to the goodness in this recommendation, working for a great boss at CableOrganizer, but more often than not when I ask the question “Who owns web analytics?” I get responses that talk about budget centers, middle-management who haven’t got budget authority or enough political clout, or worse yet, nothing but uncomfortable laughter.

Clients almost always ask “Where should web analytics live? Should it live in Finance, I.T., Marketing, or Research?” to which I almost always answer “Who is the most senior, well-connected person in your organization that is likely to really understand what web analytics is good for?” and then give their department as my answer. Here are some additional thoughts:

• Finance: Analytics living in your finance organization is fine because your CFO understands how to produce detailed analysis and make that analysis valuable internally
• Marketing: Marketing is great since in many cases marketing has the most to gain (or lose) based on web analytics data and analysis
• Research: If you have a market research organization this is also a great home since the analysis team in research usually has an excellent understanding of the customer and their (offline) behavior
• Information Technology: I personally don’t usually recommend that web analytics live in I.T. There is often too much baggage and a disconnect between I.T. and the business for this to work (but I do know of a handful of examples where I.T. ownership of analytics does work)

At the end of the day the most successful analytics organizations are those where the executive sponsor “gets it” and is able to champion for the cause at a very high level. They will need money, resources, and time from the rest of the company to deeply integrate the necessary web analytics business processes, so seniority is an absolute must.

STEP THREE: DETERMINE YOUR ANALYTICS ORGANIZATIONAL STRUCTURE

This is the step I’ve been thinking a lot about lately, how analytics organizations are structured and integrated into medium-to-large-to-very large companies. As I’m sure you know, this piece is far from a no-brainer — whether you subscribe to 10/90, 10/20/70, or some other percentage-wise distribution of effort, I think we can all agree that people are critical to web analytics success.

But as Judah deftly points out, just hiring someone is only the beginning of the work: The more important piece is determining how those resources are going to actually provide benefit back to the entire organization. You need to have a clear strategy for leveraging these resources to produce the maximum number of insights possible.

For about four yeas I have been talking about the “hub and spoke” model for web analytics organizations, especially to medium, large, and very large companies. The hub and spoke is basically a centralized/decentralized model for measurement, one that centralizes deep analysis expertise for use across the organization but mandates that each individual department and line of business takes responsibility for their own reporting needs.

The folks in the analytics hub are directly responsible for things like:

• Producing analysis, real analysis, to support business decisions
• Providing training out to the rest of the organization on tools and data
• Communicating about the goodness (or lack thereof) in the data collected
• Interfacing with the vendor(s) providing measurement software and services
• Managing multivariate tests and analyzing their results
• Working with I.T. to make changes to data collection and integration

Perhaps most importantly, the hub work directly for the executive sponsor for analytics (see STEP TWO above.) Establishing a real web analytics hub is the first thing you need to do if you want to STOP spending 80 percent of your time generating reports (something a prospect recently referred to as being a “report monkey” which they didn’t seem super-excited about …)

The folks in the individual departments and LOBs are responsible for things like:

• Paying careful attention to their key performance indicators and react to observed changes
• Spend enough time learning the available technology to answer at least basic questions when changes are observed
• Generating whatever reports are necessary on a regular basis and modifying those as required
• Interface with the analytics hub to ensure that requests for testing and analysis are clearly communicated
• Respond to test results and analysis by putting the insights generated to work for the organization

The best possible news is that the folks in the spokes don’t have to be web analytics experts! Hell, they don’t even need to read the available literature if they don’t want to (but they should.) They really only need to take enough time to learn what their KPIs are telling them and which reports in the analytics application(s) are relevant when things change.

Thinking about the relationship between the hub and spokes:

• The hub does analysis, and the spokes do reporting
• The hub executes multivariate tests, but the spokes recommend them
• The hub work directly with I.T., the spokes get to continue avoiding I.T.
• The hub helps to plan, manage, and monitor KPIs, the spokes live and die by them
• The hub runs something like Omniture Discover or IndexTools Rubix, the spokes use SiteCatalyst or Google Analytics

This is great news because there are many, many people out there that have a 0.2, a 0.33, or a 0.5 FTE for web analytics — not nearly enough time to really get deep into web analytics but enough to create the expectation that they’ll use the data to make business decisions. The hub and spoke model creates a business process to support partial FTE in their endeavor to use and benefit from web analytics, which those partial FTE seem to truly, truly appreciate!

In my experience, over time the people who really like this kind of work will pop up and ask great questions, looking to push the boundaries of their understanding of “our little craft.” They’ll read books, blogs, go to conferences, etc. and over time may realize that they really want to work in the field of web analytics full time. Which is great, because without those people flowing into the system, the multitude of recruiters and companies across the globe looking for experience web analytics professionals haven’t got a prayer.

Since Judah, Daniel, and I have been talking about the length of out posts lately I think I’ll stop here and publish Part II of this post later this week.

The key takeaways from the thoughts here are:

1. You have to have a web analytics communication strategy
2. You have to clearly define your business objectives and supporting activities
3. You need to define and establish an analytics organization
4. Your analytics organization needs to report to an appropriately senior person
5. The hub and spoke model for web analytics has many advantages, especially in large organizations
6. Web analytics done well has a tendency to make people more, not less, interested in web analytics (which is good!)

For those of you who missed my presentation yesterday, “Web Analytics: A Day a Month”, you can now listen to the re-recorded webcast at WebEx thanks to Tableau and the American Marketing Association. I say “re-recorded” since once again I managed to bring a large enough crowd to the webcast to break WebEx. Web analytics is hot!

You can listen to the webcast without having to register (still requires name and email) until next week I think by going to:

amaevents.webex.com

Here are a few other things I should mention, as long as I’m writing:

• I’m going to be in Boston next week for Judah’s Web Analytics Wednesday event (rescheduled from last month due to me being a weather-wimp) and if you’re in Boston or nearby I’d love to catch up. Please join us in Cambridge!
• The next few weeks I will be in Chicago (Jan 25th), Seattle (Jan 30th), San Jose (Jan 31st) and New York (Feb 7th) giving the keynote address at OpinionLab’s client conferences. The nice folks at OpinionLab mentioned that they’re opening up the events to non-customers so if you’d like to hear me talk about how quantitative and qualitative data combined provide a much more actionable view of the online visitor, please join us!
• The nice folks at the Direct Marketing Association who gave away PDF copies of my book Web Analytics Demystified in exchange for participation in their web analytics survey (written up by the amazing W. David Rhee) are holding a webinar on the research findings on Januay 23rd. The event is not free but the research is pretty good and if you’re in the DMA you should consider joining the call.
• The nice folks at the Web Analytics Association are also holding a research call, tomorrow (Jan 17th) in fact, on the future of the web analytics industry. I think this event is free but it might only be free to WAA members (maybe if Richard or Andrea read this they can comment for all to see!) The call is tomorrow morning at 9 AM Pacific, noon Eastern and you can register to attend at the WAA web site.
• Anil Batra has apparently jumped on the “bounce rate” bandwagon and is having a “bounce rate survey” that he’d like you to participate in. I haven’t had a chance to take it yet but I really enjoyed Anil’s salary research so I’m sure he’ll do a great job with bounce rate too!
• I’ll be back in San Diego in late February at Aaron Kahlow’s Online Marketing Summit talking about Key Performance Indicators in a Web 2.0 World.  I really enjoyed OMS last year and am looking forward to getting back to Sea World Aaron’s event!
• I had nothing to do with that movie on web analytics, despite it being filmed here in the Rose City, and have no idea what Ian is talking about.  Ian should spend less time at the movies and more time reading what experienced practitioners are saying about Gatineau.  <grin>

If I’m forgetting anything please comment below.  I think you’ll really like the webcast — the feedback I got has been excellent so far (despite some people going gossipy about the title of my last post on the subject … cage match indeed!)

Back in December of last year when I first posted on measuring visitor engagement, I hardly imagined how much interest the topic would generate. Shortly after the first post, I commented that my definition of engagement was as follows:

Engagement is an estimate of the degree and depth of visitor interaction on the site against a clearly defined set of goals.

I then went and wrote over a dozen posts, publishing feedback from some incredibly bright people and demonstrating the utility of a well-defined measure for engagement. Since that time, however, some have questioned the value of such a metric and thusly prompted me to update and publish the following calculation for visitor engagement:

I presented this calculation to a completely full room last week at Emetrics but wanted to provide an update to all my patient readers who were not able to make the event. You can download my entire Emetrics on “Web Analytics 2.0″ which includes the slides on measuring visitor engagement from the White Papers and Presentations section of my site.

I very much believe that engagement is a metric, not an excuse, and that the metric described in this post provides a powerful measurement framework for sites looking for new ways to examine and evaluate visitor interaction. I know that for my own site, the use of simple measures like “bounce rate”, “conversion rate” and “average time spent” is simply insufficient for selling anything other than my books. But I’m now in the business of selling consulting, a complex and sometimes time-consuming sale, and so I’m always on the hunt for any web analytics measure that will give me an edge and help identify truly qualified opportunities.

I believe this metric is exactly that.

This post is an extension of the work I did in late 2006 and early 2007 and was written to clarify my position, update my thinking in the context of “Web Analytics 2.0″, and reiterate my desire to have an open and honest conversation with my peers and other interested parties regarding the measurement of visitor engagement. Web analytics is hard but not impossible; the same is true regarding the calculation and use of robust measures of visitor behavior.

I believe the visitor engagement measurement to be perhaps the most important of all “Web Analytics 2.0″ measurements. Given that this model fully supports both quantitative and qualitative data, and given that the model is build as much around the measurement of “events” as much as page views, sessions, and visitors, I (perhaps haughtily) believe this calculation to be prototypical of the types of measurements we will see as we continue to explore the boundaries of “Web Analytics 2.0″ (download my presentation from SEMphonic X Change).

The Web Analytics Demystified Visitor Engagement Calculation

The latest version of my visitor engagement metric, with notes about its calculation and use, are as follows. If you’re too busy to read this entire post but would like to learn more about this measure, please write me directly and we can set up a time to discuss it.

This is a model, not an absolute calculation for all sites. I agree with other analysts and bloggers who insightfully say that there is no single calculation of engagement useful for all sites, but I do believe my model is robust and useful with only slight modification across a wide range of sites. The modification comes in the thresholds for individual indices, the qualitative component, and the measured events (see below); otherwise I believe that any site capable of making this calculation can do so without having to rethink the entire model.

The calculation needs to be made over the lifetime of visitor sessions to the site and also accommodate different time spans. This means that to calculate “percent of sessions having more than 5 page views” you need to examine all of the visitor’s sessions during the time-frame under examination and determine which had more than five page views. If the calculation is unbounded by time, you would examine all of the visitor’s sessions in the available dataset; if the calculation was bounded by the last 90 days, you would only examine sessions during the past 90 days.

The individual session-based indices are defined as follows (and these are slightly updated from past posts on the subject):

• Click-Depth Index (Ci) is the percent of sessions having more than “n” page views divided by all sessions.
• Recency Index (Ri) is the percent of sessions having more than “n” page views that occurred in the past “n” weeks divided by all sessions. The Recency Index captures recent sessions that were also deep enough to be measured in the Click-Depth Index.
• Duration Index (Di) is the percent of sessions longer than “n” minutes divided by all sessions.
• Brand Index (Bi) is the percent of sessions that either begin directly (i.e., have no referring URL) or are initiated by an external search for a “branded” term divided by all sessions (see additional explanation below)
• Feedback Index (Fi) is the percent of sessions where the visitor gave direct feedback via a Voice of Customer technology like ForeSee Results or OpinionLab divided by all sessions (see additional explanation below)
• Interaction Index (Ii) is the percent of sessions where the visitor completed one of any specific, tracked events divided by all sessions (see additional explanation below)

In addition to the session-based indices, I have added two small, binary weighting factors based on visitor behavior:

• Loyalty Index (Li) is scored as “1″ if the visitor has come to the site more than “n” times during the time-frame under examination (and otherwise scored “0″)
• Subscription Index (Si) is scored as “1″ if the visitor is a known content subscriber (i.e., subscribed to my blog) during the time-frame under examination (and otherwise scored “0″)

You take the value of each of the component indices, sum them, and then divide by “8″ (the total number of indices in my model) to get a very clean value between “0″ and “1″ that is easily converted to a percentage. Given sufficient robust technology, you can then segment against the calculated value, build super-useful KPIs like “percent highly-engaged visitors” and add the engagement metric to the reports you’re already running.

The Visitor Engagement Calculation in Detail

The Click-Depth, Recency, and Duration indices are all pretty straight forward and are more-or-less the traditional indicators that most people (incorrectly) call “measures of engagement”. Each of these are very important to the overall calculation, but none of these alone are sufficiently robust to describe “engaged” visitors. I set the “n” values for my site’s calculation based on the average value for each and this seems to work pretty well (meaning my Ci looks for sessions more than “5 page views” in depth, my Ri looks for sessions more than “5 page views” that occurred in the “past three weeks” and my Di is looking for sessions longer than about “5 minutes” in length.)

Brand Index is a little more complicated. Here I have made a list of all the terms I believe to be “branded” for my site and business, terms like eric t. peterson, web analytics demystified, web site measurement hacks, web analytics wednesday, and the big book of key performance indicators. Whenever a session begins either with no referring domain or comes from a search engine with one of these terms attached, I count this as a “branded session” and score appropriately. While this index perhaps unfairly weights towards search engines, I firmly believe that if you’re starting your session with either my branded URL, my name, or the name of one of my books that you are already engaged.

Feedback Index is the sole qualitative input to this model but it can easily be expanded if necessary. Here I am simply scoring sessions based on whether visitors are providing qualitative feedback via the OpinionLab “O” present throughout my web site or writing me directly by clicking a “mailto:” link. I’m not looking at whether the feedback is positive or negative, only whether feedback was given, operating under the belief that anyone willing to provide direct feedback is engaged.

The Feedback Index could easily be expanded by scoring based on the answer to direct questions posed to the visitor, questions like “do you find the content on this site valuable?”, “do you plan on calling Web Analytics Demystified about consulting?” and “would you described yourself as engaged with this site?” Given a sufficiently robust mechanism for making the calculation, the Feedback Index can provide a tremendously powerful input to the visitor engagement model.

The Interaction Index captures sessions in which specific “engaged events” occur other than the site’s primary conversion event — events like downloading a white paper, providing an email address, requesting a presentation or PDF, commenting on a blog post, Digging a post, emailing content to a friend, printing a page, etc. The Interaction Index is designed to capture a small weighting from those measurable goals on your site you believe to be indicative of engagement.

The Interaction Index specifically does not examine commerce transactions and other conversion events of fundamental import to the site. While I have debated this in the past, here is the rationale for recommending the exclusion of primary conversion events:

1. These events already have their own key performance indicator: conversion. Given that conversion is likely already defined for most transactional sites and tracked in great detail, adding conversion to the visitor engagement calculation is superfluous in my opinion.
2. The visitor engagement metric is designed to provide information about the large number of visitors who do not convert. Given relatively low conversion rates online, having visitor engagement be decoupled from conversion provides a cleaner measure for use in exploring non-purchaser behavior, including looking for independent correlation between the two measures.
3. By excluding conversion, the two metrics can be used side-by-side to look for visitor behaviors may not be obvious otherwise. Given the lifetime of possible visitor behaviors, having a way to look for well-engaged visitors who have not completed a transaction online or have completed a transaction outside of the available data set provides a critical view not otherwise readily attained.

The Loyalty Index is a reflection of my belief that repeat visitation behavior is perhaps the best measure of engagement available. Based on the distribution of visitor loyalty data at Web Analytics Demystified, I score “1″ when visitors have come to the site more than five times in the past 12 months.

The Subscription Index is a reflection that truly engaged visitors are able to self-identify by subscribing to our blogs or newsletters; if you have taken the time to subscribe to one of the Web Analytics Demystified blogs I believe you to be engaged. If your site does not have some type of XML-based content subscription you can either drop this index or (perhaps better) look for an opportunity to develop a subscription service, thusly giving your visitors another good engagement point.

How Does This All Work in Practice?

Careful readers will likely have already figured out that as visitors come to your site over time, their cumulative “lifetime engagement score” changes as they satisfy the criteria of each individual index. So someone coming from a Google search for “web analytics demystified” who looks at 10 pages over the course of 7 minutes, downloads a white paper and then returns to my site the next day will have a higher visitor engagement value than someone coming from a blog post who looks at 2 pages and leaves 2 minutes later, never to return.

If you think about it for just a bit, and consider the components in the full calculation, the visitor engagement metric starts to make an awful lot of sense. Consider the following:

• A visitor can quickly move through a lot of pages, getting exactly what they need, and still be scored usefully through the Click-Depth Index
• A visitor can slowly and methodically read a few pages and be scored usefully through the Duration Index
• A visitor can come to the site frequently and do little more than read a single page of content and be usefully scored through the Recency and Loyalty Indices
• A visitor can come to the site once, subscribe to the blog, return later and download a presentation, and be usefully scored through the Subscription and Interaction Indices
• A visitor can come to the site, click on dozens of pages but fail to find what they are looking for, then tell me so using my feedback mechanisms and be usefully scored through the Click-Depth and Feedback Indices

The power of the metric is appreciated when you apply it to the commonly measured dimensions found in web analytics: referring domain/URL, search engine/phrase, campaign/placement/creative, content group and page, browser/operating system, etc. Suddenly instead of looking at simple measures, you’re examining the potential of visitors coming from or going to each element in the dimension. To see the metric in action, I encourage you to read my post on the gradual building of context, at least until I’m able to publish new screenshots later this week.

Some Parting Thoughts about Measuring Visitor Engagement

Some folks have complained that this metric is “not immediately useful”, that nobody will understand it, and that it is impossible to calculate. Perhaps, but I would argue that A) no metric is truly immediately useful and B) most people don’t understand web analytics because web analytics is hard. The assumption that a diverse organization is going to be more successful using “bounce rate” because it can be glibly explained by saying “your content sucks” is just wrong — all of this stuff needs to be explained regardless of the complexity of the metrics involved.

Regarding the metric being impossible to calculate, it fully depends on which application you’re using. If you’re trying to get by using free tools then yes, you’re out of luck. But if you’re using robust tools like the high-end offerings from Unica, IndexTools, Visual Sciences, and WebTrends then you should have little trouble using the metric I describe in this post.

I personally believe that Web Analytics 2.0 both requires and allows us to be more creative and thoughtful in our use of metrics. Why not use a robust indicator if one is warranted? Especially if you’re not selling anything online, or if you’re selling high-consideration items, my visitor engagement metric can be shown to be an extremely powerful measurement.

Given the assertion that some consultants are apparently charging $200,000 USD for complex “engagement index” work, and given that someone working for Google is in the process of trying to patent a much simpler version of this equation, I am happy to give my work away to the entire industry in an effort to promote the use of more meaningful metrics to be brought to bear on increasingly complex measurement problems.

What do you think? Did you see my Emetrics presentation and still have questions? Did you read every word of my series on engagement and still not believe me? Do you need to see engagement in action before you’re willing to say it’s not just an excuse? Or are you chomping at the bit to have a robust measure like this for use on your own site?

Especially on this subject I relish your feedback, either via comments or via email — your choice! I find the subject fascinating and welcome the opportunity to discuss it you, my (hopefully) engaged readers.