How to measure visitor engagement, redux

Back in December of last year when I first posted on measuring visitor engagement, I hardly imagined how much interest the topic would generate. Shortly after the first post, I commented that my definition of engagement was as follows:

Engagement is an estimate of the degree and depth of visitor interaction on the site against a clearly defined set of goals.

I then went and wrote over a dozen posts, publishing feedback from some incredibly bright people and demonstrating the utility of a well-defined measure for engagement. Since that time, however, some have questioned the value of such a metric and thusly prompted me to update and publish the following calculation for visitor engagement:

I presented this calculation to a completely full room last week at Emetrics but wanted to provide an update to all my patient readers who were not able to make the event. You can download my entire Emetrics on “Web Analytics 2.0″ which includes the slides on measuring visitor engagement from the White Papers and Presentations section of my site.

I very much believe that engagement is a metric, not an excuse, and that the metric described in this post provides a powerful measurement framework for sites looking for new ways to examine and evaluate visitor interaction. I know that for my own site, the use of simple measures like “bounce rate”, “conversion rate” and “average time spent” is simply insufficient for selling anything other than my books. But I’m now in the business of selling consulting, a complex and sometimes time-consuming sale, and so I’m always on the hunt for any web analytics measure that will give me an edge and help identify truly qualified opportunities.

I believe this metric is exactly that.

This post is an extension of the work I did in late 2006 and early 2007 and was written to clarify my position, update my thinking in the context of “Web Analytics 2.0″, and reiterate my desire to have an open and honest conversation with my peers and other interested parties regarding the measurement of visitor engagement. Web analytics is hard but not impossible; the same is true regarding the calculation and use of robust measures of visitor behavior.

I believe the visitor engagement measurement to be perhaps the most important of all “Web Analytics 2.0″ measurements. Given that this model fully supports both quantitative and qualitative data, and given that the model is build as much around the measurement of “events” as much as page views, sessions, and visitors, I (perhaps haughtily) believe this calculation to be prototypical of the types of measurements we will see as we continue to explore the boundaries of “Web Analytics 2.0″ (download my presentation from SEMphonic X Change).

The Web Analytics Demystified Visitor Engagement Calculation

The latest version of my visitor engagement metric, with notes about its calculation and use, are as follows. If you’re too busy to read this entire post but would like to learn more about this measure, please write me directly and we can set up a time to discuss it.

This is a model, not an absolute calculation for all sites. I agree with other analysts and bloggers who insightfully say that there is no single calculation of engagement useful for all sites, but I do believe my model is robust and useful with only slight modification across a wide range of sites. The modification comes in the thresholds for individual indices, the qualitative component, and the measured events (see below); otherwise I believe that any site capable of making this calculation can do so without having to rethink the entire model.

The calculation needs to be made over the lifetime of visitor sessions to the site and also accommodate different time spans. This means that to calculate “percent of sessions having more than 5 page views” you need to examine all of the visitor’s sessions during the time-frame under examination and determine which had more than five page views. If the calculation is unbounded by time, you would examine all of the visitor’s sessions in the available dataset; if the calculation was bounded by the last 90 days, you would only examine sessions during the past 90 days.

The individual session-based indices are defined as follows (and these are slightly updated from past posts on the subject):

Click-Depth Index (Ci) is the percent of sessions having more than “n” page views divided by all sessions.
Recency Index (Ri) is the percent of sessions having more than “n” page views that occurred in the past “n” weeks divided by all sessions. The Recency Index captures recent sessions that were also deep enough to be measured in the Click-Depth Index.
Duration Index (Di) is the percent of sessions longer than “n” minutes divided by all sessions.
Brand Index (Bi) is the percent of sessions that either begin directly (i.e., have no referring URL) or are initiated by an external search for a “branded” term divided by all sessions (see additional explanation below)
Feedback Index (Fi) is the percent of sessions where the visitor gave direct feedback via a Voice of Customer technology like ForeSee Results or OpinionLab divided by all sessions (see additional explanation below)
Interaction Index (Ii) is the percent of sessions where the visitor completed one of any specific, tracked events divided by all sessions (see additional explanation below)

In addition to the session-based indices, I have added two small, binary weighting factors based on visitor behavior:

Loyalty Index (Li) is scored as “1″ if the visitor has come to the site more than “n” times during the time-frame under examination (and otherwise scored “0″)
Subscription Index (Si) is scored as “1″ if the visitor is a known content subscriber (i.e., subscribed to my blog) during the time-frame under examination (and otherwise scored “0″)

You take the value of each of the component indices, sum them, and then divide by “8″ (the total number of indices in my model) to get a very clean value between “0″ and “1″ that is easily converted to a percentage. Given sufficient robust technology, you can then segment against the calculated value, build super-useful KPIs like “percent highly-engaged visitors” and add the engagement metric to the reports you’re already running.

The Visitor Engagement Calculation in Detail

The Click-Depth, Recency, and Duration indices are all pretty straight forward and are more-or-less the traditional indicators that most people (incorrectly) call “measures of engagement”. Each of these are very important to the overall calculation, but none of these alone are sufficiently robust to describe “engaged” visitors. I set the “n” values for my site’s calculation based on the average value for each and this seems to work pretty well (meaning my Ci looks for sessions more than “5 page views” in depth, my Ri looks for sessions more than “5 page views” that occurred in the “past three weeks” and my Di is looking for sessions longer than about “5 minutes” in length.)

Brand Index is a little more complicated. Here I have made a list of all the terms I believe to be “branded” for my site and business, terms like eric t. peterson, web analytics demystified, web site measurement hacks, web analytics wednesday, and the big book of key performance indicators. Whenever a session begins either with no referring domain or comes from a search engine with one of these terms attached, I count this as a “branded session” and score appropriately. While this index perhaps unfairly weights towards search engines, I firmly believe that if you’re starting your session with either my branded URL, my name, or the name of one of my books that you are already engaged.

Feedback Index is the sole qualitative input to this model but it can easily be expanded if necessary. Here I am simply scoring sessions based on whether visitors are providing qualitative feedback via the OpinionLab “O” present throughout my web site or writing me directly by clicking a “mailto:” link. I’m not looking at whether the feedback is positive or negative, only whether feedback was given, operating under the belief that anyone willing to provide direct feedback is engaged.

The Feedback Index could easily be expanded by scoring based on the answer to direct questions posed to the visitor, questions like “do you find the content on this site valuable?”, “do you plan on calling Web Analytics Demystified about consulting?” and “would you described yourself as engaged with this site?” Given a sufficiently robust mechanism for making the calculation, the Feedback Index can provide a tremendously powerful input to the visitor engagement model.

The Interaction Index captures sessions in which specific “engaged events” occur other than the site’s primary conversion event — events like downloading a white paper, providing an email address, requesting a presentation or PDF, commenting on a blog post, Digging a post, emailing content to a friend, printing a page, etc. The Interaction Index is designed to capture a small weighting from those measurable goals on your site you believe to be indicative of engagement.

The Interaction Index specifically does not examine commerce transactions and other conversion events of fundamental import to the site. While I have debated this in the past, here is the rationale for recommending the exclusion of primary conversion events:

These events already have their own key performance indicator: conversion. Given that conversion is likely already defined for most transactional sites and tracked in great detail, adding conversion to the visitor engagement calculation is superfluous in my opinion.
The visitor engagement metric is designed to provide information about the large number of visitors who do not convert. Given relatively low conversion rates online, having visitor engagement be decoupled from conversion provides a cleaner measure for use in exploring non-purchaser behavior, including looking for independent correlation between the two measures.
By excluding conversion, the two metrics can be used side-by-side to look for visitor behaviors may not be obvious otherwise. Given the lifetime of possible visitor behaviors, having a way to look for well-engaged visitors who have not completed a transaction online or have completed a transaction outside of the available data set provides a critical view not otherwise readily attained.

The Loyalty Index is a reflection of my belief that repeat visitation behavior is perhaps the best measure of engagement available. Based on the distribution of visitor loyalty data at Web Analytics Demystified, I score “1″ when visitors have come to the site more than five times in the past 12 months.

The Subscription Index is a reflection that truly engaged visitors are able to self-identify by subscribing to our blogs or newsletters; if you have taken the time to subscribe to one of the Web Analytics Demystified blogs I believe you to be engaged. If your site does not have some type of XML-based content subscription you can either drop this index or (perhaps better) look for an opportunity to develop a subscription service, thusly giving your visitors another good engagement point.

How Does This All Work in Practice?

Careful readers will likely have already figured out that as visitors come to your site over time, their cumulative “lifetime engagement score” changes as they satisfy the criteria of each individual index. So someone coming from a Google search for “web analytics demystified” who looks at 10 pages over the course of 7 minutes, downloads a white paper and then returns to my site the next day will have a higher visitor engagement value than someone coming from a blog post who looks at 2 pages and leaves 2 minutes later, never to return.

If you think about it for just a bit, and consider the components in the full calculation, the visitor engagement metric starts to make an awful lot of sense. Consider the following:

A visitor can quickly move through a lot of pages, getting exactly what they need, and still be scored usefully through the Click-Depth Index
A visitor can slowly and methodically read a few pages and be scored usefully through the Duration Index
A visitor can come to the site frequently and do little more than read a single page of content and be usefully scored through the Recency and Loyalty Indices
A visitor can come to the site once, subscribe to the blog, return later and download a presentation, and be usefully scored through the Subscription and Interaction Indices
A visitor can come to the site, click on dozens of pages but fail to find what they are looking for, then tell me so using my feedback mechanisms and be usefully scored through the Click-Depth and Feedback Indices

The power of the metric is appreciated when you apply it to the commonly measured dimensions found in web analytics: referring domain/URL, search engine/phrase, campaign/placement/creative, content group and page, browser/operating system, etc. Suddenly instead of looking at simple measures, you’re examining the potential of visitors coming from or going to each element in the dimension. To see the metric in action, I encourage you to read my post on the gradual building of context, at least until I’m able to publish new screenshots later this week.

Some Parting Thoughts about Measuring Visitor Engagement

Some folks have complained that this metric is “not immediately useful”, that nobody will understand it, and that it is impossible to calculate. Perhaps, but I would argue that A) no metric is truly immediately useful and B) most people don’t understand web analytics because web analytics is hard. The assumption that a diverse organization is going to be more successful using “bounce rate” because it can be glibly explained by saying “your content sucks” is just wrong — all of this stuff needs to be explained regardless of the complexity of the metrics involved.

Regarding the metric being impossible to calculate, it fully depends on which application you’re using. If you’re trying to get by using free tools then yes, you’re out of luck. But if you’re using robust tools like the high-end offerings from Unica, IndexTools, Visual Sciences, and WebTrends then you should have little trouble using the metric I describe in this post.

I personally believe that Web Analytics 2.0 both requires and allows us to be more creative and thoughtful in our use of metrics. Why not use a robust indicator if one is warranted? Especially if you’re not selling anything online, or if you’re selling high-consideration items, my visitor engagement metric can be shown to be an extremely powerful measurement.

Given the assertion that some consultants are apparently charging $200,000 USD for complex “engagement index” work, and given that someone working for Google is in the process of trying to patent a much simpler version of this equation, I am happy to give my work away to the entire industry in an effort to promote the use of more meaningful metrics to be brought to bear on increasingly complex measurement problems.

What do you think? Did you see my Emetrics presentation and still have questions? Did you read every word of my series on engagement and still not believe me? Do you need to see engagement in action before you’re willing to say it’s not just an excuse? Or are you chomping at the bit to have a robust measure like this for use on your own site?

Especially on this subject I relish your feedback, either via comments or via email — your choice! I find the subject fascinating and welcome the opportunity to discuss it you, my (hopefully) engaged readers.

Posted Monday, October 22nd, 2007 | 56 responses | Add a Comment | Share, Save or Email

Kenn Gold

Ok, well you answered the one question I had from the presentation which was how come Di was not a % of Ci like Ri is. But when you went through them at eMetrics you might have missed saying it (or I missed hearing it….heh).

Sometime in the next week I will have someone on my team see if we can trend this out over the year and see what it looks like.

I think my primary concern is how to make it actionable. Because it is a combination of so many different factors, if you say are trending steadily along for a while and then your ‘engagement factor’ suddenly drops, you would then have to break it back apart to see what exactly changed, right?

share.websitemagazine.com

How to measure visitor engagement, redux…

Engagement is an estimate of the degree and depth of visitor interaction on the site against a clearly defined set of goals….

eric

Kenn: I’d love to hear what you find out and the specifics of how you’re able to make the calculation using your specific analytics application.

You make the visitor engagement metric actionable the exact same way you make any other metric actionable — by conducting careful analysis and making recommendations. In some instances you may look at the component metrics, but in others you may be able to examine the individual elements of whatever dimension you’re applying the metric to (referring domains or campaigns, for example.)

In this regard the visitor engagement metric is the same as any other metric (conversion, average page views per session, etc.)

I look forward to hearing back from you about your results!

Katie Paine

I think this is great for a web site, but it really doesn’t tell you much about the engagement level with your brand, nor does it account for what Forrester calls “intimacy.” As always, I believe it will take a combination of metrics, including some human factor analysis before we get this right. As my father told an audience of ad execs 40 years ago: “If we can put a man in orbit, why can’t we determine the effectiveness of our communications? The reason is simple and perhaps, therefore, a little old-fashioned: people, human beings with a wide range of choice. Unpredictable, cantankerous,capricious, motivated by innumerable conflicting interests, and conflicting desires.”

eric

Katie: Excellent point! I was recently at the SEMphonic X Change (which I highly recommend) listening to Digitas and Avenue A/Razorfish debate their measure of “brand engagement” when I decided what I’m really talking about is an operational measure of micro-engagement and what they were talking about was a multi-channel measure of macro-engagement.

Thanks for reminding me of that!

In terms of measuring oddly motivated, cantankerous, conflicted folks … I need to introduce you to my friend Joseph. Joseph is able to measure the impossible and make sense of the senseless. Me? I try and focus on the simple stuff that gets us 80% of the way there.

Thanks for your comment!

The Blackbeak Blog…. Arr! Web Analytics » Blog Archive » Visitor Engagement Continued…. by Eric

[...] Eric Peterson just published what I think is a comprehensive view of how to develop engagement metrics. I think basically this line (from his post) sums it up; Engagement is an estimate of the degree and depth of visitor interaction on the site against a clearly defined set of goals. [...]

Scott Klein

Some if not all of these can be captured in real time. How fruitful would it be for a website change itself in real time to maximize outcomes for a user with a given engagement level? That is, is it useful to move a user up the engagement scale by, say, offering them special content — or the opposite, starting to put up reg-walls — once they’ve identified themselves as more engaged than transient search engine traffic?

Bud Caddell

Eric, what do you think about measuring participation inequality in communities? Do you think it’s worthwhile?

Here’s my math: http://www.passion2publish.com/2007/08/measuring-parti.html

eric

Scott: It’s an excellent idea and there is very little that would prevent any vendor having sufficiently robust technology from leveraging my engagement scoring model. I suspect that this is actually not too far off what companies like Touchclarity and Kefta are doing underneath the hood.

Still, until more complex systems are available, I enthusiastically recommend working backwards from the visitor engagement metric to examine search engine traffic in an attempt to optimize those engines/phrases driving the most “engaged” visitors.

Bud: Neat idea! I am still working out how to demonstrate the statistics driving the engagement calculation but your system makes sense. In fact, I was just in New York last week working with a client that is trying out the same model! The only issue I have with your model is that in my experience it is hard to score visitors using currently available technology (but perhaps that will change over time.)

Thanks to you both for your comments!

Steve Jackson

Hi Eric,

I think this is very comprehensive. It covers all of the bases, you could also use some of the elements you describe to segment audience sessions not just single visitors. In many cases honestly I think it will be too complex for web analytics practitioners to use all the factors in their metrics. On the other hand, I can also think of a lot of companies that could use this.

Good post.
Steve.

eric

Steve: Thanks. I was telling a reporter today that I may have inadvertently given folks the impression that this metric is designed to highlight the activity of individual visitors. Sure, you can do that, but the metric is really more useful for traditional marketing and site design dimensions like referring domain, campaign, page, and content group.

When you say that the metric is too complex do you mean too complex to implement and use or too complex to understand? Either way, hopefully my next post will help change your mind about that (but thanks for voicing your concern and being willing to disagree!)

I hope all is well in Helsinki and thanks for your comment and for covering the post in your blog.

L'engagement des visiteurs : métrique du futur ou sombre fumisterie ?

[...] symbolisant l’incapacité des responsables de site à définir des KPIs, et ceux qui tentent de le formaliser, le débat n’est pas [...]

Larry Freed

Eric,
A very interesting idea. I do struggle with a couple of concepts, which hopefully will make for interesting discussion.

Is visitor engagement a measure of success? Sometimes yes, sometimes no.

First you need to define. “what is success”. The objective of websites differ from one site to another and from one audience segment to another. Does the visitor engagement calculation translate to success? Some of the elements of the visitor engagement calculation can have a contradictory impact. Higher duration, higher clickdepth and frequency of feedback do not indicate either success or failure, and could indicate frustration and failure by a user.

So what is the right measurement of success? How about, are your users satisfied and did they accomplish what they wanted to accomplish.

eric

Larry: First, I’m honored to have you comment on my blog. I’m a huge fan of your work!

Second, I’m not sure that visitor engagement is actually supposed to be a measure of success — isn’t that what “conversion rate” and “satisfaction” are measuring? I don’t think we need to call metrics “engagement” if they already have terms in use, which is exactly why I’ve taken the liberty to provide my own new measure of engagement as a straw-man for discussion, etc.

You comment that high duration, high click-depth, and frequency of feedback don’t indicate success or failure actually is the perfect example of why my metric is useful. Without analyzing the underlying data more closely it’s impossible to determine if the session is positive or negative. But in my humble opinion a long session with a lot of clicks and a lot of feedback is a strong indicator of engagement, independent of satisfaction or success!

So what I would suggest is that companies get used to the idea of measuring conversion, satisfaction, AND engagement as independent measures of visitor behavior. And, in the same way I recommend NOT including the primary conversion event in the Feedback Index (Fi), I recommend not including a calculated measure of satisfaction (like the ACSI score) in the Feedback Index. Instead I think more raw inputs that are direct measures of engagement are to be preferred (and would defer to your expertise to suggest what the most appropriate questions would be in a given situation … ;-)

Anyway, I’m glad I made you struggle with the calculation since it means I’m getting the best and the brightest to really think about what this operational measure of engagement might add to the landscape.

Thanks again for your comments!

Steve Jackson

Hi Eric,

When I said complex for many practitioners, I mean that from an implementation perspective. Understanding the idea is probably going to be fine for most people. Not that I’m saying web analytics is easy, but you explained it well. ;o)

On set-up for instance I can see how it’s a five minute job to set-up a agement with all those factors included in a tool like Visual Sciences. Unfortunately not many of the companies I work with use it.

On analyzing indivduals, Google Analytics for instance would require a complimentary system that measured individual usage and even then combining the data would be challenging. Omniture is in a similar position, as is CoreMetrics. So it makes it difficult for companies running those tools.

On analyzing mass audiences as most of the tools do I think it’s a superb set of measures, used individually or in combination as required by each business and as I said very comprehensive.

Thanks
Steve

eric

Steve: Your post is especially interesting in the context of Omniture buying Visual Sciences just now. So OMTR and WebTrends both have the ability to make this kind of calculation … others will inevitably follow, don’t you think?

Additionally, I have been talking to a number of the vendors and it’s not that this calculation is ** impossible ** for them to make, just that they’re not set up to do it today. Hopefully down the road this kind of metric will be “baked in” to the applications such that thresholds can be set, etc.

Again, thanks for your comments!

Larry Freed

Eric,
In your response to my response….Good points. Although, I still think there needs to be some separation between “good” engagement and “bad” engagement.

William C. Westmoreland, American General who commanded American military operations in the Vietnam War once said, “Militarily, we succeeded in Vietnam. We won every engagement we were involved in out there”. That is my idea of a bad engagement!

For more on the topic, check out my recent blog post.
http://www.freedyourmind.com/freed_your_mind/2007/10/on-engagement.html

-Larry

Web Analytics Demystified » Blog Archive » Nick Arnett challenges my visitor engagement calculation

[...] of the creators of Buzzmetrics) posted a very well though-out and moderately critical assessment of the visitor engagement calculation I wrote about earlier this week. Nick makes some great points and I thought it was worth addressing them while I prepare the [...]

Sébastien Brodeur

How to you conciliate Click-Depth for a 10 pages sites vs a 1000 pages sites?

Surely a 2 pages Click-Depth on a 10 pages sites versus a 5 pages Click-Depth on a 1000 pages sites is better?

I think that engagement should be calculate uniquely on each site. Compare yourself with yourself.

Bloglinks der Woche (KW 43) | Webanalyse & SEO - News

[...] How to measure visitor engagement, redux (Webanalyticsdemystified) [...]

Web Analytics Demystified » Blog Archive » Example uses of the visitor engagement metric

[...] post last week on measuring visitor engagement was pretty long by the time I outlined the calculation, so I put off publishing examples of how the [...]

Henry

Hi Eric,

First time here, and I think your proposal is really interesting.

Question, do you think values or weighting should be applied to any of your indexes or variables in the equation?

It seems that certain consumer actions by virtue are more valuable to a brand than others (i.e. x clicks vs x feedback). It very well could be that I’m lost, but it seems that the off-setting mechanism in your equation is a restriction of “n” for each index.

Why not incorporate an implied value to each action?

Henry

eric

Henry: An excellent question! I have toyed a lot with the idea of weighting some/all of the component indices but the problem is this: I don’t really have any a priori way to determine which index deserves which weighting factor. I’d have to conduct more analysis to determine whether “duration” is more important than “click-depth” and would need to compare each component to another goal (perhaps conversion or satisfaction.)

Not to say this is impossible, I just haven’t done it yet.

Also, the visitor-based scores (Loyalty Index and Subscription Index) do provide some nominal weighting (albeit small). When you’ve come to the site more than five times, or if you’re a blog subscriber, you get “points” for that.

It is definitely an area I’m looking into so keep watching this blog for an update. And thanks for writing!

Bryan Cristina

Wow Eric, you definitely are crafting some nice tools for Mathemetrics (my word :D)

At least someone is trying to tackle some way of figuring it out. It’s one of those nice measurements that never has been truly defined and changes from client to client and even day to day.

eric

Bryan: Thanks for your feedback! It’s great fun to work on the definition and a lot of really bright people are helping out.

Calculer l’engagement de vos internautes « Yesak Web Analytics

[...] > Voir l’article sur le calcul de l’engagement version 2007 > Voir l’article sur le calcul de l’engagement version 2006 [...]

Engagement is an Excuse for Deeper Analysis | plusoneanalytics.com

[...] The other bloggers came to the defense of the web analyst and to the web analytics industry itself. Eric defined engagement as: [...]

It’s all about the numbers | Daver's Blog

[...] engaged are they or how engaged arent they (which is its own post)? How many actually read your RSS? How [...]

Il branded search aumenta il visitor engagement? « MediaMeter

[...] il guru del web analytics, ha appena pubblicato un esempio dell’applicazione della metrica del visitor engagement al suo sito: WebAnalyticsDemystified. Il guru del web analytics ha definito il visitor engagement [...]

Judah Phillips at Web Analytics Demystified » Blog Archive » Thinking Back on Online Metrics in 2007 and Looking Forward to 2008

[...] Engagement, Engagement, Engagement. One of the hot topics in 2007 was a carryover from 2006. Definitions for “engagement” emerged from the worlds of advertising, social media, online metrics, and more. Engagement has been described as “turning on a prospect to a brand idea enhanced by the surrounding context” to “repeated, satisfied interactions that strengthen the emotional connection a customer has with the brand” to “apparent interest” to the more metrical “estimate of the degree and depth of visitor interaction against a clearly defined set of goals.” “Engagement” is very specific to the site being measured and full of nuance. This fact has led agencies, consultants, and various companies to create complex engagement indices consisting of measures of key behaviors. Behaviors are tallied and segmented in order to calculate an engagement metric, which is then used as the basis for site optimization. These indices go far beyond often-cited simple time-based measures of engagement. For a well-thought-of example, see Eric Peterson’s Engagement Metric. [...]

Web Analytics Demystified » Blog Archive » How to measure visitor …

[...] dushyantverma wrote an interesting post today onHere’s a quick excerptBack in December of last year when I first posted on measuring visitor engagement, I hardly imagined how much interest the topic would generate. [...]

9 experts’ thoughts on engagement » Ali Bytes » byte-sized strategies for modern living

[...] Eric Peterson: “Engagement is an estimate of the degree and depth of visitor interaction on the site against a clearly defined set of goals…I believe the visitor engagement measurement to be perhaps the most important of all ‘Web Analytics 2.0′ measurements.” Eric measures this, in my own words, by what the user sees, how recently, how often and how long. [...]

The Blackbeak Blog…. Arr! Web Analytics » Blog Archive » Measuring Online Engagement Re-visited and introducing the REAN model

[...] in this post I see agreement largely with what Eric Peterson wrote when he published his Engagement formula (also very controversial). Theo basically says “that a customer’s degree of engagement is [...]

Dennis Gorelik

Eric,

Could you give an example of recommendations that you made based on Engagement metric?
So far it’s not clear how to make this metric actionable/useful.

eric

Dennis: You may want to read back through the multitude of posts I have written on this subject, perhaps starting with this one:

http://blog.webanalyticsdemystified.com/weblog/2007/10/example-uses-of-the-visitor-engagement-metric.html

but looking back through the entire thread:

http://blog.webanalyticsdemystified.com/weblog/category/engagement

Suffice to say, the visitor engagement metric is actionable/useful in exactly the same way a metric like bounce rate is. Visitor engagement will differentiate elements in the dimension it is applied to and give you the basis for additional study.

You know, analysis. Because remember, no metric is immediately actionable or immediately useful. That is a myth.

Keep watching my blog because I’m about to release a tool that will let you approximate the calculation using Google Analytics (which I see on your site)

Thanks for your comment!

Creating of Key Perfromance Indicators Web Analytics Back to basic series | Web Analytics India Blog

[...] How to measure Visitor Engagement [...]

Web Analytics Demystified » Blog Archive » Measuring Engagement Online: The Next Stage

[...] the last few months there has been a tremendous surge in interest in my framework for measuring engagement online. Lately, some of the largest and well-known companies in the world have approached me about working [...]

Web Analytics Demystified » Blog Archive » Measuring Online Engagement: Step One

[...] to Mr. Carrabis: “Eric’s already posted that I’ll be working with him to make the formula more applicable to a wider variety of interfaces with greater general use features. I also know [...]

MetaMuse

Affinity vs. Engagement in Advertising…

Others Online has been working on some interesting reports for our partners to summarize in aggregate form what their users care about. As a matter of labeling, we’ve discussed the term affinity vs engagement. The latter has been thrown around…

Comment analyser le comportement des visiteurs — Les Bonnes Fréquentations

[...] T. Peterson a présenté une équation permettant de mesurer le degré d’engagement d’un visiteur. Quoiqu’intéressante, cette technique a été critiquée, notamment par le gourou du Web [...]

KPI Besucher Engagement - User Engagement nach Peterson | Webworking Blog

[...] zu Berechnung findet man direkt bei Peterson: How to measure visitor engagement, redux – eine kritische Betrachtung Tags: engagement, Web [...]

Community 2.0 Recap Part II: Measurement and Analytics 2.0 « MyCancerHub.com Blog

[...] account for the quality of content. A much more comprehensive calculation is covered in this post by Eric T. Peterson on WebAnalyticsDemystified. Eric’s measurement compiles the following [...]

Me gusta engagement | blog.entornao.com

[...] los conceptos que van asociados al comportamiento de las personas, el engagement, resulta un tanto complicado objetivizarlo para poder establecer una métrica sobre el mismo (ya hablamos sobre la importancia de la medida). [...]

Blackbeaks Blog….All things Analytics - » Web Analytics and Visitor Engagement… Again!

[...] blog (late 2006) and then when Eric published his engagement formula became the legendary “engagement” debate I’m sure we all remember, on Occam’s Razor, Jims site, my site and a bunch of others. [...]

Dave Manning

Hi Eric,
Great article but I found myself seeing the calculation of the Interaction Index differently. You say you’ve debated with including examination of “commerce transactions and other conversion events of fundamental import to the site”, but left it out of the equation as it already has a Key Performance Indicator – conversions. Furthermore, the second reason you give for not including that data into your calculations is “the visitor engagement metric is designed to provide information about the large number of visitors who do not convert”.

Ok, so here’s my question; what about those who ‘attempt’ to convert thru the primary activity of the site but fail to do so, for whatever reason? Isn’t that a critical piece of information that might be missed by your current formula? I think we would all agree that a consumer is engaged in your site if they are attempting to complete a primary transaction, but if they fail to convert due confusion over the process, too many pageviews necessary etc. etc. – this seems to me to be an unmeasured loss of audience, at least as the calculations are made now.

I’m not an analytical wiz but that was my initial thought when reading thru your explanation – that ‘attempted’ conversions should also be measured (albeit how is a totally different can of worms).

Did I miss something?

eric

Dave: I’m not sure you missed anything … I probably didn’t emphasize the idea enough. In a retail environment you would want the Interaction Index (Ii) to include the “Add to Cart” button, the “Checkout Now” button, and maybe even some of the steps in the cart as they are clearly important to a retailer’s notion of “engagement.” But as I said, by leaving the actual transaction out of the Interaction Index keeps that particular bias out of the calculation and better allows the direct comparison of Visitor Engagement and Conversion rates because they’re functionally measuring different aspects of the same thing (visitor tendency to purchase.)

Does that make sense?

It is worth noting that Joseph Carrabis disagrees with me on this point, something we’ve documented in our upcoming white paper on the subject. Given that Joseph is ** a lot ** smarter than I am, he’s probably correct and I may be forced to re-examine my assumption here ;-)

Thanks for your comment!

The Toolbox « Consult Keith

[...] continue to grow. It should – it’s an emerging medium and consumers’ time spent and engagement with it continues to grow. I’m not sure why there is a giant difference between good [...]

Customer Engagement - Web Analytics Reloaded? » BlogZwoNull

[...] Beitrag von Eric T Peterson in seinem eigenen Blog zum Thema Engagement [...]

Stéphane Bouchez » Blog Archive » Web Analytics and Visitor Engagement… Again!

VisualRevenue | 27 most popular Web Analytics blog posts of 2008

[...] How to measure visitor engagement, redux [...]

Marketingsherpa Blog : » 5 Favorite Articles from 2008

[...] how “engaged” visitors are with your website is not easy. This article broke down an engagement calculation created (mostly) by metrics pioneer Eric Peterson, CEO and Principal Consultant, Web Analytics [...]

Eric T. Peterson Doubts the Importance of Twitter « Greater Returns by Aaron Gray

[...] As I write this Chris Grant and John Hawbaker are having a conversation on Twitter about the the engagement model Eric Peterson has proposed. [...]

smcnally

Thanks for this terrific and detailed information regarding quantifying visitor engagement.

Understood you state that “This is a model, not an absolute calculation for all sites.” That said, is there any information regarding the adoption of this calculation that you’re able to share with your audience?

That is, of the folks with whom you consult and interact and “just know about,” are you seeing your specific engagement calc being put to use in the “real world?”

I would like to put it in use; having “case studies” or efen points of reference would help me in those efforts.

Again, many thanks for making this great info available. Please let me know if I can clarify or amplify my question at all.

Grab The Low-Hanging Fruit In The Travel Industry - ‘Hand-Raisers’ | The Single Entry Point Blog

[...] such as Eric T. Peterson from Web Analytics Demystified, calculate engagement in a more comprehensive manner. His model takes in consideration indicators such as click-depth, [...]

Add a Comment