Omniture: Visitor Engagement is just a fad!

The same guys that want you all to believe web analytics is easy has now declared that “Visitor engagement formulas are largely another fad, just like parachute pants and the Hollywood diet. It’s a measure some consultants and vendors can pitch like snake oil.”

Omniture’s point that Visitor Engagement is a bad idea because it has subjective components fails to understand the work that folks like Jim Novo, Steve Jackson, Theo Papadakis, Joseph Carrabis and others have done; it makes me wonder if the author bothered to read anyone’s work on the subject.

Worse, it makes me question Omniture’s long-term commitment to Visual Site customers since Visual (= Omniture Discover OnPremise) is, at least for now, the industry’s leading solution for creating derived measures and experimenting with visitor-level data. The point seems to be that simple measures of success, such as those provided by SiteCatalyst, are all that are required.

Hmmm …

We pretty much had this same debate a year ago when Avinash Kaushik disagreed with the use of calculated metrics to measure engagement, and I can see that Steve Jackson has already commented as such. I wouldn’t normally have written about this except the author said one smart thing when he commented you shouldn’t “try to build a better mouse trap, when you’re not taking advantage of the one you’ve got today.”

Agreed.

If you’re thinking about trying to leverage any measure of visitor engagement, regardless of which measure you choose, you should definitely make sure your web analytics house is in order first. Despite Omniture’s assertion, most people believe that web analytics is hard and requires a sometimes intense focus on people, process, and technology. If you’re not staffed appropriately, if you haven’t defined your key performance indicators, if you haven’t established core web analytics business processes, and if you haven’t worked to optimize your web analytics implementation then trust me, Visitor Engagement is not for you.

A good analogy is the one provided in Tom Davenport’s book “Competing on Analytics” where he describes how baseball teams like the Oakland A’s and my friend Judah’s beloved Boston Red Sox, and football teams like the New England Patriots have used new and innovative metrics to evaluate the performance of players, concessions workers, and the entire fan experience. Visitor Engagement is a new measure in web analytics, and thusly it will take a special type of analytical competitor to recognize the opportunity that this “uber measure” can potentially provide. And just like some teams have shown that they are not ready to adopt new measures to evaluate their business, some companies are simply not ready to explore complex key performance indicators in an effort to “Compete on Web Analytics.”

If you’re like most companies doing web analytics today, it is likely that you will benefit more from focusing internally and learning more about how to leverage people, process, and technology more effectively, rather than look externally for new metrics of success. You could get a good book on the subject of fundamental key performance indicators and spend a great deal of time implementing what you learn.

But if you’re interested in learning more about an innovative metric that describes the behavior and opportunity that exists with the 97% that don’t convert, a measure that you can apply to your advertising, content, B2B, marketing, or lead generation site that will compliment your otherwise robust key performance indicator suite, and a calculation that describes the level of Attention that visitors are paying to your site, your content, your testing and targeting, etc… well then I guess you’ll have to keep reading my blog (and Jim, and Steve, and Joseph, and a whole host of other people’s work who are committed to working these ideas out rather than just saying “balderdash!”)

If you’re not content to just keep reading and want to know more about my thoughts on Visitor Engagement, know this: I have been exceedingly clear that my measures of Visitor and Audience Engagement are new, and in their newness there is risk in the level of insight they may be able to provide you. I am not promising you better skin, new hair, or more friends, despite the validation that the measurement of engagement recently received when NextStage was granted a patent for their work on the subject. But, unlike some people, I have done my homework on the subject, and I continue to have conversations with some of the best companies in the world about how they can use new measures to improve their overall use of web analytic technology.

In the meantime, I guess I’ll put on my parachute pants, grab a glass of “Miracle juice”, and bust out the ol’ Snake Oil.

Posted Tuesday, July 15th, 2008 | 15 responses | Add a Comment | Share, Save or Email

Brad Warthan

Great posting Eric.

I too disagree with the statement that visitor engagement/involvement “formulas are largely another fad, just like parachute pants and the Hollywood diet.”

Such engagement measures are helpful for measuring overall website success when you are trying how well your message comes across with the same audience over a given time period, such as blogging.

It would be great to continue our conversation sometime on engagement from the DFW WAW you attended back February.

Best regards,

Brad Warthan

eric

Brad: Thanks! It’s interesting that there is so much passion in this debate over numbers, isn’t it? I guess I fuel that a little since I’m enthusiastic about the results that folks are seeing when they apply measures of engagement — Steve Jackson’s examples are among the best but Novo’s work with RF(M) is also brilliant.

A friend commented to me this AM that Matt Belkin comes across as somewhat of a Luddite sometimes, always advocating for the most simple solution possible rather than exhibit any kind of real thought-leadership. I don’t know if I would go that far (after all, I did invite him to contribute to my “Hacks” book and he did a really good job with that) but some of his posts make me sit back and wonder about his ulterior motives.

I’d love to continue the conversation. Can you come to the X Change in August?

Thanks for your comment!

Ian Gilyeat

Adding my voice to those already posted – vistor engagement is not a fad. It’s an evolving science and practice, just like all other analytical disciplines. I’m confident that analytical professionals will continue to reach, stretch and redefine how we use analytical measures. Taking the concept of visitor engagement and deciphering how human senses determine human engagement will likely lead us in new directions and discoveries that can be applied to web sites and how engaged visitors become.

Todd

The Moneyball analogy is good, especially on this All-Star night. Metrics aren’t one size fits all, and it’s absurd that complex metrics are being held to the same standard as our building blocks.

There are always four bases in a home run, but in what situation was the home run hit? How much does that matter to who?

eric

Ian: I see you work extensively with other segmentation strategies like RFM so I very much appreciate your comment. I hope you’ll keep following this blog since I think you’ll really appreciate the white paper that Joseph Carrabis and I have coming out in the next few weeks.

Todd: Yeah, but some people are so entrenched in the limitations of their particular technology that they cannot see any other possibilities. This was largely my problem with Avinash Kaushik’s criticism of the work — just because Google Analytics can’t make the calculation doesn’t mean the calculation is bad. Sometimes you actually need the right tool for the job, not just a nice-looking alternative.

Oh well.

Anyway, I appreciate both of your comments and do hope you’ll come back and download a copy of the white paper. Thanks again!

Marketing Productivity Blog » Blog Archive » Friction Model

[...] There’s been another eruption of discussion on Engagement. If (like me) you’re more interested in the higher level ideas not so oriented towards the “tool” aspects of this discussion, make sure you catch this post and hefty comments. See also here and here. [...]

A long term engagement: Is engagement marketing valuable or is it bunk? | Persuasive e-Marketing

[...] on the subject. I specifically recommend checking out the recent flamewar between Omniture and Eric Peterson, both industry heavyweights with very differing [...]

Christian Vermehren

Hi Eric,

I completely agree with you that visitor engagement is not a fad. I see web metrics as a whole sub-discipline in itself, which has only just begun evolving. In order to understand “the 97% that don’t convert”, as you say, we definitely must continue to explore the possibilities of constructing new interesting web metrics.

Visitor engagement is an important subject and it is therefore understandable that this metric has received so much attention. However, we should not forget other important areas. Usability, for example, which comes immediately after engagement: once you have engaged the visitors, you also must “enable” them to do what they want. But how can we measure a site’s ability to enable?

I would love to hear you opinion on this, how you see usability in relation to engagement and how you would measure it. At Netminers we are experimenting with some new metrics – “Back Navigation Rate” and “Turn Page Rate” – which we think can be used to identify certain forms of usability problems. If you are interested, please read more about them here:

http://theartofwebanalytics.com/?p=9

Thanks for an interesting blog!

Steve Jackson

@all;

I understand where Matt is coming from though his post was confrontational to say the least. I think we all pretty much started out doing web analytics by measuring conversion rates.

I’d be lying if I said in 1999 when I got into this game that I was thinking about complex measures of engagement. I admit that basically I was thinking about mainly conversion funnels. I’m sure a lot of us were.

It was only when I started consulting full time in 2003 and learning the problems that clients have that I realized that conversion alone ain’t good enough. Since most businesses are not at a strong level of adoption I can see Matt’s take on trying to learn the basics first. The mousetrap analogy that Eric commented on.

However….

Matt says “we live in a wonderful world of objectivity (for the most part)”. Unfortunately I’d argue in most enterprises that is not the situation. It’s really difficult to get any wins with just the basics. The problem in enterprises is no longer lack of information it’s information overload.

Focusing on KPIs you can take action on is key to this and as I said before combining data sources so that they work together is key to this.

I wonder therefore whether Matt has fired from the hip without fully taking stock of the situation. Since he doesn’t seem to want to answer his own blog comments it’s left for us to speculate where he and Omniture stand.

Br
Steve.

Web Analytics Demystified » Blog Archive » JupiterResearch Web Analytics Buyer’s Guide

[...] caveats in all of web analytics: it’s not the tool that matters, it’s how you use it! Omniture’s own consultants make this point when they remind us that we need to work hard to take advantage of the systems we already have in [...]

eric

Steve: Excellent summary. The thing I find hardest to reconcile with Omniture’s “simple measures are best” mantra is that they have such amazing and powerful technology at their disposal. You’ve seen Visual Site, and now that I’ve had a much closer look at some of their other technology … I am shocked at this “let’s keep banging rocks together” message from the company.

But hey, maybe you and I, and Gary, and Joseph, and Theo, and all the other folks out there working on this problem are wrong. Maybe measures of engagement have no place in web analytics today. Maybe we should just go back to looking at impressions, visits, and bounce rates …

Time will tell my friend. Time will tell.

Thanks for the constant feedback.

Web Analytics Bloglinks der Woche (KW 29 / 2008) | Web Analytics News

[...] Omniture: Visitor Engagement is just a fad! (Eric Peterson) [...]

Andrew Hally

“Engagement” isn’t an online only notion. Increasingly, we here at Unica hear our customers’ senior marketers talking about customer engagement more broadly, often in lieu of “customer loyalty” which is becoming an increasingly quaint notion in today’s world of empowered consumers. I.e., we can no longer hope for truly loyal customers but we can strive to have engaged customers.

Alex B

Interesting argument.

I think that perhaps the best point to attack Matt’s argument is in his initial premise, or at least, a part of it.

” The basic premise of measurement is that you want to measure something so you can improve it, if necessary”

While he’s correct in this statement, what I think he’s failing to give adequate credence is that web analytics has a considerable portion of art to it/interpretation. While the numbers themselves may be objective, what they represent and therefore what needs to be changed to improve those numbers can be a myriad of factors from one site to another, or even one page to another.

He seems to imply that a mashup of metrics cannot lead to action, but I’d argue that almost all of the data in general, taken at face value, cannot lead to action and will warrant further investigation.

It also seems that, where these mashup metrics would be useful is signaling when something is wrong, or right, even if other data is indicating otherwise. In other words, if metric A/B/C all decline 2%, you might not be concerned looking at them individually, but a measure that combines the three going down 6% (gross oversimplification), could be cause for concern/investigation. Thus, while not perhaps directly leading to action, they can be useful in spurring the initial investigation which has some merit unto itself, and perhaps can help show minute improvement or decline where the other metrics fail to do so (thus spurring action in some way).

Finally, this paragraph somewhat surprised me.

“First, what kind of return frequency is “often” – two visits? Four? Six? That’s subjective. What is “important” content? The home page? An article? A support document? Subjective again. And what is a “long” time on site — 5 minutes, 10 minutes? Perhaps “long” means any visit that exceeds the average for the site that week?”

I’m failing to think of one metric in web analytics that isn’t subjective by this definition. What’s a bounce? 5 seconds, 10 seconds? 30 seconds? How do you calculate a visit? Server request, java call? Etc. What’s important content? Most visited, highest “index value” (what’s a good formula for an “index value?”), largest driver of revenue, etc.

Alex

eric

Alex: Thanks for your thoughts about Omniture’s blog post. I have to say I agree with you … the post fails to address the most basic question in all of web analytics: “My {X} rate is {Y}. Is that good?” The answer to this question, and I get this from my friend Jim Sterne, is almost always “It depends.”

If Omniture’s point is that we need to continuously improve against measures then I’m in complete agreement. But without a business-relevant starting point as the basis for that improvement — a starting point that Omniture seems to be saying is bad because it is subjective — then it’s hard to get started.

Maybe someone from Omniture could write in and clarify the company’s point on this issue?

Anyway, thanks for your comment Alex!

Add a Comment