Archive for 'Engagement'

A lot of you have been following the thread in my blog about measures of engagement on the Internet. Over the past year we have certainly had a spirited discussion about the topic, and for the most part people’s interest in the subject has not apparently subsided. About six months ago I started working with Mr. Joseph Carrabis from NextStage Global on the engagement calculation and the byproduct of our work is now available as a somewhat lengthy white paper on the subject freely available to all.

You can download the white paper from the Research > Published Research section of this web site:

http://www.webanalyticsdemystified.com/link_list.asp?l=Research

The white paper includes a great deal of information about the calculation including background on it’s derivation, the calculation itself, it’s use in a business context, and the underlying mathematics.  I welcome your feedback on the paper and am more than happy to discuss the contents via phone or email.

The direct measure of a Visitor’s Engagement with a web site or set of properties is still a work in progress to be sure.  And despite some naysayers, I believe that all of us working with this or similar calculations are quite excited about the possibilities associated with moving on from more simple measures and beginning to combine metrics to create a more interesting (and potentially more valuable) view of visitor interaction on the Internet.

Let the debate begin again!

The same guys that want you all to believe web analytics is easy has now declared that “Visitor engagement formulas are largely another fad, just like parachute pants and the Hollywood diet. It’s a measure some consultants and vendors can pitch like snake oil.”

Omniture’s point that Visitor Engagement is a bad idea because it has subjective components fails to understand the work that folks like Jim Novo, Steve Jackson, Theo Papadakis, Joseph Carrabis and others have done; it makes me wonder if the author bothered to read anyone’s work on the subject.

Worse, it makes me question Omniture’s long-term commitment to Visual Site customers since Visual (= Omniture Discover OnPremise) is, at least for now, the industry’s leading solution for creating derived measures and experimenting with visitor-level data.  The point seems to be that simple measures of success, such as those provided by SiteCatalyst, are all that are required.

Hmmm …

We pretty much had this same debate a year ago when Avinash Kaushik disagreed with the use of calculated metrics to measure engagement, and I can see that Steve Jackson has already commented as such.  I wouldn’t normally have written about this except the author said one smart thing when he commented you shouldn’t “try to build a better mouse trap, when you’re not taking advantage of the one you’ve got today.”

Agreed.

If you’re thinking about trying to leverage any measure of visitor engagement, regardless of which measure you choose, you should definitely make sure your web analytics house is in order first.  Despite Omniture’s assertion, most people believe that web analytics is hard and requires a sometimes intense focus on people, process, and technology.  If you’re not staffed appropriately, if you haven’t defined your key performance indicators, if you haven’t established core web analytics business processes, and if you haven’t worked to optimize your web analytics implementation then trust me, Visitor Engagement is not for you.

A good analogy is the one provided in Tom Davenport’s book “Competing on Analytics” where he describes how baseball teams like the Oakland A’s and my friend Judah’s beloved Boston Red Sox, and football teams like the New England Patriots have used new and innovative metrics to evaluate the performance of players, concessions workers, and the entire fan experience.  Visitor Engagement is a new measure in web analytics, and thusly it will take a special type of analytical competitor to recognize the opportunity that this “uber measure” can potentially provide.  And just like some teams have shown that they are not ready to adopt new measures to evaluate their business, some companies are simply not ready to explore complex key performance indicators in an effort to “Compete on Web Analytics.”

If you’re like most companies doing web analytics today, it is likely that you will benefit more from focusing internally and learning more about how to leverage people, process, and technology more effectively, rather than look externally for new metrics of success.  You could get a good book on the subject of fundamental key performance indicators and spend a great deal of time implementing what you learn.

But if you’re interested in learning more about an innovative metric that describes the behavior and opportunity that exists with the 97% that don’t convert, a measure that you can apply to your advertising, content, B2B, marketing, or lead generation site that will compliment your otherwise robust key performance indicator suite, and a calculation that describes the level of Attention that visitors are paying to your site, your content, your testing and targeting, etc… well then I guess you’ll have to keep reading my blog (and Jim, and Steve, and Joseph, and a whole host of other people’s work who are committed to working these ideas out rather than just saying “balderdash!”)

If you’re not content to just keep reading and want to know more about my thoughts on Visitor Engagement, know this: I have been exceedingly clear that my measures of Visitor and Audience Engagement are new, and in their newness there is risk in the level of insight they may be able to provide you.  I am not promising you better skin, new hair, or more friends, despite the validation that the measurement of engagement recently received when NextStage was granted a patent for their work on the subject.  But, unlike some people, I have done my homework on the subject, and I continue to have conversations with some of the best companies in the world about how they can use new measures to improve their overall use of web analytic technology.

In the meantime, I guess I’ll put on my parachute pants, grab a glass of “Miracle juice”, and bust out the ol’ Snake Oil.

Since I am on the record as being supportive of the web analytics community around the world I wanted to make European readers aware of a webcast I will be doing next week.  Coremetrics has asked me to reprise the presentation I did on measuring visitor engagement that I did at their client summit last Fall.  The good news (for Europeans) is that the webcast is open to everyone and will be presented at 10 AM London time!

You can register for this free event at WebEx.

For those of you who can’t make the event because of holiday, or because like me the presentation will happen while you would normally be asleep, I am told that Coremetrics will be recording the presentation.  Assuming I am coherent at 2 AM my time and the recording comes of well, you should be able to download the webcast within the next few weeks at the Coremetrics web site.

I hope those of you in the European web analytics community will be able to join us next week.  I will try and leave plenty of time for questions and answers as well.

Following up on my post from Monday of this week announcing that Joseph Carrabis of NextStage Evolution will be joining “The Engagement Project” and bringing his mathematical expertise to the table, Mr. Carrabis has summarized what he’ll initially be doing for the chef in all of us.

According to Mr. Carrabis:

“Eric’s already posted that I’ll be working with him to make the formula more applicable to a wider variety of interfaces with greater general use features. I also know that I can always use help and have repeatedly and publicly stated that I don’t know web analytics.

So, first steps? A semantically exact statement of what we’re hoping to measure. I suggest this step because it’s much easier to know if your variables will result in the desired solution if you are exact in what the solution looks like and what you have to put into that solution.

Think of it this way; You want to make some chicken soup and you use your grandmother’s recipe. I want to make some chicken soup and I use my grandmother’s recipe. But your grandmother is Irish and mine is Italian. I’ll bet we’d use different spices, different vegetables, different noodles (if indeed we both did).

But I’d bet we both use chicken stock as a base. And is your chicken stock from the leftovers of a roast chicken? What spices did you use there? Or is your stock from bullion?

So the first step is to decide what we all mean by “chicken soup”. One of my mentors was a genius of an author who use to write “speculative fiction”. I would ask, “What is speculative fiction?” and he’d reply “It’s what I’m pointing at when I say it.” This is a great anecdote and an undefensible statement (except in cultural anthropology). If one person “owns” the definition of “speculative fiction”, “chicken soup” or “engagement” then that definition is only valid so long as there exists a market for that definition.

However, a definition that says something like “Basic Chicken Soup”, that is something I can start with to make “Italian Chicken Soup” and allows my Irish friend to extend it to “Irish Chicken Soup”? Now that’s a good definition.

I snuck the concept of “extendable” into the above. “Extendable” means the definition accommodates special cases (Italian, Irish, etc). Think of a recipe for Italian Chicken Soup that begins “Step 1: Make the Basic Chicken Soup. Step 2: Now add garlic, oregano, …” That “Step 2″ part means that the original definition isn’t limited, that it can be extended to incorporate specific features to make it unique to a given environment (Italian, Irish, …).

The concept of “extensible” has two parts; First, you can substitute one thing for another if they share some basic properties. For example, you can substitute a glass of wine for a glass of water in the recipe because they’re both liquids. You can’t substitute a lamb chop for a glass of water, though. Mathematically, this means that if we want to include “clickthroughs” we can use whatever product A calls clickthroughs, whatever product B calls clickthroughs, etc., so long as they all meet some definition of “clickthroughs” (I’ll let the WAA worry about things like that).

Second, “extensible” means new spices, new vegetables, new types of noodles, etc., can be used to make the chicken soup better. This means that you can add a new spice to your recipe in addition to the existing spices already in it. Extensible (in this sense) means you’re doing what you already do to make your style chicken soup and now you’ve discovered something more you can add to it to make even more “your style”. You’re not watering it down or adding more vegetables to make the soup go further. That’s scalability and the equation should be scalable without needing to define it as such.

The sum of these two concepts of “extensible” translates to “the equation is valid across all interfaces including those we haven’t thought of yet.” Mathematically extendability and extensibility form the axes of a very rich solution space.”

Joseph says “Basic Chicken Soup” and I say “a measure of the depth and degree of visitor engagement online” … clearly he and I both have our work cut out for us. If you’d like to join us in our quest for a better measure of visitor engagement online, please let me know.

In the last few months there has been a tremendous surge in interest in my framework for measuring engagement online. Lately, some of the largest and well-known companies in the world have approached me about working with them to bridge the gap between the metrics they have today and something similar to the composite metric I first described back in December 2006.

While I am tremendously flattered that I have somehow become the focal point for this conversation, I have been thinking lately about how the framework has been developed and how it might end up being used by the measurement industry in general. And while early tests using the framework I’ve described are very encouraging, the calculation in it’s current state was meant to move the discussion along and get more people to “think different” about how engagement could be calculated online.

Given that interest in the framework has clearly increased, one primary concern comes up again and again: the need to apply mathematical rigor to the framework and calculation so that A) the result is repeatable, reliable, and trustworthy and B) when naysayers inevitably emerge to criticize this small side project of mine, that I have a suitable response to their criticism, regardless of where and why it comes.

I believe that the need for “A” is obvious. The need to address “B” is perhaps less obvious, but I believe that I owe it to those of you who are investing your time, energy, and money into this framework. Especially as the stakes seem to increase exponentially with every presentation, every conversation, and every high-visibility blog post on the subject, I believe now is the time to approach the engagement framework not just as a hobby but as a serious project with committed resources.

To this end, I am extraordinarily happy to say that the single smartest person I know, Joseph Carrabis the Founder and Chief Research Officer of NextStage Evolution and NextStage Global, has offered to bring mathematical rigor and analytical precision to what I am officially dubbing “The Engagement Project.” Those of you not familiar with Joseph and his work are advised to A) meet him in person at the upcoming Emetrics Summit in San Francisco or B) read some of his recent work at iMedia Connection.

Joseph will be working to make the formula universally applicable and universally defensible. Suffice to say I can think of nobody better to bring mathematical and scientific rigor to the framework I have been evolving over the past year. Watch this blog and Joseph’s blog at BizMediaScience over the next week or so for a more complete analysis of the framework in it’s current state, something we’ve agreed is the first step towards creating a true function capable practically describing the degree and depth of engagement a visitor is displaying towards a web site over time.

At the end of the day, without regard to my framework, Joseph’s analysis, or any person or group’s particular position on the use of the word “engagement”, my goal is to solve one problem and one problem only:

If you’re interested in working with Joseph and me on The Engagement Project please feel free to contact me directly.