Archive for 'Web Analytics 3.0'

Blogger Avinash Kaushik kicked off a little debate in the blogosphere a few weeks when he declared:

“Engagement is not a metric that anyone understands and even when used it rarely drives the action / improvement on the website.

Why?

Because it is not really a metric, it is an excuse.”

Suffice to say, some pretty bright folks disagreed with Avinash, openly and vocally. Anil Jasra has a good summary of a panel from WebTrends Engage where Gary Angel, Andy Beal, Manoj Jasra, Jim Novo and Jim Sterne all apparently voiced their opinion that engagement is a metric, not an excuse.

Perhaps ironically, in an interview with Eric Enge from February of this year, Enge asked Kaushilk about my long series of posts on measuring engagement (emphasis mine)

Eric Enge: Another thing I read about recently was Eric Peterson’s notion of an engagement metric. Can you comment on that?

Avinash Kaushik: Sure. You know that Eric is obviously a leader in the industry. We are all following the trail that Eric has blazed. He is just an awesome guy and a really great thinker. And, in terms of the specific post that you are referring for engagement, I think Eric’s initial proposal for the methodology is a very good one, and it does extend the conversation in terms of what it is possible for us to measure, because Eric obviously has access to some pretty good tools that allow for deeper analysis. But my preference is to ask a random sampling of people, or every single person who comes to website, are you engaged, here is my definition of engagement, do you like this site or product, are you going to recommend it, or whatever is the case.

Now, to be fair, I agree with part of Avinash’s argument — qualitative data is a valuable input into measuring visitor engagement — I just don’t think qualitative data is the only input. Nor do I think that it is “nearly impossible to define engagement”. For over a year I have been calculating visitor engagement on my site using the following equation:

Looks complicated, huh? It is. But if you’re running a site like mine where the major outcome you’re trying to create is simply not measurable online, wouldn’t you like to have some reasonable proxy that would help you identify where your best leads are coming from, what those leads are looking at, and who your highest quality leads actually are?!

I know I do.

Obviously the equation above doesn’t tell you very much. If you want to hear the rest of the story, you have two options:

1. Come to my Web Analytics 2.0 presentation next Wednesday at 1:30 PM in the Blue Ballroom at Emetrics
2. Wait until next Thursday and download my updated Web Analytics 2.0 presentation from my web site

Ironically this little debate prompted me to stick the long-awaited explanation of how to measure and use visitor engagement into my Web Analytics 2.0 presentation. Thanks to Avinash for kicking off a nice (if a bit lopsided) debate!

See you in Washington!

Stephane at immeria has a blurb about Avinash Kaushik’s video on Web Analytics 2.0 and my post this week on Web Analytics 3.0 that I started responding to in a comment. But as typical of me the comment got really long so I will just publish it here and link it to Hamel’s blog.

Stephane, good point that I didn’t explicitly define Web Analytics 3.0 … something for a follow-up post to be sure.

To your point:

“The Web and Internet ecosystem encompass quantitative and qualitative elements, physical and virtual organisms, online and offline interactions that are functioning together within legal, ethical and technological constraints. From that angle, things like a website, competition or location can’t, by themselves, explain the complexity of what’s going on. They can merely improve the science of analysis that will eventually lead to better insight.”

While it is difficult to disagree with you, I think you’re making the same argument that Charlene Li of Forrester made regarding her definition of engagement — she commented that engagement can be indicated at a minute level, such as when a flashy print ad catches your eye. Sure, but how the hell do you MEASURE someone noticing Charlene’s flashy print ad? And how do you MEASURE your legal, ethical, and technological constraints?

Kaushik and I are in near complete agreement about Web Analytics 2.0, and I thought he did a pretty good job explaining it. A lot of people have been saying the same thing as Avinash and I for over a year (Larry Freed pops to mind). An important distinction is that both the Web Analytics 2.0 and Web Analytics 3.0 paradigms are focused on tangible, measurable aspects of our (online) lives. And, in my humble opinion, the measures we take should be practical to make.
So I agree with you, it’s not about “e” business but rather about simply doing business, you’re spot on there. But here is the problem:

Web Analytics 1.0 was a full-on after-thought … not just for companies like yours but for the entire Internet. First we had web sites then later (more or less in 1995 if you believe most time-lines) we had measurement tools built to hack web server log files (poorly) and to try and cobble together some semblance of visitor behavior. A ton of R&D and money has gone into refining Web Analytics 1.0 and today we have JavaScript page tags and sophisticated applications that are basically still an after-thought for most companies.

Web Analytics 2.0 is also an after-thought, at least for the most part. I mean, we’ve had the qualitative data in systems like ForeSee Results and Tealeaf for years, so why is it only now that we’re actively talking about combining these data into a more holistic view of the visitor? We’ve had multivariate testing systems like Offermatica and SiteSpect for years, so why is it only now that we’re actively talking about using the combination of qualitative and quantitative data to drive action? (FYI, you can download my Web Analytics 2.0 presentation from my web site if you’re interested in more of my views on the subject …)

So I guess what I’m getting at by talking about Web Analytics 3.0 at this early stage is this:

Wouldn’t it be nice if the global solution to measuring the inevitable state of “digital ubiquity” wasn’t another after-thought?

Wouldn’t it be sweet if the platform providers and device manufacturers, the standards bodies and the compliance police, all came together now instead of 10 years from now and asked “How in the world will we measure all of this?” Personally, I think so, that’s why I’m starting the conversation more-or-less five years ahead of time, so that this time we’re not all standing around trying to figure out how to answer good business questions using incomplete and inaccurate data.

Call me crazy …

So yeah, I am probably still right and wrong. And yes, you make a good point — Kaushik and I were both caught navel-gazing (again!) But if in 5 years you and I are banging around in the Yahoo! group asking people whether the “Nokia X5150J Revolution” accepts cookies and JavaScript I am going to be awfully put out, aren’t you?

Thanks very much Stephane for offering up an opinion other than “Eric and Avinash are both brilliant!” The ego stroking is great but this kind of stuff needs to be debated, openly and honestly in my humble opinion. Beers are on me in D.C.

If you’re reading the web analytics blogs, you’ve probably already heard about the recent presentations I’ve given on the subject of “Web Analytics 2.0″. The future of web analytics and the relationship between Web 2.0 technology and measurement is something I’ve been talking about for over six months — I actually have a Web Analytics 2.0 workshop that I regularly give that you can read about under Analytics Consulting on my site — but given that it is “conference season” it is no wonder that this subject is getting attention from other folks in the industry. I have given my presentation at Web Analytics Day in Brussels, SEMphonic X Change in Napa, and will be giving a variation on same at Jim Sterne’s Marketing Optimization Summit in October.

Due to demand, you can download a PDF of the presentation from the white papers section of my site. If you’re interested in learning more about Web Analytics 2.0, please give me a call and I’d be happy to discuss it with you.

Strangely enough, the slides that are generating the most interest and commentary are not those about the Web Site Optimization Ecosystem, the integration of quantitative and qualitative data, or the Web Analytics Demystified RAMP, but rather the few slides I included outlining my thoughts about Web 3.0 and what I am calling Web Analytics 3.0.

What the heck is Web Analytics 3.0?!

Before I can tell you what Web Analytics 3.0 is, I need to tell you what I think Web 3.0 is going to be. The good old Wikipedia basically dodges this by saying:

Web 3.0 is a term that has been coined with different meanings to describe the evolution of Web usage and interaction along several separate paths. These include transforming the Web into a database, a move towards making content accessible by multiple non-browser applications, the leveraging of artificial intelligence technologies, the Semantic web, the Geospatial Web, or the 3D web.

While I know that Judah is all hopped up on the notion of the semantic web, after having traveled to Tokyo and Europe in the past month, I find myself absolutely convinced that the next technology era will be characterized by our collective ability to access the Internet anyplace, anytime, using so many devices we begin to look back on computers much the same way young people do television today — as something nice to use when YouTube is unavailable. Rolf Skyberg, a disruptive innovator from eBay who I met in Rotterdam a few weeks back, called it “digital ubiquity” — the point where we forget that the Internet actually exists and take our ability to access information completely for granted.

Given so many sexy alternatives — 3D web, transforming the Internet into a database, artificial intelligence, and the such — why am I so convinced that in the next three years we’ll be talking about Web 3.0 when we talk about mobile phones and non-traditional browsers?

Easy. The financial opportunity available via the mobile Internet makes the billions transacted today look like pocket change.

Think about it:

• Most people in the U.S. haven’t seen QR codes in advertising yet, but they will.
• Most mobile users around the globe don’t have access to a brilliant browsing experience through their phones, but they will.
• Most marketers aren’t advertising on mobile platforms yet, but they will.
• The most-loved company in the world isn’t in the mobile market yet, but they will be.
• Most mobile platforms aren’t passing the phone number (or a derivative) along with HTTP requests, but they certainly could.
• Most mobile platforms aren’t passing along GPS coordinates along with HTTP requests, but they certainly could.

Just think for a minute about how your browsing experience might change if the web sites you visited remembered you and delivered a tailored experience based on your demographic profile (theoretically available via your phone number), your browsing history (accurate because you’re not deleting your phone number) and your specific geographic location when you make the request?

Now think about how the advertising buying experience would change if the same were true, not to mention behavioral targeting. I mean, given GPS and demographic data, the behavior being tracked could be “works downtown during the day, checks Facebook on his phone often, lives in the suburbs, surfs sports scores from his neighborhood bar.” The Starbucks web site could have a link at the top with a coupon to save $1 on my double-tall non-fat latte in stores 1 block, 2 blocks, and 5 blocks from my current location; the Best Buy web site could have an in-store promotion for the store I am standing in, targeted to my age and gender; and my search engine could disambiguate my searches based on my demographic profile, my geographic location, and my recent search history to serve me paid search ads designed to influence my geo-spatial movement, not just my likelihood to click.

Jeepers, huh?

Sure there are privacy issues, but given the intensely personal relationship most people have with their cell phones, and the fact that far more people in the world have mobile phones than computers (Gartner estimates 271 million units sold to end-users by Q2 2007) it is easy to make a convincing case for mobile computing and digital ubiquity defining the next technology era, much like social networking, AJAX, XML, and mashed-up business models define the current Web 2.0 era we’re living in today.

Okay, mobile is the future. So what the heck is Web Analytics 3.0?

If Web Analytics 1.0 was all about measuring page views to generate reports and define key performance indicators, and if Web Analytics 2.0 is about measuring events and integrating qualitative and quantitative data, then Web Analytics 3.0 is about measuring real people and optimizing the flow of information to individuals as they interact with the world around them.

Your log file analyzer can do that, right?

The current state of mobile measurement isn’t about Omniture and Visual Sciences, it isn’t about JavaScript and cookies, and it isn’t about page views, visits, and visitors. Web Analytics 3.0 is going to be something completely different, and it will depend on completely new technology. Anil Batra and I talked about a project he did a few years back while he was at digiMine — he hacked together WAP gateway logs into a pseduo-log file, using the phone number in place of a cookie. Brilliant, and the fact that Anil has this experience propels him to very near the head of the class for Web Analytics 3.0 analysts.

In theory, the mobile Internet has many of the same measurements as the hard-wired Internet. But as the information the platform and device providers make available changes, something I very much believe will happen, the quality and volume of information at our disposal will increase and improve. The W3C document on “Mobile Best Practices 1.0″ already exists but surprisingly enough don’t have a section about logging requests or measuring user interaction. M:Metrics is out there providing analyst reports, but the service is more similar to comScore and Nielsen than WebTrends and ClickTracks.

This post is already extremely long but I wanted to start the conversation. In future posts, as time allows, I’ll expand on some of what I believe is possible and how. In the interim, let me know what you think! Am I wrong? Is Web 3.0 bigger than mobile? Or do you already have a handle on measuring your mobile content, even without GPS and phone numbers as unique IDs? Do you personally have experience doing analysis on mobile content? If so, I’d love to hear about your experience.

As usual, I very much welcome your comments but am happy to receive your comments directly via email. Also, if you’re a mobile service provider or device manufacturer concerned with how advertisers and marketers will measure their success through your platform, application, or device, I would love to talk to you about the Web Analytics Demystified vision for Web Analytics 3.0.