Clint is asking hard questions again. This morning we find him exploring the boundaries of how traditional web analytics applications are able (or rather, unable) to effectively measure the emerging Web 2.0 world.

The essence of his post is that currently available web analytics technology is limited in what can be known and what can be reported, leaving you in a “you don’t know what you don’t know” situation. This is further aggravated by a lack of definitions and standards, which Clint wisely points out: You think varying definitions of “unique visitors” is a problem? Try figuring out how to accurately measure which weblog post should be counted from a page that has dozens of posts …

Clint’s complaint echos Xavier Casanova’s post from Monday about the gap between what traditional web analytics providers are able to provide and what he needs to gague success at Perenety. While I disagree with Xavier about his thesis that product issues are tied to financial performance in publicly traded web analytics vendors, the notion that a more flexible measurement framework is necessary in the emerging Web 2.0 world is spot on.

Clint reflects:

“Right now, I think that one of the most important steps that web analytics providers can take is to open up their platforms/architectures so that I, as a customer, can define new metrics as-I-go and have them show up where I want, how I want, when I want in the vendor’s various reporting tools.”

Since Clint lives in Southern California I offer this response:

Clint, call me. We’ll do lunch.

Matt Jacobs is back from vacation and he’s back on the money with his post this morning titled Web Analytics Data Quality: “Truly Appalling”. In his post, Matt summarizes complaints about web analytics data voiced at DM Days in New York (a summary of the panel can be read here)

Based on his post, it sounds like data quality and consistency is a topic that Matt has dealt with a bit with his clients. To this end, he offers some sage advice. My favorite is this:

“Sadly, I’ve seen my fair share of vendor selections that were based on one vendor having a more desirable feature set than another. Do not fall victim to that mistake, no matter how sexy and cool the user interface is. After all, what good is that feature set if the data itself is not as reliable as it should be? If you consider the online channel critical to your business, you don’t have room to compromise.”

Now, read between the lines here and recognize that I work at Visual Sciences, a solution that I strongly believe provides both an incredibly cool user interface and the industry’s leading data collection and data management strategy.

My bias aside, Matt gives great advice! Don’t just look at the interface, ask the hard questions: Ask about cookies, ask about how visitors and visits are defined, ask about data collection and storage options, ask about data verification strategies … take the time to research the technology, not just the user interface.

Doing your homework isn’t going to make data consistency issues between vendors and technologies disappear, but it will help you better understand why these differences exist. Sure, it’s reasonable to assume that “unique visitor” would have a single definition across vendors and technologies, but any of you who have worked with different solutions for any amount of time realize that a single definition simply does not exist.

Regardless of how you feel about data quality on the Internet, you have some obligation to your organization to understand what each of the dimensions, metrics and filters you’re using are actually telling you. I think you’ll find that when you have this understanding, the need to tear apart each report in an attempt to reconcile numbers goes away. Then, hopefully regardless of inconsistency observed in absolute numbers, you can begin to use the data to drive action.

UPDATED: Okay, so Mr. Casanova surely isn’t pleased with how that all went down. What in god’s name was Zinedine Zidane thinking? Zut alors!

Congratulations to your countrymen, Xavier, on making it to the World Cup finals! Xavier is now only hours away from having 20 more dollars to spend on his new passion.

I saw that Avinash’s recent post on The Awesome Power of Data Visualization is on the Digg home page today. Wow! Talk about a traffic driver …

Too bad all the comments are about the Government Spending visualization and not Avi’s great commentary about visualizing “web insights.” I wonder how an updated graphic will look with traffic from digg.com?

Congrats, Avinash!

I got an email last week from Marianne Llewellyn at 2xL Consulting (who, if memory serves me correctly runs consulting services at Coremetrics) asking for volunteers to write content for the upcoming University of British Columbia/Web Analytics Association course on “Creating and Managing an Analytical Business Culture” and I have to say I’m very excited.

All of the coursework that I’ve seen from the UBC/WAA so far has been great (did you know you get all of the lectures from courses one and two on a companion CD when you buy Bryan and Jeff Eisenberg’s Waiting for your Cat to Bark?) but this course looks to me like the rubber hitting the road, everything you need to know about being successful with Web Analytics from vendor selection to organizational implementation. In my opinion, too many people are still focusing only on subtle variations in the numbers and not really working on how to use analytics technology to drive business success.

I wish I could encourage you to sign up for the course but it’s already completely sold out. Hopefully UBC will add a second session sometime in early 2007. Anyway, lucky you if you’re already signed up for the course.

If you’ve taken some of the previous courses, I’d love to hear your comments about how the courses have gone.