I just received a complimentary copy of Megan Burns new report titled “The Business Case For Web Analysts: Dedicated Staff Turns Up The Volume On Productivity” and I was very impressed with the report. While I started talking loudly about the need to hire dedicated web analytics staff to fully benefit from any technology investment back in October 2004, Megan, Harley Manning, Jon Erickson and Caroline Carney went so far as to do the math and calculate the potential return on investment for this HR investment.

From the executive summary:

Companies need a way to see what customers do on their Web sites and how those activities contribute to business performance. Because Web analytics is the only practical way to do that, most managers aren’t asked to justify their initial investment in commercial Web analytics tools. But when they want increased funding — most often to bring on dedicated analysts who can make the most of the tools and data — they have a hard time making the case. To help companies decide whether or not to invest in full-time Web analytics experts, we developed a model of the economic impact that these people have on an organization. Our research shows that hiring full-time Web analytics staff brings in enough incremental value to cover the extra costs several times over.

(Boldface is my emphasis.) I’m on Megan’s DMA panel in Scottsdale so I won’t say anything else about the report for fear that she’ll ask me questions I don’t want to answer like “Who is Fred McMurry?” or “Seriously, you quit being an analyst. Are you nuts?” Still, despite being $349 (buy it from Alacrastore.com) for 12 pages of research, if you’re debating making this critical hire and struggling to convince the powers that be it would be money well spent, this report is $349 well spent.

… and I don’t just say that because the headline screams “Peterson Says Web Analytics Getting Better All the Time” or because they gave me a page-and-a-half in a six-page special report (PDF here). I was actually really impressed with the content in the report, written by many folks for whom I have tremendous respect, guys like Greg Drew, Jim Sterne and Warren Raisch. If you’re reading this blog and just toying around with web analytics, I highly recommend the report as a good primer covering much of what our collective technology is good for.

I also saw a nice reference to the article from Tim Seward at ROI Revolution in which Tim notes that I said something nice about Google Analytics.

Since comments are coming in from all directions I think it’s still easier to address them via blog posts. Especially those comments coming through the Yahoo! group which have a tendency to be quickly lost as that larger conversation progresses.

An anonymous poster commented that I had not specifically addressed RSS feeds and their measurement but was complimentary of my efforts thus far. Yeah, RSS is absolutely something I think falls into the cateory of “Web 2.0″ but not into the same bucket of measurement I’ve been talking about. All of us today have the ability to measure at least one critical aspect of our RSS feeds–how much in-bound traffic they generate–using nothing more that the core campaign analysis tools native to whichever appliciation you use.

But most people want to know more than just which posts drove clicks; consider Clint’s very well written post from July 13th on “Language Shift” where he rightly complains about the need for new metrics that speak directly to this new publication medium. I think Ivy says it best with the following:

What’s a unique visitor mean when what we care about is circulation and syndication? For my blog, I’m much more interested in the number of readers I have than how many unique cookie-based browser applications have visited my blog.

What’s a download worth when what I really need to know is how many listeners or viewers I have and how many times they have listened or watched my podcast or vlog?

What on earth is a referral in a world where it’s all about knowing who, how many and how often my post/blog/podcast/vlog has been quoted or tracked back to?

As this thing we call the ‘Internet’ or ‘Web’ evolves into a distribution platform from a content channel and destination what are the things that are important to measure and what will they be called? How will they be defined?

Indeed, the new metrics complicated by the fact that they’re not easily measured with the existing measurement paradigm. What will things be called and how will they be defined.

But I digress.

The reason I didn’t specifically address RSS in my original post was because what I was trying to describe is how other people’s content can me measured as part of your site. RSS is basically your content being measured as it is used in someone else’s site. Does that make sense? To accurately measure RSS I need a mechansim that has content report back “I was viewed here” wherever “here” may be; to measure Google Maps I need a mechanism to have someone else tell me “Your visitor did this thing” where “this thing” is an event.

Over in the Yahoo! group, Jason Egan from Scripps Networks reiterates the point I made about having the ability to measure and knowing what to measure are two completely different beasties. Jason comments that he’s already measuring RIAs but he points out that the move to a richer web environment has the potential to cripple his advertising-driven business model. Given that the current monetization point is “page views” defined traditionally, the loss of strict page refreshes caused by Flash/AJAX/etc. would reduce their available inventory and relative ranking as calculated by Nielsen/Hitwise/etc.

Yikes.

An excellent point but one I think that further drives the need to have this conversation. As the existing “governing bodies” are still working out some of the most basic definitions, the model is changing and these definitions are becoming out-of-date faster than the ink can dry on the proverbial paper. Having a measurement framework in place to count actions that occur outside of the existing page view model is certainly the first step towards understanding what ramification this new technology has on our extant business models.

Finally, Chris Harrington (who I assume is not the VP of sales at Omniture) comments that he’s already doing what I mentioned was certainly possible, manually embedding calls to a local database via the application API (SOAP in his case) primarily for debugging purposes. I’ve already written to Chris asking to know more about what he’s learned and, pending his permission, will share what I’m able in future posts.

Clint was able to comment on my last post late last evening but I tried to respond in comments but apparently something is wrong at Blogger so I’ll respond in a post and offer that if you’re trying to comment on my “measure Web 2.0″ post you can email me directly and I’ll publish the comment for you somehow.

Clint’s comment “Fie! The page view is NOT canonical in a web 2.0 world.” is spot-on correct and that’s a big part of the problem, I agree. But, and perhaps unfortunately, page view is the simplest and most significant form possible without the loss of generality offered by the vast majority of web analytics applications. So sure, in the AJAX and RIA models most of the interesting stuff happens below the level of a traditionally defined “page view” but if you don’t have any good way to measure that does it matter?

My proposal for measuring Web 2.0 assumes an end goal of integrating all site data into your current web analytics platform. For obvious reasons I’m not likely to advocate for additional silos of data, all hell bent on describing the exact same visitors in a completely different way.

So Fie! Back to you Mr. Ivy. I actually believe that the most basic event type in Web 2.0 is, and don’t laugh here, the “hit”. The same metric that I described in Web Analytics Demystified under the header of “A Small Group of Mostly Useless Terms” and that I recently told a packed room at Search Engine Strategies was an acronym for “How Idiots Track Success” is potentially returning to grace as critical to the measurement of this emerging application development model.

You’re laughing, right?

Seriously, the definition on page 46 in chapter four cites WebTrends saying “[A hit is] an action on a Web site such as when a user views a page or downloads a file.” Back in the day the fine folks at WebTrends almost surely didn’t envision measuring remotely located web-based applications like Google Maps but by capturing the idea that a hit is an action you can hopefully see where I’m going with this.

All we’re really trying to measure from Web 2.0 is actions which ideally tell us something about our visitor’s engagement. From that we can build a handful of swell key performance indicators like “Average Actions per Widget” and “Average Time Spent per Widget” and “Percent Widget Users” and so on.

Speaking of widgets, my boss serendipitously sent me a link from Fred Wilson pointing to information about how the Zevents widget supports web analytics. The guys at Zevents are absolutely thinking about it the right way but I think they need to take it a step further and make the information available via the strategy I described in my last post. They have a handful of good events described (”event views”, “venue views”, “searches”, etc.) but again, I think that the great utility of this type of reporting will come when I can integrate the data into my existing solution (whatever that may be)

So kudos to Zevents, fie to Clint and keep the comments (and email) coming!

I had the great fortune this week to have dinner with Bob Page (founder of Accrue, currently at Yahoo!), Xavier Casanova (founder of Fireclick, currently running Perenety) and Bryan and Jeffery Eisenberg (who need no introduction.) At one point Bob turned to me and commented that he had a new book he wanted me to write; Bob wanted to see my take on how to measure Web 2.0.

This is a great idea and something I’ve been thinking about a lot lately. Think about the inherent complexity involved with measuring activity on your web site, the one where you ultimately own everything. Now think about how you’d get this same type of robust measurement from a mash-up. When you have a Google map, a widget like Bitty Browser, a ZoomCloud tag cloud and more in your web page—all applications unto themselves that don’t provide ANY TYPE of reporting about visitor interaction to the best of my knowledge.

So AJAX and Web 2.0 present a whole new suite of challenges to anyone considering their use who is concerned about measurement. And I think this is a freaking fascinating subject personally–something that has the potential to change our industry–so I’ve decided to propose a measurement strategy for all Web 2.0/AJAX/RIA applications designed to be mashed into other web sites.

I mean, why not just suggest that Web 2.0 is incomplete without providing the ability to measure it’s adoption in a meaningful way? You know you we’re thinking it already, right?

So here goes …

I want to be able to, with any request for an external application, pass in a visitor identifier and have the application record event level data for me. Then, later, I want to make a different request to the application and get back event-level information for all of my visitors who interacted with the application, keyed to the visitor identifier I originially passed in.

The event-level data would differ by application type. A tag cloud would report back something like:

VISITORID001 DATE/TIME Clicked on “analytics”
VISITORID002 DATE/TIME Clicked on “peterson”
VISITORID001 DATE/TIME Clicked on “kpi book”
VISITORID003 DATE/TIME Clicked on “event api”

Whereas a more complex application like Google Maps might report back something like:

VISITORID001 DATE/TIME Zoomed map to level 4
VISITORID002 DATE/TIME Added directions to [ADDRESS]
VISITORID001 DATE/TIME Dragged map to center of [LAT/LONG]
VISITORID003 DATE/TIME Printed map

There would have to be a way to sync the dates and times such that these events could then be integrated into other clickstream data, essentially allowing these externally tracked events to be treated as “page views” in a clickstream path (”page view” being the canonical event type.) Still, those of you from the old guard–back when all we had was log files–will immediately recognize a simple log file and think to yourself either “Ahhhhh …” or “Dude, logs are so 1994!”

Regardless, those of us who use analytics packages that support hybrid data collection would simply take these logs and integrate them with our normally collected site data. This log-based approach is nice IMHO because the site owner could decide for his or herself which events they wanted included in the final analysis at integration time. Or, you could write a process that translated the text strings the application provider gave you into something more meaningful, you could categorize the events, etc.

Alternatively, assuming that few if any of the Web 2.0 application providers will actually build the necessary infrastructure to support this type of data capture and reporting, one could pass a reporting URL into the API and the application could use the reporting URL to log events using your existing meausurement application. For example, if I were to instantiate a Google Map like this:

var map = new GMap2(document.getElementById(”map”));
map.setCenter(new GLatLng(37.4419, -122.1419), 13);

Then the alternative method could be something like “setReportingURL” e.g.:

map.setReportingURL(”http://www.site.com/image.gif?ID=VISITORID001&EVENT=[E]”);

where the method would know to replace “[E]” with the encoded text string describing the actual event. This way, every time their was a meaningful event occurring, Google Maps would know to fire off a request to the tracking URL such that I could incorporate the event into my data set.

You’d almost surely have to also include the “ID=VISITORID001″ string to properly sessionize the events and associate them with a visitor. The actual value of “ID=” would be set from the client based on the value of the tracking cookie.

I’m honestly not sure which approach I like better. The former is almost surely more robust from the viewpoint of creating a standard reporting API for “Web 2.0″ applications, something Mark Baartse doubts will happen (Mark’s post being the only relevant Google result when I searched for “measuring Web 2.0″.) The latter is probably easier for the application providers to implement and more likely to work broadly given that only a handful of web analyics applications support integrating multiple data sources tied together by a common unique user identifier.

And yes, I realize that given a sufficiently robust API that you can already build this type of reporting. But man, wouldn’t it be easier if there was a standard tracking and reporting method that came with every such application that behaved the same way every time? Given that our industry has grown up almost entirely without standards, doesn’t this sound like a step in the right direction?

Keep in mind that this is only a technology proposal and only serves to highlight that every application provider would need to define and defend which events they reported and which they ignored. This is pretty much the same question we all asked ourselves seven years ago when Flash hit the scene and people started embedding script-based tracking inside Flash ActionScript.

It’s also the same question you have to ask and answer when you’re building any RIA: Which behaviors and actions are worth tracking and which should be ignored?

Maybe the practical solution is to set a level or threshold for event reporting–sort of a “few”, “many”, “firehose” hierarchy that would allow application designers to support a “log everything” mindset but allow their end users to be more conservative in what they actually collect and use. Few of the good Web 2.0 applications I’ve seen out there have enough events to really warrant this type of hierarchy; most are simple enough and likely to report click events which still provide great insight into visitor behavior.

Either way, there you have it, a line drawn in the sand. If Web 2.0 is going to change the Internet then I think the folks building these applications should play ball and report back on how our visitors are using their technologies. From the basic reporting I propose, sufficiently robust analytics packages will be able to calculate whether these applications have any effect on visitor retention, conversion, revenue, etc. Which, if you think about it, is exactly why the application providers should push reporting out ASAP.

Think I’m crazy? Have a great Web 2.0 application and want to implement my suggestion? Want to form an ad hoc Web 2.0 Measurement Standards Committee that meets on the second Wednesday of every month in a bar near you? Your comments are greatly appreciated and email is always answered.