It occurred to me today that by this time next week I will have already given my “Guru” presentation at Emetrics and will be dining with some of the brightest minds in the web analytics industry. While I have been almost zealous in my recommendation of Jim’s conference for the past six years (am I “old school”, or just “old” … you decide) I have to say I am excited about this particular event more than any other.

Why, you ask?

1. Robbin’s “Birds of a Feather” Web Analytics Blogger’s luncheon on Monday, which I think was not Robbin’s idea but nonetheless is a wonderful idea! There are dozens of web analytics bloggers now, many of whom will be at the conference, and we’re having a private little meet-up. This is cool since my friend Judah can join us now, having recently joined the blogosphere!
2. Great looking presentations from some of my favorite people in the industry, including Jim Sterne, Jason and Shane, Brett Crosby, Megan Burns, Robbin, Jason Palmer, John Marshall, Bill Gassman, Seth Romanow, Jodi McDermott, Jen Veesenmeyer, Rand Schulman, Josh Manion, Ian Houston and Judah Phillips, Kevin Heisler, Eric Hansen, Bryan Eisenberg, Bob Chatham, Gary Angel, … okay, so I pretty much just listed half of the presenters. As I look at that, do any of you remember when vendors weren’t allowed to present? I used to be special …
3. Even greater looking presentations from a ton of people I know of but have never had the chance to meet formally, including Tim Kopp, Lou Rosenfeld and Rich Wiggins, Patrick Moran, Chris Gemignani, Paul Holstein, Lissa Gatz, and more!
4. The Web Analytics Wednesday on Tuesday special event, which Jim and Matthew have graciously allowed to occur (and more importantly, have fully funded so we can have drinks, etc!) Are you already signed up? If not, sign up right now!
5. Most importantly, I get to present to the entire audience this time in my “Guru Session” where, despite what you’ll read in the literature, I will be sharing the results of my recent web analytics survey. If you’re a data junkie like me, I promise that you will not be disappointed by my presentation.

Plus, the last five minutes of my presentation are not to be missed! Trust me on this one … there was one number that came out of the research that more than any will change our understanding of the web analytics industry forever. Sadly, you’ll have to wait until 4:45 PM on Monday, May 7th to hear what it was.

Anyway, if you’re coming to Emetrics and you’d like to get together please let me know. I will be in Sunday and in-and-out of the WAA training day so will have tons of time to sit down with folks and catch up.

Oh, if you have another minute, check out this humorous tribute Rene and Aurelie did for Jim Sterne.

My friend Ian IM’d me last week and said he had confirmed comScore’s data on first-party cookie deletion. Since Ian is easily one of the sharpest people I know, I was immediately intrigued, given that he has been working on a methodology to restore deleted cookies using the browser’s cache and a dynamically generated script. Unfortunately Ian hasn’t been able to implement his strategy on a high-volume site, yet, but he did do a very robust comparison of measured site traffic data to comScore numbers.

What Ian saw by comparing the number of measured unique visitors based on accepted cookies to comScore data for the same site was, well, roughly a 2.5X inflation from panel to measured visitor counts. His monthly numbers ranged between 1.99X and 3.15X but he reports the average as 2.47X.

Ian also reported data for daily unique visitors where he saw an average inflation of 1.96X (range of 1.35X to 2.84X). Ian commented, and I agree, that the daily numbers are somewhat disconcerting given that they appear to support the notion that “serial deleters” are among the most engaged.

Keep in mind, these numbers are based on a direct comparison to the comScore panel-based numbers, numbers whose accuracy has long been questioned and continues to be questioned today.

As usual, Ian’s writing continues to be well thought out and well written, and I highly recommend reading him if you’re not already.  I also want to congratulate Ian on joining the team at WebSideStory/Visual Sciences.  The blogosphere loses a great practitioner but gains a great vendor/consultant (and to be fair, Ian has been a private consultant for as long as I have known him.)

There was a question posted to the Yahoo! group recently asking about web analytics practitioner blogs. I looked at the post and thought “what a funny thing to ask about …” but it stuck in my head. I’m not even sure if the author got many answers given that most of the folks that pop into my head are all either working for vendors or working in consulting groups.

Some of the best practitioner-bloggers, folks like Clint Ivy and Avinash Kaushik, have gone to work for vendors (and I happen to know of one more such defection about to occur but have been sworn to secrecy, for now …) Of the folks I track in my blogroll, I believe that only Dustin Wallace, Paul Strupp, and Scott Baldwin are real web analytics practitioners-not-consultants-not-vendors … surprisingly few when you consider that the vast majority of people doing web analytics are not working for vendors or consulting firms.

Well all that just changed: My friend Judah Phillips has joined the fray. Not only has Judah joined the fray, Judah is the first practitioner to join me by blogging under the Web Analytics Demystified brand.

Now, in my experience, Judah has never been at a loss for words so I won’t do him the injustice of putting words into his mouth. He’s written an introductory post and I would encourage you to give it a read.

Welcome to the blogosphere Judah, and welcome to Web Analytics Demystified.

If you’re reading this you don’t have to do anything but I have decided to go ahead and use FeedBurner for my primary RSS feed.  If you don’t mind, please update the RSS feed for this weblog to:

http://feeds.feedburner.com/ericpeterson

I know, I know … this is like the third time I’ve changed my RSS feed URL.  What can I say? The move to WordPress opened a whole world of new opportunities for my weblog and I’m slowly taking advantage of many of them.

Again, that new RSS feed URL is:

http://feeds.feedburner.com/ericpeterson

As I mentioned a few times in the Yahoo! group, I have been talking to the folks at comScore about their recent report on cookie deletion. I got an email back from Andrew Lipsman with some more information and partial answers to questions of mine and a few passed to me by other cookie-savvy folk.

According to Andrew, comScore will be publishing a more complete document describing their research methodology in the next few weeks. Until then, they’re giving me the scoop so here you have it, direct from comScore (my questions in bold type, comScore’s answers follow in normal type):

(Andrew provided this preamble to his answers …)

The reason we have done this study for two cookies is to ensure that we are very familiar with the cookie structure, the different value pairs (e.g. GUID=1234) and their purpose. We are in particular interested in ID value pair that identifies a user over time, and does not change when the cookie gets refreshed.

How did they identify the unique values of the cookies? Using the Set Cookie Response header, the Cookie Request header, or the actual storage (cookie) file?

We are reading the cookie request call and parsing out specific ID value pairs. Over time we will observe a time series for each panelist for the value of this identifier corresponding to each cookie request. Cookie reset events are based on qualified value changes for a targeted ID value-pair.

How did they take into account non-persistence and/or cookie expiration settings?

The cookie domain value-pairs were chosen to represent passively assigned unique identifiers designed to be persistent over time. Cookies of this nature should only expire in the event that the visitor never returned within a relatively long expiration window.

How did they identify First vs. Third party cookies?

We are reading specific value-pairs for specific domain cookies. The first party cookie is the cookie used by the Portal site. The third party cookie was used by the ad serving company. All information is directly observed from metered panel activity. Recall information was not a source of determining preservation rates.

How was cookie blocking treated or accounted for?

The analysis is based on a sample for which at least one cookie value was observed.

What were the domains they examined? If not the domains, what was the nature of the first-party site?

We will not disclose the names of the sites used for the analysis. First party site is a major internet portal. The third-party site is a major ad server.

What were the survey questions asked? How many people were asked and how were they selected?

All deletion/retention figures were derived from direct panel observations, not from a recall-based survey. Only qualitative information came from the survey.

Obviously some of the answers provided are lacking but I’m willing to admit that it may be more a function of my incomplete knowledge of what the comScore panel application is able to capture.

One particularly good question from a reader essentially asked if P3P-instigated cookie blocking could be artificially running up cookie reset counts (essentially counting each page request as a new cookie) to which comScore answered that the study only included panel members for which “at least one cookie value was observed.” (plus, P3P is less likely to impact the first-party cookies that I’m more interested in …)

The encouraging news is that comScore is now officially on the record as willing to produce additional documentation about the study within the next week or so. I conveyed to Andrew some of the skepticism about the results they report, skepticism I told them they would hear, and pointed him to the ongoing conversation so hopefully the community’s concerns will be directly addressed in their methodology document.

Suffice to say, if some major flaw appears in their research, the company will have major egg on their face as they approach their announced IPO. Conversely, if the research proves sound under examination, regardless of whether you’re a data purist looking for “perfection” or willing to manage based on trends however flawed the underlying data might actually be, we all have something to consider the next time someone asks us “how many visitors come to your web site?”

Perhaps the only true and precise answer is, “It depends!”

What do you think about the answers that comScore provided? As always, your comments are greatly appreciated!