The comScore study on cookie deletion is finally out

Published by Eric T. Peterson on June 6, 2007.

I just happened to write my contact at comScore today asking about their follow-up report on cookie deletion. He said it would be out today and here it is:

http://www.comscore.com/request/cookie_deletion.asp

This report does a good job of providing additional data and information about the comScore methodology in this report, something missing from the press release and critical to our collective understanding of cookie deletion. This report explicitly addresses anti-spyware and the differences in third- and first-party cookie deletion, essentially showing that there is an anti-spyware effect but it is minimal compared to manual cookie deletion which appears to be the primary culprit.

comScore also presents some of the attitudinal data they alluded to in their press release, essentially confirming what I first reported at JupiterResearch in 2005 … that most consumers aren’t really sure what cookies do.

Since I last saw the report they added a few sections — one on international traffic and one on cookie blocking. While the section on international doesn’t add much to the conversation other than to explain why panel-based and log-based systems numbers differ (something that should be fairly obvious), the cookie blocking data is pretty interesting.

According to comScore, if your web analytics application falls-back to an IP-based value for unique visitor identification in the absence of a cookie being successfully set, you’re likely worse off than you are simply dropping those visitors. Their table on page 15 shows that due to dynamic IP assignment that the average home computer has 10.5 different IP addresses in a month. Yikes!

If you’re into this stuff, or if you’re interested in how much cookie deletion might be impacting your own audience measurement, you should download the report and give it a careful read. It certainly doesn’t provide a solution to the problem, but often times knowing is half the battle.

http://www.comscore.com/request/cookie_deletion.asp

I welcome your feedback on the report and the usual comments and criticism.

About Eric T. Peterson

Eric T. Peterson is the founder of Web Analytics Demystified, Inc. and the author of Web Analytics Demystified, Web Site Measurement Hacks, and The Big Book of Key Performance Indicators. Mr. Peterson frequently presents on web analytics, is often cited in articles about digital measurement, and has been blogging on the subject since 2004.

Want to speak with Eric? Contact Web Analytics Demystified

Eric T. Peterson's Blog at Web Analytics Demystified

Posted Wednesday, June 6th, 2007 | 9 responses | Share, Save or Email

http://www.smileycat.com Christian Watson

This is a fascinating read – thanks! I am updating my report template to include the following footnote: “margin of error: +/- 150%”. That should cover me.
John Stansbury

Thanks for posting link to the study. I’m only on page 7 of the document, but why in the world did they pick December as the study period? Shouldn’t they have included another period where people exhibit more “typical” behavior? I’m also curious to explore what the exclusion of behaviors while at work means to the findings.
http://www.webanalyticsdemystified.com eric

Christian: LOL! I wonder if that will get people talking about accuracy issues internally? Of course, based on my recent study (http://www.webanalyticsdemystified.com/research/) 80% of companies are already talking about accuracy issues anyway …

John: Good questions. I’m not sure I’ve ever seen anything that suggests that cookie-deletion behavior is seasonal or otherwise influenced by time but it’s not unimaginable. I believe comScore excluded the “at work” data because many organizations have certain policies regarding the use and acceptance of cookies that may have skewed the data but that’s just a guess.
http://blog.dtelepathy.com Alex Funk

Glad they made this available for digestion. Eric – thanks for posting this for all to find. No wonder why cookies seem like a better metric to use, when I got to the chart you pointed out on page 15 it showed that more than 1/3 of computers at home use 6+ unique IP addresses each month. Anything looks better than that margin of error :)

Another interesting point they touched on, is the serial cookie deleter – who represents only 7% of the population but who represent 35% of all cookies. Maybe we need to get a pro-cookie PR and Ad campaign going, it could be be sponsored by one of the analytics associations. Couldn’t you see it now? Commercials talking about all the benefits you will enjoy just by hanging on to your cookies.

Hey- at least those conversion rates were understated last month! Now how to tell the client…
http://www.webanalyticsdemystified.com eric

Alex: I think we knew all along that IP address was not a good fall-back position but yeah, comScore really hammers that home. So now the question becomes “Does the technology you’re using, in the absence cookies, fall back on an IP-based identifier?” If the answer is yes, look out!

Regarding the serial cookie deleter … yeah, that’s a problem huh? I’m not sure a campaign would help, but knowing these folks are out there makes me increasingly suspicious about the value of “manage based on trends” mentality that some folks propose. If you manage based on trends — instead of working to understand the inaccuracies in your data — what happens if you suddenly attract a group of these serial deleters?

Just a thought.

Good luck with the client! ;-)
Winooski

“Does the technology you’re using, in the absence cookies, fall back on an IP-based identifier?”

Hmm…I’m not 100% sure. Is the answer “yes” to some of the more popular packages, for example, Google Analytics? Any idea as to which major packages default to IP-based identification when they can’t use cookies?
http://www.webanalyticsdemystified.com eric

Winooski: Like I said, that is the question. I have looking into that on my list but it’s a pretty long list. The information is out there … it’s just a matter of looking.

What package are you using?
Winooski

Hey Eric. We’re using Google Analytics, but are also shopping around for something more robust (e.g., that actually backs its own reporting up so you can’t accidentally delete all your profile data…Don’t ask me how I know that… [;-)] ). For the time being, it’s GA.
Pingback: Web Analytics Facts » ComScore rapport over cookie deletion