Archive for January, 2009

Just after President Obama was elected back in November I wrote a blog post that had been kicking around in my head for a long time calling for the “legalization” of browser cookies by Federal Government run web sites. The response to the post was great, but now it appears that the first comment from Brent Hieggelke (who was head of marketing at WebTrends for several years) was destined to become ironic.  Brent (who is my neighbor in Portland) waxed philosophical about government and cookies with this comment:

“As someone who 4 years ago spent ALL of New Years Day on the phone with the White House Communications Team because their site was “outed” by CNN and other media as <> using cookies in a completely innocent manner, I couldn’t agree more.”

Turns out that Jascha Kaykas-Wolff, the new head of marketing at WebTrends, is probably having the exact same conversation thanks to so called “privacy advocates” according to this article in InformationWeek. What’s more, the privacy advocates, rather than educating themselves about the real risks associated with the use of browser cookies are apparently patting themselves on the back for getting the Obama administration to make a simple, cosmetic change at WhiteHouse.gov regarding the use of YouTube video.

Giving himself full credit for the change, Chris Soghoian from CNET’s “surveli@nce st@te” blog says:

“It seems that someone in the White House read my blog post yesterday–as within 12 hours of the story going live, Obama’s Web team rolled out a technical fix that severely limits YouTube’s ability to track most visitors to the White House Web site.”

Congratulations Chris. Instead of giving the President’s team the lattitude to focus on, oh, THE ECONOMY, THE THREAT OF TERRORISM, THE HOUSING CRISIS, UNEMPLOYMENT, and HEALTH CARE you single-handedly managed to force the Administration to waste their time worrying about whether or not Google was getting just a little more of the world’s data.  President Obama, in the midst of rolling out a truly revolutionary use of technology in government in an effort to get more of us personally involved in our communities, our country, and our collective future, was forced by your misguided fear-mongering to stop what they were doing and address what has otherwise been hailed as a brilliant communication effort.

You sir, are the man.

Seriously people, can we stop worrying about cookies for a little while? Given all the other problems we have as a nation and as a global community, am I alone in thinking that people like Chris and his fellow “privacy advocates” need to find something else to focus their efforts on? Maybe if this community spent more time trying to help the President come up with ideas to put America back to work and less time creating fear, uncertainty, and doubt in the popular media we’d see the kind of change that our President has been talking about.

At this point I’m fairly confident that any person who has any shred of concern about their cookies being scraped, hijacked, poisoned, bombed, or otherwise maliciously used to expose their personal habits or ruin their lives has figured out how to clear or otherwise modify said cookies. Even though I started writing about the profile of the cookie deleter back in 2005, I’m still waiting for someone to give me a good reason to delete said objects that is not A) because you’re a site developer and you need to confirm how cookies are being set, B) you’re a web analytics specialist debugging tracking, C) you gamble a lot online or D) you surf a lot of porn.

If “A” or “B” I understand.  If “C” or “D” … don’t forget to clear your browser history too!

I’m being snarky, I know, and maybe I’m just taking Chris to task since he still has his street-cred inducing ponytail and I cut mine off. But at this point the hand-wringing about cookies in general much less because of the mandate set by OMB M-03-22 has become tedious and needs to stop. President Obama is working to change the way government works and I think his staff deserve some latitude when it comes to the Internet. If we want government sites to work for us, we need to let analytic technology work for them. If we want change, we need to be open to change.

Put another way, if you fear Google, don’t use their products. If you fear cookies, delete them. If you fear for your privacy online, don’t go online. Wear a foil hat. Don’t answer the phone. Don’t open the door. Don’t speak.

But please people let’s decide to take some personal responsibility on this issue and stop bugging an otherwise busy administration–whichever administration that may be. Regardless of how you feel about Barack Obama, let’s all recognize that we are facing substantially bigger challenges today than we have in recent history and since the man was fairly elected he deserves at least a chance to improve the economic conditions in the U.S. without “privacy advocates” forcing his staff to make tedious (and functionally meaningless) changes to the White House web site.

I know I’m going to get slammed for this post, that’s okay. Somebody needs to stand up for cookies and since I already tried “diplomatic” I suppose it’s time to try “direct.” Browser cookies help make it possible for great companies like CNET to provide lots of great content–including Chris’s blog! Browser cookies help justify great technology like Twitter, Facebook, and MySpace. Browser cookies power the Internet and should not be feared, especially not by President Obama.

I look forward to your comments and criticisms of my position.

Wow, once and awhile in life you get involved in something that turns out differently than you had planned. I’ve written about this in the past, but for me Web Analytics Wednesday is one of those things

Web Analytics Wednesday was started by my good friend (and soon-to-be-new-mom!) June Dershewitz and I back in 2005 with little expectation of success. Our goal? To give web analytics professionals around the world the means and motivation to gather locally and get to know each other.

Now, in 2009, by every measure we have succeeded and even exceeded expectations, having helped create regular Web Analytics Wednesday events in places like New York, San Francisco, Boston, London, Toronto, Paris, Madrid, Copenhagen, Stockholm, Sao Paulo, Sydney and smaller cities like Austin, Texas, Columbus, Ohio, and Nashville, Tennessee. These events were attended by over 4,500 web analytics professionals around the world in 2008 alone and 2009 is off to a great start with over 600 total attendees in January alone.

In 2008 we decided to do more to help Web Analytics Wednesday events get off the ground in more cities and to provide more financial support to as many groups as possible. We were able to do this with the generous support of Coremetrics and SiteSpect, the Web Analytics Wednesday Global Sponsors. Both of these companies had already been hosting events around the world, and senior management from both turned out to be enthusiastically supportive of Web Analytics Wednesday.

Today I am incredibly pleased to announce that the Web Analytics Wednesday Global Sponsors have joined Web Analytics Demystified in making a donation to Operation USA. This donation of roughly $1 per Web Analytics Wednesday participant was made in recognition of the relative success many in the web analytics industry have enjoyed in the context of the challenges faced by our fellow man, woman, and child across the globe.

Personally I have been blessed with a healthy family, a rich life, and moderate success in business, and thus have frequently been able to make charitable contributions. But as we all know, as the economy worsens, charities are often the first to feel the pinch despite the fact that an increasing number of people around the world need the support of groups like Operation USA. Because of this, I am incredibly grateful to Joe Davis, CEO of Coremetrics, and Eric Hansen, CEO of SiteSpect for their willingness to match my $1,500 contribution on behalf of both of their companies.

I hope everyone who hosts, sponsors, and participates in Web Analytics Wednesday will take the time to thank Coremetrics and SiteSpect for their generosity, either by commenting on this post or by emailing the companies directly.

On behalf of Web Analytics Demystified, June, our Global Sponsors, and all the Web Analytics Wednesday hosts we wish you all the best in 2009 and hope you’re able to make it out to an event near you soon!

UPDATE ON JANUARY 19, 2009: Peter Black from BPA Worldwide who was also on the IAB working group with Josh Chasin, wrote in and disagrees with Chasin’s characterization of who the “Unique User” language is targeting. I have email into the IAB and MRC’s George Ivie to clarify the situation. Watch this blog!

UPDATE ON JANUARY 18, 2009: Josh Chasin from comScore, who was a member of the IAB working group that defined the guidelines described in this post, wrote in to point out that I misinterpreted the IAB’s intent. While their web site clearly says …

“The IAB believes that all companies involved in audience measurement should be audited for their processes.  These audits are intended to establish the source of any measurement discrepancies and to find potential solutions.

All measurement companies that report audience metrics have a material impact on interactive marketing and decision-making. Therefore, transparency into these methodologies is critical to maintaining advertisers’ confidence in interactive, particularly now, as marketers allocate more budget to the platform.”

… according to Chasin the IAB is excluding web analytics vendors from “all companies involved in audience measurement” and the type of companies  that have a material impact on interactive marketing and decision-making. Since this doesn’t sound right to me at all I will warn the reader that some of the questions I raise in the following post may, in fact, be totally irrelevant (at least in the context of the IAB Proposed Measurement Guidelines.

If nothing else, with two days left in the open comment period, the IAB may want to use my confusion as an example and clarify the target for the recommendations made in the document.

Reader beware!

As long as we’re talking about web analytics standards I figured I would take the opportunity to offer up a few thoughts on the Interactive Advertising Bureau’s Audience Reach Measurement Guidelines that are open for public comment until January 20th. If you haven’t had a chance to read these proposed guidelines you should, especially if you have an interest in how we collectively communicate about data.

At 34 pages the document is certainly a slog to read–and I say this knowing full well that I have a tendency to write 50 page white papers! Since you’re all bright folks I’m just going to address some of the proposed language that stood out to me.  And, as always, if you have any thoughts or positions on the proposed guidelines I’d love to hear from you!

Starting in Section 1.2 the IAB clarifies the relationship between “Unique Cookies”, “Unique Browsers”, “Unique Devices” and “Unique Users / Visitors”.  The discussion about “Unique Devices” is interesting because this is a clear indication of the impact that mobile devices like the iPhone are having on audience measurement.  Things start to get really interesting, however, in Section 1.2.4 where the IAB says (emphasis mine):

“However, in order to report a Unique User, the measurement organization must utilize in its identification and attribution processes underlying data that is, at least in a reasonable proportion,, attributed directly to a person. For instance, data collected from registrants is one possible source that can be utilized in creating a Unique Users measure by a census-based measurement organization, if registrants represent a reasonable proportion of the total user-base and when appropriate scientific projection methods are used for non-registrrants (sic).  In no instance may a census measurement organization report Unique Users purely through algorithms or modeling that is not at least partially traceable to information obtained directly from people, as opposed to browsers, computers, or any other non-human element.“

Did you get that? Keep in mind that while at JupiterResearch I was among the first to publicize the decline in accuracy of visitor counting due to cookie deletion. In fact the report was subtitled “Addressing the Decline in Accuracy of Cookie-Based Measurement.” At the time people called me crazy and Seth Godin even accused me of living in an echo chamber (I have since forgiven Seth.)

Now, three years later, the IAB is expressly telling measurement vendors to stop reporting a metric called “Unique Visitors” or “Unique Users” unless they have a research-based strategy for determining the correct proportion of cookies to “real people” and have applied that calculation in a transparent way.

Whoa.

Think about this for a minute. Every one of the fine census measurement packages (nee web analytics) out there is reporting a Unique Visitor number, but I’ll go out on a limb here and propose that none of them are even vaguely adhering to the IAB proposed definition of a “Unique Visitor.”  I’ll go a step further and postulate that, at least in the base offerings, these vendors don’t currently have the technical capability required to report an estimated/algorithmically derived “Unique Visitor” count based on scientific projection methods.

If I’m wrong about this I suspect I’ll hear about it, but I don’t think I’m wrong when it comes to the base offerings like SiteCatalyst, WebTrends Web Analyics, Coremetrics 2009, etc.  And yes, I’m aware that end-users can use higher-end products like Discover on Premise and the data warehousing tools to apply a correction factor to UV counts, but that is not what the IAB is saying. This guideline is saying that correcting for cookie, browser, and device-related over-counting of unique visitors is the responsibility of the measurement vendor.

Again, whoa.

And as if that’s not a radical enough move, the document goes on to state in Section 2.2 that the vendors need to actually break out these correction factors across three components: first-cookie acceptance, deletion, and browser denial (again, emphasis mine):

“Cookie deletion rates, calibration methods and sources or estimation methods used to account for first-use, deletion, and non-accepting cookie groups should be disclosed by the audience measurement organization. The audience measurement organization should disclose census-based unique cookie counts and the estimated unique activity from first-use, deletion and non-accepting cookie groups separately and in aggregate. If the measurement organization relies on a unified model that makes reporting among these separate groups impossible, it may report these counts in aggregate only, but should be prepared to demonstrate in an audit the ability of its unified model to address each type of cookie completely.”

The IAB goes on in Section 2.4 to start to push web analytics into what is an uncomfortable position for some people, the use of algorithms and data models, to better report on unique visitors:

“As noted above, Publishers and Ad-servers will generally need to rely on algorithms (data models) to estimate the number of users attributable to the counts of Unique Cookies they develop. The underlying basis for this algorithm should be a study of actual users (i.e, people).  Ideally, such a study would be based on direct contact and/or observation of people using the browser at the time of accessing web-site content or ads with the unique cookie, as well as observation of the number of browsers in use by these users.  Additionally, inferences will need to be made about advertising activity of users with non-cookied browsers, so these types of users should also be contacted and observed.  Also, the activity of users who access content from multiple locations (home, work, school) on different browsers should be factored into these algorithms.”

Finally, the IAB is telling the vendors they need to report the results of their research to their customers, essentially exposing flaws in their technology for all to see:

“The resulting study should be representative of, and projectable to, the users of the web-site or property, and periodically re-performed to reflect gradual changes in audience.  Known weaknesses in the projection processes should be disclosed to users of Audience Reach Measurements.“

If  you’re keeping track, the IAB is telling the vendors A) to completely change their definition of “Unique Visitors”, B) to start to actively research sources of inaccuracy on behalf of their customers, and C) pro-actively report known weaknesses in their system to their customers. Anyone want to place any bets on when the vendor community will adopt these recommendations? I’m going to be a little snarky here and put my money on “never in a million years.”

Seriously you have to love the IAB for putting this out there.  Unlike the Web Analytics Association’s Standards which I believe are an excellent start but are a little soft in areas, the IAB is basically telling the measurement vendor community that they are doing the entire world a disservice by reporting unique visitor counts that are complete bollocks and they need to stop doing that post-haste! Okay, maybe I’m over-reading the document but the scope of changes required for any vendor to become IAB-compliant is dramatic, both technically and psychologically.

I’m not sure if Brandt Dainow had seen the IAB proposal when he besmirtched the fine work of the Web Analytics Association’s Standards Committee, but if you compare the two proposal documents (the WAA’s proposal can be found here in PDF form) you will detect a noticable difference.  Personally I’m glad that my good friend Judah Phillips bridged the gap between the IAB and WAA and I find myself wondering, at least a little bit, whether the IAB+WAA relationship should be even deeper.

This all brings me to an excellent point that Bryan Robertson made on my last post on standards regarding how standards are defined and moved into common use. Bryan’s thesis is based on the W3C’s move from HTML 1.0 to XHTML and his point is that this transition to the XHTML standard came about because of A) a powerful standards body, B) a vocal community, and C) passionate thought leaders.  Regarding a powerful standards body, Bryan specifically make a point that other folks have made, usually behind closed doors:

“Is the WAA powerful enough at this point in time, or do we need to continue to build momentum before the standards can be more bold? For example, is the WAA hand wringing too much over the polite “we’ll share with you if you share with us” arrangement with the IAB over standards definitions? Is the WAA in a tough position in trying to bring practitioners and vendors together at the same table?”

Bonus points to Bryan for willing to be direct on the conflict of interests arising from having two masters, vendor and practitioner.  Again, I have nothing but profound respect for Angie and all of the other members of the WAA Standards Committee, but since I do know that vendors participated in the definition process I wonder a little bit how much impact they really had.

Anyway, I’m doing all the talking here and it’s a beautiful day so I will ask what you all think — either about the IAB proposal, Brandt Dainow’s assertion, Bryan’s thesis about the strength of the WAA, or anything else that strikes your fancy. Do you think the IAB standard for “Unique Users” has a snowball’s chance of being widely implemented? Do you think Brandt Dainow makes a good point (even if he does it in a lousy way)? Do you think the WAA may be better off working more closely with the IAB on Standards, given the IAB’s relative might?

My site host assures me that my comments table will not crash again so I look forward to hearing from you all.

Readers who are enthusiastic members of the web analytics community are by now familiar with Brandt Dainow and his sometimes antagonistic missives published at iMediaConnection. While I try pretty hard to follow the old “if you can’t say something nice” rule I occasionally fail in my efforts. Perhaps the best evidence of my failing was my calling Brandt Daniow insane when he suggested that Google Analytics version 2.0 was “simply a quantum leap above any other analytics product on the planet.”

While I firmly believe that Google Analytics is a great, valuable, and appropriate application for a wide range of needs, I think that Dainow’s “quantum leap” claim and statements like “What Google has done is simply take every feature in every product on the market and put them all into one system, and then make it available for free” are so obviously hyperbolic that they beg criticism (which Mr. Dainow got in spades from many within the analytics community.)

Dainow has since turned on Google Analytics, more recently pointing out what he describes as “disturbing inaccuracies behind Google Analytics” and again getting our  attention with irresponsible statements like “Google Analytics is different from other products in that it has been intentionally designed by Google to be inaccurate over and above the normal inaccuracies that are inevitable.“

Oddly enough, his rant about Google Analytics included some statements that rubbed members of the Web Analytics Association the wrong way.  When folks like Jodi McDermott commented on the article and questioned some of Dainow’s assertions, Brandt did what any normal person would do …

… he wrote a nasty follow-up piece critical of the Web Analytics Association and the WAA Standards Committee!

I will let you read his piece yourself, but the two-sentence summary of Dainow’s opinion is that “the work of the WAA standards committee is a disaster for the web analytics community. It will take years to undo the damage and create proper precise standards that can be implemented in software. The WAA “standard” is not a standard, it’s just second-rate muttering.“

Clearly Dainow is not worried about making friends in the web analytics industry.

I personally am a big fan of the Web Analytics Association.  I am pretty loyal to some of the current Board of Directors, I’ve done a bunch in the past to support the WAA and am about to announce more of the same, and I’ve even gone out of my way to help promote the work of the Web Analytics Association Standards Committee.  So it is was with great trepidation I wrote this article’s title … but I find myself agreeing with one small part of Dainow’s otherwise unnecessary rant.

Towards the end of his article, right before he declares that some pretty nice people’s work has been little more than time wasted, he says this:

“The WAA should be setting the agenda, not following the crowd. The task of the WAA standards committee should be to determine how web analytics metrics should be calculated in order to achieve the highest degree of precision possible. The WAA should be laying out the roadmap for the way things should be. It then falls to the vendors to bring their software into line.”

I more or less made this same comment, although I like to believe I used a great deal more tact, when I commented on the original Web Analytics Association Standards published under the direction of former Director Avinash Kaushik back in August 2007.

At the time I preferred to focus on the reality of the situation–the fact that the WAA had proposed a set of standard definitions that, for good or ill, were better than anything else out there.  Instead of being openly critical of the definitions as written, I preferred to ask the question, “Now that we have these definitions, what are we going to do with them?”

While my call for a web analytics standards compliance matrix has since been answered by all of the major vendors except for Omniture, I personally don’t believe that the Standards process is serving the needs of our community as best possible.  We all continue to be vexed by a lack of standard definitions, a situation that will likely get worse with the decline of the web analytics economy.

Not having participated in the process of drafting the WAA Standards I can only express gratitude towards those members of the community who have volunteered  their valuable time for this work.  In my humble opinion, people like Jason Burby and Angie Brown are to be congratulated for their efforts, not denigrated and accused of having set our industry back into the dark ages.

But, in the spirit of having an open mind and building consensus, I would be interested in hearing my reader’s collective thoughts on Dainow’s point that the WAA should be setting standards without regard to their practicality today. Put another way, should the Association have written definitions that would be robust and useful in an analytics context and then presented that guidance to the entire community–vendor, consultant, and practitioner alike–saying “this is the result we should all be working towards.”

For example, should the WAA have been more explicit in their definition of a “visit” and proclaim that a visit is terminated after 30 minutes of inactivity, instead of saying “if an individual has not taken another action (typically additional page views) on the site within a specified time period, the visit will terminate by timing out.“  Being explicit about the timeout duration would make a clear statement about our collective expectation for the definition of a visit, and any technology or analysis that choses to use a timeout other than 30 minutes would also need to justify their decision to eschew the WAA Standard for another value.

I know that the WAA is doing the best they can, and I am enthusiastic about the work Angie, Judith and their fellow volunteers have all been doing.  But I do think Dainow’s assertion that standards should be set based on overall value to the community in the long run, not necessarily the near-term practicality, is worth exploring.  Taking this approach would definitely penalize some vendors and reduce their self-generated “compliance score” but it does kind of make sense to be working collectively towards a more precise set of definitions we can all work from.

Doesn’t it?

These are the kinds of conversations that aren’t just magically resolved and so I’m sure we’ll have to add this to the list of issues worthy of discussion the next time we all meet.  I’m sure it will come up at some of the upcoming vendor events, in San Jose at Emetrics, and likely at our own web analytics conference, the X Change (where last year Forrester analyst John Lovett led a conversation on the topic.)

As always I welcome your thoughts, feedback, open disagreement, pointing out flaws in my logic, etc.  I consider myself fortunate to have such thoughtful and experienced analytics practitioners among my most loyal readers and sincerely hope Dainow’s otherwise disturbing rant will lead to something of value for our community.