Want to learn the Adobe Analytics and Testing platforms from the team at Web Analytics Demystified? Sign up for our April 2014 "Adobe Intensive" and learn Enterprise Analytics from the best in the business!


Unique Visitors ONLY Come in One Size

Published by Eric T. Peterson on March 22, 2009 All posts from Eric T. Peterson

Back in January I published a note about the proposed IAB Audience Reach Measurement Guidelines that generated a fair amount of interest. At the time I applauded the IAB for providing guidance regarding the definition of a “unique user” or “unique visitor” while noting some concerns about how the proposed definition would actually manifest. In summary, the new IAB definition of “unique visitor” needed to have some basis in underlying data that is based on secondary research that can be directly tied to “a person.”  Now that the IAB Audience Reach Measurement Guidelines have been officially published we can use the IAB’s own words:

“… in order to report a Unique User, the measurement organization must utilitze in its identification and attribution processes underlying data that is, at least in reasonable proportion, attributed directly to a person” and “In no instance may a census measurement organization report Unique Users purely through algorithms or modeling that is not at least partially traceable to information obtained directly from people, as opposed to browsers, computers, or any other non-human element.” (Section 1.2.4)

The last little bit references, I believe, the IAB’s distinction of four types of unique “countables” — Unique Cookies (Section 1.2.1), Unique Browsers (1.2.2), Unique Devices (1.2.3) and Unique Users or Unique Visitors (1.2.4).  The term “measurement organization” was a little, well, mystifying as was evidenced in my January post, and sadly the final document does little to clarify this term other than to say the “document is principally applicable to Internet Publishers, Ad-serving organizations, Syndicated Measurement Organizations and auditors” on the IAB web site.

This definition is important since in my last post the real conundrum appeared to be that if “measurement organization” included Omniture, WebTrends, Google, Coremetrics, etc. then the IAB was essentially saying that the vendors needed to change the way they reported Unique Visitors, at least for their clients who would be subject to the perview of the IAB and MRC.  What’s more, George Ivey from MRC never got back to my repeated requests for information, despite two members of the IAB working group (Josh Chasin from comScore and Pete Black from BPA Worldwide) openly disagreeing in their interpretation of the definition …

Well, a few weeks back I got a call from Joe Laszlo, an old co-worker of mine at JupiterResearch who is now the IAB’s Director for Analytics, the guy basically responsible for the document.  I always liked Joe and it was nice to hear from him again.  And Joe did clarify for me what a “measurement organization” is … he just didn’t directly clarify the impact on web analytics vendors.

According to Joe (and he will surely correct me publicly if I am misinterpreting our conversation) the “measurement organizations” that should be guided by this new definition of “Unique Users” are publishing organizations who are outwardly reporting their metrics for consideration by advertisers in the open market. Companies like AOL, Weather.com, ESPN, etc.  This is, I think, much more clear than the sentence a few paragraphs up that includes “Syndicated Measurement Organizations and auditors” and puts at least this part of the document in context: Essentially when using numbers coming from census-based systems, the IAB and MRC want publishers to start reporting Unique Visitor counts that have some basis in reality.

Pretty hard to disagree with Joe and the IAB on that point. We all pretty much agree that cookie-based visitor counting is messed up, and I think we can even agree that the degree to which these counts are “messed up” is a function of the target audience, the duration under examination, and the type of site.  For example, we expect cookie-based counts on sites that attract highly technical users on a daily basis to be much more impacted over a 90-day measurement period than, say, sites that attract largely non-technical users on a monthly basis over the same 90-day period.

So I’ll make one really bold statement right now, the kind that I have a tendency to regret but hey, it’s Monday and I’m feeling pretty good about the coming week:

The IAB are to be applauded for taking such a bold stand on the subject of counting and reporting unique visitors based on what we traditionally consider “web analytic” data.

I said as much in my last post … right after I said that the likelihood of the web analytics vendors following these recommendations was about the same as everyone waking up tomorrow to realize that the financial meltdown was a bad dream and the Dow is still over 14,000 (zero). The team of folks that the IAB brought together, which I understand included both Omniture and WebTrends, should be congratulated for taking a firm stand on one of the most dogged issues plaguing our collective industries (web analytics, online advertising, online publishing, syndicated research, etc.) for at least the past five years.

It is about time that we all agreed that “Unique Visitor” reports coming from census-based technologies frequently have no basis in reality. Further, we should all admit that cookie deletion, cookie blocking, multiple computers, multiple devices, etc. have enough potential to distort the numbers as to render the resulting numbers useless when used to quantify the number of human beings visiting a site or property.

Yes, before you grieve on me with your “but they are probably directionally correct” response I agree with you, they probably are, but fundamentally I believe that advertising buyers are at least as interested in the raw numbers as they are the direction they are moving. I say “probably are” because if you’re not taking the IAB’s advice and reconciling census-based data with data derived directly from people, well, you’re never sure if that change in direction is because your audience is changing, technology is changing, or there is a real and substantial increase or decline.

I mentioned above that my conversation with Joe didn’t really clarify the impact on web analytics vendors under the IAB’s new definition. Since I spent a fair amount of time thinking about the IAB guideline’s impact in this regard, I will make another bigger and bolder statement:

Starting immediately, I think the web analytics vendors and any company reporting a cookie-based count that is not in compliance with the IAB’s definition of “Unique Visitor” should stop calling said metric “Unique Visitors (or Users)” and correctly rename the metric “Unique Cookies”.

Yep, I am 100% in favor of using the IAB’s new terminology and being semantically precise whenever possible. The “Unique Visitor” counts in the popular web analytics applications are always actually counting cookies and so we should just go ahead and say that explicitly by calling them “Unique Cookies”. This change would actually give the web analytics vendors a neat opportunity … to battle to be the first to have a real “Unique Visitor” count that is based, as the IAB has suggested, on underlying data that is, at least in reasonable proportion, attributed directly to a person.

How could they do this? Let me count the ways:

  1. Develop a standard practice around the use of log-in and registered user data
  2. Work with third-party partners who are focused on gathering more qualitative data (for example, Voice of Customer vendors like ForeSee Results)
  3. Work with third-party partners who are estimating cookie-deletion rates, or at least have the potential to (for example, Quantcast)
  4. Work with third-party partners who can actually calculate cookie-deletion and multiple-machine use rates with some accuracy (for example, comScore, Google, Yahoo!)

I’m sure there are a few ways I am not thinking of, but these are the big four that have been talked about since 2005. While I expect to get some grief from paying clients about this statement, and I fully expect my suggestion to be widely ignored by the vendor community (no offense taken), I think this change would be a big step towards the recognition that there is only ONE DEFINITION of a “Unique Visitor” and this definition is only tangentially related to the number of cookies being passed around.

Like Soylent Green(TM), “Unique Visitors” are PEOPLE and our industry will go a long way towards maturation when we collectively agree on this fundamental truth.  It is not to say that Unique Cookies is not a valuable count — hell, in the absence of a strategy for reconciling cookies against people-based data unique cookies are all we have. But I do not believe that after nearly 15 years we are doing the online measurement community any justice by plugging our ears and signing “LA LA LA LA I CANNOT HEAR YOU GO AWAY!!!!!”

Which brings me to my last point …

I was really, really bummed out to read Jodi McDermott’s MediaPost article titled “Unique Visitors Come in Two Shapes and Sizes.” I was bummed because I have always liked Jodi since we worked together at Visual Sciences, because I think she is a brilliant member of our community, and because I knew I was going to end up writing these words … Jodi’s thesis is wrong and does the web analytics community a dis-service in attempting to defend a mistake by asking to water down a good definition just because it isn’t “hers” (in quotes since Jodi is a member of a larger committee charged with defining standards within the WAA.)

From Jodi’s article (which I recommend you read, especially the comments, and the emphasis is mine):

Bravo to the IAB for forcing the issue with audience measurement companies to standardize the way that they report uniques, but from a Web analyst’s perspective — and as a member of the WAA Standards committee — I wish they would have not allowed the term “unique visitors” to be redefined in such a way as to allow for multiple definitions in the space. Web analysts and media planners today have a hard enough time trying to figure out which data source to use and which standard to apply when performing their job — but that issue is now compounded even more by multiple definitions of unique visitors. In defense of the IAB, its membership is comprised of some heavy-hitter companies who are not about to change that “tab” in their reporting UI that says “Unique Visitors” on it.  But in defense of  WAA individual and company members, which include vendors such as Omniture and WebTrends (who were both listed as “Project Participants” on the IAB document, interestingly enough), neither are we. The term will live on in both places.”

I think what Jodi has missed here is that the IAB has actually given the world a useful and more accurate definition of “Unique Visitors” than any used in the web analytics industry today. More importantly, given the relative weight, clout, and respect enjoyed by the IAB in the wider world, I don’t think their definition allows for “multiple definitions” … I rather think that over time the IAB expects their member companies, especially those who want to have their numbers audited and publicly used, will consider the IAB definition the definition of “Unique Visitors” and properly consider the term we web analysts widely use today to be “Unique Cookies.”

I’m not sure what Jodi means by “heavy-hitter companies who are not about to change their “tab”" since I’m aware of very few companies today that have implemented the IAB recommendation for practical and ongoing use. But I was incredulous when I read the statement regarding using the IAB’s new definition, “in defense of the WAA individual and company members, which include vendors such as Omniture and WebTrends, neither are we. The term will live on in both places.”

Seriously? Rather than start calling our cookie counts “Unique Cookies” and having a rational conversation with our bosses to explain that the technology we use is limited in its ability to discern real people, you prefer to throw down the gauntlet with the IAB and say “screw your definition?” Despite the criticism that has been both wrongly and rightly heaped on the WAA’s “standard” definitions, despite the considerable group that crafted the IAB’s definitions, and considering the fact that the WAA’s definition is wrong, you want to pick a fight?

Two wrongs never make a right, and you’re wrong twice here. Sorry.

I am not on the WAA Standards Committee, I am not on the WAA Board of Directors, and my dues with the WAA are about to lapse so I have no basis for representing the organization. Perhaps reading more into Jodi’s post given my knowledge of her passionate work in the WAA, but I would strongly encourage the current Board of Directors to examine Jodi’s statements in the context of the IAB relationship and the “bigger picture” at play.  Because while Jodi may speak for the WAA Standards Committee and by extension the entire WAA, she certainly does not speak for me.

I will gladly use the term “Unique Cookies” when I am talking about a cookie-based count and reserve the term “Unique Visitors” for those situations where I have some basis for doing so. More importantly I will encourge my clients and vendor friends to consider doing same. The IAB has given the entire measurement community a reason to take a huge leap forward and gain clarity around one of our most important metrics. To turn our back on this opportuntity because it will necessitate change, require additional explanation, or because “we like our definition better” is wrong, wrong, wrong.

Harrrrumph.

I suspect like previous posts on the subject this will generate some conversation. As usual I do not pretend to have all the answers and I welcome your feedback. I am, unfortunately, traveling all day Monday and will have limited ability to approve and respond to comments but I promise to do so as quickly as possible.

Share this blog post ...
Twitter4LinkedIn47Facebook1Google+0Email

Categorized under General Web Analytics, Random Thoughts, Vendors, Web Analytics Association

  • http://www.analog.cx/ Stephen Turner

    Wow. Strong views. Great subject for debate. And I agree with almost everything you’ve written. It’s not the IAB that has redefined “unique visitors”. It’s the web analytics vendors that have redefined “unique visitors” to suit their limited technology. We’ve all known for a decade or more that what we call “unique visitors” isn’t unique visitors, yet we’ve colluded in the pretence that it’s a good approximation.

    So yes, I wish it had been called “unique cookies” all along. Or since it might sometimes even be inferred from IP address & user-agent or something else, maybe we need a technology-neutral term: how about “circulation” (like newspapers) or something else?

    And yet. And yet. Having said all that, I honestly don’t think it’s changeable. It would require an unprecedented amount of co-operation and co-ordination from vendors, not to mention an enormous campaign of user re-education (and executive re-education!); and after all that you end up with a more precise but less user-friendly (and executive-friendly) term. I don’t think anyone cares enough. The only thing we might see is vendors giving an option of alternative language which can be turned on by customers who do care (that could just be an additional localisation, no harder than British vs American English).

  • http://www.webanalyticsdemystified.com eric

    Dr. Turner: Yeah, I guess I am turning into a grump in my old age but it is shocking to me that something that is so analytically correct is being met with resistance. I also don’t disagree with your statement that “we colluded in the pretence (sic) that it’s a good approximation” since cookies are more or less the best thing we have (or have had.) The reason we call cookies “unique visitors” is obvious — “Unique Cookies” sounds technical, is difficult to explain, and does not speak directly to management’s key concern: people.

    But given the IAB’s work I think we are finally to the point where we have an alternative and a workable one at that. All we need is someone to publish cookie deletion and multiple computer use rates by vertical on a monthly basis. comScore could do this, Nielsen could do this, Quantcast and HitWise other panel-based services could do this …

    Also, I don’t disagree with your assessment that the situation “is what it is” and is unlikely to change. But I would offer this: The vendors ** don’t really ** have to do anything — the onus is more or less squarely on the practitioner to begin to explain to management the differences between “what web analytics reports” and “the truth as best we can determine it.” That is all. So yes, it would be ** nice ** if the vendors came along, but no, I don’t expect them to change.

    I do wonder about your statement “I don’t think anyone cares enough” for one reason and one reason only: If nobody cared, why would the IAB bother rocking the boat the way they have? It would have been much easier for all those companies (see the list, it is very impressive) to decide that the status quo was the best we can do and focus their energies elsewhere. But they didn’t, and I suspect there was a reason …

    … who knows, maybe someone from IAB will comment.

    Thanks for your feedback, Stephen!

  • http://www.gartner.com Bill Gassman

    Great point of view, but it will be hard to change the lingo. I’m writing an article now, and here’s a sentence on the value of persistent cookies, using the new “Peterson” lingo.

    “For example, a weekly report that shows 1000 cookies to the Web site gains value if segmentation shows there were only 200 unique cookies, and 100 of them visited at least four times.”

    Somehow, I don’t think the CMO will get past thinking “huh” and maybe getting hungry for an Oreo.

  • http://www.webanalyticsdemystified.com eric

    (Bill commented in Twitter but I think his Tweet is worth following up on here.)

    Bill: No, I cannot imagine telling management that “we had 1,000 unique cookies visit our site” since that is clearly not what management wants to hear. But I think management would be receptive to “Based on our current knowledge of cookie deletion, multiple devices, and other obfuscating factors, we believe that the 1,000 cookies measured using {software} represents roughly 700 ** people ** visiting our site.”

    If they continue to ask, I would follow up with something like, “Our estimate is based on the observation that last month 30% of the visitors using our registered-users site who were known customers were given a new cookie. Since we expect these people to already have a cookie, our assumption is that these people either deleted their cookies or are using a different machine and thusly we use a correction factor of 0.7 against our cookie counts to arrive at “700 people.”

    My thesis is this: Management are not idiots and they understand there is an issue associated with counting visitors on the Internet. Given a decent stab at conducting the analysis and a reasonable explanation I rather suspect that they will “get it” and perhaps even appreciate the attention to detail.

    What do you think?

  • http://www.webanalyticsdemystified.com eric

    Bill: Oops, I responded to your Tweet before I saw your comment. You perhaps over-simplified my intention a little too much. Please see my comment above and see if that makes more sense. Thanks!

  • http://tompitts.org/ tom pitts

    i think changing unique visitors to unique cookies would be a good thing.

    quantcast already reports cookies and a metric they call people. this metric is algorithmically based and not equivalent to IAB’s unique visitors, but it wouldn’t be beyond the web analytics venders to come up with a similar unique people metric.

  • http://www.emetrics.org Jim Sterne

    We are definitely arguing about semantics – that’s the name of the definition game.

    While management are not idiots, they do have an infinitely small attention span.

    “Based on our current knowledge of cookie deletion, multiple devices, and other obfuscating factors….”

    Whoa, whoa, whoa! (I hear them cry). Just give me a single definition (I hear them beg) that makes sense to those who ponder such things, that can be used with consistency across all reports be they web analytics or panel based or a show of hands, so that I can generally count the number of people on my darn website. Please.”

    So let’s get specific.
    How about:

    a) individually identifiable people (IAB) – username, password, hair color, DNA sample
    AND
    b) inferred individual people (WAA) – robots & spiders filtered, cookie deletion & duplicate calcs, error rate correlations, Heisenberg uncertainty factorials, etc.

    This is what we know to be true: Jim logged in.
    This is what we believe is true: Jim came back three times without logging in.

    … and every vendor can have both metrics?

  • http://www.emetrics.org Jim Sterne

    And Eric – why the heck are you not on the WAA Standards Committee? Rather than saying “right” and “wrong,” how about being part of the solution?

  • http://www.gartner.com Bill Gassman

    The current metric “unique visitors” should really be labeled “unadjusted unique visitors”. Eric, you point out the need for new tools to estimate cookie deletion. Not everyone can afford to be a Quantcast customer. Equating anonymous user cookie deletion rate against named users is one method to estimate the deletion rate, but then we are justifying one inaccurate method with another. Managers want simplicity, even if there is complexity behind the scenes. Someone has to do the math on cookie deletion rate (per segment?) and add the “fudge factor” to the metrics that show up on the management dashboards and reports. The resultant calculated metric would be labeled with a qualifier such as “adjusted unique visitors” or “advertising unique visitors”. If we can at least get the industry to start adding a qualifier to absolute metrics, we’re getting somewhere.

  • http://blog.instantcognition.com/ Clint

    While I don’t disagree with the IAB definition of a unique visitor (and yes, there can be only one), it seems to me that the solutions to the IAB requirements for reporting unique visitors may take the census-based counts and turn them into something else.

    In other words, you would be taking one methodology (census) and applying another (sample/panel) to arrive at the unique visitor number.

    Both implementations have their flaws and weaknesses. And to quote you Eric, “two wrongs never make a right…”

    Might it not be wise to focus on the known instead of the unknown where the number of unique visitors is important?

    For instance, in some situations it is possible to say “I KNOW that we had XX number of unique, logged-in visitors over a certain time frame.”

    Everything else is an approximation and should be reported as such.

  • http://www.webanalyticsdemystified.com eric

    Tom: If Quantcast’s “people” are derived from A) a human-based panel and B) a cookie-collected measure, which if memory serves it is, then I suspect it would actually satisfy the IAB’s definition of “Unique Visitors”. If not, the ability to study the relationship between “people” and “cookies” across a large number of sites would be quite valuable to companies who didn’t have registration/other correlating data.

    Jim and Bill: Agreed this is about semantics, but it is also about the meaning associated with the words. If we agree that traditional web analytics systems count cookies — can we agree on that? — then I don’t see any inconsistency associated with calling the resulting count “Unique Cookies” (assuming the counts have been de-duplicated, which is an assumption.)

    I’m not sure I understand your point about “management begging for a single definition” Jim. Sounds like you agree with me — that Jodi’s post about using two definitions is a mistake? And I’m also not sure I understand the difference between “Jim” and “Jim returning three times” — why do we need two definitions of “Unique Visitors” to measure that?

    To Clint’s point, perhaps calling the metric you get when you take census-based counts and modify them using panel- or people-derived data “Unique Visitors” is saying too much — I agree that the result is technically not “people” — but as long as the definition of “Unique Visitor” is clear and the calculation is made transparently (something the IAB is clearly calling for) then I guess I fall back on K.I.S.S.

    Also, the solution the IAB has described is not using the panel data we have today: it is using the panel data the panels have but don’t actively report. See the comScore report on cookies for more … my understanding of this methodology is that it is not particularly inaccurate … but your assessment may differ.

    Put another way, I think the IAB definitions are very good and provide a basis for adding a “quantifier” to the “absolute metrics” Bill refers to (which are far from absolute in my experience, but I get your point.) All we have to do to take advantage of the IAB’s work is find an appropriate “factor” for each site (based on research, based on people, as the IAB has said …)

    Will this be hard? Again, yes. Will this take work? Again, yes. But I believe it can be done because I know the data is out there.

    Finally, since I am getting a LOT of email on this post from people who are somewhat more shy about the subject than I apparently am. One of the best, coming from a lead analyst looking at one of the largest groups of sites on the Internet, starts with this:

    “A huge problem as you say. I would agree with you that there are no shortcuts on this issue – analysts need to know the details whether they like it or not. If they don’t want to understand the nature of cookie tracking and how to use it effectively they are in the wrong business.

    All that is required to implement the IAB’s guideline is analysis. That’s what we do. We analyze, right?

  • Josh Chasin

    Hey Eric.

    First off, as per Joe Laszlo’s explanation of who the “audience measurement organizations” are, see? Toldja…

    It is probably trite for me to comment here, because my position on this topic is both known through my writiings, and somewhat predictable based on the fact that as Chief Research officer for comScore I’m a poster boy for the IAB standards and the conclusions therein.

    That said, I still applaud the position you have taken here. While I understand that audience measurement and web analytics are two different disciplines– with different heritages, applications, and often even different users within an organization– the two disciplines are inevitably taking furtive steps in each other’s direction. I know that many of the clients I meet with have said, “It would be great if somehow my comScore numbers and my Omniture numbers coul work more closely together.” So I think that among the universe of publishers who make regular use of both types of data, the respective value of each would be enhanced through some sort of, let’s call it orchestration, coordination, perhaps integration. (That’s why we’ve been speaking so much of late about panel-centric hybrids that integrate panel data with server data.) And essential to that is the clarity that arises from eliminating confusion at the place where the two datasets do begin to comingle.

  • http://www.gearyi.com Nicole Rawski

    While all along I have known that visitors are simply cookies and would often clarify this to management when inquiring how I knew the unique visitor metric. I will continue to speak to the activity on a website as if they are visitors. If I continued to speak to site activity as unique cookies in my reports and presentations, I think clients and management would be more confused than they sometimes already are.

    Yes, I think it is important to understand the underlying discrepancies that cookies may cause, but there has to be a term or way to speak that reminds you at the end of the day, these are people who are behind the activity on your website.

    I think it’s great to clarify this to the web analytics community, but I do not feel that any drastic changes need to occur. Excellent topic of discussion.

  • http://www.webanalyticsdemystified.com eric

    Josh: Thanks for your comment. In response I’d like to paraphrase John Lennon when he said, “You may say I’m a dreamer, but I’m not the only one, I hope some day the (analytics) world will join us, and (unique visitors) will be as one.”

    More seriously, I’m not sure I really want my Omniture data co-mingled with comaScore data. But I would really like it if you guys would sell/publish/give away all that wonderful data you have about the relationship between panel members and cookies. Hell, maybe you could sell it to the Web Analytics Association and they could give it away for the betterment of the entire community.

    Now there is an idea, huh?

    Anyway, nice to hear from you again and thanks for the comment.

  • http://www.webanalyticsdemystified.com eric

    Nicole: Thanks for your comment! I agree it’s a conversation worth having … at least now that the IAB has drawn a line and given us a good definition of “Unique Visitors” as differentiated from “Unique Cookies!”

  • http://www.emetrics.org Jim Sterne

    Eric – Upper management wants a single, functional definition. Technical accuracy is not as important as usefulness. We need two definitions so that those who derive the useful number know what we’re talking about. No executive on the planet is going to want to know how many cookies visited the website. They want to know how many hits there were. So – to cater to those who have neither the time nor the inclination, we need to hand executives a useful number based on very clear definitions. The difference between “Jim” and “Jim returning three times” is

    This is what we know to be true
    and
    This is what we believe (calculate) to be true

  • Shankar Mishra

    Interesting conversation. I couldn’t disagree with the IAB definition – that’s what the phrase means in English. Now every which way you try to calculate it, so that it’s meaningful for analysis, there is very high likelihood that you’ll be merely estimating it.

    Unique Cookies is a way to estimate UVs, Unique Cookies combined with survey may be another way to estimate it – so how about calling it “Estimated Unique Visitors” within Web Analytics world. Different tools can go on from there and claim how good their estimation may be compared to others.

  • http://www.webanalyticsdemystified.com eric

    Jim: I agree we need two definitions, this is exactly what I said in the original post. Web analytics practitioners (and theoretically the vendors) will learn to use “Unique Cookies” since that is a technically correct and 100% accurate description of the data. The IAB has done a good job of defining this term and I think we all understand it.

    When the practitioners translate “Unique Cookies” into that one magic number management needs they will ideally follow the IAB guidelines and make an attempt to “adjust” the cookie count to better represent real people. The IAB guidelines say to call this number “Unique Visitors”

    Two definitions, both really pretty clear IMHO.

    What is not clear is why anyone would want to have two completely different definitions for “Unique Visitors” for over-lapping audiences. I’m not sure that is what you’re advocating for (same word used two different ways, per Jodi’s blog post) but if it is then my friend we will agree to disagree. No harm there and thanks again for your comments!

    Shankar: Thanks for your comment. I see what you’re saying, but I think the IAB’s definition of “Unique Cookies” used in the current context (e.g., if you change “Visitors” to “Cookies” in the current web analytics applications) implies the estimate you refer to. And yes, these are all estimates, not matter how you cut it.

  • http://www.waomarketing.com/blog Jacques Warren

    What if our problem was our obsession with “unique”? Many other sectors don’t bother with it (Total viewership of Olympic Games, Museum & al. entrance, print sales/readership, etc.)

    Sure, when it comes to analyzing a *relationship*, we want to understand its intensity and its frequency. But by then, we should have something else than a cookie from that visitor.

    As for the bulk of a site traffic, after all, do we need uniques that much?

    Anyway, just a little thought on this immense topic.

  • Pingback: What Would Wonder Woman Do? » Eric T. Peterson, I’m mad at you

  • http://www.aprilwilson.net April

    This post made me so mad, my comments were too long for this little box. So I responded to you here:
    http://aprilwilson.net/blog2/2009/03/26/eric-t-peterson-im-mad-at-you/

  • Ned Kumar

    Eric, I hear your side of the debate but I also cannot refute Jodi’s comments. I think we are taking a myopic view of what unique “visitors” should be. The key is to look at how Management or folks using it expect to get out of it (not the dictionary definition of what a visitor is).

    If every person who came to a website created a login id or some sort of unique id, the IAB definition would be preferred and maybe the way to go. But the reality of the situation is that there are many situations where a user need not login or identify themselves — a perfect example would be say to track a package you shipped by FedEx or UPS. I can ship a package to you and then send the tracking number to a third party (say your admin) who can track it without identifying themselves. Are they visitors to the website? Obviously yes. Can I identify them as a person — Obviously no.

    Further there are many instances where one ‘login’ is passed around and used by multiple folks for whatever reasons (folks within a family; folks within a department etc.). So even with the correct semantic definition, I don’t think the ‘unique visitor’ count will be 100% unique visitors.

    At the end of day, (at least the way I see it) there are limitations to whichever road you take. On one hand, I agree that calling unique cookies as unique visitors is a misnomer, but then on the other hand mandating that we only call ‘identifiable visitors’ as unique visitors is impractical and myopic. I would rather use a definition that covers both and then use tools and technologies to improve the quality of that metric.

    And lastly, I am always a believer in the Pareto Chart principle — as applied to the needs and context of what you are trying to accomplish. If a definition serves most of what I want to get out of a situation, I would go with that even if it is not perfect or semantically correct.

  • http://www.webanalyticsdemystified.com eric

    I would like to publicly apologize to Jodi McDermott for the tone I used in the original post. Based on April Wilson’s “I’m mad at you!” post (worth reading) and my re-read of the original post, I was perhaps a tad harsh in my criticism. For that I am sorry.

    Jodi is a valuable WAA volunteer and has worked tirelessly over the years to make WAA efforts happen. And while I still very much disagree with her MediaPost article, I should have focused more on the essence of the article and less on Jodi’s specific language.

    I have no doubt Jodi will continue her efforts on behalf of the WAA Standards Committee and the WAA. The WAA is better for her contribution so again, apologies Jodi.

  • Pingback: Oindex -kehitysblogi » Blog Archive » Uniikkien kävijöiden problematiikka

  • http://www.redeye.com Garry Lee

    Looking back to the first response by Dr Turner highlights the most important point which is seeing how critical it is to the client and that we should try to state the definition as it relates to an individual company specifically.

    Whilst at RedEye we are immensely proud of our de-duplication algorithms and the unique way we hold data, we still recognise that it’s only for specific sites that we can actually achieve the IAB recommendations, so surely the compromise would be for vendors to state far more clearly in literature and within the product what method is used to calculate ‘unique visitors’, we are certainly not doing this as much as we should and I doubt others are either.

    As usual I think Eric stirs up a superb talking point and Jim throws in what I believe to be a good middle ground; ‘why can we not have two metrics reported within the solutions’, then when we know we have methods to calculate the IAB version (heavily log in based sites) the user has the option to hand. Again the only thing holding back such a solution is a vendor wide approach to an honest and open definition of how they calculate ‘visitors’.

    In regards the actual importance of knowing the inaccuracies, for those that want to know this information, its important (obvious but true), but its not hard to get these numbers, the reality of the excellent comScore report was that the numbers it produced differed very little from the original RedEye report produced in 2002, so it doesn’t need to be a new permanent product, a simple refresh every few years will do, as a community we now have a pretty stable set of numbers to infer the inaccuracy and going forward any studies should be conducted alongside the IAB and WAA for a consistent approach.

  • http://www.webanalyticsdemystified.com eric

    Garry: Thanks for your comment. One thing: I don’t believe that the RedEye report, my data from JupiterResearch, or the comScore report can be universally applied to cookie-based data in an attempt to calculate the IAB’s “Unique Visitors” number. While those numbers have been pretty consistent (30% over-counting or thereabouts) the data all suggests that the discrepancy is a function of the audience and audience’s vary. So ESPN will have a slightly different cookie-correction factor than Martha Stewart would, etc., right?

    Regarding the “white knight” who will ride in and start to provide (sell) cookie-correction data … I know it can be done, I’m just not sure who will do it. The first vendor I asked says it will eat into their core business to provide those numbers. But I’m going to keep asking vendors, and I suggest we all do this, until someone steps up and makes this type of data available on at least a per-vertical and per-geography basis.

    Cheers.

  • http://christopher-berry.blogspot.com/ Christopher Berry

    Unless a vendor makes a breakthrough that can detect the same human regardless of which device is used, we’ll never arrive at a valid measure for ‘people’.

    I’ll go so far as to say that it’s first time customer and repeat customer metrics that truly matter. Uniques go up – volume of conversions go up. What’s the correlation? What’s the time delayed correlation? What’s that coefficient? Is it 5? Great, awesome, good, done. We know how much we need to spend to drive what volume of ‘uniques’ to the site.

    We have perfectly good proxies for ‘people’. I’ll side with those who argue, rightly, that nobody should be doing any sort of modification of raw counts – whether they’re repeat logins or count of cookies.

    If the problem is communicating with management – use a *. Astricks the hell out it. We can’t expect anybody to become educated about anything these days.

    Great debate guys. A little heated for what I’m used to, but great.

  • Judith Pascual

    This is the debate of the month!
    I posted my thoughts on:

    http://synerinsights.blogspot.com/

    Cheers!
    Judith Pascual
    ZAAZ, Inc.
    WAA Standards Committee Co-Chair

  • Pingback: Continuing the Unique Visitors/Users conversation… | showmeanalytics.com

  • http://www.webanalyticsdemystified.com eric

    Christopher: Thanks for your comment. One question: When you say “I’ll side with those who argue, rightly, that nobody should be doing any sort of modification of raw counts – whether they’re repeat logins or count of cookies” do you mean that no type of sampling or mathematical modeling is appropriate in web analytics?

    I ask because there are whole branches of mathematics, science, and finance that make good use of “raw counts” subjected to mathematical modification through modeling, algorithms, and good old fashioned Stats 101.

    Agreed, an interesting debate. Thanks again!

  • http://www.webanalyticsdemystified.com eric

    Judith: Thanks for replying in your blog — it reminded me to add you to my RSS feed! Same as Angie you raise good points … in fact you raise a critical point when you said:

      “It’s not that I disagree with the IAB, it’s just unrealistic at this point and at this time we are attempting to establish a common language for ‘right now.’
      I think its good to expect more and move the industry forward.”

    I think that is the disconnect. The WAA Standards Committee has spent four years trying to describe a common language for “right now”. And I get that this is painstaking work and I have written fairly extensively about how wonderful a job you all are doing.

    But go back and re-read your current definition of “Unique Visitors” … while you have done a good job describing the current situation (an absolute mess), do you believe you’ve done a good job providing guidance regarding how that measure ** should ** be defined? I ask because of the statement “Because different methods are used to track unique visitors, you should ask your tool provider how they calculate this metric.”

    I guess when I read the IAB’s definition … even with the flaws it has regarding specific methodology (which are ultimately up to the MRC I guess) … I finally read a definition of “Unique Visitors” that resonated based on what ** I personally believe a Unique Visitor to actually be **

    I’m not talking about the availability of the metric or problems I may or may not have explaining the derivation to “the boss” … I’m talking about having a ** standard definition of the term that we collectively use to describe “people coming to the web site” which, in most people’s usage today IS WRONG. **

    So that is where we agree to disagree. Semantics I suppose, but an issue I see being of fundamental import as this industry evolves. In some ways this is the flat-Earth Society vs. Pythagoras and Aristotle … just because we have always used a definition does not make the definition right.

    Thanks to you and Angie both for taking the time to share your opinion and, unlike April, not making me feel like a total and complete ass in the process. While I’m a little taken-aback by the response I’m excited by the debate and looking quite forward to San Jose when we can hopefully have a nice conversation over drinks … my treat!

  • http://www.aeaw.es Sergio Maldonado

    Eric, I believe this is just the kind of open discussion we were all waiting for, so I thank you for a superb coverage and a clear position, and I really hope you can keep it up.

    I (btw, Spain’s Chapter of the WAA/AEAW) would break this down into three separate issues:

    a) Standardization
    b) Representation
    b) Accuracy

    a) In the first place, we are here because the IAB has relased the Guidelines. We know we need them. Marketing Directors are getting very tired of not being provided a simple set of common measures.

    Having finally gone through them, I get the feeling they are half-baked: Good job at accepting the limitations of the environment, insufficient job at simplifying the solution.

    I like Jim’s approach here:

    We must define what is true with tech-free terms. Or else they won’t fly.

    I agree we should stop using “Unique Visitors” while they are cookie-based… I have wasted enough time educating top management on the “inaccuracies” of the term (an effort that discredits our own industry). Now, I’d rather use “inferred individual people” than “unique cookies”.

    What about the new IAB definition for “Unique Visitors”? I don’t think we need to go all the way to IIP (question for the IAB?) so I believe login-based counts could do, and they could mean a lot in the future. But not in the “free web” (free beer) we have today (very much in line with Ned’s comment). In short: Why not just bury the term to avoid confusion, break clean from it and just leave it out of the Standards? (let me have my “logged-in” users count, which is very specific to my site).

    b) Representation. Why do we need to deal with IAB and WAA standards separately?

    I kind of find an answer in Josh’s comment: Panels (and I guess others) view Web Analytics as an outsider in the audience measurement game. But advertisers have a right to demand census-based data that goes as far as media can take it, as long as some of its measures stick to a Standard.

    SO: Time for the IAB to sit down with the WAA and come up with something really useful?

    c) Accuracy

    When reporting “inferred individual people”, panel data on cookie deletion will certainly come handy. Please, Eric, keep pushing for it! :)

  • Pingback: Asociación Española de Analítica Web

  • http://comet.co.uk Adam Marshall

    Bill Gassman said:

    >>Somehow, I don’t think the CMO will get past thinking >>“huh” and maybe getting hungry for an Oreo.

    lol. Does that make him a “hungry hungry HIPPO”?

    But yeah, an important question here is – to whom does the distinction make a real difference? We should always look to see whether exact data is actually important, or whether we can get away with just trending apples (with maybe a few pears thrown in)…

  • Judith Pascual

    Hey Eric,
    Thanks for taking time out to respond.
    I think we will get there.

    P.S. I was hoping that the real interest would have been in privacy.

  • http://www.webanalyticsdemystified.com eric

    Judith: No way, thank you for taking the time to respond to my criticism! I’m not sure why Avinash didn’t call out your post when he credited Angie this AM in Twitter but you both get credit in my book for being open to the discussion.

    Regarding privacy … I don’t think this is a privacy issue at all. Nowhere in my read of IAB guidelines or in my suggestions are we talking about identifying unique individuals. All that has been proposed is to use some secondary source ** based on counts of real people ** (which is not the same as individuals, do you see the difference?) to mathematically correct cookie-based summations.

    So, for instance, if you are lucky enough to have a client-login on your site, you would count A) the number of unique client login events and B) the number of “Unique Cookies” associated with said login events. If A = B then you have your number … Unique Cookies / 1.0 Cookies per Visitor = Unique Visitors. QED.

    If, however, you discover that you have 1.2 Cookies per Visitor (login event) on average, well, the math is still the same and your calculated Unique Visitor count < Unique Cookie count. No need to bother management with the details unless they ask, no need to identify anyone, etc. Again, QED.

    And yes, if you don’t have a login you will need to get the data somewhere else. But the data is out there and some pretty sharp folks are working on making more of it available (trust me!) If you wanted you could use the comScore data (which is consistent with RedEye and JupiterResearch, see above) and give management a range.

    In this case, using 1.3 cookies which is what I recall the number being (but I could be wrong …) Unique Cookies / 1.3 Cookies per “Visitor” = Unique Visitors will give you an “Estimated Unique Visitors” range you can present as in “Based on available data we believe the number of Unique Visitors to the site to be between X and Y during the month of Z.”

    Before anyone barfs on the idea and says “Oh man, management would be so mad if we said that!” keep in mind that if you’re presenting Unique Cookie counts to management as if they accurately represent real people today you are being purposefully misleading and non-transparent and so perhaps providing a range and an explanation is not a bad alternative?

    Still, nothing about privacy or personal identification … that is another ball of wax, perhaps for you to blog about my friend! ;-)

    Thanks again for taking the time to chime in.

  • Ned Kumar

    I still think we are missing the point by being so focused on outlawing cookie counts. At least from my limited experience, the key is not to throw cookie counting out of the window but use it judiciously along with real id counts (client-login or otherwise) whenever available and wherever applicable.

    I am an analytical guy and totally understand your suggested formula regarding cookies per visitor but to me at the end of the day it comes down to cost/benefit from a Strategy, Management needs, and Customer Experience perspective — the question being is it worth going to ‘X’ number of sources and doing ‘n’ number of calculations to get a realitvely more “accurate” count (yes, relatively — as I don’t think any formula can give you a perfect answer) when the raw counts with good assumptions and context can serve the needs well enough.

  • http://www.webanalyticsdemystified.com eric

    For those of you still paying attention to this “debate” you may want to read Neil Mason’s piece in Clickz from March 31st titled “When Is a Visitor not a Visitor?”. Neil, a Director in the Web Analytics Association says “I agree with Peterson’s assessment (if not entirely with his style and approach) that the unique-visitor metric from a Web analytics tool is potentially very misleading.”

    Neil goes on to provide good, sound, and appropriately dispassionate (compared to my use of the “bully pulpit”) and perhaps my favorite of his comments is this one:

      “One of my big concerns about the unique visitor metric from a Web analytics tool doesn’t involve its “technical efficacy” but more the messages it sends out. The implication from reporting on visitors from a Web analytics tool is that you can fully understand your Web site visitors using clickstream behavioral data.

      Because we call this metric “unique visitors,” it lulls us into a false sense of security that we’re actually tracking “visitors” rather than “devices,” and therefore don’t really need to understand anything else.”

    Brilliant. Anyway, give Neil’s article a read and I welcome your continued comments.

  • Pingback: Evropský parlament chce zakázat cookies? | Web Analytics Wednesday (CZ)

  • Pingback: Is Your Attribution Model Appropriate? | Web Analytics Demystified

  • Pingback: Why counting uniques is meaningless | Measuring Success

  • Pingback: Page Views: Part 2 - Scripted Metrics

  • Pingback: More color on Adobe + Omniture | Web Analytics Demystified

  • Pingback: ProductizeMe

  • David

    Just explain to management that the term “Unique Visitors” is like the term “Market Value” and they should get it.

  • Pingback: From Worst to Best: How to Report Metrics & Measures « Kraft T Thoughts

  • Pingback: Voice of Customer Analysis « Kraft T Thoughts

  • http://www.chooseusfirst1188.com Harley Warren

    I came across this article and posts while surfing. I find the information quite interesting. I wounder what developments have been made in the years since the oringin of the article and posts.

  • Pingback: Top Links for Help with Deciphering Web Analytics « Dr. Diane Hamilton's Blog

 


Recent Blog Posts

Current Order Value
Adam Greco, Senior Partner

I recently had a client pose an interesting question related to their shopping cart. They wanted to know the distribution of money its visitors were bringing with them to each step of the shopping cart funnel.

Continue reading this article ... ... more from Adam Greco

A Guide to Segment Sharing in Adobe Analytics
Tim Wilson, Partner

Over the past year, I've run into situations multiple times where I wanted an Adobe Analytics segment to be available in multiple Adobe Analytics platforms. It turns out…that's not as easy as it sounds. I actually went multiple rounds with Client Care once trying to get it figured out. And, I’ve found "the answer" on more than one occasion, only to later realize that that answer was a bit misguided.

Continue reading this article ... ... more from Tim Wilson

Currencies & Exchange Rates
Adam Greco, Senior Partner

If your web analytics work covers websites or apps that span different countries, there are some important aspects of Adobe SiteCatalyst (Analytics) that you must know. In this post, I will share some of the things I have learned over the years related to currencies and exchange rates in SiteCatalyst.

Continue reading this article ... ... more from Adam Greco

Linking Authenticated Visitors Across Devices
Adam Greco, Senior Partner

In the last few years, people have become accustomed to using multiple digital devices simultaneously. While watching the recent winter Olympics, consumers might be on the Olympics website, while also using native mobile or tablet apps. As a result, some of my clients have asked me whether it is possible to link visits and paths across these devices so they can see cross-device paths and other behaviors.

Continue reading this article ... ... more from Adam Greco

The 80/20 Rule for Analytics Teams
Eric T. Peterson, Senior Partner

I had the pleasure last week of visiting with one of Web Analytics Demystified’s longest-standing and, at least from a digital analytical perspective, most successful clients. The team has grown tremendously over the years in terms of size and, more importantly, stature within the broader multi-channel business and has become one of the most productive and mature digital analytics groups that I personally am aware of across the industry.

Continue reading this article ... ... more from Eric T. Peterson

Ten Things You Should ALWAYS Do (or Not Do) in Excel
Tim Wilson, Partner

Last week I was surprised by the Twitter conversation a fairly innocuous vent-via-Twitter tweet started, with several people noting that they had no idea you could simple turn off the gridlines.

Continue reading this article ... ... more from Tim Wilson

Omni Man (and Team Demystified) Needs You!
Adam Greco, Senior Partner

As someone in the web analytics field, you probably hear how lucky you are due to the fact that there are always web analytics jobs available. When the rest of the country is looking for work and you get daily calls from recruiters, it isn’t a bad position to be in! At Web Analytics Demystified, we have more than doubled in the past year and still cannot keep up with the demand, so I am reaching out to you ...

Continue reading this article ... ... more from Adam Greco

A Useful Framework for Social Media "Engagements"
Tim Wilson, Partner

Whether you have a single toe dipped in the waters of social media analytics or are fully submerged and drowning, you’ve almost certainly grappled with "engagement." This post isn’t going to answer the question "Is engagement ROI?" ...

Continue reading this article ... ... more from Tim Wilson

It’s not about "Big Data", it’s about the "RIGHT data"
Michele Kiss, Partner

Unless you’ve been living under a rock, you have heard (and perhaps grown tired) of the buzzword "big data." But in attempts to chase the "next shiny thing", companies may focus too much on "big data" rather than the "right data."

Continue reading this article ... ... more from Michele Kiss

Eric T.
Peterson

John
Lovett

Adam
Greco

Brian
Hawkins

Kevin
Willeitner

Michele
Kiss

Josh
West

Tim
Wilson

Contact Us

You can contact Web Analytics Demystified day or night via email or by reaching out to one of our Partners directly.

» Contact Information

Web Analytics Demystified, Inc.
P.O. Box 13303
Portland, OR 97213
(503) 282-2601


Useful Links