Eric Peterson's Blog at Web Analytics Demystified

This summer’s web analytics acquisition season has heated up to the point where when my phone rings and it’s John (who wakes up hours earlier) saying “comScore has acquired Nedstat” my response is “of course they have!” Not to say this isn’t an exciting acquisition, but wow the vendor landscape has changed a bunch this year …

John spent the morning on the phone with our friend Jodi McDermott and comScore’s CEO Dr. Magid Abraham talking about the decision for comScore to get more deeply into client-side measurement technology and he promises to have a more comprehensive post up in a day or two. I only wanted to weigh in and say “congratulations” to the entire team at Nedstat!

I have been lucky enough to have worked with Michael, Fred, Michiel, and Ulrike on a number of occasions and have produced two white papers with them (one on video, the other on mobile) with a third coming out in a few weeks. The management team and everyone I have interacted with at Nedstat are wonderful people and they will definitely add great value and expertise to the comScore family.

Again, watch for more detail and analysis from John in the coming days and congratulations to the comScore and Nedstat families from all of us here at Web Analytics Demystified.

Posted Wednesday, September 1st, 2010 | No responses | Add a Comment | Share, Save or Email

Back in February of this year, in partnership with BPA Worldwide, Web Analytics Demystified published a white paper detailing the risks associated with the use of Flash Local Shared Objects (LSOs) in digital measurement. Titled “The Use of Flash Objects in Visitor Tracking: Brilliant Idea or Risky Business?” the paper drilled down into how some companies are using Flash LSOs and offered  the following guidance:

1. Do not use Flash to reset browser cookies
2. Disclose the use of Local Shared Objects
3. Allow site visitors to disable Local Shared Objects

The first piece of advice turns out to be pretty important since companies are now being sued over their use of Flash to reset browser cookies. MTV, ESPN, MySpace, Hulu, ABC, NBC, Disney, and others are being dragged into a lawsuit based on their use of Quantcast and Clearspring who were identified by Soltani, et al. as using Flash LSO to reset deleted browser cookies. These lawsuits allege a “pattern of covert online surveillance” and seeks status as a class action lawsuit.

Yikes.

Fortunately for Adobe they do not seem to be one of the targets in these suits, which makes sense considering the position the company has taken regarding the use of Flash. In my interview with MeMe Rasmussen, Adobe’s Chief Privacy Officer back April of  this year, Mrs. Rasmussen explicitly stated:

“… the position we outlined in the FTC Comment on condemning the misuse of local storage, was specific to the practice of restoring browser cookies without user knowledge and express consent.  We believe that there are opportunities to provide value to our customers by combining Omniture solutions with Flash technology while honoring consumers’ privacy expectations.”

On the topic of consumer privacy and web analytics, following up my partner John’s response to the Wall Street Journal article on online privacy (“Be still my analytical heart”), I recently wrote a piece for Audience Development Magazine titled “You are all evil …” While a little tongue-in-cheek the article encourages marketers and business owners to:

1. Have a rock-solid privacy policy
2. Not use tracking software they don’t understand
3. Not be unaware of what tracking software they have deployed
4. Have a clear answer for “how and why do you track us?”
5. Be transparent as hell when anybody asks what you’re doing

As I reflect back on the guidance we have provided in the past year I run the risk of becoming quite depressed. None of our recommendations are surprising, revolutionary, or particularly Earth shattering … but not nearly enough companies are doing most of these very simple things. Given this, one possible outcome is becoming increasingly apparent …

We are going to get screwed.

Go back to Walt Mossberg’s 2005 assertion that “cookies are spyware” and the related conversation around cookie deletion and you will see a clear pattern: media (ostensibly acting in the best interest of consumers) points out that what we do is somehow devious … and we more or less ignore the problem, hoping it will go away.

My friend Bob Page once referred to something he called the “Data Chernobyl”  … a unexpected and massive meltdown in consumer trust associated with the data that we collect, store, and use to make business decisions.  When you think about it for just a little bit the idea is terrifying … because everything we do depends entirely on our ability to collect, store, and use information about consumer behavior on the Internet.

Our livelihoods depend on everyone ignoring the fact that we track, understanding why we track, or getting something tangible out of the tracking we do.  Sadly we have never offered anything tangible, we have never really made an effort to explain what we do in court of public opinion, and it is increasingly clear that the bright light shining on our trade isn’t going to fade anytime soon.

What’s worse is that we are collecting even more information across mobile, social, and other emerging channels, perfecting our ability to integrate that data into over-arching consumer data warehouses, and occasionally using techniques that even the most hard-hearted of web analysts get all geeked-out about.

We have become our own worst enemy.

Now, as I declared in the Audience Development piece, I simply do not believe that consumers are as freaked out about tracking online as the media makes them out to be … the data I have seen just doesn’t support that conclusion. But consumers aren’t the real problem: the real problem: is the media, lawyers, and potentially the Federal Government. All three of these groups continue to generate page views, make money, and “protect the common man” (sic) by throwing our industry under the bus … and we aren’t doing anything in our defense.

Dumb, dumb, dumb.

People much smarter than I am have repeatedly stated that they don’t want to engage the media or “privacy police” in a conversation that they cannot possibly win.  To a small extent this makes sense, but at some point I wonder if we are going to collectively end up looking like my four year old when he knows he’s made a mistake.  My son gets away with it because he’s awesome cute and I love him, but I am beginning to think the collective web analytics industry is not going to get away with mumbling and making lame excuses for much longer.

The advertising industry has the IAB and NAI, both of whom appear to be responding to articles, lawsuits, and Congressional investigation on many of these issues.  (If you haven’t seen it yet, have a look at this amazing “privacy matters” campaign the IAB is running.) But we are not the advertising industry, we are the web analytics and digital measurement industry, and we need to have our own voice, our own lobby, and our own representation.

Since the framework for this already exists, I am officially asking that the Web Analytics Association formalize and finalize their Industry Advocacy program and represent the digital measurement community in the forum of public opinion.

I have already volunteered to help with this effort under the Presidency of Alex Langshur and reiterate that commitment to the current Board of Directors.  The WAA needs to bring together corporate members and key practitioner representatives to quickly hash out a clear, concise, and practical position on the relationship between digital measurement technology and consumers. The current WAA Board is in perhaps the best position in years to make the decision to represent the needs of our community … but decisive action is required.

Without the WAA’s leadership on this issue I fear that over time we will lose the battle of public opinion and my tongue in cheek assessment of the “evilness” of our industry will be far less funny than it seems today.

Let’s not let that happen.

We are an awesome industry full of brilliant people.  The work we do is some of the most valuable but least understood in the interactive world.  I believe it is time to come out of the closet, accurately describe the value of the work we do, and stop shying away from a conversation we feel is stacked against us and a battle we are unsure that we can win.  If we don’t try, without a doubt, we will remain our own worst enemy.

Posted Sunday, August 29th, 2010 | 15 responses | Add a Comment | Share, Save or Email

What a crazy week it has been, what with client visits with John here in the West, web casts with the fine folks at Tealeaf and Unica, and the end of summer fast approaching at the Peterson household. I was so busy I wasn’t able to pay close attention to our X Change registrations and when I looked just now I realized something …

X Change 2010 is damn near sold out.

Thanks to some quick thinking from Joel, Grace, and Gary over at Semphonic we have a few more seats available than last year, but with nearly a month to go before we convene in Monterey, California at the beautiful Monterey Plaza Resort and Spa we have sold more seats than last year and last year was completely sold out!

You’re not gonna miss the X Change because you waited too long to sign up, are you? You aren’t going to risk missing out on the chance to discuss digital measurement in our intense and intimate conversation format with practice leaders and managers from amazing brands like Best Buy, ESPN, Expedia, Facebook, MTV, New York Times, Lowes, Turner Broadcasting, HP, Salesforce.com, Nike, Charles Schwab, Comcast, eBay, and NBC Universal, are you?

Seriously, don’t miss out.

I only wish I could make a list of all the great participant companies coming to this year’s event … but I can’t.  If I could you would see that by coming to this year’s X Change you would be joining some of the most respected brands in technology, media, healthcare, advertising, software, and retail in the world.  Worse, you would realize that missing X Change means not getting to hear first-hand how some of the greatest minds in the digital measurement industry are getting it done today.

You would be bummed.

Don’t be bummed, come to X Change 2010, September 20, 21, and 22 in Monterey, California. Register at our web site now or contact me directly for more information.

Posted Thursday, August 26th, 2010 | No responses | Add a Comment | Share, Save or Email

Today I am really excited to announce the publication of our framework for mobile and multi-channel reporting, sponsored by OpinionLab. You can download the report freely from the OpinionLab web site in trade for your name and email address.

This paper builds on our “Truth About Mobile Analytics” paper we published with our friends at Nedstat last year and focuses on both measurement in mobile applications and, more importantly, a cross-channel measurement framework built around interactions, engagement, and consumer-generated feedback.

• Interactions occur in every channel, digital or not. Online and on mobile sites we call these “visits” (although that is a made up word for interactions); in mobile apps the interaction starts when you click the icon and ends when you click “close”; in SMS it starts when you receive the message; on the phone it starts when you dial, and in stores interactions start when you walk up to an employee.
• Engagement is simply “more valuable” interactions. Regardless of your particular belief about the definition of engagement, we all know it when we see it. Online it happens after some number of minutes, or clicks, or sessions, or whatever; in mobile apps it happens when you’ve clicked enough buttons; on SMS it happens when you respond to the message; on the phone it starts when you begin a conversation, and the same is true in a physical store.  We say engagement is “more valuable” because without engagement, value is unlikely to manifest.
• Positive Feedback happens when you do a really, really good job. Measuring feedback is a critical “miss” for far too many organizations. Apples “app store” and the value of the star-rating system has essentially proven that there are massive financial differences associated with positive and negative experiences … but most companies still make the mistake of ignoring qualitative feedback altogether.

These three incredibly simple metrics can be applied to every one of your channels, your sub-channels, and  your sub-sub-channels (if you like.)  When applied you can create an apples to apples comparison between your web, mobile web, mobile apps, video, social, etc. efforts.

Then you can apply cost data, and you’re really in business.

I don’t want to say much more than that but I would really, really encourage you all to download and read this free white paper. When we put something like this out — something we believe has the power to really transform the way everyone thinks about the metrics they use to run their business, and something that has the potential to force dashboards everywhere to be scrapped and started over — we’d really like your collective feedback.

DOWNLOAD  THE WHITE PAPER NOW

Thanks to Mark, Rick, Rand, and the entire team at OpinionLab for sponsoring this work. If you’re the one person reading my blog that hasn’t seen their application in action, head on over to their site and have a look.

Posted Wednesday, August 4th, 2010 | 3 responses | Add a Comment | Share, Save or Email

Earlier this year after we launched the Analysis Exchange we put out our first challenge to the membership. We asked people to “be exceptional” in their participation, to step up and make a difference by working harder than expected, by bringing crazy passion to their work, and by participating in unexpected ways.  In exchange for “being exceptional” we said we would provide a complimentary pass to one mentor and one student to this year’s X Change conference in Monterey, California September 20, 21, and 22.

Today I am pleased to announce your exceptional winners of this challenge.

While nearly everyone who has participated in Analysis Exchange thus far has really blown my mind with their energy, their commitment, and their willingness to do something special for the larger web analytics and nonprofit communities, five people really stood out in the crowd.

• Sarah DeAtley, Mentor from Washington who worked like crazy to sign up her fellow Seattle-ites and continues to evangelize for the effort;
• Victor Acquah, Mentor from Virginia who participated in both our Alpha and Beta tests with PBS and provided tremendously valuable feedback;
• Jason Thompson, Mentor from Utah who has stepped up repeatedly to mentor projects and has helped a great deal to spread the word;
• Jan Alden Cornish, Student from California who has not only participated in multiple projects but has been an invaluable source of ideas and feedback;
• Michael Healy, Student from California who has helped out on numerous projects and who really understands what Analysis Exchange is trying to do.

Unfortunately not everyone would have been able to make the X Change this year due to previous commitments; fortunately that made our job selecting the finalists nominally easier.  To make our final decision we asked everyone to send us a short paragraph describing “what they have learned” in Analysis Exchange to date.  Here is what we heard back:

From Jason Thompson:

“First let me say, that with the amazing cast of students and mentors
that make up the Analysis Exchange, I am truly humbled to be
considered for this honor.

The Analysis Exchange has reminded me that what makes us truly rich is
not the contents of our wallets or how much money we have in our bank
accounts, whoa….sorry…started channeling Tyler Durden there for a
second, but what makes us truly rich are the relationships we have in
life.

So what have I learned thus far?  I have learned that we are all
students and that if we are open, there are many great lessons for us
to be taught.

I didn’t join the AE with the thought of getting anything in return,
although I have been given many wonderful gifts through my
participation. I joined because the AE provided me with the
opportunity to give back to an industry that has given me so much.

I honestly feel a little bit weird writing this email and the humble
Jason in me says this opportunity should go the person who would
benefit from it the most.”

From Jan Alden Cornish:

“My two Analysis Exchange student projects have demonstrated several key takeaways. First, project management principles are paramount. The project kick-of meeting should reinforce the Analysis Exchange project priorities:

• Schedule: short-term project
• Cost : limited resource availability, with the expectation that the student have the most flexibility
• Scope:  the project deliverables given the constraints

The project kick-off meeting should confirm the core roles and responsibilities of the project team members.

• Organizational Lead: the key business stakeholder who approves the project scope and project deliverables. The organizational lead may be the project manager
• Mentor : the web analysis expert / consultant. The mentor may be the project manager.
• Student: the primary execution resource

The project team has to rapidly converge on meeting times for reviewing and also for approving deliverables. The team needs to determine web / audio conferencing details: Skype, GoToMeeting, etc.

The first milestone is approving the project charter, which provides the scope definition and defines the team members’ roles and responsibilities.  Scope definition may require more than one meeting, particularly if the business plan of the nonprofit organization does not clearly set forth measurable expectations for their web presence.  For example, the organizational lead may be accountable for other time critical projects (i.e. migration to a new content management system). The Analysis Exchange project may run concurrently with other projects within the organization that impact its web presence (i.e, an outsourced web redesign).

A second key takeaway for me was that the nonprofit  world is a microcosm of the real world. Thus, risk management is key. A risk is a potential issue which might adversely impact the success of the project. Risks may be categorized as technical or organizational. The web implementation may not allow certain analysis questions to be answered.  Nonprofits often leverage third-party platforms for key business functions such as e-newsletter management, volunteer recruitment, and e-commerce. These functions might not be tightly integrated from an analytics reporting standpoint. They may “roll up” to other departments in the nonprofit’s organization. The nonprofit ’s web implementation also may have a limited deployment of Google Analytics. Other web analysis tools that are deployed may not may not offer similar functionality to Google Analytics.

Armed with these takeaways from my first two Analysis Exchange projects I look eagerly forward to my next project.”

And from Michael Healy:

“Superhuman effort isn’t worth a damn unless it achieves results. – E. Shackleton

In my experience thus far with the Analysis Exchange I learned that the bounce rate, page views, time on page and every other web metric pretty much aren’t worth anything. More accurately, they aren’t worth anything to the client unless they start to solve a business problem.

Considering that the organizations in the Analysis Exchange aren’t selling anything per se, but instead are providing a non-profit service to others presented a few challenges. Working with Gordon Holstlander, of the Circle Drive Alliance Church of Saskatoon, and Michael Helbling, my excellent mentor, I learned how to work together as a team to move beyond the prima facie challenge.

Our project involved what appeared to be a simple analysis of the home page real estate to determine the best usage of the page. I built out several personas of usage for the CDAC website which showed dynamic access to information; with different goals at different times of the day, week and year. Answering ‘it depends’ to Gordon’s original question delighted my Econ brain to no end.

Moving beyond the population of people who already accessed the site, I was able to show Gordon how to do a simple Google Trends search for the Saskatoon area. An examination of the entire CDAC website also revealed a great source of underutilized content. These two were passed onto the client for future SEO usage.

The biggest lesson in the Analysis Exchange thus far has been the open dialog and client relationships developed. Websites can be very personal things, with people at non-profits often pouring countless hours into improving them.Michael facilitated an exchange of ideas with Gordon and me; such that when I made my presentation all parties were open to improvements. That is a lesson I will lean on for the rest of my career.”

As you can see we are honored to have members who are so thoughtful, intelligent, and especially in Jan’s case, precise! In the end the decision was nearly impossible to make … until Michael Healy mentioned that he would be coming to the X Change regardless of the outcome.  To smooth that path we are sending Michael our “maximum discount” code for the conference and will ensure that he drinks his fill at the bar each evening.

Which leaves us with Jason Thompson and Jan Alden Cornish, this year’s Analysis Exchange at the X Change contest winners!

Jason and Jan will be coming to the X Change compliments of Web Analytics Demystified and Semphonic, co-hosts for the conference. I hope you will all join us in comments congratulating Jason, Jan, and all of our distinguished members! And if you are lucky enough to be joining us at this year’s X Change conference make sure to find Jason and Jan and congratulate them in person, shake their hands, and ask them about their experience in this effort.

The Analysis Exchange is always looking for more volunteer students, mentors, and nonprofit organizations. The X Change conference will be held September 20, 21, and 22 at the beautiful Monterey Plaza Hotel and Spa in Monterey, California.

Posted Monday, August 2nd, 2010 | No responses | Add a Comment | Share, Save or Email