Archive for 'X Change'

Well folks, it’s that time of year again. The winds are dying down and the flowers have all started to bloom so it must be time to make our annual pilgrimage to San Jose to bask in the glory of Jim Sterne and the Emetrics Marketing Optimization Summit! As usual I will be there and have the honor of sharing a keynote slot with my long-time friend and uber-optimizer Bryan Eisenberg!

• Emetrics Keynote: Wednesday at 1:00 PM in the Grand Ballroom

Partner John Lovett will also be there, basking in his own glory on the heels of his Web Analytics Association victory … and taking the WAA’s new Certification test. I haven’t really had much time to think about the Certification yet but will be interested to hear what John and others taking the test have to say.

I also have the rare honor of presenting with Brett Crosby, Group Product Manager for Google Analytics and one of the nicest guys in the entire industry, hand’s down. Oddly he and I are presenting IMMEDIATELY AFTER his “What’s new from Google Analytics” pitch on Tuesday … but to compensate we’re gonna try something new and have a very loose “conversation” about web analytics that is more similar to an X Change mini-huddle than a traditional presentation.

• Talking Analyics: Tuesday at 2:00 PM at The Conversion Conference (co-located w/Emetrics)

Finally I will be sharing the stage at Web Analytics Wednesday with Adam Laughlin from the nonprofit Save the Children. We will be talking about our respective community education efforts — his “Web Analytics Without Borders” WAA initiative and our own Analysis Exchange. I will be making a few exciting announcements about The Analysis Exchange next Wednesday so if you cannot attend Web Analytics Wednesday please watch my blog or follow me on Twitter.

• Web Analytics Wednesday: Wednesday at 6:00 PM at the Fairmont in San Jose

That schedule again:

• Tuesday, 2:00 PM at The Conversion Conference with Brett Crosby (Google)
• Wednesday, 1:00 PM at Emetrics with Bryan Eisenberg (Emetrics Keynote)
• Wednesday, 6:00 PM at Web Analytics Wednesday (The Fairmont Hotel, Market Street Foyer)

Thanks to Coremetrics and SAS for their generous support of Web Analytics Wednesday at Emetrics, by the way. Great companies like these are what keep WAW events around the world free and open to everyone!

See you in San Jose!

You may have already noticed this when you went to the registration page if you’re still considering the X Change next week but we officially put a cap on things last Thursday. While I’m disappointed that more people won’t be able to join us, it is incredibly gratifying to know that in the midst of the worst business economy in decades that smart people are still able to get management approval for continuing education, networking, and professional development.

I am certainly excited about the group we have coming next week: some great vendors, some awesome consultants, an incredible keynote event, and many of the best and brightest practitioners in the digital measurement industry. Excellent!

Also, as Gary pointed out in his blog post today, if you really, really, really need to join us in San Francisco and have already gone to bat for the budget, just let me know. We can always squeeze one more in, but we’ll probably make you attend Gary’s Think Tank training session just so he feels a little better (see his blog post for the back-story …)

If you can’t make it and you’re on Twitter please watch for conversation and insights on the #xchange hashtag in Twitter. Like Gary, I’m not foolish enough to promise to blog from the conference (hell, I barely blog as is anymore … too busy with work I guess!) but I will definitely try and push up 140 characters here and there, or slightly less when I co-tag with #xchange and #measure LOL!

If you’re coming to San Francisco, I look forward to seeing you next week!

P.S. piggybacking on Gary’s comments about WebTrends … I’m with Gary and Phil on the whole “9″ release. I’m encouraged by the company making a move in the right direction, but I feel that the release was dramatically over-marketed and set a new, all-time high for “hype over substance” in this industry making even the great green machine look conservative. When your own staff are forced to admit the release is not “the new UI” despite marketing’s claim that “[9's] clean, professional interface lets you creatively explore your data like never before,” well something has gone wrong somewhere.

That said, it’s great to see Alex committing to the product and, at the end of the day, it’s not what Gary, Phil, or I think … it’s what their customers and prospects think that counts. I know a handful will be at the X Change, along with someone from WebTrends marketing, so that interplay will be … ummm … interesting to watch!

If you’ve been reading my blog for any amount of time you’ve inevitably heard me comment that I think “web analytics is hard” — not complex, not mysterious … just plain difficult. It’s hard to select vendors, hard to install code, hard to train users, hard to get the “right” reports, hard to get management’s attention, hard to make the case for change … the list goes on and on (and on and on and on!)

Hard, but not impossible.

In the past few years we have definitely started seeing an increase in the number of companies that “get it” — so much so that we’re able to program an entire conference built around the superstars of web analytics. More and more I am talking to, working with, and hearing about companies who have leveraged the “web analytics is hard” mindset to properly set expectations regarding their use of technology, their deployment of people, and their use of business process to really excel at web data analysis and turn this analysis into tangible improvement for both the business and their customers.

As I collect more information about these web analytical competitors one thing that nearly always emerges as a hallmark of their success is some type of structured testing program. Of course this makes perfect sense because without testing the analyst is never really sure about the impact of their recommendations, so much so that I have often said “if you’re not testing, you’re not really doing web analytics!”

But the increase in testing has raised an interesting corollary to my “web analytics is hard” manifesto … testing is hard too!

Fortunately we’re all a little bit older and a little bit wiser this time around and we recognize that testing requires more than just throwing code on the page and clicking the “Optimize” button. Testing is a process that requires people and technology … sound familiar?

I’m bringing this up for two reasons:

1. At the X Change conference in San Francisco on September 9, 10, and 11 Matthew Wright from HP will be leading a conversation titled “Testing, Testing, Testing: Building Consensus and Evaluating Results” to discuss the nuances behind testing, things like getting stakeholder approval, planning, and clearly defining measures of success;
2. This morning the nice folks at SiteSpect published a white paper I wrote that details ten “best practices” for testing that I think a lot of folks new to testing often forget. Titled Successful Web Site Testing Strategies: Ten Best Practices for Building a World-Class Testing and Optimization Program, this white paper is freely available (requires registration) and covers nuances like testing teams, stakeholder involvement, test plans, timelines, and, of course, measurement.

If you’re working to become an analytical competitor and join the ranks of the kinds of companies who get invited to lead a conversation at X Change I highly recommend either grabbing the testing white paper, coming to the X Change, or BOTH! Especially if you’re serious about testing I think you’ll find this free white paper useful when you work to set expectations, and trust me, testing is as much about expectations as it is execution!

Register to download the free white paper on successful multivariate testing strategies!

Following up my interview with Bill Gassman a few weeks ago I realized that I would be remiss if I didn’t build on Forrester’s recent Web Analytics Wave report with an interview with John Lovett. John, like Bill, totally, totally understands the web analytics industry, and in that understanding is able to clarify the marketplace in a way few others can. Don’t believe me? Check out his response to possibly the worst article about web analytics, ever. Measured, polite, even complimentary … that’s John.

I am personally honored that John accepted my invitation to return to the X Change this year and both lead the huddle on “Industry Standards (or a lack thereof)” and co-lead a huddle on technology with Bill Gassman. If you haven’t met John personally, and if you are able to join us at the X Change, I strongly recommend you make a point of introducing yourself to him.

Finally, before my questions and John’s answers, I wanted to point out how incredibly deft Mr. Lovett really is: in response to a high-and-hard fastball question about “which vendor is really the best,” John knocked the ball clear out of the park with his answer: none of them. I’ll let you read the rest for yourself …

---

Your recent Wave report really emphasized a lot of conventional wisdom about the web analytics vendors but had some surprises for folks.  What surprised YOU the most about the Wave results?

Well Eric, I like to say that surprises are for birthdays and not for business. So in terms of actual surprises, there weren’t any big bombshells for me. I was however pleased that the vendors demonstrated innovation in a number of areas (like social media measurement) and that despite my attempts to develop extremely challenging criteria, the vendors continue to improve year over year.

One comment people have made to me is that they question the validity of comparing fee and free solutions in a single matrix due to the fundamental differences in their business model.  How would (or do) you respond to that challenge?

That’s preposterous! I respond by saying that it’s negligent not to compare free vs. fee based solutions. In today’s economic environment if you’re not watching expenses by understanding the cost to benefit ratio of your Web analytics solution, you are acting irresponsibly. Free tools have merit for many organizations as both primary and secondary tools, while fee based solutions are more appropriate for others based on their capabilities. Organizations must do their diligence to understand what they need in a Web analytics solution to decide what’s right for them, which is really the insight the Wave attempts to provide.

I asked Bill Gassman from Gartner a variation on this question recently, but do you now or see in the near future a situation where you as a Forrester analyst are advising your clients to actively consider these free solutions in addition to “traditional” web analytics solutions from Omniture, Coremetrics, and Unica?  As a follow-up, how do you see free tools impacting the market in the next 12 to 24 months?

I advocate that a single system for measurement is always the best way to go, yet recognize that this isn’t always feasible. Duality of Web analytics tools is a reality for myriad reasons. Thus, company’s need to manage their data dissemination practices to ensure comprehension and mitigate doubt. This is tricky, but certainly possible. I often help clients determine which solution is best suited to meet their needs and financial implications are always a part of that discussion.

With regard to how free tools will impact the market: we are just witnessing the beginning of the incoming tide on this one. By this I mean that “free” will continue to disrupt the market by placing pressure for improvement on all vendors. Just look at the recent Webtrends product upgrade announcement – the majority of press around it cited a “look out Google Analytics” slant. Why the comparison…they’re worried! Fee-based vendors have even more to fear now that Yahoo! Web Analytics opened up its partner program.

Another comment I hear about the Wave results, and forgive me this, is that they’re lame because they do nothing to differentiate the “market leaders” who appear as a tight cluster.  The evidence cited is that all four vendors issued press releases declaring their “market leadership” which appears technically correct based on the Wave but as the Highlander said, “There can be only one.”  First, how do you respond to this and second, who is the real market leader in web analytics?

Here’s the dirty little secret – the real market leader is the wildly talented Web analytics practitioner. It’s not the tools that differentiate it’s the craftsman. Any company that believes the Web analytic technology alone will make them incredibly successful is delusional or just plain out of touch. There is no get rich quick scheme here. Each of the leading vendors on the Wave offers a highly customized solution that can be tricked-out to meet nearly anyone’s individual needs. But this takes a great deal of work. For those organizations that are looking for the far-and-away winner in this technology category, guess what: the tools will only get you so far – you need talented people to really make it happen.

Rumors are that Omniture has a bunch of “800 lb gorillas” hanging in their offices right now.  Clearly they’re proud of their position, but last quarters results highlighted that there are clear risks to their business that are beginning to manifest.  What do you think are the greatest risks to Omniture’s business over the next 18 months?

Well, I don’t buy into rumors and sure don’t know where I left my crystal ball. But things are tough all over. As I stated earlier, free solutions are threatening all fee-based vendors and forcing them to work harder. I can tell you that measurement technologies are an imperative for executing on digital marketing endeavors. Solutions like Omniture’s, Webtrends’, Coremetrics’, Unica’s and everyone else’s will continue to play an important role in the evolution of organizations conducting business online. I believe that Web analytics is increasingly becoming an integrated service and expect to see things evolve to easier access to data through new and alternative means. The leading vendors, including Omniture, will play a role in this evolution.

What’s your taken on the current hype cycle around “open”?  Omniture bangs the Genesis drum, Coremetrics connects, and now WebTrends appears to have decided that “open” will be the foundation of their future success (or lack thereof) … but some people think that “open” is a check-box requirement, not a competitive differentiator.  What do you think?

Open is not a feature, it’s a philosophy. The ability to get data into and out of a Web analytics solution is the crux of the issue and leading vendors facilitate this through bi-directional API’s, other import and export functions and data dissemination capabilities. Webtrends is currently doing this as well as anyone, but “open” also means talking to your customers about development plans, listening to criticism and demonstrating a willingness to change. These qualities aren’t unique to Webtrends, they’re characteristics that all vendors should exhibit. Webtrends is just marketing around them and if that’s causing people to want open, then it appears to be working.

As a previous attendee to the X Change what do you like best about the conference and what would you like to see us change this year or next?

I appreciate the intimate conversational format of X Change. The huddles really facilitate deep thought, controversial leeway and provocative discussion. As someone who attends a number of conferences, it is refreshing to engage in dialogue with individuals who are passionate about what they do and to initiate a true collaborative thinking environment. As far as change goes, I really hope to be able to guide the huddles that I’m leading toward resolution. Within our industry, all too often we surface problems and issues without identifying solutions. I’ve taken your challenge to heart and hope to walk away with some tangible results from my huddles.

---

John will be joining Bill Gassman, Gary Angel, June Dershewitz, and over 100 expert users, consultants, and vendors at the 2009 X Change conference in San Francisco on September 9, 10, and 11. Registration is currently underway and we’d love to have you join us! For more information please visit:

http://www.xchangeconference.com

Bill Gassman from Gartner is one of those guys that just “gets” what we’re trying to do in the web and digital analytics industry. Perhaps because he’s been covering this space for nearly as long as I’ve been around, or perhaps because he has a deep business intelligence background and sees where all this is going. I dunno, but Bill gets it.

Recently Bill, who is incidentally coming to the 2009 X Change and leading a huddle on organizational issues and co-leading a huddle on technology with John Lovett from Forrester, published a short brief on Google Analytics that I thought really hit the nail on the head. Clear, honest, and fully taking the Enterprise into account, Bill’s report clarified a lot about how companies should be thinking about Google’s analytics solution.

Since I could not get permission to republish Bill’s report I did the next best thing — I came up with some questions and put them to the man himself. The following are my questions and Bill’s responses. Incidentally, if you want to follow-up on this interview Bill graciously said he would monitor the comments and respond there (so comment away!)

Or, you could just come to San Francisco on September 9, 10, and 11 and debate the goodness of Google Analytics with Bill in person.

Regarding your recent note on Google Analytics, can you characterize how the companies who are asking you about “free” analytics have changed in the last 12 months?  Is any one thing driving that change, do you think?

Since Google Analytics improved last October, most client inquires about Web analytics touch on Google Analytics.  That is why I published the note “Is Google Analytics Right for You?”.  (Gartner account required to access)  Marketing departments ask if it is all they need, purchasing agents wonder why they should spend money on commercial tools and corporate lawyers wonder about Google’s terms and conditions.   The economy and budget constraints trigger the questions, but the major driver to Google is its simplicity.  Many organizations do not have the processes in place to make use of the high-end products or have Web sites that do not need the sophistication they offer.  They perceive Google Analytics as good enough and “free” is a tempting offer.

If Google asked you which three things were most important to add to their functional set to be considered “Enterprise” what would those three things be?

Getting to functional parity with the commercial tools is not enough for Google Analytics to be considered enterprise class.  Google should charge for an enterprise class offering, because a lot is required as is the accountability that goes with the exchange of money.  They must also provide enterprise class support and address issues with the terms of service policy.  The key function missing is a visitor centric repository, so users can define complex multi-session segments and export visitor information to campaign and content management tools.  Google would also have to extend personalized service from customers with significant AdWords accounts to those willing to pay for it.  Finally, the terms of use policy has no service level guarantee and users must look for a FAQ to find assurances of privacy.

On the subject of “Enterprise-class” analytics … Google appears obsessed with this designation, regardless of their clear dominance from a deployment standpoint and the gains they’ve made within larger companies.  What do you think is behind their obsession?

Google appears to be fighting an asymmetric war with IBM, Microsoft and others, investing relatively little yet forcing competitors to take notice.  We see this especially for office applications and cloud computing.  Google is looking for products that give them credibility at the enterprise level, and Google Analytics is part of that story.  Other parts of the story include the recent news about the Chrome OS, Google Search Appliance, Google apps (premier edition), Geospatial Solutions, and Google App Engine for cloud computing.  While many large organizations are using some or all of these offerings, with few exceptions, only the Search Appliance has gained strategic status.  Google still brings in 97% of its revenue through advertising.  They might be showing obsession because of where they want to be, but then again, they could be throwing up a smoke screen to keep the competition too busy to attack Google on advertising.

What do you consider the single greatest risk to Google’s analytics business in the next 24 months?

There is no threat to Google’s analytics business, because Web analytics is not their business, yet.  Out of 72 million active Web servers (as reported by Netcraft), about 20,000 organizations pay for Web analytics.  Google gives away Google Analytics so that millions of Web site owners can see the impact of AdWords and buy more Google ads.  If there is a threat, it is Yahoo Web Analytics, who is using a similar tactic to go after Google’s advertising revenue.

Are you now, or do you see in the near future, a situation where as a Gartner analyst you are advising your clients to actively consider free solutions from Google and Yahoo alongside “traditional” web analytics solutions like Omniture, WebTrends, and Coremetrics?

Running two sets of tools on the same Web pages can be a recipe for trouble, because reported numbers will not match, reducing respect and therefore value for both tools.  There are situations however where two tools make sense.  It would be great if all organizations had the leadership, investment, skills and processes to use commercial tools to meet everyone’s needs, but for too many, it has not worked out that way.  When analytic resources are limited, it is pragmatic to focus commercial tools on the high-value parts of the site and let other site stakeholders use free tools. Analysis is a critical part of a customer centric Web strategy.  If some departments are happy with the free tools and a central group cannot support them, it is OK to let chaos reign until the business justification, investment and leadership are available to do things right.