Thanks to everyone who has been so engaged in the debate over Technorati’s utility as a data source for ranking blogs.  I guess I opened a can of worms with that post but the debate has been just great!  But on to bigger and better things …I’m just about to board my flight to Tokyo, Japan to give the keynote presentation at Digital Forest’s Marketing ROI Day conference on August 1st.  I’m very excited about this opportunity and to finally meeting my generous hosts at Digital Forest.  If you are reading this and live in-or-near Tokyo, please come to Marketing ROI Day and meet me in person!

Incidentally, I’ve finally updated my presentation schedule.  You can learn where I’ll be presenting pretty much through the end of the year at this URL:

http://www.webanalyticsdemystified.com/link_list.asp?l=Presentation

Finally, I think I mentioned that I’m writing for DM News now.  My second article, titled “Hiring Myths for Web Data Talent” is now available online (and theoretically in print as well!)  In this article I address four key issues that anyone looking to hire experienced talent — regardless of which analytics platform they’ll be using — needs to consider.  I encourage each of you to read the article, but here is the top-line summary of hiring myths:

1. That an analyst is always the most important “first hire”
2. That a mathematics background is a must
3. That web analysts salaries can be easily compared to common IT functions
4. That a good hire guarantees positive return on investment

I welcome your comments and feedback regarding my presentation calendar, the DM News article, and pretty much anything else you’re interested in chatting about.

A few days ago my friend Avinash Kaushik wrote and asked me for my Feedburner subscriber numbers as an input into his very popular ranking of web analytics blogs. It gave me an opportunity to do something I’d meant to do for awhile — request that Avinash drop my site from his ranking system (which he agreed to do, thanks Avinash!)

I asked Avinash to drop me from the rankings for two reasons:

1. I’m pretty well established as a web analytics blogger, having done this for a pretty long time. I started blogging about the topic while I was at JupiterResearch back in 2004 and have been writing basically the same blog at a variety of URLs continually since that time. I figure it’s far better for Avinash’s list to highlight some newer folks in the web analytics blogosphere — great writers like Ian Thomas, Gary Angel, Judah Phillips, and Aurelie Pols!
2. Technorati, which Avinash uses as the basis for his ranking system, is an extremely poor data source for ranking weblogs.

While it may be a unique source of this type of data, Technorati provides a lousy basis for accurately ranking blogs and appears to be very easily fooled by anyone actively working to increase their Technorati ranking.

Why would I say such a thing, you ask? An excellent question, but I have what I think is a pretty good answer (especially if you’ve ever had any concerns about the quality of data you use in your analysis …)

First, and maybe this is something I’m just being dumb about and is easily corrected, if you’ve had a blog for any amount of time and have moved URLs for any reason, Technorati seems incapable of re-grouping URLs for a single blog. Have a look at the following:

As you can see here, based on a search for “web analytics” grouped using the “Blogs” tab in Technorati, my blog shows up as two distinct entries from two slightly different URLs. Both have slightly different levels of authority. When you drill down into “authority” which seems to provide at least partial basis for the ranking system Technorati uses, you’ll see slightly different results for each of these URLs:

The first blog URL lists “437 blog reactions to Web Analytics Demystified”

The second blog URL lists “461 blog reactions to Web Analytics Demystified”

What’s worse is that the exact same blog and the exact same content appear further down the same page of results:

Here the blog URL is www.webanalyticsdemystified.com/weblog which was the original blog URL back when I was on the Blogger platform. Perhaps because this is the oldest URL Technorati has in the system, this URL has the greatest reaction:

Now, I wondered if perhaps each of these listings were de-duplicated and could perhaps be added up or something — no such luck it appears. There is a ton of duplication and thusly my blog is pretty much just broken up into three pieces which certainly must make it hard for anyone trying to assess the overall reaction to my writing over the past 3.5 years.

I asked around a bit to see if anyone knew why this happens, and Judah Phillips (who admits he’s been watching his Technorati ranking lately given his relative newness to the blogosphere) pointed me to this entry in the Technotati FAQ:

http://support.technorati.com/faq/topic/56

Just in case you don’t want to read the FAQ entry, I will summarize: You are more or less out of luck. According to the FAQ “we are unable transfer or combine links from different URLs at this time.” I suppose their answer makes sense, but it doesn’t make how Technorati treats blogs that have moved any more useful or appropriate.

Oh well.

The second problem I have with Technorati is that it is either not paying very close attention to where these “blog reactions” are coming from or the system is very easily gamed. Consider the blogs in the number 2, 4, and 5 slots when you search for “web analytics” blogs at Technorati:

There is my friend Avinash, Mr. Marshall Sponder from the Web Analytics Association and KnowMoreMedia, and the entire team at FutureNow, Inc. This is what you expect to see based on Avinash’s ranking system (although I think he might be excluding the guys from FutureNow, I’m not sure …) given that, according to Avinash:

“The evolution of the ranking system continues with a couple of tweaks to the ranking this time around. The primary determinant of the rank in the list below is still Technorati (click here).”

The problem arises when you start to examine the sites that make up the Authority calculation:

Here you can see that Avinash (who is widely loved, I love Avinash too!) has done a great job at generating reaction to his blog, getting 6,013 sites to link back to his content and having a Techorati “authority score” of 948. Very cool … that is until you start to examine the actual sites and blogs linking back to Avinash, at which point you notice something like this entry (#10 on the first page of results when I snapped this screenshot):

Hmmm, that is from Avinash’s own site. That’s odd, isn’t it, that Avinash’s own site would be included in his authority calculation? I thought so, so I quickly looked at the first ten pages of results:

What I found was that 44 percent of the top 50 sites listed as providing “blog reaction” to Occam’s Razor were Avinash’s own (albeit slightly different) URLs.

I’m not sure why that is, do you know?

I figured this might just be some strange anomaly so I took a look at the same thing for Marshall Sponder’s blog, WebMetricsGuru. Marshall didn’t seem to have the same problem, fortunately, but of the 52,152 reactions to Marshall’s blog contained in Technorati, it appears that the dramatic majority come from un-targeted links to his site from other KnowMoreMedia properties:

Similar to the problem with Avinash’s listing, 80 percent of the top 50 sites listed as providing blog reaction to WebMetricsGuru were from KnowMoreMedia. When I continued looking at the results, this percentage actually went up to 83 percent of the top 100 sites “reacting” to Marshall.

I might be thinking about this the wrong way, but that hardly seems like the kind of “reaction” most bloggers are looking for.

Well, at this point I had to look at the Eisenberg’s blog which is a little newer in the blogosphere. Here I saw the exact same problem I found in Avinash’s listing, 34% of the top 50 sites listed as providing blog reaction to GrokDotCom are from, yep, you guessed it, GrokDotCom.

I also noticed that in the GrokDotCom authority listings that some sites appeared again and again and again:

Again, I don’t know what’s going on here but as the basis of “blog popularity” this data seems pretty suspect to me.

Perhaps I’m naive, or perhaps I’m just plain confused here, but the Technorati ranking system doesn’t seem to provide very useful results based on the inconsistency I am describing. Maybe I just happened to stumble on three anomalies — other blogs listed in Avinash’s ranking don’t seem to have the same problems but some surely do. For what it’s worth, none of my three blogs (?!?) listed in Technorati appear to have the problem described above — maybe that’s what I’m doing wrong!

Now perhaps I am thinking about the authority calculation incorrectly. According to Technorati:

“Technorati Authority is the number of blogs linking to a website in the last six months. The higher the number, the more Technorati Authority the blog has.

It is important to note that we measure the number of blogs, rather than the number of links. So, if a blog links to your blog many times, it still only count as +1 toward your authority. Of course, new links mean the +1 will last another 180 days.”

This sounds good and kind of makes sense, except that you can see where KnowMoreMedia is kind of cheating Marshall by having all those completely irrelevant links back to his blog that artificially run up the number of “blog reactions” and likely his authority score.

Also, I kept finding examples of blogs that when I looked at the blogs linking to them, I kept finding the same problem described above — the blog being assessed linking back to itself. Here’s an example from one of the posts/pages on Avinash’s site that has an authority ranking of “12″:

You can’t see all of it but there are 13 reactions to Avinash’s post, and given Technorati’s ranking you would expect 12 different blogs, one of which would have two posts linking to the page, right? Wrong. What you get is five different blogs, two of which are Avinash’s own work (albeit in two different domains) and three of which (the “SEO, SEM, Social Media, web analytics” listings in the image above) appear to be the exact same content in different domains.

If you examine the URLs in the authority listings that come from Avinash’s site, you’ll see that they all have slightly different URLs. But clearly they are all from the exact same blog. If, based on the FAQ answer I gave above about why my blog is listed three times, this is how Technorati is calculating authority … essentially Technorati is saying that every distinct URL is a distinct blog.

Huh?

Seems like a pretty easy system to “game” to me, or one that is easily fooled and mostly useless. At this point I’m even more confused about Technorati’s ability to de-duplicate blogs as an input to their authority ranking.

Please don’t get me wrong, I think Avinash is brilliant for publishing a list of popular blogs (especially one that ranks his own site as #1, how amazing and magnificent is that!) I have learned to respect Marshall Sponder’s ability to write (and write, and write, and write) and I obviously get on well with Bryan and Jeffery Eisenberg (Bryan is one of Web Analytics Demystified, Inc.’s trusted advisors.) And I sincerely, sincerely hope that each of these fine gentlemen will see that this blog post is far from a criticism of their talents and passions.

But given that I’ve always worked with my clients to make sure they had the best data possible to serve as inputs for their analysis, something about the data reported by Technorati just doesn’t pass the “old smell test”.

Honestly, if anyone out there can help me understand what Technorati is doing and why a ranking system that is apparently so easily corrupted by self-reference and link farming is useful, I’m more than happy to hear from you! Feel free to email me directly or simply comment on this post. Until that time, I will view any ranking system based on Technorati data as quite suspect.

Perhaps you will as well …

I welcome your comments, criticisms, insights and feedback. If someone from Technorati wants to email or call me and explain what the heck I’m doing wrong and why everything I’ve written in this post is incorrect, I’ll gladly listen. And if the answer makes sense to me, I’ll even more gladly apologize for being so confused! If you think I’m carping, whining, or just being critical of Technorati’s data for no good reason, let me hear it! Frankly I sometimes worry that we don’t have enough engaged, thoughtful debate in the web analytics blogosphere …

Yesterday, while I was in Los Angeles at Jim Sterne’s “Guru’s of Marketing Optimization” event, Web Analytics Demystified released the second set of research based on our March 2007 web analytics survey.  This time we looked at the difference in usage patterns reported by respondents primarily using “free” tools (for example, Google Analytics or Analog) and those using “for-fee” solutions like SiteCatalyst, HBX, WebTrends, etc.

If you haven’t already, you can download the research from this link:

http://www.webanalyticsdemystified.com/research/

The three key insights in the data, in my opinion, were:

• Those deploying free web analytics solutions are more likely to treat web analytics as a casual endeavor, with 35 percent of survey respondents using free solutions reporting only an ad hoc use of their measurement tools, compared to less than 20 percent of those using licensed solutions.
• Companies using free tools are dramatically understaffed for web analytics, with 42 percent of respondents using free tools reporting having no dedicated resources, compared to only 18 percent of those using licensed solutions.
• Individuals using free tools have less experience with web analytics in general, with 64 percent of respondents using free tools reporting less than two years of experience, compared to 32 percent of those using licensed solutions.

While some may incorrectly interpret this report as some kind of damnation of free tools nothing could be further from the truth.  And while I comment in the report on page 3 that “there appears to be a very strong correlation between a lack of investment in web analytics technology and a sub-optimal use of this technology” anyone paying much attention should hardly be surprised by this statement.

It is my firm belief, and I emphasize this on page 4, that any company using any application regardless of price can be tremendously successful in their use of web analytics.  I say this because I know that success means different things to different companies.  I also say this because I firmly believe that being successful with web analytics has very little to do with technology, at least for the majority of companies doing web analytics today.

This was my position in 2004 when I published Web Analytics: Staffing, Spending, and Vendor Selection while at JupiterResearch–that people are as-or-more important to your ability to “do” web analytics than the technology you select.  My position today is slightly revised, given my views on the importance of process to web analytics, but until you’re doing moderately complicated things with your web analytics package, based on clearly defined business objectives, the technology you deploy is not the limiting factor: you are.

The recommendations I make in the report are pretty straightforward:

1. Companies who have decided to standardize on free solutions need to work overtime to be successful in their endeavors.  Ian Thomas over at Lies Damned Lies thinks I have this backwards, commenting that companies are not deciding necessarily to standardize on free solutions as much as they lack the ability or desire to invest in web analytics and so pick a free solution by default.  Fair enough, but given the industrial-grade love fest surrounding Google Analytics, one that appears to be about to be duplicated with the release of Thomas’s “Gatineau” product later this summer, the guidance stands: If you’ve decided to use GA (or Analog, or Gatineau) in lieu of a paid solution, don’t fool yourself into thinking that your choice in any way, shape, or form makes web analytics any easier.
2. Companies who have decided to standardize on free solutions must spend the money they’ve saved on technology to hire smart people.  On this point I must agree with the spirit of guidance given by the wonderful and well-loved Mr. Avinash Kaushik in his $90/$10 rule for web analytics spending.  Again, assuming that you’re not just “messing about” with web analytics but have decided to standardize on a free solution and intend to use that solution to create valuable business insights and to optimize the relationship between your site, your visitors, and your marketing endeavors, take Avinash’s advice and hire (or allocate) someone really smart to learn the technology and what it can tell you about your business.
3. Companies serious about improving their web site but unable to commit the necessary resources should consider licensed web analytics solutions.  I was not surprised that this statement more than any in the report got people’s hackles up but in most cases the response appears to me more emotional than logical.  All I’m saying is “if you aren’t going to help yourself, select a technology vendor that is motivated to help you!”  The for-fee vendors have near universally spent a great deal of time and money on their account management and professional services teams, and these people are ostensibly there to help their customers achieve “magnificent web analytics success” when they’re not able to commit to web analytics and hire dedicated resources.

There is a ton of great information in the report and I would very much encourage everyone interested in the great subject of web analytics to spend the time to read these 19 pages.  You can download the complete report here:

http://www.webanalyticsdemystified.com/research/

As is always the case, I welcome your comments, your feedback, and your honest opinion about my research.  Do you think I’m picking on free solutions?  Or perhaps I’m picking on for-fee solutions?  I’d love to hear what you’re thinking …

Jeremiah Owyang from PodTech has been bugging me to start a web analytics group at Facebook.  I initially resisted but upon kicking the tires at Facebook a bit it seemed like a good idea.  If you’re already in the Web 2.0 Measurement Working Group you should have seen an email from me about shutting it down at Google Groups.  Either way, the group is now totally open to everyone and can be joined here:

http://www.facebook.com/group.php?gid=2668335473

My hope is that by putting the Web 2.0 Measurement Group on the premier “Web 2.0″ platform that it will drive interest and promote conversation.  If nothing else it is a reflection that sometimes you can bring audience to you and sometimes you have to go where the audience is.

I hope you’ll join me at Facebook. 

Five little words: Rand don’t pull no punches.

Don’t believe me? Check out some of the commentary Rand offers up about his former employer (and mine, Visual Sciences/WebSideStory) in an interview just published with Eric Enge of Stone Temple Consulting. I only wish Eric would have conducted the interview in the last few days following Visual Sciences announcement that they are going to miss Q2 earnings by a few million dollars (OUCH!) and they’ve retained Goldman Sachs to help them shop the company, just to see what Rand would say.

Rand and Eric even discussed me of all people, with Rand offering up:

“Eric Peterson and I have been in two companies together. WebTrends, and WebSideStory, and we certainly worked together very closely when he was an analyst at Jupiter. We know each other pretty well. Eric and I don’t always agree on everything. I come from a pure solutions point of view, because my DNA is in marketing. I know what I want to achieve with these applicaitons, and Eric has an approach to these things more from a technology point of view. But, more and more, his point of view and mine are becoming very similar. I respect him, because he will tell you what he thinks. We’ve had our differences, but he and I are good friends, and I think it’s good to have differences. It makes the world go round. I think he is going to do really well as an independent by the way.”

While our time together at WebTrends was non-overlapping, and I would more or less describe our close work together while I was at Jupiter as, um, confrontational, I more or less agree with everything else he says in this paragraph (not the entire interview!) And yeah, we drink beer together and have funny conversations about industry folks and he has a unique sense of perspective that most people don’t have, so yeah he and I are friends.

Oh, and I appreciate his vote of confidence regarding the formation of Web Analytics Demystified.

Wait, what? You don’t know who Rand Schulman is?! You’re kidding me! Rand is an icon in the web analytics industry and, if I may say so, has one butt-kicking web site for his private venture, Rand Schulman Partners.

Anyway, thanks Rand for the nice comments and I certainly wish you and Unica all the best out there. Judah loves your stuff, that’s for sure.