Matt Jacobs is back from vacation and he’s back on the money with his post this morning titled Web Analytics Data Quality: “Truly Appalling”. In his post, Matt summarizes complaints about web analytics data voiced at DM Days in New York (a summary of the panel can be read here)

Based on his post, it sounds like data quality and consistency is a topic that Matt has dealt with a bit with his clients. To this end, he offers some sage advice. My favorite is this:

“Sadly, I’ve seen my fair share of vendor selections that were based on one vendor having a more desirable feature set than another. Do not fall victim to that mistake, no matter how sexy and cool the user interface is. After all, what good is that feature set if the data itself is not as reliable as it should be? If you consider the online channel critical to your business, you don’t have room to compromise.”

Now, read between the lines here and recognize that I work at Visual Sciences, a solution that I strongly believe provides both an incredibly cool user interface and the industry’s leading data collection and data management strategy.

My bias aside, Matt gives great advice! Don’t just look at the interface, ask the hard questions: Ask about cookies, ask about how visitors and visits are defined, ask about data collection and storage options, ask about data verification strategies … take the time to research the technology, not just the user interface.

Doing your homework isn’t going to make data consistency issues between vendors and technologies disappear, but it will help you better understand why these differences exist. Sure, it’s reasonable to assume that “unique visitor” would have a single definition across vendors and technologies, but any of you who have worked with different solutions for any amount of time realize that a single definition simply does not exist.

Regardless of how you feel about data quality on the Internet, you have some obligation to your organization to understand what each of the dimensions, metrics and filters you’re using are actually telling you. I think you’ll find that when you have this understanding, the need to tear apart each report in an attempt to reconcile numbers goes away. Then, hopefully regardless of inconsistency observed in absolute numbers, you can begin to use the data to drive action.