In the last few months there has been a tremendous surge in interest in my framework for measuring engagement online. Lately, some of the largest and well-known companies in the world have approached me about working with them to bridge the gap between the metrics they have today and something similar to the composite metric I first described back in December 2006.

While I am tremendously flattered that I have somehow become the focal point for this conversation, I have been thinking lately about how the framework has been developed and how it might end up being used by the measurement industry in general. And while early tests using the framework I’ve described are very encouraging, the calculation in it’s current state was meant to move the discussion along and get more people to “think different” about how engagement could be calculated online.

Given that interest in the framework has clearly increased, one primary concern comes up again and again: the need to apply mathematical rigor to the framework and calculation so that A) the result is repeatable, reliable, and trustworthy and B) when naysayers inevitably emerge to criticize this small side project of mine, that I have a suitable response to their criticism, regardless of where and why it comes.

I believe that the need for “A” is obvious. The need to address “B” is perhaps less obvious, but I believe that I owe it to those of you who are investing your time, energy, and money into this framework. Especially as the stakes seem to increase exponentially with every presentation, every conversation, and every high-visibility blog post on the subject, I believe now is the time to approach the engagement framework not just as a hobby but as a serious project with committed resources.

To this end, I am extraordinarily happy to say that the single smartest person I know, Joseph Carrabis the Founder and Chief Research Officer of NextStage Evolution and NextStage Global, has offered to bring mathematical rigor and analytical precision to what I am officially dubbing “The Engagement Project.” Those of you not familiar with Joseph and his work are advised to A) meet him in person at the upcoming Emetrics Summit in San Francisco or B) read some of his recent work at iMedia Connection.

Joseph will be working to make the formula universally applicable and universally defensible. Suffice to say I can think of nobody better to bring mathematical and scientific rigor to the framework I have been evolving over the past year. Watch this blog and Joseph’s blog at BizMediaScience over the next week or so for a more complete analysis of the framework in it’s current state, something we’ve agreed is the first step towards creating a true function capable practically describing the degree and depth of engagement a visitor is displaying towards a web site over time.

At the end of the day, without regard to my framework, Joseph’s analysis, or any person or group’s particular position on the use of the word “engagement”, my goal is to solve one problem and one problem only:

If you’re interested in working with Joseph and me on The Engagement Project please feel free to contact me directly.

I am incredibly honored to be able to announce that Web Analytics Demystified has partnered with the fine folks at Semphonic to present X Change 2008 in San Francisco at the Ritz Carlton, August 18 and 19, 2008.

In my decade in the web analytics industry I have attended and presented at hundreds of events around the world. While each have their own unique value proposition and are truly great in their own way, X Change is the only conference I left both completely energized about our industry and actively scheming about how I could become more involved in presenting the event. I said as much after last year’s event.

You can read more in the press release but here’s what you’re not going to read in the release:

• For me, X Change was like no conference I’ve ever attended as a practitioner, consultant, analyst, or business owner. At the end of the day I didn’t want the conversation to end!
• The key is the “huddles”: far more structured than “birds of a feather” sessions or round tables, yet informal enough to encourage everyone to participate, and loaded with analytics experience and talent.
• The small size of the conference is intentional, and Gary, Joel, and I are working to bring some of biggest brains and folks working on the most exciting projects out there to participate. The net result is an amazing ratio of experts per attendee.
• No booths, no vendor pitches, no literature you don’t want under your door. At least last year, the vendor representatives were senior people working at the top of their game (folks like Matt Belkin, Olivier Silvestre, Aaron Gray, Rand Schulman, etc.)
• More web analytics bloggers per capita than any conference ever thrown. (Okay, maybe that isn’t really a selling point but it led to a lot of really good PR …)

Again, I am incredibly pleased with Gary and Joel’s decision to allow Web Analytics Demystified a greater level of participation in the event this year. If you’re a long-time reader of my blog and would like more information about X Change, please don’t hesitate to contact me directly.

I hope to see you in San Francisco in August!

What does the future hold for web analytics, indeed? During my tenure at JupiterResearch I was more-or-less paid to predict the future, but at best I was right maybe 50% of the time. I think I predicted (correctly) that Google would give Urchin away, but I probably also predicted that Microsoft would acquire WebTrends (incorrectly) as a result. Such is life.

Fortunately I am smart enough to surround myself with really smart people, which is what I have done at the newest Web Analytics Demystified weblog: The Future of Web Analytics, Demystified.

What Joseph Carrabis (my partner) and I are doing at “The Future of” blog is creating an opportunity for some of the brightest voices in our community to wax philosophical about where we’re all going and what things will look like when we get there. In the last two months we’ve had excellent conversations started by the likes of Joseph Carrabis (NextStageEvolution), Rene Dechamps Otamendi (OX2), and most recently Mr. Ian Thomas of Microsoft fame.

The unique thing about this web analytics blog is that posts and comments are basically peer; we’re looking for long, well thought out comments that add something to the conversation. And as odd as it sounds we’re not approving navel gazing, fawning, and trackback/ping so that we can keep the conversation moving!

If you’re a long-time reader of my personal weblog I would strongly encourage you to subscribe to The Future of Web Analytics, Demystified. More importantly, if you’ve got a big brain and want to help us work collectively in an effort to figure the future out before we get there, we welcome your comments. Alternatively, if you have a prediction, see a problem, or want help resolving a problem that you struggle with, I’d love to hear from you about being an author in the Future Collective.

(Last week I published PART I of this post which you should read first if you haven’t already done so.)

STEP FOUR: DETERMINE YOUR KEY PERFORMANCE INDICATORS AND CRITICAL REPORTS

You’re probably thinking “shouldn’t we have done this after we defined our business objectives and activities?” Conventional wisdom would probably say you should, but in my experience if you don’t have a clear process for leveraging those key performance indicators (KPIs) and critical reports, you may end up with one of three things:

1. A huge report of 40 KPIs distributed across the organization that few people are likely to read and even fewer likely to act upon
2. No KPIs distributed at all, and the expectation that everyone will simply “log in” and get the information on their own
3. Well-defined and clearly articulated KPIs distributed hierarchically throughout the organization (because hey maybe you read a great book on the subject at some point)

The problem is that only the third possibility will deeply benefit your organization. I know that some people talk about hundreds of internal users who really get web analytics and all make superb decisions with the data, but this is very much the exception, not the rule. Remember, in our Web Analytics Demystified Spring Survey 69 percent of respondents said that they did not believe the majority of people using web analytics data in their organization actually understood that data.

It is far better for your analytics hub, as mandated by their executive sponsor in agreement with his or her peers throughout the organization, work directly with the individual spokes to ensure that appropriate KPIs are defined and the basis for those measures is clear. The hub then follows-up with appropriate explanation about the measures, including training on the reports and data that forms the basis of the indicators.

Your critical reports are directly tied to your key performance indicators (which remember are tied directly to your business objectives.) If you belong to the marketing organization than your KPIs will be measures like “Campaign Response Rate”, “Campaign Conversion Rate” and “Campaign Cost per Click”. Obviously as these KPIs change, appropriate tactical resources in the marketing spoke will review campaign response, conversion, and cost reports in your analytics application.

Your KPIs and critical reports will differ dramatically depending on what department you work for and where in that department you work — remember that the best practice for key performance indicator distribution is to deliver the specifically and hierarchically. Most attempts that I have seen to send “everything to everybody” have failed (often miserably).

STEP 5: DETERMINE HOW YOU’LL DELIVER ANALYSIS

Once you know what your KPIs and critical reports will look like, the next step is to determine how you’ll produce and deliver analysis. Let’s assume for a moment that you’ve got a hub-and-spoke model in place and the hub is receiving regular requests for more information, insights, and recommendations. The question then becomes “how will you deliver those insights and recommendations?”

As I said last week, there is no one “right” way to communicate about web analytics data but there are many, many wrong ways. The central challenge when delivering analysis stems from the fact that so few people really understand what web analytics terms mean, what the limitations of the technology are, and what is possible and impossible to report on. But it’s not like you can just give up and ignore the confusion, so what’s a great analyst to do?

The answer is “work harder, and think outside the box” (to use an overused term). While reports and raw data are best delivered using the Bottom Line Up Front (BLUF) method, analysis really needs to be more engaging. Remember: when you deliver analysis, what you really need to do is to convince the listeners that they need to take some action. To do this you absolutely have to be engaging.

Things that have worked for clients of mine in the past include:

• Well-delivered presentations, given IN PERSON, not just sent via email in hopes that people will review and understand
• Well-written documents, followed by a meeting to make sure that everyone READ the document and is on the same page
• Short summary documents, written up like a newsletter or newspaper article, designed to get people to attend a meeting or presentation

Since we’re in a Web 2.0 world, and since many of you are increasingly comfortable using new technology, a few other things you may want to consider include:

• An internal analysis Wiki that people can subscribe to and participate in. The Wiki is a good idea because it allows you to capture the conversation in a searchable format
• A regular analysis podcast, providing an update on past analysis and summarizing the data currently being reviewed
• A analysis video or vidcast, created with tools like TechSmith Camasis that allow you to easily blend images, live screen capture (useful when showing people live data in your analytics application), and annotation

The advantage the final two ideas confer is their ability to be downloaded to an MP3 player like the iPod or iPhone. If you have busy executives, you might be better able to reach them if you give them something to watch on the airplane or listen to on the drive home.

Keep in mind that none of these “Web 2.0″ strategies should replace well-written, well-presented analysis, delivered in person whenever possible and making specific recommendations for changes (including a testing plan when possible!)

STEP 6: PUT IT ALL TOGETHER!

Assuming you’ve completed the previous five steps, you now have a functional web analytics organization, one capable of delivering relevant reports and producing actionable analysis. Now the challenge is to stop spending all of your time generating reports and start delivering analysis!

Unfortunately, for many organizations this is really, really difficult. Even when there are dedicated resources — people specifically hired to do web “analytics” (not web “reporting”) — far too many bright folks end us spending all of their time churning out reports. Even worse, these reports often go unread, unused, and unnoticed despite the real and opportunity costs associated with generating them.

To be really, really successful with web analytics you have to train the organization to stop looking for reports and start asking for analysis, insights, and recommendations. While every situation is different, ask yourself how closely your organization follows these steps:

1. Automated KPI reports arrive, highlighting a potential problem associated with a core business objective
2. Line of business analytics resources consult critical reports directly looking for a reasonable explanation
3. Failing a reasonable explanation, business resources request analysis resources from the analytics hub
4. Analytics hub double-checks LOB’s cursory analysis, confirming the need for deeper exploration
5. Analytics hub prioritizes analysis with the business based on pre-agreed criteria
6. Analysis is delivered back to the business along with recommendations and a testing plan
7. Recommendations are reviewed by the business, test plan is agreed upon
8. Tests are run, results are socialized as follow-up to the original analysis
9. Incremental value of change is recorded to help calculate web analytics return on investment

Individual departments are still getting their reports, but they’re generating them by themselves. Senior managers have an appropriate view into the metrics, and their own resources to evaluate observed changes. Those resources have a way to get help when help is needed. Help (the hub) isn’t bogged down generating ad hoc reports all the time and is able to focus on high-value priorities. People produce analysis and make recommendations. Recommendations are tested. Optimization happens.

Kinda brings a tear to your eye, doesn’t it?

I know there are a hundred other things that come up in the line of business for any of you who are working practitioners, but having a clear communication strategy is the first step towards whittling that list down to something reasonable and, more importantly, valuable to your organization. Defining your business objectives, clarifying ownership and organization structures, establishing KPIs and critical reports, and knowing what your analysis output will actually look like is fundamental.

Defining your web analytics communication strategy will let the data work for you, not make you work for the data. It will help you move from making purely tactical decisions and start using web analytics strategically as part of your entire business. Over time you’ll find that a clear strategy, no surprise, helps the entire organization better understand web analytics in general and the value your investment can provide. And perhaps most importantly, a clear strategy will cut down on the volume of under-used, unused, and ignored reports traveling across your network.

If you’re interested in defining a web analytics communication strategy in your organization, I’d love to talk to you. If you don’t need help, I’m still happy to provide encouragement. If I can help you, great. If I can’t help you, I bet I know somebody who can!

Ha, made you look. Actually what is going on is some Web Analytics Wednesday antics in San Francisco where last week Web Analytics Demystified, Inc. sponsored an event hosted by the great June Dershewitz (read June’s blog!) You can see some of the other photos June took at Facebook

Speaking of Web Analytics Wednesday, we’ve made some pretty cool additions to the event system recently, including:

• Better visibility for sponsors, which seems to have increased the number of event sponsors and thusly the volume of free food and drink being provided
• Sponsorship from Web Analytics Demystified, Inc., to help smaller events in cities around the world get started. If you’re thinking about having an event, we’d love to hear from you!
• The ability to share your experience at Web Analytics Demystified, which will inevitably help market these events to other practitioners around the globe
• A snazzy interactive map done in Flash (alternates with other Flash content) to highlight how much of the world is participating in these events

I just participated in an EXCELLENT Web Analytics Wednesday event in New York hosted by Joel Collymore and Derek Monteverdi and sponsored by OpinionLab. Web Analytics Book has a nice write-up of the event and it was nice to have met Sebastian and the nearly 50 other folks who showed up.

Anyway, thanks to June for the fun photo and to all of you who are out there hosting, sponsoring, and most importantly participating in Web Analytics Wednesday events. If you have any questions about how you can host or sponsor an event, please drop me a line anytime.