Is there really a problem with the way web analytics is performed? I think there probably is, but I don’t pretend to know the answer for sure – or to believe that it’s a question that can be easily answered.
(You may want to visit http://www.semphonic.com/resources/whitepapers.asp and download the White Paper on Functionalism as a detailed technical companion piece to this series).
I’ll start by saying that I have a deep and abiding distrust of the punditocracy in any industry (yes, I know, fear the enemy – he is yourself!). I sometimes get asked by people who I think the experts in web analytics are. I generally dodge this sort of question. But what I’m thinking is, that it isn’t anyone you read or have ever heard of. Because the real experts are all too busy doing the work!
It seems to me you can divide pundits up into three basic categories. By far the largest are the group who never knew anything and never will but have a gift for “punditry.” Then there is a substantial group who once were quite expert in their field, but as they achieved a name spent less and less time practicing and more time preaching. This group ranges from a little out of touch to wildly out touch – depending largely on how long they’ve been well known. Then there is a tiny percentage who somehow manage to be both expert and preacher. Do I aspire to be in this last group? Of course, but there are already people in SEMphonic who know more than I do about some aspects of web analytics. That’s just life. So I’d guess I’d settle for being only a little out-of-touch.
Which is a long-winded way of saying that just because lots of people who “talk” think that web analytics isn’t being done well doesn’t mean it’s true. I know that when I’ve read those things in the past, my reaction was generally – “Hell, they should hire us.” And they should have. Nor is SEMphonic world-girdling enough so that I can pretend that my view of what we see with our clients when we start engagements is necessarily representative of the whole world. Our clients may start out knowing more or less than what the average out there is – it’s hard for me to tell.
So I’m just going to say what my sense is, without making any stronger claim. It seems to me that most organizations I talk to and meet with aren’t really using web analytics. They have it now. They have good software tools. They usually have web analysts. But those web analysts seem to spend their life running ad hoc reports – and those ad hoc reports seem to do nothing more than scratch the surface of business or marketing problems. The reports are driven by the marketing managers, not the analysts (which isn’t necessarily all bad) but are usually of the very un-interesting sort like “how many visitors do X?”
In the background, the web analysts get a chance, every now and then, to build a bigger analysis. And when they do, it’s usually to the effect that these six types of visitors exist on the site and conversion for Group A (Top Prospects) could be improved and satisfaction for Group B (customer support) has slipped a little. This sets off a flurry of activity designed (but in no way driven by the analysis) to solve the problem. Generally, it doesn’t, and the process is repeated sometime later.
You’ll have to decide for yourself if you think I’ve described your situation (or anyones), but if you accept that there is quite a bit of this kind of thing, the next question is why. I’ve heard any number of theories, but here are the ones that seem to me to make enough sense to consider: 1) that’s the way every field is when you get right down to it; 2) the web analytics tools aren’t sophisticated enough to do real analysis; 3) organizations lack the essential data to close-the-loop on their analysis and make it meaningful; and 4) web analytics lacks a good methodology that most practitioners can understand and use.
I fall into Camp Four, obviously, and that’s what Functionalism is about. But what about 1-3?
Let’s start with the “that’s just the way it is everywhere” notion. Everyone knows the old joke about the making of hot dogs, and it’s kind of true about large companies as well. When you see what goes on at the best of them, you wonder how they ever make money (and how it is that you haven’t). That is life – there is a lot of friction in even very well run companies – and things can look like bloody hell on the inside while outside the cash registers are ringing feverishly. But I come from the world of credit-card database marketing – and I knew a lot of analysts in that world. And my sense was that they had a much better idea of what they were about and had a much bigger influence on the actual business than is true for most web analysis. So while I think there’s truth in the “it’s hell everywhere” theory, I don’t think it captures the whole story. Things can be better – even if they are never going to look really great.
The idea behind #2 also has some truth in it. But I think the truth is more historical. Five years ago, web analytics tools were useless. Three years ago, they were bad. Neither claim is true today. The pace of tool improvement in the past few years has been pretty impressive. So while tools used to be a serious problem, I simply don’t accept the notion that you can’t do worthwhile analysis with the enterprise tools in common use today. Are they perfect? Far from it – but they are more than acceptable and getting better fast. Much faster, in my opinion, than the analysis coming out of them. Hence my feeling that there’s a problem. Four years ago, you couldn’t tell there wasn’t a problem with methods – because you couldn’t do anything! Now, you can.
Idea #3 strikes a chord as well. I have seen many situations – especially in multi-channel businesses, where the most important data are never either collected or brought together in one place. This IS frustrating. And, it makes a great excuse for people within an organization to do nothing. Indeed, the very existence of the excuse may be the biggest problem. Because you can almost always do useful analysis without closing the loop. Functionalism is certainly designed to be one solution for just this problem. But there are many others – and many kinds of techniques that with only a little creativity can drive useful analysis. So while I accept that this is a real problem, I don’t think it really explains what’s actually happening when people do web analytics. It’s actually a whole different hurdle that needs to be addressed in its own right.
Which brings me back to #4 – that businesses have not developed a reasonable process for doing web analytics that makes it fairly likely that the output will be both useful and used. Functionalism, of course, is not the only methodological solution to this problem that anyone has proposed. And so what I’m thinking is that the next blog will actually cover some thoughts on what I take to be the most interesting alternative to Functionalism – a set of methods that focus on visitor segmentation and that I’m going to lump together and discuss what I take to be both good and problematic in their general approach.
Tag:
Add to 
Bookmark Murdok:
Gary Angel is the author of the “SEMAngel blog – Web Analytics and Search Engine Marketing practices and perspectives from a 10-year experienced guru.
