Topics from X Change
I’m going to weave in some posts on learnings and discussion resulting from X Change while I finish my SEM Analytics series, and I thought I’d start with one that came out of my SEM Analytics Huddle and so fits right in!
We spent a good chunk of time in the SEM Analytics Huddle talking through some basic issues in getting setup for measurement and structuring PPC campaigns to manage and monitor them effectively.
One thing that struck me in the discussion was that most of the Agency experts agreed that when they took over a program they usually found themselves rebuilding it from scratch. There’s an element of ethnocentrism here, but I think it’s more than that. PPC programs tend to start fairly clean and become increasingly messy as they are managed over time. So unless a PPC firm is really on top of things, a program that’s been running for awhile is often a mess.
In an earlier post, I mentioned that one pretty good way to see if your Agency is managing your campaign well is to review your Ad Groups and look to see if they make sense. If search terms that are fundamentally different in business concept are grouped together, then your buyer isn’t doing a very good job.
But as we talked through this issue at X Change, I was struck by the fact that the process as described was almost 100% subjective. Buyers basically seem to rely on their personal understanding of the business and the search terms to make these judgments. That’s fine as far it goes, but there are some simple analytic techniques that can help test these judgments – and I get the feeling they are rarely used.
Here’s a simple real-world example. Should “stock quote” and “real time stock quote” be grouped together? If you know the trading industry, you’d probably say no. The addition of real time to the phrase is a significant addition – one that constitutes a demand for particular type of service and that is fundamentally different from the search for a quote on a specific stock. But while subjectively I’d make this distinction, I might not be right. And there are other cases where I’d have no particularly strong judgment. Take “stock ticker,” for example. Should “stock ticker” be treated the same as “stock quote” or “real time stock quote” or is it a different beast entirely? How about “free stock quote” being grouped with “stock quote?”
These simple examples are repeated hundreds of times in most programs and each represents a potentially significant empirical question – one that can’t necessarily be resolved by any amount of learned discussion.
In many cases, the correct answer may depend not on abstract considerations of how the words are used in the language but on the real-world products, services and web pages a particular business deploys. “Stock Ticker” might best map to “stock quote” for one company but “real-time stock quote” in another. The PPC buyer doesn’t work in a world where every distinct concept is distinct to the business. What’s more, the buyer frequently has to group words together to achieve any measurable statistical significance in the tail and to provide any level of management. So there is always a cost to splitting words out – and that cost needs to be constantly weighed against the potential benefits.
These practical considerations mean that you can’t realistically just expect to treat every word as a distinct entity. But if you have to group search terms, how do you know your groups are any good?
There is actually a simple way to test your groups. Keep in mind, that what you are striving for in a group of words is homogeneity. The more similar the words, the better the grouping, and the more disparate the words, the more likely the grouping is to show significant differences in internal performance.
So to check the quality of your groupings, you can review the search term performance within the group to look for significant variations. There are two places to do this. First, you can check within your bid management system for variations in click through. These variations are often indicative, but they can also be misleading because the results are shaped by a surrounding marketplace that isn’t necessarily constant.
On the web analytics side, however, you can get a purer read. If visits sourced from particular search terms within a group behave significantly differently, then the group isn’t homogenous. You can track at several levels here – but I don’t generally recommend focusing on distant measures like conversion. Instead, I expect problems in homogeneity to show up almost immediately on the web site.
So what I’d recommend looking at are two simple measures by keyword within a group. The first is % single access pages (SAP). If a search term within a group has significantly different SAP performance, then it probably doesn’t belong in the group.
Secondly, I’d look at 1st click to see which link path is chosen. If the visitors for a specific term tend to jump in a different direction than those from the rest of the group, then the search term probably doesn’t belong.
By measuring the variation in these two measures within an Ad Group, you can quickly test your hypotheses about Search Term homogeneity within an Ad Group. Having this empirical data is especially important for Agency buyers who are taking over new accounts and may not have learned all the subtleties of the business yet.
In our experience, these two simple measures will catch the vast majority of Ad Group mismatches.
If you’ve setup your measurement to include Ad Groups, you can generally automate a significant amount of this in reporting. It should be possible to capture the core statistics (Single Access Pages and Top Next Pages/Links) for both Group and individual terms. With a snapshot like this, you should be able to check your assumptions quickly and adjust your Ad Groups appropriately.
Since truly homogenous Ad Groups are absolutely essential to effective PPC Management and optimization, this is much bigger deal than most people realize.
There are some arguments that web analytics just won’t help solve. There are others where appropriate web measurement can effectively end the discussion. You probably wouldn’t have thought that deciding on the subjective placement of search terms in Ad Groups was one of the latter – but it’s just one of the many surprising ways that web measurement can support better Search Engine Marketing.
