The classic web analytics funnel analysis is simple indeed.
The key metric is the percentage of times the process is abandoned on each step. The implicit assumption is that steps with the highest abandonment rates are the biggest problem.
When you actually do this type of measurement, the most common finding is that the conversion funnel step abandonment rates look like a big U. The first step of a form process often has a high-abandonment rate, followed by a series of steps with small and relatively similar abandonment rates, concluding with a final confirmation step that also has a high-abandonment rate.
Different design philosophies and implementations will produce different curves, but this is the pattern I’d describe as “classic.” Given this common pattern (or the existence of any common pattern other than an equal rates), is it meaningful to suggest that the step with the highest exit rate is reflective of a problem? Not really. That would only be true if the natural forms abandonment model was that each step had an equal chance of abandonment.
Once you discard that assumption, it’s obvious that no particular level of step abandonment is positive evidence that the step is somehow broken or worse than any other step. Thinking the issue through, it should also be obvious that no particular level (or even change) in step abandonment is necessarily evidence of a problem.
This is a similar point to one I’ve made before many times about reporting. Just as no one Conversion Rate (or even an improvement in Conversion Rate) is positively good, a step abandonment rate must be viewed within a larger context.
Let me give you an example. Suppose you have form process with four steps and you produce a report like this:
Abandonment Rate
October
November
Step 1
42%
53%
Step 2
11%
14%
Step 3
9%
12%
Step 4
48%
51%
Has the form gotten worse? Maybe, but perhaps not. Indeed, with no physical change in the process the real-world odds are heavily against such a conclusion. Far more likely is that the shape and quality of traffic into the form has changed for the worse. If you just started a major PPC effort, it may be that you are driving far more, and somewhat less qualified, traffic than before. Form performance will reflect that.
This point is especially critical to understand if you change your conversion process AND change your site design at the same time. This combination may produce worse Step Conversion rates even with a Form that has been signficantly improved – a fact that is almost always missed or mis-interpreted.
I believe this illustrates two critical points. First, a step abandonment rate taken as a single fact means nothing. Like so many individual metrics, it is dangerous and misleading when used as a single point of decision-making. Second, measured form performance is very dependent on exogenous factors.
This should give pause to anyone who thinks that the basic process of measuring conversion processes is to find the steps with high abandonment and fix them!
I’m going to step back for a moment and layout some more general principles for thinking about measuring conversion processes.
It seems to me that in building up a system of measurement, we have to a bit of a theory about form-based processes on the web. Here are some basic rules (some purely theoretical some based on actual experience) I think might make up such a theory:
- All forms will experience some friction.
- Each field and each form will contribute at least a small amount of friction to the process.
- Some fields inherently involve significant friction. These include commitment steps, complex steps and steps requiring personal or unusual information.
- Some visitors do not intend to finish a process – and these visitors may abandon at different steps. So eliminating a Form with a 10% drop rate will not produce an equal effect on final conversion.
- The lower the pre-qualification of visitors into a Forms process, the higher the effect of friction.
- Abandonment on a Form is most common when the user has not changed the field focus.
I believe these principles are sound, though probably not even close to exhaustive. Still, they produce some fairly straightforward analytic consequences:
- Some abandonment will occur in any Form.
- Some spots always have higher abandonment – this doesn’t mean they are “worse.”
- Some fields will have a higher abandonment – this doesn’t mean they are “worse.”
- Elimination of a step or field should always have at least a tiny incremental positive effect.
- Behavioral Analysis can measure the friction involved in a step – but not necessarily provide direction about how to reduce that friction.
- Changing the shape of the traffic into the form will likely affect BOTH the rate of step abandonment and the ratio of step abandonment between steps.
It should be obvious from this that the basic measurement task in a conversion process is not to simply identify step completion rates.
From the above discussion, it should be clear that a high abandonment rate doesn’t imply failure.
On the other hand, it is a purely empirical point whether or not such steps are easier or more difficult to improve than lower friction steps. Based on our real-world experience, I’d argue that the weight of empirical evidence is, in fact, in the contrary direction. It’s often easier to improve conversion processes by focusing on relatively low-friction steps.
A similar lesson applies to field-based abandonment. While the field focus (the place where the cursor was when the user left the Form) on abandonment is sometimes significant, it is quite often simply the first (or last) field on the form. Neither is particularly meaningful. So while looking at field-based abandonment can occasionally be helpful, it is hardly the slam-dunk analysis that people unpracticed in web analytics often expect.
In this post, I focused mainly on the wrong way to approach conversion process analysis. In my next post, I’ll take up some of what I believe are the real analytic tasks around form measurement!
