It’s difficult to find someone who doesn’t believe it’s beneficial to make a more usable design. However, in today’s design environment, it’s often difficult to justify the expense of usability work against other business priorities.
Poor design creates friction. Friction can manifest itself as a shopper abandoning a sale. Alternatively, employees may spend more time than necessary obtaining a critical piece of information. The less the design jives with what the user is trying to accomplish, the more friction increases.
Whenever friction occurs, someone becomes frustrated. The user may feel the frustration because they can’t complete their task. Employers may become frustrated because their charges are taking too long for critical functions. A poor design results in high friction. High friction results in high frustration.
Measuring Frustration in Dollars
While it’s hard to measure whether a design is poor or not, we can easily measure when frustration occurs. With a little digging, we can usually associate a dollar cost to the frustration, thereby giving us a way to estimate, in financial terms, the cost of the poor design.
Frustration usually shows up in one of four ways:
For example, a problem that causes users to call the organization’s toll-free number increases the number of required call center operators. Similarly, an overly complex intranet design can make employees waste work time completing what should be simple information gathering tasks.
Each organization “feels” frustration with a monetary impact. We can measure the cost of handling a support call by dividing the support budget for a year by the quantity of calls they handle.
When we estimate how many calls deal with a particular problem, we can use the cost-per-average-call to estimate the cost of handling the problem through support. Since we can easily tell if an alternative design produces less frustration, we can then estimate the cost difference, giving us a big piece of the return-on-investment puzzle.
Case Study: Amtrak.com’s Cost of Frustration
As an example, let’s look at Amtrak.com, the official web site for America ‘s passenger railroad system. The folks at Amtrak have designed their site to allow customers to make reservations online.
Imagine we’ve observed during usability testing that Amtrak.com makes it very difficult to complete a registration. Only one out of every four attempts to book a registration online actually succeeds.
A quick analysis of the site’s logs shows that the average reservation is for $220. It also shows that there are currently 10,000 reservations successfully completed every month, producing a monthly revenue stream of $2,200,000.
Our inspection of the site’s logs shows the same patterns we see in the labs: only 25% of the people who start reservations actually complete them. That means that 30,000 reservations a month aren’t completed. Using our average reservation, that roughly puts the monthly failed registrations at $6,600,000 or $79,200,000 a year.
That’s a lot of money for Amtrak to recapture. However, our studies show that many of those folks won’t actually register, even if the site was much more usable. The current design forces a potential traveler to start the registration process just to see what a fare costs or when trains run between two cities. Many of these travelers will go for cheaper or more convenient travel and will never register.
Therefore, we should estimate our frustration cost carefully, removing these “no-go” users from our estimates. However, we may find it difficult to predict the percentage of visitors accurately without knowing their purchase intentions. In this case, we’ll conservatively estimate that only 20% of people who are not currently registering today would do so with an easier-to-use interface. (By estimating conservatively, we make it easier for others to put faith in our calculations while also allowing for the happy surprise of exceeding our goals.)
20% of our 30,000 uncompleted registrations a month is 6,000 people who we think will register with an improved interface. That means that a well-designed reservation system could increase revenues by $1,320,000 a month or $15,840,000 a year.
That means that the Cost of Frustration for Amtrak.com is almost $16 million annually. That ought to get someone’s attention.
Practical Calculations
To calculate Amtrak’s cost of frustration, we needed to combine our usability testing results with company financial activity and web site log files. The more data sources we can use, the more solid our numbers become.
When we believe that the frustration results in lost revenue, we’ll often look to identify those problems that get in the way of sales. However, for other types of frustration costs, we need to change how we calculate the costs.
Increased Expenses: We look at all the costs that the organization incurs because of the problem. These can be extra support calls, replacement materials, and increased server load.
Lost Productivity: This is slightly more difficult to estimate. Ideally, you could take the annual cost of the affected employees and divide it by hours they work per year. That would give you a cost-per-hour for a productive employee. Multiply that by the number of unproductive hours per year and you have an annual productivity loss figure.
However, it’s sometimes hard to calculate the cost of the affected employees. Figuring out people’s salaries, benefits, and other costs (such as the electricity they use) can be time consuming and politically difficult. Sometimes, it’s easier for us if we talk about what those employees would do instead of dealing with the loss productivity. If we can talk about how salespeople would have 15% more time selling, we can often frame the frustration costs in terms of the gains from the more productive activities.
Wasted Development Time: In many of today’s software products and web sites, there are entire areas of functionality that users never see or use. (One only needs to pull down the menus in their word processor to see the vast quantity of items they’ve never actually explored.) Developing these features and functions takes time, yet if no one knows they are there, what benefit is the publisher getting?
Looking at the list of features set for development, we can see how much each one will cost to develop (using similar calculations to those we used for lost productivity). Relatively simple usability tests can help us identify if users can discover the features (and will use them once they’ve discovered them). We can calculate the costs developers waste building things that people won’t use.
Looking for Pain
Sometimes it might be difficult to burst into the CEO’s office and declare you can save the corporation millions with solid usability work, even though you’ve carefully calculated your cost of frustration. Instead, you need to go to the next best place: the source of the pain.
Usually, in an organization, where there is frustration, there is pain. Somebody is not getting what they want because of the frustration. If the web site is so unusable that it stops shoppers in their place, then the sales and marketing people aren’t making their numbers. If the support phones ring too often, the support center manager has a staffing problem.
We’ve found that, by following the pain to its source, we can often find important allies who may even have the budget to fund the usability work. At a minimum, having one more champion on our side is extremely beneficial.
Leveraging the Cost of Frustration
The cost of frustration is one of our favorite techniques for demonstrating the value of our work. If we can pinpoint how frustrating the interfaces are and how that frustration is influencing the business, it becomes very easy to convince stakeholders that they need to change their designs.
We’ve found that, once we start focusing on the underlying cost of frustration figures, we end up with a very effective metric that we can use throughout the development process. It helps us identify which designs are most effective and gives us a tool to explain the benefits of good design.
A software developer and programmer, Jared founded User Interface Engineering in 1988. He has more than 15 years of experience conducting usability evaluations on a variety of products, and is an expert in low-fidelity prototyping techniques. Visit http://www.uie.com/ for more usability information. You can reach Jared by calling our office or by sending mail to jspool@uie.com.
