Every usability practitioner, early or later, run into a simple question: how many users do I need to conduct a good usability test?
Jacob Nielsen argues in his alertbox that five users are enough to detect 80% of all usability problems. On the other hand Christine Perfetti and Lori Landesman claim that eight people are not sufficient to discover all usability problems. So what is the answer to our question?
Before giving an answer to our question we have to take into consideration some factors that influence the number of people that had to be hired in order to conduct usability tests:
* Project budget
* Project timeline
* Project complexity
Project budget
The biggest obstacle for usability is represented by project budget. Here, in Italy, companies does not invest in usability and all related subjects are treated as optional services in estimates. Clients infact prefer to get discounts in the final price by cutting off usability costs (including user testing) or more frequent investing available budget in other services like home page flash animations. These prejudices toward usability influence in the negative the final product and when clients realize the mistake they made, it is too late to find solutions without investing more money. So there’s no budget to hire people and conduct usability tests.
Project timeline
Clients believe that a good web site can be done in a week and insist to get it published in five days: one day to get material, one day to prepare layout, two days to develop pages and last day to make funcionality test and deploy site. There’s no option to consider usability studies in such a timeline. So there’s no time to hire people and conduct usability tests.
Project complexity
To analyze this factor I will take as an example web site projects that can be divided into four types (naturally this is a simplified categorization):
* small: web sites that go from a simple waiting page to 10 static pages
* medium: web sites that exceed 10 static pages or include few dynamic contents and special functionalities
* large: web sites based on dynamic contents and include a lot of special functionalities
* web applications
Logically usability analysis is useless in small size web sites due to their small complexity and most important for their reduced budget. In medium size web sites and in web applications usability analysis can be done either using an heuristic evaluation or usability test with no more than five users. Naturally, if funds and time are sufficient it is better to use both options. In large web sites depending on budget (timelines in this kind of web site are generally more relaxed) is better to conduct a heuristic evaluation and usability test with no more than ten users.
The answer
So depending on the external factors the best result can be achieved by making a usability test with no more than ten people, in most cases closer to five. Moreover it is better to conduct usability tests integrating them with an iterative approach ideally following this procedure:
* Conduct a heuristic evaluation in prototyping phase so biggest usability issues can be resolved early
* Conduct a usability test with four users at the end of the design process in order to correct design usability problems and collect feedback on interfaces
* Conduct a comprehensive usability test with four users at the end of the development phase in order to correct interaction usability problems
* Conduct a usability test with two users at the end of the deploy process in order to verify that all usability issues have been corrected and also to gather feedback that can be used as recommendations for a future version.
Source: How many users makes a good usability test?
More details: Interno Tredici
Add to document.write(“Del.icio.us”) | Digg | Yahoo! My Web
Technorati:
Menghini Calderon Fabrizio Antonio is an IT engineer that actually works in a web agency located in Rome, Italy, where is responsible of usability, human factors and information architecture and also is in charge of developing web sites and web applications using CSS, XHTML and PHP.
To find more about Fabrizio visit Interno Tredici.
