Mozilla has a new project called Test Pilot, which it uses to test usability of its products. It collects data from users on a strictly voluntary basis, and shares that data with the public anonymously so others can use it too. In fact, the testing platform itself will be available to the public as well.
Wired.com writer Michael Calore reports, “Any research institution that needs usability data can draft a request to query the hive mind of worldwide Firefox users. Tests can be conducted in a matter of hours with virtually no overhead — a dream compared to the hours and days spend recruiting, screening and testing participants in traditional usability studies.”
What exactly does Mozilla hope to find out with this? Things like: how many tabs users have open at a time in Firefox, how often the stop button is pushed, how often do people open new tabs for searches, etc. The possibilities are endless, and in the end could prove very useful for third parties.
Here’s the rundown of exactly how it works according to Mozilla itself:
– We develop and promote a formal Test Pilot program with a Firefox add-on at its core.
– The first time the Test Pilot add-on is run, it asks a few simple non-personally-identifiable questions in order to put the user into a demographic bucket, e.g. technical level, locale, etc., and to let them opt in to additional anonymous instrumentation.
– Test Pilot will then notify its users when a new experiment is available for testing. If the user opts in, it will download the required software (if any) and load any information required to get started with the new experiment, e.g. overview, use cases, etc.
– After either a specified amount of time or upon completion of a specific action, Test Pilot will prompt the user for feedback. The feedback form will only ask a few questions selected from a much larger set. A link will be provided to provided more comprehensive unstructured feedback or bug reports.
– The set of questions posed for feedback will be randomly distributed within each demographic bucket to ensure statistical significance of the results.
– Anonymized aggregate results and analysis will then be posted automatically to the Test Pilot site.
– All participants will receive a “flight badge” displayed in their Test Pilot profile and available to embed on blogs, social networks, etc.
Mozilla stresses that the project is not for general market research though, and is specifically for usability and feature research. Usability is a too-often-overlooked aspect of web development and design, and I think webmasters could really benefit from this type of thing. It certainly seems worth keeping an eye on Mozilla’s results at the very least.
