From large scale stat providers right down to the smallest detail it is easy to take a statistic out of context and draw false conclusions from it.
Some examples:
Examples of bogus statistics and the value of evaluating stats out of context.
- MySpace has more traffic than Yahoo! – over-represents automated spam and does not account for the targeting and value of some of Yahoo!’s search traffic and vertical content. In addition the story about this conveniently separated the Yahoo! traffic into multiple smaller streams, and typically counts pageviews when many of Yahoo!’s products use AJAx.
- Yahoo! has 25% of the search market – over-represents Yahoo! by counting searches on vertical properties and internal searches. Their actual volume is probably closer to 10%…it is amazing how quickly their marketshare has been eroding.
- Looking at search marketshare by referral data to large websites – does not take into account algorithmic bias of some engines toward large or small websites.
- Alexa data for my site – because it is an internet marketing blog, it is heavily overrepresented.
- Lower conversion rate for leads after redesign – I redesigned a friend’s site and made it easier for people to contact them or get price estimates. Originally their site was unattractive and it was hard to contact them or get a quote. After making it easier to do those the end conversion rate of the people who did those action items was slightly lower due to it being so much easier to do them.
- Higher AdSense earnings per click or clickthrough rate on a finance site on Christmas eve – the people searching may be more desperate, and thus more willing to click on anything, and they may also search a disproportionately higher rate for higher value terms.
- Seasonal bias – I have a seasonal site which I fixed broken links right around when it was going out of season. It did not make more, but fell less hard than it would have. When the seasons changed again the earnings shot through the roof.
Why do Bogus Stats Matter?
As a marketer it is important to realize how statistics can lie for two main reasons.
1. so you are measuring the right stuff
2. so you can present market data in a way that biases your story such that it is remarkable and easy to spread
1.) After Brian Clark rewrote my sales letter my number of ebook sales per day jumped up in the short term, but that was largely because
- a trusted voice recommended it
- other trusted voices echoed that trusted voice
- a new audience got exposure to my site
A better measure of the effectiveness of the new sales letter would be to look if the percent, conversion rate, or number of affiliate sales goes up. An even better measure would be a Google AdWords A/B split test.
2.) The MySpace has more traffic than Yahoo! story was a way to promote the Hitwise statistic service. But you do not even need to collect a bunch of expensive market research data to create a piece of market research data that would easily spread. Simply cross reference a few free or cheap publicly accessible tools like Quantcast, Compete.com, Alexa, Technorati mentions by day, Google Trends, Spyfu, Key Compete, buy AdWords to track search volume, and the number of Google search results over time.
Statistics & Humor
It is easy to find errors or weird biases when you look for ideas connected to weird human actions. And if they are funny, the emotional responses will help them spread quickly. For example, you can use Google trends to research which country or city the most perverted country in the world.
Serious Statistics
When you cite stats from any trusted brand you leverage the strength of their brand.
Many statistics are nothing but self promotional public relations drivel. If you do a great job with your public relations then you can make reasonably believable stats sound factual, even if the collection method is biased. Leverage the bias or errors in some of the publicly available tools and then try to spread that information to the media. Once you get a trusted third party to buy off on it, you then use that exposure as leverage to make the data even more concrete and believable, and to get additional exposure.
Selling Stats to an Audience:
How hard would it be to make the Digg homepage with a title like An Analytical View of Digg’s Growth or Digg Traffic to Pass AOL by January 2008, especially if you cited third party research that backed up your claims?
How hard is it to spread stories on blogs about how important bloggers are? If it has a few images and stats in it a story feels far more concrete and is easy to spread, especially if spreading it makes the messenger feel important / valuable.
FurlBookmark murdok:
Aaron Wall is the author of SEO Book, an ebook offering the latest
search engine optimization tips and strategies. From SEOBook.com Aaron
gives away free advice and search engine optimization tools. He is a
regular conference speaker, partner in Clientside SEM, and runs the
Threadwatch community.
