Joseph Milana, chief scientist at Fair Isaac, discussed the early results of his company’s click fraud study in an exclusive chat with murdok.
Click Fraud Unbeatable Without Advertiser Data
Pathological traffic that impacted a small sample of less than a dozen websites resulted in advertisers being billed for illicit clicks that the search engines did not catch. The early results of a Fair Isaac study of click fraud showed some channels could hit ad campaign budgets at a rate of 10 to 15 percent.
Also, Fair Isaac contends that Google’s specific estimates of what their unsupervised detection efforts find in click fraud is “not believable,” based on their experience detecting fraud in other industries. Without advertiser data, search engines cannot defeat click fraud.
We spoke with the newly popular Dr. Milana about his presentation at InterACT 2007 regarding click fraud. Fair Isaac did share the 35-slide presentation with us, and Dr. Milana spoke at length about how Fair Isaac uses technology to detect fraud.
Part of his research and development work involves looking for places to apply his firm’s technology. Most people know of Fair Isaac for their work on credit scores, but there is a lot of high-tech they can and have brought to other fields.
“It caught our attention,” Dr. Milana said of click fraud, which arrived on their radar about two years ago. The issue for them would be to employ their signature Falcon technology to assessing click fraud.
Fair Isaac turned to SEMPO for assistance with the project. SEMPO encouraged advertisers to provide data to Dr. Milana so they could profile transactions, i.e., clicks, look for anomalies, and determine a pathology score for that visit.
(We contacted current SEMPO president Gord Hotchkiss about the Fair Isaac study. Though he could not comment on Fair Isaac’s findings due to the small size of the dataset, he did say, “At this point, our primary concern is increasing the size of the data set (getting more advertisers to volunteer their data) to allow Fair Isaac to extend their methodology to a more representative data set.”)
“We see some nuances, and want more data,” Dr. Milana said of their early results. They broke pathological traffic into two categories: malicious competitors, a threat they see as benign; and clicks for profit, where affiliate traffic in particular was a problem.
He noted how their technology has been proven in the payment card industry. Applying it to click fraud assessment, where search engines do not have the advertiser data, meant doing anomaly detection to find illicit traffic.
“Advertisers were charged” for this traffic, Dr. Milana said. His preliminary conclusion contends that search engines cannot solve the click fraud problem alone without having advertiser data available.
A request to Shuman Ghosemajumder at Google for his company’s feedback on Fair Isaac’s early results brought us a statement from Google on click fraud:
We take the protection of our advertisers very seriously. The amount we filter proactively is consistently less than 10% of all clicks but individual invalid click rates can vary by advertiser, campaign and even keyword.
The relatively rare cases we find of advertisers being affected by undetected click fraud constitute less than 0.02% of all clicks.
Without knowing more about Fair Isaac’s data gathering and methodology it is difficult to comment on their study except to say that a handful is not a representative sample size for our hundreds of thousands of advertisers.
Here is what we do know: Fair Isaac had access to advertiser data from a small sample of websites. Their modeling methods as applied to determining click fraud found some high instances of those advertisers being charged for illicit clicks, at a much higher rate than search engines claim gets through to those ad campaigns.
Considering Fair Isaac’s interest in finding new markets for their products, we wanted to know if Dr. Milana entered this study with a formed hypothesis about click fraud. He said he had “an open mind going in.” Since stats on click fraud were so varied, Fair Isaac had no preconceived notions about what they might find.
Dr. Milana’s main point holds that click fraud can’t be beaten with search data alone. Unlike the payment card industry, there are no objective click reviewers out there. Advertiser data needs to be part of the equation because search engines can’t see the true impact of a click.
(We asked Google about the need for advertiser data to provide a better picture of click fraud, but we did not receive a response by press time.)
The percentage of click fraud Fair Isaac found was in line with numbers reported regularly by Click Forensics. Tom Cuthbert, Click Forensics’ CEO, told murdok that web marketers need to take the numbers seriously. “14.8 percent of pay-per-click advertising dollars is a huge amount of money,” he said.
For Dr. Milana, the early research will proceed into more in-depth work, as they gain more advertiser data to use in their assessments. The search advertising industry can probably get away with being somewhat dismissive of the early Fair Isaac report due to the small number of sites being studied.
The real test comes when Fair Isaac has a larger body of data from advertisers to calculate. If a wider sample proves the accuracy of the first report, advertisers will really escalate their demand for greater transparency into search engine efforts to stop click fraud.
