October 22, 2006

Click Fraud, Google, and Telepathy

Greetings. As noted today in a Washington Post article, the growing problem of "click fraud" increasingly threatens the confidence powering the economic engine at Google and other firms that depend on consumer click-through activities for significant portions of their revenue stream.

Google says that less than 10% of ad clicks through their system are illegitimate. Taking that at face value, it can still add up to an awful lot of fraud given the total amount of clicking going on out on the Web these days. And of course, when someone says that fraud is at such and such a level, what they really mean is detected fraud. Undetected fraud, like other proverbial "perfect crimes," doesn't show up in your stats since you don't know about it in the first place.

Google and other ad-based services are deploying ever more sophisticated computational mechanisms in their efforts to detect click fraud whenever possible, primarily by analyzing click patterns, click sources, and related metrics.

I suspect that in the long run this approach will prove to be insufficient for the problem at hand. The reason is that click fraud represents what I call a "Turing-Plus" problem.

That is, when it comes to detecting click fraud, it isn't enough to know that you're dealing with a human rather than an automated clicking system -- since click fraud is increasingly performed by paid human agents. Rather, what we really want is to telepathically "look inside" the clickers' heads to determine if they legitimately have interest in the product or service that they're clicking on.

Given the extremely limited availability of telepathic Web services these days, less esoteric techniques are still theoretically available. For example, we could implement mass surveillance of individual user Web viewing activities on a relatively gigantic scale, with broad data integration and correlation to yield higher-level patterns of user behavior. It is possible to postulate ways in which this sort of data could indeed be used to minimize click fraud by maximizing our knowledge of each user's detailed behavior on the Web -- but the obvious negative privacy implications would be enormous.

Given these realities, what could be done right now to reduce click fraud without introducing serious collateral damage to privacy and other consumer concerns?

One possibility would be to move toward a "two-step" process for reaching clicked ads. Rather than being taken immediately to a selected ad, the user would need to perform some action that would ot only help to affirm the probability of their being human, but that also would slow the click-through process. Slow this sequence down sufficiently, and you may significantly reduce the economic viability of those entities who are increasingly major "warpers" of the click-through model (by paying their agents specifically to click on particular ads).

There are indeed some problems with this two-step approach. We don't know by how much human click agents would need to be slowed to significantly reduce the impact of their operations, or to what extent such organizations could expand their agent base in an attempt to compensate.

The exact mechanisms provided for the two-step process could be critical, both to effectiveness and longevity. Variations on current "CAPTCHA" technology (the sometimes badly implemented "Turing Test" systems that display alphanumeric image sequences for the user to type back as "confirmation") may have some merit, though they also have significant limitations. Other approaches would also need to be investigated and developed.

An even more obvious problem with any multi-stage click approach is that many advertisers may object to any methodology that appears to insert temporal or other barriers between consumers and the ease with which they can reach an ad. But would all advertisers feel this way, especially when the alternative is likely to be increasing levels of undetected click fraud that they ultimately pay for? At least as an option, I suspect that a well-designed two-step click-through process could become increasingly attractive.

Ultimately though, it's hard to see how the current single-stage click-through environment can be successfully leveraged indefinitely for the benefit of advertisers and service providers, in the face of the globally dispersed resources available to perpetrator of click fraud. Sooner or later, something's got to give.


Posted by Lauren at October 22, 2006 04:12 PM | Permalink
Twitter: @laurenweinstein
Google+: Lauren Weinstein