Greetings. I'd appreciate feedback from the Internet community regarding the following issue.
Search engines have of course become the primary means by which vast numbers of people find all manner of information. For many firms, if you don't have a high rank with Google, it's as if you don't exist (or at least, many companies appear to feel that way).
Increasingly, cases are appearing of individuals and organizations being defamed or otherwise personally damaged -- lives sometimes utterly disrupted -- by purpose-built, falsified Web pages, frequently located in distant jurisdictions. Search engine results are typically the primary means by which such attacks are promulgated and sustained by providing a continuing stream of viewers to those Web pages. Due to ranking algorithms, attempts to counter such attacks with other Web pages may not be widely seen since they are not directly associated with the attacking pages.
Courts appear generally reluctant to order offending Web page take downs in such cases, except where intellectual property (e.g. DMCA orders) are involved, and take downs do not necessarily inform viewers of the ongoing controversy in a logically connected manner. Additionally, "remedies" that result in suppression of information, rather than providing additional information, are generally ineffective and counter to the "open information whenever possible" philosophy that many of us share.
Question: Would it make sense for search engines, only in carefully limited, delineated, and serious situations, to provide on some search results a "Disputed Page" link to information explaining the dispute in detail, as an available middle ground between complete non-action and total page take downs?
Search engine firms have generally taken the view that they are akin to telephone directories, and bear no responsibility for the content of the pages that they reference. Similarly, when ostensibly aggrieved parties approach these firms with concerns about "offending" pages, the usual response is that the search firms can do nothing about those pages, and that any complaints need to be taken to the Web page owner or associated ISP. From a practical and jurisdictional standpoint, this turns out to be impossible in many cases.
We clearly do not want to hold search engines responsible for other sites' content, even when locally cached. To do so would likely obliterate the entire search engine model and industry under a storm of litigation, to everyone's detriment. It must be noted, however, that increasing calls for holding search engines responsible in just such a manner are being heard in some political and judicial circles, likely out of frustration with the status quo, which currently tends not to offer reasonable dispute resolution paths in most situations. This is a serious warning sign, suggesting that we should consider some new approaches on our own, or risk draconian and damaging legislation.
The telephone directory argument also has some problems. Unlike typical phone books, search engines are not passive publishers of information. In addition to third-party ads tied to the core listings, a key facet of search engines is intensive ranking and decision-based ordering of content listings, usually via highly proprietary algorithms. Such ranking provides a high percentage of the value-added represented by search engine results.
So while search engines are not responsible and should not be held responsible for the content of the outside pages and data they index, they are very much directly involved as decision-making gatekeepers (albeit, usually through fully automated algorithms) that determine to a major extent which individual Web pages are likely -- or unlikely -- to be discovered by Internet users.
More questions: Given the power that search engines possess in these regards, do they bear any responsibility for helping to untangle serious disputes regarding the pages they reference and often profit from? If search engines do not voluntarily move in this direction, do they risk damaging legislation written without a genuine understanding of the complex technical and business issues involved?
In my view, an evolution by search engines to deal with these situations should be predicated on that key concept of maximizing the availability of information. Page take downs -- which are likely to be ineffective in the long run as noted -- should be a last resort. Similarly, a total laissez faire approach is also unlikely to be tolerated indefinitely by the political and judicial establishments.
So returning to where we started... Could some sort of "dispute link" -- tied directly to information regarding particularly serious page disputes -- provide a reasonable means to help ameliorate these situations without risking the more destructive alternatives? If so, how would such a system be effectively implemented in a practical fashion? How could such a system be structured to avoid being swamped by relatively trivial complaints?
Would providing related dispute links only to persons with court orders make sense to limit potential abuse of the mechanism, or would requiring the use of the expensive and delay-prone courts be far too restrictive a qualification? Could such a dispute system operate purely on a voluntary basis? (Voluntary would be very much preferred in my opinion.) What are the cost factors involved in such a system and how could they be reasonably addressed?
Overall then, is it possible to structure such a system along these lines so that it is practical, workable, and also palatable to the major search engine firms, as an alternative to barreling along toward an onerous and likely politically motivated crackdown down the line?
Or would this concept just never work -- and that crackdown is inevitable?
Your thoughts would be appreciated. Thanks very much.
Blog Update (June 17, 2007): Extending Google Blacklists for Dispute Resolutions