July 18, 2010

The Google Search "Secrets" in Plain View

Greetings. As I've mentioned previously, I tend to receive several hundred e-mails daily that relate to Google one way or another, many of which contain requests for advice regarding perceived or real Google-related issues. I try to help when I can.

And my concerns about what I consider to be significant shortcomings in Google's user communications structure -- especially when dealing with relatively unusual or serious problems -- are fairly well known.

But recent calls for regulatory oversight of Google Search are way off-base, and -- beyond the obvious First Amendment concerns -- threaten to undermine Google's efforts to provide the best possible natural (organic, algorithmic) search results via Google's continuing work to avoid distortions in or gaming of those results.

The fact that Google permits highly controversial search results to maintain algorithmically determined high rankings, even when it would be much easier from a public relations standpoint for Google to suppress those results, are another indication of Google's laudable efforts not to disrupt natural results rankings with manual alterations.

It's notable that in all of the countless cases where people have come to me (sometimes utterly convinced that Google has a vendetta against them) with complaints about their Web site "vanishing" from Google listings or not achieving the kind of result rankings that they felt were deserved, I've never once seen a case where any unfair or unreasonable actions by Google were actually in play in the situation.

In fact, in virtually all of these cases the problems have boiled down to one of two issues.

The first is that the sites have become contaminated with malware, often without the site owners' knowledge. This can result in Google quite reasonably flagging the sites as potentially dangerous to users, with resulting undesirable (but completely appropriate under the circumstances) effects. Even when site owners protest that their sites are clean, on closer inspection it turns out (in my experience) that their sites have been compromised in some manner.

An even more common case is that sites have not been organized in ways that make it possible for Google to effectively crawl their contents, or the sites include elements (often at the urging of unscrupulous SEO -- "Search Engine Optimization" -- firms), that violate the site guidelines Google has established to help avoid gaming of results to the detriment of Google's users overall.

There isn't anything intrinsically evil about SEO per se. In fact -- and here come those "secrets" in plain view that I promised -- Google has put major efforts into making available absolutely comprehensive resources and tools for webmasters, yet it appears that many or most Web site owners don't even realize that these exist.

Google's Webmaster Central is a universe of information, tools, video tutorials, and all manner of other resources that webmasters can use to better understand how Google crawls their sites, potential problems in sites; mechanisms to inform Google how sites are organized to enable efficient and complete crawling of text, video, and other formats; ways to gather metrics on how people discover sites; and so much more.

Over on YouTube, the Google Webmaster Central Channel contains hundreds of videos on related topics that should be of interest to anyone running a Web site, including many Q&A videos from Matt Cutts who heads up Google's "Webspam" team -- which is directly involved in these sorts of search quality issues (he's also a good guy -- I recommend paying attention to his suggestions!)

All of this isn't to assert that every problem anyone may have with Google will be solved via these Google resources -- nor to say that effective means to solving every other possible sort of Google-related problem necessarily even currently exists.

But for many common situations -- the kind where people may feel that Google is unfairly ranking their site -- or similar scenarios -- I believe that a reasoned analysis of the circumstances, especially in conjunction with the Google Webmaster Resources discussed above, will demonstrate that Google bends over backwards not only to keep their natural search results rankings as useful and honest as possible, but also that Google has worked very hard to explain how to optimize sites for best results -- and has provided tools to help make this as straightforward and painless as possible for webmasters.

--Lauren--

Posted by Lauren at July 18, 2010 02:03 PM | Permalink
Twitter: @laurenweinstein
Google+: Lauren Weinstein