Every webmaster dreams of having tons of pages indexed in Google and other search engines. The thought of receiving millions of visitors each month is enough to make any site owner shiver with joy. Why then did Webmaster World, the nets largest community for webmasters, close the doors to all robots…including Google, Yahoo, and Microsoft?
In a bold move to end the madness of “rogue bots”, Brett Tabke, has dis-allowed all robots from spidering his massive site.
“Seeing what effect it will have on unauthorized bots. We spend 5-8hrs a week here fighting them. It is the biggest problem we have ever faced.”
Webmaster World has long been the premier destination for webmasters seeking knowledge about all aspects of site development. For many users, the best way to find things around Webmaster World is via Google search. The internal search is so terrible, members are allowed to enter a custom Google search code on the top of each page.
Being that Brett’s site is so established, it will be interesting to see this play out. Will his site be affected by the eventual downfall in search traffic? Does the site have enough critical mass to sustain traffic levels and growth without the major search engines?
These are all interesting questions that we should ask ourselves. Many webmasters rely completely on the 3 major search engines to provide traffic.
Webmaster World is currently testing a solution for the internal site search in an effort to combat user’s reliance on external search.
I’m excited to watch this story unfold. The largest webmaster community is the net’s guinea pig for this extremely bold move.