Comments on: Collection of Robots.txt Files

By: Free Web Directory

Free Web Directory — Wed, 03 Jun 2009 08:17:11 +0000

nice stuff, you make ideas for me to change my robots.txt

By: jo

jo — Mon, 18 May 2009 13:03:57 +0000

what kind of robot.txt should be used for this site:

http://alphapal.net/fee-calculator.php

it is a paypal fee calculator

By: Elle

Elle — Sun, 10 May 2009 20:54:15 +0000

Should the /wp-content folder be included as well…where all the themes and plugins reside? I’ve not noticed it listed on any robot.txt file anywhere..

Thanks.

By: Bang Kritikus

Bang Kritikus — Mon, 02 Feb 2009 04:11:32 +0000

But, my blogspot’s robot.txt is not editable..

By: 国外最好的wordpress初学者文章（原文） | 精华之家|HIGHTHOUSE

国外最好的wordpress初学者文章（原文） | 精华之家|HIGHTHOUSE — Sun, 31 Aug 2008 01:09:22 +0000

[...] Collection of Robots.txt Files [...]