Comments on: Create a robots.txt file http://www.dailyblogtips.com/create-a-robotstxt-file/ Fri, 19 Mar 2010 18:01:24 -0500 hourly 1 By: Keith Davis http://www.dailyblogtips.com/create-a-robotstxt-file/comment-page-2/#comment-1027265 Keith Davis Wed, 25 Nov 2009 19:46:22 +0000 http://www.dailyblogtips.com/create-a-robotstxt-file/#comment-1027265 Hi Dan That's my next step for my site... creating a robots.txt file. I've always understood the problem of duplicate content, but the penny has only just dropped about how easy it is to create duplicate content with a wordpress site. Great solution, to use a robots.txt file. Hi Dan
That’s my next step for my site… creating a robots.txt file.

I’ve always understood the problem of duplicate content, but the penny has only just dropped about how easy it is to create duplicate content with a wordpress site.

Great solution, to use a robots.txt file.

]]> By: Daily Good Tips http://www.dailyblogtips.com/create-a-robotstxt-file/comment-page-2/#comment-944035 Daily Good Tips Sat, 08 Aug 2009 03:36:29 +0000 http://www.dailyblogtips.com/create-a-robotstxt-file/#comment-944035 Create robot.txt just can be created in wordpress or website with wordpress but I think robot.txt creating can not be applied in blogspot template. We just talk google if we want to change the robot.txt. Create robot.txt just can be created in wordpress or website with wordpress but I think robot.txt creating can not be applied in blogspot template. We just talk google if we want to change the robot.txt.

]]> By: Kitchener Waterloo SEO http://www.dailyblogtips.com/create-a-robotstxt-file/comment-page-2/#comment-939119 Kitchener Waterloo SEO Sat, 01 Aug 2009 02:57:10 +0000 http://www.dailyblogtips.com/create-a-robotstxt-file/#comment-939119 I like your idea of blocking whole directories instead of mentioning specific files (which I've seen done). For example: User agent: * Disallow: /my_passwords_file.html Robot.txt are considered best practice, but they can leave your website open to exploits if your not careful. I like your idea of blocking whole directories instead of mentioning specific files (which I’ve seen done). For example:

User agent: *

Disallow: /my_passwords_file.html

Robot.txt are considered best practice, but they can leave your website open to exploits if your not careful.

]]> By: blogging posts http://www.dailyblogtips.com/create-a-robotstxt-file/comment-page-2/#comment-905132 blogging posts Fri, 26 Jun 2009 16:18:13 +0000 http://www.dailyblogtips.com/create-a-robotstxt-file/#comment-905132 I real want to know, how can we do with the image, dis or allow in robots.txt. I real want to know, how can we do with the image, dis or allow in robots.txt.

]]> By: Kamal Hasa http://www.dailyblogtips.com/create-a-robotstxt-file/comment-page-2/#comment-894876 Kamal Hasa Sun, 14 Jun 2009 06:02:33 +0000 http://www.dailyblogtips.com/create-a-robotstxt-file/#comment-894876 Thank you for this post. But askapache has got a better topic on the same subject.. Thank you for this post. But askapache has got a better topic on the same subject..

]]>