Subscribe in a reader
HOME
How to remove content from Google
I have to admit not taking the time to really get to know Googlebot so now that Webmaster Central is introducing him/her/it, I am looking into how to remove content from Google.
Did you know that robots only do what you tell them?
Search engine robots, including our very own Googlebot, are incredibly polite.
Google offers several ways to remove indexed content including a URL removal tool which is tedious and extremely misunderstood. As a novice webmaster I once used it to remove pages when I should have added a 301 redirect to .htaccess. If you also spend time in Google Groups you will find people who are fearful of using the removal tool just as expected.
You can remove the following from Google’s index via robots.txt exclusion
Remove your entire website
Remove part of your website
Remove snippets
Remove cached pages
Remove an outdated link
Remove an image from Google Image Search
Remove a blog from Blog Search
Remove a RSS or Atom feed (i.e., block Feedfetcher)
Remove transcoded pages
Read: How can I remove content from Google’s index?
I am looking for a way to remove all content back to a specific date, it does not appear that Google offers this type of exclusion yet so there is a limit to what you can do via robots.txt.
Similar Post:
- Remove yourself from the Wayback Machine
- What are supplemental results?
- Why to not noindex your feed
March 6th, 2007 at 5:48 pm
You may have already thought of this, but you can just modify the header.php file of your WordPress theme to check the post date and then insert a noindex meta robots tag if the date is back far enough. Over time the search engines should remove the content as they revisit the posts and pick up the tag.
March 6th, 2007 at 7:50 pm
Marios - Great idea, I actually never wanted to remove content from Google until a few days ago.
March 6th, 2007 at 9:28 pm
Actually robots (good ones) will do what you tell them NOT to do, there isn’t really a way to make them do exactly what you want them to, like crawl my site fully 3 times a day. By this I mean of course you could tell a robot to noindex a page, but it if takes 3 weeks to come back and visit it nothing will happen. Thus the importance of the removal system.
It is a robots EX-clusion protocol after all.
March 7th, 2007 at 7:16 am
You could also create an XML sitemap with the old content not showing in it. Took me about 1.5-2 weeks to clear out some old crap on a site I was working on.
December 28th, 2007 at 12:29 am
How do I stop the constant automatic Google searches that pop up? I’m not sure what I did to cause it, but my work is frequently interrupted by the Google search page. Please help.
January 6th, 2008 at 8:38 am
What are you referring to THELMS? Sounds like you got spyware on your computer.