AverageCoder.Net - Programming - Software - Techie Stuffs

How to remove your indexed pages or even your entire site from Google and Yahoo!

Posted by Daniel - 3,965 Views

The idea to start writing this post was started a couple months ago when I began to migrate my previous blog to this blog. I am amazed by WordPress Blogging Platform for its simplicity, plug-n-play feature and the ease of maintenance BUT My previous blog - which is still active for .NET experiment purposes - is using windows based server with no PHP and MySQL installed so there is now way to have a WordPress based blog on that server, SO I decided to get a linux based hosting service in order to be able to setup a WordPress blog, and here it is my current blog powered by WordPress.

Another reason why I decided to migrate is because I want to develop my blog but I don’t want the process distract my focus on my main job. With WordPress I can get the speed. For single needed feature there are so many plugins available to be plugged with little efforts and there are plenty of support I can get from the community.

OK enough with the opening story, let’s get into the main issue. There are pages from my previous blog which are indexed by Google and Yahoo! With the release of my new blog I want those indexed pages from my previous blog deleted from their index. Is it possible?

I use both of their webmaster tools, Google Webmaster Tools and Yahoo! Site Explorer. Following is the explanation of what I’ve done using those tools to delete indexed pages from my previous blog or the more precise word is to remove my previous blog from both Google and Yahoo! I will assume that you also use Yahoo! Site Explorer and Google Webmaster Tools.

Yahoo! Site Explorer

There are explanations on how to delete your site or your URLs from Yahoo! search index. You can read the explanation here. They recommend you to use robots.txt to prevent the indexing after the deletion, this is the note from their site:

When you’ve used Delete URL/Path to remove a URL from our index, we recommend using robots.txt to exclude the content from being added to our indexed again. Robots.txt excludes Yahoo!’s crawler (Yahoo! Slurp) from re-indexing your content.

To delete your indexed page or your entire site via Yahoo! Site Explorer Control Panel, you can follow their instruction here.

To get notified by the deletion process, go to [Preferences] - find the [Communications] section and then check the [Notify me of all actions I perform on my authenticated websites] checkbox.

UPDATE: The link below shows the approved removal request from Yahoo! Search Result:

Screenshot showing the approved removal request from Yahoo! Search

Google Webmaster Tool

Google Webmaster Tools also provides the feature to remove URLs which are already indexed by Google. 3 things that you should have them done before requesting a URL removal are below:

1. Make sure the content is no longer live on the web. Requests for the page or image you want to remove must return an HTTP 404 (not found) or 410 status code.
2. Block the content using a meta noindex tag.
3. Block the content using a robots.txt file.

From my experience, doing point number 2 and 3 aren’t enough, point number 1 is mandatory. My first removal request with the page still exists on my website has been denied, but when I removed the page and then submit another request, it was successful.

To remove URLs indexed by Google via Google Webmaster Tools you can navigate to [Tools] - [Remove URLs] and then click the [New Removal Request] button. You will then need to select one of several removal types provided.

• Individual URLs: web pages, images, or other files Remove outdated or blocked web pages, images, and other documents from appearing in Google search results.
• A directory and all subdirectories on your site Remove all files and subdirectories in a specific directory on your site from appearing in Google search results.
• Your entire site Remove your site from appearing in Google search results.
• Cached copy of a Google search result Remove the cached copy and description of a page that is either outdated or to which you’ve added a noarchive meta tag.

Select one of the removal types based on what you need, click the [Next] button, enter the URL needs to be removed then finally hit the [Add] button. Your request will be placed on the queue for processing.

As far as I know, Google Webmaster Tools doesn’t provide notification feature. You should check the status of your request manually from the control panel.

UPDATE: The link below shows the approved removal request from Google Search Result:

Screenshot showing the approved removal request from Google Search

That’s all. To check the pages of my previous blog on Google or Yahoo! Search you can use “site:averagecoder.com” query. I hope the result is the same on your browser, none of the pages from my previous blog still indexed by both search engines.

If you find anything incorrect on my post please say so, or if you find another way to remove indexed page(s) from any search engine, I would really love to hear it, please let me know…

Cheers :)

__________________________

Related posts brought to you by Yet Another Related Posts Plugin.

7 Responses to “How to remove your indexed pages or even your entire site from Google and Yahoo!”

1. Johnny Idol Says:
   February 14th, 2008 at 5:58 pm

   Nice post, it might come in handy sooner or later!

2. Daniel Says:
   February 15th, 2008 at 12:17 am

   @Johnny: Yep, my other consideration why I need to remove my previous blog from those search engines is I don’t want to have two identical article content indexed by them, some folks said it’s not good for SEO - I am not sure since I am not an SEO expert but it’s not a bad thing to follow the advice right? Even in reality no one can certainly sure about what those search engines do to treat our pages except their programmers :D

   If we deal with human, I can just say that those identical articles are mine, but unfortunately we’re dealing with “engine” here, so let’s treat the engine like an engine - with a good moral of course :)

3. Webmaster Uk Says:
   February 28th, 2009 at 9:16 pm

   Thank you really. This is the information that I’ve been looking for.

4. Tennessee SEO Says:
   May 6th, 2009 at 4:00 pm

   With WordPress I can get the speed. For single needed feature there are so many plugins available to be plugged with little efforts and there are plenty of support I can get from the community.

5. pos Says:
   May 6th, 2009 at 11:24 pm

   Great stuff!

6. Google WMC user Says:
   May 10th, 2009 at 6:21 am

   The part of your blog concerning “remove entire site from google” is a little misleading. It’s because google is a little misleading. If you remove your entire site from google using the WMC, and later decide to add your site back, then all of those “removed” urls will be right back in the search engine results. They do not really provide a way to delete your entire site, unless your intention is to never ever use the site again. The only way to truly remove your entire site from google search results, is to remove them one by one, url by url in the WMC. Then and only then will those pages be “deleted” from the search results. The remove entire site is only temporary until you add it back. This is an important note as I had to learn it the hard way.

7. Daniel Says:
   May 11th, 2009 at 6:11 am

   You’re right, good point. I saw it worked because I never use my old site anymore, thanks for sharing the tips :)

Leave a Reply

Name (required)

Mail (will not be published) (required)

Website

Options for your comment:

Notify me via e-mail if anyone post more comment to this post

Enable CommentLuv which will try to get your last blog post to be inserted at the end of your comment