AverageCoder.Net - Programming - Software - Techie Stuffs

How to remove your indexed pages or even your entire site from Google and Yahoo!

Posted by Daniel - 12,824 Views

The idea to start writing this post was started a couple months ago when I began to migrate my previous blog to this blog. I am amazed by WordPress Blogging Platform for its simplicity, plug-n-play feature and the ease of maintenance BUT My previous blog - which is still active for .NET experiment purposes - is using windows based server with no PHP and MySQL installed so there is now way to have a WordPress based blog on that server, SO I decided to get a linux based hosting service in order to be able to setup a WordPress blog, and here it is my current blog powered by WordPress.

Another reason why I decided to migrate is because I want to develop my blog but I don’t want the process distract my focus on my main job. With WordPress I can get the speed. For single needed feature there are so many plugins available to be plugged with little efforts and there are plenty of support I can get from the community.

OK enough with the opening story, let’s get into the main issue. There are pages from my previous blog which are indexed by Google and Yahoo! With the release of my new blog I want those indexed pages from my previous blog deleted from their index. Is it possible?

I use both of their webmaster tools, Google Webmaster Tools and Yahoo! Site Explorer. Following is the explanation of what I’ve done using those tools to delete indexed pages from my previous blog or the more precise word is to remove my previous blog from both Google and Yahoo! I will assume that you also use Yahoo! Site Explorer and Google Webmaster Tools.

Yahoo! Site Explorer

There are explanations on how to delete your site or your URLs from Yahoo! search index. You can read the explanation here. They recommend you to use robots.txt to prevent the indexing after the deletion, this is the note from their site:

When you’ve used Delete URL/Path to remove a URL from our index, we recommend using robots.txt to exclude the content from being added to our indexed again. Robots.txt excludes Yahoo!’s crawler (Yahoo! Slurp) from re-indexing your content.

To delete your indexed page or your entire site via Yahoo! Site Explorer Control Panel, you can follow their instruction here.

To get notified by the deletion process, go to [Preferences] - find the [Communications] section and then check the [Notify me of all actions I perform on my authenticated websites] checkbox.

UPDATE: The link below shows the approved removal request from Yahoo! Search Result:

Screenshot showing the approved removal request from Yahoo! Search

Google Webmaster Tool

Google Webmaster Tools also provides the feature to remove URLs which are already indexed by Google. 3 things that you should have them done before requesting a URL removal are below:

1. Make sure the content is no longer live on the web. Requests for the page or image you want to remove must return an HTTP 404 (not found) or 410 status code.
2. Block the content using a meta noindex tag.
3. Block the content using a robots.txt file.

From my experience, doing point number 2 and 3 aren’t enough, point number 1 is mandatory. My first removal request with the page still exists on my website has been denied, but when I removed the page and then submit another request, it was successful.

To remove URLs indexed by Google via Google Webmaster Tools you can navigate to [Tools] - [Remove URLs] and then click the [New Removal Request] button. You will then need to select one of several removal types provided.

• Individual URLs: web pages, images, or other files Remove outdated or blocked web pages, images, and other documents from appearing in Google search results.
• A directory and all subdirectories on your site Remove all files and subdirectories in a specific directory on your site from appearing in Google search results.
• Your entire site Remove your site from appearing in Google search results.
• Cached copy of a Google search result Remove the cached copy and description of a page that is either outdated or to which you’ve added a noarchive meta tag.

Select one of the removal types based on what you need, click the [Next] button, enter the URL needs to be removed then finally hit the [Add] button. Your request will be placed on the queue for processing.

As far as I know, Google Webmaster Tools doesn’t provide notification feature. You should check the status of your request manually from the control panel.

UPDATE: The link below shows the approved removal request from Google Search Result:

Screenshot showing the approved removal request from Google Search

That’s all. To check the pages of my previous blog on Google or Yahoo! Search you can use “site:averagecoder.com” query. I hope the result is the same on your browser, none of the pages from my previous blog still indexed by both search engines.

If you find anything incorrect on my post please say so, or if you find another way to remove indexed page(s) from any search engine, I would really love to hear it, please let me know…

Cheers :)