If you’ve ever had pages on your site that are no longer live, but still get indexed and appear within the search results, then you’ll no doubt be aware of what it takes to remove them.
Usually your first port of call will be to update the robots.txt file. This will stop the page being crawled and can eliminate some of the issues. However, for some inexperienced website owners, identifying and updating this kind of technical information can pose significant issues.
Therefore yesterday’s announcement from Google that it is integrating a URL removal option within Webmaster Tools is certainly a positive. Effectively you can now simply show the search engine what you don’t want to index just by copying and pasting the URL. This will see it removed from the search results for at least 90 days.
However, Google does still recommend that you follow the normal procedure if you want to achieve a permanent removal of a URL. This includes setting up a custom 404 or 410 permanent removal holding page. Updating the .robots.txt and applying the noindex Meta attribute.
It’s important to remove erroneous URLs as soon as possible, purely because these can damage your site. This is particularly true if the page has been replaced on a new domain or another location within the site and contains the same content. But it can also impact how your site is crawled and the perception that search engines have about it – i.e. it’s not great having a load of pages that don’t work, visitors won’t much care for that.
To find out more about the new feature, visit the Google Webmaster Blog.