Call 03332 207 677

Tara West

How to Deal with Duplicate Content on a Website

9th Jun 2011 SEO 1 minute to read


Copy Paste KeyboardDuplicate content has been in the SEO headlines more and more recently since the notorious Farmer / Panda update began.

This post talks about some of the ways to deal with some duplicate content on a site including Robots.txt, Meta tags, redirects and canonical tags. Each solution is explained in a simple to understand fashion.

You can use your robots.txt file to block content which you do not want indexed. With this option you can block entire groups of URLs if they are within the same folder, for example /price. This is ideal if your duplicates are all produced with the same pattern and you can identify them through the folder structure. If your duplicates are completely random, you can also add individual page URLs to be blocked with a robots.txt

You can also use a Meta robots tag to ask search engines not to index content. This should be placed on every page which you do not want indexed. This example makes sure the links on the page are still followed by the search engines and it is just the one page which is not indexed:

<meta name=”robots” content=”noindex,follow” />

You can place permanent 301 redirects on the URLs of the duplicate content, to point users and search engines to the standard version. This can be very effective providing a permanent 301 redirect is used.

The canonical tag is another option for telling Google which pages you do and do not want indexed. The canonical tag should be placed on all the duplicate pages and states which is the standard version which should be indexed.

<link rel=”canonical” href=”www.the-correct-url-here.com” />

Whichever solution you use for combating your duplicate content, it is important to remember to always use the standard version of the URL when creating internal links.

Image Source

Copy Paste Keyboard via BigStock

Share this post

Tara West
About the author

Tara West

Tara West is an experienced SEO and PPC specialist who worked at Koozai, with particular expertise within AdWords PPC and Remarketing. She has worked on a wide variety of verticals, from plumbing and travel to fashion and beauty.

results-bg

Fixing Website Redirects and Canonical Issues

Your Free Whitepaper

23 pages of goodness

Redirects Whitepaper Cover

What do you think?

  • aspect-ratio UK Search Awards Sam Noble Winner
    Cat Birch

    Samantha Noble Wins UK Search Personality 2016 at UK Search Awards

    The UK Search Awards is an exciting night for all; brimming with excitement, anticipation and pride, these awards showcase the very best in Digital Marketing. Our pride was overflowing as we watched our very own Sam Noble be honoured with the top prize of the night: UK Search Personality 2016.

    (more…)

    Cat Birch
    @MissCatBirch
    2nd Dec 2016
    News
  • aspect-ratio UK Search Awards Sam Noble Winner
    Cat Birch

    Koozai Shortlisted for Three Awards at the UK Search Awards 2016

    The UK Search Awards are here and we’re pleased to announce that we’ve been shortlisted for three awards at this prestigious ceremony.

    What an incredible year 2016 has been for us so far. We’ve celebrated turning 10 as a company, and in the last few months alone we’ve been nominated for a range of campaigns, won a bunch of awards and now we’ve been shortlisted for even more success.

    (more…)

    Cat Birch
    @MissCatBirch
    30th Nov 2016
    News

Digital Ideas Monthly

Sign up now and get our free monthly email. It’s filled with our favourite pieces of the news from the industry, SEO, PPC, Social Media and more. And, don’t forget - it’s free, so why haven’t you signed up already?