7 Ways to Find (and Fix!) Duplicate Content

man searching for duplicate contentDuplicate content happens.

You may be aware that content theft can create duplicate content and search engine penalties for your website or blog.

But, sometimes we inadvertently create duplicate content on our own websites, by not understanding content creation best practices!

Recent search engine updates have made it more important than ever to identify and address duplicate content … before it impacts your organic search ranking or link popularity.

Here’s how to detect and fix both types:

Detect and Fix Onsite Duplicate Content

Webmaster tools provide easy ways to help address duplicate content issues within your site or blog.  Here’s a gameplan:

  • Top Level Domains: Select Preferred URL Structure

To avoid creating multiple URLs delivering the same content, decide what your preferred URL structure is.

For example, should our website address be? Or should it be simply ? For top level domains (tLDs), do you want the site to appear with or without the ‘www’?  It doesn’t matter which you pick, as long as you pick and use one or the other.

Once decided, this preference can be set within Google’s free Webmaster Tools under ‘Configuration’.

With the tLD sorted, you can now address individual instances of duplicate content.

  • URL Structure at the Page Level: Unique and Consistent

Determine which unique URL you would prefer to use for each piece of content. Flatter URL structures that keep the content closer to the root domain are better for SEO and can help influence a higher rate of clicks on calls-to-action.

Here’s a hypothetical example:

Once you’ve selected a preferred URL structure, be consistent.  Use the preferred URLs throughout site navigation, in anchor text links and in sitemap files.

  • Apply 301 Redirects

 A 301 redirect points search engines from a page with an earlier URL structure to the page with the newly structured URL , so that search engines do not perceive duplicate content - a fantastic way to reunify duplicate content.

If you find duplication based on earlier URL structures, use a 301 redirect to indicate that the content has permanently moved.

  • Implement Canonical Tag

301 redirects are server settings.

If you’re not comfortable implementing a server change or your hosting provider does not support the use of 301 redirects, the canonical tag can also be used.

All major search engines including Google, Bing and Yahoo currently support the use of the canonical tag (rel=canonical), which can help point search engines to your preferred URLS when inserted into the page code of your site.

The official Google Webmaster Tools explanation, including an example, can be found here.

(PS - Here’s  an explanation and example of this in use, for commenter Bruce:

There’s a very simple way out of this: the canonical tag. In the *<head>* of the page, you put the preferred URL in a tag like this (this is a sample from our own blog):


The href should be the URL you like most for the content. If a search engine sees duplicate content on the site, it’ll use the canonical URL as the one and only one page on the site; there will be no dup content penalty).

  • URL Parameter Handling Tool

For duplicate content issues that arise due to multiple URLs with query string parameters, consider using the URL parameter handling tool within Google Webmaster Tools.

(A query string is the part of a URL that contains data to be passed to web applications. Query strings contain parameters or variables. Sometimes these parameters impact the content of the page).

To clarify, here’s an example:

www.XYZClothing.com/products/women?category=dresses&color=green is a query string with parameters for ‘Category’ and ‘Color’.

Other URL parameters do not impact page content and are solely for tracking (like a session ID) or sorting purposes.

www.XYZClothing.com/products/women?category=dresses&sort=price_ascending delivers the same content as www.XYZClothing.com/products/women?category=dresses.

Google’s Webmaster tools to detect and clarify these situations.

  • Reconsider Robot.txt Use

Of note, Google no longer recommends blocking access to duplicate content with a robot.txt file.

(This file is in the root directory of the website (www.XYZClothing.com/robots.txt) and it instructs search bots on what to index within the site).

If you are currently managing onsite duplicate content with a robot.txt file, you can read more about Google’s newest recommendations here .

Detect and Fix Offsite Duplicate Content 

What about offsite issues that create duplicate content?

Guest blogging, article syndication and maliciously pirated or scraped content can all negatively impact your organic search ranking, author rank and most importantly – control over your own original content.

An advanced plagiarism detection tool such as iCopyright’s premium Discovery™ service, continuously scans the web to protect your content.

Discovery crawls the internet every day looking for duplicates of your content.

And, unlike simple plagiarism detection tools,  Discovery automates monitoring AND the entire infringement resolution process, from first contact to sending the DMCA takedown notice should that be needed to resolve the situation.

Takeaway for bloggers, writers and publishers: 

You work hard to create unique content.  Make sure it appears both on site and across the web as it should!

  • Use Webmaster tools to identify where the issues exist and fix duplicate content, both on and off your website or blog.
  • Using an advanced duplicate content detection and resolution tool can make this job much easier.

Attending to these 7 suggestions now will pay search big result dividends later!

To find Offsite Duplicate Content, Try Discovery Free For 30 Days!

Content theft is a reality in today’s online world.

To learn more about how iCopyright’s digital copyright solution can help you protect your online content, watch a quick video … then take advantage of our f!

Free eBook: Learn from Your Publishing Peers!

Download our eBook filled with content strategies and insights from online editors, bloggers and content marketers on the cutting edge.         


5 Tech Tools for Better Blog Writing & Content Creation

What’s your biggest blogging challenge? Is it finding time to write? Coming up with your next topic? Or do you need a little help getting found? While technology won’t help you define your “secret sauce,” there are some remarkable tech tools to help make your content creation and blog writing smarter: 1. Website Analytics Website [...]

How To Protect Your Blog Content From RSS Feed Scrapers

Does your RSS feed inadvertently contribute to content theft? In a prior post, we talked about how peer pressure can work to fight online content piracy – particularly for naïve infringement.  But peer pressure alone can’t always work.  That’s because some content is stolen by bots and automated programs that scrape your RSS feed. Like [...]

How to Monetize your Blog: Make it “License friendly” (Part 2)

  The blogger’s online business toolkit seems to grow daily:  advertising, affiliate sales, sponsored posts, selling products and services, consulting, speaking, selling books, and more. Whether you do ALL of those things — or NONE of those things — there’s one more revenue opportunity that is often overlooked. Make your original online content license friendly. [...]

LicenseFriendly & Fair Use / Fair Dealing

Adjective.  li·cense·friend·ly.  Describes a website that simultaneously makes it (i) easy for users to license content and   (ii) difficult for users to steal content.  For example: “The Daily Planet sure is licensefriendly!  I licensed an article for republication on my blog today with a single click and for only $5.  That was a no-brainer [...]

How To Monetize Your Blog (Part 1)

Blogging takes incredible discipline and hard work. Leading blogger Chris Brogan  (@chrisbrogan on twitter) famously wrote that it took him eight years to get his first 100 subscribers! Learning how to connect with readers, and attract a following may be the hardest part.  But sooner or later, if you stick with it, you’ll find where your [...]

Fighting Online Content Theft: Peer Pressure Can Work!

It seems that almost every day we learn of some new instance of online content theft. Some of it is unintentional – because people just don’t understand what content is OK to re-use and when. Sometimes it’s blatant thievery – people taking content that they haven’t created for their own purposes. So when it comes [...]

Canadian Copyright Law: Fair Dealing

We’ve written previously in this blog about the concept of fair use under U.S. copyright law, and its impact on publishers and folks reusing their content.  In addition, our licensing service helps publishers provide tips to readers for evaluating when permission should be obtained to reuse content. While copyright law in the United States has [...]

The Most Common Form of Copyright Infringement, and How to Avoid It

Sometimes, conversations about copyright veer into the realm of ethics and lawfulness, but today we’d like to have a more pragmatic conversation. If you publish online, you want people to spread your content. It’s also likely that being known as the source of what you write has a direct impact on your income. The messy [...]

Copyright and Fair Use: Keep Your Blog from Crossing The Line

Bloggers have an innate desire to share, exchange, and discuss ideas. The free exchange of ideas makes the blogosphere a uniquely engaging and inspiring place. One minute you’re reading a blog post that engages you, and before you know it, you are inspired to write your next post! What then informs our ability to share and [...]

Blog Copyright: 8 Do’s and Don’ts to Protect Your Content

Have you worked hard to create original content on a website or blog, only to have someone “borrow” it? That’s not right! Take time to review your blog: copyright is automatic, but it’s up to you to make sure that the right measures are in place to make compliance seamless. Here are 8 Do’s and Don’ts, to [...]

What is Online Content Worth?

This is one of the most complex issues of our day. Everyone — from traditional newspaper, periodical, and book publishers to blogger moms — is trying to figure it out. So, too, is every individual consumer of digital content. There are those who argue that if it’s online, it should be free. But why should [...]

5 Critical Copyright Guidelines for Bloggers

What are the basic rules of copyright protection for bloggers? First and foremost:  When it comes to writing, sharing, and republishing other people’s content, the golden rule applies:  Do unto others, as you would have them do unto you. Always get permission before posting another person’s copyrighted work on your blog or website! This is [...]

How Writers, Freelancers, and Publishers Can Profit From the Meltwater Decision

If you follow copyright matters, you know that Meltwater was recently handed a legal blow. In Part 1 of this series we summarized why the court ruled that Meltwater’s online republication of excerpts is copyright infringement, and not fair use. Meltwater was systematically republishing excerpts of another writer’s original expression, without getting permission or paying compensation. We’d [...]

© Copyright. . Info iCopyright. All Rights Reserved Terms | Site Map