If search engines detect duplicate content on your web site, it can
have an adverse effect on your rankings. When a search engine finds
the same content on more than one page, they tend to consider it a
form of spam. In this post, I'll review some of the ways duplicate
content may appear on your site and a few strategies to avoid being
penalized by search engines.
In general, the rule of thumb to follow when creating
content for your web site is this:
The same content should never be available from more
than one URL.
If your web site is violating the above rule, then the
site's search engine rankings may be suffering as a result.
Any blatant violations of this rule should be removed from
your web site immediately. There are, however, a few
scenarios that can result in duplicate content violations
that are not intentional or even obvious. A few examples are
presented below.
Multiple Domains
If you own more than one domain name and point these domains
to the same page, a duplicate content problem can result.
It's fine to have multiple domains, but it's important to
use the correct type of
redirect to point any additional domains you have to the
primary domain. If a search engine finds the correct
redirect, it will follow it, land on the primary domain's
page, and no duplicate content penalty will result. The
proper redirect to use in these cases is called a
301 Redirect, which indicates that the requested page
has been assigned a new permanent URL. By contrast, a
302 Redirect should only be used for pages that have
been temporarily removed and will be restored in the
future. Using the wrong type of redirect for your multiple
domains will cause duplicate content penalties and possibly
result in your web site being completely removed for a
search engine's index.
Dynamic URLs
Most search engines do not have any problem indexing dynamic
URLs, which are typically associated with sites that are
database driven. However, sometimes webmasters make the
mistake of setting up the site so that the same content
might be presented for several different dynamically
generated URLs. This might occur if the content is delivered
based on a selection of search parameters, for example. It's
important to avoid these types of situations, because search
engines consider each of these dynamic URLs as unique. If
duplicate content is found, then a penalty might be applied
to the web site.
Sorting Content
Some web pages present content that can be sorted in various
ways for the convenience of the user. For example, you might
have a site that sells floor tiles and allow the various
tile types to be sorted by texture, color, style, etc. The
sorting feature is an excellent way to help your customer
decide on the right floor tile, but it can cause duplicate
content issues for your web site. Often, the various sort
combinations are presented using different URLs, even
though the core content of the pages are the same. Search
engines do not distinguish the various sort pages as unique
content, and consider these multiple URLs to be duplicate
content. One good strategy to avoid this problem is to use
your
robots.txt file to allow only one of your sort options
to be accessible to search engine spiders. Make use of
web analytics to identify the page that has the highest
customer conversion rate, and present only that page to
search engines.
What if my web site content appears on other sites?
It sometimes happens that the content you present on your
web site is posted on another site, like a blog site about
the topic. For example, you might write an article that is
published on several other web sites related to the
article's topic. Do these reproductions of your content
result in a duplicate content penalty for your web site?
Thankfully, the answer to that question is no. Search
engines look at the
hostname for these pages, and will typically only
penalize a web site if the duplicate content pages are
served up by the same host. In addition, search engines make
efforts to locate the originating source of the content and
present its URL in search results.
Conclusion
Duplicate content issues can be problematic for webmasters
and possibly result in penalties that can lower search
engine rankings. The good news is that these penalties are
easy to avoid with a little education and understanding of
how they can arise in the first place.
