Fix Duplicate Site Content: Avoid Google SERP Drop and Duplicate Content Penalty

• Tweet

• Tweet

Presence of duplicate content in Google index may result in drastic drop of Google Website Positioning Rank. Duplicate website content could directly impact keyword positioning in search results and also sometimes the duplicate content penalty may be severe .
Good Search Engine Result Position (SERP) is very important in determining the success of any website or blog. Duplicate content can hurt the hard earned high SERP rank and the hours of SEO activities may not yield any results. Removal of duplicate content will ensure that your site more crawlable. Duplicate content may result in Google penalizing the site.
Related:

• Top 6 Mistakes You Should Avoid in SEO
• Google Content Farm Algorithm Benefits and Risks
• 5 Important steps to avoid Search Engine Blacklisting.

What is duplicate content and how to check duplicate content?

As per Google “Duplicate content generally refers to substantive blocks of content within or across domains that either completely match other content or are appreciably similar“. All major search engine especially Google will penalize the sites which deliberately duplicates content across domains in an attempt to manipulate search engine rankings or win more traffic.


But sometimes content duplication can occur because of multiple distinct URLs. For example some URLs with query string may return the same original URL with the original content. This also results in duplicate content. If you are using WordPress as your blogging platform then the query string “?replytocom” will result in duplicate content. Other common query string that is creating issue is “?ftr=vidpgurl”. We will see how to tackle all these issues in this article.
Similarly printer-only versions of web pages can also result in duplicate content.
Related:

• How to Avoid Duplicate Content on WordPress Blog
• Online Plagiarism : How to Find if Someone is Copying Your Blogs Content?

Check duplicate content and Fix Them

Primary step to avoid duplicate content is to do a manual verification of your site or blog content. If there is any duplicate content with duplicate meta information or title is present, remove them immediately. These types of contents comes under the category of content is deliberately duplicated across domains in an attempt to manipulate search engine rankings.

Control duplicate contents using “canonicalization”

If your blog or site has same content available via different URLs you can use Canonicalization to indicate the original content to be indexed by search engines. This is achieved by adding a element with the attribute rel=”canonical” to the section of the non-canonical pages.
For example if you have two pages with identical contents as below,
http://www.mysite.com/originalcontent.html
and
http://www.mysite.com/originalcontent.html?cat=original
Both returns the same content. Here you need to add the rel=”canonical” to the section of the second page. For example the http://www.mysite.com/originalcontent.html?cat=original page should have the following in the section.

Control duplicate content using 301 redirects in .htaccess file

Read: .htaccess Tutorial -A guide and .htaccess tricks and tips
You can configure 301 redirects (“RedirectPermanent”) in your .htaccess file to avoid duplicate contents. This configuration will redirect users, and search engine spiders like Googlebot, MSNbot etc.
If your web server is Apache then you can edit the .htaccess file  to setup 301 redirects. This is particularly useful if query string is creating duplicate content issue. You can strip of the query string and redirect the traffic to original URL. You just need to enter the following in the .htaccess file.

1
2
3

RewriteEngine On
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^?]*)\?
RewriteRule \.html$ http://www.mysite.com/%1? [R=301,L]

Remember to replace ” http://www.mysite.com ” with your site.
But remeber that the above will strip all the query strings from all the incoming requests. To avoid this you can specify specific query string as given below. The below exaple will strip “ftr=vidpgurl” query string if present.

1
2
3
4
5

RewriteEngine On
 
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^?]*)\?
RewriteCond %{query_string} ^ftr=vidpgurl
RewriteRule \.html$ http://www.mysite.com/%1? [R=301,L]

Use Parameter handling Tool of Google Webmaster Central

You can use the Google Webmaster Central’s  Parameter handling tool to configure the query string parameters. This is explained in the below article.
Handle ?ftr=vidpgurl and ?replytocom URLs: Avoid Duplicate Content Penalty