Duplicate Content: What Causes It, What Google Does, and How to Fix It
"Will I get penalised for duplicate content?" is one of the questions I hear most from small business owners, usually after a tool flagged a few hundred duplicate pages. The short answer: almost certainly not. Google has said repeatedly that duplicate content on its own isn't grounds for a penalty unless it's deceptive, like scraping other sites to manipulate rankings. The longer answer is that duplicates still cause real problems, just quieter ones.
What counts as duplicate content
Duplicate content is substantial blocks of text that are the same or very similar, either across URLs on your own site or across different sites. Two broad kinds:
- Internal duplicates: the same content reachable at more than one URL on your site.
- External duplicates: your content appearing on other sites, or content you copied from elsewhere appearing on yours.
What Google actually does
When Google finds several versions of the same content, it groups them and picks one to show, the canonical. The others are typically filtered out of results. The problems this causes:
- Google may pick the wrong version. A URL with tracking parameters might be shown instead of your clean one.
- Signals get split. If some sites link to one version and others link to another, the authority is divided until Google consolidates them. Link equity explains why that matters.
- Crawling gets wasted. On large sites, thousands of duplicate URLs can eat into the attention Google gives to your important pages.
- Thin variations underperform. Pages that differ only by a city name or a colour rarely rank well individually.
Common causes and fixes
| Cause | Example | Usual fix |
|---|---|---|
| HTTP vs HTTPS, www vs non-www | http://example.com and https://www.example.com | 301 redirect to one version |
| Trailing slash variants | /page and /page/ | Redirect to one; use it consistently |
| URL parameters | ?sort=price, ?utm_source=whatsapp | Canonical to the clean URL |
| Ecommerce filters | /bedsheets?colour=blue&size=king | Canonical to category, or index only valuable filters |
| Products in several categories | /kitchen/pan-1/ and /sale/pan-1/ | One product URL, canonical set |
| Print or AMP versions | /page/print/ | Canonical to main page |
| Paginated archives | /blog/page/2/ | Self-canonical per page; don't canonical all to page 1 |
| Staging site indexed | staging.example.com | Password protect or noindex |
Canonical tags
A canonical tag tells Google which URL you prefer:
<link rel="canonical" href="https://example.com/bedsheets/block-print-king/">
It's a strong hint, not a command. Google may ignore it if other signals disagree, for example if your internal links and sitemap point to a different version. The canonical tags guide covers setup and common mistakes.
301 redirects
When a duplicate URL has no reason to exist, redirect it permanently to the main version. This is the strongest signal and also helps users.
Noindex
For pages users need but search engines don't, like internal search results or thank-you pages, a noindex tag keeps them out of the index. Don't combine noindex with a canonical pointing elsewhere; they send mixed signals.
The ecommerce problem: manufacturer descriptions
Shopify and WooCommerce stores often paste the supplier's product description. So do dozens of other stores selling the same item. Google will pick one version to rank, and it's usually the site with more authority, often a large marketplace.
The fix is unglamorous: write your own descriptions for products that matter. Add details only you know, like how the fabric feels after washing, real photos, sizing advice from returns, and answers to questions customers have asked. Start with your best sellers and highest-margin items rather than trying to rewrite 3,000 SKUs at once.
Location pages
Service businesses love creating "Plumber in Jaipur", "Plumber in Ajmer", "Plumber in Kota" pages with identical text except the city name. These are near-duplicates and usually perform badly. If you serve multiple areas, each page needs something specific: local projects, area-specific pricing or timings, real staff, local testimonials. If you can't say anything different about a location, one well-written service-area page is often better.
Syndicated and scraped content
If you republish articles on Medium, LinkedIn or a partner site, ask for a canonical tag pointing to your original, or at least a clear link back. Even then, the larger site may outrank you, so many publishers wait until Google has indexed the original before syndicating.
If a scraper copies your content, it usually won't outrank you. If it does and it's hurting you, you can file a copyright removal request with Google.
Finding duplicates
- Search Console: the Pages report shows URLs marked "Duplicate without user-selected canonical" or "Duplicate, Google chose different canonical than user".
- Site crawlers: Screaming Frog, Sitebulb and similar tools flag duplicate titles, descriptions and content.
- Exact phrase search: paste a distinctive sentence from your page into Google in quotes to see where else it appears.
- Duplicate titles are a quick proxy; check them with a crawler or the SERP preview tool when rewriting.
What not to worry about
Quoting a source, legal disclaimers in footers, a short boilerplate "about us" blurb on every page, and the same shipping policy on product pages are all normal. Google expects this.
Duplicates are part of broader site hygiene, which the technical SEO checklist covers. Open Search Console's Pages report today, look for the "Google chose different canonical" status, and check whether Google's choice or yours is right.