What is duplicate content? Definition, causes, and how to fix it

Most of it is accidental. A page that answers over http and https, with and without www, is four URLs to a crawler carrying one text. Google says plainly that "some duplicate content on a site is normal and it's not a violation of Google's spam policies." The trouble starts when you leave Google to decide which copy counts, or when the copies were made in order to rank.

Is there a duplicate content penalty?

Not for ordinary duplication. Google settled this in 2008: "There's no such thing as a 'duplicate content penalty.' At least, not in the way most people mean when they say that." Duplication on a site "is not grounds for action on that site unless it appears that the intent of the duplicate content is to be deceptive and manipulate search engine results."

Deliberate copying is different, and today's spam policies name it: scraping, which includes "republishing content from other sites without adding any original content or value", and scaled content abuse, which includes generating many pages "without adding value for users." Sites that break these policies "may rank lower in results or not appear in results at all."

What ordinary duplication costs is quieter:

• The wrong URL can rank. Google picks the version it considers best; the canonical you set is "a hint, not a rule."
• Signals can split. Copies Google fails to detect cannot be consolidated, which "may dilute the strength of that content's ranking signals."
• Crawling is wasted. Time Googlebot spends on duplicates is time it does not spend getting "to the rest of your content."
• Copies rarely beat the original. Google's example is an affiliate site that republishes Amazon's listings and struggles to rank, not because of a penalty, but because it offers "the exact same listing" as a far stronger site.

How Google handles duplicates

Google calls the process canonicalization. When pages share the same or very similar main content, "it clusters them together" and marks the one that is "objectively the most complete and useful for search users" as the canonical. That page is crawled most often and is the one search results usually show. The main signals behind the choice are HTTPS, redirects, sitemap inclusion and rel="canonical" annotations.

Translation is not duplication: pages in different languages count as duplicates "only if the primary content is in the same language." Same-language regional versions, such as US and UK pages with the same text, are duplicates, and Google asks for canonical tags and hreflang together on them.

Common causes and the usual fix

Cause	Example	Usual fix
Protocol and host variants	Every page reachable on http and https, with and without www	301 redirect to one version
URL parameters	?sort=price, ?color=green, ?utm_source=newsletter	Canonical tag pointing at the clean URL
CMS archives	Tag, category and date archives listing the same excerpts	Keep the archives that work as hub pages; noindex or remove thin ones
Staging and demo copies	A test copy of the site left open to crawlers	Password-protect it
Syndication	Your article republished on a partner's site	The partner blocks indexing of its copy
Templated pages	The same paragraphs with a city or product name swapped	Write each page for its own query, or merge them
Same-language regional copies	US and UK pages with identical text	Canonical tags plus hreflang
Headless setups	The same post reachable on the CMS's own domain and on the front end	Keep the CMS domain out of the index; canonical to the public URL

WordPress core prints a canonical tag only on singular views such as posts and pages, so archives depend on your theme or SEO plugin. On a headless WordPress site, the WordPress install often still answers on its own domain, and that copy should stay out of the index. For syndication, Google advises against a cross-site canonical and calls blocking indexing on the partner's side "the most effective solution."

Templated pages are the cause most likely to become a policy problem. A generator that drops a new noun into the same text produces near-duplicates by design, which is where useful programmatic SEO ends and scaled content begins. The post on doing programmatic SEO on WordPress covers making each generated page differ in substance.

How to find duplicate content

Three checks cover most sites:

1. Search Console's Page indexing report. It lists the URLs Google treats as duplicates under statuses such as "Duplicate without user-selected canonical" and "Duplicate, Google chose different canonical than user". The URL Inspection tool shows which canonical Google chose for any URL.
2. A site crawl. An SEO crawler such as Screaming Frog groups pages with identical titles or near-identical text, and URLs that differ only by a parameter.
3. A quoted search. Search a distinctive sentence from the page in quotation marks. Other URLs that come back, on your site or elsewhere, are copies.

How to fix it

Google ranks its methods by strength: redirects and rel="canonical" annotations are each "a strong signal", sitemap inclusion "a weak signal". They stack, so two together raise the chance that your preferred URL is shown.

• 301 redirect when the duplicate should stop existing: the http version, the www variant, the old URL after a merge.
• Canonical tag when the duplicate has to stay reachable: filtered and sorted views, links with tracking parameters. Give the main page a canonical that points at itself too.
• Rewrite or merge when the duplicates are near-identical posts you wrote yourself. Two articles on one question compete with each other; content pruning covers when to merge them and redirect the loser.
• noindex only for a copy that should not appear in search at all, such as a partner's syndicated version. Google advises against noindex for choosing between duplicates on your own site, "because it will completely block the page from Search."

Link internally to the canonical URL, since consistent internal linking is one more signal of the version you prefer. Do not hide duplicates with robots.txt: Google "may still index URLs that are disallowed in robots.txt without their content." And expect a delay, because Google "might hold pages in a duplicate cluster for up to two weeks."

How Structura handles this

Structura writes one article per keyword and never writes the same keyword twice. If a scheduled post would go live at an address your site already publishes under, the run stops and you get a notification; only posts you request yourself with Generate Now skip that check. Every post is a fresh draft, never a variation spun from another article.

That is also why Structura does not generate template pages. Page generators merge a spreadsheet into one layout, which suits real structured data such as prices but repeats the same prose on every page. The programmatic SEO page compares the two: Structura covers a list of queries with a written article for each, not a thousand copies of one template.

It does not audit or fix duplication that already exists on your site, and it does not edit posts you wrote yourself. Canonical tags and redirects stay with WordPress and your SEO plugin; on a headless site, rendering the canonical tag is your front end's job.

FAQ