What counts as duplicate content?
Duplicate content means the same, or nearly the same, main text available at more than one address. On a small site it almost always happens by accident. The Crumb and Crust menu can be reached at /menu.html, /menu, and /menu.html?sort=price, and each looks like a different URL to a crawler even though the server sends the same file. Other common causes are http:// and https:// versions, www. and bare domains, printable versions, and two pages a person wrote separately that say the same thing.
Copying a paragraph of opening hours onto several pages is fine. Duplication becomes a search question when the main content of whole pages matches, because the search engine then has several candidates for the same answer and shows one of them.
Is there a penalty for duplicate content?
For ordinary duplicates, no. Google has said many times that it does not punish a site because the same page is reachable at two addresses. It groups the copies, picks one as the canonical, the main copy, and shows that one in results. The others are left out of results, and signals such as links pointing at the copies are mostly credited to the canonical.
The real cost is loss of control. If you do not say which address you prefer, the search engine picks for you, and it may choose /menu.html?sort=price or the http:// version. Crawlers also spend time fetching copies. Copying other people's content at scale to fill a site is a different matter: that falls under Google's spam policies, and can lead to a site being demoted or removed.
How do you tell search engines which copy is the main one?
There are several signals, and they work best when they agree. A 301 redirect is the strongest: the old address stops serving a page at all. A <link rel="canonical"> tag in the <head> names the preferred address while keeping both addresses working for visitors, which suits sorted or filtered views. Internal links that always point at the preferred address, and a sitemap that lists only preferred addresses, back it up.
Use a redirect when the copy has no reason to exist, such as an old page or the www. version. Use a canonical tag when the copy is useful to visitors but should not appear in results, such as a sorted list. A page can name itself as canonical; that is a sensible default, because every odd variant with tracking parameters then carries the same hint.
A canonical tag is a strong hint, and Google may still choose a different copy if other signals disagree, for example if the sitemap lists one address and every link points at another.
<head>
<link rel="canonical" href="https://crumbandcrust.example/menu.html">
</head>
What is a thin page?
A thin page has almost nothing on it for a visitor: "Coming soon", a heading and a single line, a tag page listing one post, or a page that repeats the menu in other words with nothing new. Search engines often crawl pages like this and then leave them out. In Search Console they usually show up in the Pages report as "Crawled, currently not indexed".
There is no word count that makes a page good enough. A bus timetable or a price list can be short and very useful. The question is whether a person who lands on the page gets what the search promised. The lab uses 30 words of main content as its rule of thumb so that it can give a clear result, and says so in the Index view; real search engines judge usefulness in their own ways.
You have three good options for a thin page. Fill it with something real, merge it into a page that already covers the topic and redirect the old address, or leave it unpublished until it is ready. A noindex tag is a fourth option for pages visitors need but searchers do not, such as an order confirmation.
What is a soft 404, and why do search engines dislike it?
When someone asks for a page that does not exist, the server should answer with the status code 404 Not Found. A soft 404 is a missing page that answers 200 OK anyway, often by showing the home page or a page that says "Sorry, not found" with a success code. Visitors see something, so the problem is easy to miss, but crawlers get told that every mistyped address is a real page, all with the same content.
Search engines detect these and treat them as errors. Search Console lists them in the Pages report as "Soft 404". The fix is on the server: return 404 for missing pages, or 410 for ones you removed on purpose, and use a 301 when the content has moved somewhere else. In the lab, missing_pages = home in site.conf creates this problem and missing_pages = 404 fixes it.
How does the lab choose between duplicates, and how is that different from Google?
The lab compares the main text of every indexed page. When two pages share 90 per cent or more of their words and neither names a canonical, it keeps one: the one in the sitemap, then the one with more internal links, then the one without a query string, then the shortest address. It is a fixed rule, so you can predict the result.
Google uses many more signals, including redirects, canonical tags, sitemaps, internal links, HTTPS, and hreflang for language versions, and it can change its mind as the site changes. It may also group pages the lab would call different, or keep two that the lab would merge. Use the lab to understand the idea, and use the URL Inspection tool in Search Console to see which canonical Google actually chose for a real address.
Where do you find these problems in Search Console?
Open the Pages report under Indexing. The "Why pages aren't indexed" table lists reasons, each with a count and example URLs. The ones linked to this topic are:
- Duplicate without user-selected canonical: Google found copies and picked one itself.
- Duplicate, Google chose different canonical than user: your canonical tag was overruled, usually because other signals disagree.
- Alternate page with proper canonical tag: working as intended, the copy points at the main page.
- Crawled, currently not indexed: often thin or low-value pages, sometimes just new ones waiting.
- Soft 404: missing pages that answer 200, or pages that look empty.
What do AI tools get wrong about duplicates and thin pages?
Many AI answers still warn of a "duplicate content penalty" and suggest rewriting every product description so no two sentences match. That wastes time on a small site, where a canonical tag or a redirect is the right fix. Others add a canonical tag to every page pointing at the home page, which tells search engines that every page is a copy of the home page; Google ignores most such tags, and the ones it follows remove good pages from results.
The bigger trap runs the other way. It is now easy to ask a tool for a hundred pages such as "Bakery in each nearby town" with the town name swapped. Google's scaled content abuse policy covers exactly this: many pages made mainly to rank, with little original value, however they were produced. A few honest pages about what you actually offer are worth more than a pile of near copies. Be equally wary of advice that more words always rank higher; padding a thin page with filler does not make it useful.
Field notes
- Duplicate content is the same or nearly the same text at more than one address.
- Most duplicates bring no penalty, and search engines pick one copy to show.
- Point to the main copy with redirects, canonical tags and consistent internal links.
- A thin page adds little for a reader, so improve it, merge it or remove it.
Easy to get wrong
A canonical tag names the main copy of this page. Pointing every page at the home page claims they are all copies of it, so search engines either ignore the tags or drop real pages.
Near-empty pages are usually crawled and left out, and visitors who find them leave. A page is worth publishing once it answers a question.
Answering every unknown address with the home page and a 200 status creates soft 404s. A real 404 tells crawlers the page is not there, and a custom 404 page can still help visitors find their way.
Questions people ask
Should I use noindex or a canonical tag for a sorted version of a page?
A canonical tag. It keeps the sorted view working for visitors and points search engines at the main page, so links to either address count for the main one. A noindex on a page that also has a canonical tag sends mixed signals.
Is it a problem that my opening hours appear on every page?
No. Repeated footers, addresses and opening hours are normal. Duplication becomes a question when the main content of whole pages is the same.
How long should a page be so it is not thin?
There is no length rule. Write what a visitor needs to know about that one thing, and stop. A short page that answers the question is better than a long one padded with filler.
Can I copy a supplier's product descriptions?
You can, and nothing bad happens to your site for it, but the supplier's page and every other shop using the same text already exist. Adding your own notes, prices, photos and stock details gives a searcher a reason to pick your page.