Why it matters
On a very large site, crawlers may not reach every URL quickly. Removing duplicates, errors and endless URL variations helps them spend time on pages that matter.
Where you will see it
The free XML sitemap validator looks at this on a live page and explains what it finds in plain English: find the sitemap, validate the XML and limits, and spot-check that listed URLs really load.
Related terms
- XML sitemap: A file that lists the URLs you want search engines to know about, optionally with last-modified dates.
- Redirect chain: Several redirects in a row before the final page, for example http to https to www.
- Duplicate content: The same or very similar content available at more than one URL. It is not a penalty in itself, but search engines have to pick one version.
Go deeper
- XML sitemaps: How an XML sitemap works, its format and limits, which URLs belong in it, the common mistakes that make search engines ignore it, and how to check yours.