Why it matters
It keeps a page out of search results, which is right for private or thin pages. Combined with a robots.txt block, crawlers cannot even see the directive, so use one or the other deliberately.
Example
<meta name="robots" content="noindex, nofollow">Where you will see it
The free Robots.txt tester looks at this on a live page and explains what it finds in plain English: see which crawlers can fetch a URL, spot syntax mistakes and blocked CSS or JavaScript.
Related terms
- X-Robots-Tag: An HTTP response header that carries the same directives as the robots meta tag, such as noindex.
- robots.txt: A plain-text file at the root of a site that tells crawlers which paths they may crawl.
- Indexing: A search engine storing a page in its index so that it can appear in results.
Go deeper
- robots.txt explained: What robots.txt does and does not do, how Allow and Disallow rules are matched, the mistakes that hide sites from search, and how to handle AI crawlers.