An HTML meta tag or HTTP header that tells search engines not to include a specific page in their index, keeping it out of search results entirely.
The noindex directive is the most direct way to prevent a page from appearing in search engine results. It can be delivered two ways: as a `<meta>` tag in the HTML `<head>` of the page, or as an `X-Robots-Tag` HTTP response header (useful for non-HTML resources like PDFs and images).
A noindexed page can still be crawled and followed, it just does not appear in the index. This is distinct from `robots.txt` `Disallow`, which prevents crawling entirely. A page blocked in robots.txt may still end up in the index (without content) if other sites link to it. A noindexed page that is crawlable is guaranteed to stay out of the index.
Noindex is the correct tool for pages you do not want indexed but still want to function: login pages, internal search results, thank-you pages, staging environments, duplicate variants, thin category pages, user profile pages with no public value. Using robots.txt for these instead is a common mistake because it prevents crawling but not indexing.
The critical rule: never noindex a page you also want to block in robots.txt. If the page is disallowed in robots.txt, Google cannot crawl it to see the noindex tag, so the directive is never respected. Always allow crawling for any page you want noindexed.
Both delivery methods are equivalent for Google. Pick whichever fits your architecture:
<!-- Meta tag in HTML head -->
<!DOCTYPE html>
<html>
<head>
<meta name="robots" content="noindex, follow">
<title>Internal Search Results</title>
</head>
<body>...</body>
</html># HTTP header, works for any file type HTTP/2 200 content-type: application/pdf x-robots-tag: noindex
# Nginx, send noindex header for all PDFs
location ~* \.pdf$ {
add_header X-Robots-Tag "noindex" always;
}The `follow` value in `content="noindex, follow"` tells Google to still crawl outgoing links from the page, which is usually what you want for internal pages. Use `noindex, nofollow` only if you also want to prevent link discovery.
Every article on our blog was written by Acta AI. No edits. No ghostwriter.
Read Our BlogStart Free Trial