What Is Noindex? How the Meta Robots Tag Controls Indexing

Noindex is a robots meta directive that instructs search engines not to include a page in their search index. Implemented as <meta name="robots" content="noindex" /> in the HTML head or via the X-Robots-Tag HTTP header, it tells crawlers: you may crawl this page, but do not show it in search results. It is fundamentally different from blocking a URL in robots.txt, which prevents crawling entirely and can leave a previously indexed URL stuck in the index with no updated signals.

Noindex vs. Nofollow vs. Disallow

Noindex — page can be crawled, should not appear in results. Nofollow — do not pass link equity through links on this page (applied at page or individual link level). Robots.txt disallow — do not crawl this URL at all. Combining noindex, nofollow on the same page is valid for utility pages you want crawlers to see but not rank or follow links from — thank-you pages, internal search results, staging environments.

When to Use Noindex

Appropriate targets: thin or duplicate pages with no unique value, paginated archives where page 1 should rank alone, internal site search result pages, login and account pages, campaign landing pages not meant for organic discovery, and content behind paywalls where you still want crawl for other reasons. Tag and author archive pages on blogs often benefit from noindex when they create near-duplicate collections.

The Invisible Noindex Mistake

If a page is blocked by robots.txt, Googlebot cannot see the noindex tag on it. The page may remain indexed indefinitely with an old cached version. Fix order: remove the robots.txt block first, allow crawling, let Googlebot fetch the noindex directive, then wait for deindexing. This commonly trips up staging-to- production migrations and "soft launch" workflows.

Noindex is one control in the indexing toolkit alongside canonical tags, sitemaps, and Search Console removals. Our technical SEO guide explains when to noindex vs. canonicalize vs. delete, and how each choice affects crawl budget and link equity flow.