SEO glossary
What is Indexability?
Learn what indexability means in SEO—whether a URL is eligible and permitted to enter the search index after crawl—and how noindex, canonicals, and quality gates differ from crawlability.
Definition
Indexability is the set of technical and policy conditions under which a crawled URL may be stored in a search engine's index, determined by publisher directives, HTTP signals, canonical rules, and engine quality thresholds.
Indexability: permission to enter the index
Indexability answers a precise question after crawl succeeds: is this URL allowed and suitable to become an indexed page? It is the eligibility layer between fetch and corpus membership—governed by your explicit directives, implicit canonical signals, and search engine quality policies.
Teams that fix crawlability but ignore indexability still see empty Search Console performance charts. The bot arrived; the indexing pipeline said no.
Indexability vs crawlability vs indexing
| Concept | Question it answers | Primary levers |
|---|---|---|
| Crawlability | Can bots fetch this URL? | robots.txt, server errors, orphans |
| Indexability | May this URL be stored in the index? | noindex, canonical, quality |
| Indexing | Did the engine commit a record? | Pipeline outcome after evaluation |
Crawlable + Indexable → may become Indexed
Crawlable + Non-indexable → crawled, excluded (by design or quality)
Non-crawlable → indexability untested; URL may stale in index
Indexability is prospective—a URL's configured and structural right to enter the index. Indexed is retrospective—the engine already said yes.
Technical indexability signals
Publishers control most hard gates:
| Signal | Location | Indexability effect |
|---|---|---|
noindex | <meta name="robots"> | Block index storage after crawl |
noindex | X-Robots-Tag HTTP header | Same; works for PDFs, images |
rel=canonical to other URL | <link rel="canonical"> | This URL may not earn standalone index entry |
| 301/308 to canonical target | Server redirect | Index representation follows target |
nofollow alone | meta or link | Does not block page indexability |
robots.txt Disallow | robots.txt | Blocks crawl → indirect non-indexing |
<!-- Indexable by default when absent -->
<head>
<title>Indexable Product Page</title>
<link rel="canonical" href="https://example.com/product/widget">
</head>
<!-- Explicitly non-indexable -->
<meta name="robots" content="noindex, follow">
Validate both HTML and HTTP header layers—CDNs sometimes inject conflicting X-Robots-Tag values.
Soft indexability: canonical and duplication
Even without noindex, URLs may be effectively non-indexable as standalone documents:
- Faceted
?sort=priceURLs canonicalized to/category/shoes - Paginated
?page=2withrel=canonicalto page 1 (when appropriate) - HTTPS and HTTP duplicates where one variant wins
- Cross-domain syndication with canonical to publisher
<link rel="canonical" href="https://publisher.com/original-article">
The syndicated copy is crawlable but not indexable as its own result—by design.
Quality-based indexability (engine-side)
Google documents that crawling does not guarantee indexing. Engines apply soft indexability heuristics:
| Pattern | Typical indexing response |
|---|---|
| Near-duplicate thin pages | Crawled — currently not indexed |
| Empty templates / soft 404 | Excluded or dropped |
| User-generated spam profiles | Selective exclusion |
| Doorway geo pages | Aggressive filtering |
You cannot toggle these with meta tags alone. Fix content, consolidation, and site architecture.
Indexability audit workflow
- Export URL inventory — CMS, sitemap, analytics landing pages.
- Crawl with directive capture — robots meta, headers, canonicals, status codes.
- Segment by template — product, blog, faceted, paginated.
- Compare to GSC exclusions — "Excluded by noindex," "Duplicate," etc.
- Fix at template root — one plugin misconfiguration affects thousands of URLs.
Template: /tags/*
URLs: 4,200
noindex: 0
canonical to self: 12%
canonical to /blog/: 88% ← intentional consolidation
indexability verdict: only self-canonical tag pages indexable
Indexability for non-HTML resources
PDFs, images, and video landing pages carry indexability too:
Content-Type: application/pdf
X-Robots-Tag: noindex
Sitemaps list URLs; they do not override noindex. Video watch pages need clear indexability on the landing URL you want in results.
Indexability during migrations
Redirects affect indexability of both source and target:
| Migration pattern | Source indexability | Target indexability |
|---|---|---|
| 301 to equivalent page | Transfers; source deindexes over time | Should be indexable |
| 302 temporary | Source may persist in index | Target may not consolidate |
| Missing redirect | Source 404 → deindex | Target may lack signals |
Launch checklists must verify target URLs are indexable and sources redirect cleanly.
Indexability vs "index, follow"
Default behavior when no restrictive directive exists:
<meta name="robots" content="index, follow">
Explicit index is rarely needed—it is the default. Teams add meta robots when they need exceptions (noindex, nosnippet, noarchive).
Common indexability mistakes
- Staging
noindexon production — most common catastrophic error. - Canonical to homepage on all pages — mass effective deindexing of interiors.
- Conflicting canonical and hreflang — wrong regional URL indexed.
- Parameter URLs without canonical — indexable duplicates explode.
- Assuming disallow deindexes — use noindex on URLs that must drop from results.
Indexability checklist for new templates
Before shipping:
- Confirm no
noindexin production builds - Set self-referencing canonical on indexable templates
- Block indexability on faceted/sort variants via canonical or noindex
- Match HTTP header and HTML robots directives
- Test with URL Inspection on beta URLs behind auth bypass for bots if needed
How Crawlox helps
Crawlox reports indexability signals—noindex, X-Robots-Tag, canonical targets, and status codes—across your entire URL set, not just spot checks. Template-level dashboards reveal when a plugin silently marks thousands of product pages non-indexable, or when faceted URLs lack consolidation, so you fix eligibility before indexing pipelines reject good content.
Related terms
Crawlability
Whether bots can fetch a URL—the prerequisite before indexability matters.
Indexing
The process that evaluates indexability signals and writes to the index.
Deindexing
When indexability changes to exclusion and URLs leave the index.
Indexed Page
A URL that passed indexability checks and was stored in the index.
Frequently asked questions
Is indexability the same as crawlability?
No. Crawlability is whether a bot can download the URL. Indexability is whether that URL should be stored in the index after fetch. A page can be crawlable but non-indexable (noindex), or theoretically indexable but unreachable if crawl is blocked.
What makes a page non-indexable?
Common blockers include noindex directives, canonical pointing elsewhere, authentication walls, low-quality or duplicate content thresholds, and certain robots meta combinations. robots.txt disallow prevents crawl, which indirectly blocks indexing.
Can a page be indexable but not indexed?
Yes. Indexability is eligibility; indexing is the outcome. Quality filters, crawl delays, and queue depth can leave indexable URLs waiting in 'discovered — currently not indexed' states.
How do I audit indexability at scale?
Crawl with a tool that reports meta robots, X-Robots-Tag, canonicals, and status codes per URL. Cross-check with Search Console Page indexing exclusions. Fix template-level issues, not just homepage samples.
Does nofollow affect indexability?
nofollow governs link treatment, not whether the page itself may be indexed. A nofollow link does not make the destination non-indexable. Use noindex on the page you want excluded from results.
References
Explore authoritative guidance and frameworks related to indexability.
Explore every glossary definition
Return to the glossary to search by term, alias, starting letter, or category.