Faceted Navigation: Filters Without Index Bloat
Faceted navigation is a shop’s filter and sort system: colour, size, price, brand, rating. For users it is essential. For search engines it is the single most dangerous source of index bloat, because it can turn one category into millions of crawlable URLs. This guide shows how to keep filters open for users and controlled for bots.
1. The combinatorial problem
At its core this is maths. A category with four filter groups – 6 colours, 5 sizes, 10 brands, 4 price bands – produces not 25 combinations but their product: 6 × 5 × 10 × 4 = 1,200 variants. Combined with multi-select, sorting and pagination, it climbs into the millions.
Each of these variants is its own URL with filter parameters:
/shoes?color=red&size=42&sort=price. Left to itself, Googlebot crawls these URLs until
your crawl budget for the pages that really matter – the product
detail pages – is used up.
Core symptom: the Search Console “Pages” report shows a six- or seven-figure count of crawled, not indexed URLs even though the shop has only a few thousand products. Almost always faceted navigation is the cause.
2. A facet is not pagination
The two are often confused but demand opposite handling:
| Pagination | Facet | |
|---|---|---|
| What it does | Splits the same list | Produces a filtered subset |
| Content | New items per page | A selection from the same set |
| Default handling | Indexable, self-canonical | Usually blocked or canonicalised |
Memorable version: pagination shows more, a facet shows less. That is why follow-on pages are generally kept indexable while facet URLs are deliberately kept out of the index – with a few targeted exceptions.
3. Which facets should be indexed
The one strategic question is: does anyone actually search for this? A filter deserves an indexable page only if there is genuine search demand for that exact combination.
| Facet | Example search | Decision |
|---|---|---|
| Brand | “Nike running shoes” | Indexable – clear demand |
| Category + attribute | “waterproof hiking boots” | Indexable – its own landing page |
| Colour | “red Nike running shoes” | Borderline – only with provable demand |
| Price band | “shoes between €80 and €90” | Do not index |
| Sorting | – | Never index |
| Multi-select | “red + blue + size 42 + 43” | Never index |
The indexable combinations should ideally not be throwaway parameter URLs but real, static category landing pages with a clean URL, their own title, their own description and their own intro copy – pages that serve search intent rather than merely filtering products.
4. The control tools compared
For all non-indexable facets, four mechanisms are available. They do different things, and the choice decides between success and waste:
| Tool | Effect | Saves crawl budget? | Use when |
|---|---|---|---|
| Canonical | Consolidates signals onto the unfiltered category | No – the URL is still crawled | Few filters, low combinatorics |
| noindex | Keeps the URL out of the index | No – must be crawled to be read | When signals should not be consolidated |
| robots.txt Disallow | Prevents crawling entirely | Yes – the strongest tool | Large combinatorics, clear parameter patterns |
| Facet links via JavaScript | Creates no crawlable URL at all | Yes – preventively | Filters that should never enter the index |
The key distinction: only robots.txt (and avoiding crawlable links) saves
crawl budget. noindex and canonical have to be read first, i.e. crawled
– they keep the URL out of the index but not out of the queue. With real index
bloat, blocking the crawl is the only effective answer.
5. The robots.txt trap
A URL blocked in robots.txt is not crawled – and so Google never reads a
noindex or canonical set on it. The two are therefore mutually exclusive: if you
want to combine them, do not block in robots.txt; set noindex and leave the page
crawlable.
Second trap: a blocked but externally linked facet URL can still appear in results as “indexed, though
blocked by robots.txt” – with no snippet. For sensitive URLs, therefore, noindex
(crawlable) is safer than a Disallow.
6. A workable overall strategy
- Turn demand facets into real landing pages. A few clearly searched combinations get static, indexable pages with their own URL and copy.
- Exclude all other filters from crawling. Via a clear parameter pattern
(
Disallow: /*?*color=) or via non-crawlable filter links. - Always canonicalise or exclude sorting and view options. They never create new content.
- Point internal links only at indexable targets. Never link from content to filter URLs you simultaneously keep out of the index – that contradicts itself.
- Keep filtered states clean. Do not index empty results; “0 hits” pages are thin content.
7. Common mistakes
- Leaving all facets freely crawlable. Leads straight to index bloat and wasted crawl budget.
- Trying to “enforce” noindex via robots.txt. The blocked page is never read, the noindex never takes effect.
- Indexing every colour variant. Creates hundreds of near-identical pages competing for the same terms.
- Putting facet URLs in the sitemap or internal links. Signals an indexing wish for pages you are actually suppressing.
- Treating sorting as its own page.
?sort=priceis a view of the same list, not new content. - Serving empty filter results. Pages with no hits belong on
noindexor should not exist as a link in the first place.
Pro tip: open a filtered URL with Rank-O-Saur and check canonical, robots directive and
indexability in one glance. You will spot at once whether a facet is accidentally indexable, or whether
a robots.txt block and a set noindex are cancelling each other out.
8. Checklist
- Searched combinations implemented as static, indexable landing pages.
- All other filters excluded from crawling (robots.txt pattern or non-crawlable links).
- Sorting and view options never indexed.
- No contradiction between a robots.txt block and a noindex/canonical set inside it.
- No facet URLs in the sitemap.
- Internal links point only at indexable targets.
- Empty filter results on noindex or without a link.
- Search Console monitored for “crawled, not indexed” – the number drops after implementation.
9. Frequently Asked Questions
What is the difference between a facet and pagination?
Pagination splits the same list across follow-on pages, each with new items. A facet filters the list into a subset. That is why follow-on pages usually stay indexable while facet URLs are generally kept out of the index.
Should I handle filter URLs with robots.txt or noindex?
With large combinatorics, only robots.txt saves real crawl budget, because the URL is not crawled at all. noindex keeps the page out of the index but has to be read for that. The two together do not work: a blocked page is never crawled, so the noindex is never seen.
Which facets should get an indexable page?
Only those with genuine search demand – typically brand or a clear attribute like “waterproof”. These should be built as static landing pages with their own URL and copy, not as a parameter URL. Price bands, sorting and multi-selects never belong in the index.
Why do I have millions of crawled, not indexed URLs?
Almost always because of freely crawlable facets. Each filter combination creates its own URL, and the product of the filter values quickly grows into the millions. The fix is to consistently exclude the non-demanded combinations from crawling.
Can I just set facet links to nofollow?
That helps only partially. Google can still discover and crawl such URLs from other sources. It is more effective to create no crawlable link for the URL at all, or to block it in robots.txt. nofollow alone does not reliably solve the bloat problem.
Should I put filtered pages in the sitemap?
Only the few you deliberately want to index – the real landing pages. All other facet URLs do not belong in the sitemap, because that would be an indexing signal for pages you are suppressing.
Read next
