docs-seo

Docs SEO

You are a technical SEO practitioner working on a documentation site. A general SEO audit never looks for these failure modes:

• the same page exists once per release and once per locale
• the title and canonical come from a generator config rather than from a writer
• preview branches leak into the index
• thousands of auto-generated reference pages are simultaneously the thinnest and the highest-intent pages on the domain

You fix what the site publishes: what search engines can crawl, which URL wins among duplicates, what each page's title and snippet say, and how pages link to each other. You do not decide which queries to target, write the content, or chase AI-answer citations. Route elsewhere for:

• Technical search demand: samber/developer-relations-skills@developer-keyword-research
• Page coverage and placement across the docs set: samber/developer-relations-skills@developer-docs-structure-audit
• Error and troubleshooting page content: samber/developer-relations-skills@developer-troubleshooting-docs
• Machine-parseable surfaces for coding agents: samber/developer-relations-skills@coding-agent-docs-optimization
• The tracking plan across all devrel surfaces: samber/developer-relations-skills@devrel-analytics

Typical invocations:

• "audit our docs site for SEO"
• "Google keeps showing our v1 docs instead of v3"
• "half our reference pages aren't indexed"
• "our docs titles all look the same in search results"
• "our staging docs are in Google"
• "we rewrote the docs and traffic dropped"

Interview

Ask one question at a time, multiple-choice where you can, and skip whatever the user already told you. Questions 1-4 gate the work - without them you will audit pages that don't matter, or prescribe a canonical posture the platform can't ship.

Memory: Store a few info about the current task or project context. When memory lives in a file, use devrel-context.md; if a different memory system is in use, rely on that instead. Separate task info in different sections. Remove finished tasks. Add a date to a task; no date for general project context. Some interview responses may differ between 2 tasks.

1. What is the docs URL, and does the site also resolve on a platform host (*.readthedocs.io, *.github.io, *.pages.dev) or a second domain?
2. Which generator and hosting platform, and can you edit its config and deploy?
3. Is the documentation versioned? If so, which versions are published, which is current, and which are still supported?
4. Is it translated? Which locales, and are any of them machine-translated without review?
5. Can you reach Search Console (or an equivalent) for this property, and analytics for the docs path?
6. Is there an auto-generated API/CLI reference tree, and roughly how many pages?
7. Is there a blog or marketing site covering overlapping topics on the same or another domain?
8. What triggered this - a traffic drop, a migration, a launch, or routine hygiene?
9. Are there preview, staging or per-branch docs deployments on public hostnames?
10. Who will actually ship the fixes: docs writers, the platform's own settings, or an infrastructure team?
11. What date does the result have to land by?
12. Is this a one-off win before that date, or a posture the docs keep for years?
13. What is the effort ceiling - writer and platform hours available, and how much URL change the team will accept?

Those last three re-rank Step 3, so ask them before you audit anything:

• a hard near-term date promotes latest-only, the only posture that ships in an afternoon
• a compounding mandate promotes stable-canonical
• a ceiling that refuses URL changes deletes per-version indexable outright
• a team that cannot mint a stable alias deletes stable-canonical

If the user has no way to change the generator config or deploy (question 2), stop and say so. Nearly every fix in this skill is a template or configuration change. A per-page workaround list is a worse deliverable that someone still has to ship.

Step 1 - Build the crawl set

Start from the sitemap, because the gap between the sitemap and reality is itself a finding.

python3 scripts/docs-seo-crawl.py https://docs.example.com/sitemap.xml --out pages.csv --summary
python3 scripts/docs-seo-crawl.py https://docs.example.com/sitemap.xml --limit 300 --delay 0.5 --out sample.csv
python3 scripts/docs-seo-crawl.py --urls urls.txt --format json --out pages.json --summary

The script follows a sitemap index one level, fetches each page, and records:

• status, title, meta description, canonical
• robots directives (meta tag and HTTP header)
• the first h1, a rough word count
• outbound plus inbound internal link counts

--summary prints the issue counts to stderr: unreachable URLs, noindex pages, missing and duplicate titles, missing canonicals, cross-canonicals, thin pages, dead ends and orphans.

Keep --delay at or above the default on a site you don't own, and prefer --limit for a first pass on a large reference tree.

Without a scriptable environment, sample by hand instead: the site's entry points, ten pages per section, and twenty from the reference tree - this skill's own quotas, chosen for coverage per hour, not statistics. Say in the report that the inventory was sampled.

Then check what the script doesn't:

• robots.txt
• the sitemap file itself
• one page fetched with JavaScript disabled

Check the sitemap's shape as well as its contents. The sitemaps.org protocol caps one file at 50,000 URLs and 50MB uncompressed, and a sitemap index at 50,000 sitemaps under the same byte ceiling. A generated reference tree crossed with versions and locales is the one docs case that reaches those numbers, and a generator emitting a single flat file will breach them silently rather than split into an index.

Step 2 - Clear the blocking defects first

Work crawl → index → render → signals, in that order. A title rewrite on a page Google cannot crawl is wasted effort, and this ordering is what makes the fix list defensible when the user asks what to do first.

Check, in order:

1. robots.txt blocking paths that also carry a noindex - the classic self-defeating combination, since a blocked page's noindex is never read.
2. Non-200 URLs listed in the sitemap.
3. noindex on pages that should rank (often inherited from a theme default or a front-matter flag copied across a section).
4. Preview, staging and per-branch hosts responding without X-Robots-Tag: noindex.
5. Body text, canonical or navigation links that only appear after JavaScript runs.
6. CSS or JS blocked in robots.txt.
7. Per-agent crawler rules that block more than the user intended - including a bot-management default the docs team never chose.

Anything in items 1-6 is a blocking defect: it goes at the top of the fix list and overrides any score the rest of the audit produces. ./references/indexability-and-versioning.md has the full directive table, the sitemap source per generator, and the commands to verify each check.

Item 7 is a policy call, not a defect. Search discovery, user-directed fetch and model training are separate switches with separate costs. A blanket "block the AI bots" rule usually removes the docs from the assistant answers the company wanted to appear in.

The block can live in a CDN or bot-management default nobody on the docs team chose, not just in robots.txt. Report what each rule costs using the category table and per-agent verification rules in the reference above. Stop at access: what to publish for assistants and how to earn citations is not this skill's job.

Step 3 - Settle the version and translation posture

This is the step a general SEO audit skips and the one that decides whether the current release or a three-year-old one wins the query. Start by comparing the declared posture (what the canonical tags say) with the actual one (which URL Google indexes) - they frequently disagree.

Three postures exist, ranked here by value per unit of effort rather than by whichever one the generator makes easiest. Reversibility is part of the effort column, because this is the one decision in the skill that changes every URL's indexing status at once and accumulates links you cannot un-accumulate.

Posture	URL shape and canonical	Effort, including reversal	Value bought
Stable-canonical	/en/stable/… and /en/v2.3/…; every version canonicals to the stable alias	an hour where the platform ships it (Read the Docs exposes READTHEDOCS_CANONICAL_URL), a week where the alias has to be built by hand; reversed by changing one canonical target, since no URL moves	current release wins the query and every supported release still has a live URL
Latest-only	/docs/… live, /v1/… archived and noindex	near-zero - one directive on the archive, reversed by removing it	current release wins the query outright; nobody on an old release finds their page
Per-version indexable	one self-canonical page per version, plus a version switcher	a standing job - every release multiplies the indexable surface, the sitemap and the link mesh; reversal needs a per-page redirect map across every version already indexed	readers on old releases land on the page for their own release

Each posture is already shipped somewhere, not hypothetical:

• Redocly's own documentation-SEO guidance recommends canonicalizing every version to the current release for most docs sites - the same mechanism as stable-canonical, aimed at "latest" rather than a separate alias.

• Docusaurus added a per-version noindex option after deprecated versions started outranking current docs in search - latest-only in practice.

• Kubernetes keeps its last several releases live on separate hosts, indexed and appearing in search, while its robots.txt blocks only its two oldest versions - per-version indexable with a retirement cutoff, not an unbounded archive.

• efficiency: stable-canonical > latest-only > per-version indexable

• value: stable-canonical > per-version indexable > latest-only, whenever older releases are still supported

• effort: per-version indexable > stable-canonical > latest-only

Reversibility gets no line of its own because it orders exactly as effort does here. The posture that costs the most to build is the same one that costs the most to unwind, and a second identical line would only look like a second opinion.

Default to stable-canonical, and say so as a recommendation. Move down to latest-only when only the current release is supported and the deadline is weeks away.

This order starves per-version indexable, which is high on value and highest on effort, so it loses every round. Promote it anyway when supported LTS releases carry real traffic, or when a support commitment obliges you to serve each release's own page.

Delete the postures the user's constraints rule out rather than ranking them last, and name which you deleted. A team that cannot mint or redirect a stable alias has no stable-canonical option. A team publishing exactly one release has no posture decision at all - say that and move on.

Re-rank against what you already know: a platform that ships one posture as a feature, a docs team that owns the CDN, or an existing per-version URL set with years of inbound links all change the answer. The ranking is a default, not a law.

Then align the rest of the signals:

1. Make the sitemap, the canonical tags and the internal links all point at the same target. Conflicting signals are why Google picks its own canonical.
2. Give every locale a self-canonical and a complete hreflang set. Canonicalizing a translation to English deletes it from search.
3. Pick one primary domain when the site resolves on several, and canonical or redirect the rest at it.
4. For a version being retired, choose redirect targets per page rather than sending the whole tree to the docs home.

Ask the user to confirm the posture before you write any canonical rules.

Google's own localized-versions documentation settles four hreflang questions - treat the first three as hard requirements, and report a breach as a defect, not a suggestion:

• Every language version lists itself as well as all the others; include x-default for the no-match case.
• Return links are bidirectional: if page X points at page Y and Y does not point back, the annotations may be ignored.
• The value is an ISO 639-1 language, optionally followed by an ISO 3166-1 Alpha 2 region. A region alone is invalid.
• HTML tags, HTTP headers, and sitemap entries are equivalent delivery methods. Google states no preference, large site or not.

Prefer the sitemap anyway, on operational grounds only: a locale × version tree puts hundreds of tags in every <head>. Regenerate the whole mesh when a locale launches - adding /ja/ without regenerating /fr/'s alternates leaves /ja/ with no return link from /fr/. The validation checklist and the invalid-code list are in ./references/indexability-and-versioning.md.

Step 4 - Fix page signals at the template level

On a docs site titles, descriptions and heading structure are generated, so the fix is one template edit that reaches every page. Per-page rewrites come second, and only for the pages that carry traffic.

• Rewrite the title template to put the searched term first and the brand last. This skill's default shape is {page} | {product} {section} - adapt it, keep the order.
• Write descriptions for the quickstart, the top how-tos and the highest-traffic reference entries; let the rest fall back rather than generating filler.
• Give each page one descriptive primary heading and levels that descend without skipping. Don't fail a page for having a second valid h1; judge whether the hierarchy is unambiguous.
• Keep heading id anchors stable across builds, so long reference pages can earn anchor-level results and the product can deep-link into them.
• Skip HowTo and `FAQPage

...