SEO Launch Playbook — 50,000-Page Knowledge Base
Expert playbook for publishing the id86.net brain knowledge base (technical cheatsheets, references, runbooks) at scale, without triggering Google's scaled-content-abuse / helpful-content suppression.
0. Core truth
Google will not index 50k pages quickly, and it will not rank pages that are just reformatted command lists. Your lever is unique, firsthand, well-structured reference content with strong internal linking. Volume only helps if each page earns its place. Expect a 6–12 month ramp before the long tail compounds.
1. Pre-launch blockers
Already flagged in your own docs — critical at 50k:
| Issue | Why it matters | Fix |
|---|---|---|
Sitemap returns HTML, not XML (/sitemaps.xml) | Google can't discover 50k URLs | Fix SEOPress sitemap generation; verify Content-Type: application/xml |
| Cloudflare Zero Trust blocks Googlebot | Stale "excluded by noindex" in GSC; zero crawl | At launch: scoped Access bypass for verified bot UAs, or remove Access |
Duplicate mirror: Docusaurus id86.net vs WordPress id86.net | Same content indexed twice → canonical dilution | Pick one canonical home (WP). Noindex/canonical the Docusaurus mirror |
Source-of-truth ambiguity: scs/data/id86/md vs ID86-instant-documentation repo | Risk of publishing partial/draft content | One canonical repo; status: draft for WIP, promote only after QA |
2. Content quality gate
Your files already come in pairs (e.g. …-reference.md + …-cheatsheet.md for the same topic). At 50k this is a massive duplicate/thin-content risk.
- Dedup: MinHash/SimHash across all md files; merge pages >80% similar or canonicalize variants.
- Thin-content rule: A page needs ≥300 words of unique value — when to use, examples, pitfalls, version notes, related commands. Bare command lists → merge into the reference page or noindex.
- Title/slug optimization: Target long-tail queries (
bash cheat sheet,git quick reference). Frontmattertitle+slug+descriptionmust be unique per page. - H2-first rule: Already enforced (template renders H1). Keep it.
- Content types: Use the 10 registered types consistently; they drive badges and help Google understand page purpose.
- Freshness:
date+ modified date via[id86_modified_date]shortcode. Refresh high-impression pages quarterly.
3. Architecture & internal linking
50k orphan pages won't rank.
- Categories = silos (top-level folders). Each needs a strong hub/pillar page linking to all children.
- Tags governance: Auto-creating a tag per md tag explodes into thousands of thin archives. Curate a controlled vocabulary; noindex tag archives with <10 posts; merge synonyms.
- Automated related posts: Inject "See also / Related" blocks by category + tag overlap. This is your #1 crawl-priority driver.
- Breadcrumbs + HTML sitemap + XML sitemap index (one file per category, ≤10k URLs each).
- Pagination: Efficient (avoid
OFFSETon 50k posts). - Noindex: search result pages, filter/sort URLs, empty archives.
4. Scale the sync pipeline
Current sync.py collects all *.md and sends one giant payload. At 50k this times out or OOMs.
- Chunked sync: Add
--batch-size 500or use--only PATH; process one silo at a time. - Incremental by hash:
_id86_source_hashskip already exists — protect it; never--forceglobally at 50k. - Run from VPS for bulk: Use
tools/id86-sync.sh/ systemd unit for the heavy initial load; Actions minutes + SCP limits will bite. Keep Actions for incremental pushes. - DB performance: 50k posts × ~6 meta keys. Ensure
wp_postmetacomposite index on(meta_key, meta_value(191)). Monitor slow queries. - Meilisearch reindex: Once per sync (good). For bulk, reindex once at the very end.
- Cache purge: Per-URL
PURGESINGLE× 50k is too heavy. Use--purge-allonce after the full batch; targeted purges for incremental syncs. Cloudflare purge API allows only 30 URLs/request — verify chunking or purge everything for bulk.
5. Technical SEO & performance
- Core Web Vitals: Code Block Pro pre-renders Shiki (good). But 50k posts with heavy code = large HTML. LiteSpeed cache + Cloudflare CDN. Measure LCP/CLS/INP on a sample of 20 pages post-launch.
- Schema: SEOPress → Article/TechArticle. Cheatsheets →
TechArticle/HowTo; glossary →DefinedTermSet. Rich results improve CTR on long-tail. - robots.txt: Plain text, include sitemap URL, allow CSS/JS. Verify with
check-indexing.py. - Canonical tags: Self-referencing canonical on every post.
- Image optimization: WebP + lazy load + proper sizing; SEOPress AI alt-text in batches.
- Mobile-first: Test 50 URLs in Mobile-Friendly Test; code blocks must scroll horizontally.
6. Staged rollout
| Week | Volume | Gate to advance |
|---|---|---|
| 1–2 | 500–1,000 (one or two silos) | Indexing ≥60%, no CWV regressions, no GSC errors, sitemap valid XML |
| 3–4 | 1,500–2,000 | Indexing ≥70%, impressions appearing |
| 5–8 | 3,000–5,000 | "Discovered not indexed" <30% |
| 9+ | 5,000–10,000 | Only if all signals green |
- Publish as
draftfirst, spot-check 5% in WP editor, then flip topublishin chunks. - Never publish >10% of existing URL count in a single day.
- Keep
status: draftin frontmatter for WIP.
7. Indexing acceleration
No Indexing API (correct — it's for JobPosting/BroadcastEvent only).
- Sitemap submitted once in GSC; new posts discovered on each crawl pass. Fix XML bug first.
- IndexNow via SEOPress Pro — fires on publish for Bing/Yandex/Seznam/Naver. Enable it.
- WP publish ping — already triggered by
wp_insert_post. - Internal-link amplification: "Latest in [topic]" block on category hub pages drives crawl priority more than pinging.
- GSC URL Inspection → Request indexing for top ~10–12 pillar/hub pages per day (quota-limited).
- Log-file analysis weekly: are bots hitting new URLs?
8. Launch checklist (Cloudflare Access)
Execute in order when going public:
- Decide: remove Zero Trust OR add scoped Access bypass for verified bot UAs on
id86.net. - Verify Googlebot gets real
200on/and/sitemaps.xmlfrom outside Cloudflare. - GSC → URL Inspection →
https://id86.net/→ Request indexing. - Confirm stale "noindex" clears after new crawl.
- Confirm Search Analytics starts accumulating.
- Add a second GSC verification method (HTML file) so ownership isn't lost if DNS TXT is removed.
9. Monitoring dashboard (weekly)
| Metric | Source | Red flag |
|---|---|---|
| Indexing rate | GSC Coverage | <60% after 4 weeks |
| "Discovered – not indexed" | GSC | Rising month-over-month |
| "Crawled – not indexed" | GSC | >20% of submitted |
| Impressions/clicks per new URL | GSC | Flat after 8 weeks → prune/rewrite |
| CWV p75 (LCP/CLS/INP) | CrUX/GSC | Any "Poor" |
| Crawl stats | GSC | Sudden drops = server issues |
| Manual actions | GSC | Any = stop publishing immediately |
10. Risk guardrails
- Scaled content abuse policy: mass-publishing pages whose primary purpose is ranking, without unique value, is spam — regardless of AI/human.
- Helpful Content System is site-wide: if a large fraction of 50k is unhelpful, your existing good pages lose rankings too.
- Don't create doorway pages, auto-generated location pages, or "best X" spam.
- Don't buy links; earn them via original data, tools, or definitive references.
- Don't index thin tag/category archives.
- Prune monthly: bottom 10% by impressions after 90 days → noindex or merge.
- Rollback plan: if indexing stalls or rankings drop, pause publishing for 2 weeks and audit.
11. 90-day calendar
- Days 1–14: Quality gate + dedup + fix sitemap XML + canonical mirror decision + staging + technical fixes.
- Days 15–30: Tranche 1 (500–1k, one silo), monitor indexing, fix crawl issues, build internal links, launch Cloudflare Access bypass.
- Days 31–60: Tranches 2–3 (up to 5k), add pillar/hub pages, schema, author entity, IndexNow, E-E-A-T pages.
- Days 61–90: Scale to 10k/week if green; prune underperformers; launch link-building; refresh top 100 pages.
- Ongoing: weekly GSC review, monthly content refresh, quarterly prune of bottom 10%.
12. Leveraging your existing stack
- Add
publish_decision,tranche,publish_date,internal_links_added,indexed_status,gsc_impressions_30dcolumns to your inventory CSV. - Automate internal-link injection at sync time (match by category + tag overlap).
- Add post-sync verification:
wp post list --post_status=publishcount matches manifest; sitemap regenerated; IndexNow fired; report written. - Use
--only <path>for chunked syncs and--purge-allfor bulk launches. - Keep
tools/check-indexing.py --developmentin CI to catch regressions without unblocking crawlers prematurely.
Bottom line
For a 50k-page personal knowledge base, the winning formula is: unique firsthand reference content + siloed architecture with automated internal linking + staged rollout with indexing gates + monthly pruning of the bottom 10%. Your sync pipeline, SEOPress, and Cloudflare stack already support this — the missing pieces are chunked sync, tag governance, the sitemap XML fix, and the Cloudflare Access launch decision.