Skip to main content

SEO Launch Playbook — 50,000-Page Knowledge Base

Expert playbook for publishing the id86.net brain knowledge base (technical cheatsheets, references, runbooks) at scale, without triggering Google's scaled-content-abuse / helpful-content suppression.

0. Core truth​

Google will not index 50k pages quickly, and it will not rank pages that are just reformatted command lists. Your lever is unique, firsthand, well-structured reference content with strong internal linking. Volume only helps if each page earns its place. Expect a 6–12 month ramp before the long tail compounds.

1. Pre-launch blockers​

Already flagged in your own docs — critical at 50k:

IssueWhy it mattersFix
Sitemap returns HTML, not XML (/sitemaps.xml)Google can't discover 50k URLsFix SEOPress sitemap generation; verify Content-Type: application/xml
Cloudflare Zero Trust blocks GooglebotStale "excluded by noindex" in GSC; zero crawlAt launch: scoped Access bypass for verified bot UAs, or remove Access
Duplicate mirror: Docusaurus id86.net vs WordPress id86.netSame content indexed twice → canonical dilutionPick one canonical home (WP). Noindex/canonical the Docusaurus mirror
Source-of-truth ambiguity: scs/data/id86/md vs ID86-instant-documentation repoRisk of publishing partial/draft contentOne canonical repo; status: draft for WIP, promote only after QA

2. Content quality gate​

Your files already come in pairs (e.g. …-reference.md + …-cheatsheet.md for the same topic). At 50k this is a massive duplicate/thin-content risk.

  • Dedup: MinHash/SimHash across all md files; merge pages >80% similar or canonicalize variants.
  • Thin-content rule: A page needs ≥300 words of unique value — when to use, examples, pitfalls, version notes, related commands. Bare command lists → merge into the reference page or noindex.
  • Title/slug optimization: Target long-tail queries (bash cheat sheet, git quick reference). Frontmatter title + slug + description must be unique per page.
  • H2-first rule: Already enforced (template renders H1). Keep it.
  • Content types: Use the 10 registered types consistently; they drive badges and help Google understand page purpose.
  • Freshness: date + modified date via [id86_modified_date] shortcode. Refresh high-impression pages quarterly.

3. Architecture & internal linking​

50k orphan pages won't rank.

  • Categories = silos (top-level folders). Each needs a strong hub/pillar page linking to all children.
  • Tags governance: Auto-creating a tag per md tag explodes into thousands of thin archives. Curate a controlled vocabulary; noindex tag archives with <10 posts; merge synonyms.
  • Automated related posts: Inject "See also / Related" blocks by category + tag overlap. This is your #1 crawl-priority driver.
  • Breadcrumbs + HTML sitemap + XML sitemap index (one file per category, ≤10k URLs each).
  • Pagination: Efficient (avoid OFFSET on 50k posts).
  • Noindex: search result pages, filter/sort URLs, empty archives.

4. Scale the sync pipeline​

Current sync.py collects all *.md and sends one giant payload. At 50k this times out or OOMs.

  • Chunked sync: Add --batch-size 500 or use --only PATH; process one silo at a time.
  • Incremental by hash: _id86_source_hash skip already exists — protect it; never --force globally at 50k.
  • Run from VPS for bulk: Use tools/id86-sync.sh / systemd unit for the heavy initial load; Actions minutes + SCP limits will bite. Keep Actions for incremental pushes.
  • DB performance: 50k posts × ~6 meta keys. Ensure wp_postmeta composite index on (meta_key, meta_value(191)). Monitor slow queries.
  • Meilisearch reindex: Once per sync (good). For bulk, reindex once at the very end.
  • Cache purge: Per-URL PURGESINGLE × 50k is too heavy. Use --purge-all once after the full batch; targeted purges for incremental syncs. Cloudflare purge API allows only 30 URLs/request — verify chunking or purge everything for bulk.

5. Technical SEO & performance​

  • Core Web Vitals: Code Block Pro pre-renders Shiki (good). But 50k posts with heavy code = large HTML. LiteSpeed cache + Cloudflare CDN. Measure LCP/CLS/INP on a sample of 20 pages post-launch.
  • Schema: SEOPress → Article/TechArticle. Cheatsheets → TechArticle/HowTo; glossary → DefinedTermSet. Rich results improve CTR on long-tail.
  • robots.txt: Plain text, include sitemap URL, allow CSS/JS. Verify with check-indexing.py.
  • Canonical tags: Self-referencing canonical on every post.
  • Image optimization: WebP + lazy load + proper sizing; SEOPress AI alt-text in batches.
  • Mobile-first: Test 50 URLs in Mobile-Friendly Test; code blocks must scroll horizontally.

6. Staged rollout​

WeekVolumeGate to advance
1–2500–1,000 (one or two silos)Indexing ≥60%, no CWV regressions, no GSC errors, sitemap valid XML
3–41,500–2,000Indexing ≥70%, impressions appearing
5–83,000–5,000"Discovered not indexed" <30%
9+5,000–10,000Only if all signals green
  • Publish as draft first, spot-check 5% in WP editor, then flip to publish in chunks.
  • Never publish >10% of existing URL count in a single day.
  • Keep status: draft in frontmatter for WIP.

7. Indexing acceleration​

No Indexing API (correct — it's for JobPosting/BroadcastEvent only).

  • Sitemap submitted once in GSC; new posts discovered on each crawl pass. Fix XML bug first.
  • IndexNow via SEOPress Pro — fires on publish for Bing/Yandex/Seznam/Naver. Enable it.
  • WP publish ping — already triggered by wp_insert_post.
  • Internal-link amplification: "Latest in [topic]" block on category hub pages drives crawl priority more than pinging.
  • GSC URL Inspection → Request indexing for top ~10–12 pillar/hub pages per day (quota-limited).
  • Log-file analysis weekly: are bots hitting new URLs?

8. Launch checklist (Cloudflare Access)​

Execute in order when going public:

  1. Decide: remove Zero Trust OR add scoped Access bypass for verified bot UAs on id86.net.
  2. Verify Googlebot gets real 200 on / and /sitemaps.xml from outside Cloudflare.
  3. GSC → URL Inspection → https://id86.net/ → Request indexing.
  4. Confirm stale "noindex" clears after new crawl.
  5. Confirm Search Analytics starts accumulating.
  6. Add a second GSC verification method (HTML file) so ownership isn't lost if DNS TXT is removed.

9. Monitoring dashboard (weekly)​

MetricSourceRed flag
Indexing rateGSC Coverage<60% after 4 weeks
"Discovered – not indexed"GSCRising month-over-month
"Crawled – not indexed"GSC>20% of submitted
Impressions/clicks per new URLGSCFlat after 8 weeks → prune/rewrite
CWV p75 (LCP/CLS/INP)CrUX/GSCAny "Poor"
Crawl statsGSCSudden drops = server issues
Manual actionsGSCAny = stop publishing immediately

10. Risk guardrails​

  • Scaled content abuse policy: mass-publishing pages whose primary purpose is ranking, without unique value, is spam — regardless of AI/human.
  • Helpful Content System is site-wide: if a large fraction of 50k is unhelpful, your existing good pages lose rankings too.
  • Don't create doorway pages, auto-generated location pages, or "best X" spam.
  • Don't buy links; earn them via original data, tools, or definitive references.
  • Don't index thin tag/category archives.
  • Prune monthly: bottom 10% by impressions after 90 days → noindex or merge.
  • Rollback plan: if indexing stalls or rankings drop, pause publishing for 2 weeks and audit.

11. 90-day calendar​

  • Days 1–14: Quality gate + dedup + fix sitemap XML + canonical mirror decision + staging + technical fixes.
  • Days 15–30: Tranche 1 (500–1k, one silo), monitor indexing, fix crawl issues, build internal links, launch Cloudflare Access bypass.
  • Days 31–60: Tranches 2–3 (up to 5k), add pillar/hub pages, schema, author entity, IndexNow, E-E-A-T pages.
  • Days 61–90: Scale to 10k/week if green; prune underperformers; launch link-building; refresh top 100 pages.
  • Ongoing: weekly GSC review, monthly content refresh, quarterly prune of bottom 10%.

12. Leveraging your existing stack​

  • Add publish_decision, tranche, publish_date, internal_links_added, indexed_status, gsc_impressions_30d columns to your inventory CSV.
  • Automate internal-link injection at sync time (match by category + tag overlap).
  • Add post-sync verification: wp post list --post_status=publish count matches manifest; sitemap regenerated; IndexNow fired; report written.
  • Use --only <path> for chunked syncs and --purge-all for bulk launches.
  • Keep tools/check-indexing.py --development in CI to catch regressions without unblocking crawlers prematurely.

Bottom line​

For a 50k-page personal knowledge base, the winning formula is: unique firsthand reference content + siloed architecture with automated internal linking + staged rollout with indexing gates + monthly pruning of the bottom 10%. Your sync pipeline, SEOPress, and Cloudflare stack already support this — the missing pieces are chunked sync, tag governance, the sitemap XML fix, and the Cloudflare Access launch decision.