reverse-proxy-cms-indexing

Fix robots.txt, sitemap, and canonical URL mismatches for CMS sites behind Host-rewriting reverse proxies.

3|1|Updated Apr 8, 2026
One-click install
npx skills add https://github.com/ever-just/agentskills --skill reverse-proxy-cms-indexing-ever-just
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: reverse-proxy-cms-indexing
Source: https://github.com/ever-just/agentskills/tree/main/skills/reverse-proxy-cms-indexing
Command: npx skills add https://github.com/ever-just/agentskills --skill reverse-proxy-cms-indexing-ever-just

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? When nginx or Cloudflare rewrites the Host header before a CMS like Odoo sees the request, the app builds URLs from the internal host instead of the public domain. This silently breaks robots.txt (emitting Disallow: /), points sitemaps and canonical tags at internal hosts, and leaves internal tenant hosts crawlable as duplicate sites — blocking search engines and AI crawlers from indexing the site. ## Core Features & Use Cases - Robots.txt domain-mismatch fix: Patch the Odoo website.robots template via a DB inherit view so the Disallow rule stops firing when the internal host fails the indexability check. - Proxy-level sitemap and canonical rewriting: Add nginx sub_filter location blocks that rewrite the internal host to the public domain in sitemap XML and robots.txt responses. - Duplicate-host noindexing: Configure a dedicated nginx server block that adds X-Robots-Tag noindex headers to the internal tenant host without breaking backend or OAuth links. - Use Case: A multi-tenant Odoo platform serves customer domains through a wildcard reverse proxy, and one tenant's site is not being indexed. Use this Skill to diagnose the Host-header mismatch, patch the robots template, rewrite the sitemap at the proxy, and safely validate and roll back the shared nginx config. ## Quick Start Diagnose why my Odoo site behind the nginx reverse proxy is not getting indexed and fix the robots.txt and sitemap host mismatch.

Frequently Asked Questions about reverse-proxy-cms-indexing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
Why does robots.txt say Disallow: / when my site should be indexable?▼

When a reverse proxy rewrites the Host header, the CMS sees the internal host instead of the public domain. Odoo's website.robots template then fails its indexability check and emits Disallow: /, blocking all crawlers even though pages return 200.

How do I fix sitemap URLs pointing to an internal host behind nginx?▼

Add nginx location blocks for sitemap XML and robots.txt that use sub_filter to rewrite the internal host to the public domain in the response body. Set Accept-Encoding to empty so sub_filter sees uncompressed content, and keep the rewritten Host header so tenant matching still works.

Should I redirect or noindex an internal tenant host that is publicly crawlable?▼

Use X-Robots-Tag noindex headers, not a redirect. The internal host is often the application's base URL, so backend, OAuth, and email links must keep resolving. A dedicated nginx server block for that exact host adds the header without affecting other tenants.

Why don't Odoo template edits show up live after editing via shell?▼

Running workers serve a cached compiled template, and cache-clearing sequences do not propagate across processes. Restart the Odoo container, or better, write the view via XML-RPC or in-worker tools, which invalidate the cache immediately without a restart.

How do I safely edit shared production nginx config without downtime?▼

Back up the live file first, make only additive edits such as new server or location blocks, then validate with nginx -t inside the container. Restart only if validation passes; otherwise restore the backup, since the running config stays active until restart.