crawl-website
OfficialDiscover and snapshot relevant website pages.
Authortheworkflowco
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Discover and scrape relevant pages from a company website to build context for research briefs and knowledge bases.
Core Features & Use Cases
- Discover pages via domain-wide search to identify candidate URLs.
- Filter for high-value pages (homepage, about, product, pricing, FAQ, team, customers) to focus scraping efforts.
- Save scraped content as Markdown files under docs/research/website/ for easy documentation and reuse.
Quick Start
Run the crawl-website skill on a target domain and save discovered pages as markdown under docs/research/website/.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: crawl-website Download link: https://github.com/theworkflowco/gtm-agent-example/archive/main.zip#crawl-website Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.