Skills
.Work. You
Rest

crawl

Community

Crawl websites for content and data.

Software Engineering#data extraction#documentation#web scraping#tavily#crawling#content collection
AuthorHuynhSang2005
Version1.0.0
Installs0

System Documentation

What problem does it solve?

This Skill automates the process of crawling websites to extract content from multiple pages, making it ideal for downloading documentation, knowledge bases, or web content for offline access or analysis.

Core Features & Use Cases

  • Website Crawling: Extracts content from specified URLs and their linked pages.
  • Content Extraction: Saves crawled pages as local markdown files or extracts relevant chunks for LLM context.
  • Focused Crawling: Supports filtering by paths and instructions for targeted data collection.
  • Use Case: Download an entire documentation website for offline reference or to build a local knowledge base.

Quick Start

Crawl the website at https://docs.example.com and save the content to a local directory named ./docs.

Dependency Matrix

Required Modules

None required

Components

scripts

💻 Claude Code Installation

Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.

Please help me install this Skill:
Name: crawl
Download link: https://github.com/HuynhSang2005/blog-personal-nextjs/archive/main.zip#crawl

Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
View Source Repository

Agent Skills Search Helper

Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.