Scraping
CommunityWeb scraping via progressive escalation.
AuthorRooseveltAdvisors
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill provides a robust solution for extracting web content, even from sites that employ bot detection, CAPTCHAs, or require JavaScript rendering, by progressively escalating through different scraping methods.
Core Features & Use Cases
- Four-Tier Scraping: Utilizes WebFetch, custom Curl, Browser Automation (Playwright), and Bright Data MCP as fallbacks.
- URL Scraping: Reliably fetches content from any single URL.
- Site Crawling: Capable of crawling multiple pages within a site.
- Use Case: When a website blocks direct access or requires JavaScript to load content, this Skill can automatically switch to more advanced methods like Bright Data to ensure content retrieval.
Quick Start
Use the BrightData skill to scrape the content from the URL https://example.com.
Dependency Matrix
Required Modules
None requiredComponents
scriptsreferences
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: Scraping Download link: https://github.com/RooseveltAdvisors/claude-agent-stack/archive/main.zip#scraping Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.