skills.rest
azaanaliraza
azaanalirazaCOMMUNITY
Explore all skills by this author →
GitHub

scrapling

Scrapes web pages using HTTP fetching, stealth browser automation, and spider crawling via Scrapling.

Updated Apr 18, 2026
Search Tags:#data extraction#python#web scraping#scrapling#cloudflare bypass#stealth browser#spider crawling
One-click install
npx skills add https://github.com/azaanaliraza/operarius --skill scrapling-azaanaliraza
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scrapling
Source: https://github.com/azaanaliraza/operarius/tree/main/src-tauri/bin/hermes/optional-skills/research/scrapling
Command: npx skills add https://github.com/azaanaliraza/operarius --skill scrapling-azaanaliraza

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires scrapling.

What problem does it solve? Extracting data from websites often fails when pages require JavaScript rendering, sit behind Cloudflare protection, or span many linked pages. This Skill provides CLI and Python workflows for scraping static, dynamic, and anti-bot-protected sites with a single framework. ## Core Features & Use Cases - Three Fetching Strategies: Use Fetcher for fast HTTP requests, DynamicFetcher for JS-rendered pages, and StealthyFetcher for Cloudflare bypass and anti-bot evasion. - Spider Framework: Crawl multiple pages with link following, concurrent requests, multi-session routing, and pause/resume checkpoints. - Rich Element Selection: Query content with CSS selectors, XPath, regex, text matching, and similar-element detection. - Use Case: Scrape a Cloudflare-protected product catalog by running scrapling extract stealthy-fetch with --solve-cloudflare, then export results to Markdown or JSON. ## Quick Start Use the scrapling skill to extract the main content from https://example.com and save it as a Markdown file.

Frequently Asked Questions about scrapling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape a website with Python using Scrapling?▼

Install scrapling with pip, then use Fetcher.get(url) for static pages and select elements with CSS selectors like page.css('h1::text').get(). For persistent cookies across requests, use FetcherSession with browser impersonation.

How to bypass Cloudflare protection when web scraping?▼

Use StealthyFetcher with solve_cloudflare=True, block_webrtc=True, and hide_canvas=True to evade Cloudflare Turnstile and fingerprinting. Note that Cloudflare solving adds 5-15 seconds per fetch, so enable it only when needed.

What is the difference between Fetcher, DynamicFetcher, and StealthyFetcher?▼

Fetcher handles static pages and APIs over plain HTTP and is fastest. DynamicFetcher runs a real browser for JS-rendered SPAs. StealthyFetcher adds anti-fingerprinting and Cloudflare bypass for protected sites.

Why does DynamicFetcher fail after installing scrapling?▼

DynamicFetcher and StealthyFetcher require browser binaries installed via the scrapling install command after pip install. Without this step, browser-based fetchers will fail at runtime.

Can Scrapling crawl multiple pages and resume after interruption?▼

Yes, the Spider framework supports concurrent requests, link following via response.follow, and multi-session routing. Pass a crawldir path to the spider to checkpoint progress and resume crawling after interruption.

What are the limitations of stealth web scraping with Scrapling?▼

StealthyFetcher runs a real browser, so concurrent usage consumes significant memory and CPU. Cloudflare solving adds latency, and users must comply with robots.txt, website Terms of Service, and applicable scraping laws.

Related Skills

Explore More Software Engineering Skills→
ZJU-REALZJU-REAL
195

firecrawl-map

Discover and filter website URLs using the Firecrawl command-line interface.

Official
Basic
TurnaboutHeroTurnaboutHero
12

data-wizard

Automate ETL and data transformation for CSV, JSON, and database sources using Python and pandas.

Community
Advanced
aryaniyapsaryaniyaps
26

firecrawl-build-scrape

Extract HTML, markdown, links, and screenshots from known URLs.

Community
Intermediate
mazreanmazrean

building-static-site-rss-feeds

Generate RSS 2.0, Atom 1.0, and JSON feeds for static sites.

Community
Intermediate