skills.rest
danielrosehill
danielrosehillCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

scrape-article

Scrapes article content from URLs via localhost and saves markdown captures with frontmatter.

Updated Apr 23, 2026
Search Tags:#markdown#web scraping#news#camoufox#scrapling#article capture#geo-restricted
One-click install
npx skills add https://github.com/danielrosehill/Claude-Local-Web-Capture-Plugin --skill scrape-article-danielrosehill
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scrape-article
Source: https://github.com/danielrosehill/Claude-Local-Web-Capture-Plugin/tree/main/skills/scrape-article
Command: npx skills add https://github.com/danielrosehill/Claude-Local-Web-Capture-Plugin --skill scrape-article-danielrosehill

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires scrapling.

What problem does it solve? Geo-restricted and bot-protected sites block hosted scraping services, so articles from sources like Israeli news sites cannot be captured through cloud readers. This Skill fetches pages from the user's own machine, preserving the local IP, and saves clean markdown captures. ## Core Features & Use Cases - Tiered scraping ladder: Escalates from Scrapling static Fetcher to StealthyFetcher (Camoufox) to PlayWrightFetcher, stopping at the first rung that returns a real article body. - Strategy caching: Records per-domain scraping strategy in sites.yaml so repeat captures start at the correct rung automatically. - Structured markdown output: Saves captures with YAML frontmatter (URL, title, author, published date, language, word count, rung used) into a date-organized directory. - Use Case: A researcher pastes a ynet or Haaretz URL and asks to capture the article; the Skill scrapes it locally, extracts the body text, and saves a timestamped markdown file under articles/YYYY/MM/. ## Quick Start Capture this article and save it as markdown: https://www.ynet.co.il/news/article/example

Frequently Asked Questions about scrape-article

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape articles from geo-restricted news sites?▼

Run the scrape from your own machine so requests exit via your local IP instead of a hosted reader. This Skill uses Scrapling's Fetcher by default and escalates to stealth or JS-rendering fetchers when the static request is blocked.

What is the difference between Scrapling Fetcher and StealthyFetcher?▼

Fetcher is a plain HTTP client with lxml parsing, fastest for server-rendered pages. StealthyFetcher uses Camoufox, a stealth Firefox build, to bypass Cloudflare and bot detection when static requests return 403 or empty bodies.

Can this scrape paywalled or login-required articles?▼

No. This Skill explicitly stops at headless rungs and refuses paywalled or session-bound pages. For authenticated content it directs you to the scrape-authenticated skill, which drives a real logged-in Chrome via bb-browser.

Why does my scraped article contain navigation and footer text?▼

The extractor fell back to the full document instead of the article element. Strip nav, footer, aside, script, and style elements before reading markdown, or add a site-specific CSS selector to sites.yaml for reliable body extraction.

Where are scraped article captures saved?▼

Captures save to <repo_root>/captures/articles/YYYY/MM/ when run inside a git repository, otherwise to ~/local-web-capture/articles/YYYY/MM/. An explicit --out path overrides both locations.

Related Skills

Explore More Content & Communication Skills→
fengshao1227fengshao1227
5.9k

scrapling

Scrapes web pages and extracts data using scrapling Fetchers with Cloudflare bypass support.

Community
Advanced
ignromanovignromanov
20

capture

Capture URLs, files, and clipboard data into frontmatter-rich markdown files.

Community
Intermediate
silva2kandsilva2kand

price-tracker

Monitor and compare e-commerce product prices across multiple websites.

Community
Intermediate
ybbms777ybbms777
1

agent-browser

Automate web browser tasks via Chrome/Chromium CDP or Lightpanda.

Community
Advanced