skills.rest
lsongdev
lsongdevCOMMUNITY
Explore all skills by this author →
Website𝕏@lsongdevGitHub

web-content-extractor

Extract clean main content from webpage URLs as Markdown using Defuddle or Jina AI reader services.

Updated Jul 20, 2026
Search Tags:#markdown#web scraping#content extraction#curl#defuddle#jina ai#article reader
One-click install
npx skills add https://github.com/lsongdev/skills --skill web-content-extractor-lsongdev
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-content-extractor
Source: https://github.com/lsongdev/skills/tree/main/web-content-extractor
Command: npx skills add https://github.com/lsongdev/skills --skill web-content-extractor-lsongdev

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Webpages are cluttered with ads, navigation menus, and boilerplate that make it hard to read or reuse the actual article content. This Skill converts any webpage URL into clean Markdown containing only the main body text. ## Core Features & Use Cases - Clean Content Extraction: Fetches a webpage through the Defuddle service to strip ads, navigation, and clutter, returning only the main article as Markdown. - Automatic Fallback: Switches to the Jina AI reader service (r.jina.ai) when Defuddle fails or returns an error. - Use Case: When a user shares a blog post URL and asks for a summary, the Skill fetches the clean article text via curl and uses it to answer questions or save the content to a file. ## Quick Start Extract the main article content from https://example.com/article and save it as clean Markdown.

Frequently Asked Questions about web-content-extractor

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract the main content from a webpage URL?▼

Prepend https://defuddle.md/ to the target URL and fetch it with curl, for example: curl -sL "https://defuddle.md/https://example.com/article". The response is clean Markdown containing only the main article content without ads or navigation.

What is the difference between Defuddle and Jina AI reader?▼

Defuddle (defuddle.md) is the primary service used to convert webpages into clean Markdown. Jina AI reader (r.jina.ai) serves as the fallback option when Defuddle fails, returns empty output, or produces an error.

What should I do if webpage content extraction fails?▼

If the Defuddle request fails or returns an error, retry with the Jina AI reader by fetching https://r.jina.ai/ prepended to the target URL. Also verify the target URL includes the https:// protocol and is quoted in the curl command.

Does the extractor work with URLs missing the https protocol?▼

Yes, but you must prepend https:// to the target URL before appending it to the service URL. URLs without a protocol cannot be fetched correctly by the extraction services.

Why should URLs be quoted in the curl command?▼

Quoting prevents the shell from interpreting special characters like & or ? in the URL, which would otherwise break the command or truncate the URL being fetched.

Related Skills

Explore More Content & Communication Skills→
OpenLinkSoftwareOpenLinkSoftware
35

opml-rss-reader

Discover and query OPML/RSS feeds from URLs using SPARQL/SPASQL templates.

Official
Advanced
lordkinglordking

web-article-collector

Extract article content from public webpages and save Markdown notes.

Community
Advanced
RonanCodesRonanCodes

ingest-web

Extract web article content into markdown with YAML frontmatter.

Community
Intermediate
appautomatonappautomaton
18

html-to-markdown

Converts HTML or live web pages into structured Markdown with metadata and link inventories.

Official
Intermediate