web-fetcher

Convert web pages into clean Markdown via OpenCLI or Jina Reader.

36|1|Updated Mar 24, 2026
One-click install
npx skills add https://github.com/jiahao-shao1/sjh-skills --skill web-fetcher
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-fetcher
Source: https://github.com/jiahao-shao1/sjh-skills/tree/main/skills/web-fetcher
Command: npx skills add https://github.com/jiahao-shao1/sjh-skills --skill web-fetcher

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Fetch any URL as clean Markdown. Use instead of WebFetch for JS-rendered pages, login-required platforms (Twitter/X, zhihu, reddit, weibo, xiaohongshu, bilibili, etc.), and complex pages. Routes known platforms through OpenCLI (browser login state), others through Jina Reader / defuddle.md / markdown.new. Invoke when the user provides a URL to read, extract, summarize, or convert to markdown.

Core Features & Use Cases

  • Known platforms via OpenCLI (uses browser login state) for reliable rendering and access control.
  • Generic URLs through a deterministic fallback chain: Jina Reader → defuddle.md → markdown.new.
  • Use Case: convert research articles, blog posts, or documentation pages into Markdown for offline reading or integration into notes.

Quick Start

Fetch a URL and output the cleaned Markdown to stdout or save it to a file.

Frequently Asked Questions about web-fetcher

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I convert a web page into clean Markdown for documentation?▼

To convert a web page into clean Markdown, this tool routes generic URLs through a deterministic chain of Jina Reader, defuddle.md, and markdown.new, producing output suitable for documentation and research workflows.

Can I extract Markdown from login-protected platforms like Reddit or Twitter?▼

Yes, you can extract Markdown from login-protected platforms like Reddit or Twitter by routing requests through OpenCLI, which leverages your existing browser login state to access and render restricted content reliably.

Does web scraping JavaScript-rendered pages require complex setup?▼

Scraping JavaScript-rendered pages requires minimal setup when using a deterministic fallback chain. It relies on either configured OpenCLI or built-in readers like Jina Reader to fetch and convert dynamic content automatically.

What is the best way to fetch generic URLs when OpenCLI is not configured?▼

The best way to fetch generic URLs without OpenCLI is using a deterministic fallback chain: Jina Reader attempts first, followed by defuddle.md, and finally markdown.new, ensuring reliable Markdown extraction for most web pages.

Why use a deterministic fetching chain instead of standard WebFetch for web scraping?▼

A deterministic fetching chain is preferred over standard WebFetch because it successfully handles JavaScript-rendered pages and login-required platforms, producing clean Markdown output where standard fetching methods fail to load dynamic content.