skills.rest
madyankin
madyankinCOMMUNITY
Explore all skills by this author →
Website𝕏@madyankinGitHub

read-web-page

Extract clean markdown from web pages with trafilatura and sanitize prompt injection patterns.

5|Updated Sep 24, 2017
Search Tags:#markdown#security#sanitization#web scraping#content extraction#trafilatura#prompt injection
One-click install
npx skills add https://github.com/madyankin/dotfiles --skill read-web-page-madyankin
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: read-web-page
Source: https://github.com/madyankin/dotfiles/tree/main/.config/agents/skills/read-web-page
Command: npx skills add https://github.com/madyankin/dotfiles --skill read-web-page-madyankin

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires trafilatura, transformers, torch, and includes scripts (resource) components.

What problem does it solve? Fetching web pages directly returns navigation, ads, and boilerplate that waste tokens, and raw web content can carry prompt injection attacks targeting AI agents. This Skill extracts readable article text and sanitizes it before the agent processes it. ## Core Features & Use Cases - Clean Content Extraction: Uses the trafilatura CLI to strip navigation, ads, and clutter from articles, documentation, and blog posts, outputting plain text, XML, JSON, or CSV. - Two-Pass Sanitization: A regex pass redacts injection patterns, LLM template tokens, and dangerous code execution calls, while an optional Prompt Guard pass uses Meta Llama-Prompt-Guard-2-86M to classify and redact injection or jailbreak content. - URL Safety Validation: The read-web.py script enforces https-only URLs, rejects credentials in URLs, and blocks hosts resolving to private, loopback, link-local, or cloud metadata addresses. - Use Case: When a user pastes a URL to an article and asks for a summary, the agent fetches it through trafilatura, pipes the output through the sanitizer, and receives fenced untrusted content safe to analyze. ## Quick Start Ask the agent to read and summarize the article at a specific https URL you provide in chat.

Frequently Asked Questions about read-web-page

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract clean text from a web page with trafilatura?▼

Run trafilatura with the -u flag followed by an https URL to extract readable content without navigation or ads. Pipe the output through sanitize-web-content.py so injection patterns are redacted before an AI agent processes the text.

How do I protect an AI agent from prompt injection in web content?▼

Pipe fetched web content through sanitize-web-content.py, which strips injection phrases, LLM template tokens, and dangerous code execution patterns via regex. Installing transformers and torch adds a Prompt Guard pass that classifies and redacts injection or jailbreak windows.

What output formats does trafilatura support?▼

Trafilatura outputs plain text by default, with --xml for XML with metadata, --json for JSON metadata fields like title and date, and --csv for CSV output. Batch extraction is supported with -i for URL lists and -o for an output directory.

Why does the URL validation reject some addresses?▼

The read-web.py script only allows https URLs and resolves the host to verify every address is publicly routable. It rejects loopback, RFC-1918 private ranges, link-local addresses, and cloud metadata endpoints like 169.254.169.254 to prevent SSRF attacks.

Does the sanitizer work without transformers and torch installed?▼

Yes, the regex pass always runs and strips known injection patterns, template tokens, and dangerous code blocks. The Prompt Guard ML pass is skipped silently when transformers and torch are not installed, and a comment in the output notes their absence.

Related Skills

Explore More Software Engineering Skills→
augustsclaugustscl
2

rss-aggregator

Aggregate and summarize recent RSS feed entries from OPML lists with a days-based cutoff.

Community
Intermediate
penseraipenserai

digest

Summarize gopher-cli RSS feed top stories into Markdown digests stored in the vault.

Community
Intermediate
Nandansai08Nandansai08
1

source-credibility-check

Audit a single source or claim by tracing citations to their primary origin.

Community
Intermediate
octos-orgoctos-org
1.0k

deep-crawl

Crawl JavaScript-rendered websites with headless Chrome via CDP and output structured JSON reports.

Official
Advanced