scrape-web-page

Use interactive mode with Playwright: load the page, then click the load-more button in a loop until it disappears, becomes disabled, or the row count stops growing. After the page settles, extract the requested data from the final DOM and save it as JSON.

Use network mode: attach a Playwright response listener, load the page, and record every JSON endpoint it requests. Then call the endpoint directly with curl or Fetcher.get and cache it in sites.yaml so future runs skip browser automation entirely.

Raw mode fetches the HTML as served, rendered mode executes JavaScript first, and interactive mode additionally scripts user actions like clicks, typing, and scrolling. Start with raw since rendered is the slowest, and escalate only when the goal requires it.

Yes, all requests originate from the user's own machine, preserving their local IP address. The Skill explicitly forbids routing through hosted readers like Jina, Firecrawl, or ScrapingBee, which is the entire reason the plugin exists.

The page likely hides data behind user interaction such as a city dropdown, search box, or show-all button that a one-shot fetch never triggers. Switch to interactive mode, or run network mode first to find the backing JSON API and call it directly.

Use scrape-article for clean, readability-friendly article pages where a main text body exists. Use this general scraper for everything else: SPAs, catalogs, search results, dashboards, and pages needing raw HTML, DOM, or network capture.