firecrawl

Run firecrawl search with your query and add --scrape to fetch each result's page content in one step. Use --json and -o to save structured output to a file, avoiding repeated scrape calls on the same URLs.

Use firecrawl scrape with the --wait-for flag to give the page time to render before extraction. If scraping still fails because interaction is required, escalate to firecrawl browser sessions with click, fill, and snapshot commands.

Use map when you know the domain but need to locate specific subpages, especially with --search to filter URLs. Use crawl only when you need bulk content from an entire site section, such as a full docs tree or blog archive.

No, the browser command should not be used on sites with Cloudflare challenges, CAPTCHAs, or bot detection, including Google and Bing. For those sites, use firecrawl search or find public mirrors and alternative sources instead.

The npm global bin directory may not be in your PATH after installing with npm install -g firecrawl-cli. Verify with firecrawl --status, or fall back to running it via npx firecrawl-cli.

Run firecrawl --status to see authentication state, concurrency usage, and remaining credits, or firecrawl credit-usage for detailed consumption. Scale task scope down if credits or concurrency are insufficient.