skills.rest
sparkst
sparkstCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

Web Research Executor

Orchestrates parallel Tavily and Brave web searches with deduplication and caching.

Updated Jan 29, 2026
Search Tags:#rate limiting#web search#brave search#research automation#deduplication#tavily#parallel search
One-click install
npx skills add https://github.com/sparkst/sparkry-claude-skills --skill web-research-executor-sparkst
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: Web Research Executor
Source: https://github.com/sparkst/sparkry-claude-skills/tree/main/plugins/research-workflow/skills/research/web-exec
Command: npx skills add https://github.com/sparkst/sparkry-claude-skills --skill web-research-executor-sparkst

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? Running broad market or technical research manually means issuing many queries across multiple search APIs, handling rate limits, and cleaning up duplicate results by hand. This Skill automates that execution layer so research queries defined in a plan file are searched in parallel and returned as a single structured source list. ## Core Features & Use Cases - Parallel Multi-Tool Search: Executes queries concurrently across Tavily, Brave web search, and Brave news search with intelligent query routing. - Rate Limiting and Fallbacks: Applies exponential or linear backoff on 429 errors and falls back from Tavily to Brave when a tool fails. - Deduplication and Caching: Normalizes URLs, hashes content to remove duplicates, and caches results for 24 hours to avoid redundant API calls. - Use Case: Given a research/plan.json containing 12 market-sizing queries, run the orchestrator to produce research/sources.json with deduplicated, scored sources and execution metrics. ## Quick Start Run the parallel search script against my research/plan.json and save the deduplicated results to research/sources.json.

Frequently Asked Questions about Web Research Executor

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run parallel web searches with Tavily and Brave Search?▼

Run scripts/parallel_search.py with --plan pointing to a plan.json containing your queries. The script executes all queries concurrently across Tavily and Brave using asyncio, then merges and deduplicates results into a single sources.json output.

What API keys are required for parallel web search orchestration?▼

You need a TAVILY_API_KEY environment variable for Tavily searches and a BRAVE_API_KEY for Brave web and news searches. The script reads these from the environment at runtime.

How does the search script handle rate limits from Tavily or Brave?▼

Tavily uses exponential backoff (1s, 2s, 4s, 8s) on 429 errors, while Brave uses linear backoff. After three retries the query fails gracefully, and the orchestrator can fall back from Tavily to Brave as a secondary tool.

Does the search orchestrator cache results to avoid duplicate API calls?▼

Yes. Results are cached in a .cache directory keyed by an MD5 hash of the tool and query. Cached entries are reused for up to 24 hours, after which a fresh API call is made.

How are duplicate search results removed across multiple search tools?▼

Sources are deduplicated by normalizing URLs (stripping tracking parameters like utm_source) and hashing content excerpts. Any source matching a previously seen URL or content hash is dropped before writing the final output.

Related Skills

Explore More Education & Research Skills→
WILLOSCARWILLOSCAR
497

idea-brief

Generate a structured brainstorm brief and query plan from user input.

Community
Intermediate
belokonmbelokonm
8

research-analyst

Coordinate multi-source research into evidence-based narrative reports.

Community
Advanced
andy-austinandy-austin

secure-web-search

Perform privacy-preserving web searches with source credibility checks and claim validation.

Community
Intermediate
bogdanbaciu21bogdanbaciu21
2

deep-research-agents

Coordinate multiple deep-research providers for complex research requests.

Community
Advanced