bggg-data-x

Collect public X posts from logged-in Chrome DOM into normalized JSONL.

588|92|Updated May 2, 2026
One-click install
npx skills add https://github.com/binggandata/bggg-skills --skill bggg-data-x
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: bggg-data-x
Source: https://github.com/binggandata/bggg-skills/tree/main/bggg-data-x
Command: npx skills add https://github.com/binggandata/bggg-skills --skill bggg-data-x

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

X/Twitter blocks most automated access behind login walls, paid APIs, and internal GraphQL endpoints, making it hard to gather auditable public post data for voice-of-customer research. This Skill collects visible posts through the user's own logged-in Chrome session and normalizes them into a consistent, auditable JSONL dataset.

Core Features & Use Cases

  • Deterministic Query Planning: Builds a validated request plan from a TSV of queries with language, round, row cap, and sort mode, generating encoded X search URLs.
  • Visible-DOM Collection: Extracts post text, Tweet IDs, canonical URLs, timestamps, author handles, and engagement labels using a fixed selector contract, with scroll-loop checkpointing and per-query source packages.
  • Normalization and Audit: Merges per-query packages, deduplicates by Tweet ID, parses engagement metrics, matches keywords, and emits a summary with missing-data and failure counts.
  • Use Case: A researcher running multilingual VOC analysis prepares queries in English and Japanese, collects up to 250 posts per query from the Latest tab, and receives a deduplicated JSONL corpus plus an audit summary for downstream reporting.

Quick Start

Use bggg-data-x to search X in my logged-in Chrome for these keyword queries and save the collected posts as normalized JSONL with an audit summary.

Frequently Asked Questions about bggg-data-x

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I collect X/Twitter posts without the paid API?▼

Drive the user's already logged-in Chrome session, open X search URLs, and extract rendered tweet cards from the visible DOM using selectors like article[data-testid="tweet"]. Results are saved per query and normalized into JSONL without touching internal GraphQL or cookies.

How to scrape X search results with a logged-in Chrome session?▼

Build a query plan from a TSV file, open each generated search URL in one Chrome tab, validate the selector contract, then scroll and accumulate posts keyed by Tweet ID. Stop after six stale scrolls or a 180-scroll hard cap and save one source package per query.

Does this approach export cookies or session tokens from Chrome?▼

No. The collection reads only visible rendered DOM content and explicitly forbids inspecting or exporting cookies, local storage, credentials, or profile data. Authentication relies entirely on the user's existing signed-in browser session.

What data fields are captured for each X post?▼

Each row preserves the numeric Tweet ID, canonical status URL, timestamp, exact post text, author handle and display block, engagement aria-label, full card text, query, language hint, and collection time. Normalization also parses replies, reposts, likes, bookmarks, and views.

What are the limitations of collecting X data via DOM scraping?▼

X search is a visible sample, not a complete census, so results depend on sort mode, query syntax, and date slicing. Collection stops on challenge pages, rate limits, or blank timelines, and search-engine snippets must never substitute for original post text.