mine-recording

Extract workflow steps, field labels, and business logic from screenshare walkthrough videos.

Updated Aug 19, 2026
One-click install
npx skills add https://github.com/nperepichka/Antigravity --skill mine-recording-nperepichka
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: mine-recording
Source: https://github.com/nperepichka/Antigravity/tree/main/config/skills/mine-recording
Command: npx skills add https://github.com/nperepichka/Antigravity --skill mine-recording-nperepichka

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? Customer walkthrough recordings (Loom, Meet, Zoom captures) are often the only ground truth for a system you cannot access directly, but manually scrubbing hours of video to document screens, fields, and spoken business rules is slow and error-prone. This Skill turns a recording into a buildable spec by joining extracted UI frames with a timestamped transcript. ## Core Features & Use Cases - Frame Extraction with Crop & Scene Detection: Probe video geometry, crop out webcam overlays, and extract distinct UI frames using ffmpeg scene detection tuned for form-heavy screens (threshold 0.02). - Two-Tier Local Transcription: Transcribe the full recording locally with Whisper (base.en), then re-run targeted segments with larger models to verify domain terminology. - Master Index Generation: Join frames and transcript into a timecoded Markdown index for writing findings, vocabulary files, and question lists. - Use Case: You receive a 90-minute Zoom recording of a customer demoing an insurance portal. Run probe, frames, transcript, and index to produce a findings document with navigation paths, field inventories, branch points, and traps — ready to feed a live capture session for selectors. ## Quick Start Ask the agent to mine the walkthrough video at a given path by running the probe, frames, transcript, and index commands with a run id, then review the generated index under .local/recordings.

Frequently Asked Questions about mine-recording

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract workflow steps from a screen recording video?▼

Run the probe command to sample stills and pick a crop, then extract distinct UI frames with ffmpeg scene detection, transcribe audio locally with Whisper, and generate a master index joining frames to transcript timestamps. Review the frames the index points at to write findings.

What ffmpeg scene detection threshold works for form walkthrough videos?▼

Start at 0.02 rather than ffmpeg's default 0.3, because portal forms barely change the whole-frame score. Testing showed 0.08 missed entire wizard pages while 0.02 captured about 9 frames per 180 seconds, roughly 300 frames for a 1.5-hour call.

Can I transcribe a recording locally without cloud services?▼

Yes, the pipeline uses whisper-cli (whisper.cpp) with models under ~/whisper-models/ or Python whisper/faster-whisper. Run a fast base.en pass over the whole video, then re-transcribe targeted segments with medium.en or large-v3-turbo to verify domain terms.

Why crop the video before running scene detection?▼

Call recordings pin a webcam feed in a corner, and a talking head moves far more than a form does, so full-frame detection produces thousands of useless cuts. Probe the geometry first and crop out the webcam, participant strip, and browser chrome.

Can a walkthrough video provide CSS selectors for automation?▼

No, a video only provides labels, flow, and vocabulary — never DOM ids or option values. Mine the recording first to learn the flow, then use a live browser capture tool like rpa-capture to obtain exact stable selectors.