video-understand

Extract key frames and transcribe audio from video files using ffmpeg and Whisper.

1|Updated Aug 28, 2026
One-click install
npx skills add https://github.com/mooch10/mochiptos --skill video-understand-mooch10
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: video-understand
Source: https://github.com/mooch10/mochiptos/tree/main/frontend/.agents/video-understand
Command: npx skills add https://github.com/mooch10/mochiptos --skill video-understand-mooch10

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires openai-whisper, and includes scripts (resource) and references (resource) components.

What problem does it solve? Understanding what a video contains normally requires watching it manually or paying for cloud transcription APIs. This Skill analyzes video files entirely on your local machine, extracting representative frames and transcribing speech without any API keys or external services. ## Core Features & Use Cases - Frame Extraction: Pull key frames using scene-change detection, codec keyframes, or fixed intervals, with automatic subsampling to a configurable frame limit. - Local Audio Transcription: Convert speech to timestamped text using OpenAI Whisper running locally, with selectable model sizes from tiny to large. - Structured JSON Output: Receive video metadata, frame paths with timestamps, and transcript segments in a single machine-readable JSON document. - Use Case: Given a screen recording of a product demo, extract scene-change frames and a full transcript so an AI can visually inspect the frames and summarize the demo content without watching the video. ## Quick Start Analyze the video file demo.mp4 by extracting its key frames and transcribing its audio locally.

Frequently Asked Questions about video-understand

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I transcribe a video file locally without an API key?▼

Run the understand_video.py script on your video file, which extracts audio with ffmpeg and transcribes it using the openai-whisper package running entirely on your machine. No cloud API keys are required, and you can choose Whisper model sizes from tiny to large.

How to extract key frames from a video with ffmpeg?▼

The script offers three extraction modes: scene mode detects scene changes using ffmpeg's scene threshold filter, keyframe mode extracts codec I-frames, and interval mode samples frames at evenly spaced timestamps. Scene mode falls back to interval mode when no scene changes are detected.

What are the requirements for running local video transcription?▼

You need ffmpeg and ffprobe installed on your system, which are mandatory for frame extraction and audio processing. Transcription additionally requires the openai-whisper Python package or the whisper CLI, installable via pip.

Can I analyze YouTube videos directly with this tool?▼

No, YouTube URLs are explicitly rejected by the script. You must first download the video using a separate download tool, then run the analysis script on the downloaded local file.

Why is my video transcription output null?▼

The transcript and text fields are null when the --no-transcribe flag is used, when Whisper is not installed, or when the video has no audio track. Install openai-whisper and ensure the video contains an audio stream to get transcription results.