What problem does it solve? Timing MLX models is error-prone because MLX evaluates lazily, so naive timers measure graph construction instead of compute. This Skill enforces a credible benchmarking protocol for mlx-vlm so performance comparisons, regression checks, and upstream PR evidence are trustworthy. ## Core Features & Use Cases - Correct timing discipline: warmup iterations, median-of-N reporting, and mx.eval/mx.synchronize before stopping timers. - Peak-memory protocol: mx.reset_peak_memory/mx.get_peak_memory measurement paired with latency so regressions in memory are caught alongside speed. - A/B across MLX versions: conda + pip workflow for comparing local MLX builds or pinned wheels through check_models, with like-for-like comparison rules. - Use Case: You modified an MLX kernel and need numbers for an upstream PR. Follow the protocol to produce a median-of-7 latency table with peak memory, a correctness assertion, and full environment provenance (chip, RAM, versions) that maintainers will accept. ## Quick Start Ask the assistant to benchmark an MLX model change following the mlx-vlm benchmarking protocol with warmup, median-of-N timing, and peak-memory reporting.
