chat-perf

Benchmarks chat rendering performance and detects memory leaks in VS Code builds.

10|1|Updated Jul 13, 2026
One-click install
npx skills add https://github.com/KevinHuangIsLearning/shortestpath-ide --skill chat-perf-kevinhuangislearning
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: chat-perf
Source: https://github.com/KevinHuangIsLearning/shortestpath-ide/tree/main/.github/skills/chat-perf
Command: npx skills add https://github.com/KevinHuangIsLearning/shortestpath-ide --skill chat-perf-kevinhuangislearning

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires playwright, @vscode/test-electron, and includes scripts (resource) components.

What problem does it solve? Chat UI changes can silently regress rendering performance or introduce memory leaks that only surface after release. This Skill automates benchmarking of the VS Code chat panel against baseline builds and runs memory leak checks, turning subjective performance concerns into statistically validated pass/fail verdicts. ## Core Features & Use Cases - Perf regression testing: Launches VS Code via Playwright Electron, streams mock LLM responses, and compares timing, layout, and rendering metrics against a baseline build using Welch's t-test. - Memory leak detection: Sends sequential messages with forced GC between each, then uses linear regression on heap and DOM node samples to compute per-message growth rates. - Flexible build comparison: Compares local dev builds, production builds, downloaded release versions, or any two commits, with per-build VS Code settings overrides for A/B testing. - Use Case: After modifying chatListRenderer.ts, run the perf regression test with 5 runs against the 1.115.0 release baseline to confirm no statistically significant slowdown before merging. ## Quick Start Run the chat performance regression test comparing my local dev build against the VS Code 1.115.0 baseline with 5 runs and tell me if any metrics regressed.

Frequently Asked Questions about chat-perf

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run a chat performance regression test in VS Code?▼

Run npm run perf:chat with optional flags like --scenario, --runs, and --baseline-build. By default it downloads VS Code 1.115.0 as a baseline, benchmarks both builds via Playwright Electron, and compares metrics using Welch's t-test.

How do I check for memory leaks in the VS Code chat panel?▼

Run npm run perf:chat-leak to send sequential messages in one session with forced GC between each. It uses linear regression on heap and DOM node samples to compute per-message growth, flagging slopes above 2 MB per message as likely leaks.

Can I compare two local VS Code builds for performance?▼

Yes, both --build and --baseline-build accept local paths to VS Code executables, enabling apples-to-apples comparisons between any two dev or production builds. Local path baselines are never cached since the build may change between runs.

Why does the perf test report a regression as likely noise?▼

A metric is only flagged as a regression when it exceeds the threshold and is statistically significant (p < 0.05) under Welch's t-test. Results use IQR-based outlier removal and medians, so noisy runs with high variance are reported as not significant.

What metrics cause the chat perf CI check to fail?▼

Only timeToFirstToken, timeToComplete, layoutDurationMs, forcedReflowCount, and longTaskCount trigger failures when statistically significant. Metrics like layoutCount, recalcStyleCount, and heap deltas are informational only because they are noisy or compositor-driven.