skills.rest
KevinHuangIsLearning
KevinHuangIsLearningCOMMUNITY
Explore all skills by this author →
GitHub

smoke-tests

Runs and debugs VS Code end-to-end smoke tests across Electron, web, and remote suites.

10|1|Updated Jul 13, 2026
Search Tags:#github actions#smoke tests#flaky tests#end-to-end testing#azure devops#vs code#ci debugging
One-click install
npx skills add https://github.com/KevinHuangIsLearning/shortestpath-ide --skill smoke-tests-kevinhuangislearning
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: smoke-tests
Source: https://github.com/KevinHuangIsLearning/shortestpath-ide/tree/main/.github/skills/smoke-tests
Command: npx skills add https://github.com/KevinHuangIsLearning/shortestpath-ide --skill smoke-tests-kevinhuangislearning

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Running VS Code smoke tests involves knowing the right npm scripts, grep filters, and CI artifact layouts, and flaky CI failures are hard to reproduce without a structured approach. ## Core Features & Use Cases - Test execution guidance: Covers npm run smoketest and smoketest-no-compile, grep filtering with -g, and options like --tracing, --web, --remote, and --build. - Flaky test hunting: Provides a temporary repeat-loop technique for GitHub Actions and Azure DevOps workflows to reproduce intermittent CI failures with fail-fast behavior. - CI log analysis: Explains how to download smoke test log artifacts via gh run download or az pipelines runs artifact download and where to find runner logs, Copilot runtime logs, and Playwright screenshots. - Use Case: A smoke test fails intermittently only in CI. Add the temporary loop step to the PR workflow, push a throwaway branch, reproduce the failure, then download the logs artifact and inspect the Copilot runtime logs to find the root cause. ## Quick Start Run the smoke tests filtered to a specific suite with tracing enabled using npm run smoketest with the -g flag.

Frequently Asked Questions about smoke-tests

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run VS Code smoke tests?▼

Run npm run smoketest to compile and execute the smoke tests, or npm run smoketest-no-compile to run already-compiled tests. Pass extra arguments after -- to the runner, such as -g to filter suites by title.

How do I run only one smoke test suite?▼

Use the -g option with a substring of the suite title, for example npm run smoketest -- -g "Agents Window" --tracing. The pattern matches mocha suite titles, so one substring can select multiple related suites.

How do I reproduce a flaky smoke test that only fails in CI?▼

Add a temporary loop step to the CI workflow that runs the suspect suite around 20 times and exits on the first failure. Push it on a throwaway branch, iterate until the flake reproduces, and remove the loop before merging.

How do I download smoke test logs from CI?▼

For GitHub Actions use gh run download with the run id and artifact name like logs-macos-arm64-electron-1. For Azure DevOps use az pipelines runs artifact download with the build id and artifact name.

Why did my smoke test time out waiting for a Copilot response?▼

Check the Copilot runtime process logs under the suite's copilot-runtime-logs directory in the logs artifact. They record startup, auth, model requests, and protocol errors that explain timeouts the test error alone does not.

What is the difference between smoke tests and unit tests in VS Code?▼

Smoke tests in test/smoke drive a full VS Code instance through end-to-end UI flows via npm run smoketest. Unit tests are .test.ts files run through scripts/test.sh, and integration tests use scripts/test-integration.sh.

Related Skills

Explore More Software Engineering Skills→
mako-aimako-ai
11

build-desktop-app

Automate build, test, packaging, and release workflows for Mako Desktop and Local Agent.

Official
Advanced
TyrizxTyrizx

smoke-tests

Runs VS Code end-to-end smoke tests and diagnoses flaky CI failures.

Official
Intermediate
namhoanggnamhoangg

run-smoke-tests

Automate end-to-end smoke tests and diagnose failures with traces and logs.

Community
Intermediate
posit-devposit-dev
4.2k

integration-tests

Runs VS Code integration tests via test-integration scripts with filtering options.

Official
Intermediate