skills.rest
az0512124155azz-sys
az0512124155azz-sysCOMMUNITY
Explore all skills by this author →
GitHub

smoke-tests

Runs VS Code end-to-end smoke tests and diagnoses flaky CI failures.

Updated Sep 3, 2026
Search Tags:#smoke tests#flaky tests#playwright#end-to-end testing#vs code#mocha#ci debugging
One-click install
npx skills add https://github.com/az0512124155azz-sys/CodeForge --skill smoke-tests-az0512124155azz-sys
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: smoke-tests
Source: https://github.com/az0512124155azz-sys/CodeForge/tree/main/.github/skills/smoke-tests
Command: npx skills add https://github.com/az0512124155azz-sys/CodeForge --skill smoke-tests-az0512124155azz-sys

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Running and debugging VS Code smoke tests involves knowing the right npm scripts, grep filters, and CI artifact layouts, and intermittent CI-only failures are especially hard to reproduce and diagnose without a systematic approach. ## Core Features & Use Cases - Smoke Test Execution: Run full end-to-end UI flows with npm run smoketest or smoketest-no-compile, filtering suites with mocha grep patterns and options like --tracing, --web, --remote, and --build. - Flaky Test Hunting: Apply a temporary repeat-loop technique in GitHub Actions or Azure DevOps pipelines that reruns a suspect suite many times and fails fast to capture traces and screenshots of intermittent failures. - CI Failure Diagnosis: Download and interpret smoke test log artifacts from GitHub Actions (gh run download) or Azure DevOps (az pipelines runs artifact download), including runner logs, Copilot runtime logs, extension-host logs, and Playwright screenshots. - Use Case: A smoke test fails only in CI on macOS. Add a temporary 20-iteration loop step to the PR workflow, reproduce the failure, download the logs-macos-arm64-electron-1 artifact, and inspect the Copilot runtime logs to find the timeout cause. ## Quick Start Run the smoke tests filtered to the Agents Window suite with tracing enabled using npm run smoketest.

Frequently Asked Questions about smoke-tests

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run VS Code smoke tests?▼

Run npm run smoketest to compile and execute the smoke tests, or npm run smoketest-no-compile to run already-compiled tests. Pass extra arguments after -- to the runner, such as -g to grep-filter suites or --tracing to capture Playwright traces.

How do I run only one smoke test suite?▼

Use the -g option with a substring of the suite title, for example npm run smoketest -- -g "Agents Window" --tracing. The pattern is a mocha grep filter, so it matches every suite whose title contains that substring.

How do I reproduce a flaky smoke test that only fails in CI?▼

Add a temporary loop step to the CI workflow that runs the suspect suite around 20 times and exits on the first failure, preserving traces and screenshots. Push it on a throwaway branch and remove the loop before merging.

How do I download smoke test logs from GitHub Actions or Azure DevOps?▼

For GitHub Actions use gh run download <run-id> -n logs-<os>-<arch>-<suite>-<attempt>. For Azure DevOps use az pipelines runs artifact download with the run id and artifact name, which varies between product-build and suite-split CI pipelines.

Why does a Copilot smoke test time out waiting for a response?▼

Check the Copilot runtime process logs under the suite's copilot-runtime-logs directory in the logs artifact first. They record startup, auth, model requests, and protocol errors that explain timeouts the test error alone does not.

What is the difference between smoke tests, unit tests, and integration tests in VS Code?▼

Smoke tests in test/smoke/ drive a full VS Code instance through end-to-end UI flows via npm run smoketest. Unit tests (.test.ts) run through scripts/test.sh, and integration tests (.integrationTest.ts) run through scripts/test-integration.sh.

Related Skills

Explore More Software Engineering Skills→
microsoftmicrosoft
190.1k

smoke-tests

Run and debug VS Code end-to-end smoke tests across Electron, web, and remote platforms.

Official
Intermediate
SavigeSystemZSavigeSystemZ

etherweave-debug-playbook

Diagnose and guide fixes for EtherWeave build, runtime, and UI failures.

Official
Advanced
namhoanggnamhoangg

run-smoke-tests

Automate end-to-end smoke tests and diagnose failures with traces and logs.

Community
Intermediate
posit-devposit-dev
4.2k

smoke-tests

Runs and debugs VS Code end-to-end smoke tests across Electron, web, and remote CI environments.

Official
Intermediate