skills.rest
Niiihuel
NiiihuelCOMMUNITY
Explore all skills by this author →
GitHub

smoke-tests

Runs VS Code smoke tests and diagnoses flaky CI failures with tracing and log artifacts.

1|Updated Aug 27, 2026
Search Tags:#smoke tests#flaky tests#playwright#end-to-end testing#vs code#mocha#ci debugging
One-click install
npx skills add https://github.com/Niiihuel/openide --skill smoke-tests-niiihuel
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: smoke-tests
Source: https://github.com/Niiihuel/openide/tree/main/vscode/.github/skills/smoke-tests
Command: npx skills add https://github.com/Niiihuel/openide --skill smoke-tests-niiihuel

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Running VS Code end-to-end smoke tests and debugging intermittent CI failures is error-prone: developers must know the right npm scripts, grep filters, build flags, and where CI publishes per-platform logs. This Skill consolidates those commands and debugging workflows into one reference. ## Core Features & Use Cases - Smoke test execution: Run npm run smoketest or smoketest-no-compile with grep filters, --tracing, --web, --remote, and --build options for Electron, browser, and remote suites. - Flaky test hunting: Apply a temporary repeat-loop technique in GitHub Actions or Azure DevOps YAML to reproduce intermittent CI failures and capture traces on the first failure. - CI log analysis: Download and interpret smoke test log artifacts (smoke-test-runner.log, Copilot runtime logs, extension-host logs, Playwright screenshots) from both GitHub Actions and Azure DevOps. - Use Case: A smoke test fails only in CI on macOS. Use this Skill to add a temporary 20-iteration loop to the PR workflow, reproduce the flake, download the logs-macos-arm64-electron-1 artifact, and inspect the Copilot runtime logs to find the timeout cause. ## Quick Start Ask the assistant to run only the Agents Window smoke test suite with tracing enabled against the current build.

Frequently Asked Questions about smoke-tests

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run VS Code smoke tests?▼

Run npm run smoketest to compile and execute the smoke tests, or npm run smoketest-no-compile to run already-compiled tests. Pass extra arguments after -- to the runner, such as -g to filter suites by title.

How do I run only one smoke test suite?▼

Use the -g (or -f) grep option with a suite title substring, for example npm run smoketest -- -g "Agents Window" --tracing. The pattern matches mocha test and suite titles, so a substring can select multiple related suites.

How do I debug a flaky smoke test that only fails in CI?▼

Add a temporary loop step to the CI workflow that runs the suspect suite 20 times and exits on the first failure, with a generous timeout. Push it on a throwaway branch, reproduce the flake, then remove the loop before merging.

Where are VS Code smoke test logs in CI?▼

Both GitHub Actions and Azure DevOps publish the .build/logs directory as a downloadable artifact named logs-<os>-<arch>-<suite>-<attempt>. Download it with gh run download or az pipelines runs artifact download, then inspect smoke-test-runner.log and per-suite Copilot runtime logs.

What is the difference between smoke tests, unit tests, and integration tests in VS Code?▼

Smoke tests in test/smoke/ drive a full VS Code instance through end-to-end UI flows via npm run smoketest. Unit tests (.test.ts) run via scripts/test.sh, and integration tests (.integrationTest.ts) run via scripts/test-integration.sh.

Related Skills

Explore More Software Engineering Skills→
hodorhodor

build

Compile source files and run unit tests with a single command.

Community
Intermediate
mvillmowmvillmow
18

fix-ci-failures

Diagnose CI failures by inspecting logs and reproducing issues locally.

Community
Advanced
namhoanggnamhoangg

run-smoke-tests

Automate end-to-end smoke tests and diagnose failures with traces and logs.

Community
Intermediate
jimehjimeh

smoke-tests

Runs VS Code end-to-end smoke tests and diagnoses flaky CI failures.

Community
Intermediate