skills.rest
JohnRebellion
JohnRebellionCOMMUNITY
Explore all skills by this author →
GitHub

ai-regression-testing

Writes sandbox-mode API regression tests to catch AI-introduced code regressions.

Updated Apr 18, 2026
Search Tags:#vitest#next.js#regression testing#api testing#ai code review#bug-check#sandbox mode
One-click install
npx skills add https://github.com/JohnRebellion/.claude-public --skill ai-regression-testing-johnrebellion
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ai-regression-testing
Source: https://github.com/JohnRebellion/.claude-public/tree/main/claude/skills/ai-regression-testing
Command: npx skills add https://github.com/JohnRebellion/.claude-public --skill ai-regression-testing-johnrebellion

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? When an AI writes code and then reviews its own work, it carries the same assumptions into both steps, creating systematic blind spots where bugs survive review. This Skill provides testing patterns that mechanically catch AI-introduced regressions—especially sandbox/production path mismatches—without requiring a database. ## Core Features & Use Cases - Sandbox-Mode API Testing: Configure Vitest with Next.js App Router to test API routes with mocked sandbox mode, eliminating database dependencies. - Regression Test Patterns: Ready-to-use test templates for the most common AI regressions: SELECT clause omissions, sandbox/production response mismatches, error state leakage, and missing optimistic-update rollbacks. - Bug-Check Workflow Integration: A structured workflow that runs automated tests and builds before any AI code review, then writes a regression test for every bug fixed. - Use Case: After an AI assistant fixes an API bug, run the bug-check workflow; the test suite immediately flags if the fix was applied to the production path but forgotten in the sandbox path. ## Quick Start Ask the AI to write a sandbox-mode regression test for the API endpoint where a bug was just fixed, then run the test suite before any code review.

Frequently Asked Questions about ai-regression-testing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test Next.js API routes without a database?▼

Use sandbox mode with Vitest by setting environment variables like SANDBOX_MODE=true in a setup file, then call route handlers directly with a NextRequest test helper. This runs API tests in under a second with no database connection.

How to catch regressions when AI writes and reviews code?▼

Write automated tests for every bug found, naming them after the bug they prevent, and run tests as the mandatory first step of any review workflow. AI self-review repeats the same blind spots, so only mechanical tests reliably catch regressions.

What is the most common AI-introduced regression pattern?▼

Sandbox/production path mismatch is the most frequent pattern, where a fix is applied to one code path but not the other. Tests that assert both paths return the same response shape catch this immediately.

Does Vitest work with Next.js App Router route handlers?▼

Yes, Vitest can import and invoke GET, POST, and other route handlers directly by constructing NextRequest objects. Configure the node environment and path aliases in vitest.config.ts to resolve imports like @/app/api routes.

Should I aim for high test coverage with AI-generated code?▼

No, write tests only where bugs were actually found rather than targeting coverage percentages. AI tends to repeat the same category of mistake, so bug-driven tests prevent real regressions without wasted effort.

Related Skills

Explore More Software Engineering Skills→
akulkarniakulkarni
14

add-backend-testing

Configure Vitest and an isolated test database schema for tRPC router integration tests.

Community
Intermediate
excattexcatt
14

pytest-runner

Executes Python test suites with pytest and generates detailed coverage reports.

Community
Advanced
gofhirgofhir
2

go-code-review

Analyze Go source code with golangci-lint and go vet for bugs and race conditions.

Official
Intermediate
VYRE-StudiosVYRE-Studios

webapp-testing

Automate end-to-end web testing across browsers and APIs with Playwright or Cypress.

Official
Intermediate