skills.rest
adriancodes
adriancodesCOMMUNITY
Explore all skills by this author →
GitHub

ecc-ai-regression-testing

Write regression tests for bugs found in AI-generated API and frontend code.

Updated Apr 18, 2025
Search Tags:#vitest#test automation#next.js#regression testing#api testing#ai code review#sandbox mode
One-click install
npx skills add https://github.com/adriancodes/dotfiles --skill ecc-ai-regression-testing-adriancodes
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ecc-ai-regression-testing
Source: https://github.com/adriancodes/dotfiles/tree/main/dot_agents/skills/ecc-ai-regression-testing
Command: npx skills add https://github.com/adriancodes/dotfiles --skill ecc-ai-regression-testing-adriancodes

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? When the same AI model writes code and then reviews it, it carries identical assumptions into both steps, creating systematic blind spots where bugs survive review. This Skill provides testing patterns that catch these recurring AI regressions with automated tests instead of trusting self-review. ## Core Features & Use Cases - Sandbox-Mode API Testing: Set up Vitest with Next.js App Router to test API routes without a database by forcing sandbox mode. - Regression Pattern Catalog: Covers common AI regression patterns including sandbox/production path mismatch, SELECT clause omission, error state leakage, and missing optimistic-update rollback. - Bug-Driven Test Strategy: Write tests only where bugs were found, naming tests after the defect they prevent (e.g., "BUG-R1 regression") rather than chasing coverage percentages. - Use Case: After an AI assistant fixes a missing notification_settings field in a profile API, write a contract test asserting all required fields exist in both sandbox and production response paths so the same omission cannot recur. ## Quick Start Ask the AI to write a regression test for the bug just fixed in your API route, asserting the response contains all required fields in both sandbox and production paths.

Frequently Asked Questions about ecc-ai-regression-testing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I write regression tests for AI-generated code?▼

Write tests immediately after finding a bug, targeting the exact defect rather than general coverage. Assert the API response contract (required fields, types) and name the test after the bug it prevents, such as "BUG-R1 regression".

How to test Next.js API routes without a database?▼

Use Vitest with sandbox mode forced via environment variables in the setup file, then call route handlers directly with a NextRequest built by a test helper. This runs route logic against mock data without any database connection.

Why does AI code review miss bugs it introduced?▼

The same model carries identical assumptions into both writing and reviewing, so it validates its own blind spots. Automated tests break this loop because they execute the code against explicit contracts instead of reasoning about it.

What is sandbox/production path mismatch in API testing?▼

It occurs when a fix updates the production code path but forgets the sandbox or mock path, so the two return different response shapes. Catch it by asserting both paths return the same required fields.

Should I aim for high test coverage with AI-assisted development?▼

No. The recommended strategy is bug-driven testing: write tests only where bugs were actually found, since AI tends to repeat the same category of mistake. Test count grows organically with fixes without wasted effort.

Related Skills

Explore More Software Engineering Skills→
Sma1lboySma1lboy
136

tdd

Writes failing regression tests before fixing bugs with clear test paths.

Community
Intermediate
dtabordadtaborda

contract-testing

Validate JSON contract fixtures against Zod schemas with Vitest.

Community
Intermediate
ArthTech-AIArthTech-AI

qa

Run full-stack QA checks with pytest, Jest, ruff, TypeScript, and Playwright.

Official
Intermediate
UnicornUnicorn

e2e-test-agent

Generate end-to-end test suites for web and full-stack applications.

Official
Advanced