skills.rest
docktermj
docktermjCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

auto-test

Probes the Senzing MCP server for drift and runs sandboxed simulated bootcamp walks.

Updated Jul 13, 2026
Search Tags:#automation#testing#drift detection#mcp#regression#claude plugin#senzing
One-click install
npx skills add https://github.com/docktermj/senzing-bootcamp-claude-plugin-development --skill auto-test-docktermj
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: auto-test
Source: https://github.com/docktermj/senzing-bootcamp-claude-plugin-development/tree/main/.claude/skills/auto-test
Command: npx skills add https://github.com/docktermj/senzing-bootcamp-claude-plugin-development --skill auto-test-docktermj

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? The Senzing Bootcamp plugin depends on a live MCP server that can change without notice, and manually dry-running the bootcamp to catch regressions is slow and expensive. This Skill automates both checks: it detects server drift and contract breakage for free, and optionally walks the bootcamp with a simulated Bootcamper. ## Core Features & Use Cases - MCP drift and conformance probing: Talks JSON-RPC directly to the Senzing MCP server with zero tokens, diffing live schemas and prose-declared value sets against a committed baseline, and verifying every tool call the plugin documents. - Simulated bootcamp walks: Drives two independent Claude processes (guide and Bootcamper persona) in an isolated git-worktree sandbox, then lints the transcript against interaction invariants. - CI/cron-ready reporting: Exits 1 on BREAKING findings and writes machine-readable JSON reports, so it drops into scheduled jobs unchanged. - Use Case: A maintainer schedules the MCP probe every four hours to catch server-side breaking changes, and runs a nightly walk with rotating personas (terse, confused, impatient) as a regression net before releases. ## Quick Start Run an automated sandboxed test of the Senzing Bootcamp plugin and report any MCP server drift or breaking changes.

Frequently Asked Questions about auto-test

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test a Claude Code plugin against a live MCP server?▼

Run autotest.py to probe the MCP server over JSON-RPC with zero tokens, diffing live tool schemas against a committed baseline. Add --walk to also drive a two-process simulated bootcamp walk and lint the transcript for invariant violations.

How do I detect MCP server drift on a schedule?▼

Run the MCP-only check from cron, for example every four hours with --json appended to a log. The probe exits 1 on BREAKING findings, so it drops into CI unchanged, and mcp_probe.py update refreshes the baseline after deliberately accepting a server change.

Why does the MCP conformance check probe values instead of reading JSON-Schema enums?▼

The Senzing server declares no JSON-Schema enums, so closed value sets exist only in description prose. The probe sends a deliberately invalid value and parses the rejection message, which names the accepted values, then verifies suspicious literals with an actual call.

Can a clean automated walk prove the bootcamp works?▼

No. A simulated Bootcamper is more cooperative than a real one, so findings are trustworthy but a clean run is weak evidence. The report states coverage limits explicitly, and a human dry run is still required.

What safety guardrails does the automated test enforce?▼

submit_feedback and download_resource are blocked by flag rather than instruction, the sandbox never lives under /tmp, each run pins the plugin to a git commit, and the guide process is never told it is under test.

Related Skills

Explore More Software Engineering Skills→
docktermjdocktermj

implement-spec

Implement developer-defined specifications with live Senzing MCP server verification.

Community
Advanced
nicorevonicorevo

debugging-and-error-recovery

Diagnose test failures, build errors, and runtime bugs through structured root-cause triage.

Community
Intermediate
zts212653zts212653
2.6k

bootcamp-guide

Coordinate structured bootcamp phases for collaborative multi-agent training.

Community
Advanced
VilnaCRM-OrgVilnaCRM-Org
1

bats-fixture-full-validation

Validates edited Bats helpers and stubs by running the full test suite and ShellCheck.

Official
Intermediate