ablate-ai-layer

Measures which AI agent instructions still matter by running controlled ablation experiments in git worktrees.

Updated Aug 28, 2026
One-click install
npx skills add https://github.com/az9713/claude-code-hooks-tutorial --skill ablate-ai-layer-az9713
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ablate-ai-layer
Source: https://github.com/az9713/claude-code-hooks-tutorial/tree/main/skills/.claude/skills/ablate-ai-layer
Command: npx skills add https://github.com/az9713/claude-code-hooks-tutorial --skill ablate-ai-layer-az9713

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? AI instruction files like CLAUDE.md and AGENTS.md accumulate rules written for older, weaker models, and reading them cannot reveal which rules are now dead weight competing for context. This Skill answers the question experimentally: it runs the same real coding task many times with the instruction layer intact and with it stripped, then grades every rule against what actually changed. ## Core Features & Use Cases - Layer Mapping: Inventories every AI instruction artifact across Claude Code, Cursor, Windsurf, Cline, Copilot, and AGENTS.md, classifying each as always-loaded, on-demand, or enforcement, and reports the per-session token cost. - Automated Ablation Runs: Executes control and stripped arms in throwaway git worktrees built from HEAD, never touching the working tree, with configurable run counts, models, and concurrency. - Blind Grading Rubric: Provides a reference rubric for grading each rule per run as followed, violated, or not applicable, blind to which arm produced each diff. - Use Case: A team suspects their 400-line CLAUDE.md is bloated. The Skill maps the layer, runs a real feature task twice per arm, and produces a per-rule table showing which rules are load-bearing, which the model follows anyway, and which are ignored even when loaded. ## Quick Start Ask the agent to ablate the AI layer of this repository and tell you which CLAUDE.md rules can be safely deleted.

Frequently Asked Questions about ablate-ai-layer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test whether my CLAUDE.md rules are still needed?▼

Run an ablation experiment: execute the same real coding task multiple times with CLAUDE.md present and with it removed, then grade each rule against the resulting diffs. Rules the model follows in both arms have expired; rules only followed in the control arm are load-bearing.

What is AI layer ablation for coding agents?▼

AI layer ablation is an experiment that strips a repository's agent instruction files and compares agent output against a control arm. It reveals which instructions still change behavior and which have become dead weight as models improved.

Does ablation testing work with Cursor rules and AGENTS.md?▼

Yes, the layer mapping is agent-agnostic and detects CLAUDE.md, AGENTS.md, .cursor/rules, .windsurfrules, .clinerules, and copilot-instructions files. A --runner flag supports non-Claude agents that read a prompt on stdin.

Will running an ablation experiment modify my working tree?▼

No, every run executes in a detached git worktree built from HEAD in a temporary directory outside the repo and deleted afterwards. Uncommitted changes are never touched, though they are also not part of the experiment.

Why do ablation results differ between new files and edited files?▼

Existing code substitutes for the rules file: a stripped agent copies conventions from neighboring code in edited files but falls back to defaults in new files. Grade new-file evidence more heavily when deciding whether a rule is load-bearing.

When should I not delete a rule after an ablation experiment?▼

Never delete rules the probe task never exercised, such as security or release procedures, since an untested rule looks identical to a useless one in the data. Also confirm results with a second task before deleting anything large.