agentifind-benchmark

What problem does it solve?

This Skill automates the setup and execution of benchmarks to measure the effectiveness of the CODEBASE.md guide for AI agents, identifying improvements in efficiency and accuracy.

Core Features & Use Cases

• Automated Benchmark Setup: Creates necessary hook scripts and configuration files for running parallel agent tests.
• Parallel Agent Execution: Runs two agents simultaneously – one with access to CODEBASE.md and one without – for direct comparison.
• Violation Logging: Tracks and logs any attempts by the agent without the guide to access restricted files, demonstrating the guide's protective role.
• Detailed Metric Collection: Captures tool call counts, file access patterns, and agent turns for comprehensive analysis.
• Use Case: A development team wants to quantify the performance boost provided by their new codebase documentation tool. They use this Skill to run a series of representative coding tasks, comparing how quickly and accurately agents complete them with and without the documentation.

Quick Start

Run the agentifind-benchmark skill to set up the necessary hooks and benchmark templates for your repository.