perform-agent-builder-eval

What problem does it solve?

This Skill automates the setup, execution, and teardown of agent-builder evaluation environments, simplifying the process of testing and benchmarking AI agents.

Core Features & Use Cases

• Environment Orchestration: Initializes and stops the necessary Elasticsearch, Kibana, and EDOT stack components.
• Parameter Collection: Gathers essential evaluation parameters, including GCS credentials, snapshot selection, and connector configurations.
• Command Generation: Outputs the precise command needed to initiate the evaluation run in a separate terminal.
• Use Case: A machine learning engineer needs to evaluate a new retrieval-augmented generation (RAG) model against a specific dataset. This Skill sets up the required backend services, prompts the engineer for necessary inputs, and provides the exact command to start the evaluation.

Quick Start

Use the perform-agent-builder-eval skill with the 'init' argument to set up the evaluation environment.