What problem does it solve? Building reinforcement learning environments for agent training requires correctly implementing the Atropos BaseEnv interface, wiring up multi-turn agent loops with tool calling, and writing reward functions that score rollouts — a process full of subtle pitfalls like misreading AgentResult fields or polluting training metrics during evaluation. ## Core Features & Use Cases - Environment Implementation Guide: Covers the six required methods (setup, get_next_item, format_prompt, compute_reward, evaluate, wandb_log) with correct code patterns for the HermesAgentBaseEnv interface. - Reward Function Patterns: Provides LLM-judge, binary verification via ToolContext sandbox execution, and multi-signal weighted scoring approaches. - Three CLI Modes: Documents serve (live training), process (offline JSONL data generation), and evaluate (standalone benchmarking) workflows with provider-specific flags for OpenRouter, VLLM, and OpenAI-compatible APIs. - Use Case: You are creating a new RL environment that trains an agent to answer research questions using tools. Use this Skill to scaffold the environment class, implement a reward function that verifies answers in the agent's sandbox, and run a process-mode smoke test before launching training. ## Quick Start Ask the agent to create a new Atropos RL environment in the hermes-agent repo that implements the HermesAgentBaseEnv interface with a reward function and run it in process mode for one test item.
