skills.rest
imsanghaar
imsanghaarCOMMUNITY
Explore all skills by this author →
GitHub

testing-ai-agents

Test AI agent code with pytest, mocking LLM calls via respx.

Updated Aug 23, 2026
Search Tags:#testing#tdd#pytest#fastapi#llm#ai agents#mocking
One-click install
npx skills add https://github.com/imsanghaar/agentfactory --skill testing-ai-agents
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: testing-ai-agents
Source: https://github.com/imsanghaar/agentfactory/tree/main/docs/_skills_archive/warm/testing-ai-agents
Command: npx skills add https://github.com/imsanghaar/agentfactory --skill testing-ai-agents

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pytest, pytest-asyncio, httpx, respx, pytest-cov, pytest-mockllm, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides a comprehensive framework for ensuring the correctness and reliability of AI agent code through robust testing practices, distinguishing between code correctness and LLM reasoning quality.

Core Features & Use Cases

  • TDD for Agent Code: Implement Test-Driven Development specifically for AI agent applications, focusing on deterministic code correctness.
  • Mocking LLM Calls: Effectively mock external LLM API calls using libraries like respx and pytest-mockllm to isolate and test your agent's logic without incurring API costs or latency.
  • Testing Frameworks: Integrates seamlessly with pytest, pytest-asyncio, and httpx for testing FastAPI endpoints, SQLModel operations, and agent tools.
  • Use Case: Ensure your AI agent correctly parses tool calls, interacts with your database as expected, and handles API errors gracefully, all without making live LLM calls during development.

Quick Start

Use the testing-ai-agents skill to set up pytest for your FastAPI application, including mocking LLM calls with respx.

Frequently Asked Questions about testing-ai-agents

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test AI agent code with pytest without making live LLM API calls?▼

You can test AI agent code by mocking LLM calls using respx and pytest-mockllm. This isolates agent logic for deterministic code correctness testing, avoiding API latency and costs during development.

What is the best way to mock LLM API calls in pytest for FastAPI applications?▼

The best way to mock LLM API calls in pytest is using respx and pytest-mockllm. These libraries intercept external HTTP requests, allowing you to test FastAPI endpoints and agent logic deterministically.

Does testing AI agents with pytest focus on LLM reasoning quality or code correctness?▼

Testing AI agents with pytest focuses strictly on deterministic code correctness rather than probabilistic LLM reasoning quality. It validates tool call parsing, database interactions, and API error handling.

Can I use pytest-asyncio and httpx to test FastAPI endpoints for AI agents?▼

Yes, you can use pytest-asyncio and httpx to test FastAPI endpoints for AI agents. This integration supports asynchronous testing patterns and SQLModel operations within your agent applications.

How do I implement Test-Driven Development for AI agent tools and SQLModel operations?▼

Implement TDD for AI agent tools by writing deterministic pytest tests before code. Use pytest-mockllm to mock LLM calls and verify SQLModel database interactions without live API dependencies.

Why should I mock LLM calls when testing AI agent tools instead of using live APIs?▼

Mocking LLM calls when testing AI agent tools ensures deterministic test results, eliminates API latency, and avoids incurring costs. It verifies tool call parsing and error handling without external dependencies.

Related Skills

Explore More Software Engineering Skills→
mifunedevmifunedev
17

Test Backend

Run and verify FastAPI backend tests with pytest and pytest-asyncio.

Official
Intermediate
mvillmowmvillmow
18

quality-coverage-report

Generate HTML and terminal coverage reports for Python pytest runs.

Community
Intermediate
seikaikyoseikaikyo
2

fastapi-patterns

Build FastAPI backends with SQLModel, Neon PostgreSQL, and Render deployment.

Community
Advanced
z1439527767z1439527767

agent-evaluation

Evaluate and benchmark LLM agents for reliability and behavioral quality.

Community
Advanced