caveman-setup

Wire repository LLM callsites through the Caveman gateway for spend measurement.

1|Updated Apr 23, 2026
One-click install
npx skills add https://github.com/tkogut/agents-os-core --skill caveman-setup-tkogut
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: caveman-setup
Source: https://github.com/tkogut/agents-os-core/tree/main/vault/.agents/skills/caveman-setup
Command: npx skills add https://github.com/tkogut/agents-os-core --skill caveman-setup-tkogut

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Teams shipping LLM-powered features often have no visibility into what each request costs or which callsites generate spend. This Skill integrates a repository with the Caveman gateway so every LLM request is measured and priced, without changing application behavior. ## Core Features & Use Cases - Callsite Discovery: Scans dependency files and source code for LLM SDKs (OpenAI, Anthropic, Vercel AI SDK, LangChain, LiteLLM) and raw provider HTTP calls before making any change. - Gateway Wiring: Redirects each callsite's base URL through the gateway path /w/<app> with the x-cave-api-key header, supporting both stored provider keys and bring-your-own-key (byok) modes. - Verified Integration: Sends one real minimal verification request and reports the actual HTTP status and token usage, with verbatim failure templates for unreachable gateways, invalid keys, or routing errors. - Use Case: A developer with a Python support bot using the OpenAI SDK runs this Skill to route all calls through the gateway, then sees per-request costs grouped under the app's slug on the Caveman dashboard. ## Quick Start Ask the agent to set up Caveman in this repository using the provided gateway URL, API key, provider key mode, and dashboard URL.

Frequently Asked Questions about caveman-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add LLM spend tracking to an existing repository?▼

Run this Skill with a gateway URL, API key, provider-key mode, and dashboard URL. It finds every LLM callsite, repoints each base URL through the gateway path /w/<app> with an x-cave-api-key header, and verifies with one real request.

Which LLM SDKs does the Caveman gateway integration support?▼

Published wiring recipes cover OpenAI (TypeScript and Python), Anthropic, Vercel AI SDK, LangChain/LangGraph, LiteLLM, and raw HTTP calls. Frameworks like google-genai, crewai, and pydantic-ai are handled via matching integration docs.

Does routing through the gateway change my app's LLM behavior?▼

No. Record mode is byte-preserving: it measures requests and costs while changing nothing else. No optimization is enabled, and verified savings remain $0 until an optimizer is explicitly turned on and passes its eval gate.

Where should the Caveman API key be stored?▼

The key goes only into the repo's existing env file (such as .env or .env.local), which must be gitignored. It is never hardcoded in source, never committed, and never printed in full.

What happens if the gateway verification request fails?▼

The Skill reports a verbatim failure template matching the error: unreachable gateway, 401 invalid API key, 404 route not found, or an upstream provider error. An unverified integration is always reported as unverified, never as success.