cost-control-token-review

Start by instrumenting per-request token logging for input, output, and cached tokens, then rank request types by cost. Apply prompt caching to stable prefixes, route simple tasks to cheaper models, batch latency-tolerant jobs, and trim context, validating quality before any model downgrade.

Verify the cached prefix is at least 1,024 tokens, place the cache breakpoint after stable content and before variable content, and move timestamps or user names out of the system prompt. A cache hit rate below 50% for repetitive workloads indicates a configuration problem.

Use the Batch API for workloads where results are consumed hours after submission, such as nightly document processing, bulk evaluations, and offline classification. Both Anthropic and OpenAI Batch APIs offer roughly 50% discounts with no quality change.

Orchestrator token counts hide the true cost because worker fanout and synthesis multiply tokens per user query. Calculate total tokens end-to-end across orchestrator, all workers, and synthesis; actual cost can be 20x or more above the orchestrator-only estimate.

Skip it for one-time prototyping expenses, when monthly API cost is below roughly $100, or when the real problem is output quality rather than spend. GPU or self-hosted compute costs belong to a cloud spend review, not token optimization.