ai-systems-architect

Start by separating retrieval, reranking, generation, and application layers with explicit contracts between them. Define evaluation datasets and failure taxonomies before launch, and add observability for traces, latency, cost, and answer quality.

Choose the smallest reliable architecture first and only add multi-agent complexity when a single agent demonstrably cannot meet the requirement. Each added agent introduces orchestration, evaluation, and failure-handling overhead that must be justified.

An evaluation plan should include representative datasets, a failure taxonomy, quality metrics, red-teaming scenarios, and guardrail checks. It should be designed before production launch and tied to observable traces of prompts, tool calls, and outputs.

Avoid agents when a deterministic workflow or simple model call satisfies the requirement, since agents add latency, cost, and unpredictable failure modes. Consequential actions also need human-approval boundaries rather than full autonomy.

Use model routing, caching, batching, and streaming to manage inference economics, and track cost and latency per request through observability tooling. Re-evaluate model choices when scale, budget, or quality requirements change.