testing-strategies

Write Tier 2 integration tests by building a workflow with WorkflowBuilder and executing it with LocalRuntime against real infrastructure like a Docker PostgreSQL container. Do not mock databases or SDK nodes, and use the runtime context manager for proper resource cleanup.

The 3-tier strategy splits tests into Tier 1 unit tests (fast, mocking allowed, about 70% of tests), Tier 2 integration tests (real databases and APIs, about 25%), and Tier 3 end-to-end tests (full user flows over real HTTP, about 5%).

No, mocking databases, HTTP calls, or LLM providers is forbidden in Tiers 2-3 because it hides real issues like constraint violations, timeouts, and race conditions. Use Docker-based test databases and cached LLM responses instead.

Use the shared parametrized fixtures in tests/shared/runtime/ so the same test runs once with LocalRuntime and once with AsyncLocalRuntime. The execute_runtime helper normalizes parameters and return structures between the two runtimes.

Start the Docker test services with the test-env script in tests/utils, which provides PostgreSQL on port 5433, Redis on 6380, MinIO on 9001, and Elasticsearch on 9201. Then run pytest against the integration test directory with appropriate timeouts.

LocalRuntime requires use as a context manager in the current version, so wrap execution in a with block to guarantee cleanup. Also ensure fixtures truncate test tables and tear down database connections after each test.