test-qa

Start with test case design questions that require candidates to structure answers by dimensions like function, boundary, exception, security, and performance before listing cases. Then probe automation framework depth, performance testing process, and quality metric experience with follow-ups on real numbers.

Ask about automation framework layering, flaky test governance, test data isolation, and code reading ability. For experienced candidates, require concrete data such as case counts, nightly failure rates, real defects found, and platform adoption numbers.

Ask them to walk through a full load test: scenario modeling, data preparation, monitoring metrics, and bottleneck triage order when error rates spike. Strong candidates mention connection pools, thread pools, GC, and the risk that the load generator itself becomes the bottleneck.

Red flags include listing test cases without dimensional structure, frameworks that are just script piles, treating code coverage as a quality goal, and inability to cite real numbers like failure rates or defects found. These indicate lack of structured thinking or unverified project claims.

Ask how they would test a non-deterministic system like a customer service agent, covering assertion design, evaluation dataset construction, and fallback verification. Strong answers include LLM-as-judge with human calibration and layered fallback validation.