debug-batch-api

Start with the job ID query across all batch-api services, then check sweep and finalize events such as sweep.poll_failed and finalize_one.finalize_failed. The symptom decision table maps "batch stuck" to provider poll failures, Pub/Sub retries, or missing provider job IDs.

A 402 means the settlement outcome is explicitly payment_required, typically a failed or insufficient-balance charge. Query batch_api.get_batch.results_payment_blocked with the job or entity ID; a null settlement is a metadata-only 200 pending state, not a provider failure.

Query service:batch-api* with the job ID under @data.jsonPayload.extra.job_id, since each Cloud Run worker role logs under its own service. Cloudflare ingress logs use source:cloudflare @script_name:batch-api instead.

Check emit_generations.done and emit_generation.missing_usage events for the job. Missing or malformed provider usage triggers fallback estimation, so inspect the estimated, unaccounted, and is_usage_complete fields rather than treating estimates as exact usage.

Yes. Pull output/raw_response and output/results from the GCS bucket customer-data-batch-api-prod under the billable entity and job ID path. A field present only in the raw response was dropped by the parse/render transform.

Datadog retains logs for 14 days, so investigations older than that window cannot rely on log queries. GCS batch artifacts follow a separate 30-day retention lifecycle rule.