What problem does it solve? Building and maintaining dbt pipelines for BigQuery requires deep knowledge of dbt project structure, profiles configuration, SQL optimization patterns, and BigQuery-specific quirks like BigLake Iceberg naming. This Skill guides an AI agent through the full lifecycle of creating, modifying, testing, and troubleshooting dbt projects so pipelines compile correctly and follow best practices. ## Core Features & Use Cases - dbt Project Scaffolding: Initializes new dbt projects in a dedicated folder with correct dbt_project.yml and profiles.yml configuration mapped to BigQuery settings. - SQL Optimization: Automatically rewrites inefficient patterns (e.g., IN (SELECT ...) to EXISTS) and proposes conditional optimizations like UNION ALL or APPROX_COUNT_DISTINCT with user confirmation. - Validation & Testing: Compiles projects with dbt compile, runs dbt test, and generates unit tests alongside models using schema-derived test data. - Troubleshooting: Diagnoses local and remote orchestration failures (e.g., Cloud Composer DAG errors, missing profile errors) using gcloud logging and root cause analysis. - Use Case: Ask the agent to create a new dbt model that transforms raw events from a BigQuery source table into a cleaned, documented, and unit-tested staging model with proper ref() and source() references. ## Quick Start Create a dbt model that aggregates daily revenue from my BigQuery orders table, then compile and test the project.
