What problem does it solve? Building Dataform pipelines for BigQuery requires correct SQLX syntax, proper source declarations, incremental table configuration, and validation against real schemas. This Skill guides the AI through the full workflow of creating, modifying, and validating Dataform ELT pipelines while avoiding common mistakes like misusing operations blocks for appends or skipping schema checks. ## Core Features & Use Cases - Pipeline Generation: Creates SQLX actions, source declarations, and GCS ingestion external tables following Dataform best practices. - Validation Workflow: Compiles pipelines with dataform compile and validates with dataform run --dry-run or bq query --dry_run without executing real runs. - Automatic Cleaning & Optimization: Applies data cleaning protocols and BigQuery SQL optimizations to every transformation, with summary sections in responses. - Use Case: You need to ingest CSV files from GCS into an existing BigQuery table daily. The Skill initializes the Dataform project, creates an external table declaration, builds an incremental SQLX action matching the target schema, and validates the pipeline via dry run. ## Quick Start Ask the AI to create a Dataform pipeline that loads data from a GCS bucket into a BigQuery table using incremental appends.
