What problem does it solve? Moving or ingesting data from BigQuery or Google Cloud Storage often introduces hidden data quality issues like malformed dates, garbage values, inconsistent units, and schema mismatches. This Skill automates profiling, cleansing, and verification so pipelines built on Dataform, dbt, or BigQuery start from trustworthy data. ## Core Features & Use Cases - Dataplex-Based Profiling: Runs automated data profile scans on source tables (with dynamic sampling for tables over 1M rows) to detect nulls, anomalies, and format variance before writing any transformation. - SQL Transformation Generation: Produces cleansing SQL covering schema alignment, type conversion with SAFE.PARSE functions, JSON parsing, array handling, and STRUCT field mapping. - Quality Review Loop: Re-profiles a temporary output table after transformation and compares profiles column-by-column to catch regressions like unexpected NULL increases. - Use Case: When asked to copy a GCS dataset into a BigQuery destination table, the Skill profiles the source, generates cleaning SQL matched to the destination schema, and verifies the result with a post-transformation Dataplex scan. ## Quick Start Ask the agent to clean and load a BigQuery or GCS source into a destination table, for example: profile and clean the table my-project.raw.events and load it into my-project.curated.events.
