What problem does it solve? Data ingested from BigQuery or Google Cloud Storage often contains garbage values, inconsistent formats, and schema mismatches that silently corrupt downstream pipelines. This Skill automates profiling, cleansing, and verification so every data movement or ingestion task is backed by evidence-based transformations. ## Core Features & Use Cases - Dataplex-Based Profiling: Runs automated data profile scans on source tables to detect nulls, anomalies, and format variance before writing any transformation logic. - SQL Transformation Generation: Produces cleansing SQL for schema alignment, type conversion, JSON parsing, array handling, and STRUCT field mapping in BigQuery. - Quality Review Loop: Re-profiles transformed output on a temporary sample table and compares profiles column-by-column to catch regressions like unexpected NULL increases. - Use Case: When asked to copy a GCS file into a BigQuery table, the Skill profiles the source, generates cleaning SQL matched to the destination schema, and verifies the result with a post-transformation Dataplex scan. ## Quick Start Ask the assistant to clean and load a BigQuery table or GCS file into a destination table, and it will profile the source, generate cleansing SQL, and verify the results automatically.
