What problem does it solve? Data ingested from BigQuery or Google Cloud Storage often contains malformed values, inconsistent formats, and schema mismatches that silently corrupt downstream pipelines. This Skill automates profiling, cleansing, and verification so every data movement or ingestion task produces validated, schema-aligned output. ## Core Features & Use Cases - Dataplex-Based Profiling: Automatically scans source tables (with dynamic sampling for tables over 1M rows) to detect nulls, garbage values, and format variance before writing any transformation. - SQL Transformation Generation: Produces cleaning logic for schema alignment, type conversion with SAFE.PARSE functions, JSON parsing, array handling, and STRUCT field mapping. - Quality Review Loop: Re-profiles a temporary output table and compares against the source profile to catch regressions like unexpected NULL increases. - Use Case: When asked to copy a GCS dataset into a BigQuery destination table, the Skill profiles the source, generates cleansing SQL matched to the destination schema, and verifies the result with a post-transformation Dataplex scan. ## Quick Start Ask the assistant to clean and load the BigQuery table my-project.my_dataset.my_table into the destination table using the data-autocleaning skill.
