05-schema-validation
OfficialEnsure Gold layer data integrity at runtime.
Data & Analytics#data integrity#data quality#schema validation#delta lake#gold layer#merge operation
Authordatabricks-solutions
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill prevents common Gold layer bugs caused by schema mismatches between DataFrames and target Delta tables, ensuring data quality and preventing runtime errors during merge operations.
Core Features & Use Cases
- Runtime Schema Validation: Compares DataFrame columns and data types against the actual DDL schema of the target Gold table before a MERGE operation.
- Error Prevention: Catches
UNRESOLVED_COLUMNerrors and other schema-related issues before they impact production. - Use Case: When preparing a DataFrame for a
dim_usertable merge, this Skill will verify that all required columns from thedim_userDDL are present in the DataFrame and that no extra columns are included, preventing merge failures.
Quick Start
Validate the schema of the updates_df DataFrame against the catalog.schema.dim_user table before performing a merge.
Dependency Matrix
Required Modules
None requiredComponents
scriptsreferences
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: 05-schema-validation Download link: https://github.com/databricks-solutions/vibe-coding-workshop-template/archive/main.zip#05-schema-validation Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.