databricks-data-engineering
CommunityBuild robust medallion data pipelines.
Data & Analytics#orchestration#etl#data-quality#data-engineering#databricks#delta-lake#medallion-architecture
Authorandregit2026
Version1.0.0
Installs0
System Documentation
What problem does it solve?
The skill provides a framework for building production-grade data pipelines using the medallion architecture (Bronze/Silver/Gold) in Databricks, covering ingestion, transformation, quality checks, Delta Lake optimization, and orchestration to accelerate reliable analytics.
Core Features & Use Cases
- End-to-end medallion pipeline design: Bronze layer for raw ingestion, Silver layer for cleaned and validated data, and Gold layer for business-ready analytics.
- Delta Lake optimization, schema evolution, and idempotent upserts for reliable data pipelines.
- Orchestration of multi-stage notebooks and workflows, with automated quality checks and scalable performance.
- Use cases include ETL/ELT data ingestion, incremental processing, and analytics dashboards powered by clean, trusted data.
Quick Start
Set up a three-layer medallion pipeline (Bronze-Silver-Gold) to ingest, clean, and model data, then run the deployment workflow.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: databricks-data-engineering Download link: https://github.com/andregit2026/Databricks_DQ_Business/archive/main.zip#databricks-data-engineering Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.