dfdl_ref
CommunityMaster DataFusion planning with DeltaLake.
Authorpaul-heyse
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Provide a comprehensive operations manual for DataFusion + DeltaLake integration, detailing how to wire the core query engine with the storage layer, including planning, pushdown, scan providers, and schema bridging, while guiding lookup patterns and avoiding API guesswork.
Core Features & Use Cases
- In-depth coverage of DataFusion catalog/schema management, external tables, and predicate pushdown with DeltaLake integration nuances.
- Practical guidance on programmatic plan construction, subqueries, and UDF usage across Rust and Python bindings, including how planning surfaces map to execution.
- DeltaLake-specific integration notes (time travel, file pruning, MVCC log semantics) with example workflows for DataFusion planning, SQL/DDL usage, and data registration.
Quick Start
Load a DeltaTable into a DataFusion context and run a simple plan to observe scan pruning and predicate pushdown.
Dependency Matrix
Required Modules
None requiredComponents
references
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: dfdl_ref Download link: https://github.com/paul-heyse/CodeAnatomy/archive/main.zip#dfdl-ref Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.