dlt-extract

What problem does it solve?

This Skill streamlines the process of building portable DLT pipelines, specifically for extracting data from file-based sources and supporting consulting engagements where clients need to run pipelines themselves.

Core Features & Use Cases

• File Source Ingestion: Handles CSV, Excel, Parquet, JSON, SharePoint, and SFTP files.
• Destination Swapping: Enables pipelines to target DuckDB for development and Snowflake/BigQuery for production using environment variables.
• Schema Contracts: Enforces data quality and handles schema drift.
• Portable Pipelines: Facilitates client handoff with pip install and environment variable configurations.
• Use Case: You need to ingest daily CSV reports from a client's SFTP server, process them locally with DuckDB, and then deploy the same pipeline to load data into Snowflake.

Quick Start

Use the dlt-extract skill to create a portable DLT pipeline that reads CSV files from a local directory and loads them into DuckDB.