pipeline-design
OfficialDesign reliable ETL/ELT pipelines with patterns.
Authortimequity
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Helps architects design ETL/ELT pipelines with reliability, observability, idempotence, and recoverability.
Core Features & Use Cases
- Batch pattern: Source → Extract → Stage → Transform → Load with checkpoints
- Streaming pattern: Source → Kafka/Kinesis → Process → Sink with state store
- Design principles: Idempotent, Incremental, Observable, Testable, Recoverable
- Sample SQL patterns: staging deduplication and transform-to-target
Quick Start
Sketch a simple batch pipeline from a source to a target, annotating at least one checkpoint.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: pipeline-design Download link: https://github.com/timequity/plugins/archive/main.zip#pipeline-design Please download this .zip file, extract it, and install it in the .claude/skills/ directory.