etl-guidelines

What problem does it solve?

Manual data entry and inconsistent data imports lead to errors, duplicates, and wasted time. This Skill provides a standardized, automated, and error-resistant framework for all data ingestion tasks, ensuring data integrity and saving countless hours of manual reconciliation.

Core Features & Use Cases

• Idempotent Imports: Prevent duplicate records with external_hash and get_or_create patterns, ensuring data integrity on re-runs and eliminating manual cleanup.
• Quality Gates: Automatically validate data quality (e.g., max invalid rows, duplicates) before committing changes, preventing bad data from entering your system and reducing post-import errors.
• Dry-Run & Apply Workflow: Safely preview import results without modifying the database, then apply changes with confidence and automatic rollback on error, giving you peace of mind.
• Use Case: Imagine you receive weekly Excel files with event schedules. Use this Skill to automatically import new events, update existing ones, and flag any data quality issues, all while ensuring no duplicates are created and your database remains clean.

Quick Start

Use the etl-guidelines skill to generate a Django management command template for importing event data from an XLSX file, including idempotence and quality gates.