memory-spool-drain

What problem does it solve?

This Skill ensures that event data, which may have failed to be written directly to ClickHouse, is reliably captured from spool files, preventing data loss.

Core Features & Use Cases

• Spool File Processing: Reads JSONL files from a designated spool location.
• Data Ingestion: Writes the contents of spool files to ClickHouse.
• Error Handling: Implements retry mechanisms with backoff and quarantines problematic files after a maximum number of attempts.
• Use Case: When the primary data pipeline experiences temporary network issues, this skill acts as a fallback, processing the queued spool files once the connection is restored.

Quick Start

Run the memory spool drain skill for today's date.