celery-expert

What problem does it solve?

This Skill streamlines the design, implementation, and optimization of distributed task queues using Celery, ensuring reliable background job processing and workflow orchestration.

Core Features & Use Cases

• Task Design & Reliability: Define robust, idempotent tasks with proper error handling and retry mechanisms.
• Performance Optimization: Tune Celery for maximum throughput and efficiency through chunking, routing, and connection pooling.
• Monitoring & Security: Implement best practices for observing task execution and securing your Celery infrastructure.
• Use Case: You need to process thousands of user sign-ups asynchronously. This Skill will guide you in setting up Celery workers, defining tasks that can handle failures gracefully, and ensuring the system scales efficiently.

Quick Start

Use the celery-expert skill to design a Celery task for processing user uploads with retry logic.