training-pipelines

What problem does it solve?

Training pipelines simplify the creation, orchestration, and deployment of end-to-end machine learning training workflows, reducing setup time and avoiding boilerplate.

Core Features & Use Cases

• End-to-end orchestration: Define and manage data loading, preprocessing, training, evaluation, and model registration in a reproducible pipeline.
• Distributed training: Support for multi-GPU and distributed training using PyTorch DDP with proper data sharding and synchronization.
• Hyperparameter tuning: Integrates with Optuna to explore configurations and find optimal models.
• Kubeflow deployment: Provides templates to deploy pipelines to Kubeflow or similar orchestration platforms.
• Real-world use case: A team trains multiple experiments across GPU clusters with automated validation and artifact storage.

Quick Start

Use the training-pipelines skill to spin up a simple training workflow on a GPU-enabled environment. For example: claude "training-pipelines - [describe a task]"