skills.rest
Clay-HHK
Clay-HHKCOMMUNITY
Explore all skills by this author →
GitHub

pytorch-lightning

Organizes PyTorch training code with automatic distributed training, callbacks, and checkpointing.

1|Updated Mar 5, 2026
Search Tags:#deep learning#callbacks#distributed training#training loop#fsdp#ddp#pytorch lightning
One-click install
npx skills add https://github.com/Clay-HHK/claude-skills --skill pytorch-lightning-clay-hhk
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: pytorch-lightning
Source: https://github.com/Clay-HHK/claude-skills/tree/main/pytorch-lightning
Command: npx skills add https://github.com/Clay-HHK/claude-skills --skill pytorch-lightning-clay-hhk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires lightning, torch, transformers, and includes references (resource) components.

What problem does it solve? Writing raw PyTorch training loops requires repetitive boilerplate for device management, distributed synchronization, checkpointing, and logging, which introduces bugs and slows down research iteration. ## Core Features & Use Cases - Structured Training Loops: Organize model code into a LightningModule with training_step, validation_step, and configure_optimizers while the Trainer handles devices, precision, and logging. - Automatic Distributed Training: Switch between DDP, FSDP, and DeepSpeed strategies with a single parameter, scaling from one GPU to multi-node clusters without code changes. - Callbacks and Tuning: Use ModelCheckpoint, EarlyStopping, and LearningRateMonitor callbacks, plus integrations with Ray Tune, Optuna, and WandB sweeps for hyperparameter search. - Use Case: Convert an existing PyTorch training script into a LightningModule, then launch 8-GPU DDP training with BF16 precision and automatic best-model checkpointing using one Trainer configuration. ## Quick Start Convert my PyTorch training loop into a PyTorch Lightning module and run it on multiple GPUs with DDP and early stopping.

Frequently Asked Questions about pytorch-lightning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I convert a PyTorch training loop to PyTorch Lightning?▼

Move your model into a LightningModule subclass, put the forward and loss logic in training_step, and return the optimizer from configure_optimizers. Then call trainer.fit(model, train_loader) and the Trainer handles device placement, backward passes, and logging.

How do I run multi-GPU training with PyTorch Lightning?▼

Set accelerator='gpu', devices to the GPU count, and strategy='ddp' in the Trainer. Lightning automatically spawns processes, distributes data with DistributedSampler, and synchronizes gradients without code changes.

PyTorch Lightning vs Hugging Face Accelerate: which should I use?▼

Lightning provides a structured framework with built-in callbacks, checkpointing, and logging, while Accelerate makes minimal changes to existing PyTorch loops for more flexibility. Choose Lightning for standardized team workflows and Accelerate for lightweight migration of existing code.

Does PyTorch Lightning support FSDP and DeepSpeed for large models?▼

Yes, Lightning includes FSDPStrategy with FULL_SHARD sharding and DeepSpeedStrategy with ZeRO stages 2 and 3 plus CPU offloading. These strategies support models from 7B to 70B+ parameters with bf16 precision.

Why is my validation loop not running in PyTorch Lightning?▼

Validation only runs when you pass a validation dataloader to trainer.fit(model, train_loader, val_loader) and define a validation_step method. Calling fit with only the train loader skips validation entirely.

How do I fix out-of-memory errors during Lightning training?▼

Reduce batch size, enable gradient accumulation with accumulate_grad_batches, or switch to bf16 precision to cut memory roughly in half. For very large models, use FSDP with cpu_offload or DeepSpeed ZeRO-3 with optimizer offloading.

Related Skills

Explore More Software Engineering Skills→
Clay-HHKClay-HHK
1

ray-data

Process large ML datasets with distributed streaming execution across CPU and GPU clusters.

Community
Intermediate
RachasumanthRachasumanth

inference-deploy

Deploy trained machine learning models for inference using vLLM, TGI, Ollama, and llama.cpp.

Community
Advanced
DifanaDAPDifanaDAP

weights-and-biases

Automate ML experiment tracking and hyperparameter sweeps via the Python API.

Community
Advanced
math-incmath-inc
1.2k

pytorch-lightning

Automate end-to-end PyTorch training workflows with PyTorch Lightning.

Official
Advanced