mkl-debug-ml-training

Diagnose training failures, NaNs, and gradient issues in PyTorch, Lightning, and TensorFlow.

11|1|Updated Sep 13, 2026
One-click install
npx skills add https://github.com/00200200/maintainer-skills-lab --skill mkl-debug-ml-training-00200200
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: mkl-debug-ml-training
Source: https://github.com/00200200/maintainer-skills-lab/tree/main/providers/cursor/.cursor/skills/mkl-debug-ml-training
Command: npx skills add https://github.com/00200200/maintainer-skills-lab --skill mkl-debug-ml-training-00200200

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Training runs fail in opaque ways: NaN losses, missing gradients, misleading loss values from shape mismatches, and non-reproducible results. This Skill turns a reported symptom into an observable, bounded failure so the root cause can be identified and verified instead of masked. ## Core Features & Use Cases - Failure isolation: Reduce a failing training run to a fixed batch while preserving shapes, label semantics, and value ranges, then trace finite values through inputs, loss, gradients, and parameter updates. - Framework-specific controls: Apply the correct debugging levers for PyTorch (autograd, anomaly detection, AMP), Lightning (fast_dev_run, manual vs automatic optimization), and TensorFlow/Keras (GradientTape, run_eagerly, op determinism). - Verified fixes: Compare baseline and candidate fix from equivalent model and optimizer state with unchanged assertions and principled numerical tolerances. - Use Case: Your PyTorch training loss looks wrong despite correct predictions. The Skill identifies a broadcasting bug where [N, 1] predictions minus [N] labels produce an [N, N] residual, then verifies the shape fix against the baseline. ## Quick Start Debug my PyTorch training run where the loss suddenly becomes NaN after a few steps and tell me the root cause with a verified fix.

Frequently Asked Questions about mkl-debug-ml-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I debug NaN loss during PyTorch training?▼

Trace finite values through inputs, outputs, loss, gradients, and updated parameters to find the first unexpected value. Clipping or lowering the learning rate only hides the symptom unless supported by the diagnosed cause; anomaly detection can localize backward failures narrowly.

Why are my model gradients None or zero during training?▼

Missing gradients usually come from unexpected detach() calls, .item() conversions, tensor reconstruction, or parameters not registered with the optimizer. In TensorFlow, check that operations occur inside the GradientTape and that NumPy conversions have not severed differentiation.

How do I reproduce a training bug without running full training?▼

Reduce the run to a fixed batch while keeping the relevant shapes, label semantics, masks, and value ranges. In Lightning, bound work with integer limit_train_batches and limit_val_batches rather than fast_dev_run when the bug involves loggers or checkpointing.

Does model.eval() disable gradient computation in PyTorch?▼

No. model.eval() changes layers like dropout and batch normalization but does not disable autograd. Conversely, torch.no_grad() disables gradient recording without selecting evaluation behavior; the failing path may need one, both, or neither.

Can seeding guarantee reproducible training results across machines?▼

No. Seed controls do not promise identical results across framework releases or CPU/GPU execution. Deterministic algorithms in PyTorch or tf.config.experimental.enable_op_determinism() can help but may cost performance or reject unsupported operations.

When should I not use this debugging approach?▼

Do not use it for open-ended architecture design or hyperparameter search; it targets a concrete training bug or regression. Also note that a CPU or eager-mode reproduction isolates a bug but does not establish behavior on the original accelerator or compiled graph.