skills.rest
Clay-HHK
Clay-HHKCOMMUNITY
Explore all skills by this author →
GitHub

model-pruning

Prune LLM weights using Wanda, SparseGPT, and N:M sparsity to reduce model size.

1|Updated Mar 5, 2026
Search Tags:#pytorch#sparsity#inference optimization#llm compression#model pruning#wanda#sparsegpt
One-click install
npx skills add https://github.com/Clay-HHK/claude-skills --skill model-pruning-clay-hhk
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-pruning
Source: https://github.com/Clay-HHK/claude-skills/tree/main/model-pruning
Command: npx skills add https://github.com/Clay-HHK/claude-skills --skill model-pruning-clay-hhk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, transformers, accelerate, and includes references (resource) components.

What problem does it solve? Large language models are expensive to store and slow to serve, and this Skill compresses them by removing redundant weights to cut memory footprint by up to 50% with minimal accuracy loss, without requiring retraining. ## Core Features & Use Cases - One-Shot Pruning: Apply Wanda (weight magnitude × activation) or SparseGPT (second-order Hessian) methods to prune models to 50% sparsity with under 1% accuracy loss and no retraining. - Hardware-Friendly N:M Sparsity: Apply 2:4 or 4:8 structured sparsity patterns compatible with NVIDIA sparse tensor cores for up to 2× inference speedup. - Use Case: You need to deploy LLaMA-2-7B on constrained hardware. Use Wanda with 128 calibration samples to prune it to 50% sparsity, then evaluate with lm-eval to confirm accuracy degradation stays below 1%. ## Quick Start Prune my LLaMA-2-7B model to 50% sparsity using Wanda and evaluate the accuracy on standard benchmarks.

Frequently Asked Questions about model-pruning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I prune a LLaMA model without retraining?▼

Use one-shot pruning methods like Wanda or SparseGPT, which remove weights based on importance scores computed from a small calibration dataset of about 128 samples. Wanda computes importance as weight magnitude times input activation norm, achieving 50% sparsity with under 1% accuracy loss.

Wanda vs SparseGPT: which pruning method should I use?▼

Wanda is simpler and faster with O(n) complexity per layer, while SparseGPT uses second-order Hessian information for slightly better quality (-0.4% vs -0.8% accuracy at 50% sparsity). Use Wanda unless you need the absolute best quality and can afford longer computation.

Does unstructured pruning speed up LLM inference?▼

No, unstructured pruning reduces memory by 50% but provides no speedup because the sparsity pattern is irregular. For actual speedup, use N:M structured sparsity like 2:4, which achieves 2× speedup on NVIDIA GPUs with sparse tensor cores.

What sparsity level is safe for pruning LLMs?▼

50% sparsity is the recommended balance, causing roughly 1% accuracy loss with Wanda or SparseGPT. Conservative 30% sparsity loses under 0.5%, while aggressive 70-90% sparsity causes 2-5% or greater degradation and should use iterative pruning with fine-tuning.

Why does one-shot pruning fail at high sparsity levels?▼

Removing too many weights at once destroys model capacity without any recovery mechanism. At sparsity above 70%, use iterative pruning with fine-tuning between steps, or gradual magnitude pruning that increases sparsity progressively during training.

Related Skills

Explore More Software Engineering Skills→
huggingfacehuggingface
10.9k

huggingface-datasets

Explore and retrieve metadata from Hugging Face datasets via the Dataset Viewer API.

Official
Intermediate
michaelalbermichaelalber
2

model-optimization

Quantize, prune, and convert ML models to TensorRT, TFLite, and ONNX.

Community
Advanced
AnandaAnugrahHandyantoAnandaAnugrahHandyanto

evaluating-llms-harness

Benchmark LLM performance across 60+ tasks using lm-evaluation-harness.

Community
Advanced
Supporter09Supporter09

knowledge-distillation

Compress large language models via teacher-to-student knowledge distillation.

Community
Advanced