quantizing-models-bitsandbytes
Community8-bit/4-bit quantization for memory-efficient LLMs.
Authorovachiever
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill explains how to quantize large language models to 8-bit or 4-bit precision, enabling significant memory savings with minimal accuracy loss, and covers QLoRA workflows and advanced quantization options.
Core Features & Use Cases
- Memory Reduction: 50% (8-bit) to 75% (4-bit) memory savings for LLMs.
- Quantization Modes: INT8, NF4, FP4 with configurable compute dtype and double quantization.
- Practical Workflows: QLoRA training, 8-bit optimizers, and mixed-precision deployments.
Quick Start
Configure 4-bit quantization with NF4 for a large model and load via transformers.
Dependency Matrix
Required Modules
None requiredComponents
references
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: quantizing-models-bitsandbytes Download link: https://github.com/ovachiever/droid-tings/archive/main.zip#quantizing-models-bitsandbytes Please download this .zip file, extract it, and install it in the .claude/skills/ directory.