skills.rest
Clay-HHK
Clay-HHKCOMMUNITY
Explore all skills by this author →
GitHub

unsloth

Guides LLM fine-tuning with Unsloth using LoRA, QLoRA, and GRPO reinforcement learning.

1|Updated Mar 5, 2026
Search Tags:#fine-tuning#lora#qlora#gguf#grpo#unsloth#llm training
One-click install
npx skills add https://github.com/Clay-HHK/claude-skills --skill unsloth-clay-hhk
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: unsloth
Source: https://github.com/Clay-HHK/claude-skills/tree/main/unsloth
Command: npx skills add https://github.com/Clay-HHK/claude-skills --skill unsloth-clay-hhk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires unsloth, torch, transformers, trl, datasets, peft, and includes references (resource) components.

What problem does it solve? Fine-tuning large language models typically demands excessive VRAM and slow training runs, blocking developers with consumer GPUs from customizing models like Llama, Qwen, Gemma, or Mistral for their own tasks. ## Core Features & Use Cases - Memory-Efficient Fine-Tuning: Apply LoRA and QLoRA (4-bit) training to fit models from 3B to 70B+ parameters on limited VRAM, with documented VRAM requirement tables per model size. - Reinforcement Learning: Train reasoning models with GRPO, GSPO, DPO, ORPO, and KTO using up to 80% less VRAM than standard RL setups. - Model Export & Deployment: Save fine-tuned models to GGUF for Ollama and llama.cpp, or to 16-bit for vLLM and SGLang serving. - Use Case: Fine-tune Llama 3.1 (8B) on a custom instruction dataset in a free Colab notebook, then export it to GGUF and run it locally in Ollama. ## Quick Start Ask the assistant to show how to fine-tune a Llama model with Unsloth QLoRA on a custom dataset and export it to GGUF for Ollama.

Frequently Asked Questions about unsloth

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune a Llama model with Unsloth?▼

Load the model with FastLanguageModel.from_pretrained using 4-bit quantization, attach LoRA adapters, prepare a formatted dataset, and train with the TRL SFTTrainer. Unsloth provides free Colab notebooks for Llama, Qwen, Gemma, and other models that walk through the full process.

How much VRAM do I need for QLoRA fine-tuning?▼

QLoRA (4-bit) requires roughly 5 GB for a 7B model, 6 GB for 8B, 8.5 GB for 14B, and 41 GB for 70B. LoRA in 16-bit needs significantly more, such as 19 GB for 7B. If you hit out-of-memory errors, lower the batch size to 1, 2, or 3.

What is the difference between LoRA and QLoRA in Unsloth?▼

LoRA trains small adapter matrices in 16-bit while freezing the base model, reducing trainable parameters. QLoRA additionally quantizes base weights to 4-bit, cutting memory use dramatically. Unsloth recommends starting with QLoRA since its dynamic 4-bit quants have negligible accuracy loss.

Does Unsloth support reinforcement learning with GRPO?▼

Yes, Unsloth supports GRPO, GSPO, DPO, ORPO, and KTO for reinforcement learning, using up to 80% less VRAM than standard implementations. It includes notebooks for training reasoning models and vision-language models with GRPO.

Can I run Unsloth on Windows or AMD GPUs?▼

Unsloth works on Linux and Windows (including WSL), and supports NVIDIA GPUs with CUDA capability 7.0 or higher, plus AMD and Intel GPUs. An official Docker image unsloth/unsloth is available for setup-free training.

How do I export a fine-tuned model to Ollama or GGUF?▼

After training, save the model merged to 16-bit and convert it to GGUF format for use with Ollama, llama.cpp, Jan, or Open WebUI. For production serving, save to 16-bit for vLLM or SGLang deployment instead.

Related Skills

Explore More Software Engineering Skills→
cuba6112cuba6112

unsloth-datasets

Standardize ShareGPT and Alpaca datasets for Unsloth fine-tuning.

Community
Intermediate
cryptopaficryptopafi

unsloth

Guide LoRA and QLoRA fine-tuning of LLMs with Unsloth.

Community
Advanced
JeffallanJeffallan
10.9k

fine-tuning-expert

Automate fine-tuning workflows for large language models using LoRA/QLoRA and PEFT methods.

Community
Advanced
ragnarokhaaragnarokhaa
4

huggingface-accelerate

Train PyTorch models across GPUs and TPUs with a unified distributed API.

Community
Advanced