skills.rest
superfhp
superfhpCOMMUNITY
Explore all skills by this author →
GitHub

unsloth

Fine-tune LLMs with LoRA and QLoRA using less VRAM and faster training.

Updated May 13, 2026
Search Tags:#fine-tuning#lora#qlora#gguf#grpo#unsloth#llm training
One-click install
npx skills add https://github.com/superfhp/lumi-agent-body --skill unsloth-superfhp
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: unsloth
Source: https://github.com/superfhp/lumi-agent-body/tree/main/skills/mlops/training/unsloth
Command: npx skills add https://github.com/superfhp/lumi-agent-body --skill unsloth-superfhp

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires unsloth, torch, transformers, trl, datasets, peft, and includes references (resource) components.

What problem does it solve? Fine-tuning large language models typically demands expensive GPUs and long training times. This Skill provides comprehensive Unsloth documentation and guidance to run LoRA/QLoRA fine-tuning and reinforcement learning 2-5x faster with up to 70-80% less VRAM, making local training feasible on consumer hardware. ## Core Features & Use Cases - Efficient Fine-Tuning: Guidance for LoRA, QLoRA, full fine-tuning, and continued pretraining of models like Llama, Qwen, Gemma, Mistral, Phi-4, and gpt-oss. - Reinforcement Learning: Instructions for GRPO, GSPO, DPO, ORPO, and KTO training, including vision RL and reasoning model creation. - Model Export & Deployment: Steps to save models to GGUF, Ollama, vLLM, and SGLang formats for local inference and serving. - Use Case: Fine-tune Llama 3.1 (8B) on a custom dataset using a free Colab notebook with QLoRA, then export it to GGUF and run it locally in Ollama. ## Quick Start Ask the assistant to show how to fine-tune a Llama model with Unsloth using QLoRA on a single GPU.

Frequently Asked Questions about unsloth

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune a Llama model with Unsloth?▼

Load the model with FastLanguageModel.from_pretrained using load_in_4bit for QLoRA, attach LoRA adapters, prepare your dataset, and train with the TRL SFTTrainer. Unsloth provides free Colab notebooks for Llama, Qwen, Gemma, and other models to follow step by step.

What is the difference between LoRA and QLoRA fine-tuning?▼

LoRA fine-tunes small adapter matrices in 16-bit precision while freezing the base model. QLoRA additionally quantizes base weights to 4-bit, drastically reducing VRAM usage. Unsloth recommends starting with QLoRA since its dynamic 4-bit quants have negligible accuracy loss.

How much VRAM do I need for Unsloth fine-tuning?▼

QLoRA 4-bit training needs about 5 GB for a 7B model and 41 GB for a 70B model, while 16-bit LoRA needs roughly 19 GB and 164 GB respectively. If you run out of memory, lower the batch size to 1, 2, or 3.

Does Unsloth work on Windows and AMD GPUs?▼

Yes. Unsloth supports Linux and Windows (including WSL), and works on NVIDIA GPUs with CUDA capability 7.0 or higher, plus AMD and Intel GPUs. An official Docker image unsloth/unsloth is also available for setup-free training.

How do I export a fine-tuned model to Ollama or GGUF?▼

After training, save the model by merging to 16-bit and converting to GGUF format, which can then be loaded into Ollama, llama.cpp, Open WebUI, or Jan. Unsloth documentation includes dedicated guides for saving to GGUF, Ollama, vLLM, and SGLang.

Can Unsloth train vision and text-to-speech models?▼

Yes. Unsloth supports vision/multimodal fine-tuning for models like Qwen3-VL and Gemma 3, TTS models such as Orpheus and Sesame-CSM, and even vision reinforcement learning with GRPO and GSPO.

Related Skills

Explore More Software Engineering Skills→
cuba6112cuba6112

unsloth-datasets

Standardize ShareGPT and Alpaca datasets for Unsloth fine-tuning.

Community
Intermediate
cuba6112cuba6112

unsloth-gguf

Export fine-tuned machine learning models to GGUF format with quantization and LoRA merging.

Community
Intermediate
JeffallanJeffallan
10.9k

fine-tuning-expert

Automate fine-tuning workflows for large language models using LoRA/QLoRA and PEFT methods.

Community
Advanced
hhhi21ghhhi21g

unsloth

Fine-tune AI models with Unsloth for faster training and reduced memory usage.

Community
Advanced