skills.rest
organvm-i-theoria
organvm-i-theoriaOFFICIAL
Explore all skills by this author →
GitHub

local-llm-fine-tuning

Guides dataset preparation and LoRA/QLoRA fine-tuning of open-weights LLMs on consumer hardware.

Updated May 16, 2026
Search Tags:#llm#fine-tuning#lora#jsonl#dataset#qlora#hugging face
One-click install
npx skills add https://github.com/organvm-i-theoria/_agent-ontology --skill local-llm-fine-tuning-organvm-i-theoria
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: local-llm-fine-tuning
Source: https://github.com/organvm-i-theoria/_agent-ontology/tree/main/.agents/skills/local-llm-fine-tuning
Command: npx skills add https://github.com/organvm-i-theoria/_agent-ontology --skill local-llm-fine-tuning-organvm-i-theoria

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Fine-tuning open-weights models like Llama, Mistral, and Gemma locally involves confusing choices around data formats, hyperparameters, VRAM limits, and tooling. This Skill walks users through the entire process so they can adapt a base model to their task without wasting GPU time on misconfigured runs. ## Core Features & Use Cases - Dataset Preparation Guidance: Explains JSONL formats (Alpaca, ShareGPT, OpenAI messages, completion) with conversion and validation scripts, plus a data quality checklist. - Hyperparameter Recommendations: Provides concrete LoRA/QLoRA settings (rank, alpha, dropout, learning rate, batch size) tuned to dataset size and model scale, with VRAM requirement tables. - Tool Selection Advice: Compares Unsloth, Axolotl, and raw Transformers/PEFT workflows with ready-to-use sample configurations. - Use Case: A user wants to teach Llama-3-8B a specific output format using 800 examples on a single consumer GPU. The Skill recommends the Alpaca JSONL format, a QLoRA config with r=16 and 4-bit quantization, and flags overfitting risks. ## Quick Start Ask the assistant to help you fine-tune a Llama or Mistral model on your dataset, and it will recommend the data format, hyperparameters, and tooling for your hardware.

Frequently Asked Questions about local-llm-fine-tuning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune a Llama model on my own dataset?▼

Format your data as JSONL in Alpaca or ShareGPT format, then train with LoRA using Unsloth, Axolotl, or Transformers/PEFT. Start with rank 16, learning rate 2e-4, and 2 epochs for medium-sized datasets.

What is the difference between LoRA and QLoRA?▼

LoRA trains small low-rank adapter matrices instead of full model weights. QLoRA additionally loads the base model in 4-bit quantization, cutting VRAM usage roughly in half so 7B models fit in 6-8 GB of memory.

Unsloth vs Axolotl for fine-tuning, which should I use?▼

Unsloth is fastest for single-GPU training with minimal setup. Axolotl suits reproducible, config-file-driven runs. Use raw Transformers with PEFT when you need full control over a custom training script.

How much VRAM do I need to fine-tune a 7B model?▼

With QLoRA 4-bit quantization, a 7B model needs roughly 6-8 GB of VRAM. Reduce memory further by enabling gradient checkpointing, lowering batch size with gradient accumulation, or shortening max sequence length.

Why is my fine-tuned model outputting garbage or repeating itself?▼

Common causes are a mismatched chat template, incorrect EOS token configuration, or severe overfitting from too many epochs. Verify the template matches your base model and reduce epochs or add dropout.

How many training examples do I need for fine-tuning?▼

Style transfer works with 50-500 examples, new tasks need 500-5000, and domain adaptation needs 1000-10000 or more. Quality matters more than quantity: 500 excellent examples beat 5000 mediocre ones.

Related Skills

Explore More Software Engineering Skills→
phonowellphonowell
1

book-sft-pipeline

Convert books into SFT-ready JSONL datasets and train LoRA models.

Community
Advanced
cuba6112cuba6112

unsloth-gguf

Export fine-tuned machine learning models to GGUF format with quantization and LoRA merging.

Community
Intermediate
roanbrasilroanbrasil
5

fine-tuning-llms

Guide LLM fine-tuning workflows with PEFT methods and deployment.

Community
Advanced
NaveedTechLabNaveedTechLab
11

ai-integration-specialist

Integrate LLMs, RAG systems, and WhatsApp Business API workflows.

Community
Advanced