llm-fine-tuning

What problem does it solve?

This Skill provides the infrastructure and code to fine-tune Large Language Models (LLMs) using various techniques like QLoRA, LoRA, and full fine-tuning, enabling customization for specific domains and tasks.

Core Features & Use Cases

• QLoRA/LoRA Fine-Tuning: Efficiently fine-tune large models on consumer-grade GPUs.
• Distributed Training: Supports DeepSpeed and FSDP for multi-node, multi-GPU training.
• Dataset Preparation: Guides on preparing datasets for instruction tuning, DPO, or RLHF.
• Model Export: Merging LoRA adapters and exporting models for serving.
• Use Case: Fine-tune an open-source LLM on your company's internal documentation to create a specialized chatbot that can answer employee questions accurately.

Quick Start

Use the llm-fine-tuning skill to set up QLoRA fine-tuning for a Llama 3.1 model using the provided Axolotl configuration.