implementing-llms-litgpt

What problem does it solve?

This Skill provides clean, production-ready implementations for training and fine-tuning Large Language Models (LLMs) using Lightning AI's LitGPT framework.

Core Features & Use Cases

• Model Implementation: Offers 20+ pretrained LLM architectures (Llama, Gemma, Phi, Qwen, Mistral) with single-file code.
• Fine-Tuning: Supports efficient fine-tuning methods like LoRA and QLoRA, suitable for custom datasets.
• Pretraining: Enables training new models from scratch on custom data.
• Deployment: Includes tools for model conversion and API deployment.
• Use Case: A researcher wants to fine-tune a Qwen model on a proprietary dataset for a specific domain. This Skill provides the necessary tools and recipes to achieve this efficiently.

Quick Start

Use the litgpt skill to fine-tune the microsoft/phi-2 model on your custom dataset located at data/my_dataset.json using LoRA.