skills.rest
waybarrios
waybarriosCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

trl-training

Train and fine-tune transformer language models using TRL CLI commands.

507|40|Updated Apr 26, 2026
Search Tags:#llm#fine-tuning#lora#dpo#trl#sft#hugging face
One-click install
npx skills add https://github.com/waybarrios/opencode-power-pack --skill trl-training-waybarrios
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: trl-training
Source: https://github.com/waybarrios/opencode-power-pack/tree/main/skills/trl-training
Command: npx skills add https://github.com/waybarrios/opencode-power-pack --skill trl-training-waybarrios

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires trl, transformers, accelerate, peft, datasets.

What problem does it solve? Post-training large language models with techniques like SFT, DPO, GRPO, RLOO, and reward modeling requires knowing many CLI flags, dataset formats, and distributed training configurations, which is error-prone without expert guidance. ## Core Features & Use Cases - Multiple Training Methods: Covers SFT, DPO, GRPO, RLOO, and Reward Model training through the TRL CLI with ready-to-run command templates. - LoRA and PEFT Support: Provides parameter-efficient fine-tuning commands with LoRA rank and alpha settings for memory-constrained environments. - Distributed Training: Documents Accelerate integration with multi-GPU, FSDP, and DeepSpeed ZeRO configurations, plus YAML config files for reproducibility. - Use Case: Fine-tune Qwen2-0.5B on the Capybara instruction dataset with LoRA adapters, then align it with DPO on ultrafeedback preference data, all via guided CLI commands. ## Quick Start Ask the assistant to fine-tune a small model like Qwen/Qwen2-0.5B on the trl-lib/Capybara dataset using TRL SFT with LoRA adapters.

Frequently Asked Questions about trl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune a language model with TRL SFT?▼

Run the trl sft command with --model_name_or_path, --dataset_name, and training hyperparameters like learning rate and epochs. Add --use_peft with --lora_r and --lora_alpha to train LoRA adapters instead of full weights.

What is the difference between DPO and GRPO in TRL?▼

DPO aligns models using static preference datasets with chosen and rejected pairs, while GRPO is an online RL method that samples multiple outputs and optimizes based on their relative rewards. DPO needs preference data; GRPO needs a reward function.

Does TRL support multi-GPU distributed training?▼

Yes, TRL integrates with Hugging Face Accelerate for multi-GPU and multi-node training. Use --num_processes for multi-GPU or --accelerate_config with predefined configs like fsdp2, zero2, or zero3 for FSDP and DeepSpeed.

How do I fix CUDA out of memory during TRL training?▼

Reduce --per_device_train_batch_size and increase --gradient_accumulation_steps, enable --use_peft for LoRA training, and use --gradient_checkpointing. You can also try a smaller model or shorter sequence lengths.

Can I use a YAML config file with TRL CLI commands?▼

Yes, all TRL CLI arguments can be specified in a YAML config file passed via --config. You can still override individual values on the command line, for example trl sft --config sft_config.yaml --learning_rate 1.0e-5.

Related Skills

Explore More Software Engineering Skills→
huggingfacehuggingface
10.9k

huggingface-llm-trainer

Train language and vision models with TRL methods on Hugging Face Jobs.

Official
Advanced
ml-exploreml-explore
771

swift-mlx-lm

Run local LLM and VLM inference and fine-tuning on Apple Silicon with MLX Swift.

Official
Advanced
ISAQQSAIISAQQSAI

model-trainer

Train models with TRL on Hugging Face Jobs using SFT, DPO, GRPO, and Reward Modeling workflows.

Community
Advanced
heysuhasheysuhas
2

slime-rl-training

Integrate Megatron-LM and SGLang for large-scale LLM reinforcement learning training.

Community
Advanced