Configure and run SimPO training for reference-free LLM preference optimization.
npx skills add https://github.com/ever-oli/io --skill simpo-training-ever-oli
Please help me install this Agent Skill. Skill: simpo-training Source: https://github.com/ever-oli/io/tree/main/skills/mlops/training/simpo Command: npx skills add https://github.com/ever-oli/io --skill simpo-training-ever-oli
SimPO provides a reference-free approach to preference optimization for aligning large language models, enabling simpler and faster training without a separate reference model.
Run the provided script (scripts/run_simpo.py) with a valid YAML config to begin SimPO alignment.
High-intent search queries and answers about installing and using this skill.
SimPO enables reference-free preference optimization by aligning LLMs without a separate reference model. You configure training parameters like beta, gamma_beta_ratio, and loss_type in a YAML file to run the alignment script directly on preference datasets.
SimPO eliminates the separate reference model required by standard RLHF, simplifying the pipeline and reducing memory overhead. It uses a length-normalized reward formulation with configurable beta and gamma_beta_ratio parameters instead of comparing against a fixed reference policy.
Yes, SimPO supports end-to-end training workflows for large language models ranging from 7B to 70B parameters. It operates using standard dependencies including torch, transformers, and accelerate to handle distributed training configurations.
You configure dataset mixing and loss functions through YAML-based config fields. Define data proportions using dataset_mixer, select the loss_type, and adjust sft_weight to apply optional SFT regularization alongside the preference optimization loss.
SimPO requires hyperparameters like gamma_beta_ratio and beta because it uses a reference-free reward calculation. These parameters control the target reward margin and regularization strength, ensuring the model converges correctly without a reference policy.
MesferAliImplement SimPO training for LLM alignment using PyTorch and Transformers.
masermediagroup-stackEvaluate LLM systems using automated metrics, LLM-as-judge, RAGAS, and benchmarks.
zhouboyu-xrealOptimizes LLM preferences with SimPO using torch, transformers, datasets, trl, and accelerate.
huggingfaceTrain and fine-tune language models on Hugging Face Jobs using TRL methods.
