skills.rest
waybarrios
waybarriosCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

train-sentence-transformers

Train and fine-tune SentenceTransformer bi-encoders, CrossEncoder rerankers, and SparseEncoder models.

507|40|Updated Apr 26, 2026
Search Tags:#embeddings#fine-tuning#sentence-transformers#reranker#hard negatives#splade#contrastive loss
One-click install
npx skills add https://github.com/waybarrios/opencode-power-pack --skill train-sentence-transformers-waybarrios
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: train-sentence-transformers
Source: https://github.com/waybarrios/opencode-power-pack/tree/main/skills/train-sentence-transformers
Command: npx skills add https://github.com/waybarrios/opencode-power-pack --skill train-sentence-transformers-waybarrios

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sentence-transformers, datasets, trackio, and includes scripts (resource) and references (resource) components.

What problem does it solve? Training embedding and reranking models with sentence-transformers involves many failure-prone decisions: matching losses to dataset shapes, choosing evaluators and metric keys, configuring training arguments, and avoiding silent pitfalls like saturated Sigmoid activations or dense SPLADE outputs. This Skill routes each training task to the correct production template and reference documentation so the resulting training script follows proven patterns. ## Core Features & Use Cases - Three model-type tracks: Dedicated loss, evaluator, and template guidance for SentenceTransformer bi-encoders, CrossEncoder rerankers, and SparseEncoder (SPLADE) models. - Production training templates: Ready-to-copy scripts covering distillation, LoRA, Matryoshka, multi-dataset, multilingual, and listwise variants, plus a hard-negative mining CLI. - Operational guardrails: Enforced baseline evaluation, smoke tests, VERDICT result lines, Hub push handling, and troubleshooting recipes for common failures. - Use Case: Fine-tune a bi-encoder on domain query-passage pairs with mined hard negatives, evaluate with NanoBEIR, and push the improved model to the Hugging Face Hub. ## Quick Start Ask the agent to fine-tune a sentence-transformers embedding model on your dataset and let it select the right loss, evaluator, and training template for the task.

Frequently Asked Questions about train-sentence-transformers

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune a SentenceTransformer model on my own data?▼

Identify your data shape first, then pick the matching loss: pairs use MultipleNegativesRankingLoss, scored pairs use CoSENTLoss, and labeled pairs use OnlineContrastiveLoss. Copy the production template script, replace the model and dataset names, smoke-test with max_steps=1, then run the full training.

What loss should I use for sentence-transformers training?▼

Match the loss to your dataset shape, not the reverse. Use MultipleNegativesRankingLoss for (anchor, positive) pairs, CoSENTLoss for float-scored pairs, BinaryCrossEntropyLoss for cross-encoder labeled pairs, and SpladeLoss wrapping a sparse loss for SPLADE models.

How do I mine hard negatives for retrieval model training?▼

Use the mine_hard_negatives utility with a retriever model over your (anchor, positive) dataset, sampling from the top-100 retrieved candidates. Optionally filter false negatives with a cross-encoder using max_score or relative_margin thresholds, then train with a contrastive loss.

Can I train sentence-transformers models on multiple GPUs?▼

Yes, training uses accelerate under the hood, so accelerate launch enables DDP without code changes. For models over 3B parameters, configure FSDP or DeepSpeed via accelerate config, though evaluators do not run mid-training under FSDP.

Why does my CrossEncoder eval nDCG collapse after distillation training?▼

The default Sigmoid activation saturates raw logits during predict(), collapsing eval rankings for non-BCE losses. Construct the model with activation_fn=torch.nn.Identity() for distillation, listwise, and pairwise losses; only BCE and CrossEntropy losses tolerate the default Sigmoid.

Does SPLADE training work with decoder LLM base models?▼

No, SPLADE requires a fill-mask or AutoModelForMaskedLM-compatible checkpoint, so decoder LLMs cannot be used. Use encoder-only MLM models like BERT, DistilBERT, or XLM-RoBERTa, or continue from an existing SPLADE checkpoint such as naver/splade family models.

Related Skills

Explore More Data & Analytics Skills→
noahsilve123noahsilve123
1

train-sentence-transformers

Automate training and fine-tuning of sentence-transformers models with Hugging Face Hub.

Community
Advanced
WhaleylawWhaleylaw

archon-rag-specialist

Index and retrieve semantically relevant document chunks for retrieval-augmented generation with Archon.

Community
Advanced
hao-ai-labhao-ai-lab
3.9k

summarize-run

Retrieve W&B or local metrics and generate markdown comparison reports.

Official
Basic
aisa-groupaisa-group
90

rag-implementation

Build retrieval-augmented generation systems with embeddings, vector stores, and reranking.

Official
Intermediate