skills.rest
castform-ai
castform-aiOFFICIAL
Explore all skills by this author →
GitHub

generate-data

Create train and eval JSONL datasets for Castform runs.

49|3|Updated Jun 27, 2025
Search Tags:#rag#traces#jsonl#training data#dataset generation#castform
One-click install
npx skills add https://github.com/castform-ai/benchmax --skill generate-data
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: generate-data
Source: https://github.com/castform-ai/benchmax/tree/main/src/benchmax/cli/scaffold/skills/generate-data
Command: npx skills add https://github.com/castform-ai/benchmax --skill generate-data

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you create the datasets needed to start and validate a Castform run without manually assembling every example format or source type.

Core Features & Use Cases

  • Generic dataset creation: Write train and eval JSONL files for prompt and ground-truth tasks.
  • RAG data generation: Produce question-answer pairs from a real corpus for retrieval and citation workflows.
  • Trace-based data extraction: Turn recorded agent traces into structured training rows for model improvement.
  • Use Case: A team can prepare a small baseline dataset, generate corpus-based QA pairs, or convert Braintrust traces into training data before launching a run.

Quick Start

Use this Skill to generate Castform-ready train and eval datasets from your task description, corpus, or traces.

Frequently Asked Questions about generate-data

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate training datasets from a text corpus for RAG workflows?▼

To generate RAG training datasets, you provide a Castform-compatible corpus and the Skill produces question-answer pairs formatted as JSONL. These files are rollout-ready for retrieval, citation workflows, and reward computation.

Can I turn recorded agent traces into JSONL training data?▼

Trace-based data extraction converts recorded agent traces into structured JSONL training rows. The Skill processes Castform-compatible trace inputs to build datasets for model improvement and baseline validation.

What format do training and eval datasets need to be in for Castform runs?▼

Training and eval datasets require generic prompt and ground_truth JSONL formats for Castform runs. Task rows must contain valid prompt fields, and the Skill preserves rollout-ready formats for reward computation and baseline validation.

Do I need existing task rows to create evaluation datasets?▼

Yes, creating evaluation datasets requires task rows with valid prompt fields. The Skill uses these rows alongside Castform-compatible corpus or trace inputs to generate rollout-ready eval files for baseline validation.

What is the best way to build train and eval datasets without manual formatting?▼

Automated dataset generation builds train and eval JSONL files from your task descriptions, corpora, or traces without manual assembly. It outputs rollout-ready formats for reward computation and baseline validation.

Related Skills

Explore More Data & Analytics Skills→
castform-aicastform-ai
49

design-environment

Design castform BaseEnv training environments for reinforcement-learning workflows.

Official
Advanced
castform-aicastform-ai
49

view-progress

Monitor castform training runs and inspect status, reward scalars, and logs.

Official
Intermediate
AgentientAgentient
2

rag-patterns

Configure Vertex AI RAG pipelines for grounding agent outputs in private data sources.

Official
Advanced
Bakery88Bakery88

comfyui-lora-data-gathering

Gather, curate, and auto-caption LoRA training data into a ready dataset folder.

Community
Advanced