trulens-dataset-curation

What problem does it solve?

Creating reliable evaluation data for LLMs is time-consuming and error-prone. This Skill helps you build ground-truth datasets that enable consistent measurement of model outputs against expected responses, chunks, and retrieval results, empowering teams to track improvements across versions.

Core Features & Use Cases

• Ground-truth dataset structure: query, query_id, expected_response, and expected_chunks fields.
• Easy persistence and retrieval with TruLens to support versioned evaluations.
• Facilitate sharing and collaboration across teams to standardize evaluation.

Quick Start

1. Install prerequisites: pip install trulens pandas
2. Initialize TruSession from trulens.core import TruSession session = TruSession()
3. Build a dataset import pandas as pd data = { "query": ["What is TruLens?"], "expected_response": ["TruLens is an open-source library for evaluating and tracing AI agents."], "expected_chunks": [["TruLens"]] } ground_truth_df = pd.DataFrame(data)
4. Persist and load session.add_ground_truth_to_dataset(dataset_name="my_dataset", ground_truth_df=ground_truth_df, dataset_metadata={"domain":"TruLens QA","version":"1.0"}) df = session.get_ground_truth("my_dataset")