eval-session-classify

Classify Datadog assistant sessions into satisfaction verdicts using session_id and LLM Observability traces.

150|23|Updated Feb 3, 2026
One-click install
npx skills add https://github.com/datadog-labs/agent-skills --skill eval-session-classify
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-session-classify
Source: https://github.com/datadog-labs/agent-skills/tree/main/dd-llmo/eval-session-classify
Command: npx skills add https://github.com/datadog-labs/agent-skills --skill eval-session-classify

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill enables automated classification of user satisfaction and intent in Datadog assistant sessions by analyzing session identifiers and related observability traces to produce a structured verdict.

Core Features & Use Cases

  • Input-driven session classification using a session_id and ROUTE_CONTEXT-derived signals (pre-, during-, and post-session data).
  • Integration of LLM Observability evaluations, trace data, and RUM signals to produce robust verdicts.
  • Output structured satisfaction verdicts with actionable recommendations for product teams.

Quick Start

Provide a standalone classification for a session_id to determine whether the user’s intent was satisfied.

Frequently Asked Questions about eval-session-classify

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I classify user intent satisfaction using LLM observability traces in Datadog?▼

Session intent satisfaction is classified by analyzing a session_id alongside Datadog LLM Observability traces and RUM signals across pre-, during-, and post-event windows to produce a structured satisfaction verdict.

What is the best way to automate session classification verdicts for Datadog assistant sessions?▼

Automated session classification verdicts are generated by integrating ROUTE_CONTEXT details, LLM Observability evaluations, and RUM signals to robustly measure whether user intent was satisfied.

How does the classification pipeline handle missing LLM observability traces during session evaluation?▼

The classification pipeline validates the presence of required data sources and handles missing LLM observability traces safely to ensure robust satisfaction verdicts without processing failures.

Can I use RUM signals and session_id to determine if a user's intent was met during a Datadog session?▼

Yes, you can provide a standalone session_id to measure whether user intent was satisfied by integrating RUM signals and LLM Observability traces across the session timeline.

Why does my session satisfaction classification require pre-, during-, and post-event data windows?▼

Pre-, during-, and post-event data windows are required to capture the complete session context, ensuring the classification pipeline produces robust and accurate satisfaction verdicts.