What problem does it solve? Manually collecting LLM evaluation results (questions, answers, accuracy scores, token usage, latency) from Langfuse datasets and runs is tedious and error-prone. This Skill automates pulling raw benchmark data for financial LLM evaluations into structured CSV files. ## Core Features & Use Cases - Automated Data Extraction: Connects to a Langfuse instance via API/SDK to pull dataset items, run records, traces, scores, token usage, and latency. - Structured CSV Export: Writes consolidated evaluation rows (question, reference answer, actual answer, model, difficulty, Accuracy, reasoning_quality, tokens, cost) to /mnt/workspace/data/val_FullReport.csv. - Partial-Data Handling: Documents how to proceed when only some datasets (e.g., knowledge QA but not sentiment classification) have runs available. - Use Case: Before generating a financial LLM benchmark report comparing Claude model versions on CFA-style questions, run this Skill to fetch the latest evaluation data from Langfuse. ## Quick Start Activate the project virtual environment and run the fetch-data skill to export the latest Langfuse evaluation data to CSV.
