arize-prompt-optimization

What problem does it solve?

This Skill helps you systematically improve the performance of your Large Language Model (LLM) prompts by leveraging production trace data, evaluations, and annotations, ensuring your prompts are effective and efficient.

Core Features & Use Cases

• Prompt Extraction: Automatically identify and extract prompts from various span types within your trace data.
• Performance Analysis: Gather and analyze performance signals like evaluation scores, human annotations, and error rates associated with specific prompts.
• Data-Driven Optimization: Utilize a structured meta-prompting approach to generate improved prompt versions based on identified failure patterns.
• Use Case: Debug a chatbot prompt that is generating factually incorrect responses by analyzing traces of user interactions, identifying common errors, and using the skill's meta-prompt to refine the system instructions for better accuracy.

Quick Start

Use the arize-prompt-optimization skill to extract the current prompt from the latest LLM span in your project.