zz-plugin-profile

Compute evidence blocks and conformance verdicts from plugin run traces without model calls.

Updated Sep 11, 2026
One-click install
npx skills add https://github.com/zhixuan312/zz-stack --skill zz-plugin-profile-zhixuan312
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: zz-plugin-profile
Source: https://github.com/zhixuan312/zz-stack/tree/main/catalog/zz/zz-plugin-eval/skills/zz-plugin-profile
Command: npx skills add https://github.com/zhixuan312/zz-stack --skill zz-plugin-profile-zhixuan312

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Evaluating a plugin requires real evidence, but raw run logs can be misleading — a number without its denominator can be wrong by orders of magnitude. This Skill computes the trace evidence block and conformance results for a plugin version deterministically, with an explicit sufficiency verdict, so no model ever guesses at the data. ## Core Features & Use Cases - Trace Evidence Block: Counts usable runs, returns, unplaced steps, per-tool calls and refusals, and never-called tools, all derived from the event log with explicit coverage denominators. - Sufficiency Verdict: Reports whether the evidence is sufficient for judging (5 usable runs floor), while treating a thin trace block as a valid outcome rather than a failure. - Three-Valued Conformance: Reports R1–R14 as true, false, or not_measured, refusing to score clauses the contract does not measure. - Use Case: After locating a third-party plugin with no local run history, run plugin_conform to get a starting ruler of which requirements are measured, then decide whether there is enough evidence to judge. ## Quick Start Run the profile stage for the located plugin and version to compute its trace evidence block, coverage, and conformance verdict.

Frequently Asked Questions about zz-plugin-profile

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate whether a plugin has enough run history to judge?▼

Run plugin_profile with the plugin name and version to compute the trace evidence block. The block reports a sufficiency verdict: 5 usable runs is the floor for a signal to exist. A thin trace block is reported as thin but does not stop the evaluation flow.

What does the trace evidence block contain for a plugin version?▼

The block counts runs and usable runs, returns (stages entered after later stages), unplaced steps the manifest declares no stage for, per-tool calls and refusals, never-called tools, and the coverage denominator all figures rest on.

Why does plugin_conform return not_measured for some requirements?▼

Five of the R1-R14 clauses are always not_measured because the contract refuses to score them rather than guess. Clauses that do not apply to a flow plugin also return not_measured, never true, so passes are only granted when actually measured.

What happens when a plugin has no usable runs at all?▼

A plugin nobody has used is treated as a correct and complete outcome, not a failure. The locate and profile stages still run and report; only both blocks being empty is a stop condition, and no substitute evidence should be invented.

Why must coverage be read before interpreting trace numbers?▼

Every figure in the trace block rests on the resolvable subset of events, such as 198 of 510. A number without its denominator can be wrong by orders of magnitude while looking healthy, so the coverage line must be stated when narrating the profile.