Score a script against a fixed rubric using only script and rubric paths.
npx skills add https://github.com/Jason5330/ai-self-eval --skill cheat-score-blind
Please help me install this Agent Skill. Skill: cheat-score-blind Source: https://github.com/Jason5330/ai-self-eval/tree/main/skills/cheat-score-blind Command: npx skills add https://github.com/Jason5330/ai-self-eval --skill cheat-score-blind
This internal sub-agent provides a fully isolated, multi-dimensional rubric score for a given script by consuming only script_path and rubric_notes_path, ensuring no leakage of actual outcomes or state data into the main conversation.
Spawn cheat-score-blind with a given script_path and rubric_notes_path to obtain a strict JSON score.
High-intent search queries and answers about installing and using this skill.
Isolated rubric scoring evaluates a script using only the script_path and rubric_notes_path inputs, returning a deterministic JSON object with N dimension scores (0-5) and a confidence enum. This prevents actual outcomes or state data from contaminating the evaluation.
A blind rubric evaluation outputs a strict JSON object containing N dimension scores ranging from 0 to 5, a confidence enum, and a one-line rationale for each dimension. This deterministic format is suitable for calibration workflows.
To generate deterministic JSON scores, provide a script_path and rubric_notes_path to a blind scoring sub-agent. The evaluation isolates scoring to a fixed rubric and returns a strict JSON object with per-dimension scores, confidence levels, and rationales.
Yes, blind scoring specifically prevents data contamination by reading only the specified script and rubric files. It does not read state files, predictions, sessions, or any outcome data, ensuring a clean anchor for model evaluation.
You need a blind rubric evaluation when generating a clean calibration anchor without data leakage. It is required when strict output formats with per-dimension scores and confidence levels are necessary for objective model evaluation workflows.
No, cheat-score-blind does not read state files, predictions, sessions, or any outcome data. It strictly consumes script_path and rubric_notes_path to produce an objective JSON score and prevent data contamination.
wgpsecInspect outputs, logs, and evidence files for flag{...} patterns to determine CTF flag validity.
sean-m-cooperAudit codebases across nine dimensions and produce per-ecosystem scores with JSON evidence.
Jason5330Convert user-stated tasks into structured cheat-specs for Cheat-Predict.
XBuilderLABScore scripts blindly using rubric notes without user context or past data.
