baseline-assessment

Establish the runnable baseline, evaluation protocol, and target metrics before research changes.

Updated Aug 18, 2026
One-click install
npx skills add https://github.com/Nitrogen216/awesome-dsh-mods --skill baseline-assessment-nitrogen216
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: baseline-assessment
Source: https://github.com/Nitrogen216/awesome-dsh-mods/tree/main/modes/dsh-autoresearch/skills/baseline-assessment
Command: npx skills add https://github.com/Nitrogen216/awesome-dsh-mods --skill baseline-assessment-nitrogen216

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Research experiments often proceed without a trustworthy reference point, making it impossible to tell whether a new method actually improves on the status quo. This Skill freezes the current baseline, evaluation protocol, primary metric, and failure profile before any method changes begin. ## Core Features & Use Cases - Baseline Inspection: Reads the active implementation, evaluation code, configs, data splits, and raw results to identify the actual method under test rather than relying on stale summaries. - Baseline Artifact Generation: Creates research/BASELINE.md capturing run id, commit, config, seeds, budget, aggregate metrics, comparators, target delta, and localized failure profiles. - Research Contract Setup: Generates research/RESEARCH_CONTRACT.md from a preset template, persisting the entrance mode, deliverable, and fixed evaluation protocol. - Use Case: Before modifying a model training pipeline, run this Skill to record the official baseline's measured performance and failure modes, so every subsequent experiment is compared against a fixed, reproducible reference. ## Quick Start Assess the current implementation, evaluation protocol, and baseline metrics, then write the baseline artifact before any research changes.

Frequently Asked Questions about baseline-assessment

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I establish a baseline before changing a research method?▼

Inspect the current implementation, evaluation code, configs, splits, and raw results, then record the baseline run id, commit, seeds, budget, and aggregate metric in a baseline artifact. Run the official baseline only when comparable evidence is absent, and never tune it during this phase.

What should a research baseline document include?▼

A baseline document should include the objective and primary metric direction, run id, commit, config, command, split, seeds, budget, raw-result paths, aggregate and per-seed metrics, the strongest existing comparator, target delta, guardrail metrics, and a localized failure profile.

Can I reuse an existing baseline for my experiment?▼

Reuse an existing baseline only when its code, data split, preprocessing, training budget, and metric definition match the intended comparison. Otherwise the comparison is invalid and you should run the official baseline or report the comparability blocker.

What happens if no runnable baseline exists for a broad research direction?▼

Identify the strongest feasible reference implementation and establish a minimal faithful benchmark. If no runnable comparison is possible under the authorized resources, report that concrete blocker instead of inventing a baseline.

When should the evaluation protocol be changed after baseline assessment?▼

The evaluation protocol stays fixed through screening and confirmation unless the user explicitly changes it. Material changes to the objective, metric, protocol, data use, or compute budget require user authorization, with oracle consultation only when multiple defensible alternatives remain.