confidence-calibration

Generates calibrated confidence scores for analytical outputs using three-pass prompting.

Updated May 13, 2026
One-click install
npx skills add https://github.com/Syntari-International-Inc/syntari-nexus-plugin --skill confidence-calibration-syntari-international-inc
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: confidence-calibration
Source: https://github.com/Syntari-International-Inc/syntari-nexus-plugin/tree/main/skills/confidence-calibration
Command: npx skills add https://github.com/Syntari-International-Inc/syntari-nexus-plugin --skill confidence-calibration-syntari-international-inc

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Standard AI-generated analyses express identical certainty regardless of underlying data quality, leaving users unable to judge how much to trust a credit memo, deal score, or underwriting projection. This Skill measures internal consistency across calibration perspectives to produce a quantified confidence score for any analytical output. ## Core Features & Use Cases - Three-Pass Calibration Protocol: Runs conservative, neutral, and optimistic analysis passes, then measures agreement to derive a confidence score from 0.00 to 1.00. - Structured Confidence Assessment Block: Appends a standardized section with score, calibration stance, confidence drivers, data gaps, and three-pass agreement level. - Domain-Specific Calibration: Applies tailored calibration drivers for credit memos, deal screening, CRE underwriting, and financial models. - Use Case: After generating a 100-point deal screening score, run this protocol to discover that only two of three passes agree on the thesis direction, yielding a 0.62 confidence score with flagged data gaps in management quality assessment. ## Quick Start Ask the AI to run confidence calibration on the investment analysis it just produced and append the confidence assessment block.

Frequently Asked Questions about confidence-calibration

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I measure confidence in AI-generated investment analysis?▼

Run the SteerConf three-pass protocol: generate conservative, neutral, and optimistic versions of the analysis, then measure agreement across passes. The consistency level maps to a confidence score from 0.00 to 1.00, combined with data quality and source diversity factors.

What is the SteerConf confidence calibration protocol?▼

SteerConf is a prompt-only calibration technique that halves calibration error without infrastructure. It generates three analysis passes with different assumption stances and derives confidence from their agreement, weighted by verified datapoints and source diversity.

How is the confidence score calculated?▼

The score combines three weighted components: base agreement across passes (60%), ratio of verified datapoints (25%), and source diversity (15%). Scores above 0.80 indicate high consistency, while scores below 0.20 signal conflicting interpretations requiring human review.

Does confidence calibration work with credit memos and deal screening?▼

Yes, the protocol integrates with credit memos, deal screening, CRE underwriting, and financial models. Each domain has specific calibration drivers, such as independent data source counts for credit memos and T12 quality for CRE underwriting.

When should a low confidence score trigger human review?▼

Scores in the 0.20-0.49 range indicate high uncertainty where the three passes diverge, and scores below 0.20 indicate conflicting conservative and optimistic conclusions. Both cases flag insufficient data or fundamental ambiguity requiring human judgment.