confidence-score

Appends structured confidence assessment tables to every naniteAgent troubleshooting response.

Updated Aug 13, 2026
One-click install
npx skills add https://github.com/crystalzhangbai/TS-Agent-Azure --skill confidence-score-crystalzhangbai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: confidence-score
Source: https://github.com/crystalzhangbai/TS-Agent-Azure/tree/main/bridge/naniteagent-playground/naniteagent/skills/confidence-score
Command: npx skills add https://github.com/crystalzhangbai/TS-Agent-Azure --skill confidence-score-crystalzhangbai

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? AI troubleshooting responses often lack transparency about how reliable their conclusions are, leaving engineers unsure whether to trust a diagnosis or escalate. This Skill enforces a mandatory, standardized confidence assessment on every response so users can gauge reliability at a glance. ## Core Features & Use Cases - Three-Dimension Scoring: Rates Data Source (40%), Query Accuracy (35%), and Diagnosis (25%) with 🟢/🟡/🔴 icons and computes a weighted overall percentage. - Auto-Scoring Triggers: Automatically adjusts scores based on concrete events such as KQL query row counts, MCP tool failures, schema verification, and speculative language. - Special-Case Handling: Provides scoring rules for configuration guides, code reviews, dashboard links, and multi-step investigations. - Use Case: After running an azuremcp Kusto query that returns BGP flap data for an ExpressRoute circuit, the agent appends a confidence table showing 🟢 across all dimensions with a 92% overall score, signaling the findings are safe to use in case notes. ## Quick Start Ask the agent to troubleshoot an Azure networking issue and include the confidence score table assessing data source, query accuracy, and diagnosis for its findings.

Frequently Asked Questions about confidence-score

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add a confidence score to AI troubleshooting responses?▼

Apply the confidence score framework which appends a structured table rating Data Source, Query Accuracy, and Diagnosis with 🟢/🟡/🔴 icons. The overall score is computed as a weighted average: 40% data source, 35% query accuracy, and 25% diagnosis.

How is the overall confidence score calculated?▼

The overall score uses the formula: Data Source × 0.40 + Query Accuracy × 0.35 + Diagnosis × 0.25. Icons map to numeric values where 🟢 equals 90%, 🟡 equals 67%, and 🔴 equals 25%.

What triggers a low confidence score in KQL query troubleshooting?▼

A 🔴 rating is triggered when a Kusto query returns zero rows even after broadening the time range, when the MCP server is unreachable, when unverified table or column names are used, or when the root cause is stated without supporting data evidence.

Does confidence scoring apply to configuration guides and code reviews?▼

Yes, special-case rules cover non-diagnostic responses. Configuration guides score Data Source 🟢 when based on verified documentation with Query Accuracy and Diagnosis marked N/A, while code reviews score Diagnosis based on review quality.

When should I escalate instead of trusting a troubleshooting response?▼

Escalate or gather more data when the overall score falls below 50% (🔴), which indicates speculative diagnosis without live telemetry. Scores between 50-84% (🟡) warrant critical review and possibly additional queries before acting.