review-comment-futures

Predict outcomes of PR review comments and track calibration over time.

Updated Jul 28, 2026
One-click install
npx skills add https://github.com/christian-byrne/comfy-skills --skill review-comment-futures-christian-byrne
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: review-comment-futures
Source: https://github.com/christian-byrne/comfy-skills/tree/main/skills/review-comment-futures
Command: npx skills add https://github.com/christian-byrne/comfy-skills --skill review-comment-futures-christian-byrne

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? Code reviewers rarely get feedback on whether their comments land well. This Skill turns external PR reviews into deliberate practice by recording a prediction for each comment (accept, pushback, or ignore) and scoring those predictions after the PR closes, revealing calibration patterns over time. ## Core Features & Use Cases - Prediction Logging: Annotate each posted review comment with a predicted outcome, confidence level, and rationale, stored as JSONL telemetry. - Automated Scoring: A script classifies actual outcomes via the GitHub API by inspecting review threads for author replies and resolution status. - Calibration Reports: Generate accuracy, per-outcome precision/recall, Brier score, and confidence-bucket statistics to spot anti-patterns like high ignore rates or overconfidence. - Use Case: After reviewing a colleague's PR on an external repository, log predictions for each comment, then run the scoring script a week later to see which comments were accepted, debated, or ignored. ## Quick Start After I post review comments on an external PR, help me record a prediction with confidence for each comment and log it to the telemetry file.

Frequently Asked Questions about review-comment-futures

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I track whether my code review comments are effective?▼

Log a prediction (accept, pushback, or ignore) with a confidence score for each comment you post on an external PR. After the PR closes, run the scoring script to compare predictions against actual outcomes and review your calibration report.

How are review comment outcomes scored automatically?▼

The scoring script checks each comment's review thread via the GitHub API. A reply from the PR author agreeing or a resolved thread counts as accept, a disagreeing reply counts as pushback, and no response after the PR closes counts as ignore.

What metrics does the calibration report include?▼

The report shows overall accuracy, per-outcome precision and recall for accept, pushback, and ignore, a Brier score for calibration quality, and accuracy broken down by confidence bucket (high, medium, low).

Can I score predictions for only one pull request?▼

Yes, run the scoring script with the --pr flag followed by owner/repo#number to score a single PR. You can also use --dry-run to preview scoring without writing changes, or --stats for the calibration report.

What does a high ignore rate on review comments mean?▼

A high ignore rate suggests your comments are being treated as noise and not addressed. The recommended response is to be more selective about which comments you post and focus on higher-signal feedback.