skills.rest
pjherron
pjherronCOMMUNITY
Explore all skills by this author →
GitHub

model-roster-review

Review and build LLM model rosters for agentic routing tiers following ADR 0007.

Updated Jul 16, 2026
Search Tags:#llm routing#opencode#ollama#model audit#benchmark evaluation#model roster#agentic models
One-click install
npx skills add https://github.com/pjherron/hypoc --skill model-roster-review-pjherron
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-roster-review
Source: https://github.com/pjherron/hypoc/tree/main/hypoc/skills/model-roster-review
Command: npx skills add https://github.com/pjherron/hypoc --skill model-roster-review-pjherron

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Choosing which local or hosted models should fill each routing tier is error-prone: configs drift from what is actually pulled, benchmarks go stale, and broken or oversized models silently occupy slots. This Skill provides a structured audit and selection method so every roster entry is evidence-backed and runs on the target hardware. ## Core Features & Use Cases - Roster auditing: Reconciles configured models against ollama list, flags configured-but-not-pulled and pulled-but-broken entries, and sweeps registry pages for known issues. - Evidence-based ranking: Ranks candidates by agentic benchmarks (SWE-bench Verified, Terminal-Bench, Tau2, NL2Repo) rather than general benchmarks, with dated scores. - Hardware and jurisdiction fit: Assigns models to machine classes by RAM footprint and records vendor origin, hosting, and retention policy in structured fields. - Use Case: A team maintaining an OpenCode/Ollama setup runs a quarterly refresh: the Skill audits the current config, evaluates a newly released model against incumbents, and outputs a JSON roster keyed by router tier with keep/drop/monitor dispositions. ## Quick Start Ask the assistant to review the current ollama model roster against ADR 0007 and produce an updated tiered roster with dispositions.

Frequently Asked Questions about model-roster-review

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I audit an existing ollama model roster?▼

Compare the configured provider block in opencode.json against the output of ollama list. Configured-but-not-pulled and pulled-but-broken entries are flagged with reasons, then each survivor is checked for hardware fit and known registry issues.

Which benchmarks matter for choosing agentic coding models?▼

Rank candidates by SWE-bench Verified, Terminal-Bench, Tau2 for tool-calling reliability, and NL2Repo for repo-scale work. General benchmarks like MMLU are tiebreakers only because they do not predict tool-loop discipline.

How do I evaluate a newly released model against my current roster?▼

A new model must displace the incumbent primary on agentic benchmark evidence, fit a hardware class, and pass jurisdiction checks. Cross-lab benchmark deltas under about 3 points are treated as noise and should not trigger churn.

Does model size determine roster quality for local LLMs?▼

No. Past the workhorse class, dense 70B models lose to 30-35B MoE models on both speed and agentic benchmarks. The rule of thumb is a modelfile under roughly 60 percent of the target machine's RAM, spending headroom on context instead of parameters.

What happens to broken but promising models?▼

Models with known issues, such as macOS Metal bugs flagged on the registry page, go on a monitor list with a date and re-evaluation trigger. They are never silently dropped and never rostered until fixed.

Related Skills

Explore More Software Engineering Skills→
IrisGoLabIrisGoLab
6

win-ai-local

Manage Ollama models and run local LLM inference on Windows 10/11.

Official
Intermediate
AradotsoAradotso
2

duel-agents-routing

Route prompts across OpenAI and Anthropic models to minimize cost.

Official
Advanced
CleanExpoCleanExpo

token-budgeter

Enforce per-tier token budgets for Claude executions using tier-specific pricing.

Community
Basic
chiptoe-svgchiptoe-svg

add-ollama-tool

Expose locally hosted Ollama models as callable tools for NanoClaw container agents.

Community
Intermediate