model-guided-research

Simulates falsifiable hypotheses against kept run logs before changing a live Grok Bot.

Updated Sep 11, 2026
One-click install
npx skills add https://github.com/JYeswak/grok_bot_playground --skill model-guided-research-jyeswak
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-guided-research
Source: https://github.com/JYeswak/grok_bot_playground/tree/main/plugin/skills/model-guided-research
Command: npx skills add https://github.com/JYeswak/grok_bot_playground --skill model-guided-research-jyeswak

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Product questions about a Grok Bot are usually answered by changing the live Bot first and reading the damage later. This Skill replaces that pattern with a local simulation loop: frame falsifiable hypotheses, replay kept logs and run arithmetic locally, and produce a readout where every number traces back to a reproducible run log. ## Core Features & Use Cases - Hypothesis framing: Restates each product question as a claim that can fail, with a measure and a fail line, plus written assumptions fixed before any number is produced. - Local simulation with kept logs: Replays kept run logs, sweeps stated parameter ranges, or computes arithmetic with python3, recording inputs, code, seeds, and outputs so any readout can be rerun to the same numbers. - Honest readouts: Reports effect with interval, verdict (supported, unsupported, or underpowered), the assumptions it leans on, and the single cheapest next live measurement. - Use Case: Before shortening a draft prompt on a live Bot, simulate the variant against kept logs to test whether it cuts mean draft length by 15% without raising owner rewrite rate, then ship the readout with its reproducing log. ## Quick Start Ask the assistant to use the model-guided-research skill to test what would happen if a prompt, threshold, or schedule changed on the Bot, before making any live change.

Frequently Asked Questions about model-guided-research

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test a Grok Bot prompt change before deploying it?▼

Frame the change as a falsifiable hypothesis with a measure and fail line, then simulate it locally by replaying kept run logs or sweeping parameter ranges with python3. The readout reports the effect, its interval, and a verdict before anything touches the live Bot.

What is a run log in simulation-based product research?▼

A run log records each hypothesis's inputs, assumption version, code or snippet, seed, raw outputs, and the exact command that reproduces them. A readout without a kept log is treated as an opinion, and a log that does not reproduce is a finding, not a foundation.

Can simulated results prove a Bot change will work in production?▼

No. The skill refuses causal claims from simulated data alone: it states the association, then names the specific confounder a live measurement would rule out. Every readout ends with the cheapest next live observation that would confirm or overturn the verdict.

When should I not use local simulation for Bot questions?▼

Do not use it for live experiments on the production Bot, incident response, or any change that sends, publishes, installs, or spends. Those need their own routine and explicit owner approval; simulation is read-only plus local compute.

What does an underpowered simulation verdict mean?▼

Underpowered means the available runs cannot detect the change being tested at a meaningful size. The skill reports this honestly with one line stating what change is detectable at current runs and how many runs the desired change would need.