deep-q-rl

Train Q-networks with Russian Doll MCTS in scored discrete-action environments.

Updated Apr 20, 2026
One-click install
npx skills add https://github.com/thistleknot/skills --skill deep-q-rl
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: deep-q-rl
Source: https://github.com/thistleknot/skills/tree/main/deep-q-rl
Command: npx skills add https://github.com/thistleknot/skills --skill deep-q-rl

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill turns scored, discrete-action decision problems into an efficient training loop by learning a value function while using a progressive, search-guided policy improvement strategy.

Core Features & Use Cases

  • Dense score-based learning: builds a Q-style value head from a per-state evaluate(state) correlate instead of relying only on sparse terminal rewards.
  • Russian Doll MCTS with value-head leaves: runs progressive narrowing search so wide action spaces remain tractable, using the network (and a heuristic fallback) to evaluate search leaves.
  • AHA mistake correction: detects evaluation drops after a chosen action during training and applies immediate corrective replay signal.
  • Training progress annealing: anneals MCTS iteration counts, exploration, and funnel widths as the value function becomes more reliable.

Use it for environments like board games, turn-based strategy, or any simulation where you can enumerate discrete actions, encode state tensors, and compute a current-player-perspective scalar score that correlates with ultimate success.

Quick Start

Use the deep-q-rl skill to train an agent by implementing the ScoredEnvironment interface with encode_state, evaluate, legal_actions, apply, and is_terminal for your environment, then run self-play or rollout-based training with Russian Doll MCTS and AHA enabled for training.

Frequently Asked Questions about deep-q-rl

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train reinforcement learning policies in environments with dense state scoring instead of sparse terminal rewards?▼

Deep Q-learning with Russian Doll MCTS trains policies from dense state scores by building a Q-style value head from a per-state evaluate correlate, using a target network and replay buffer to optimize decisions without waiting for sparse terminal rewards.

How does Russian Doll MCTS keep progressive narrowing search tractable for wide discrete action spaces?▼

Russian Doll MCTS applies progressive narrowing search to wide discrete action spaces by using action categorization with weighted sampling and evaluating search leaves with a value-head Q-network, annealing funnel widths and exploration as the value function becomes more reliable.

Can I use self-play to train a value function for turn-based strategy games if I can encode state tensors and enumerate legal actions?▼

Yes, self-play training works for turn-based strategy games or board games where you can encode state tensors, enumerate discrete legal actions, compute a current-player-perspective scalar score, and simulate transitions with an apply function.

What is AHA mistake correction in deep Q-learning and when does it apply corrective replay signals?▼

AHA mistake correction detects evaluation drops after a chosen action during training and applies immediate corrective replay signal, improving policy decisions by penalizing moves that cause immediate state score degradation during self-play rollouts.

Do I need a bounded and calibrated evaluate score to use a Q-network with Bellman targets for policy training?▼

Yes, a bounded and calibrated evaluate score correlate is required to train the value-head Q-network with Bellman targets, ensuring the scalar per current-player POV accurately correlates with ultimate success for progressive narrowing search optimization.

How do I start training an agent using scored discrete-action environments with deep Q-learning?▼

Implement the ScoredEnvironment interface with encode_state, evaluate, legal_actions, apply, and is_terminal functions for your environment, then run self-play or rollout-based training with Russian Doll MCTS and AHA mistake correction enabled for progressive policy improvement.