codex-readiness-unit-test

Evaluates AGENTS.md and PLANS.md documentation quality with deterministic checks and in-session LLM evals.

Updated Feb 4, 2026
One-click install
npx skills add https://github.com/pchemguy/AISandbox --skill codex-readiness-unit-test-pchemguy
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: codex-readiness-unit-test
Source: https://github.com/pchemguy/AISandbox/tree/main/docs/AgentSkills/openai/skills/skills/.experimental/codex-readiness-unit-test
Command: npx skills add https://github.com/pchemguy/AISandbox --skill codex-readiness-unit-test-pchemguy

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? Teams using Codex agents often lack a way to verify that their AGENTS.md and PLANS.md onboarding documentation is complete, concise, and actually executable. This Skill runs a structured readiness report combining filesystem checks with in-session LLM evaluations, without requiring any external API keys or SDKs. ## Core Features & Use Cases - Deterministic Checks: Verifies AGENTS.md exists, stays under 300 lines, references a valid planning markdown file, and that config.toml is present in the repo or user .codex directory. - In-Session LLM Evaluations: Assesses project context clarity, copy-pastable build/test commands, and documented dev/build/test loops using strict JSON evaluator prompts with a retry-and-fix loop. - Execute Mode with Safety Controls: Runs the documented build/test loop via a confirmed plan.json with denylisted command blocking, soft/hard timeouts, and per-step logs. - Use Case: Before onboarding a new agent to a repository, run the report to confirm AGENTS.md documents the build and test commands correctly, then execute mode validates the documented loop actually passes. ## Quick Start Ask the agent to run the Codex Readiness unit test report in read-only mode against the current working directory.

Frequently Asked Questions about codex-readiness-unit-test

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run a Codex readiness check on my AGENTS.md file?▼

Run the collect_evidence.py and deterministic_rules.py scripts against your current working directory, then complete the in-session LLM checks and generate the report with scoring.py. The skill writes report.json, report.html, and summary.json under .codex-readiness-unit-test/<timestamp>/.

What checks validate AGENTS.md documentation quality?▼

Deterministic checks verify AGENTS.md exists, stays under 300 lines, references an existing planning markdown file, and that config.toml is present. LLM checks evaluate project context clarity, copy-pastable build/test commands, and documented dev/build/test loops.

Does the readiness test require an OpenAI API key?▼

No API key or external HTTP calls are required. All LLM evaluation happens in-session using the provided reference prompts, and deterministic checks use only local filesystem scanning and command execution.

What is the difference between read-only and execute mode?▼

Read-only mode runs evidence collection, deterministic rules, and LLM checks without executing any commands, marking the execution check as NOT_RUN. Execute mode additionally runs a user-confirmed plan.json of build/test commands and produces execution logs and an execution summary.

How are unsafe commands handled during execute mode?▼

Commands matching the denylist, such as rm -rf, git clean -xfd, mkfs, or dd, are never executed and are marked FAIL. Each command also runs under a soft timeout of 600 seconds with a hard cap of three times that value.