computah

Controls desktop applications through screenshots, mouse clicks, and keyboard input on Windows and Linux.

1|Updated Dec 8, 2025
One-click install
npx skills add https://github.com/inayayousfi/myconfig --skill computah-inayayousfi
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: computah
Source: https://github.com/inayayousfi/myconfig/tree/main/dotfiles/ai/.agents/skills/computah
Command: npx skills add https://github.com/inayayousfi/myconfig --skill computah-inayayousfi

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? When no purpose-built tool, API, or MCP exists for a task, automation stalls. This Skill lets an AI operate the computer the way a person does — capturing the screen, locating targets visually, and acting through mouse and keyboard — so canvas apps, games, remote desktops, and installers remain automatable. ## Core Features & Use Cases - Cross-platform input scripts: Unified command interface (capture, crop, move, click, drag, scroll, type, key) backed by native tooling on Windows, WSL, and Linux (X11/Wayland via spectacle, grim, xdotool, ydotool, wtype). - Capture-locate-act-verify loop: Every action is verified with a fresh screenshot, with crop-based zooming for targets smaller than ~40px and explicit stop conditions after repeated misses. - Safety guardrails: Confirmation before first capture, non-destructive typing rules, and mandatory stops before deleting, overwriting, or acting on payment and account surfaces. - Use Case: Automate a GUI-only installer on a Linux desktop: capture the screen, locate the Next button, click it, and verify each dialog advances — no accessibility APIs required. ## Quick Start Ask the agent to open an application and complete a GUI task using the computah desktop-control skill, confirming the first screen capture.

Frequently Asked Questions about computah

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate a GUI application that has no API?▼

Use screenshot-based desktop control: capture the screen, locate the target visually, then act with synthetic mouse and keyboard input. This skill provides scripts for Windows, WSL, and Linux that expose capture, click, type, and key commands through one interface.

How to click small or hard-to-target elements on screen?▼

Crop the existing screenshot around the target region and read the crop at native resolution, then add the crop origin back to get screen coordinates. Estimates from downscaled full screenshots are only accurate to roughly ±15px, so cropping is required for targets under ~40px.

Does this desktop automation work on Wayland and macOS?▼

Linux Wayland sessions are supported through grim, ydotool, and wtype backends, and X11 through xdotool, scrot, and spectacle. macOS is not implemented; the skill instructs the agent to stop and tell the user.

Why did my automated click miss the target?▼

Misses usually come from coordinate scaling: if the image-reading facility downscales the capture, visual estimates must be adjusted by its reported factor before being passed to the script. After two misses on the same target, stop and change approach rather than retrying.

When should I use desktop control instead of an API or MCP tool?▼

Only when no purpose-built tool fits the task, or after that tool fails and cannot be repaired. Mouse and keyboard automation works everywhere but is slower and more fragile than native APIs, so it is the fallback, not the default.