cua-driver

Automates native macOS apps and browser UI through the local computer-use MCP server.

Updated Jul 7, 2026
One-click install
npx skills add https://github.com/jamsdoescode/Erik-s-Portfolio --skill cua-driver-jamsdoescode
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: cua-driver
Source: https://github.com/jamsdoescode/Erik-s-Portfolio/tree/main/openclicky/AppResources/OpenClicky/OpenClickyBundledSkills/cua-driver
Command: npx skills add https://github.com/jamsdoescode/Erik-s-Portfolio --skill cua-driver-jamsdoescode

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Driving real macOS applications and browser interfaces programmatically is fragile: synthetic clicks steal focus, accessibility trees are sparse, and actions silently fail without verification. This Skill gives agents a disciplined, background-safe loop for operating GUI apps without disrupting the user's foreground work. ## Core Features & Use Cases - Snapshot-Verified Automation: Enforces a snapshot-before-and-after invariant using get_window_state so every click, keystroke, or value change is confirmed against the accessibility tree. - Background-Safe Operation: Launches apps and opens URLs via launch_app without focus steal, cursor warping, or Space switching, so users keep typing in their current app. - Web App Handling: Covers Chromium, WebKit, Electron, and Tauri quirks including sparse AX trees, minimized-window keyboard-commit failures, and the windows-over-tabs pattern. - Use Case: Ask the agent to open a URL in your default browser, fill a form in Numbers, or read your calendar events, and it drives the real app in the background while verifying each step. ## Quick Start Ask the agent to open the Downloads folder in Finder or navigate to a website in your default browser, and it will launch the app, snapshot its window state, act by element index, and verify the result.

Frequently Asked Questions about cua-driver

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate a macOS app without stealing focus?▼

Use launch_app with the app's bundle id to start it in the background, then call get_window_state to snapshot its accessibility tree and act via element_index clicks. Never use the open CLI or AppleScript activate, which always foreground the target.

How do I open a URL in the user's default browser programmatically?▼

Resolve the default HTTPS browser bundle id from LaunchServices, then call launch_app with that bundle id and the urls array. This opens a new background window using the user's normal logged-in profile, avoiding the focus-stealing Cmd+L omnibox flow.

Why does pressing Return not work on a minimized browser window?▼

Keyboard commit events reach the app but renderer focus is not established on minimized windows, so Return fires as a silent no-op or system beep. Use set_value to write the field directly, or AX-click a Submit or Go button instead.

Why do I get 'No cached AX state' errors when clicking elements?▼

Element indices are cached per (pid, window_id) pair and replaced on every snapshot, so indices from a previous turn or a different window are stale. Call get_window_state with the same window_id in the same turn before acting.

Can this skill record or replay automation trajectories?▼

No. OpenClicky's managed runtime does not expose recording or replay tools in this build. For regression evidence, save screenshots, logs, and a plain-language step list instead of attempting trajectory capture.

What are the limitations for canvas apps like Figma or Blender?▼

Canvas and custom-drawn surfaces are not reachable through the accessibility tree, so element_index actions fail. Coordinate clicks are only valid through explicit OpenClicky pixel routes; otherwise the agent should stop and report the missing capability.