What problem does it solve? Driving native desktop applications (browsers, Finder, Mail, chat clients) programmatically usually requires moving the user's cursor and stealing keyboard focus, interrupting their work. This Skill lets an AI agent click, type, scroll, and drag inside the user's real apps in the background, on macOS, Windows, and Linux. ## Core Features & Use Cases - Screenshot capture with numbered element overlays: Capture any app's window with Set-of-Mark overlays and an accessibility-tree index, so elements can be targeted by index instead of pixel coordinates. - Full GUI action vocabulary: Click, double-click, right-click, drag, scroll, type, and send platform-specific keyboard shortcuts, with optional inline post-action captures for verification. - Background operation: Input is routed to the target app without raising windows, switching virtual desktops, or moving the real OS cursor. - Use Case: Ask the agent to reply to a message in a native chat app while you keep typing in your editor; it captures the chat window, clicks the input field by element index, types the reply, and verifies with a follow-up screenshot. ## Quick Start Ask the agent to open your browser, capture the current window, and click a specific element using the computer_use tool.
