computer-control

List open windows with the windows action, focus the target with focus_window, then call read to get a numbered map of its controls. Click, fill, or scroll elements by their ref, which uses the OS accessibility API and needs no coordinates.

Call read on the focused window to enumerate controls via UI Automation, then pass the control's ref to click. The tool invokes the control's automation pattern directly, falling back to a real click only if no pattern is exposed.

The read and find actions rely on UI Automation, which is wired for Windows only. On other platforms, use the pixel path: take a screenshot and act on coordinates, while window listing still works everywhere.

Coordinates are pixels of the last screenshot, not desktop pixels, so they are only valid for the captured screen. Either capture the correct screen first or act by ref, which works regardless of which monitor the window is on.

Windows blocks synthetic input across the privilege boundary silently, so clicks to elevated windows do nothing. The window actions refuse outright; the app must be driven by hand or the agent restarted as administrator.

Use the browser tool for any web page, since it reads the DOM and clicks by element number, which is faster and cheaper in tokens. Computer control is for native applications, installers, and dialogs with no other interface.