skills.rest
tribgames
tribgamesOFFICIAL
Explore all skills by this author →
GitHub

computer-use

Controls the local Windows desktop through the Mixdog computer tool.

6|Updated Jun 25, 2026
Search Tags:#accessibility#computer use#windows#desktop automation#gui testing#input control#ui observation
One-click install
npx skills add https://github.com/tribgames/mixdog --skill computer-use-tribgames
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: computer-use
Source: https://github.com/tribgames/mixdog/tree/main/src/defaults/skills/computer-use
Command: npx skills add https://github.com/tribgames/mixdog --skill computer-use-tribgames

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Automating native Windows applications, external browser windows, and OS dialogs requires reliable UI observation and input delivery that shell commands and web tools cannot provide. ## Core Features & Use Cases - Window observation and capture: Inspect windows via accessibility trees, OCR marks, screenshots, and zoomed frame regions before acting. - Structured input actions: Click, type, scroll, drag, and send keys using semantic refs, with background or foreground delivery modes. - Verification and recovery: Use bounded verify predicates, diagnose backend readiness, and follow strict pause/resume and cleanup flows after interruptions. - Use Case: Ask the agent to fill a form in an external Chrome window or operate a native Windows settings dialog while preserving the user's session and focus. ## Quick Start Use the computer tool to capture the external Chrome window and click the submit button in the open form.

Frequently Asked Questions about computer-use

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate clicks and typing in a Windows desktop app?▼

Capture the target window to get fresh semantic refs, then issue an act call with a click followed by type or key actions. The tool returns a new observation after each act so you can verify the result before continuing.

When should I use computer use instead of browser automation?▼

Use computer use for user-designated external Chrome, Edge, or Firefox windows and native app UI. Mixdog in-app browser pages belong to the browser tool, and URL research belongs to web_fetch or web_search.

What is the difference between background and foreground input delivery?▼

Background delivery sends native window messages without moving the physical pointer and is the default. Foreground uses the real cursor and keyboard focus when a gesture requires it or the user requests a visible demonstration.

Why are my element refs rejected as expired?▼

Refs, marks, and frames expire after 60 seconds or after any UI mutation. Use the recovery observation returned by the failed action if available, otherwise capture the window again to obtain fresh refs.

Can computer use bypass CAPTCHAs or permission dialogs?▼

No. Permission denials, CAPTCHAs, 2FA prompts, and user stops are never fallback signals. The skill reports the situation or hands off to the user rather than attempting any bypass.

What happens if the user interrupts an automation session?▼

Input stops and only list, diagnose, and wait_for_user remain available while paused. After a resume, a fresh capture is required; after timeout or cancellation, no further input is authorized.

Related Skills

Explore More Software Engineering Skills→
leogurjaleogurja
1

computer-use

Inspect and control visible local app windows through the Orca CLI.

Community
Intermediate
JhonMA82JhonMA82

executing-plans

Executes written implementation plans task-by-task with review checkpoints and verification steps.

Community
Intermediate
ainlperainlper

github-deep-research

Research GitHub repositories and generate structured markdown reports.

Community
Advanced
yunseo-kimyunseo-kim
1

mcporter

Interact with MCP servers and tools via a command-line interface.

Community
Intermediate