zipbox-desktop

Controls a live X11/VNC desktop with xdotool mouse input and wmctrl window management.

1|Updated Jun 17, 2026
One-click install
npx skills add https://github.com/tribes-protocol/trading-harness --skill zipbox-desktop-tribes-protocol
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: zipbox-desktop
Source: https://github.com/tribes-protocol/trading-harness/tree/main/skills/zipbox-desktop
Command: npx skills add https://github.com/tribes-protocol/trading-harness --skill zipbox-desktop-tribes-protocol

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Automating a real, user-visible desktop requires more than HTTP calls: you need to move the mouse naturally, click, type, scroll, and manage windows across native applications on a running X11/KasmVNC display. This Skill provides the operational playbook for driving that desktop the way a person would, while the user watches and can take over. ## Core Features & Use Cases - Desktop Discovery & Toolchain Setup: Measure the true display geometry with xdpyinfo and wmctrl, and bootstrap xdotool, wmctrl, scrot, and imagemagick idempotently. - Human-Like Input: Generate eased, curved mouse paths in Python, plus clicking, dragging, typing, and scrolling via xdotool. - Window Management: List, activate, move, resize, maximize, and tile windows side by side with wmctrl, including the critical fix of removing maximized state before any move/resize. - Screenshot Capture: Capture the full root window with scrot and hand it to a vision-capable reader when the harness cannot render images. - Use Case: A user asks you to open two browsers side by side and compare pages. You measure the root dimensions, launch the windows, remove their maximized state, tile each to half the screen, and capture a screenshot to confirm the layout. ## Quick Start Ask the agent to start the desktop with zipbox-desktop up, then move the mouse to a target and click it while tiling two windows side by side on display :1.

Frequently Asked Questions about zipbox-desktop

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I control an X11 desktop programmatically with xdotool?▼

Use xdotool to move the mouse, click, type, and send keys against a target display such as :1. Combine it with wmctrl for window management and scrot for screenshots, after installing the toolchain with apt-get.

How do I make automated mouse movement look natural?▼

Generate an eased, slightly curved path in Python instead of teleporting with xdotool mousemove. Apply an ease-in-out curve over many small steps with a short sleep between moves, and add a gentle sinusoidal arc to mimic a human hand.

Why does wmctrl -e fail to move or resize a window?▼

A maximized window silently ignores wmctrl -e because _NET_WM_STATE_MAXIMIZED_VERT and _HORZ are set. Remove the maximized state with wmctrl -b remove,maximized_vert,maximized_horz, wait briefly, then retry the move or resize.

Why do my clicks land in the wrong place on a VNC desktop?▼

Root window pixels differ from CSS or logical pixels because the browser runs with a device-scale factor. Measure the true root size with xdpyinfo and multiply page coordinates by the scale factor before clicking.

When should I use desktop control instead of browser DevTools automation?▼

Use desktop control for native apps, window chrome, file managers, and cross-application window tiling. Use the DevTools-based browser path for precise in-page clicks and text extraction, which it handles more cleanly than synthetic mouse events.