Detect UI elements in images using a YOLO-based detector.
npx skills add https://github.com/Throokie/claude-code-skills --skill ui-detector
Please help me install this Agent Skill. Skill: ui-detector Source: https://github.com/Throokie/claude-code-skills/tree/main/skills/ui-detector Command: npx skills add https://github.com/Throokie/claude-code-skills --skill ui-detector
在 UI 自动化场景下，提供一个基于 YOLO 的界面元素检测工具，帮助快速识别按钮、输入框、二维码等元素，减少手动标注与脚本开发成本。
Create a new project with ui-detector new <project>, then start annotation with ui-detector annotate <project> to train and detect UI elements.
High-intent search queries and answers about installing and using this skill.
To detect UI elements in screenshots for automated testing, use a YOLO-based detector to identify buttons, inputs, and QR codes. This approach streamlines semi-automated interface interactions by automatically locating elements in desktop or mobile images.
To train a custom YOLO model for GUI components, use an integrated annotation and training workflow. Create an isolated project, annotate your specific UI element data, and execute the training process to build a detector tailored to your automation needs.
Yes, you can use computer vision to automate mobile and desktop interface interactions. A YOLO-based detector supports on-demand inference across both platforms, enabling automated clicking and interaction with detected elements like buttons and inputs.
The best way to manage multiple UI automation datasets is using multi-project isolation. Creating independent workspaces for different automation tasks prevents data混淆, ensuring your training workflows and annotations remain cleanly separated.
Yes, you need Python 3.11 to run these YOLO-based GUI testing scripts. The detector fulfills explicit dependencies on this specific Python version and requires a portable project structure with a dedicated scripts directory to function properly.
connectwithprakashSet up Python projects with uv for dependency management and environment configuration.
testdriveraiAutomate click, type, hover, and scroll actions for end-to-end tests with TestDriver.
AvatarGanymedeAutomate end-to-end web application testing with Playwright.
PPPPandaControls mouse, keyboard, and screen capture for GUI workflows via pyautogui-based actions with optional OpenCV template matching.
