mac-use
CommunityOCR-driven macOS GUI automation.
Authorpolyuiislab
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Automate repetitive GUI interactions on macOS by recognizing on-screen text and providing a reliable, element-based way to interact with apps.
Core Features & Use Cases
- OCR-based text detection and element identification in macOS apps
- Numbered element clicking, typing, scrolling, and key presses across macOS windows
- Robust window activation, coordinate mapping, and fallback canvas coordinates for unlabeled icons
- Use cases include automating routine UI tasks, data entry, and GUI testing on macOS
Quick Start
Open a macOS app, run the screenshot command to detect elements, then click a numbered element or type text.
Dependency Matrix
Required Modules
pyobjc-framework-VisionpyautoguiPillow
Components
scripts
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: mac-use Download link: https://github.com/polyuiislab/infiAgent/archive/main.zip#mac-use Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.