Skills
.Work. You
Rest

find-ui-element

Official

Locate UI elements visually.

Software Engineering#automation#visual search#bounding box#screenshot analysis#coordinate extraction#ui element
AuthorX-School-Academy
Version1.0.0
Installs0

System Documentation

What problem does it solve?

This Skill helps locate specific user interface elements within screenshots, which is crucial for automating interactions with graphical applications or websites when direct element selectors are unavailable.

Core Features & Use Cases

  • Visual Element Identification: Finds buttons, links, text fields, checkboxes, and other interactive elements based on their visual appearance in an image.
  • Coordinate Extraction: Provides precise bounding box coordinates and center points for identified elements.
  • Iterative Refinement: Employs a strategy of initial visual estimation, followed by verification with small crops, and then grid-based searching in nearby regions if the initial guess is incorrect.
  • Use Case: Automating a desktop application where UI elements change dynamically or lack stable identifiers. You can use this skill to find a "Submit" button in a screenshot to click it.

Quick Start

Use the find-ui-element skill to locate the button with the text 'Login' in the provided screenshot.

Dependency Matrix

Required Modules

None required

Components

scripts

💻 Claude Code Installation

Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.

Please help me install this Skill:
Name: find-ui-element
Download link: https://github.com/X-School-Academy/skill-pilot/archive/main.zip#find-ui-element

Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
View Source Repository

Agent Skills Search Helper

Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.