1 个仓库
Awesome GitHub RepositoriesVisual UI Labeling
Overlays numerical markers or labels on detected user interface components for AI referencing.
Distinct from Object Detection: Focuses on adding referenceable markers to UI elements rather than just detecting their existence
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Visual UI Labeling. Refine with filters or upvote what's useful.
Awesome Visual UI Labeling GitHub Repositories
用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
- othersideai/self-operating-computer
OthersideAI/self-operating-computer
10,153在 GitHub 上查看This project is a computer control framework that uses multimodal vision models to simulate mouse and keyboard inputs for automating desktop tasks. It functions as an autonomous agent and vision-based orchestrator that interprets screen visuals to interact with user interfaces. The system employs vision language models and object detection to locate and click interface elements. It utilizes visual grounding to overlay numerical markers on UI components and uses optical character recognition to map on-screen text to precise pixel coordinates. The framework supports voice-controlled computing
Overlays numerical markers on detected UI components to help the AI reference specific elements by ID.
Pythonautomationopenaipyautogui
