1 repo
Awesome GitHub RepositoriesSemantic Mapping Engines
Mechanisms for translating raw visual data into labeled interactive interface primitives.
Distinguishing note: Focuses on the mapping of visual features to functional interface labels.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Semantic Mapping Engines. Refine with filters or upvote what's useful.
Awesome Semantic Mapping Engines GitHub Repositories
Find the best repos with AI.We'll search the best matching repositories with AI.
- microsoft/OmniParser
microsoft/OmniParser
24,377View on GitHubOmniParser is a multimodal interaction engine designed to function as a desktop automation agent. It interprets visual screen information to execute complex, multi-step tasks across operating system environments by bridging visual interface perception with language models. Through a continuous cycle of observation and command execution, the system grounds high-level natural language instructions into precise, coordinate-based actions. The project distinguishes itself by utilizing vision-based parsing to interact with software interfaces without requiring access to underlying application progr
Translates raw pixel data into labeled interactive components by matching visual features against interface primitives.
Jupyter Notebook
