1 repo
Awesome GitHub RepositoriesVisual Interface Parsers
Systems that decompose graphical interfaces into structured semantic elements for machine reasoning.
Distinguishing note: Focuses on the hierarchical decomposition of visual input rather than general image processing.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Visual Interface Parsers. Refine with filters or upvote what's useful.
Awesome Visual Interface Parsers GitHub Repositories
Find the best repos with AI.We'll search the best matching repositories with AI.
- microsoft/OmniParser
microsoft/OmniParser
24,377View on GitHubOmniParser is a multimodal interaction engine designed to function as a desktop automation agent. It interprets visual screen information to execute complex, multi-step tasks across operating system environments by bridging visual interface perception with language models. Through a continuous cycle of observation and command execution, the system grounds high-level natural language instructions into precise, coordinate-based actions. The project distinguishes itself by utilizing vision-based parsing to interact with software interfaces without requiring access to underlying application progr
Decomposes complex desktop screenshots into structured semantic elements to simplify visual input for reasoning models.
Jupyter Notebook
