Awesome GitHub RepositoriesProcess-Level Input Injectors

UI-TARS-desktop is a cross-platform desktop application designed to automate software interface interactions. It functions as a local agent environment that interprets graphical user interfaces through multimodal visual-language model reasoning, allowing it to navigate and manipulate software by simulating human-like mouse and keyboard inputs. The platform distinguishes itself by executing all visual recognition and decision-making logic directly on the host machine. This local inference model ensures that screen data and sensitive information remain private, as no processing is offloaded to

Cua is an agent benchmarking and desktop automation platform designed to evaluate autonomous agents and execute repetitive tasks within isolated, virtualized environments. It provides a framework for provisioning consistent workspaces and measuring agent performance against standardized desktop operations. The platform distinguishes itself by integrating virtual machine orchestration with headless interaction capabilities. By leveraging hypervisor-based virtualization, it runs operating systems at near-native speeds, while its automation layer injects commands directly into application proces

cc-haha is a cross-platform desktop agent and computer use framework that enables large language models to control local operating systems through screenshots, clicks, and keystrokes. It functions as an AI coding workbench and orchestration platform, allowing for the management of multi-project workflows and the coordination of multiple agents executing complex tasks in parallel. The system includes a model backend gateway to connect various artificial intelligence providers and local models to autonomous agents. It features a centralized permission gate for authorizing sensitive commands and

Agent-S is a multimodal AI agent and LLM desktop automation framework designed to control operating systems through graphical user interface interactions. It functions as a computer use interface, utilizing vision-language grounding to translate natural language goals into precise screen coordinates and system actions. The project differentiates itself by combining structured accessibility tree inspection with vision-based element localization. It manages cross-application workflows by mapping conceptual descriptions to physical pixels and simulating low-level keyboard and mouse events to mov

Apollo is a remote desktop streaming server designed to transmit a low-latency video feed of a host screen to remote clients using hardware-accelerated encoding. It functions as a remote input controller and virtual display manager, allowing the host to match the resolution and framerate of connected devices. The system includes a gamepad input emulator that simulates controller hardware on the host machine to translate remote inputs into local events. Server configurations and client pairing are managed through a web-based remote access interface. The project also covers bidirectional clipb

Input Leap is a software KVM switch and multi-machine input controller that allows a single keyboard and mouse to control several physical workstations over a local network. It functions as a network input emulator, sending keyboard and mouse signals to remote operating systems to manage a multi-machine workspace as if it were a single system with multiple monitors. The project also serves as a cross-platform clipboard synchronizer, enabling the sharing of copied text and data between different operating systems so that content can be pasted across all connected machines.

Kinto is an application-specific keymap engine and contextual keyboard remapper. It functions as a configuration-driven utility that automatically switches keyboard mappings and modifier keys based on the active application window. The project acts as a cross-platform input manager and desktop shortcut emulator, specifically designed to replicate macOS keyboard shortcuts and layout behaviors on Linux and Windows. This enables a Mac-style workflow on Linux by matching keyboard shortcut behaviors across different operating systems. The system provides capabilities for custom input remapping an

OptiKey is an assistive technology suite and gaze-based input system designed to provide computer access and communication tools. It functions as an eye-tracking virtual keyboard and augmentative and alternative communication software, allowing users to perform text entry and system control via gaze-tracking hardware or other pointer devices. The system differentiates itself through an input-agnostic selection layer that supports diverse sources, including eye trackers, game controllers, and head mice. It utilizes a dwell-time trigger mechanism and gaze fixation selection to translate eye mov

AI0x0.com ist ein multimodaler KI-Desktop-Assistent und Cross-Application-Wrapper. Er bietet ein schwebendes Interface-Overlay, das Large Language Models in jede aktive Softwareanwendung integriert, um globale Abfragen und Textautomatisierung zu erleichtern. Das System zeichnet sich durch die Fähigkeit aus, Echtzeit-Bildschirmaufnahmen zur visuellen Analyse zu verarbeiten und eine Voice-Pipeline für freihändige Speech-to-Text- und Text-to-Speech-Interaktion zu nutzen. Es ermöglicht zudem die direkte KI-Content-Injektion durch Simulation von Tastatureingaben, um generierte Antworten in aktive Softwarefelder einzufügen. Das Projekt enthält eine RAG-Wissensdatenbank (Retrieval-Augmented Generation), die lokale Dokumentenbibliotheken und Echtzeit-Webdaten durchsucht. Es unterstützt eine Multi-Modell-Provider-Schnittstelle zum Wechseln zwischen verschiedenen KI-APIs, ein Plugin-System für Drittanbieter-Integrationen und Tools zur Generierung von Multimedia-Artikeln. Benutzer können benutzerdefinierte funktionale Presets verwalten und den Gesprächsverlauf für zukünftige Abrufe mit Lesezeichen versehen.

AntiMicroX is a gamepad input manager and automation tool that translates gamepad inputs into keyboard and mouse commands. It functions as an input remapper capable of translating physical controller buttons and joystick movements into system actions for applications that lack native controller support. The system features window detection to automatically switch between custom input profiles based on the currently active application. It includes tools for game controller automation through complex macros and provides hardware standardization to ensure diverse controller models behave consist

OpenTabletDriver is a user-mode graphics tablet driver and USB HID input manager that translates raw USB input from digitizer hardware into system cursor and pen events. It serves as a hardware abstraction layer that normalizes input from various tablet brands into a consistent API, providing a cross-platform pen interface with pressure and tilt sensitivity. The project distinguishes itself through a plugin-based architecture and a profile-based configuration system, allowing for custom filters, interpolators, and persistent hardware setups. It includes specialized tools for hardware driver d

OSWorld is an evaluation framework and multimodal agent benchmark designed to test the ability of large language models to complete complex tasks within virtualized operating system environments. It provides a virtualized desktop sandbox and a virtual machine orchestrator to deploy, snapshot, and reset cloud-based desktops, ensuring reproducible test states for AI agent interactions. The system distinguishes itself by providing an OS-level action space that translates model decisions into mouse clicks, keyboard inputs, and system commands. It employs a standardized interface to integrate vari