Awesome GitHub RepositoriesProcess-Level Input Injectors

UI-TARS-desktop is a cross-platform desktop application designed to automate software interface interactions. It functions as a local agent environment that interprets graphical user interfaces through multimodal visual-language model reasoning, allowing it to navigate and manipulate software by simulating human-like mouse and keyboard inputs. The platform distinguishes itself by executing all visual recognition and decision-making logic directly on the host machine. This local inference model ensures that screen data and sensitive information remain private, as no processing is offloaded to

Cua is an agent benchmarking and desktop automation platform designed to evaluate autonomous agents and execute repetitive tasks within isolated, virtualized environments. It provides a framework for provisioning consistent workspaces and measuring agent performance against standardized desktop operations. The platform distinguishes itself by integrating virtual machine orchestration with headless interaction capabilities. By leveraging hypervisor-based virtualization, it runs operating systems at near-native speeds, while its automation layer injects commands directly into application proces

cc-haha is a cross-platform desktop agent and computer use framework that enables large language models to control local operating systems through screenshots, clicks, and keystrokes. It functions as an AI coding workbench and orchestration platform, allowing for the management of multi-project workflows and the coordination of multiple agents executing complex tasks in parallel. The system includes a model backend gateway to connect various artificial intelligence providers and local models to autonomous agents. It features a centralized permission gate for authorizing sensitive commands and

Agent-S is a multimodal AI agent and LLM desktop automation framework designed to control operating systems through graphical user interface interactions. It functions as a computer use interface, utilizing vision-language grounding to translate natural language goals into precise screen coordinates and system actions. The project differentiates itself by combining structured accessibility tree inspection with vision-based element localization. It manages cross-application workflows by mapping conceptual descriptions to physical pixels and simulating low-level keyboard and mouse events to mov

Apollo is a remote desktop streaming server designed to transmit a low-latency video feed of a host screen to remote clients using hardware-accelerated encoding. It functions as a remote input controller and virtual display manager, allowing the host to match the resolution and framerate of connected devices. The system includes a gamepad input emulator that simulates controller hardware on the host machine to translate remote inputs into local events. Server configurations and client pairing are managed through a web-based remote access interface. The project also covers bidirectional clipb

Input Leap is a software KVM switch and multi-machine input controller that allows a single keyboard and mouse to control several physical workstations over a local network. It functions as a network input emulator, sending keyboard and mouse signals to remote operating systems to manage a multi-machine workspace as if it were a single system with multiple monitors. The project also serves as a cross-platform clipboard synchronizer, enabling the sharing of copied text and data between different operating systems so that content can be pasted across all connected machines.

Kinto este un motor de keymap specific aplicației și un remapper contextual de tastatură. Acesta funcționează ca un utilitar bazat pe configurare care comută automat mapările tastaturii și tastele modificatoare în funcție de fereastra aplicației active. Proiectul acționează ca un manager de input cross-platform și emulator de scurtături desktop, conceput special pentru a replica scurtăturile de tastatură și comportamentele de layout macOS pe Linux și Windows. Acest lucru permite un flux de lucru în stil Mac pe Linux prin potrivirea comportamentelor scurtăturilor de tastatură pe diferite sisteme de operare. Sistemul oferă capabilități pentru remapparea personalizată a input-ului și capacitatea de a defini scurtături specifice aplicației. Gestionează maparea contextuală a tastaturii pentru a menține consistența în layout-urile tastaturii și tiparele tastelor modificatoare pe diverse medii desktop.

OptiKey este o suită de tehnologie asistivă și un sistem de input bazat pe privire, conceput pentru a oferi acces la calculator și instrumente de comunicare. Funcționează ca o tastatură virtuală cu eye-tracking și software de comunicare augmentativă și alternativă, permițând utilizatorilor să introducă text și să controleze sistemul prin hardware de eye-tracking sau alte dispozitive de pointer. Sistemul se diferențiază printr-un strat de selecție agnostic față de input care suportă surse diverse, inclusiv eye-trackere, controllere de joc și head mouse-uri. Utilizează un mecanism de declanșare bazat pe dwell-time și selecție prin fixarea privirii pentru a traduce mișcările ochilor în click-uri de mouse, apăsări de taste și navigare pe ecran. Software-ul acoperă o gamă largă de capabilități, inclusiv simularea input-ului la nivel de OS pentru emularea mouse-ului și tastaturii, un motor de predicție bazat pe dicționar pentru tastare accelerată și definiții de layout bazate pe XML pentru structuri de tastatură personalizate. De asemenea, încorporează sinteza text-to-speech, table de comunicare bazate pe simboluri pentru utilizatorii non-verbali și instrumente avansate de automatizare pentru secvențierea comenzilor complexe. Interfața este extrem de personalizabilă, suportând configurații multi-monitor, controale de transparență și mărirea zonei țintă pentru a îmbunătăți precizia.

AI0x0.com is a multimodal AI desktop assistant and cross-application wrapper. It provides a floating interface overlay that integrates large language models into any active software application to facilitate global querying and text automation. The system distinguishes itself through the ability to process real-time screen captures for visual analysis and utilize a voice pipeline for hands-free speech-to-text and text-to-speech interaction. It further enables direct AI content injection by simulating keyboard input to insert generated responses into active software fields. The project includ

AntiMicroX is a gamepad input manager and automation tool that translates gamepad inputs into keyboard and mouse commands. It functions as an input remapper capable of translating physical controller buttons and joystick movements into system actions for applications that lack native controller support. The system features window detection to automatically switch between custom input profiles based on the currently active application. It includes tools for game controller automation through complex macros and provides hardware standardization to ensure diverse controller models behave consist

OpenTabletDriver is a user-mode graphics tablet driver and USB HID input manager that translates raw USB input from digitizer hardware into system cursor and pen events. It serves as a hardware abstraction layer that normalizes input from various tablet brands into a consistent API, providing a cross-platform pen interface with pressure and tilt sensitivity. The project distinguishes itself through a plugin-based architecture and a profile-based configuration system, allowing for custom filters, interpolators, and persistent hardware setups. It includes specialized tools for hardware driver d

OSWorld is an evaluation framework and multimodal agent benchmark designed to test the ability of large language models to complete complex tasks within virtualized operating system environments. It provides a virtualized desktop sandbox and a virtual machine orchestrator to deploy, snapshot, and reset cloud-based desktops, ensuring reproducible test states for AI agent interactions. The system distinguishes itself by providing an OS-level action space that translates model decisions into mouse clicks, keyboard inputs, and system commands. It employs a standardized interface to integrate vari