Awesome GitHub RepositoriesProcess-Level Input Injectors

UI-TARS-desktop is a cross-platform desktop application designed to automate software interface interactions. It functions as a local agent environment that interprets graphical user interfaces through multimodal visual-language model reasoning, allowing it to navigate and manipulate software by simulating human-like mouse and keyboard inputs. The platform distinguishes itself by executing all visual recognition and decision-making logic directly on the host machine. This local inference model ensures that screen data and sensitive information remain private, as no processing is offloaded to

Cua is an agent benchmarking and desktop automation platform designed to evaluate autonomous agents and execute repetitive tasks within isolated, virtualized environments. It provides a framework for provisioning consistent workspaces and measuring agent performance against standardized desktop operations. The platform distinguishes itself by integrating virtual machine orchestration with headless interaction capabilities. By leveraging hypervisor-based virtualization, it runs operating systems at near-native speeds, while its automation layer injects commands directly into application proces

cc-haha is a cross-platform desktop agent and computer use framework that enables large language models to control local operating systems through screenshots, clicks, and keystrokes. It functions as an AI coding workbench and orchestration platform, allowing for the management of multi-project workflows and the coordination of multiple agents executing complex tasks in parallel. The system includes a model backend gateway to connect various artificial intelligence providers and local models to autonomous agents. It features a centralized permission gate for authorizing sensitive commands and

Agent-S is a multimodal AI agent and LLM desktop automation framework designed to control operating systems through graphical user interface interactions. It functions as a computer use interface, utilizing vision-language grounding to translate natural language goals into precise screen coordinates and system actions. The project differentiates itself by combining structured accessibility tree inspection with vision-based element localization. It manages cross-application workflows by mapping conceptual descriptions to physical pixels and simulating low-level keyboard and mouse events to mov

Apollo is a remote desktop streaming server designed to transmit a low-latency video feed of a host screen to remote clients using hardware-accelerated encoding. It functions as a remote input controller and virtual display manager, allowing the host to match the resolution and framerate of connected devices. The system includes a gamepad input emulator that simulates controller hardware on the host machine to translate remote inputs into local events. Server configurations and client pairing are managed through a web-based remote access interface. The project also covers bidirectional clipb

Input Leap is a software KVM switch and multi-machine input controller that allows a single keyboard and mouse to control several physical workstations over a local network. It functions as a network input emulator, sending keyboard and mouse signals to remote operating systems to manage a multi-machine workspace as if it were a single system with multiple monitors. The project also serves as a cross-platform clipboard synchronizer, enabling the sharing of copied text and data between different operating systems so that content can be pasted across all connected machines.

Kinto es un motor de mapas de teclado específico para aplicaciones y un remapeador de teclado contextual. Funciona como una utilidad basada en configuración que cambia automáticamente los mapas de teclado y las teclas modificadoras según la ventana de la aplicación activa. El proyecto actúa como un gestor de entrada multiplataforma y emulador de atajos de escritorio, diseñado específicamente para replicar los atajos de teclado y comportamientos de diseño de macOS en Linux y Windows. Esto permite un flujo de trabajo al estilo Mac en Linux al igualar los comportamientos de los atajos de teclado en diferentes sistemas operativos. El sistema proporciona capacidades para el remapeo de entrada personalizado y la capacidad de definir atajos específicos de la aplicación. Gestiona el mapeo de teclado contextual para mantener la consistencia en los diseños de teclado y los patrones de teclas modificadoras en varios entornos de escritorio.

OptiKey es una suite de tecnología asistiva y sistema de entrada basado en la mirada, diseñado para proporcionar acceso a computadoras y herramientas de comunicación. Funciona como un teclado virtual de seguimiento ocular y software de comunicación aumentativa y alternativa, permitiendo a los usuarios realizar entradas de texto y control del sistema a través de hardware de seguimiento ocular u otros dispositivos punteros. El sistema se diferencia por una capa de selección agnóstica a la entrada que soporta diversas fuentes, incluyendo rastreadores oculares, controladores de juegos y ratones de cabeza. Utiliza un mecanismo de disparo por tiempo de permanencia (dwell-time) y selección por fijación de la mirada para traducir los movimientos oculares en clics de ratón, pulsaciones de teclado y navegación por pantalla. El software cubre una amplia gama de capacidades, incluyendo simulación de entrada a nivel de SO para emulación de ratón y teclado, un motor de predicción basado en diccionario para escritura acelerada y definiciones de diseño basadas en XML para estructuras de teclado personalizadas. También incorpora síntesis de texto a voz, tableros de comunicación basados en símbolos para usuarios no verbales y herramientas de automatización avanzadas para secuenciar comandos complejos. La interfaz es altamente personalizable, soportando configuraciones de múltiples monitores, controles de transparencia y magnificación de áreas objetivo para mejorar la precisión.

AI0x0.com is a multimodal AI desktop assistant and cross-application wrapper. It provides a floating interface overlay that integrates large language models into any active software application to facilitate global querying and text automation. The system distinguishes itself through the ability to process real-time screen captures for visual analysis and utilize a voice pipeline for hands-free speech-to-text and text-to-speech interaction. It further enables direct AI content injection by simulating keyboard input to insert generated responses into active software fields. The project includ

AntiMicroX is a gamepad input manager and automation tool that translates gamepad inputs into keyboard and mouse commands. It functions as an input remapper capable of translating physical controller buttons and joystick movements into system actions for applications that lack native controller support. The system features window detection to automatically switch between custom input profiles based on the currently active application. It includes tools for game controller automation through complex macros and provides hardware standardization to ensure diverse controller models behave consist

OpenTabletDriver is a user-mode graphics tablet driver and USB HID input manager that translates raw USB input from digitizer hardware into system cursor and pen events. It serves as a hardware abstraction layer that normalizes input from various tablet brands into a consistent API, providing a cross-platform pen interface with pressure and tilt sensitivity. The project distinguishes itself through a plugin-based architecture and a profile-based configuration system, allowing for custom filters, interpolators, and persistent hardware setups. It includes specialized tools for hardware driver d

OSWorld is an evaluation framework and multimodal agent benchmark designed to test the ability of large language models to complete complex tasks within virtualized operating system environments. It provides a virtualized desktop sandbox and a virtual machine orchestrator to deploy, snapshot, and reset cloud-based desktops, ensuring reproducible test states for AI agent interactions. The system distinguishes itself by providing an OS-level action space that translates model decisions into mouse clicks, keyboard inputs, and system commands. It employs a standardized interface to integrate vari