Awesome GitHub RepositoriesProcess-Level Input Injectors

UI-TARS-desktop is a cross-platform desktop application designed to automate software interface interactions. It functions as a local agent environment that interprets graphical user interfaces through multimodal visual-language model reasoning, allowing it to navigate and manipulate software by simulating human-like mouse and keyboard inputs. The platform distinguishes itself by executing all visual recognition and decision-making logic directly on the host machine. This local inference model ensures that screen data and sensitive information remain private, as no processing is offloaded to

Cua is an agent benchmarking and desktop automation platform designed to evaluate autonomous agents and execute repetitive tasks within isolated, virtualized environments. It provides a framework for provisioning consistent workspaces and measuring agent performance against standardized desktop operations. The platform distinguishes itself by integrating virtual machine orchestration with headless interaction capabilities. By leveraging hypervisor-based virtualization, it runs operating systems at near-native speeds, while its automation layer injects commands directly into application proces

cc-haha is a cross-platform desktop agent and computer use framework that enables large language models to control local operating systems through screenshots, clicks, and keystrokes. It functions as an AI coding workbench and orchestration platform, allowing for the management of multi-project workflows and the coordination of multiple agents executing complex tasks in parallel. The system includes a model backend gateway to connect various artificial intelligence providers and local models to autonomous agents. It features a centralized permission gate for authorizing sensitive commands and

Agent-S is a multimodal AI agent and LLM desktop automation framework designed to control operating systems through graphical user interface interactions. It functions as a computer use interface, utilizing vision-language grounding to translate natural language goals into precise screen coordinates and system actions. The project differentiates itself by combining structured accessibility tree inspection with vision-based element localization. It manages cross-application workflows by mapping conceptual descriptions to physical pixels and simulating low-level keyboard and mouse events to mov

Apollo is a remote desktop streaming server designed to transmit a low-latency video feed of a host screen to remote clients using hardware-accelerated encoding. It functions as a remote input controller and virtual display manager, allowing the host to match the resolution and framerate of connected devices. The system includes a gamepad input emulator that simulates controller hardware on the host machine to translate remote inputs into local events. Server configurations and client pairing are managed through a web-based remote access interface. The project also covers bidirectional clipb

Input Leap is a software KVM switch and multi-machine input controller that allows a single keyboard and mouse to control several physical workstations over a local network. It functions as a network input emulator, sending keyboard and mouse signals to remote operating systems to manage a multi-machine workspace as if it were a single system with multiple monitors. The project also serves as a cross-platform clipboard synchronizer, enabling the sharing of copied text and data between different operating systems so that content can be pasted across all connected machines.

Kinto 是一个特定于应用的键位映射引擎与上下文键盘重映射器。它作为一个配置驱动的实用程序，根据当前活动的应用程序窗口自动切换键盘映射与修饰键。 该项目充当跨平台输入管理器与桌面快捷键模拟器，专门用于在 Linux 与 Windows 上复制 macOS 的键盘快捷键与布局行为。这通过在不同操作系统间匹配键盘快捷键行为，实现了 Linux 上的 Mac 风格工作流。 该系统提供了自定义输入重映射的能力，以及定义特定应用快捷键的功能。它管理上下文键盘映射，以在各种桌面环境中保持键盘布局与修饰键模式的一致性。

OptiKey 是一个辅助技术套件和基于注视的输入系统，旨在提供计算机访问和通信工具。它作为眼动追踪虚拟键盘和增强与替代通信软件，允许用户通过眼动追踪硬件或其他指针设备执行文本输入和系统控制。 该系统通过支持多种来源（包括眼动仪、游戏控制器和头部鼠标）的输入无关选择层脱颖而出。它利用停留时间触发机制和注视固定选择，将眼球运动转换为鼠标点击、键盘敲击和屏幕导航。 该软件涵盖了广泛的能力，包括用于鼠标和键盘模拟的 OS 级输入模拟、用于加速打字的字典驱动预测引擎，以及用于自定义键盘结构的基于 XML 的布局定义。它还结合了文本转语音合成、用于非语言用户的基于符号的通信板，以及用于对复杂命令进行排序的高级自动化工具。 该界面高度可定制，支持多显示器配置、透明度控制和目标区域放大，以提高精度。

AI0x0.com 是一款多模态 AI 桌面助手和跨应用程序包装器。它提供了一个浮动界面覆盖层，将大语言模型集成到任何活动的软件应用程序中，以促进全局查询和文本自动化。 该系统通过处理实时屏幕截图进行视觉分析以及利用语音管道进行免提语音转文本和文本转语音交互的能力而脱颖而出。它还通过模拟键盘输入将生成的响应插入到活动软件字段中，从而实现直接的 AI 内容注入。 该项目包括一个检索增强生成 (RAG) 知识库，可搜索本地文档库和实时 Web 数据。它支持用于在不同 AI API 之间切换的多模型提供程序接口、用于第三方集成的插件系统，以及用于生成多媒体文章的工具。 用户可以管理自定义功能预设并收藏对话历史记录以供将来检索。

AntiMicroX is a gamepad input manager and automation tool that translates gamepad inputs into keyboard and mouse commands. It functions as an input remapper capable of translating physical controller buttons and joystick movements into system actions for applications that lack native controller support. The system features window detection to automatically switch between custom input profiles based on the currently active application. It includes tools for game controller automation through complex macros and provides hardware standardization to ensure diverse controller models behave consist

OpenTabletDriver is a user-mode graphics tablet driver and USB HID input manager that translates raw USB input from digitizer hardware into system cursor and pen events. It serves as a hardware abstraction layer that normalizes input from various tablet brands into a consistent API, providing a cross-platform pen interface with pressure and tilt sensitivity. The project distinguishes itself through a plugin-based architecture and a profile-based configuration system, allowing for custom filters, interpolators, and persistent hardware setups. It includes specialized tools for hardware driver d

OSWorld is an evaluation framework and multimodal agent benchmark designed to test the ability of large language models to complete complex tasks within virtualized operating system environments. It provides a virtualized desktop sandbox and a virtual machine orchestrator to deploy, snapshot, and reset cloud-based desktops, ensuring reproducible test states for AI agent interactions. The system distinguishes itself by providing an OS-level action space that translates model decisions into mouse clicks, keyboard inputs, and system commands. It employs a standardized interface to integrate vari