Awesome GitHub RepositoriesProcess-Level Input Injectors

UI-TARS-desktop is a cross-platform desktop application designed to automate software interface interactions. It functions as a local agent environment that interprets graphical user interfaces through multimodal visual-language model reasoning, allowing it to navigate and manipulate software by simulating human-like mouse and keyboard inputs. The platform distinguishes itself by executing all visual recognition and decision-making logic directly on the host machine. This local inference model ensures that screen data and sensitive information remain private, as no processing is offloaded to

Cua is an agent benchmarking and desktop automation platform designed to evaluate autonomous agents and execute repetitive tasks within isolated, virtualized environments. It provides a framework for provisioning consistent workspaces and measuring agent performance against standardized desktop operations. The platform distinguishes itself by integrating virtual machine orchestration with headless interaction capabilities. By leveraging hypervisor-based virtualization, it runs operating systems at near-native speeds, while its automation layer injects commands directly into application proces

cc-haha is a cross-platform desktop agent and computer use framework that enables large language models to control local operating systems through screenshots, clicks, and keystrokes. It functions as an AI coding workbench and orchestration platform, allowing for the management of multi-project workflows and the coordination of multiple agents executing complex tasks in parallel. The system includes a model backend gateway to connect various artificial intelligence providers and local models to autonomous agents. It features a centralized permission gate for authorizing sensitive commands and

Agent-S is a multimodal AI agent and LLM desktop automation framework designed to control operating systems through graphical user interface interactions. It functions as a computer use interface, utilizing vision-language grounding to translate natural language goals into precise screen coordinates and system actions. The project differentiates itself by combining structured accessibility tree inspection with vision-based element localization. It manages cross-application workflows by mapping conceptual descriptions to physical pixels and simulating low-level keyboard and mouse events to mov

Apollo is a remote desktop streaming server designed to transmit a low-latency video feed of a host screen to remote clients using hardware-accelerated encoding. It functions as a remote input controller and virtual display manager, allowing the host to match the resolution and framerate of connected devices. The system includes a gamepad input emulator that simulates controller hardware on the host machine to translate remote inputs into local events. Server configurations and client pairing are managed through a web-based remote access interface. The project also covers bidirectional clipb

Input Leap is a software KVM switch and multi-machine input controller that allows a single keyboard and mouse to control several physical workstations over a local network. It functions as a network input emulator, sending keyboard and mouse signals to remote operating systems to manage a multi-machine workspace as if it were a single system with multiple monitors. The project also serves as a cross-platform clipboard synchronizer, enabling the sharing of copied text and data between different operating systems so that content can be pasted across all connected machines.

Kinto هو محرك خرائط مفاتيح خاص بالتطبيق ومُعيد تعيين لوحة مفاتيح سياقي. يعمل كأداة قائمة على التكوين تقوم تلقائياً بتبديل تعيينات لوحة المفاتيح ومفاتيح التعديل بناءً على نافذة التطبيق النشطة. يعمل المشروع كمدير إدخال عبر الأنظمة الأساسية ومحاكي اختصارات سطح المكتب، مصمم خصيصاً لتكرار اختصارات لوحة مفاتيح macOS وسلوكيات التخطيط على Linux و Windows. يتيح ذلك سير عمل بنمط Mac على Linux من خلال مطابقة سلوكيات اختصارات لوحة المفاتيح عبر أنظمة تشغيل مختلفة. يوفر النظام قدرات لإعادة تعيين الإدخال المخصص والقدرة على تحديد اختصارات خاصة بالتطبيق. يدير تعيين لوحة المفاتيح السياقي للحفاظ على الاتساق في تخطيطات لوحة المفاتيح وأنماط مفاتيح التعديل عبر بيئات سطح المكتب المختلفة.

OptiKey هي مجموعة تقنيات مساعدة ونظام إدخال قائم على تتبع العين مصمم لتوفير الوصول إلى الحاسوب وأدوات التواصل. تعمل كلوحة مفاتيح افتراضية لتتبع العين وبرمجيات تواصل بديلة ومعززة، مما يسمح للمستخدمين بإدخال النصوص والتحكم في النظام عبر أجهزة تتبع العين أو أجهزة التأشير الأخرى. يتميز النظام بطبقة اختيار محايدة للمدخلات تدعم مصادر متنوعة، بما في ذلك أجهزة تتبع العين، وأجهزة التحكم في الألعاب، وفأرة الرأس. يستخدم آلية تشغيل وقت المكوث (dwell-time) واختيار تثبيت العين لترجمة حركات العين إلى نقرات فأرة، وضغطات لوحة مفاتيح، وتنقل في الشاشة. تغطي البرمجيات مجموعة واسعة من الإمكانيات، بما في ذلك محاكاة الإدخال على مستوى نظام التشغيل لمحاكاة الفأرة ولوحة المفاتيح، ومحرك تنبؤ قائم على القاموس لتسريع الكتابة، وتعريفات تخطيط قائمة على XML لهياكل لوحة المفاتيح المخصصة. كما تدمج تركيب تحويل النص إلى كلام، ولوحات تواصل قائمة على الرموز للمستخدمين غير الناطقين، وأدوات أتمتة متقدمة لتسلسل الأوامر المعقدة. الواجهة قابلة للتخصيص بدرجة عالية، وتدعم تكوينات الشاشات المتعددة، وعناصر التحكم في الشفافية، وتكبير منطقة الهدف لتحسين الدقة.

AI0x0.com is a multimodal AI desktop assistant and cross-application wrapper. It provides a floating interface overlay that integrates large language models into any active software application to facilitate global querying and text automation. The system distinguishes itself through the ability to process real-time screen captures for visual analysis and utilize a voice pipeline for hands-free speech-to-text and text-to-speech interaction. It further enables direct AI content injection by simulating keyboard input to insert generated responses into active software fields. The project includ

AntiMicroX is a gamepad input manager and automation tool that translates gamepad inputs into keyboard and mouse commands. It functions as an input remapper capable of translating physical controller buttons and joystick movements into system actions for applications that lack native controller support. The system features window detection to automatically switch between custom input profiles based on the currently active application. It includes tools for game controller automation through complex macros and provides hardware standardization to ensure diverse controller models behave consist

OpenTabletDriver is a user-mode graphics tablet driver and USB HID input manager that translates raw USB input from digitizer hardware into system cursor and pen events. It serves as a hardware abstraction layer that normalizes input from various tablet brands into a consistent API, providing a cross-platform pen interface with pressure and tilt sensitivity. The project distinguishes itself through a plugin-based architecture and a profile-based configuration system, allowing for custom filters, interpolators, and persistent hardware setups. It includes specialized tools for hardware driver d

OSWorld is an evaluation framework and multimodal agent benchmark designed to test the ability of large language models to complete complex tasks within virtualized operating system environments. It provides a virtualized desktop sandbox and a virtual machine orchestrator to deploy, snapshot, and reset cloud-based desktops, ensuring reproducible test states for AI agent interactions. The system distinguishes itself by providing an OS-level action space that translates model decisions into mouse clicks, keyboard inputs, and system commands. It employs a standardized interface to integrate vari