Awesome GitHub RepositoriesMouse Automation

Open Interpreter is an autonomous agent runtime that translates natural language instructions into executable code to interact with local software and operating systems. It functions as an orchestration framework that connects language models to a secure execution environment, enabling the development of agents capable of managing system resources and performing complex tasks. To ensure safety, the system mandates explicit user verification before executing any generated code and provides robust isolation through containerized sandboxing. The project distinguishes itself through its deep inte

Karabiner-Elements is a system-level utility designed for advanced keyboard and mouse customization. It functions as a background service that intercepts raw hardware input signals at the driver level, allowing for the transformation of key presses and pointer movements before they reach the operating system. By utilizing virtual input device emulation, the software re-injects modified events into the system stream, enabling complex remapping, macro execution, and hardware-specific control. The project distinguishes itself through a sophisticated state-based logic engine that enables context-

This project is a comprehensive technical interview preparation resource and computer science interview guide. It serves as an educational reference for developers to study core software engineering fundamentals and common coding patterns required for employment screenings. The repository provides detailed guides and references covering data structures and algorithms, networking and security, operating systems, and web development. It specifically focuses on the implementation and complexity analysis of sorting, searching, and graph algorithms. The material encompasses a wide breadth of comp

Hammerspoon is a programmable automation engine for macOS that enables deep system-level control through a Lua scripting environment. By bridging high-level scripts with native Objective-C APIs, it allows users to interact with the operating system's accessibility tree, intercept hardware input streams, and manage the lifecycle of running applications. The project distinguishes itself through an event-driven architecture that registers asynchronous hooks for system notifications and hardware events. This allows for real-time automation, such as remapping keyboard and mouse inputs, managing wi

chromedp is a browser automation framework and driver that controls web browsers via the Chrome DevTools Protocol. It functions as a headless browser automation tool and web browser controller, enabling the programmatic management of browser sessions, targets, and network responses through a remote debugging interface. The project provides specialized capabilities for Chrome DevTools Protocol automation, including headless browser testing, web scraping and data extraction, and mobile device emulation. It also supports browser-based visual regression by capturing precise screenshots of web pag

Robotjs is a native Node.js automation library and desktop input simulator. It uses C++ bindings to provide low-level access to operating system functions, allowing for the programmatic control of the mouse and keyboard and the analysis of screen pixels. The library functions as a toolkit for automating user interfaces and desktop workflows, including those within Electron applications. It enables the simulation of key presses and mouse movements to automate interactions with desktop software and perform automated data entry. Its capabilities extend to screen pixel analysis, where it capture

PyAutoGUI is a Python GUI automation library and desktop automation framework. It provides a set of tools for programmatically controlling the mouse and keyboard to automate user interface interactions across different operating systems. The project functions as a cross-platform input simulator and computer vision screen scanner. It enables the simulation of keystrokes and cursor movements to perform repetitive tasks and utilizes screen analysis to locate specific images or pixel colors on the display. Its capability surface includes mouse and keyboard input simulation, screen image capture,

Agent-S is a multimodal AI agent and LLM desktop automation framework designed to control operating systems through graphical user interface interactions. It functions as a computer use interface, utilizing vision-language grounding to translate natural language goals into precise screen coordinates and system actions. The project differentiates itself by combining structured accessibility tree inspection with vision-based element localization. It manages cross-application workflows by mapping conceptual descriptions to physical pixels and simulating low-level keyboard and mouse events to mov

Robotgo is a cross-platform desktop automation framework for the Go programming language. It provides a comprehensive toolkit for programmatically interacting with graphical user interfaces, enabling developers to simulate human input, manage application windows, and monitor system-wide hardware events. The library distinguishes itself through its low-level system integration, utilizing a foreign function interface to interact directly with native operating system APIs. It employs pixel-buffer memory mapping and real-time screen capture to perform visual element identification, allowing for i

Bytebot is an LLM desktop automation framework and virtual Linux desktop environment. It enables AI agents to plan and execute mouse and keyboard actions on a virtual computer using natural language, allowing for autonomous desktop automation and the integration of legacy systems that lack native APIs. The system operates as an LLM API gateway and a Model Context Protocol server, routing requests across multiple language model providers with integrated load balancing and rate limiting. It provides isolated, containerized environments where agents use visual reasoning to interpret screenshots

SerpentAI is a game AI development kit and computer vision framework designed for building autonomous agents that interact with video games. It serves as a game input automation tool and a machine learning model integration engine, allowing developers to create agents that perceive game states and execute actions. The framework utilizes a plugin-based agent architecture to provide modular extensions for game-specific logic and behaviors. It features a specialized system for training, bundling, and deploying machine learning classifiers to recognize visual contexts and game states in real time

BetterJoy es una capa de software y driver para conectar mandos de Nintendo Switch Pro y Joy-Cons a un entorno Windows vía Bluetooth o USB. Funciona como un emulador de mando XInput y mapeador de entradas de gamepad, traduciendo las entradas del mando de Nintendo Switch a señales XInput para su uso con juegos de PC y emuladores. El proyecto incluye un emulador de ratón por giroscopio que mapea el giroscopio interno de los mandos de Nintendo Switch al movimiento del cursor del ratón del sistema. También proporciona una interfaz de configuración para remapear los botones del mando a teclas de teclado personalizadas. El software cubre la conectividad de hardware para mandos inalámbricos y cableados, traducción de entradas para software de juegos y la conversión de datos del giroscopio en coordenadas de pantalla relativas.

Phoenix es un automatizador de espacios de trabajo y gestor de ventanas para macOS que utiliza un motor de scripting de JavaScript para controlar comportamientos a nivel de sistema. Funciona como un puente de automatización de AppleScript, permitiendo a los usuarios manipular programáticamente los estados de las aplicaciones, la geometría de las ventanas y las interacciones del escritorio. El proyecto permite la creación de flujos de trabajo personalizados mediante la vinculación de atajos de teclado a funciones de JavaScript. Esto permite la automatización de acciones complejas del sistema, como organizar diseños de aplicaciones en múltiples pantallas y gestionar espacios virtuales. Su superficie de capacidades cubre la gestión de ventanas y aplicaciones, la ejecución asíncrona de comandos de shell y la automatización a nivel de sistema. También incluye utilidades para gestionar coordenadas de pantalla 2D, programar tareas recurrentes e interactuar con el usuario mediante notificaciones del sistema y ventanas modales. La aplicación admite un modo demonio para la operación en segundo plano y monitorea el sistema de archivos para recargar los archivos de configuración sin necesidad de reiniciar.

OptiKey es una suite de tecnología asistiva y sistema de entrada basado en la mirada, diseñado para proporcionar acceso a computadoras y herramientas de comunicación. Funciona como un teclado virtual de seguimiento ocular y software de comunicación aumentativa y alternativa, permitiendo a los usuarios realizar entradas de texto y control del sistema a través de hardware de seguimiento ocular u otros dispositivos punteros. El sistema se diferencia por una capa de selección agnóstica a la entrada que soporta diversas fuentes, incluyendo rastreadores oculares, controladores de juegos y ratones de cabeza. Utiliza un mecanismo de disparo por tiempo de permanencia (dwell-time) y selección por fijación de la mirada para traducir los movimientos oculares en clics de ratón, pulsaciones de teclado y navegación por pantalla. El software cubre una amplia gama de capacidades, incluyendo simulación de entrada a nivel de SO para emulación de ratón y teclado, un motor de predicción basado en diccionario para escritura acelerada y definiciones de diseño basadas en XML para estructuras de teclado personalizadas. También incorpora síntesis de texto a voz, tableros de comunicación basados en símbolos para usuarios no verbales y herramientas de automatización avanzadas para secuenciar comandos complejos. La interfaz es altamente personalizable, soportando configuraciones de múltiples monitores, controles de transparencia y magnificación de áreas objetivo para mejorar la precisión.

This is a Model Context Protocol server that exposes Windows desktop automation and system administration functions to large language models. It provides programmatic control of mouse, keyboard, windows, and UI elements on Windows through simulated user input, while also enabling LLMs to manage the Windows registry, processes, files, and execute PowerShell commands through a remote interface. The server supports multiple transport protocols including stdio, SSE, and streamable HTTP, allowing flexible integration with different language model clients. It implements OAuth 2.0 with PKCE for secu

P4wnP1 is a hardware-based USB HID attack platform and peripheral emulator. It functions as a tool for emulating USB keyboards and mice to execute automated keystroke payloads, as well as a WiFi-enabled remote access tool that provides a wireless bridge for network relay and SSH access. The project is distinguished by its ability to establish covert bidirectional communication channels and remote shells using raw HID reports, specifically to bridge air-gapped systems. It further enables wireless network interception and the routing of network traffic over WiFi to facilitate man-in-the-middle

AutoKey is a Linux GUI macro manager and Python desktop automation tool. It functions as a background daemon that simulates keyboard and mouse events via the X11 protocol to automate interactions with graphical user interfaces. The software provides a Python scripting engine for executing stateful automation and a text expansion utility that replaces short abbreviations with predefined content or scripted actions in real time. It differentiates itself by combining low-level input simulation with the ability to manage global state and persistent variables across different scripts. The project

xdotool is a command-line utility for X11 desktop automation that simulates keyboard and mouse input and manages application windows. It functions as a scriptable tool that maps sequential text instructions to low-level library calls to control the X11 graphical environment. The project enables the generation of synthetic keyboard and mouse events to automate interactions with graphical user interfaces. It also acts as a window manager controller, allowing users to programmatically move, resize, focus, and organize application windows and virtual desktops. The tool covers a wide range of cap