Projects sharing features with Autokey

Agent-S is a multimodal AI agent and LLM desktop automation framework designed to control operating systems through graphical user interface interactions. It functions as a computer use interface, utilizing vision-language grounding to translate natural language goals into precise screen coordinates and system actions. The project differentiates itself by combining structured accessibility tree inspection with vision-based element localization. It manages cross-application workflows by mapping conceptual descriptions to physical pixels and simulating low-level keyboard and mouse events to mov

This is a Model Context Protocol server that exposes Windows desktop automation and system administration functions to large language models. It provides programmatic control of mouse, keyboard, windows, and UI elements on Windows through simulated user input, while also enabling LLMs to manage the Windows registry, processes, files, and execute PowerShell commands through a remote interface. The server supports multiple transport protocols including stdio, SSE, and streamable HTTP, allowing flexible integration with different language model clients. It implements OAuth 2.0 with PKCE for secu

Robotjs is a native Node.js automation library and desktop input simulator. It uses C++ bindings to provide low-level access to operating system functions, allowing for the programmatic control of the mouse and keyboard and the analysis of screen pixels. The library functions as a toolkit for automating user interfaces and desktop workflows, including those within Electron applications. It enables the simulation of key presses and mouse movements to automate interactions with desktop software and perform automated data entry. Its capabilities extend to screen pixel analysis, where it capture

Hammerspoon is a programmable automation engine for macOS that enables deep system-level control through a Lua scripting environment. By bridging high-level scripts with native Objective-C APIs, it allows users to interact with the operating system's accessibility tree, intercept hardware input streams, and manage the lifecycle of running applications. The project distinguishes itself through an event-driven architecture that registers asynchronous hooks for system notifications and hardware events. This allows for real-time automation, such as remapping keyboard and mouse inputs, managing wi

This project is an autonomous desktop automation agent that interprets natural language instructions to control applications, browser interfaces, and system terminals. It functions as a cross-platform utility designed to manage complex workflows by integrating visual screen analysis with system-level input simulation. The agent distinguishes itself through its ability to perform tasks asynchronously, ensuring that web and terminal operations run in the background without interrupting the active user session or desktop focus. By combining computer vision to map interface elements with event-dr

xdotool is a command-line utility for X11 desktop automation that simulates keyboard and mouse input and manages application windows. It functions as a scriptable tool that maps sequential text instructions to low-level library calls to control the X11 graphical environment. The project enables the generation of synthetic keyboard and mouse events to automate interactions with graphical user interfaces. It also acts as a window manager controller, allowing users to programmatically move, resize, focus, and organize application windows and virtual desktops. The tool covers a wide range of cap

This project is a cross-platform input interception library and global keyboard event simulator for Windows and Linux. It provides a programmatic interface to monitor global keyboard events, simulate hardware input, and block system events to create custom hotkeys and shortcuts. The tool functions as a keyboard macro automation utility and a text expansion engine. It can record keystroke sequences and replay them with precise timing, or detect specific typed character sequences and replace them with predefined strings using simulated backspaces. The library covers a broad range of input cont

OptiKey is an assistive technology suite and gaze-based input system designed to provide computer access and communication tools. It functions as an eye-tracking virtual keyboard and augmentative and alternative communication software, allowing users to perform text entry and system control via gaze-tracking hardware or other pointer devices. The system differentiates itself through an input-agnostic selection layer that supports diverse sources, including eye trackers, game controllers, and head mice. It utilizes a dwell-time trigger mechanism and gaze fixation selection to translate eye mov

Open-computer-use is a framework designed to connect vision-capable language models to isolated cloud-based desktop environments. It functions as an agentic interface that enables autonomous systems to interact with graphical user interfaces by simulating mouse movements, keyboard keystrokes, and shell commands. By bridging language models with remote workspaces, the platform facilitates the execution of complex, long-running tasks within secure, sandboxed environments. The platform distinguishes itself through its ability to orchestrate thousands of concurrent, isolated instances, making it

Kit is a desktop automation framework and scriptable UI toolkit designed for building personalized productivity tools. It serves as a cross-platform CLI wrapper and macOS system automator, providing an environment to execute scripts that manage operating system tasks, file management, and application workflows. The project distinguishes itself with a dedicated LLM integration layer for structured data extraction and text generation, alongside a specialized UI framework for creating interactive input forms, HTML windows, and floating widgets. It features deep macOS integration through AppleScr

Handy is a local speech-to-text automation tool designed to convert spoken audio into text and inject it directly into active desktop applications. By running machine learning models entirely on the host hardware, it provides a private, offline-first environment for dictation and command execution. The system functions as a background service that manages microphone input, transcription state, and text output, enabling hands-free typing across various software environments. The project distinguishes itself through a modular pipeline that integrates local language models for post-transcription

Nerd-dictation is a command-line utility that provides offline speech-to-text transcription and system-wide desktop automation. It functions as a local recognition engine, processing microphone audio streams directly on the host machine without requiring cloud connectivity or external network services. The tool distinguishes itself by integrating transcription directly into the desktop environment through system-level input injection, which simulates keyboard events to type recognized speech into active application windows. It supports grammar-constrained decoding, allowing users to define sp

SerpentAI is a game AI development kit and computer vision framework designed for building autonomous agents that interact with video games. It serves as a game input automation tool and a machine learning model integration engine, allowing developers to create agents that perceive game states and execute actions. The framework utilizes a plugin-based agent architecture to provide modular extensions for game-specific logic and behaviors. It features a specialized system for training, bundling, and deploying machine learning classifiers to recognize visual contexts and game states in real time

Karabiner-Elements is a system-level utility designed for advanced keyboard and mouse customization. It functions as a background service that intercepts raw hardware input signals at the driver level, allowing for the transformation of key presses and pointer movements before they reach the operating system. By utilizing virtual input device emulation, the software re-injects modified events into the system stream, enabling complex remapping, macro execution, and hardware-specific control. The project distinguishes itself through a sophisticated state-based logic engine that enables context-

Claude Code is a command-line interface and multi-agent orchestration framework designed for autonomous software engineering. It enables AI agents to perform codebase modifications, debugging, and Git workflow management while coordinating multiple specialized agents to decompose and execute complex engineering tasks in parallel. The system distinguishes itself through a high degree of isolation and safety, utilizing Git worktrees to create independent working directories for concurrent agents and implementing a tiered permission system that combines user rules, project policies, and OS-level

Open Interpreter is an autonomous agent runtime that translates natural language instructions into executable code to interact with local software and operating systems. It functions as an orchestration framework that connects language models to a secure execution environment, enabling the development of agents capable of managing system resources and performing complex tasks. To ensure safety, the system mandates explicit user verification before executing any generated code and provides robust isolation through containerized sandboxing. The project distinguishes itself through its deep inte

This project is a desktop automation scripting framework and shell script command runner. It allows users to execute custom scripts and workflows from a central launcher to automate repetitive desktop tasks and system operations. The framework functions as a unified command interface and a JSON input automation bridge, serializing search queries into arguments passed to the execution environment. It integrates large language model prompts directly into the desktop interface to automate content generation and answer questions. The system handles a broad range of capabilities including the exe

Openwork is an AI agent for desktop automation that uses large language models to execute browser tasks, manage local files, and automate desktop workflows. It operates on a local-first execution model, translating natural language prompts into sequences of tool calls to perform digital chores. The system functions as a framework for defining and saving repeatable sequences of actions as reusable skills. It integrates large language models with third-party services and local APIs to synchronize data and share files. The agent includes capabilities for headless browser automation to conduct r

Yabai is a system-level utility for macOS that provides automated window management and workspace organization. It functions as a tiling window manager that uses binary space partitioning to arrange application windows into grid layouts, bypassing standard desktop constraints through direct interaction with the system window server. The project distinguishes itself by offering granular control over the graphical user interface via a socket-based command interface and event-driven system hooks. This allows for the programmatic manipulation of window states, display configurations, and virtual

UI-TARS-desktop is a cross-platform desktop application designed to automate software interface interactions. It functions as a local agent environment that interprets graphical user interfaces through multimodal visual-language model reasoning, allowing it to navigate and manipulate software by simulating human-like mouse and keyboard inputs. The platform distinguishes itself by executing all visual recognition and decision-making logic directly on the host machine. This local inference model ensures that screen data and sensitive information remain private, as no processing is offloaded to

skhd is a keyboard shortcut manager and hotkey daemon for macOS. It functions as an input remapper that maps custom keyboard shortcuts to shell commands and system actions via a plain text configuration file. The utility supports modal keyboard workflows, allowing groups of shortcuts to activate only when the system is in a specific operational mode. It also enables application-specific hotkeys and application blacklisting, ensuring shortcuts only trigger based on which application is currently in focus. Additional capabilities include the remapping of hardware media keys and the synthesis o

This project is a collection of toolsets for executing visual effects, simulating operating system-level input, and manipulating graphical user interface window states. It functions as a Windows API automation tool and GUI manipulator designed to programmatically alter the behavior and appearance of active application windows. The toolkit features a desktop visual effects engine capable of applying rotational animations, hue shifts, and view transformations to the display. It includes an input simulator that can spawn multiple independent mouse cursors and automate user interactions across th

Osaurus is a local AI workflow engine and LLM agent orchestration framework designed for private execution on local hardware. It functions as a desktop application automator and a voice-controlled AI interface, enabling the development of autonomous agents that can write code, execute tools, and operate a computer without keyboard or mouse input. The system is distinguished by its ability to control native desktop applications via accessibility APIs and manage web interactions through a headless browser automation tool. It supports a local-first execution model and on-premises deployment with

Free RPA tool by AI Singapore

Mjolnir is a macOS automation framework and extensible scripting engine. It provides a system for creating custom productivity workflows, managing application states, and controlling the macOS desktop interface programmatically. The project functions as a global hotkey manager that binds keyboard shortcuts to trigger automated scripts across the operating system. It includes a macOS application controller to inspect active windows and manage system-wide user interface interactions. The environment supports extensibility through a pluggable package management system, allowing for the installa

Hermes-agent is an autonomous AI agent framework and runtime designed to execute complex tasks and synthesize new skills from execution traces. It includes a provider-agnostic gateway for routing requests across multiple model backends and a serverless runtime that suspends idle agent instances and resumes them on demand across containers and virtual machines. The project provides a desktop automation toolset that controls native GUI workflows on Linux by querying accessibility APIs and injecting input events. It further distinguishes itself with the ability to generate procedural skills from

This project is a library of automation scripts designed for the BoxJs environment. It functions as an external API client and integration interface for interacting with remote servers. The system provides a persistent local data store to maintain state between script executions by saving and retrieving strings and JSON objects. It enables data exchange through structured JSON objects and supports both synchronous and asynchronous network requests using custom headers.