# zai-org/open-autoglm **Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/zai-org-open-autoglm).** 23,532 stars · 3,714 forks · Python · apache-2.0 ## Links - GitHub: https://github.com/zai-org/Open-AutoGLM - Homepage: https://autoglm.z.ai/blog - awesome-repositories: https://awesome-repositories.com/repository/zai-org-open-autoglm.md ## Topics `agent` `phone-use-agent` ## Description Open-AutoGLM is an autonomous agent framework designed to perform complex user workflows on mobile devices. By translating natural language instructions into precise sequences of taps, scrolls, and text inputs, the system enables the automation of mobile application interactions and testing. The platform distinguishes itself through a combination of vision-language processing and reinforcement learning. It converts graphical user interfaces into structured data, allowing agents to parse screen elements and map natural language commands to coordinate-based actions. To ensure reliability, the system employs heuristic-based error recovery to navigate around interface interruptions such as pop-ups, advertisements, and network delays. The framework provides a secure, containerized environment for executing these tasks, which isolates agent processes to protect sensitive data and maintain audit trails. Additionally, it functions as a training platform where agents refine their decision-making policies through repeated reinforcement learning cycles within virtualized mobile environments. ## Tags ### Testing & Quality Assurance - [UI Automation](https://awesome-repositories.com/f/testing-quality-assurance/automation-interaction-tools/ui-automation.md) — Translates natural language commands into precise mobile UI interactions for automated testing and workflow execution. - [Mobile Testing Frameworks](https://awesome-repositories.com/f/testing-quality-assurance/software-testing/testing-frameworks/test-frameworks/test-levels-and-types/mobile-testing-frameworks.md) — Automates complex mobile user workflows by translating natural language into precise interface interactions. - [Browser Automation Interfaces](https://awesome-repositories.com/f/testing-quality-assurance/software-testing/testing-frameworks/test-frameworks/browser-and-ui-testing/browser-automation-frameworks/browser-automation-interfaces.md) — Provides interfaces for programmatic interaction with mobile environments to bypass obstacles during automated testing. ([source](https://autoglm.z.ai/blog)) ### Artificial Intelligence & ML - [Large Language Models](https://awesome-repositories.com/f/artificial-intelligence-ml/large-language-models.md) — Leverages large language models to enable autonomous navigation and multi-step workflow execution. - [Reinforcement Learning Environments](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/reinforcement-learning-environments.md) — Provides a training platform for refining agent behaviors through reinforcement learning in virtualized mobile interfaces. ([source](https://autoglm.z.ai/blog)) - [Autonomous Agents](https://awesome-repositories.com/f/artificial-intelligence-ml/autonomous-agents.md) — Develops intelligent agents capable of navigating diverse mobile interfaces through reinforcement learning. - [Vision-Language Grounding Models](https://awesome-repositories.com/f/artificial-intelligence-ml/vision-language-grounding-models.md) — Maps natural language instructions to spatial coordinates on mobile interfaces using vision-language grounding models. - [Agent Task Execution](https://awesome-repositories.com/f/artificial-intelligence-ml/agent-task-execution.md) — Executes automated software tasks within secure, isolated containers with audit logging. - [Model Feedback Loops](https://awesome-repositories.com/f/artificial-intelligence-ml/model-feedback-loops.md) — Implements feedback loops to iteratively refine agent decision-making policies based on task completion performance. - [Interface Representations](https://awesome-repositories.com/f/artificial-intelligence-ml/markovian-state-representations/interface-representations.md) — Converts raw graphical user interfaces into structured data to maintain context during automated navigation. ### DevOps & Infrastructure - [Cloud Sandbox Provisioning](https://awesome-repositories.com/f/devops-infrastructure/infrastructure/private-enterprise-management/cloud-infrastructure-management/cloud-sandbox-provisioning.md) — Provides a secure, containerized cloud environment for executing and auditing automated mobile tasks. ### User Interface & Experience - [Action Sequence Composers](https://awesome-repositories.com/f/user-interface-experience/action-sequence-composers.md) — Chains complex user input sequences into executable workflows for mobile device automation. ([source](https://autoglm.z.ai/blog)) - [Heuristic Selectors](https://awesome-repositories.com/f/user-interface-experience/ui-element-selectors/heuristic-selectors.md) — Uses heuristic-based selectors to navigate dynamic mobile interfaces and recover from unexpected interruptions like pop-ups. ### Part of an Awesome List - [Multimodal Models](https://awesome-repositories.com/f/awesome-lists/ai/multimodal-models.md) — Agentic multimodal model for automated device interaction. ### Security & Cryptography - [Agent Security Runtimes](https://awesome-repositories.com/f/security-cryptography/agent-security-runtimes.md) — Enforces security policies and isolation for agent execution environments within cloud containers. ([source](https://autoglm.z.ai/blog)) ### Mobile Development - [Interaction Navigators](https://awesome-repositories.com/f/mobile-development/mobile-operating-systems/interaction-navigators.md) — Navigates through pop-ups and network latency to maintain reliable operation chains in mobile applications. ### Software Engineering & Architecture - [Sandboxed Execution Environments](https://awesome-repositories.com/f/software-engineering-architecture/sandboxed-execution-environments.md) — Provides isolated runtime environments for executing automated agent tasks securely within virtualized containers.