# magnitudedev/browser-agent

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/magnitudedev-browser-agent).**

_How this analysis was created: the description and tags below were written by an AI model that read this project's README and public documentation pages; stars, license and language come straight from the GitHub API. The model does not read the source code._

4,139 stars · 247 forks · TypeScript · Apache-2.0

## Links

- GitHub: https://github.com/magnitudedev/browser-agent
- Homepage: https://magnitude.run
- awesome-repositories: https://awesome-repositories.com/repository/magnitudedev-browser-agent.md

## Topics

`ai` `automation` `browser` `framework` `javascipt` `playwright` `rpa` `selenium` `test` `typescript`

## Description

Browser Agent is a TypeScript toolkit for operating live web pages with an AI agent that works from screenshots. A vision-grounded model locates interface elements and returns exact pixel positions, and the agent then acts through mouse clicks, drags, and keyboard input. Because it does not depend on page markup, DOM structure, or APIs, actions work on any rendered page.

Goals are written in plain English. A language model turns them into ordered steps and loops between observing the screen and acting until the task is done, with no selectors or scripts to write. The same approach can complete tasks across several web applications that have no programmatic integration. Developers can mix fine-grained primitive actions with higher-level multi-step flows, and can override actions and prompts at the agent or individual action level.

The project also covers several related uses. A Model Context Protocol server exposes browser control as tool calls, so external AI clients can drive and inspect pages. A test runner takes plain-English goals and visual assertions and runs them against web apps locally or in CI/CD pipelines. A data extraction capability reads visible page content and returns it in a shape defined by a schema, including derived values the agent works out itself.

The repository is a multi-package monorepo with dependency-ordered, cached builds, linting, and type-checking. It includes a template-based scaffolding command that generates a ready-to-run starter project.

## Tags

### Artificial Intelligence & ML

- [Natural Language Browser Control Platforms](https://awesome-repositories.com/f/artificial-intelligence-ml/agentic-systems-frameworks/agent-orchestration-multi-agent/autonomous-agents/autonomous-browser-agents/natural-language-browser-control-platforms.md) — Drives live web pages from plain-language instructions using vision-based understanding, planning, and precise mouse and keyboard actions. ([source](https://github.com/magnitudedev/browser-agent#readme))
- [Browser Automation Agents](https://awesome-repositories.com/f/artificial-intelligence-ml/browser-automation-agents.md) — Provides an AI agent that operates live web pages from screenshots using mouse and keyboard input, without relying on page markup or APIs.
- [Natural Language Browser Control](https://awesome-repositories.com/f/artificial-intelligence-ml/natural-language-browser-control.md) — Interpret plain-English goals and step-by-step instructions, then carry them out on a live web page by using vision to understand the interface. ([source](https://github.com/magnitudedev/browser-agent/blob/main/README.md))
- [Pixel Coordinate Grounding](https://awesome-repositories.com/f/artificial-intelligence-ml/pixel-coordinate-grounding.md) — Uses a visually grounded model to return exact pixel positions for interface elements, so actions never depend on page markup. ([source](https://github.com/magnitudedev/browser-agent/blob/main/README.md))
- [Plain-Language Task Requests](https://awesome-repositories.com/f/artificial-intelligence-ml/plain-language-task-requests.md) — Takes plain-English goals and carries them out step by step on a live web page without selectors or scripts.
- [Vision-Based GUI Automation](https://awesome-repositories.com/f/artificial-intelligence-ml/vision-based-gui-automation.md) — Controls web pages by analyzing screenshots and simulating mouse and keyboard input at pixel positions on any site.
- [Vision-Language Grounding Models](https://awesome-repositories.com/f/artificial-intelligence-ml/vision-language-grounding-models.md) — Runs a visually grounded model on screenshots to return exact pixel positions of interface elements, independent of page markup.
- [Browser Control Servers](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-agent-servers/browser-control-servers.md) — Exposes browser control through a Model Context Protocol server so external AI clients can drive and inspect web pages.
- [Cross-Application Workflow Automation](https://awesome-repositories.com/f/artificial-intelligence-ml/cross-application-workflow-automation.md) — Completes tasks spanning several web applications by driving their interfaces directly when no API or integration exists.
- [Model Context Protocol Servers](https://awesome-repositories.com/f/artificial-intelligence-ml/model-context-protocol-servers.md) — Ships a Model Context Protocol server that exposes browser actions as tools external AI clients can call on demand.
- [Schema-Based Page Extraction](https://awesome-repositories.com/f/artificial-intelligence-ml/schema-based-page-extraction.md) — Reads what is visible on a page and returns it in a schema-defined shape, including derived values like ratings the agent infers. ([source](https://github.com/magnitudedev/browser-agent#readme))

### Part of an Awesome List

- [Keyboard and Mouse Input Simulations](https://awesome-repositories.com/f/awesome-lists/devtools/keyboard-mouse/keyboard-and-mouse-input-simulations.md) — Translates model-chosen pixel positions into low-level mouse clicks, drags, and keyboard events that work on any rendered page.

### Data & Databases

- [Web Data Extraction Tools](https://awesome-repositories.com/f/data-databases/web-data-extraction-tools.md) — Reads visible page content into clean schema-shaped data, including derived details the agent works out itself.

### Software Engineering & Architecture

- [AI-Driven Schema Extractions](https://awesome-repositories.com/f/software-engineering-architecture/content-schemas/ai-driven-schema-extractions.md) — Reads visible page content and returns it in a schema-defined shape, including derived values the model works out itself.

### Testing & Quality Assurance

- [Browser Automation Testing](https://awesome-repositories.com/f/testing-quality-assurance/software-testing/testing-frameworks/test-frameworks/browser-and-ui-testing/browser-automation-frameworks/browser-automation-testing.md) — Writes and runs browser tests that check what a web app looks like and does, including in CI/CD pipelines.
- [Natural Language Test Generators](https://awesome-repositories.com/f/testing-quality-assurance/software-testing/testing-frameworks/test-frameworks/execution-and-infrastructure/automated-test-generators/natural-language-test-generators.md) — Runs tests written as plain-English goals and visual assertions against web apps locally and in CI/CD pipelines.
- [Visual Tests](https://awesome-repositories.com/f/testing-quality-assurance/visual-tests.md) — Ships a built-in test runner that checks the visual output of web apps and runs in CI/CD pipelines. ([source](https://github.com/magnitudedev/browser-agent#readme))

### User Interface & Experience

- [Visual Element Locators](https://awesome-repositories.com/f/user-interface-experience/dom-element-selectors/visual-element-locators.md) — Locates page elements from screenshots and returns exact pixel positions instead of relying on DOM selectors. ([source](https://github.com/magnitudedev/browser-agent#readme))
