Integration
Agent Browser Bridge AI
By The high-fidelity browser control layer for AI Agents OpenClaw and Hermes agent. 10x Token Efficiency ! Precise, Human-like, and Production-ready.
Install
npm install browser-agentbridge-aiConfiguration Example
{
"steps": [
{ "type": "navigate", "url": "https://example.com" },
{ "type": "annotate" },
{ "type": "click", "ref": 3 },
{ "type": "summary" }
]
}README
# AgentBridge v3.2
> **The high-fidelity browser control layer for AI Agents.**
> Precise, Human-like, and Production-ready.
>
[https://github.com/alexandre-leng/Browser-AgentBridge-AI](https://github.com/alexandre-leng/browser-agentbridge-ai)
>
[](https://opensource.org/licenses/MIT)
[](https://playwright.dev/)
[](https://github.com/alexandre-leng/agentbridge/actions/workflows/ci.yml)
[](https://www.npmjs.com/package/browser-agentbridge-ai)
---
## 🌟 Why AgentBridge?
Most AI agents "guess" where to click using screenshots. AgentBridge provides a **DOM-first** approach:
- **99% Precision**: No more x,y coordinate guessing. Interact with elements using stable numerical IDs.
- **10x Token Efficiency**: Send a 200-token element list instead of a 2000-token high-res screenshot.
- **Anti-Detection**: Built-in human-like mouse movements (Bezier curves), varied typing speeds, and stealth scripts.
- **Cross-Platform**: Native CLI for Windows and Linux/macOS.
- **MCP-native**: Official stdio MCP server for Codex, Claude Desktop, and other MCP clients.
- **Traceable**: Session traces and benchmark artifacts for debugging, replay, and regressions.
---
## 🚀 Quick Start (2 minutes)
### 1. Install
```bash
npm install browser-agentbridge-ai
npx playwright install chromium
```
### 2. Launch
```bash
npm start
# → http://localhost:8080/viewer
# → ws://localhost:8080/ws/browser-bridge
```
### 3. Test it
```bash
# CLI
npx agentbridge navigate https://example.com
# Or open http://localhost:8080/viewer in your browser
```
### Docker (alternative)
```bash
docker build -t browser-bridge .
docker run -p 8080:8080 browser-bridge
```
### MCP Server
```bash
npm run mcp
```
After `npm run build`, the package exposes `agentbridge-mcp`. The MCP server registers focused tools (`browser_status`, `navigate`, `annotate_page`, `click_ref`, `type_ref`, `extract_schema`, `human_timing_get`, `human_timing_set`, `human_antispam_check`). A low-level `browser_command` escape hatch is available behind `BRIDGE_MCP_ALLOW_RAW=1`.
The MCP resource `api` (`agentbridge://api`) exposes the registered bridge command list, and the `browser_task` prompt gives agents a refs-first task template.
### Agent Skill Install
Install AgentBridge as an AgentSkills-compatible skill for local agents:
```bash
npm run build
npx agentbridge install openclaw --global
npx agentbridge install hermes
```
Targets:
- `openclaw`: writes `agentbridge/SKILL.md` and `PROMPTS.md` into `./skills` by default, or `~/.openclaw/skills` with `--global`.
- `hermes`: writes the skill into `~/.hermes/skills/agentbridge`.
- `all`: installs both adapters.
Use `--workspace <path>` to stage into a workspace instead of a user-level directory, and `--dry-run` to preview paths.
Reusable browser scripts can be run from JSON:
```bash
npx agentbridge script ./examples/browser-script.json
```
```json
{
"steps": [
{ "type": "navigate", "url": "https://example.com" },
{ "type": "annotate" },
{ "type": "click", "ref": 3 },
{ "type": "summary" }
]
}
```
---
## 🛠️ The `bridge` CLI
AgentBridge includes a powerful CLI to interact with the browser from any terminal or script.
### One-liner Workflows (Batch Mode)
Execute complex sequences in a single request to eliminate network latency:
```bash
.\bridge.cmd run "navigate google.com" "annotate" "click 7" "type 7 'weather paris'" "press Enter" "summary"
```
### Interactive REPL
Perfect for manual testing or continuous agent dialogue:
```bash
.\bridge.cmd repl
bridge> navigate https://google.com
bridge> annotate
bridge> click 7
```
---
## 🤖 Agent-Ready API
Designed specifically for LLMs (Claude, GPT, Gemini).
- **`page.annotate`**: Generates a numbered screenshot + structured element list.
- **`agent.click {ref: N}`**: Clicks the element with ID `N` using human-like motion.
- **`agent.type {ref: N, text: "..."}`**: Focuses and types with realistic delays.
- **`web.search {query, limit?, engine?, pages?}`**: Searches the web, paginates, deduplicates URLs, and returns a run report.
- **`dom.extract {type: "search-results"|"form"|"table"|"google-maps"|"listings"|"marketplace"}`**: Returns clean JSON instead of a wall of text.
- **`dom.extract {schema}`**: Extracts typed fields from CSS selectors.
- **`dom.extract {schema, llm: true}`**: Produces a strict JSON extraction prompt for an external LLM client.
- **`dom.visibleText {filterAny, filterLines}`**: Extracts visible text with Windows-safe comma filters.
- **`human.clickText {text, timeoutMs}`**: Finds visible text, logs each stage, and falls back to ref-clicking when coordinate clicks fail.
- **`human.timing.*`**: Lets an agent read, tune, and reset consultation timings while a session is running.
- **`human.antispam.check`**: Returns a structured anti-spam warning without throwing, so an agent can pause or hand off cleanly.
### Runtime Human Timing
AgentBridge separates interaction speed from consultation speed. Mouse movement and typing stay humanized, while page-reading pauses can be adjusted live by the agent when a site starts reacting badly to rapid browsing.
```bash
node bridge-cli.cjs timing get
node bridge-cli.cjs timing set consultSpeed=1.6 minFocusedMs=3500 feedbackIntervalMs=800
node bridge-cli.cjs antispam
```
Typical agent loop:
1. Navigate or annotate.
2. Call `human.read`, `human.scan`, or `human.findText`.
3. Watch `human.feedback` WebSocket events for `phase`, `remainingMs`, `progress`, and active `timing`.
4. If the page feels sensitive, call `human.timing.set` with a higher `consultSpeed` or larger minimum pauses.
5. Call `human.antispam.check` before continuing with more clicks or searches.
`consultSpeed` is a multiplier: `1` is default, `1.5` is slower, `0.75` is faster. Prefer slowing down and handing off to a human if `human_antispam_check` reports a block; AgentBridge is designed for polite automation, not bypassing protections.
Schema extraction example:
```json
{
"type": "dom.extract",
"payload": {
"schema": {
"fields": {
"title": { "selector": "h1", "type": "string", "required": true },
"links": { "selector": "a", "attribute": "href", "type": "array" }
}
}
}
}
```
---
## 🏗️ Project Architecture
```mermaid
graph TD
A[AI Agent / CLI] -- JSON/WS --> B[Bridge Server]
B -- Commands --> C[Modular Handlers]
C -- Playwright --> D[Browser Instance]
D -- Events --> C
C -- Annotated Frames --> B
B -- Frames/Results --> A
```
- **`src/browser/handlers/`**: Domain-driven command handlers (Navigation, DOM, Extraction...).
- **`src/browser/agent.ts`**: The "Eyes" — ARIA tree extraction and visual annotation.
- **`src/browser/human.ts`**: The "Hands" — Bezier mouse curves and typing jitter.
- **`src/transport/ws.ts`**: The "Nerves" — High-speed WebSocket communication.
---
## 🧪 Reliability & Testing
We take stability seriously. The bridge includes a comprehensive test suite powered by **Vitest**:
```bash
npm test
```
- ✅ **Resolver Integrity**: Ensures XPath/CSS/Text detection is flawless. Empty/undefined queries now throw an explicit error (`dom.click: requires query, selector or text`) instead of crashing.
- ✅ **Human Dynamics**: Validates mouse movement physics and typing patterns.
---
## 👁️ Human Realism (`src/browser/human.ts`)
AgentBridge's anti-detection isn't just stealth scripts — every interaction is shaped to match human motor patterns.
| Aspect | Implementation |
|---|---|
| **Mouse trajectory** | Cubic Bezier with random arc, 24-90 steps adaptive to distance. Cursor position is tracked server-side, so each move starts from the real last position (no teleport from `(0,0)`). |
| **Mouse velocity** | **Smoothstep easing** `t' = t²(3-2t)` — slow start, fast middle, slow end. Linear `t` was a strong bot signal (constant velocity). |
| **Typing rhythm** | 40-160 ms per character, 3% chance of long pause (200-500 ms reflection). |
| **Typos & correction** | ~2.5% chance of pressing a QWERTY-neighbor key, then `Backspace`, then the correct key. The strongest defeat for keystroke-pattern detectors. |
| **Scroll inertia** | Wheel deltas follow exponential decay `e^(-1.2t)` — strong initial impulse then taper, mimicking real mouse-wheel physics. |
| **Visible cursor** | Red dot (turns green during click) painted via injected `<div>` with CSS `transition: transform 40ms linear`. The browser interpolates between updates, so we throttle paints to 1-in-3 steps to save IPC round-trips with no visual difference. Disable with `BRIDGE_VISIBLE_CURSOR=0`. |
Fine-tune at runtime:
```jsonc
{ "type": "human.timing.set", "payload": {
"consultSpeed": 1.4,
"minFocusedMs": 3000,
"feedbackIntervalMs": 800
}}
```
---
## 🛡️ Stealth & Anti-detection (`src/browser/stealth.ts`)
Twelve patches injected via `addInitScript` before any page script executes:
| # | Patch | Purpose |
|---|---|---|
| 1 | `navigator.webdriver` → `undefined` | Defeats the most basic check. |
| 2 | `window.chrome` full object (`app`, `runtime`, `loadTimes`, `csi`) | Real Chrome has it — headless doesn't. |
| 3 | `navigator.plugins` (5 PDF viewers) | Empty plugins array is a headless signal. |
| 4 | `navigator.mimeTypes` | Coherent with plugins. |
| 5 | `navigator.languages` → `['fr-FR', 'fr', 'en-US', 'en']` | Matches the `locale` Playwright contextOpt. |
| 6 | `navigator.deviceMemory` → `8` | Default is missing in headless. |
| 7 | `navigator.hardwareConcurrency` → `8` | Same. |
| 8 | `Permissions.query` patched | `notifications` returns the real `Notification.permission` instead of leaking automation state. |
| 9 | Canvas fingerprint noise | Imperceptible per-session noise on `toDataURL` / `toBlob` defeats exact-
... (truncated)
integration
Comments
Sign in to leave a comment