Tools
Computer Use
By Community OpenClaw plugin for GPT-5.4 computer-use orchestration through an external executor.
Install
npm install
pnpmConfiguration Example
{ "action": "start", "task": "Open example.com and report the page title" }README
# OpenClaw Computer Use Plugin
Community OpenClaw plugin for GPT-5.4 computer-use orchestration.
This package is the external-plugin version of the original OpenClaw core PR
proposal. It keeps the same responsibility split:
- GPT-5.4 handles UI understanding and next-step planning.
- OpenClaw coordinates the task through an optional tool.
- An external executor owns screenshots, input execution, confirmation,
isolation, retries, and replay.
This repository does not ship a full desktop executor yet. It defines the
plugin boundary and executor HTTP contract that a local executor can implement.
## Status
Experimental community plugin scaffold.
Implemented:
- `computer-use` optional tool registration
- `start`, `status`, `confirm`, and `cancel` actions
- fixed executor endpoint from plugin config
- explicit confirmation default: missing `allow` rejects
- manifest schema requiring `executorBaseUrl`
- focused tests for request construction and safety defaults
Not implemented:
- bundled desktop runner
- real screenshot capture
- real mouse or keyboard execution
- end-to-end GPT-5.4 Responses API loop
## Install
Local development:
```bash
openclaw plugins install -l .
openclaw plugins enable computer-use
```
After npm publication:
```bash
openclaw plugins install @ethereall11/openclaw-computer-use
openclaw plugins enable computer-use
```
## Example Config
```json5
{
plugins: {
entries: {
"computer-use": {
enabled: true,
config: {
executorBaseUrl: "http://127.0.0.1:8100",
executorAuthToken: "local-dev-token",
defaultProvider: "openai",
defaultModel: "gpt-5.4",
defaultRequireConfirmation: true,
defaultMaxSteps: 25,
defaultTimeoutMs: 120000,
},
},
},
},
agents: {
list: [
{
id: "main",
tools: {
allow: ["computer-use"],
},
},
],
},
}
```
## Tiny Demo
The smallest working flow is:
1. Configure the plugin with a fixed `executorBaseUrl`.
2. Run any HTTP service that implements the four task endpoints below.
3. Call the `computer-use` tool with `action: "start"`.
4. Poll with `status`, then use `confirm` or `cancel` when needed.
Minimal tool-call sequence:
```json
{ "action": "start", "task": "Open example.com and report the page title" }
```
Example follow-up calls after the executor returns `taskId: "task_123"`:
```json
{ "action": "status", "taskId": "task_123" }
{ "action": "confirm", "taskId": "task_123", "allow": true }
{ "action": "cancel", "taskId": "task_123" }
```
Minimal executor stub shape:
```ts
app.post("/v1/tasks", (_req, res) => {
res.json({ taskId: "task_123", status: "blocked", needsConfirmation: true });
});
app.get("/v1/tasks/:taskId", (req, res) => {
res.json({ taskId: req.params.taskId, status: "blocked" });
});
app.post("/v1/tasks/:taskId/confirm", (req, res) => {
res.json({ taskId: req.params.taskId, status: req.body.allow ? "running" : "rejected" });
});
app.post("/v1/tasks/:taskId/cancel", (req, res) => {
res.json({ taskId: req.params.taskId, status: "cancelled" });
});
```
## Executor Contract
The executor is intentionally outside this plugin.
Required endpoints:
- `POST /v1/tasks`
- `GET /v1/tasks/:taskId`
- `POST /v1/tasks/:taskId/confirm`
- `POST /v1/tasks/:taskId/cancel`
See [docs/design/gpt-5.4-computer-use-plugin.md](docs/design/gpt-5.4-computer-use-plugin.md)
for the proposed request and response shape.
## Development
```bash
pnpm install
pnpm test
```
## Origin
This work started as OpenClaw PR
[#40640](https://github.com/openclaw/openclaw/pull/40640). The PR was closed
because OpenClaw maintainers prefer optional tool plugins to ship through
ClawHub, npm, or community repositories instead of being bundled in core.
tools
Comments
Sign in to leave a comment