Unity

README

# 🦞 OpenClaw Unity Plugin

> **TL;DR:** Vibe-code your game development remotely from anywhere! 🌍
> 
> **한줄요약:** 이제 집밖에서도 원격으로 바이브코딩으로 게임 개발 가능합니다! 🎮

Connect Unity to [OpenClaw](https://github.com/openclaw/openclaw) AI assistant via HTTP. Works in **Editor mode** without hitting Play!

[![Unity](https://img.shields.io/badge/Unity-2021.3+-black?logo=unity)](https://unity.com)
[![OpenClaw](https://img.shields.io/badge/OpenClaw-2026.2.3+-green)](https://github.com/openclaw/openclaw)
[![License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Buy Me A Coffee](https://img.shields.io/badge/Buy%20Me%20A%20Coffee-support-yellow?logo=buy-me-a-coffee)](https://buymeacoffee.com/tomleelive)

## ✨ Key Features

- 🎮 **Works in Editor & Play Mode** - No need to hit Play to use AI tools
- 🔌 **Auto-Connect** - Connects when Unity starts, maintains connection across mode changes
- 📋 **Console Integration** - Capture and query Unity logs for debugging
- 🎬 **Scene Management** - List, load, and inspect scenes
- 🔧 **Component Editing** - Add, remove, and modify component properties
- 📸 **Debug Tools** - Screenshots, hierarchy view, and more
- 🎯 **Input Simulation** - Keyboard, mouse, and UI interaction for game testing
- 🔄 **Editor Control** - Trigger recompilation and asset refresh remotely
- 🔒 **Security Controls** - Configure what operations are allowed

## Requirements

| Component | Version |
|-----------|---------|
| **Unity** | 2021.3+ |
| **OpenClaw** | 2026.2.3+ |

## Installation

### Option 1: Git URL (Recommended)

1. Open Unity Package Manager (`Window > Package Manager`)
2. Click `+` → `Add package from git URL...`
3. Enter:
   ```
   https://github.com/TomLeeLive/openclaw-unity-plugin.git
   ```

### Option 2: Local Package

1. Clone this repository
2. In Unity: `Window > Package Manager` → `+` → `Add package from disk...`
3. Select the `package.json` file

## Quick Start

### 1. Install OpenClaw Gateway Extension (Required)

Copy the gateway extension files to OpenClaw:

```bash
# Copy extension files
cp -r OpenClawPlugin~/* ~/.openclaw/extensions/unity/

# Restart gateway to load the extension
openclaw gateway restart

# Verify
openclaw unity status
```

> **Note:** `OpenClawPlugin~` contains the gateway extension that enables `unity_execute` and `unity_sessions` tools. This is required for OpenClaw to communicate with Unity.

### 2. Install Unity Package

See [Installation](#installation) above for Git URL or local package setup.

### 3. Configure in Unity

1. Open `Window > OpenClaw Plugin`
2. Set Gateway URL: `http://localhost:18789` (default)
3. Connection is automatic when Unity starts
4. Status shows green when connected

### 4. Chat with OpenClaw

Ask OpenClaw to inspect your scene, create objects, or debug issues - all without entering Play mode!

### 5. Install OpenClaw Skill (Optional)

The companion skill provides workflow patterns and tool references for the AI:

```bash
# Clone skill to OpenClaw workspace
git clone https://github.com/TomLeeLive/openclaw-unity-skill.git ~/.openclaw/workspace/skills/unity-plugin
```

The skill provides:
- Quick reference for all 52 tools
- Common workflow patterns (scene inspection, UI testing, etc.)
- Detailed parameter documentation
- Troubleshooting guides

> **Note:** The skill is separate from the gateway extension. The extension enables the tools; the skill teaches the AI how to use them effectively.

## 📚 Documentation

- **[Development Guide](Documentation~/DEVELOPMENT.md)** - Architecture, extending tools, and contribution guidelines
- **[Testing Guide](Documentation~/TESTING.md)** - Complete testing guide with examples

## Available Tools (52 total)

### Console (3 tools)
| Tool | Description |
|------|-------------|
| `console.getLogs` | Get Unity console logs (with type filter) |
| `console.getErrors` | Get error/exception logs (with optional warnings) |
| `console.clear` | Clear captured logs |

### Scene (5 tools)
| Tool | Description |
|------|-------------|
| `scene.list` | List all scenes in build settings |
| `scene.getActive` | Get active scene info |
| `scene.getData` | Get scene hierarchy data |
| `scene.load` | Load a scene by name (Play mode) |
| `scene.open` | Open a scene in Editor mode |

### GameObject (8 tools)
| Tool | Description |
|------|-------------|
| `gameobject.find` | Find by name, tag, or component type |
| `gameobject.getAll` | Get all GameObjects with filtering |
| `gameobject.create` | Create GameObject or primitive |
| `gameobject.destroy` | Destroy a GameObject |
| `gameobject.delete` | Delete a GameObject (alias for destroy) |
| `gameobject.getData` | Get detailed object data |
| `gameobject.setActive` | Enable/disable object |
| `gameobject.setParent` | Change parent |

### Transform (6 tools)
| Tool | Description |
|------|-------------|
| `transform.getPosition` | Get world position (x, y, z) |
| `transform.getRotation` | Get rotation in Euler angles |
| `transform.getScale` | Get local scale |
| `transform.setPosition` | Set world position |
| `transform.setRotation` | Set rotation (Euler) |
| `transform.setScale` | Set local scale |

### Component
| Tool | Description |
|------|-------------|
| `component.add` | Add component to object |
| `component.remove` | Remove component |
| `component.get` | Get component data |
| `component.set` | Set field/property value |
| `component.list` | List available types |

### Script
| Tool | Description |
|------|-------------|
| `script.execute` | Execute simple commands |
| `script.read` | Read script file contents |
| `script.list` | List script files in project |

### Application
| Tool | Description |
|------|-------------|
| `app.getState` | Get play mode, FPS, etc. |
| `app.play` | Enter play mode (Editor) |
| `app.pause` | Toggle pause (Editor) |
| `app.stop` | Exit play mode (Editor) |

### Debug
| Tool | Description |
|------|-------------|
| `debug.log` | Write to console |
| `debug.screenshot` | Capture screenshot (with UI) |
| `debug.hierarchy` | Text hierarchy view |

### Editor (NEW in v1.2.0)
| Tool | Description |
|------|-------------|
| `editor.refresh` | Refresh AssetDatabase (triggers recompile) |
| `editor.recompile` | Request script recompilation |
| `editor.focusWindow` | Focus Editor window (game/scene/console/hierarchy/project/inspector) |
| `editor.listWindows` | List all open Editor windows |

### Input Simulation (NEW in v1.2.0)
| Tool | Description |
|------|-------------|
| `input.keyPress` | Press and release a key |
| `input.keyDown` | Press and hold a key |
| `input.keyUp` | Release a key |
| `input.type` | Type text into input field |
| `input.mouseMove` | Move mouse cursor |
| `input.mouseClick` | Click at position |
| `input.mouseDrag` | Drag from A to B |
| `input.mouseScroll` | Scroll wheel |
| `input.getMousePosition` | Get current cursor position |
| `input.clickUI` | Click UI element by name |

> ⚠️ **Input Simulation Limitation**: Keyboard/mouse simulation works for **UI interactions** (Button clicks, InputField typing) but NOT for gameplay input using `Input.GetKey()` or legacy Input Manager. This is a Unity limitation - `Input.GetKey()` reads directly from the OS input buffer. For automated gameplay testing, use `transform.setPosition` to move objects directly, or migrate to Unity's **new Input System** which supports programmatic input injection.

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                     Unity Editor                             │
│                                                              │
│  ┌────────────────────────────────────────────────────────┐ │
│  │           OpenClawEditorBridge                          │ │
│  │           [InitializeOnLoad]                            │ │
│  │                                                          │ │
│  │  • EditorApplication.delayCall → safe init              │ │
│  │  • EditorApplication.update → connection polling        │ │
│  │  • SessionState → survives Play mode transitions        │ │
│  └──────────────────────┬─────────────────────────────────┘ │
│                         │                                    │
│                         ▼                                    │
│  ┌────────────────────────────────────────────────────────┐ │
│  │         OpenClawConnectionManager                       │ │
│  │         (Singleton - shared across modes)               │ │
│  │                                                          │ │
│  │  • HTTP polling for commands                            │ │
│  │  • Main thread execution queue                          │ │
│  │  • Automatic reconnection                               │ │
│  │  • JSON parsing with nested object support              │ │
│  └──────────────────────┬─────────────────────────────────┘ │
│                         │                                    │
│                         ▼                                    │
│  ┌────────────────────────────────────────────────────────┐ │
│  │           OpenClawTools (50 tools)                      │ │
│  │                                                          │ │
│  │  • Scene/GameObject/Component manipulation              │ │
│  │  • Debug tools (screenshot, hierarchy)                  │ │
│  │  • Input simulation (keyboard, mouse, UI)               │ │
│  │  • Editor control (recompile, refresh)                  │ │
│  └────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
                              │
                              │ HTTP
                              ▼
┌──────────────────────────────────────────────────────────────┐
│                   OpenClaw Gateway                            │
│                   http://localhost:18789                      │
│                                                               │
│  Endpoints:                                                   │
│  • POST /unity/re

... (truncated)