Lossless Claw

Name: Lossless Claw
Rating: 3.5 (1 reviews)
Author: Martian-Engineering

By Martian-Engineering 👁 26 views ▲ 0 votes

Lossless Claw — LCM (Lossless Context Management) plugin for OpenClaw

GitHub

Install

npm install @martian-engineering/lossless-claw

Configuration Example

{
  "plugins": {
    "paths": [
      "node_modules/@martian-engineering/lossless-claw"
    ],
    "slots": {
      "contextEngine": "lossless-claw"
    }
  }
}

README

# lossless-claw

Lossless Context Management plugin for [OpenClaw](https://github.com/openclaw/openclaw), based on the [LCM paper](https://voltropy.com/LCM). Replaces OpenClaw's built-in sliding-window compaction with a DAG-based summarization system that preserves every message while keeping active context within model token limits.

## What it does

When a conversation grows beyond the model's context window, OpenClaw (just like all of the other agents) normally truncates older messages. LCM instead:

1. **Persists every message** in a SQLite database, organized by conversation
2. **Summarizes chunks** of older messages into summaries using your configured LLM
3. **Condenses summaries** into higher-level nodes as they accumulate, forming a DAG (directed acyclic graph)
4. **Assembles context** each turn by combining summaries + recent raw messages
5. **Provides tools** (`lcm_grep`, `lcm_describe`, `lcm_expand`) so agents can search and recall details from compacted history

Nothing is lost. Raw messages stay in the database. Summaries link back to their source messages. Agents can drill into any summary to recover the original detail.

**It feels like talking to an agent that never forgets. Because it doesn't. In normal operation, you'll never need to think about compaction again.**

## Installation

### Prerequisites

- OpenClaw with context engine support (josh/context-engine branch or equivalent)
- Node.js 22+
- An LLM provider configured in OpenClaw (used for summarization)

### Install the plugin

**From npm** (recommended):

```bash
npm install @martian-engineering/lossless-claw
```

**From source** (for development):

```bash
git clone https://github.com/Martian-Engineering/lossless-claw.git
cd lossless-claw
npm install
```

### Configure OpenClaw

Add the plugin to your OpenClaw config (`~/.openclaw/openclaw.json`):

```json
{
  "plugins": {
    "paths": [
      "node_modules/@martian-engineering/lossless-claw"
    ],
    "slots": {
      "contextEngine": "lossless-claw"
    }
  }
}
```

If installed from source, use the absolute path to the cloned repo instead:

```json
{
  "plugins": {
    "paths": [
      "/path/to/lossless-claw"
    ],
    "slots": {
      "contextEngine": "lossless-claw"
    }
  }
}
```

The `slots.contextEngine` setting tells OpenClaw to route all context management through LCM instead of the built-in legacy engine.

Restart OpenClaw after configuration changes.

## Configuration

LCM is configured through a combination of plugin config and environment variables. Environment variables take precedence for backward compatibility.

### Plugin config

Add an `lossless-claw` block under `plugins.config` in your OpenClaw config:

```json
{
  "plugins": {
    "config": {
      "lossless-claw": {
        "enabled": true,
        "freshTailCount": 32,
        "contextThreshold": 0.75,
        "incrementalMaxDepth": 1
      }
    }
  }
}
```

### Environment variables

| Variable | Default | Description |
|----------|---------|-------------|
| `LCM_ENABLED` | `true` | Enable/disable the plugin |
| `LCM_DATABASE_PATH` | `~/.openclaw/lcm.db` | Path to the SQLite database |
| `LCM_CONTEXT_THRESHOLD` | `0.75` | Fraction of context window that triggers compaction (0.0–1.0) |
| `LCM_FRESH_TAIL_COUNT` | `32` | Number of recent messages protected from compaction |
| `LCM_LEAF_MIN_FANOUT` | `8` | Minimum raw messages per leaf summary |
| `LCM_CONDENSED_MIN_FANOUT` | `4` | Minimum summaries per condensed node |
| `LCM_CONDENSED_MIN_FANOUT_HARD` | `2` | Relaxed fanout for forced compaction sweeps |
| `LCM_INCREMENTAL_MAX_DEPTH` | `0` | How deep incremental compaction goes (0 = leaf only) |
| `LCM_LEAF_CHUNK_TOKENS` | `20000` | Max source tokens per leaf compaction chunk |
| `LCM_LEAF_TARGET_TOKENS` | `1200` | Target token count for leaf summaries |
| `LCM_CONDENSED_TARGET_TOKENS` | `2000` | Target token count for condensed summaries |
| `LCM_MAX_EXPAND_TOKENS` | `4000` | Token cap for sub-agent expansion queries |
| `LCM_LARGE_FILE_TOKEN_THRESHOLD` | `25000` | File blocks above this size are intercepted and stored separately |
| `LCM_SUMMARY_MODEL` | *(from OpenClaw)* | Model for summarization (e.g. `anthropic/claude-sonnet-4-20250514`) |
| `LCM_SUMMARY_PROVIDER` | *(from OpenClaw)* | Provider override for summarization |
| `LCM_INCREMENTAL_MAX_DEPTH` | `0` | Depth limit for incremental condensation after leaf passes |

### Recommended starting configuration

```
LCM_FRESH_TAIL_COUNT=32
LCM_INCREMENTAL_MAX_DEPTH=1
LCM_CONTEXT_THRESHOLD=0.75
```

- **freshTailCount=32** protects the last 32 messages from compaction, giving the model enough recent context for continuity.
- **incrementalMaxDepth=1** enables automatic condensation of leaf summaries after each compaction pass (without this, only leaf summaries are created and condensation only happens during manual `/compact` or overflow).
- **contextThreshold=0.75** triggers compaction when context reaches 75% of the model's window, leaving headroom for the model's response.

## How it works

See [docs/architecture.md](docs/architecture.md) for the full technical deep-dive. Here's the summary:

### The DAG

LCM builds a directed acyclic graph of summaries:

```
Raw messages → Leaf summaries (d0) → Condensed (d1) → Condensed (d2) → ...
```

- **Leaf summaries** (depth 0) are created from chunks of raw messages. They preserve timestamps, decisions, file operations, and key details.
- **Condensed summaries** (depth 1+) merge multiple summaries at the same depth into a higher-level node. Each depth tier uses a different prompt strategy optimized for its level of abstraction.
- **Parent links** connect each condensed summary to its source summaries, enabling drill-down via `lcm_expand_query`.

### Context assembly

Each turn, the assembler builds model context by:

1. Fetching the conversation's **context items** (an ordered list of summary and message references)
2. Resolving each item into an `AgentMessage`
3. Protecting the **fresh tail** (most recent N messages) from eviction
4. Filling remaining token budget from oldest to newest, dropping the oldest items first if over budget
5. Wrapping summaries in XML with metadata (id, depth, timestamps, descendant count)

The model sees something like:

```xml
<summary id="sum_abc123" kind="condensed" depth="1" descendant_count="8"
         earliest_at="2026-02-17T07:37:00" latest_at="2026-02-17T15:43:00">
  <parents>
    <summary_ref id="sum_def456" />
    <summary_ref id="sum_ghi789" />
  </parents>
  <content>
    ...summary text...
  </content>
</summary>
```

This gives the model enough information to know what was discussed, when, and how to drill deeper via the expansion tools.

### Compaction triggers

Compaction runs in two modes:

- **Proactive (after each turn):** If raw messages outside the fresh tail exceed `leafChunkTokens`, a leaf pass runs. If `incrementalMaxDepth > 0`, condensation follows.
- **Reactive (overflow/manual):** When total context exceeds `contextThreshold × tokenBudget`, a full sweep runs: all eligible leaf chunks are compacted, then condensation proceeds depth-by-depth until stable.

### Depth-aware prompts

Each summary depth gets a tailored prompt:

| Depth | Kind | Strategy |
|-------|------|----------|
| 0 | Leaf | Narrative with timestamps, file tracking, preserves operational detail |
| 1 | Condensed | Chronological session summary, deduplicates against `previous_context` |
| 2 | Condensed | Arc-focused: goals, outcomes, what carries forward. Self-contained. |
| 3+ | Condensed | Durable context only: key decisions, relationships, lessons learned |

All summaries end with an "Expand for details about:" footer listing what was compressed, guiding agents on when to use `lcm_expand_query`.

### Large file handling

Files over `largeFileTokenThreshold` (default 25k tokens) embedded in messages are intercepted during ingestion:

1. Content is stored to `~/.openclaw/lcm-files/<conversation_id>/<file_id>.<ext>`
2. A ~200 token exploration summary replaces the file in the message
3. The `lcm_describe` tool can retrieve the full file content on demand

This prevents large file pastes from consuming the entire context window.

## Agent tools

LCM registers four tools that agents can use to search and recall compacted history:

### `lcm_grep`

Full-text and regex search across messages and summaries.

```
lcm_grep(pattern: "database migration", mode: "full_text")
lcm_grep(pattern: "config\\.threshold", mode: "regex", scope: "summaries")
```

Parameters:
- `pattern` — Search string (regex or full-text)
- `mode` — `"regex"` (default) or `"full_text"`
- `scope` — `"messages"`, `"summaries"`, or `"both"` (default)
- `conversationId` — Scope to a specific conversation
- `allConversations` — Search across all conversations
- `since` / `before` — ISO timestamp filters
- `limit` — Max results (default 50, max 200)

### `lcm_describe`

Inspect a specific summary or stored file by ID.

```
lcm_describe(id: "sum_abc123")
lcm_describe(id: "file_def456")
```

Returns the full content, metadata, parent/child relationships, and token counts. For files, returns the stored content.

### `lcm_expand_query`

Deep recall via delegated sub-agent. Finds relevant summaries, expands them by walking the DAG down to source material, and answers a focused question.

```
lcm_expand_query(
  query: "database migration",
  prompt: "What migration strategy was decided on?"
)

lcm_expand_query(
  summaryIds: ["sum_abc123"],
  prompt: "What were the exact config changes?"
)
```

Parameters:
- `prompt` — The question to answer (required)
- `query` — Text query to find relevant summaries (when you don't have IDs)
- `summaryIds` — Specific summary IDs to expand (when you have them)
- `maxTokens` — Answer length cap (default 2000)
- `conversationId` / `allConversations` — Scope control

Returns a compact answer with cited summary IDs.

### `lcm_expand`

Low-level DAG expansion (sub-agent only). Main agents should use `lcm_ex

... (truncated)

tools