chore: Run pnpm format:fix.
This commit is contained in:
@@ -4,13 +4,17 @@ read_when:
|
||||
- Planning a unified network protocol for nodes + operator clients
|
||||
- Reworking approvals, pairing, TLS, and presence across devices
|
||||
---
|
||||
|
||||
# Clawnet refactor (protocol + auth unification)
|
||||
|
||||
## Hi
|
||||
|
||||
Hi Peter — great direction; this unlocks simpler UX + stronger security.
|
||||
|
||||
## Purpose
|
||||
|
||||
Single, rigorous document for:
|
||||
|
||||
- Current state: protocols, flows, trust boundaries.
|
||||
- Pain points: approvals, multi‑hop routing, UI duplication.
|
||||
- Proposed new state: one protocol, scoped roles, unified auth/pairing, TLS pinning.
|
||||
@@ -18,6 +22,7 @@ Single, rigorous document for:
|
||||
- Migration plan, risks, open questions.
|
||||
|
||||
## Goals (from discussion)
|
||||
|
||||
- One protocol for all clients (mac app, CLI, iOS, Android, headless node).
|
||||
- Every network participant authenticated + paired.
|
||||
- Role clarity: nodes vs operators.
|
||||
@@ -27,6 +32,7 @@ Single, rigorous document for:
|
||||
- Single machine should appear once (no UI/node duplicate entry).
|
||||
|
||||
## Non‑goals (explicit)
|
||||
|
||||
- Remove capability separation (still need least‑privilege).
|
||||
- Expose full gateway control plane without scope checks.
|
||||
- Make auth depend on human labels (slugs remain non‑security).
|
||||
@@ -38,6 +44,7 @@ Single, rigorous document for:
|
||||
## Two protocols
|
||||
|
||||
### 1) Gateway WebSocket (control plane)
|
||||
|
||||
- Full API surface: config, channels, models, sessions, agent runs, logs, nodes, etc.
|
||||
- Default bind: loopback. Remote access via SSH/Tailscale.
|
||||
- Auth: token/password via `connect`.
|
||||
@@ -48,6 +55,7 @@ Single, rigorous document for:
|
||||
- `docs/gateway/protocol.md`
|
||||
|
||||
### 2) Bridge (node transport)
|
||||
|
||||
- Narrow allowlist surface, node identity + pairing.
|
||||
- JSONL over TCP; optional TLS + cert fingerprint pinning.
|
||||
- TLS advertises fingerprint in discovery TXT.
|
||||
@@ -58,6 +66,7 @@ Single, rigorous document for:
|
||||
- `docs/gateway/bridge-protocol.md`
|
||||
|
||||
## Control plane clients today
|
||||
|
||||
- CLI → Gateway WS via `callGateway` (`src/gateway/call.ts`).
|
||||
- macOS app UI → Gateway WS (`GatewayConnection`).
|
||||
- Web Control UI → Gateway WS.
|
||||
@@ -65,11 +74,13 @@ Single, rigorous document for:
|
||||
- Browser control uses its own HTTP control server.
|
||||
|
||||
## Nodes today
|
||||
|
||||
- macOS app in node mode connects to Gateway bridge (`MacNodeBridgeSession`).
|
||||
- iOS/Android apps connect to Gateway bridge.
|
||||
- Pairing + per‑node token stored on gateway.
|
||||
|
||||
## Current approval flow (exec)
|
||||
|
||||
- Agent uses `system.run` via Gateway.
|
||||
- Gateway invokes node over bridge.
|
||||
- Node runtime decides approval.
|
||||
@@ -78,6 +89,7 @@ Single, rigorous document for:
|
||||
- Multi‑hop, UI tied to node host.
|
||||
|
||||
## Presence + identity today
|
||||
|
||||
- Gateway presence entries from WS clients.
|
||||
- Node presence entries from bridge.
|
||||
- mac app can show two entries for same machine (UI + node).
|
||||
@@ -98,7 +110,9 @@ Single, rigorous document for:
|
||||
# Proposed new state (Clawnet)
|
||||
|
||||
## One protocol, two roles
|
||||
|
||||
Single WS protocol with role + scope.
|
||||
|
||||
- **Role: node** (capability host)
|
||||
- **Role: operator** (control plane)
|
||||
- Optional **scope** for operator:
|
||||
@@ -109,17 +123,20 @@ Single WS protocol with role + scope.
|
||||
### Role behaviors
|
||||
|
||||
**Node**
|
||||
|
||||
- Can register capabilities (`caps`, `commands`, permissions).
|
||||
- Can receive `invoke` commands (`system.run`, `camera.*`, `canvas.*`, `screen.record`, etc).
|
||||
- Can send events: `voice.transcript`, `agent.request`, `chat.subscribe`.
|
||||
- Cannot call config/models/channels/sessions/agent control plane APIs.
|
||||
|
||||
**Operator**
|
||||
|
||||
- Full control plane API, gated by scope.
|
||||
- Receives all approvals.
|
||||
- Does not directly execute OS actions; routes to nodes.
|
||||
|
||||
### Key rule
|
||||
|
||||
Role is per‑connection, not per device. A device may open both roles, separately.
|
||||
|
||||
---
|
||||
@@ -127,12 +144,15 @@ Role is per‑connection, not per device. A device may open both roles, separate
|
||||
# Unified authentication + pairing
|
||||
|
||||
## Client identity
|
||||
|
||||
Every client provides:
|
||||
|
||||
- `deviceId` (stable, derived from device key).
|
||||
- `displayName` (human name).
|
||||
- `role` + `scope` + `caps` + `commands`.
|
||||
|
||||
## Pairing flow (unified)
|
||||
|
||||
- Client connects unauthenticated.
|
||||
- Gateway creates a **pairing request** for that `deviceId`.
|
||||
- Operator receives prompt; approves/denies.
|
||||
@@ -144,18 +164,23 @@ Every client provides:
|
||||
- Client persists token, reconnects authenticated.
|
||||
|
||||
## Device‑bound auth (avoid bearer token replay)
|
||||
|
||||
Preferred: device keypairs.
|
||||
|
||||
- Device generates keypair once.
|
||||
- `deviceId = fingerprint(publicKey)`.
|
||||
- Gateway sends nonce; device signs; gateway verifies.
|
||||
- Tokens are issued to a public key (proof‑of‑possession), not a string.
|
||||
|
||||
Alternatives:
|
||||
|
||||
- mTLS (client certs): strongest, more ops complexity.
|
||||
- Short‑lived bearer tokens only as a temporary phase (rotate + revoke early).
|
||||
|
||||
## Silent approval (SSH heuristic)
|
||||
|
||||
Define it precisely to avoid a weak link. Prefer one:
|
||||
|
||||
- **Local‑only**: auto‑pair when client connects via loopback/Unix socket.
|
||||
- **Challenge via SSH**: gateway issues nonce; client proves SSH by fetching it.
|
||||
- **Physical presence window**: after a local approval on gateway host UI, allow auto‑pair for a short window (e.g. 10 minutes).
|
||||
@@ -167,17 +192,21 @@ Always log + record auto‑approvals.
|
||||
# TLS everywhere (dev + prod)
|
||||
|
||||
## Reuse existing bridge TLS
|
||||
|
||||
Use current TLS runtime + fingerprint pinning:
|
||||
|
||||
- `src/infra/bridge/server/tls.ts`
|
||||
- fingerprint verification logic in `src/node-host/bridge-client.ts`
|
||||
|
||||
## Apply to WS
|
||||
|
||||
- WS server supports TLS with same cert/key + fingerprint.
|
||||
- WS clients can pin fingerprint (optional).
|
||||
- Discovery advertises TLS + fingerprint for all endpoints.
|
||||
- Discovery is locator hints only; never a trust anchor.
|
||||
|
||||
## Why
|
||||
|
||||
- Reduce reliance on SSH/Tailscale for confidentiality.
|
||||
- Make remote mobile connections safe by default.
|
||||
|
||||
@@ -186,26 +215,31 @@ Use current TLS runtime + fingerprint pinning:
|
||||
# Approvals redesign (centralized)
|
||||
|
||||
## Current
|
||||
|
||||
Approval happens on node host (mac app node runtime). Prompt appears where node runs.
|
||||
|
||||
## Proposed
|
||||
|
||||
Approval is **gateway‑hosted**, UI delivered to operator clients.
|
||||
|
||||
### New flow
|
||||
1) Gateway receives `system.run` intent (agent).
|
||||
2) Gateway creates approval record: `approval.requested`.
|
||||
3) Operator UI(s) show prompt.
|
||||
4) Approval decision sent to gateway: `approval.resolve`.
|
||||
5) Gateway invokes node command if approved.
|
||||
6) Node executes, returns `invoke-res`.
|
||||
|
||||
1. Gateway receives `system.run` intent (agent).
|
||||
2. Gateway creates approval record: `approval.requested`.
|
||||
3. Operator UI(s) show prompt.
|
||||
4. Approval decision sent to gateway: `approval.resolve`.
|
||||
5. Gateway invokes node command if approved.
|
||||
6. Node executes, returns `invoke-res`.
|
||||
|
||||
### Approval semantics (hardening)
|
||||
|
||||
- Broadcast to all operators; only the active UI shows a modal (others get a toast).
|
||||
- First resolution wins; gateway rejects subsequent resolves as already settled.
|
||||
- Default timeout: deny after N seconds (e.g. 60s), log reason.
|
||||
- Resolution requires `operator.approvals` scope.
|
||||
|
||||
## Benefits
|
||||
|
||||
- Prompt appears where user is (mac/phone).
|
||||
- Consistent approvals for remote nodes.
|
||||
- Node runtime stays headless; no UI dependency.
|
||||
@@ -215,16 +249,19 @@ Approval is **gateway‑hosted**, UI delivered to operator clients.
|
||||
# Role clarity examples
|
||||
|
||||
## iPhone app
|
||||
|
||||
- **Node role** for: mic, camera, voice chat, location, push‑to‑talk.
|
||||
- Optional **operator.read** for status and chat view.
|
||||
- Optional **operator.write/admin** only when explicitly enabled.
|
||||
|
||||
## macOS app
|
||||
|
||||
- Operator role by default (control UI).
|
||||
- Node role when “Mac node” enabled (system.run, screen, camera).
|
||||
- Same deviceId for both connections → merged UI entry.
|
||||
|
||||
## CLI
|
||||
|
||||
- Operator role always.
|
||||
- Scope derived by subcommand:
|
||||
- `status`, `logs` → read
|
||||
@@ -237,18 +274,24 @@ Approval is **gateway‑hosted**, UI delivered to operator clients.
|
||||
# Identity + slugs
|
||||
|
||||
## Stable ID
|
||||
|
||||
Required for auth; never changes.
|
||||
Preferred:
|
||||
|
||||
- Keypair fingerprint (public key hash).
|
||||
|
||||
## Cute slug (lobster‑themed)
|
||||
|
||||
Human label only.
|
||||
|
||||
- Example: `scarlet-claw`, `saltwave`, `mantis-pinch`.
|
||||
- Stored in gateway registry, editable.
|
||||
- Collision handling: `-2`, `-3`.
|
||||
|
||||
## UI grouping
|
||||
|
||||
Same `deviceId` across roles → single “Instance” row:
|
||||
|
||||
- Badge: `operator`, `node`.
|
||||
- Shows capabilities + last seen.
|
||||
|
||||
@@ -257,32 +300,39 @@ Same `deviceId` across roles → single “Instance” row:
|
||||
# Migration strategy
|
||||
|
||||
## Phase 0: Document + align
|
||||
|
||||
- Publish this doc.
|
||||
- Inventory all protocol calls + approval flows.
|
||||
|
||||
## Phase 1: Add roles/scopes to WS
|
||||
|
||||
- Extend `connect` params with `role`, `scope`, `deviceId`.
|
||||
- Add allowlist gating for node role.
|
||||
|
||||
## Phase 2: Bridge compatibility
|
||||
|
||||
- Keep bridge running.
|
||||
- Add WS node support in parallel.
|
||||
- Gate features behind config flag.
|
||||
|
||||
## Phase 3: Central approvals
|
||||
|
||||
- Add approval request + resolve events in WS.
|
||||
- Update mac app UI to prompt + respond.
|
||||
- Node runtime stops prompting UI.
|
||||
|
||||
## Phase 4: TLS unification
|
||||
|
||||
- Add TLS config for WS using bridge TLS runtime.
|
||||
- Add pinning to clients.
|
||||
|
||||
## Phase 5: Deprecate bridge
|
||||
|
||||
- Migrate iOS/Android/mac node to WS.
|
||||
- Keep bridge as fallback; remove once stable.
|
||||
|
||||
## Phase 6: Device‑bound auth
|
||||
|
||||
- Require key‑based identity for all non‑local connections.
|
||||
- Add revocation + rotation UI.
|
||||
|
||||
@@ -292,36 +342,42 @@ Same `deviceId` across roles → single “Instance” row:
|
||||
|
||||
- Role/allowlist enforced at gateway boundary.
|
||||
- No client gets “full” API without operator scope.
|
||||
- Pairing required for *all* connections.
|
||||
- Pairing required for _all_ connections.
|
||||
- TLS + pinning reduces MITM risk for mobile.
|
||||
- SSH silent approval is a convenience; still recorded + revocable.
|
||||
- Discovery is never a trust anchor.
|
||||
- Capability claims are verified against server allowlists by platform/type.
|
||||
|
||||
# Streaming + large payloads (node media)
|
||||
|
||||
WS control plane is fine for small messages, but nodes also do:
|
||||
|
||||
- camera clips
|
||||
- screen recordings
|
||||
- audio streams
|
||||
|
||||
Options:
|
||||
1) WS binary frames + chunking + backpressure rules.
|
||||
2) Separate streaming endpoint (still TLS + auth).
|
||||
3) Keep bridge longer for media‑heavy commands, migrate last.
|
||||
|
||||
1. WS binary frames + chunking + backpressure rules.
|
||||
2. Separate streaming endpoint (still TLS + auth).
|
||||
3. Keep bridge longer for media‑heavy commands, migrate last.
|
||||
|
||||
Pick one before implementation to avoid drift.
|
||||
|
||||
# Capability + command policy
|
||||
|
||||
- Node‑reported caps/commands are treated as **claims**.
|
||||
- Gateway enforces per‑platform allowlists.
|
||||
- Any new command requires operator approval or explicit allowlist change.
|
||||
- Audit changes with timestamps.
|
||||
|
||||
# Audit + rate limiting
|
||||
|
||||
- Log: pairing requests, approvals/denials, token issuance/rotation/revocation.
|
||||
- Rate‑limit pairing spam and approval prompts.
|
||||
|
||||
# Protocol hygiene
|
||||
|
||||
- Explicit protocol version + error codes.
|
||||
- Reconnect rules + heartbeat policy.
|
||||
- Presence TTL and last‑seen semantics.
|
||||
@@ -330,23 +386,23 @@ Pick one before implementation to avoid drift.
|
||||
|
||||
# Open questions
|
||||
|
||||
1) Single device running both roles: token model
|
||||
1. Single device running both roles: token model
|
||||
- Recommend separate tokens per role (node vs operator).
|
||||
- Same deviceId; different scopes; clearer revocation.
|
||||
|
||||
2) Operator scope granularity
|
||||
2. Operator scope granularity
|
||||
- read/write/admin + approvals + pairing (minimum viable).
|
||||
- Consider per‑feature scopes later.
|
||||
|
||||
3) Token rotation + revocation UX
|
||||
3. Token rotation + revocation UX
|
||||
- Auto‑rotate on role change.
|
||||
- UI to revoke by deviceId + role.
|
||||
|
||||
4) Discovery
|
||||
4. Discovery
|
||||
- Extend current Bonjour TXT to include WS TLS fingerprint + role hints.
|
||||
- Treat as locator hints only.
|
||||
|
||||
5) Cross‑network approval
|
||||
5. Cross‑network approval
|
||||
- Broadcast to all operator clients; active UI shows modal.
|
||||
- First response wins; gateway enforces atomicity.
|
||||
|
||||
|
||||
@@ -9,19 +9,22 @@ read_when:
|
||||
# Exec host refactor plan
|
||||
|
||||
## Goals
|
||||
|
||||
- Add `exec.host` + `exec.security` to route execution across **sandbox**, **gateway**, and **node**.
|
||||
- Keep defaults **safe**: no cross-host execution unless explicitly enabled.
|
||||
- Split execution into a **headless runner service** with optional UI (macOS app) via local IPC.
|
||||
- Provide **per-agent** policy, allowlist, ask mode, and node binding.
|
||||
- Support **ask modes** that work *with* or *without* allowlists.
|
||||
- Support **ask modes** that work _with_ or _without_ allowlists.
|
||||
- Cross-platform: Unix socket + token auth (macOS/Linux/Windows parity).
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No legacy allowlist migration or legacy schema support.
|
||||
- No PTY/streaming for node exec (aggregated output only).
|
||||
- No new network layer beyond the existing Bridge + Gateway.
|
||||
|
||||
## Decisions (locked)
|
||||
|
||||
- **Config keys:** `exec.host` + `exec.security` (per-agent override allowed).
|
||||
- **Elevation:** keep `/elevated` as an alias for gateway full access.
|
||||
- **Ask default:** `on-miss`.
|
||||
@@ -34,17 +37,21 @@ read_when:
|
||||
- **No XPC helper:** stick to Unix socket + token + peer checks.
|
||||
|
||||
## Key concepts
|
||||
|
||||
### Host
|
||||
|
||||
- `sandbox`: Docker exec (current behavior).
|
||||
- `gateway`: exec on gateway host.
|
||||
- `node`: exec on node runner via Bridge (`system.run`).
|
||||
|
||||
### Security mode
|
||||
|
||||
- `deny`: always block.
|
||||
- `allowlist`: allow only matches.
|
||||
- `full`: allow everything (equivalent to elevated).
|
||||
|
||||
### Ask mode
|
||||
|
||||
- `off`: never ask.
|
||||
- `on-miss`: ask only when allowlist does not match.
|
||||
- `always`: ask every time.
|
||||
@@ -52,49 +59,59 @@ read_when:
|
||||
Ask is **independent** of allowlist; allowlist can be used with `always` or `on-miss`.
|
||||
|
||||
### Policy resolution (per exec)
|
||||
1) Resolve `exec.host` (tool param → agent override → global default).
|
||||
2) Resolve `exec.security` and `exec.ask` (same precedence).
|
||||
3) If host is `sandbox`, proceed with local sandbox exec.
|
||||
4) If host is `gateway` or `node`, apply security + ask policy on that host.
|
||||
|
||||
1. Resolve `exec.host` (tool param → agent override → global default).
|
||||
2. Resolve `exec.security` and `exec.ask` (same precedence).
|
||||
3. If host is `sandbox`, proceed with local sandbox exec.
|
||||
4. If host is `gateway` or `node`, apply security + ask policy on that host.
|
||||
|
||||
## Default safety
|
||||
|
||||
- Default `exec.host = sandbox`.
|
||||
- Default `exec.security = deny` for `gateway` and `node`.
|
||||
- Default `exec.ask = on-miss` (only relevant if security allows).
|
||||
- If no node binding is set, **agent may target any node**, but only if policy allows it.
|
||||
|
||||
## Config surface
|
||||
|
||||
### Tool parameters
|
||||
|
||||
- `exec.host` (optional): `sandbox | gateway | node`.
|
||||
- `exec.security` (optional): `deny | allowlist | full`.
|
||||
- `exec.ask` (optional): `off | on-miss | always`.
|
||||
- `exec.node` (optional): node id/name to use when `host=node`.
|
||||
|
||||
### Config keys (global)
|
||||
|
||||
- `tools.exec.host`
|
||||
- `tools.exec.security`
|
||||
- `tools.exec.ask`
|
||||
- `tools.exec.node` (default node binding)
|
||||
|
||||
### Config keys (per agent)
|
||||
|
||||
- `agents.list[].tools.exec.host`
|
||||
- `agents.list[].tools.exec.security`
|
||||
- `agents.list[].tools.exec.ask`
|
||||
- `agents.list[].tools.exec.node`
|
||||
|
||||
### Alias
|
||||
|
||||
- `/elevated on` = set `tools.exec.host=gateway`, `tools.exec.security=full` for the agent session.
|
||||
- `/elevated off` = restore previous exec settings for the agent session.
|
||||
|
||||
## Approvals store (JSON)
|
||||
|
||||
Path: `~/.openclaw/exec-approvals.json`
|
||||
|
||||
Purpose:
|
||||
|
||||
- Local policy + allowlists for the **execution host** (gateway or node runner).
|
||||
- Ask fallback when no UI is available.
|
||||
- IPC credentials for UI clients.
|
||||
|
||||
Proposed schema (v1):
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
@@ -123,24 +140,31 @@ Proposed schema (v1):
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- No legacy allowlist formats.
|
||||
- `askFallback` applies only when `ask` is required and no UI is reachable.
|
||||
- File permissions: `0600`.
|
||||
|
||||
## Runner service (headless)
|
||||
|
||||
### Role
|
||||
|
||||
- Enforce `exec.security` + `exec.ask` locally.
|
||||
- Execute system commands and return output.
|
||||
- Emit Bridge events for exec lifecycle (optional but recommended).
|
||||
|
||||
### Service lifecycle
|
||||
|
||||
- Launchd/daemon on macOS; system service on Linux/Windows.
|
||||
- Approvals JSON is local to the execution host.
|
||||
- UI hosts a local Unix socket; runners connect on demand.
|
||||
|
||||
## UI integration (macOS app)
|
||||
|
||||
### IPC
|
||||
|
||||
- Unix socket at `~/.openclaw/exec-approvals.sock` (0600).
|
||||
- Token stored in `exec-approvals.json` (0600).
|
||||
- Peer checks: same-UID only.
|
||||
@@ -148,16 +172,19 @@ Notes:
|
||||
- Short TTL (e.g., 10s) + max payload + rate limit.
|
||||
|
||||
### Ask flow (macOS app exec host)
|
||||
1) Node service receives `system.run` from gateway.
|
||||
2) Node service connects to the local socket and sends the prompt/exec request.
|
||||
3) App validates peer + token + HMAC + TTL, then shows dialog if needed.
|
||||
4) App executes the command in UI context and returns output.
|
||||
5) Node service returns output to gateway.
|
||||
|
||||
1. Node service receives `system.run` from gateway.
|
||||
2. Node service connects to the local socket and sends the prompt/exec request.
|
||||
3. App validates peer + token + HMAC + TTL, then shows dialog if needed.
|
||||
4. App executes the command in UI context and returns output.
|
||||
5. Node service returns output to gateway.
|
||||
|
||||
If UI missing:
|
||||
|
||||
- Apply `askFallback` (`deny|allowlist|full`).
|
||||
|
||||
### Diagram (SCI)
|
||||
|
||||
```
|
||||
Agent -> Gateway -> Bridge -> Node Service (TS)
|
||||
| IPC (UDS + token + HMAC + TTL)
|
||||
@@ -166,6 +193,7 @@ Agent -> Gateway -> Bridge -> Node Service (TS)
|
||||
```
|
||||
|
||||
## Node identity + binding
|
||||
|
||||
- Use existing `nodeId` from Bridge pairing.
|
||||
- Binding model:
|
||||
- `tools.exec.node` restricts the agent to a specific node.
|
||||
@@ -177,88 +205,110 @@ Agent -> Gateway -> Bridge -> Node Service (TS)
|
||||
- `nodeId` prefix (>= 6 chars)
|
||||
|
||||
## Eventing
|
||||
|
||||
### Who sees events
|
||||
|
||||
- System events are **per session** and shown to the agent on the next prompt.
|
||||
- Stored in the gateway in-memory queue (`enqueueSystemEvent`).
|
||||
|
||||
### Event text
|
||||
|
||||
- `Exec started (node=<id>, id=<runId>)`
|
||||
- `Exec finished (node=<id>, id=<runId>, code=<code>)` + optional output tail
|
||||
- `Exec denied (node=<id>, id=<runId>, <reason>)`
|
||||
|
||||
### Transport
|
||||
|
||||
Option A (recommended):
|
||||
|
||||
- Runner sends Bridge `event` frames `exec.started` / `exec.finished`.
|
||||
- Gateway `handleBridgeEvent` maps these into `enqueueSystemEvent`.
|
||||
|
||||
Option B:
|
||||
|
||||
- Gateway `exec` tool handles lifecycle directly (synchronous only).
|
||||
|
||||
## Exec flows
|
||||
|
||||
### Sandbox host
|
||||
|
||||
- Existing `exec` behavior (Docker or host when unsandboxed).
|
||||
- PTY supported in non-sandbox mode only.
|
||||
|
||||
### Gateway host
|
||||
|
||||
- Gateway process executes on its own machine.
|
||||
- Enforces local `exec-approvals.json` (security/ask/allowlist).
|
||||
|
||||
### Node host
|
||||
|
||||
- Gateway calls `node.invoke` with `system.run`.
|
||||
- Runner enforces local approvals.
|
||||
- Runner returns aggregated stdout/stderr.
|
||||
- Optional Bridge events for start/finish/deny.
|
||||
|
||||
## Output caps
|
||||
|
||||
- Cap combined stdout+stderr at **200k**; keep **tail 20k** for events.
|
||||
- Truncate with a clear suffix (e.g., `"… (truncated)"`).
|
||||
|
||||
## Slash commands
|
||||
|
||||
- `/exec host=<sandbox|gateway|node> security=<deny|allowlist|full> ask=<off|on-miss|always> node=<id>`
|
||||
- Per-agent, per-session overrides; non-persistent unless saved via config.
|
||||
- `/elevated on|off|ask|full` remains a shortcut for `host=gateway security=full` (with `full` skipping approvals).
|
||||
|
||||
## Cross-platform story
|
||||
|
||||
- The runner service is the portable execution target.
|
||||
- UI is optional; if missing, `askFallback` applies.
|
||||
- Windows/Linux support the same approvals JSON + socket protocol.
|
||||
|
||||
## Implementation phases
|
||||
|
||||
### Phase 1: config + exec routing
|
||||
|
||||
- Add config schema for `exec.host`, `exec.security`, `exec.ask`, `exec.node`.
|
||||
- Update tool plumbing to respect `exec.host`.
|
||||
- Add `/exec` slash command and keep `/elevated` alias.
|
||||
|
||||
### Phase 2: approvals store + gateway enforcement
|
||||
|
||||
- Implement `exec-approvals.json` reader/writer.
|
||||
- Enforce allowlist + ask modes for `gateway` host.
|
||||
- Add output caps.
|
||||
|
||||
### Phase 3: node runner enforcement
|
||||
|
||||
- Update node runner to enforce allowlist + ask.
|
||||
- Add Unix socket prompt bridge to macOS app UI.
|
||||
- Wire `askFallback`.
|
||||
|
||||
### Phase 4: events
|
||||
|
||||
- Add node → gateway Bridge events for exec lifecycle.
|
||||
- Map to `enqueueSystemEvent` for agent prompts.
|
||||
|
||||
### Phase 5: UI polish
|
||||
|
||||
- Mac app: allowlist editor, per-agent switcher, ask policy UI.
|
||||
- Node binding controls (optional).
|
||||
|
||||
## Testing plan
|
||||
|
||||
- Unit tests: allowlist matching (glob + case-insensitive).
|
||||
- Unit tests: policy resolution precedence (tool param → agent override → global).
|
||||
- Integration tests: node runner deny/allow/ask flows.
|
||||
- Bridge event tests: node event → system event routing.
|
||||
|
||||
## Open risks
|
||||
|
||||
- UI unavailability: ensure `askFallback` is respected.
|
||||
- Long-running commands: rely on timeout + output caps.
|
||||
- Multi-node ambiguity: error unless node binding or explicit node param.
|
||||
|
||||
## Related docs
|
||||
|
||||
- [Exec tool](/tools/exec)
|
||||
- [Exec approvals](/tools/exec-approvals)
|
||||
- [Nodes](/nodes)
|
||||
|
||||
@@ -6,20 +6,24 @@ description: Track outbound session mirroring refactor notes, decisions, tests,
|
||||
# Outbound Session Mirroring Refactor (Issue #1520)
|
||||
|
||||
## Status
|
||||
|
||||
- In progress.
|
||||
- Core + plugin channel routing updated for outbound mirroring.
|
||||
- Gateway send now derives target session when sessionKey is omitted.
|
||||
|
||||
## Context
|
||||
Outbound sends were mirrored into the *current* agent session (tool session key) rather than the target channel session. Inbound routing uses channel/peer session keys, so outbound responses landed in the wrong session and first-contact targets often lacked session entries.
|
||||
|
||||
Outbound sends were mirrored into the _current_ agent session (tool session key) rather than the target channel session. Inbound routing uses channel/peer session keys, so outbound responses landed in the wrong session and first-contact targets often lacked session entries.
|
||||
|
||||
## Goals
|
||||
|
||||
- Mirror outbound messages into the target channel session key.
|
||||
- Create session entries on outbound when missing.
|
||||
- Keep thread/topic scoping aligned with inbound session keys.
|
||||
- Cover core channels plus bundled extensions.
|
||||
|
||||
## Implementation Summary
|
||||
|
||||
- New outbound session routing helper:
|
||||
- `src/infra/outbound/outbound-session.ts`
|
||||
- `resolveOutboundSessionRoute` builds target sessionKey using `buildAgentSessionKey` (dmScope + identityLinks).
|
||||
@@ -30,11 +34,13 @@ Outbound sends were mirrored into the *current* agent session (tool session key)
|
||||
- Gateway send derives a target session key when none is provided (default agent), and ensures a session entry.
|
||||
|
||||
## Thread/Topic Handling
|
||||
|
||||
- Slack: replyTo/threadId -> `resolveThreadSessionKeys` (suffix).
|
||||
- Discord: threadId/replyTo -> `resolveThreadSessionKeys` with `useSuffix=false` to match inbound (thread channel id already scopes session).
|
||||
- Telegram: topic IDs map to `chatId:topic:<id>` via `buildTelegramGroupPeerId`.
|
||||
|
||||
## Extensions Covered
|
||||
|
||||
- Matrix, MS Teams, Mattermost, BlueBubbles, Nextcloud Talk, Zalo, Zalo Personal, Nostr, Tlon.
|
||||
- Notes:
|
||||
- Mattermost targets now strip `@` for DM session key routing.
|
||||
@@ -44,12 +50,14 @@ Outbound sends were mirrored into the *current* agent session (tool session key)
|
||||
- Gateway send lowercases provided session keys before mirroring.
|
||||
|
||||
## Decisions
|
||||
|
||||
- **Gateway send session derivation**: if `sessionKey` is provided, use it. If omitted, derive a sessionKey from target + default agent and mirror there.
|
||||
- **Session entry creation**: always use `recordSessionMetaFromInbound` with `Provider/From/To/ChatType/AccountId/Originating*` aligned to inbound formats.
|
||||
- **Target normalization**: outbound routing uses resolved targets (post `resolveChannelTarget`) when available.
|
||||
- **Session key casing**: canonicalize session keys to lowercase on write and during migrations.
|
||||
|
||||
## Tests Added/Updated
|
||||
|
||||
- `src/infra/outbound/outbound-session.test.ts`
|
||||
- Slack thread session key.
|
||||
- Telegram topic session key.
|
||||
@@ -60,10 +68,12 @@ Outbound sends were mirrored into the *current* agent session (tool session key)
|
||||
- Derives session key when omitted and creates session entry.
|
||||
|
||||
## Open Items / Follow-ups
|
||||
|
||||
- Voice-call plugin uses custom `voice:<phone>` session keys. Outbound mapping is not standardized here; if message-tool should support voice-call sends, add explicit mapping.
|
||||
- Confirm if any external plugin uses non-standard `From/To` formats beyond the bundled set.
|
||||
|
||||
## Files Touched
|
||||
|
||||
- `src/infra/outbound/outbound-session.ts`
|
||||
- `src/infra/outbound/outbound-send-service.ts`
|
||||
- `src/infra/outbound/message-action-runner.ts`
|
||||
|
||||
@@ -4,21 +4,25 @@ read_when:
|
||||
- Defining or refactoring the plugin architecture
|
||||
- Migrating channel connectors to the plugin SDK/runtime
|
||||
---
|
||||
|
||||
# Plugin SDK + Runtime Refactor Plan
|
||||
|
||||
Goal: every messaging connector is a plugin (bundled or external) using one stable API.
|
||||
No plugin imports from `src/**` directly. All dependencies go through the SDK or runtime.
|
||||
|
||||
## Why now
|
||||
|
||||
- Current connectors mix patterns: direct core imports, dist-only bridges, and custom helpers.
|
||||
- This makes upgrades brittle and blocks a clean external plugin surface.
|
||||
|
||||
## Target architecture (two layers)
|
||||
|
||||
### 1) Plugin SDK (compile-time, stable, publishable)
|
||||
|
||||
Scope: types, helpers, and config utilities. No runtime state, no side effects.
|
||||
|
||||
Contents (examples):
|
||||
|
||||
- Types: `ChannelPlugin`, adapters, `ChannelMeta`, `ChannelCapabilities`, `ChannelDirectoryEntry`.
|
||||
- Config helpers: `buildChannelConfigSchema`, `setAccountEnabledInConfigSection`, `deleteAccountFromConfigSection`,
|
||||
`applyAccountNameToChannelSection`.
|
||||
@@ -28,14 +32,17 @@ Contents (examples):
|
||||
- Docs link helper: `formatDocsLink`.
|
||||
|
||||
Delivery:
|
||||
|
||||
- Publish as `openclaw/plugin-sdk` (or export from core under `openclaw/plugin-sdk`).
|
||||
- Semver with explicit stability guarantees.
|
||||
|
||||
### 2) Plugin Runtime (execution surface, injected)
|
||||
|
||||
Scope: everything that touches core runtime behavior.
|
||||
Accessed via `OpenClawPluginApi.runtime` so plugins never import `src/**`.
|
||||
|
||||
Proposed surface (minimal but complete):
|
||||
|
||||
```ts
|
||||
export type PluginRuntime = {
|
||||
channel: {
|
||||
@@ -49,8 +56,11 @@ export type PluginRuntime = {
|
||||
ctx: unknown;
|
||||
cfg: unknown;
|
||||
dispatcherOptions: {
|
||||
deliver: (payload: { text?: string; mediaUrls?: string[]; mediaUrl?: string }) =>
|
||||
void | Promise<void>;
|
||||
deliver: (payload: {
|
||||
text?: string;
|
||||
mediaUrls?: string[];
|
||||
mediaUrl?: string;
|
||||
}) => void | Promise<void>;
|
||||
onError?: (err: unknown, info: { kind: string }) => void;
|
||||
};
|
||||
}): Promise<void>;
|
||||
@@ -87,7 +97,12 @@ export type PluginRuntime = {
|
||||
matchesMentionPatterns(text: string, regexes: RegExp[]): boolean;
|
||||
};
|
||||
groups: {
|
||||
resolveGroupPolicy(cfg: OpenClawConfig, channel: string, accountId: string, groupId: string): {
|
||||
resolveGroupPolicy(
|
||||
cfg: OpenClawConfig,
|
||||
channel: string,
|
||||
accountId: string,
|
||||
groupId: string,
|
||||
): {
|
||||
allowlistEnabled: boolean;
|
||||
allowed: boolean;
|
||||
groupConfig?: unknown;
|
||||
@@ -129,6 +144,7 @@ export type PluginRuntime = {
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- Runtime is the only way to access core behavior.
|
||||
- SDK is intentionally small and stable.
|
||||
- Each runtime method maps to an existing core implementation (no duplication).
|
||||
@@ -136,49 +152,59 @@ Notes:
|
||||
## Migration plan (phased, safe)
|
||||
|
||||
### Phase 0: scaffolding
|
||||
|
||||
- Introduce `openclaw/plugin-sdk`.
|
||||
- Add `api.runtime` to `OpenClawPluginApi` with the surface above.
|
||||
- Maintain existing imports during a transition window (deprecation warnings).
|
||||
|
||||
### Phase 1: bridge cleanup (low risk)
|
||||
|
||||
- Replace per-extension `core-bridge.ts` with `api.runtime`.
|
||||
- Migrate BlueBubbles, Zalo, Zalo Personal first (already close).
|
||||
- Remove duplicated bridge code.
|
||||
|
||||
### Phase 2: light direct-import plugins
|
||||
|
||||
- Migrate Matrix to SDK + runtime.
|
||||
- Validate onboarding, directory, group mention logic.
|
||||
|
||||
### Phase 3: heavy direct-import plugins
|
||||
|
||||
- Migrate MS Teams (largest set of runtime helpers).
|
||||
- Ensure reply/typing semantics match current behavior.
|
||||
|
||||
### Phase 4: iMessage pluginization
|
||||
|
||||
- Move iMessage into `extensions/imessage`.
|
||||
- Replace direct core calls with `api.runtime`.
|
||||
- Keep config keys, CLI behavior, and docs intact.
|
||||
|
||||
### Phase 5: enforcement
|
||||
|
||||
- Add lint rule / CI check: no `extensions/**` imports from `src/**`.
|
||||
- Add plugin SDK/version compatibility checks (runtime + SDK semver).
|
||||
|
||||
## Compatibility and versioning
|
||||
|
||||
- SDK: semver, published, documented changes.
|
||||
- Runtime: versioned per core release. Add `api.runtime.version`.
|
||||
- Plugins declare a required runtime range (e.g., `openclawRuntime: ">=2026.2.0"`).
|
||||
|
||||
## Testing strategy
|
||||
|
||||
- Adapter-level unit tests (runtime functions exercised with real core implementation).
|
||||
- Golden tests per plugin: ensure no behavior drift (routing, pairing, allowlist, mention gating).
|
||||
- A single end-to-end plugin sample used in CI (install + run + smoke).
|
||||
|
||||
## Open questions
|
||||
|
||||
- Where to host SDK types: separate package or core export?
|
||||
- Runtime type distribution: in SDK (types only) or in core?
|
||||
- How to expose docs links for bundled vs external plugins?
|
||||
- Do we allow limited direct core imports for in-repo plugins during transition?
|
||||
|
||||
## Success criteria
|
||||
|
||||
- All channel connectors are plugins using SDK + runtime.
|
||||
- No `extensions/**` imports from `src/**`.
|
||||
- New connector templates depend only on SDK + runtime.
|
||||
|
||||
@@ -5,19 +5,23 @@ read_when:
|
||||
- Working on config migrations or doctor workflows
|
||||
- Handling plugin config schemas or plugin load gating
|
||||
---
|
||||
|
||||
# Strict config validation (doctor-only migrations)
|
||||
|
||||
## Goals
|
||||
|
||||
- **Reject unknown config keys everywhere** (root + nested).
|
||||
- **Reject plugin config without a schema**; don’t load that plugin.
|
||||
- **Remove legacy auto-migration on load**; migrations run via doctor only.
|
||||
- **Auto-run doctor (dry-run) on startup**; if invalid, block non-diagnostic commands.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Backward compatibility on load (legacy keys do not auto-migrate).
|
||||
- Silent drops of unrecognized keys.
|
||||
|
||||
## Strict validation rules
|
||||
|
||||
- Config must match the schema exactly at every level.
|
||||
- Unknown keys are validation errors (no passthrough at root or nested).
|
||||
- `plugins.entries.<id>.config` must be validated by the plugin’s schema.
|
||||
@@ -26,11 +30,12 @@ read_when:
|
||||
- Plugin manifests (`openclaw.plugin.json`) are required for all plugins.
|
||||
|
||||
## Plugin schema enforcement
|
||||
|
||||
- Each plugin provides a strict JSON Schema for its config (inline in the manifest).
|
||||
- Plugin load flow:
|
||||
1) Resolve plugin manifest + schema (`openclaw.plugin.json`).
|
||||
2) Validate config against the schema.
|
||||
3) If missing schema or invalid config: block plugin load, record error.
|
||||
1. Resolve plugin manifest + schema (`openclaw.plugin.json`).
|
||||
2. Validate config against the schema.
|
||||
3. If missing schema or invalid config: block plugin load, record error.
|
||||
- Error message includes:
|
||||
- Plugin id
|
||||
- Reason (missing schema / invalid config)
|
||||
@@ -38,6 +43,7 @@ read_when:
|
||||
- Disabled plugins keep their config, but Doctor + logs surface a warning.
|
||||
|
||||
## Doctor flow
|
||||
|
||||
- Doctor runs **every time** config is loaded (dry-run by default).
|
||||
- If config invalid:
|
||||
- Print a summary + actionable errors.
|
||||
@@ -48,7 +54,9 @@ read_when:
|
||||
- Writes updated config.
|
||||
|
||||
## Command gating (when config is invalid)
|
||||
|
||||
Allowed (diagnostic-only):
|
||||
|
||||
- `openclaw doctor`
|
||||
- `openclaw logs`
|
||||
- `openclaw health`
|
||||
@@ -59,6 +67,7 @@ Allowed (diagnostic-only):
|
||||
Everything else must hard-fail with: “Config invalid. Run `openclaw doctor --fix`.”
|
||||
|
||||
## Error UX format
|
||||
|
||||
- Single summary header.
|
||||
- Grouped sections:
|
||||
- Unknown keys (full paths)
|
||||
@@ -66,6 +75,7 @@ Everything else must hard-fail with: “Config invalid. Run `openclaw doctor --f
|
||||
- Plugin load failures (plugin id + reason + path)
|
||||
|
||||
## Implementation touchpoints
|
||||
|
||||
- `src/config/zod-schema.ts`: remove root passthrough; strict objects everywhere.
|
||||
- `src/config/zod-schema.providers.ts`: ensure strict channel schemas.
|
||||
- `src/config/validation.ts`: fail on unknown keys; do not apply legacy migrations.
|
||||
@@ -75,6 +85,7 @@ Everything else must hard-fail with: “Config invalid. Run `openclaw doctor --f
|
||||
- CLI command gating in `src/cli`.
|
||||
|
||||
## Tests
|
||||
|
||||
- Unknown key rejection (root + nested).
|
||||
- Plugin missing schema → plugin load blocked with clear error.
|
||||
- Invalid config → gateway startup blocked except diagnostic commands.
|
||||
|
||||
Reference in New Issue
Block a user