Commit Graph

20 Commits

Author SHA1 Message Date
Glucksberg 9361e33b6d fix(openclaw): inject context via system prompt instead of overwriting MEMORY.md (#1386)
* fix(openclaw): inject context via system prompt instead of overwriting MEMORY.md

The OpenClaw plugin was overwriting each agent's MEMORY.md with a large
auto-generated observation dump (~12-15KB) on every before_agent_start
and tool_result_persist event. This conflicts with OpenClaw's design
where MEMORY.md is agent-curated long-term memory.

Migrate context injection from file-based (writeFile MEMORY.md) to
OpenClaw's native before_prompt_build hook, which returns context via
appendSystemContext. This keeps MEMORY.md under agent control while
still providing cross-session observation context to the LLM.

Changes:
- Add before_prompt_build hook that returns { appendSystemContext }
- Remove writeFile/MEMORY.md sync from before_agent_start
- Remove MEMORY.md sync from tool_result_persist (observations still recorded)
- Add 60s TTL cache to avoid re-fetching context on every LLM turn
- Add syncMemoryFileExclude config for per-agent opt-out
- Remove dead workspaceDirsBySessionKey tracking map
- Rewrite test suite to verify prompt injection instead of file writes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ui): align settings defaults with backend and use nullish coalescing

The web UI had two issues causing settings inflation:

1. DEFAULT_SETTINGS in the UI used FULL_COUNT='5' and all token columns
   'true', while SettingsDefaultsManager (backend) uses FULL_COUNT='0'
   and token columns 'false'. Opening the settings modal and saving
   without changes would silently inflate the context.

2. useSettings used || for fallback, which treats '0' and 'false' as
   falsy — even when the backend correctly returns these values, the UI
   would replace them with inflated defaults. Changed to ?? (nullish
   coalescing) so only null/undefined trigger the fallback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(openclaw): update integration docs for system prompt injection

Reflect the migration from MEMORY.md file writes to before_prompt_build
hook-based context injection:

- Update architecture diagram and overview to show new hook flow
- Replace "MEMORY.md Live Sync" section with "System Prompt Context Injection"
- Update event lifecycle steps (before_agent_start, tool_result_persist)
- Add before_prompt_build step with TTL cache description
- Document new syncMemoryFileExclude config parameter
- Update session tracking to reflect removed workspaceDirsBySessionKey

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix terminology and update SKILL.md for system prompt injection

Replace "prompt injection" with "context injection" in docs to avoid
confusion with the OWASP security term. Update openclaw/SKILL.md to
reflect the new before_prompt_build hook and remove stale MEMORY.md
references.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Alex Newman <thedotmack@gmail.com>
2026-03-17 17:14:30 -07:00
Alex Newman c6f932988a Fix 30+ root-cause bugs across 10 triage phases (#1214)
* MAESTRO: fix ChromaDB core issues — Python pinning, Windows paths, disable toggle, metadata sanitization, transport errors

- Add --python version pinning to uvx args in both local and remote mode (fixes #1196, #1206, #1208)
- Convert backslash paths to forward slashes for --data-dir on Windows (fixes #1199)
- Add CLAUDE_MEM_CHROMA_ENABLED setting for SQLite-only fallback mode (fixes #707)
- Sanitize metadata in addDocuments() to filter null/undefined/empty values (fixes #1183, #1188)
- Wrap callTool() in try/catch for transport errors with auto-reconnect (fixes #1162)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix data integrity — content-hash deduplication, project name collision, empty project guard, stuck isProcessing

- Add SHA-256 content-hash deduplication to observations INSERT (store.ts, transactions.ts, SessionStore.ts)
- Add content_hash column via migration 22 with backfill and index
- Fix project name collision: getCurrentProjectName() now returns parent/basename
- Guard against empty project string with cwd-derived fallback
- Fix stuck isProcessing: hasAnyPendingWork() resets processing messages older than 5 minutes
- Add 12 new tests covering all four fixes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix hook lifecycle — stderr suppression, output isolation, conversation pollution prevention

- Suppress process.stderr.write in hookCommand() to prevent Claude Code showing diagnostic
  output as error UI (#1181). Restores stderr in finally block for worker-continues case.
- Convert console.error() to logger.warn()/error() in hook-command.ts and handlers/index.ts
  so all diagnostics route to log file instead of stderr.
- Verified all 7 handlers return suppressOutput: true (prevents conversation pollution #598, #784).
- Verified session-complete is a recognized event type (fixes #984).
- Verified unknown event types return no-op handler with exit 0 (graceful degradation).
- Added 10 new tests in tests/hook-lifecycle.test.ts covering event dispatch, adapter defaults,
  stderr suppression, and standard response constants.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix worker lifecycle — restart loop coordination, stale transport retry, ENOENT shutdown race

- Add PID file mtime guard to prevent concurrent restart storms (#1145):
  isPidFileRecent() + touchPidFile() coordinate across sessions
- Add transparent retry in ChromaMcpManager.callTool() on transport
  error — reconnects and retries once instead of failing (#1131)
- Wrap getInstalledPluginVersion() with ENOENT/EBUSY handling (#1042)
- Verified ChromaMcpManager.stop() already called on all shutdown paths

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix Windows platform support — uvx.cmd spawn, PowerShell $_ elimination, windowsHide, FTS5 fallback

- Route uvx spawn through cmd.exe /c on Windows since MCP SDK lacks shell:true (#1190, #1192, #1199)
- Replace all PowerShell Where-Object {$_} pipelines with WQL -Filter server-side filtering (#1024, #1062)
- Add windowsHide: true to all exec/spawn calls missing it to prevent console popups (#1048)
- Add FTS5 runtime probe with graceful fallback when unavailable on Windows (#791)
- Guard FTS5 table creation in migrations, SessionSearch, and SessionStore with try/catch

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix skills/ distribution — build-time verification and regression tests (#1187)

Add post-build verification in build-hooks.js that fails if critical
distribution files (skills, hooks, plugin manifest) are missing. Add
10 regression tests covering skill file presence, YAML frontmatter,
hooks.json integrity, and package.json files field.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix MigrationRunner schema initialization (#979) — version conflict between parallel migration systems

Root cause: old DatabaseManager migrations 1-7 shared schema_versions table with
MigrationRunner's 4-22, causing version number collisions (5=drop tables vs add column,
6=FTS5 vs prompt tracking, 7=discovery_tokens vs remove UNIQUE).  initializeSchema()
was gated behind maxApplied===0, so core tables were never created when old versions
were present.

Fixes:
- initializeSchema() always creates core tables via CREATE TABLE IF NOT EXISTS
- Migrations 5-7 check actual DB state (columns/constraints) not just version tracking
- Crash-safe temp table rebuilds (DROP IF EXISTS _new before CREATE)
- Added missing migration 21 (ON UPDATE CASCADE) to MigrationRunner
- Added ON UPDATE CASCADE to FK definitions in initializeSchema()
- All changes applied to both runner.ts and SessionStore.ts

Tests: 13 new tests in migration-runner.test.ts covering fresh DB, idempotency,
version conflicts, crash recovery, FK constraints, and data integrity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix 21 test failures — stale mocks, outdated assertions, missing OpenClaw guards

Server tests (12): Added missing workerPath and getAiStatus to ServerOptions
mocks after interface expansion. ChromaSync tests (3): Updated to verify
transport cleanup in ChromaMcpManager after architecture refactor. OpenClaw (2):
Added memory_ tool skipping and response truncation to prevent recursive loops
and oversized payloads. MarkdownFormatter (2): Updated assertions to match
current output. SettingsDefaultsManager (1): Used correct default key for
getBool test. Logger standards (1): Excluded CLI transcript command from
background service check.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix Codex CLI compatibility (#744) — session_id fallbacks, unknown platform tolerance, undefined guard

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix Cursor IDE integration (#838, #1049) — adapter field fallbacks, tolerant session-init validation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix /api/logs OOM (#1203) — tail-read replaces full-file readFileSync

Replace readFileSync (loads entire file into memory) with readLastLines()
that reads only from the end of the file in expanding chunks (64KB → 10MB cap).
Prevents OOM on large log files while preserving the same API response shape.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix Settings CORS error (#1029) — explicit methods and allowedHeaders in CORS config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: add session custom_title for agent attribution (#1213) — migration 23, endpoint + store support

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: prevent CLAUDE.md/AGENTS.md writes inside .git/ directories (#1165)

Add .git path guard to all 4 write sites to prevent ref corruption when
paths resolve inside .git internals.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix plugin disabled state not respected (#781) — early exit check in all hook entry points

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix UserPromptSubmit context re-injection on every turn (#1079) — contextInjected session flag

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix stale AbortController queue stall (#1099) — lastGeneratorActivity tracking + 30s timeout

Three-layer fix:
1. Added lastGeneratorActivity timestamp to ActiveSession, updated by
   processAgentResponse (all agents), getMessageIterator (queue yields),
   and startGeneratorWithProvider (generator launch)
2. Added stale generator detection in ensureGeneratorRunning — if no
   activity for >30s, aborts stale controller, resets state, restarts
3. Added AbortSignal.timeout(30000) in deleteSession to prevent
   indefinite hang when awaiting a stuck generator promise

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 19:34:35 -05:00
Alex Newman ab2dbb7dc7 Rename Telegram bot commands from hyphens to underscores (#1126)
Telegram Bot API only allows a-z, 0-9, and underscores in command
names. Rename /claude-mem-feed → /claude_mem_feed and
/claude-mem-status → /claude_mem_status.

Fixes #1108

Co-authored-by: Manantra <113709296+Manantra@users.noreply.github.com>
2026-02-16 00:33:55 -05:00
Daniel M. be474ea595 Fix command handlers to return { text } per OpenClaw plugin API (#1115)
The OpenClaw plugin API requires command handlers to return
{ text: string } objects. Returning plain strings causes
"reply missing text/media" errors and commands silently fail
to send responses to Telegram/other channels.
2026-02-16 00:32:04 -05:00
Alex Newman cef15011c2 openclaw: add Claude-Mem search and timeline commands (#1069)
Co-authored-by: Alex Newman <alexnewman@Alexs-Mac-mini.local>
2026-02-16 00:30:08 -05:00
Glucksberg 55e0e323b9 feat: universalize observation feed emojis (#1100)
* feat: universalize observation feed emojis with config-driven system

Replace hardcoded AGENT_EMOJI_MAP with a three-tier approach:
1. User-pinned emojis via observationFeed.emojis.agents config
2. Deterministic auto-assign from pool using agentId hash
3. Configurable fallbacks for primary, Claude Code, and default emojis

Claude Code sessions now display "Claude Code Session" instead of the
working directory name. All emoji settings are exposed in the plugin
configSchema so the onboarding wizard AI can discover and configure them.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(feed): keep Claude Code project id in source labels

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 00:29:55 -05:00
Alex Newman 6cad77328b fix(openclaw): fix MEMORY.md project query mismatch and add feed botToken support
Three fixes for the OpenClaw plugin:

1. Fix MEMORY.md sync returning empty content
   - syncMemoryToWorkspace() was querying basename(workspaceDir) as the
     project name (e.g. 'workspace'), but observations are stored under
     agent-scoped names like 'openclaw-main'
   - Now queries both the base project and agent-scoped project name
   - Passes EventContext through so the correct project can be derived

2. Add dedicated botToken support for observation feed
   - New optional 'botToken' field in observationFeed config
   - When set, sends observations directly via Telegram Bot API instead
     of routing through the gateway's channel plugin
   - Allows using a separate bot for the observation stream

3. Fix plugin kind for memory slot compatibility
   - Changed plugin kind from 'integration' to 'memory' so OpenClaw
     recognizes it as a valid memory slot plugin
   - Fixes 'memory slot plugin not found' warning when
     plugins.slots.memory = 'claude-mem' is configured
2026-02-12 23:48:44 -05:00
Alex Newman c469e0acc3 fix: remove opinionated filters from OpenClaw plugin
Remove arbitrary TOOL_RESULT_MAX_LENGTH truncation, capture all content
blocks instead of only the first, observe all tool uses including
memory_ tools, and filter context injection by workspace directory
instead of hardcoded project name.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 22:14:16 -05:00
Alex Newman 6535ad597f fix: use event.prompt instead of ctx.sessionKey for prompt storage in OpenClaw plugin
The before_agent_start handler was passing ctx.sessionKey (e.g. "agent:main:main")
as the prompt to the worker API, causing the viewer to display the session key
instead of actual user prompt text. Now correctly reads event.prompt from
OpenClaw's BeforeAgentStartEvent.

Also adds message_received hook to capture inbound user prompts from messaging
channels (Telegram, Discord, etc.) and stores them via the worker API.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 16:17:32 -05:00
Glucksberg 63827c9dcb fix: type ObservationSSEPayload.project as nullable
The project field can be null/undefined for malformed SSE payloads.
Update the type and getSourceLabel signature to match the runtime
null guard.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 03:17:32 +00:00
Glucksberg 809175612c feat(openclaw): enable observation feed for OpenClaw agent sessions
Three fixes to make OpenClaw agent observations work end-to-end:

1. Session init in before_agent_start — the worker's privacy check
   requires a stored user prompt; without calling /api/sessions/init,
   all observations were skipped as "private"

2. Race condition fix in agent_end — await summarize before sending
   complete, preventing session deletion before in-flight observation
   POSTs arrive

3. OAuth token pass-through in buildIsolatedEnv — spawned Claude CLI
   processes now receive CLAUDE_CODE_OAUTH_TOKEN from the worker's
   env when no explicit API key is configured

Also adds agent-specific emoji mapping and dynamic project naming
for the Telegram observation feed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 02:30:18 +00:00
Alex Newman 05b615c858 Fix SSE stream URL consistency, multi-line data parsing, and test mocks
- Use workerBaseUrl() for SSE stream URL instead of hardcoded localhost
- Concatenate all SSE data: lines per frame per SSE spec
- Update WhatsApp mock to accept third options argument
- Restrict SSE mock server to only respond on /stream path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 21:54:02 -05:00
Alex Newman e13562e4cb Clean up session tracking on session_end to prevent unbounded map growth
gateway_start only fires on full process restart. Without cleanup,
sessionIds and workspaceDirsBySessionKey grow indefinitely across
/new and /reset cycles. session_end now deletes entries for the
completed session key.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 21:42:23 -05:00
Alex Newman c7f7f87321 Move session init to session_start and after_compaction hooks
Init was incorrectly placed in before_agent_start, which fires on every
agent attempt (retries, context overflow, auth rotation). Session init
should fire once on /new or /reset (session_start) and after compaction
(after_compaction). before_agent_start now only syncs MEMORY.md and
tracks workspace dirs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 21:04:52 -05:00
Alex Newman 11532a36fb Fix sendToChannel to use explicit OpenClaw SDK function mapping
Replace dynamic function name construction with CHANNEL_SEND_MAP that
matches the actual PluginRuntime.channel structure. Fixes WhatsApp
(sendMessageWhatsApp) and iMessage (sendMessageIMessage) casing, and
adds WhatsApp's required verbose option. Also adds null guard on SSE
observation payload before type casting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 20:14:41 -05:00
Alex Newman 7b7a92e35a MAESTRO: Add observation I/O, MEMORY.md live sync, and gateway lifecycle support
Merge crab-mem observation recording with existing SSE broadcasting to
create a complete OpenClaw plugin. Records observations from embedded
runner sessions via worker HTTP API, and continuously syncs MEMORY.md
to agent workspaces so agents always have fresh context.

- Add event handlers: before_agent_start, tool_result_persist, agent_end, gateway_start
- Add MEMORY.md live sync on every agent start and tool use (fire-and-forget)
- Add worker HTTP client (POST, fire-and-forget POST, GET text)
- Add /claude-mem-status health check command
- Add workspace dir tracking across session events
- Expand test suite from 17 to 36 tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 19:12:12 -05:00
Alex Newman 1b9f601c41 MAESTRO: Fix OpenClaw SDK API mismatch — use real PluginApi interface
E2E testing against the official OpenClaw Docker image revealed the plugin
was built against a custom interface that didn't match the real SDK:
- api.log() → api.logger.info/warn/error() (PluginLogger interface)
- api.getConfig() → api.pluginConfig (direct property)
- command handler (args[], ctx) → (ctx) with ctx.args string
- service stop optional, service context typed

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 21:28:08 -05:00
Alex Newman db207807cb MAESTRO: Address PR review feedback — fix connection lifecycle, lazy channel access, buffer safety
- Move sseAbortController/connectionState from module globals into closure for multi-instance safety
- Make start() idempotent by aborting existing connection before creating a new one
- Track connectionPromise and await it on stop() for proper cleanup
- Guard channel API access lazily to prevent crash when integrations are missing
- Add 1MB MAX_SSE_BUFFER_SIZE to prevent unbounded buffer growth
- Log malformed JSON parse errors instead of silently ignoring
- Replace error: any with proper instanceof Error type narrowing
- Remove hardcoded user paths from TESTING.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 20:06:29 -05:00
Alex Newman f8d8de53e8 MAESTRO: Implement SSE observation feed consumer with channel routing and exponential backoff
Replaces stub start/stop methods with working SSE consumer that connects to
claude-mem worker's /stream endpoint, parses new_observation events, and
forwards formatted messages to configured OpenClaw channels (Telegram, Discord,
Signal, Slack, WhatsApp, Line). Includes reconnection with exponential backoff
(1s-30s), connection state tracking, and on/off command toggle. Added 17 tests
covering unit and SSE integration scenarios.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 18:44:35 -05:00
Alex Newman baa37eba07 MAESTRO: Add OpenClaw plugin entry point with service and command registration
Creates openclaw/src/index.ts with:
- Inline OpenClawPluginApi interface definition
- registerService for claude-mem-observation-feed (stub start/stop for Phase 2)
- registerCommand for /claude-mem-feed status command
- Plugin initialization logging

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 18:37:49 -05:00