Commit Graph

337 Commits

Author SHA1 Message Date
Alex Newman 0e502dbd21 feat: add smart-explore AST-based code navigation (#1244)
* feat: add smart-file-read module for token-optimized semantic code search

- Created package.json for the smart-file-read module with dependencies and scripts.
- Implemented parser.ts for code structure parsing using tree-sitter, supporting multiple languages.
- Developed search.ts for searching code files and symbols with grep-style and structural matching.
- Added test-run.mjs for testing search and outline functionalities.
- Configured TypeScript with tsconfig.json for strict type checking and module resolution.

* fix: update .gitignore to include _tree-sitter and remove unused subproject

* feat: add preliminary results and skill recommendation for smart-explore module

* chore: remove outdated plan.md file detailing session start hook issues

* feat: update Smart File Read integration plan and skill documentation for smart-explore

* feat: migrate Smart File Read to web-tree-sitter WASM for cross-platform compatibility

* refactor: switch to tree-sitter CLI for parsing and enhance search functionality

- Updated `parser.ts` to utilize the tree-sitter CLI for AST extraction instead of native bindings, improving compatibility and performance.
- Removed grammar loading logic and replaced it with a path resolution for grammar packages.
- Implemented batch parsing in `parseFilesBatch` to handle multiple files in a single CLI call, enhancing search speed.
- Refactored `searchCodebase` to collect files and parse them in batches, streamlining the search process.
- Adjusted symbol extraction logic to accommodate the new parsing method and ensure accurate symbol matching.

* feat: update Smart File Read integration plan to utilize tree-sitter CLI for improved performance and cross-platform compatibility

* feat: add smart-file-read parser and search to src/services

Copy validated tree-sitter CLI-based parser and search modules from
smart-file-read prototype into the claude-mem source tree for MCP
tool integration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: register smart_search, smart_unfold, smart_outline MCP tools

Add 3 tree-sitter AST-based code exploration tools to the MCP server.
Direct execution (no HTTP delegation) — they call parser/search
functions directly for sub-second response times.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add tree-sitter CLI deps to build system and plugin runtime

Externalize tree-sitter packages in esbuild MCP server build. Add
10 grammar packages + CLI to plugin package.json for runtime install.
Remove unused @chroma-core/default-embed from plugin deps.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: create smart-explore skill with 3-layer workflow docs

Progressive disclosure workflow: search -> outline -> unfold.
Documents all 3 MCP tools with parameters and token economics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add comprehensive documentation for the smart-explore feature

- Introduced a detailed technical reference covering the architecture, parser, search engine, and tool registration for the smart-explore feature in claude-mem.
- Documented the three-layer workflow: search, outline, and unfold, along with their respective MCP tools.
- Explained the parsing process using tree-sitter, including language support, query patterns, and symbol extraction.
- Outlined the search module's functionality, including file discovery, batch parsing, and relevance scoring.
- Provided insights into build system integration and token economics for efficient code exploration.

* chore: remove experiment artifacts, prototypes, and plan files

Remove A/B test docs, prototype smart-file-read directory, and
implementation plans. Keep only production code.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: simplify hooks configuration and remove setup script

* fix: use execFileSync to prevent command injection in tree-sitter parser

Replaces execSync shell string with execFileSync + argument array,
eliminating shell interpretation of file paths. Also corrects
file_pattern description from "Glob pattern" to "Substring filter".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 21:00:26 -05:00
Alex Newman ad3d236cec fix: resolve hook crashes and CLAUDE_PLUGIN_ROOT fallback (#1215, #1220) (#1229)
* fix: resolve PostToolUse hook crashes and 5s latency (#1220)

Three compounding bugs caused hook failures:

1. Missing break statements in worker-service.ts switch — if async
   code threw before process.exit(), execution fell through to
   subsequent cases. Added break to all 7 cases missing them.

2. Unhandled promise rejection on main() — added .catch() that logs
   the error and exits 0 (per project exit code strategy: don't block
   Claude Code or leave Windows Terminal tabs open).

3. Redundant start commands in hooks.json — PostToolUse,
   UserPromptSubmit, and Stop groups each had a standalone start
   command that was redundant (the hook case already calls
   ensureWorkerStarted internally). The redundant start also caused
   5s latency via bun-runner.js collectStdin() timeout since Claude
   Code never closes stdin.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add CLAUDE_PLUGIN_ROOT fallback for Stop hooks (#1215)

Upstream Claude Code bug (anthropics/claude-code#24529) leaves
CLAUDE_PLUGIN_ROOT unset for Stop hooks on macOS and ALL hooks
on Linux. Two-layer defense:

1. Shell-level: hooks.json commands now use inline fallback
   _R="${CLAUDE_PLUGIN_ROOT}"; [ -z "$_R" ] && _R="$HOME/...";
   falling back to the known marketplace install path.

2. Script-level: bun-runner.js self-resolves plugin root from
   its own filesystem location via import.meta.url, and fixes
   broken /scripts/... paths that result from empty expansion.

Added test to verify all hook commands include the fallback path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 19:31:26 -05:00
Alex Newman c6f932988a Fix 30+ root-cause bugs across 10 triage phases (#1214)
* MAESTRO: fix ChromaDB core issues — Python pinning, Windows paths, disable toggle, metadata sanitization, transport errors

- Add --python version pinning to uvx args in both local and remote mode (fixes #1196, #1206, #1208)
- Convert backslash paths to forward slashes for --data-dir on Windows (fixes #1199)
- Add CLAUDE_MEM_CHROMA_ENABLED setting for SQLite-only fallback mode (fixes #707)
- Sanitize metadata in addDocuments() to filter null/undefined/empty values (fixes #1183, #1188)
- Wrap callTool() in try/catch for transport errors with auto-reconnect (fixes #1162)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix data integrity — content-hash deduplication, project name collision, empty project guard, stuck isProcessing

- Add SHA-256 content-hash deduplication to observations INSERT (store.ts, transactions.ts, SessionStore.ts)
- Add content_hash column via migration 22 with backfill and index
- Fix project name collision: getCurrentProjectName() now returns parent/basename
- Guard against empty project string with cwd-derived fallback
- Fix stuck isProcessing: hasAnyPendingWork() resets processing messages older than 5 minutes
- Add 12 new tests covering all four fixes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix hook lifecycle — stderr suppression, output isolation, conversation pollution prevention

- Suppress process.stderr.write in hookCommand() to prevent Claude Code showing diagnostic
  output as error UI (#1181). Restores stderr in finally block for worker-continues case.
- Convert console.error() to logger.warn()/error() in hook-command.ts and handlers/index.ts
  so all diagnostics route to log file instead of stderr.
- Verified all 7 handlers return suppressOutput: true (prevents conversation pollution #598, #784).
- Verified session-complete is a recognized event type (fixes #984).
- Verified unknown event types return no-op handler with exit 0 (graceful degradation).
- Added 10 new tests in tests/hook-lifecycle.test.ts covering event dispatch, adapter defaults,
  stderr suppression, and standard response constants.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix worker lifecycle — restart loop coordination, stale transport retry, ENOENT shutdown race

- Add PID file mtime guard to prevent concurrent restart storms (#1145):
  isPidFileRecent() + touchPidFile() coordinate across sessions
- Add transparent retry in ChromaMcpManager.callTool() on transport
  error — reconnects and retries once instead of failing (#1131)
- Wrap getInstalledPluginVersion() with ENOENT/EBUSY handling (#1042)
- Verified ChromaMcpManager.stop() already called on all shutdown paths

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix Windows platform support — uvx.cmd spawn, PowerShell $_ elimination, windowsHide, FTS5 fallback

- Route uvx spawn through cmd.exe /c on Windows since MCP SDK lacks shell:true (#1190, #1192, #1199)
- Replace all PowerShell Where-Object {$_} pipelines with WQL -Filter server-side filtering (#1024, #1062)
- Add windowsHide: true to all exec/spawn calls missing it to prevent console popups (#1048)
- Add FTS5 runtime probe with graceful fallback when unavailable on Windows (#791)
- Guard FTS5 table creation in migrations, SessionSearch, and SessionStore with try/catch

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix skills/ distribution — build-time verification and regression tests (#1187)

Add post-build verification in build-hooks.js that fails if critical
distribution files (skills, hooks, plugin manifest) are missing. Add
10 regression tests covering skill file presence, YAML frontmatter,
hooks.json integrity, and package.json files field.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix MigrationRunner schema initialization (#979) — version conflict between parallel migration systems

Root cause: old DatabaseManager migrations 1-7 shared schema_versions table with
MigrationRunner's 4-22, causing version number collisions (5=drop tables vs add column,
6=FTS5 vs prompt tracking, 7=discovery_tokens vs remove UNIQUE).  initializeSchema()
was gated behind maxApplied===0, so core tables were never created when old versions
were present.

Fixes:
- initializeSchema() always creates core tables via CREATE TABLE IF NOT EXISTS
- Migrations 5-7 check actual DB state (columns/constraints) not just version tracking
- Crash-safe temp table rebuilds (DROP IF EXISTS _new before CREATE)
- Added missing migration 21 (ON UPDATE CASCADE) to MigrationRunner
- Added ON UPDATE CASCADE to FK definitions in initializeSchema()
- All changes applied to both runner.ts and SessionStore.ts

Tests: 13 new tests in migration-runner.test.ts covering fresh DB, idempotency,
version conflicts, crash recovery, FK constraints, and data integrity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix 21 test failures — stale mocks, outdated assertions, missing OpenClaw guards

Server tests (12): Added missing workerPath and getAiStatus to ServerOptions
mocks after interface expansion. ChromaSync tests (3): Updated to verify
transport cleanup in ChromaMcpManager after architecture refactor. OpenClaw (2):
Added memory_ tool skipping and response truncation to prevent recursive loops
and oversized payloads. MarkdownFormatter (2): Updated assertions to match
current output. SettingsDefaultsManager (1): Used correct default key for
getBool test. Logger standards (1): Excluded CLI transcript command from
background service check.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix Codex CLI compatibility (#744) — session_id fallbacks, unknown platform tolerance, undefined guard

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix Cursor IDE integration (#838, #1049) — adapter field fallbacks, tolerant session-init validation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix /api/logs OOM (#1203) — tail-read replaces full-file readFileSync

Replace readFileSync (loads entire file into memory) with readLastLines()
that reads only from the end of the file in expanding chunks (64KB → 10MB cap).
Prevents OOM on large log files while preserving the same API response shape.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix Settings CORS error (#1029) — explicit methods and allowedHeaders in CORS config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: add session custom_title for agent attribution (#1213) — migration 23, endpoint + store support

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: prevent CLAUDE.md/AGENTS.md writes inside .git/ directories (#1165)

Add .git path guard to all 4 write sites to prevent ref corruption when
paths resolve inside .git internals.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix plugin disabled state not respected (#781) — early exit check in all hook entry points

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix UserPromptSubmit context re-injection on every turn (#1079) — contextInjected session flag

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix stale AbortController queue stall (#1099) — lastGeneratorActivity tracking + 30s timeout

Three-layer fix:
1. Added lastGeneratorActivity timestamp to ActiveSession, updated by
   processAgentResponse (all agents), getMessageIterator (queue yields),
   and startGeneratorWithProvider (generator launch)
2. Added stale generator detection in ensureGeneratorRunning — if no
   activity for >30s, aborts stale controller, resets state, restarts
3. Added AbortSignal.timeout(30000) in deleteSession to prevent
   indefinite hang when awaiting a stuck generator promise

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 19:34:35 -05:00
Alex Newman 5f28550551 MAESTRO: fix MCP type coercion for batch endpoints, add defensive observation error handling
Add string-to-array coercion for ids and memorySessionIds in DataRoutes.ts
batch endpoints so MCP clients sending "[1,2,3]" or "1,2,3" instead of
native arrays no longer get 400 errors. Wrap observation storage path in
SessionRoutes.ts with try/catch returning 200 on recoverable errors instead
of 500, preventing hook breakage.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 16:41:28 -05:00
Alex Newman 0a26bb18bf fix: update footer text to reference claude-mem skill instead of MCP search tools
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 03:47:02 -05:00
Alex Newman 7966c6cba9 fix: rename save_memory and fix MCP search instructions + startup hook (#1210)
* fix: rename save_memory to save_observation and fix MCP search instructions

Stop the primary agent from proactively saving memories by renaming
save_memory to save_observation with a neutral description. Remove
"Saving Memories" section from SKILL.md. Update context formatters
and output styles to reference the mem-search skill instead of raw
MCP tool names.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: split SessionStart hooks so smart-install failure doesn't block worker start

smart-install.js and worker-start were in the same hook group, so if
smart-install exited non-zero the worker never started. Split into
separate hook groups so they run independently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: worker startup waits for readiness before hooks fire

Move initializationCompleteFlag to set after DB/search init (not MCP),
add waitForReadiness() polling /api/readiness, and extract shared
pollEndpointUntilOk helper to DRY up health/readiness checks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 03:30:31 -05:00
Alex Newman e788fd3676 fix: prevent duplicate worker daemons and zombie processes (#1178)
* fix: prevent duplicate worker daemons and zombie processes

Three root causes of chroma-mcp timeouts:

1. HTTP shutdown (POST /api/admin/shutdown) closed resources but never
   called process.exit(). Zombie workers stayed alive, background tasks
   reconnected to chroma-mcp, spawning duplicate subprocesses that all
   contended for the same persistent data directory.

2. No guard against concurrent daemon startup. When hooks fired
   simultaneously, multiple daemons started before either wrote a PID
   file. The loser got EADDRINUSE but stayed alive because signal
   handlers registered in the constructor prevented exit.

3. Corrupt 147GB HNSW index file caused all chroma queries to timeout
   (MCP error -32001). Data fix: deleted corrupt collection, backfill
   rebuilds from SQLite.

Code fixes:
- Add PID-based guard in daemon startup: exit if PID file process alive
- Add port-based guard in daemon startup: exit if port already bound
  (runs before WorkerService constructor registers keepalive handlers)
- Add process.exit(0) after HTTP shutdown/restart completes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: aggressive startup cleanup and one-time chroma wipe for upgrade

Kill orphaned worker-service.cjs and chroma-mcp processes immediately
at startup (no age gate) while keeping 30-min threshold for mcp-server.
Wipe corrupt chroma data once on upgrade from pre-v10.3 versions —
backfill rebuilds from SQLite automatically.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: wrap shutdown handlers in try/finally to guarantee process.exit

If onShutdown() or onRestart() threw, process.exit(0) was never reached,
leaving the daemon alive as a zombie. Also removed redundant require('fs')
calls in process-manager tests where ESM imports already existed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 20:10:28 -05:00
Alex Newman 40daf8f3fa feat: replace WASM embeddings with persistent chroma-mcp MCP connection (#1176)
* feat: replace WASM embeddings with persistent chroma-mcp MCP connection

Replace ChromaServerManager (npx chroma run + chromadb npm + ONNX/WASM)
with ChromaMcpManager, a singleton stdio MCP client that communicates with
chroma-mcp via uvx. This eliminates native binary issues, segfaults, and
WASM embedding failures that plagued cross-platform installs.

Key changes:
- Add ChromaMcpManager: singleton MCP client with lazy connect, auto-reconnect,
  connection lock, and Zscaler SSL cert support
- Rewrite ChromaSync to use MCP tool calls instead of chromadb npm client
- Handle chroma-mcp's non-JSON responses (plain text success/error messages)
- Treat "collection already exists" as idempotent success
- Wire ChromaMcpManager into GracefulShutdown for clean subprocess teardown
- Delete ChromaServerManager (no longer needed)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review — connection guard leak, timer leak, async reset

- Clear connecting guard in finally block to prevent permanent reconnection block
- Clear timeout after successful connection to prevent timer leak
- Make reset() async to await stop() before nullifying instance
- Delete obsolete chroma-server-manager test (imports deleted class)
- Update graceful-shutdown test to use chromaMcpManager property name

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: prevent chroma-mcp spawn storm — zombie cleanup, stale onclose guard, reconnect backoff

Three bugs caused chroma-mcp processes to accumulate (92+ observed):

1. Zombie on timeout: failed connections left subprocess alive because
   only the timer was cleared, not the transport. Now catch block
   explicitly closes transport+client before rethrowing.

2. Stale onclose race: old transport's onclose handler captured `this`
   and overwrote the current connection reference after reconnect,
   orphaning the new subprocess. Now guarded with reference check.

3. No backoff: every failure triggered immediate reconnect. With
   backfill doing hundreds of MCP calls, this created rapid-fire
   spawning. Added 10s backoff on both connection failure and
   unexpected process death.

Also includes ChromaSync fixes from PR review:
- queryChroma deduplication now preserves index-aligned arrays
- SQL injection guard on backfill ID exclusion lists

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 18:32:38 -05:00
Alex Newman 5d79bb7a7a fix: prevent zombie process accumulation by verifying subprocess exit (#1168) (#1175)
Two changes fix the observer process resource leak:

1. Add ensureProcessExit to generator finally blocks in SessionRoutes and
   worker-service, matching the pattern already working in SDKAgent.

2. Add stale session reaper (every 2m) that removes sessions with no active
   generator and no pending work after 15m idle. This unblocks the orphan
   reaper which previously skipped processes for "active" sessions.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 16:33:23 -05:00
Alex Newman b88251bc8b fix: self-healing claimNextMessage prevents stuck processing messages (#1159)
* fix: self-healing claimNextMessage prevents stuck processing messages

claimAndDelete → claimNextMessage with atomic self-healing: resets stale
processing messages (>60s) back to pending before claiming. Eliminates
stuck messages from generator crashes without external timers. Removes
redundant idle-timeout reset in worker-service.ts. Adds QUEUE to logger
Component type.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update stale comments in SessionQueueProcessor to reflect claim-confirm pattern

Comments still referenced the old claim-and-delete pattern after the
claimNextMessage rename. Updated to accurately describe the current
lifecycle where messages are marked as processing and stay in DB until
confirmProcessed() is called.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: move Date.now() inside transaction and extract stale threshold constant

- Move Date.now() inside claimNextMessage transaction closure so timestamp
  is fresh if WAL contention causes retry
- Extract STALE_PROCESSING_THRESHOLD_MS to module-level constant
- Add comment clarifying strict < boundary semantics

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 23:15:46 -05:00
Alex Newman ca8421611c fix: backfill Chroma vector DB for all projects on startup (#1154)
* fix: backfill all Chroma projects on worker startup

ChromaSync.ensureBackfilled() existed but was never called. After
v10.2.2's bun cache clear destroyed the ONNX model cache, Chroma only
had ~2 days of embeddings while SQLite had 49k+ observations.

- Add static backfillAllProjects() to ChromaSync — iterates all projects
  in SQLite, creates temporary ChromaSync per project, runs smart diff
- Call backfillAllProjects() fire-and-forget on worker startup
- Add 'CHROMA_SYNC' to logger Component type (pre-existing gap)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: sanitize project names for Chroma collection naming

Replace characters outside [a-zA-Z0-9._-] with underscores so projects
like "YC Stuff" map to collection "cm__YC_Stuff" instead of failing
Chroma's collection name validation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: route backfill to shared cm__claude-mem collection, harden sanitization

- Use single ChromaSync('claude-mem') in backfillAllProjects() instead of
  per-project instances, matching how DatabaseManager and SearchManager
  operate — fixes critical bug where backfilled data landed in orphaned
  collections that no search path reads from
- Strip trailing non-alphanumeric chars from sanitized collection names
  to satisfy Chroma's end-character constraint
- Guard backfill behind Chroma server readiness to avoid N spurious error
  logs when Chroma failed to start
- Use CHROMA_SYNC log component consistently for backfill messages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: pass project as parameter to ensureBackfilled instead of mutating instance state

Eliminates shared mutable state in backfillAllProjects() loop. Project
scoping is now passed explicitly via parameter to both ensureBackfilled()
and getExistingChromaIds(), keeping a single Chroma connection while
avoiding fragile instance property mutation across iterations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 22:47:46 -05:00
Alex Newman 224567f980 fix: prevent ONNX model cache corruption from bun cache clears
Remove nuclear `bun pm cache rm` from smart-install.js and
sync-marketplace.cjs (only needed for removed sharp dependency).
Add `bun install` in cache version directory after sync so worker
can resolve dependencies. Move HuggingFace model cache to
~/.claude-mem/models/ so reinstalls don't corrupt it. Add self-healing
retry for Protobuf parsing failures.

Fixes recurring issues #1104, #1105, #1110.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 19:54:17 -05:00
Alex Newman f24251118e fix: bun install, node-addon-api for sharp, consolidate PendingMessageStore (#1140)
* fix: use bun install in sync, add node-addon-api for sharp, consolidate PendingMessageStore

- Switch sync-marketplace from npm to bun install
- Add node-addon-api as dev dep so sharp builds under bun
- Consolidate duplicate PendingMessageStore instantiation in worker-service finally block

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* build assets

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 18:05:42 -05:00
Alex Newman d2e926fbf7 fix: post-merge breakage (Gemini, idle timeout, sharp cache) (#1138)
* fix: add gemini-3-flash to validModels array

The model was defined in the type union and RPM limits but missing from
the runtime validModels array, causing silent fallback to gemini-2.5-flash.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: skip processing when Gemini returns empty observation response

Empty responses were silently consuming messages from the queue via
processAgentResponse. Now skips processing on empty content, leaving
the message in processing status for stale recovery.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: prevent idle timeout from triggering infinite restart loop

When a session hits the 3-minute idle timeout, the finally block was
seeing stale processing messages and restarting the generator endlessly.
Now tracks idle timeout as a distinct exit reason via session flag,
resets stale messages, and skips restart.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: clear stale Bun native module cache on update

Bun's global cache retains sharp/libvips native binaries with broken
dylib references after version upgrades. Clear ~/.bun/install/cache/@img/
before install in both the end-user (smart-install) and dev (sync-marketplace)
paths to prevent ERR_DLOPEN_FAILED errors in Chroma sync.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback (empty summary response, session-scoped reset, shell injection)

- Apply same empty-response guard to summary path as observation path in GeminiAgent
- Add optional sessionDbId param to resetStaleProcessingMessages for session-scoped resets
- Use JSON.stringify for gitignore pattern escaping, filter negation patterns

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:46:30 -05:00
michelhelsdingen 51719d23a4 feat: configurable subprocess pool limit for SDK agents (#995)
* feat: configurable subprocess pool limit for SDK agents

Prevents runaway accumulation of Claude SDK agent subprocesses by
enforcing a configurable concurrency limit.

- New CLAUDE_MEM_MAX_CONCURRENT_AGENTS setting (default: 2)
- Promise-based waitForSlot() in ProcessRegistry (not polling per
  review feedback on #830)
- Waiters are notified via unregisterProcess when a slot frees up
- SDKAgent.startSession() waits for a slot before spawning
- 60s timeout prevents indefinite waits

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>

* fix: remove unused originalUnregister const and getActiveCount import

Cleanup from Greptile review:
- Remove dead `originalUnregister` variable in ProcessRegistry
- Remove unused `getActiveCount` import in SDKAgent

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Happy <yesreply@happy.engineering>
2026-02-16 00:31:17 -05:00
ixnaswang 81013e1310 fix: SDK Agent fails on Windows when username contains spaces (#1022)
* fix: SDK Agent fails on Windows when username contains spaces

Fixes spawn failure on Windows when the user's path contains spaces
(e.g., C:\Users\Anderson Wang\).

Root cause:
- SDKAgent.ts returns full auto-detected path with spaces
- ProcessRegistry.ts cannot execute .cmd files when path contains spaces

Solution:
- SDKAgent: On Windows, prefer "claude.cmd" via PATH instead of full path
- ProcessRegistry: Use cmd.exe /d /c wrapper for .cmd files on Windows

This preserves argument boundaries (e.g., empty string values) while
properly handling paths with spaces.

Fixes #1014

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* docs: add Windows spawn path with spaces fix documentation

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-16 00:31:11 -05:00
Mark L 6d1f17adee fix: use Bun runtime for Windows daemon spawn (#1086)
Co-authored-by: root <root@localhost.localdomain>
2026-02-16 00:31:04 -05:00
TerrifiedBug 0a40c4c596 fix: include project in ChromaDB where clause for vector search (#1112)
When searching with a project parameter, the ChromaDB vector query was
not filtering by project. It only filtered by doc_type. This caused
larger projects to dominate the top-N results returned by ChromaDB,
effectively crowding out results from smaller projects before the
post-hoc SQLite project filter could take effect.

For example, with project A having 19,000 embeddings and project B
having 700, a search scoped to project B would return mostly project A
results from ChromaDB. After SQLite filtered by project, only 1-3
results from B would survive instead of the expected 20+.

The fix adds the project to the ChromaDB where clause using $and when
both doc_type and project filters are needed. This is applied in both
ChromaSearchStrategy.buildWhereFilter() and SearchManager.search().

Co-authored-by: TARS <tars@openclaw.local>
2026-02-16 00:30:29 -05:00
Kamran Khalid 02f7c3c9d0 fix(security): validate and restrict /api/instructions operation and topic params (CWE-22, CWE-1321) (#986) 2026-02-16 00:29:08 -05:00
Mark L a94ddc504f fix(cursor): remove obsolete cursor-hooks directory gate (#1087)
Co-authored-by: root <root@localhost.localdomain>
2026-02-16 00:26:37 -05:00
zhaixingzi 454e9c5870 fix: resolve duplicate assistant messages in OpenRouter agent (#1074)
This commit addresses the issue of duplicate assistant messages appearing
in the conversation history by commenting out the lines that were
unnecessarily pushing assistant responses to the conversationHistory array.

The processAgentResponse function already handles adding assistant messages
to the conversation history, so these additional pushes were causing
duplicate entries.

Changes made:
- Commented out session.conversationHistory.push calls for assistant responses
  in three locations within OpenRouterAgent.ts:
  1. In the init response handling (around line 117)
  2. In the observation response handling (around line 188)
  3. In the summary response handling (around line 230)

This ensures that assistant messages are only added once to the conversation
history, preventing duplication while maintaining the intended functionality.

Co-authored-by: 张坤 <zhangkun@example.com>
2026-02-16 00:26:07 -05:00
SaneApps 2f337dab13 fix: use Gemini v1 API endpoint instead of v1beta (#1082)
v1beta does not support newer models like gemini-3-flash, causing
silent 404 errors that back up the observation queue indefinitely.
Users with CLAUDE_MEM_GEMINI_MODEL=gemini-3-flash get zero observations
stored, with no visible error — the queue just grows silently.

Changes:
- Switch API URL from v1beta/models to v1/models (generateContent
  works identically on both endpoints)
- Add gemini-3-flash to GeminiModel type and RPM limits
- Update test to match new endpoint

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 00:26:01 -05:00
Alex Newman 055888e181 fix: address PR review feedback for subprocess cleanup and binary resolution
Wrap SDK query loop in try/finally so subprocess cleanup runs on error paths.
Swap Chroma binary check order to try project-level .bin first (common case).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 23:24:00 -05:00
Alex Newman 67ba17cc8a fix: use WASM backend for Chroma embeddings to fix cross-platform issues
Chroma requires client-side embeddings — the server is storage only.
The previous commit incorrectly removed @chroma-core/default-embed.

Uses DefaultEmbeddingFunction({ wasm: true }) which forces the WASM
backend instead of native ONNX binaries. Same model (all-MiniLM-L6-v2),
same embeddings, but works on all platforms without segfaults or
ENOENT errors (#1104, #1105, #1110).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 23:14:21 -05:00
Alex Newman e1ef14dbcc fix: resolve orphaned subprocesses and Chroma HTTP regressions
- Add subprocess cleanup after SDK query loop completes, using existing
  ProcessRegistry infrastructure (getProcessBySession + ensureProcessExit)
- Replace npx-based Chroma binary spawning with absolute path resolution
  via require.resolve, falling back to npx with explicit cwd (#1120)
- Remove @chroma-core/default-embed client-side dependency; let Chroma
  HTTP server handle embeddings server-side (#1104, #1105, #1110)

Closes #1010, #1089, #1090, #1068, #1120, #1104, #1105, #1110

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 22:04:52 -05:00
Alex Newman c27314f896 fix: address PR review comments for chroma server lifecycle 2026-02-13 23:39:30 -05:00
Alex Newman 1b68c55763 fix: resolve SDK spawn failures and sharp native binary crashes
- Strip CLAUDECODE env var from SDK subprocesses to prevent "cannot be
  launched inside another Claude Code session" error (Claude Code 2.1.42+)
- Lazy-load @chroma-core/default-embed to avoid eagerly pulling in
  sharp native binaries at bundle startup (fixes ERR_DLOPEN_FAILED)
- Add stderr capture to SDK spawn for diagnosing future process failures
- Exclude lockfiles from marketplace rsync and delete stale lockfiles
  before npm install to prevent native dep version mismatches

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 22:47:27 -05:00
Alex Newman ed313db742 Merge main into feat/chroma-http-server
Resolve conflicts between Chroma HTTP server PR and main branch changes
(folder CLAUDE.md, exclusion settings, Zscaler SSL, transport cleanup).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 21:02:54 -05:00
Alex Newman 5de728612e chore: bump version to 10.0.6
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 00:02:37 -05:00
Alex Newman f05f9ca735 Merge remote-tracking branch 'origin/main' into openclaw-installer
# Conflicts:
#	plugin/scripts/mcp-server.cjs
#	plugin/scripts/worker-service.cjs
2026-02-12 22:04:03 -05:00
Alex Newman 05e904e613 feat: enhance /api/health with version, uptime, workerPath, and AI status
Replace hardcoded TEST-008 build ID with real package version. Add worker
filesystem path, uptime counter, and AI provider status (including last
interaction success/failure tracking) to the health endpoint response.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 21:16:22 -05:00
Alex Newman 98d87d7573 chore: bump version to 10.0.4
Reverts v10.0.3 chroma-mcp spawn storm fix (broken release).
Restores codebase to v10.0.2 state.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 21:36:34 -05:00
Alex Newman 3f01baebfe Merge remote-tracking branch 'origin/main' into fix/chroma-mcp-spawn-storm
# Conflicts:
#	src/services/worker-service.ts
#	tests/infrastructure/process-manager.test.ts
2026-02-11 15:43:08 -05:00
Alex Newman 0b214a59a1 chore: bump version to 10.0.2
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 15:25:50 -05:00
Rod Boev 2c5c99c0c7 fix: use etime-based sorting instead of PID ordering for process guards
Addresses Greptile review feedback:
- ChromaSync: replace PID-based sort with ps etime column + parseElapsedTime()
  for reliable age ordering (PIDs wrap and don't guarantee ordering)
- ProcessManager: filter out entries with unparseable etime (-1) before
  sorting to prevent sort corruption in cleanupExcessChromaProcesses()
2026-02-11 07:19:28 -05:00
Rod Boev a3f9e7f638 fix: prevent chroma-mcp spawn storm with 5-layer defense (641 processes → max 2)
During SIGHUP testing with 6+ active sessions, ChromaSync.ensureConnection()
had no mutex — concurrent fire-and-forget syncObservation() calls each spawned
a chroma-mcp subprocess via StdioClientTransport, creating 641 orphans in ~5min.
Error-driven reconnection formed a positive feedback loop amplifying the storm.

Defense layers:
- Layer 0: Connection mutex via promise memoization (prevents concurrent spawns)
- Layer 1: Pre-spawn process count guard using execFileSync('ps') (kills excess)
- Layer 2: Hardened close() with try-finally + Unix pkill in GracefulShutdown
- Layer 3: Count-based orphan reaper in ProcessManager (not age-based)
- Layer 4: Circuit breaker stops retries after 3 consecutive failures for 60s

Closes #1063, closes #695
Relates to #1010, #707
2026-02-11 07:19:28 -05:00
Rod Boev 4e67393d27 fix: prevent daemon silent death from SIGHUP + unhandled errors
Root cause: registerSignalHandlers() handled SIGTERM/SIGINT but not
SIGHUP. When the parent hook process exits, the kernel sends SIGHUP
to the daemon, causing immediate termination (default signal action).

Belt-and-suspenders fix:
1. SIGHUP handler: ignore in daemon mode, graceful shutdown otherwise
2. setsid: spawn daemon in new session on Linux (prevents SIGHUP delivery)
3. Global unhandledRejection/uncaughtException guards in daemon mode
2026-02-11 00:35:53 -05:00
Alex Newman af95461a70 Merge branch 'main' into fix/hook-resilience-worker-lifecycle
# Conflicts:
#	plugin/scripts/mcp-server.cjs
#	plugin/scripts/worker-service.cjs
2026-02-10 23:37:33 -05:00
Rod Boev 418e38ee46 fix: hook resilience and worker lifecycle improvements (#957, #923, #984, #987, #1042)
Reduce timeouts to eliminate 10-30s startup delay when worker is dead
(common on WSL2 after hibernate). Add stale PID detection, graceful
error handling across all handlers, and error classification that
distinguishes worker unavailability from handler bugs.

- HEALTH_CHECK 30s→3s, new POST_SPAWN_WAIT (5s), PORT_IN_USE_WAIT (3s)
- isProcessAlive() with EPERM handling, cleanStalePidFile()
- getPluginVersion() try-catch for shutdown race (#1042)
- isWorkerUnavailableError: transport+5xx+429→exit 0, 4xx→exit 2
- No-op handler for unknown event types (#984)
- Wrap all handler fetch calls in try-catch for graceful degradation
- CLAUDE_MEM_HEALTH_TIMEOUT_MS env var override with validation
2026-02-10 15:34:35 -05:00
xingyu e4e1d3fb92 fix: Windows platform improvements — re-enable Chroma, fix DB race, simplify env isolation
1. ProcessManager: Migrate spawnDaemon() from WMIC to PowerShell Start-Process
   - WMIC deprecated in Windows 11, PowerShell inherits env vars properly
   - Use -WindowStyle Hidden to prevent console popups
   - Fix redundant backslash escaping in PowerShell $_ variables

2. ChromaSync: Re-enable vector search on Windows
   - Remove overly defensive platform check that disabled all semantic search
   - Worker daemon starts with -WindowStyle Hidden; child processes inherit
   - MCP SDK's StdioClientTransport uses shell:false, no new console created

3. worker-service: Unified DB-ready gate middleware
   - Replace single-endpoint /api/sessions/init wait with global middleware
   - Hold all DB-dependent requests until database is initialized (30s timeout)
   - Whitelist static assets, /health, and viewer page for immediate response
   - Separate dbReadyPromise (DB only) from initializationComplete (full init)
   - Fixes "Database not initialized" errors on /stream, /summarize, /init

4. EnvManager: Switch from allowlist to blocklist for subprocess env
   - Only strip ANTHROPIC_API_KEY to prevent Issue #733 billing hijack
   - Pass through all other vars (ANTHROPIC_AUTH_TOKEN, ANTHROPIC_BASE_URL, etc.)
   - Simpler, less fragile than maintaining an exhaustive system vars allowlist
2026-02-07 18:30:57 +08:00
Alex Newman 5969d670d0 chore: bump version to 9.1.1
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 02:18:44 -05:00
Alex Newman 8dfcb5e612 chore: bump version to 9.1.0
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 01:05:38 -05:00
Alex Newman ff503d08a7 MAESTRO: Merge PR #657 - Add generate/clean CLI commands for CLAUDE.md management
Cherry-picked source changes from PR #657 (224 commits behind main).
Adds `claude-mem generate` and `claude-mem clean` CLI commands:
- New src/cli/claude-md-commands.ts with generateClaudeMd() and cleanClaudeMd()
- Worker service generate/clean case handlers with --dry-run support
- CLAUDE_MD logger component type
- Uses shared isDirectChild from path-utils.ts (DRY improvement over PR original)

Skipped from PR: 91 CLAUDE.md file deletions (stale), build artifacts,
.claude/plans/ dev artifact, smart-install.js shell alias auto-injection
(aggressive profile modification without consent).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 05:52:54 -05:00
Alex Newman 98920bd860 MAESTRO: Merge PR #662 - Add save_memory MCP tool for manual memory storage
Adds save_memory MCP tool allowing users to manually save observations
for semantic search. Source changes cherry-picked from PR #662 by
@darconada (build artifact conflicts resolved by direct application).

Closes #645.

Co-Authored-By: darconadalabarga <darconada@arsys.es>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 04:13:44 -05:00
Alex Newman 5dffb1ebb0 MAESTRO: fix(hooks): add session-complete handler to enable orphan reaper cleanup
Cherry-picked from PR #844 by @thusdigital. Sessions stayed in active
sessions map forever after summarize, causing the orphan reaper to think
all processes were still active. Adds session-complete as Stop phase 2
hook that calls POST /api/sessions/complete to remove sessions from the
active map, allowing the reaper to correctly identify and clean up
orphaned worker processes. Fixes #842.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 03:23:13 -05:00
Alex Newman da1d2cd36a MAESTRO: fix(db): prevent FK constraint failures on worker restart
Cherry-picked source changes from PR #889 by @Et9797. Fixes #846.

Key changes:
- Add ensureMemorySessionIdRegistered() guard in SessionStore.ts
- Add ON UPDATE CASCADE migration (schema v21) for observations and session_summaries FK constraints
- Change message queue from claim-and-delete to claim-confirm pattern (PendingMessageStore.ts)
- Add spawn deduplication and unrecoverable error detection in SessionRoutes.ts and worker-service.ts
- Add forceInit flag to SDKAgent for stale session recovery

Build artifacts skipped (pre-existing dompurify dep issue). Path fixes (HealthMonitor.ts, worker-utils.ts)
already merged via PR #634.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 03:16:17 -05:00
Alex Newman 8030c44af4 MAESTRO: fix(cursor): use bun runtime and fix hooks directory detection
Cherry-picked source changes from PR #721. Fixes two Cursor standalone
setup bugs:

1. findCursorHooksDir() now checks for hooks.json (unified CLI mode)
   in addition to legacy common.sh/common.ps1 scripts
2. installCursorHooks() now uses bun instead of node for hook commands
   since worker-service.cjs depends on bun:sqlite
3. Added findBunPath() to detect bun executable across platforms

Build artifacts skipped (pre-existing dompurify viewer dep issue).
Source-only cherry-pick, TypeScript compilation clean for modified file.

Co-Authored-By: polux0 <aleksaprosperitylabs@gmail.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 03:09:26 -05:00
Alex Newman 75a0f2e981 fix: respect CLAUDE_CONFIG_DIR for plugin paths (#626)
Add MARKETPLACE_ROOT constant to paths.ts and update 5 source files to use
centralized path constants instead of hardcoded ~/.claude paths. Preserves
backwards compatibility when CLAUDE_CONFIG_DIR is not set.

Based on PR #634 by @Kuroakira, cherry-picked onto main due to build artifact
merge conflicts (source changes applied cleanly).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 03:00:08 -05:00
Alex Newman 91e1d5baad fix: correct Gemini model name from gemini-3-flash to gemini-3-flash-preview
The Gemini API requires the -preview suffix for the Gemini 3 Flash model.
gemini-3-flash does not exist - only gemini-3-flash-preview is available.
This was causing 404 errors when users selected this model option.

Closes #831

Co-Authored-By: Glucksberg <markuscontasul@gmail.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 02:55:30 -05:00
Alex Newman 3ad53733e8 MAESTRO: Merge PR #884 adding Zscaler SSL certificate support for ChromaDB vector search
Adds automatic detection and handling of Zscaler enterprise security certificates
on macOS. Combines standard certifi CA certificates with Zscaler certificates into
a single bundle, passed via SSL_CERT_FILE/REQUESTS_CA_BUNDLE/CURL_CA_BUNDLE env vars
to the chroma-mcp subprocess. Certificate bundle is cached for 24 hours. Falls back
gracefully when Zscaler is not present, with no impact on non-Zscaler environments.

Co-Authored-By: RClark4958 <rickdclark48@gmail.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 02:15:19 -05:00