Compare commits

...

67 Commits

Author SHA1 Message Date
Alex Newman 8c79b99384 chore: bump version to 10.6.0
Publish to npm / publish (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 17:15:27 -07:00
Glucksberg 9361e33b6d fix(openclaw): inject context via system prompt instead of overwriting MEMORY.md (#1386)
* fix(openclaw): inject context via system prompt instead of overwriting MEMORY.md

The OpenClaw plugin was overwriting each agent's MEMORY.md with a large
auto-generated observation dump (~12-15KB) on every before_agent_start
and tool_result_persist event. This conflicts with OpenClaw's design
where MEMORY.md is agent-curated long-term memory.

Migrate context injection from file-based (writeFile MEMORY.md) to
OpenClaw's native before_prompt_build hook, which returns context via
appendSystemContext. This keeps MEMORY.md under agent control while
still providing cross-session observation context to the LLM.

Changes:
- Add before_prompt_build hook that returns { appendSystemContext }
- Remove writeFile/MEMORY.md sync from before_agent_start
- Remove MEMORY.md sync from tool_result_persist (observations still recorded)
- Add 60s TTL cache to avoid re-fetching context on every LLM turn
- Add syncMemoryFileExclude config for per-agent opt-out
- Remove dead workspaceDirsBySessionKey tracking map
- Rewrite test suite to verify prompt injection instead of file writes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ui): align settings defaults with backend and use nullish coalescing

The web UI had two issues causing settings inflation:

1. DEFAULT_SETTINGS in the UI used FULL_COUNT='5' and all token columns
   'true', while SettingsDefaultsManager (backend) uses FULL_COUNT='0'
   and token columns 'false'. Opening the settings modal and saving
   without changes would silently inflate the context.

2. useSettings used || for fallback, which treats '0' and 'false' as
   falsy — even when the backend correctly returns these values, the UI
   would replace them with inflated defaults. Changed to ?? (nullish
   coalescing) so only null/undefined trigger the fallback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(openclaw): update integration docs for system prompt injection

Reflect the migration from MEMORY.md file writes to before_prompt_build
hook-based context injection:

- Update architecture diagram and overview to show new hook flow
- Replace "MEMORY.md Live Sync" section with "System Prompt Context Injection"
- Update event lifecycle steps (before_agent_start, tool_result_persist)
- Add before_prompt_build step with TTL cache description
- Document new syncMemoryFileExclude config parameter
- Update session tracking to reflect removed workspaceDirsBySessionKey

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix terminology and update SKILL.md for system prompt injection

Replace "prompt injection" with "context injection" in docs to avoid
confusion with the OWASP security term. Update openclaw/SKILL.md to
reflect the new before_prompt_build hook and remove stale MEMORY.md
references.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Alex Newman <thedotmack@gmail.com>
2026-03-17 17:14:30 -07:00
Alex Newman 9e7b08445f Add table for project preview and star history 2026-03-17 14:48:16 -07:00
Alex Newman 033c1c4503 Restore Star History section in README
Reintroduced the Star History section with updated HTML structure.
2026-03-17 14:44:23 -07:00
Alex Newman 8d74031213 Remove Star History Chart from README
Removed Star History Chart image from README.
2026-03-17 14:43:38 -07:00
Alex Newman 3bc3697648 Enhance README with star history section
Added star history chart and image to README.
2026-03-17 14:43:07 -07:00
Alex Newman 4d7b29786b Add Star History section to README
Added Star History section with a chart link.
2026-03-17 14:41:54 -07:00
Alex Newman 4c697899e0 Update README to reflect changes in $CMEM information
Removed outdated links and added information about the $CMEM token.
2026-03-16 23:56:17 -07:00
Alex Newman ef0a07f606 Update official links in README.md (#1381) 2026-03-16 23:40:25 -07:00
Alex Newman 472ed8e1e0 docs: update CHANGELOG.md for v10.5.6 2026-03-16 14:55:10 -07:00
Alex Newman 5ccd81b8a3 chore: bump version to 10.5.6
Publish to npm / publish (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 14:54:32 -07:00
Alex Newman 678ae1e7d3 chore: rebuild worker-service.cjs with latest source changes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 14:53:03 -07:00
Alex Newman 80a8c90a1a feat: add embedded Process Supervisor for unified process lifecycle (#1370)
* feat: add embedded Process Supervisor for unified process lifecycle management

Consolidates scattered process management (ProcessManager, GracefulShutdown,
HealthMonitor, ProcessRegistry) into a unified src/supervisor/ module.

New: ProcessRegistry with JSON persistence, env sanitizer (strips CLAUDECODE_*
vars), graceful shutdown cascade (SIGTERM → 5s wait → SIGKILL with tree-kill
on Windows), PID file liveness validation, and singleton Supervisor API.

Fixes #1352 (worker inherits CLAUDECODE env causing nested sessions)
Fixes #1356 (zombie TCP socket after Windows reboot)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add session-scoped process reaping to supervisor

Adds reapSession(sessionId) to ProcessRegistry for killing session-tagged
processes on session end. SessionManager.deleteSession() now triggers reaping.
Tightens orphan reaper interval from 60s to 30s.

Fixes #1351 (MCP server processes leak on session end)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add Unix domain socket support for worker communication

Introduces socket-manager.ts for UDS-based worker communication, eliminating
port 37777 collisions between concurrent sessions. Worker listens on
~/.claude-mem/sockets/worker.sock by default with TCP fallback.

All hook handlers, MCP server, health checks, and admin commands updated to
use socket-aware workerHttpRequest(). Backwards compatible — settings can
force TCP mode via CLAUDE_MEM_WORKER_TRANSPORT=tcp.

Fixes #1346 (port 37777 collision across concurrent sessions)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove in-process worker fallback from hook command

Removes the fallback path where hook scripts started WorkerService in-process,
making the worker a grandchild of Claude Code (killed by sandbox). Hooks now
always delegate to ensureWorkerStarted() which spawns a fully detached daemon.

Fixes #1249 (grandchild process killed by sandbox)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add health checker and /api/admin/doctor endpoint

Adds 30-second periodic health sweep that prunes dead processes from the
supervisor registry and cleans stale socket files. Adds /api/admin/doctor
endpoint exposing supervisor state, process liveness, and environment health.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add comprehensive supervisor test suite

64 tests covering all supervisor modules: process registry (18 tests),
env sanitizer (8), shutdown cascade (10), socket manager (15), health
checker (5), and supervisor API (6). Includes persistence, isolation,
edge cases, and cross-module integration scenarios.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert Unix domain socket transport, restore TCP on port 37777

The socket-manager introduced UDS as default transport, but this broke
the HTTP server's TCP accessibility (viewer UI, curl, external monitoring).
Since there's only ever one worker process handling all sessions, the
port collision rationale for UDS doesn't apply. Reverts to TCP-only,
removing ~900 lines of unnecessary complexity.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove dead code found in pre-landing review

Remove unused `acceptingSpawns` field from Supervisor class (written but
never read — assertCanSpawn uses stopPromise instead) and unused
`buildWorkerUrl` import from context handler.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* updated gitignore

* fix: address PR review feedback - downgrade HTTP logging, clean up gitignore, harden supervisor

- Downgrade request/response HTTP logging from info to debug to reduce noise
- Remove unused getWorkerPort imports, use buildWorkerUrl helper
- Export ENV_PREFIXES/ENV_EXACT_MATCHES from env-sanitizer, reuse in Server.ts
- Fix isPidAlive(0) returning true (should be false)
- Add shutdownInitiated flag to prevent signal handler race condition
- Make validateWorkerPidFile testable with pidFilePath option
- Remove unused dataDir from ShutdownCascadeOptions
- Upgrade reapSession log from debug to warn
- Rename zombiePidFiles to deadProcessPids (returns actual PIDs)
- Clean up gitignore: remove duplicate datasets/, stale ~*/ and http*/ patterns
- Fix tests to use temp directories instead of relying on real PID file

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 14:49:23 -07:00
Vincent Leraitre 237a4c37f8 fix: always pass --ssl flag to chroma-mcp in remote mode (#1286)
* fix: always pass --ssl flag to chroma-mcp in remote mode

The chroma-mcp CLI defaults to SSL when using --client-type http.
When CLAUDE_MEM_CHROMA_SSL is false (the common case for local
ChromaDB servers), buildCommandArgs() omitted --ssl entirely,
causing chroma-mcp to attempt an SSL connection to a plain HTTP
server and fail with "Could not connect to a Chroma server".

Always pass --ssl with an explicit true/false value so the user's
CLAUDE_MEM_CHROMA_SSL setting is faithfully forwarded.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add regression tests for ChromaMcpManager SSL flag fix

Adds 4 focused test cases verifying buildCommandArgs() produces correct
--ssl args, covering SSL=false, SSL=true, unset (defaults to false), and
local mode (no --ssl flag). Requested by @xkonjin in PR #1286 review.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: rebuild checked-in bundles to include SSL flag fix

Rebuild all bundles against upstream/main so the --ssl <true|false>
fix is present in the runtime artifacts that hooks and the marketplace
plugin actually execute.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 20:03:58 -07:00
laihenyi 626654f816 fix: prevent infinite restart loop on FOREIGN KEY constraint errors (#1334)
The pending-work-restart logic had no retry limit, causing infinite loops
when sessions encountered FOREIGN KEY constraint failures. This led to
2000+ error log entries per minute and eventual worker crash via SIGTERM.

Two fixes:
1. Add 'FOREIGN KEY constraint failed' to unrecoverable error patterns
   so it short-circuits immediately instead of falling through to restart
2. Add MAX_PENDING_RESTARTS (3) limit to pending-work-restart path as a
   safety net for any future unhandled persistent errors

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 20:03:48 -07:00
secyunshu ed5189ebe9 fix: merge SessionStart hooks to run sequentially (#1341)
The SessionStart hook was incorrectly split into two separate matchers
with the same pattern "startup|clear|compact", causing them to run
in parallel per Claude Code's hook execution model. This resulted in
a race condition where both hooks tried to bind to port 37777
simultaneously, causing "port in use" errors on first startup.

This fix consolidates all SessionStart commands into a single
matcher, ensuring they execute sequentially.

Fixes regression introduced in commit d93bde0.

Co-authored-by: yunshu <yunshu@moresec.cn>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 20:03:44 -07:00
enzoricciulli e7ba9acaa7 fix: add content-hash dedup to batch observation store methods (#1302)
storeObservations() and storeObservationsAndMarkComplete() were missing
the content-hash deduplication that storeObservation() (singular) already
had via computeObservationContentHash() and findDuplicateObservation().

This caused the Gemini provider (and potentially others that return
multiple observations per response) to insert 2-10x duplicate rows per
tool use, since the batch methods inserted unconditionally without
checking content_hash.

The fix adds the same dedup pattern from storeObservation() to both
batch methods:
1. Compute content hash via computeObservationContentHash()
2. Check for existing observation within 30s window via findDuplicateObservation()
3. Skip insert and reuse existing ID if duplicate found
4. Include content_hash column in INSERT statement

Fixes #1158 (duplicate observations with Gemini provider)

Co-authored-by: Enzo Ricciulli <e.ricciulli@systhema.ai>
2026-03-12 20:01:53 -07:00
antmid ad902bedd9 fix: auto-repair malformed database schema from cross-version sync (#1308)
When a claude-mem DB is synced between machines running different versions,
orphaned indexes can reference non-existent columns (e.g. idx_observations_content_hash
referencing content_hash). This causes SQLite to throw "malformed database schema"
on ALL queries, including PRAGMAs, creating a silent 503 failure loop.

The fix detects this on startup, uses Python's sqlite3 module to drop the
orphaned schema objects (bun:sqlite doesn't support writable_schema modifications),
resets migration versions, and lets the idempotent migration system recreate
everything properly.

Fixes #1307

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 20:01:51 -07:00
GigiTiti-Kai b88566dcdd fix(ui): include SSE live data when project filter is active (#1315)
When a project filter was selected in the Web UI, all SSE live data
(observations, summaries, prompts) was completely discarded. Only
paginated API data was shown, meaning new real-time events were
invisible until the user refreshed the page.

Fix: filter SSE data by project before merging with paginated data,
instead of discarding it entirely.

Fixes #1313

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 20:01:48 -07:00
Rajiv Sinclair 1fac57535e fix: gracefully handle missing transcript files in worktree sessions (#1326)
When Claude Code runs in a worktree (via Agent tool with isolation: "worktree"),
the transcript path points to the worktree's project directory. After the
worktree is cleaned up, the Stop hook fires but the transcript file no longer
exists, causing extractLastMessage() to throw. This error triggers Claude to
respond, which fires another Stop hook, creating an infinite error loop.

Changed throws to warn-and-return-empty so the summarize hook exits cleanly
with exit 0 instead of cascading errors.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 19:59:47 -07:00
AlexWorland 10e980cd69 fix: remove unrecognized fields from Claude Code Stop hook output (#1291)
* fix: remove unrecognized fields from Claude Code Stop hook output

Claude Code validates Stop hook JSON output against its hook contract
schema which only accepts {decision?, reason?, systemMessage?}. The
formatOutput() function was returning {continue, suppressOutput} which
are not part of the Claude Code hook API, causing "JSON validation
failed" errors on every session stop.

Return an empty object {} for the default case (no hookSpecificOutput),
preserving only systemMessage when present. This is valid for all hook
event types and eliminates the schema validation error.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add unhappy-path tests for formatOutput per PR review

Add edge case coverage for malformed input (undefined/null), falsy
systemMessage values, non-contract field stripping, and contract key
allowlist. Also add defensive null guard to formatOutput matching
normalizeInput pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Alex Worland <alexworland@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 19:59:45 -07:00
Nir Alfasi 38d9ac7adb fix: prevent zombie subprocess accumulation by only trusting exitCode (#1226) (#1325)
proc.killed only means Node sent a signal — the process can still be alive.
This caused premature pool slot release, allowing unbounded process spawning.

- ensureProcessExit: remove proc.killed from early-exit checks, only trust exitCode
- Fix 3 call-site guards that skipped cleanup for signaled-but-alive processes
- Add TOTAL_PROCESS_HARD_CAP=10 safety net in waitForSlot()
- After SIGKILL, wait up to 1s via exit event instead of blind 200ms sleep
- Reduce reaper interval from 5min to 1min, idle threshold from 2min to 1min

Closes #1226

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 19:59:42 -07:00
GigiTiti-Kai 23058d4b0c fix: move session-complete from Stop to SessionEnd hook (#1330)
The `session-complete` command in the Stop hook runs every turn,
killing the SDK agent via SIGTERM. This causes the next summarize
call to receive an empty response because the agent needs to restart.

By moving `session-complete` to SessionEnd (which only fires on
actual session termination), the SDK agent stays alive between turns
and summarize works reliably.

Fixes #1314

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 19:59:39 -07:00
Ben Younes 503bda4868 fix: add null guards for getChromaSync() when Chroma is disabled (#1336)
When CLAUDE_MEM_CHROMA_ENABLED=false, getChromaSync() returns null.
Two call sites were missing null guards, causing "null is not an object"
errors on every UserPromptSubmit / session init.

Fixes #1294

Vibe-coded by Ousama Ben Younes

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 19:58:03 -07:00
Ben Younes 4616f7ab1c fix: output valid JSON from smart-install.js hook to prevent SessionStart error (#1337)
smart-install.js used stdio: 'inherit' for execSync calls, leaking
plain-text install output (bun/npm progress) to stdout. Claude Code
expects hook output to start with '{' (valid JSON), so this caused
a confusing "SessionStart:startup hook error" on every session start.

Changes:
- Pipe stdout on all execSync calls to prevent non-JSON stdout leak
- Output {"continue":true,"suppressOutput":true} on both success and
  error paths so Claude Code always receives valid JSON
- Add tests verifying no stdio:'inherit' remains and JSON is output

Fixes #1253

Vibe-coded by Ousama Ben Younes

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 19:58:00 -07:00
Umut Polat 73113321a1 fix: respect dateStart/dateEnd filters in Chroma search path (#1343)
When a search query includes dateStart/dateEnd parameters, the Chroma
semantic search path (PATH 2) ignored them entirely and only applied a
hardcoded 90-day recency window. This meant date-filtered searches
returned results from outside the requested range.

Now the Chroma path checks for a user-provided dateRange first. If
present, it filters results by the requested start/end bounds. The
90-day window is only used as the default when no date filter is
specified, preserving backward compatibility.

Fixes #1324

Signed-off-by: umut-polat <52835619+umut-polat@users.noreply.github.com>
2026-03-12 19:57:58 -07:00
Umut Polat 88be01910b fix: respect env vars and settings.json for DATA_DIR resolution (#1344)
SettingsDefaultsManager.get() returned only the hardcoded default,
ignoring both environment variables and settings.json overrides. This
meant CLAUDE_MEM_DATA_DIR set via env or settings file had no effect
on paths.ts (and other callers using get()).

Two changes:
- get() now checks process.env before falling back to the default
- paths.ts resolves DATA_DIR with full priority: env var > settings.json
  at the default location > hardcoded default

The settings file read uses a synchronous bootstrap pattern to avoid
circular dependencies (DATA_DIR is needed to locate the settings file,
so we check the default path only).

Fixes #1303

Signed-off-by: umut-polat <52835619+umut-polat@users.noreply.github.com>
2026-03-12 19:57:55 -07:00
Umut Polat 9dbf63f5d4 fix: prevent LLM from using <observation> tags in summary responses (#1345)
The SDK agent's conversation history is heavily biased toward
<observation> output. By the time a summarize prompt arrives, the
in-context conditioning can cause the LLM to respond with <observation>
tags instead of the expected <summary> tags. parseSummary() then returns
null and the summary is silently lost.

Two changes:
- Add explicit mode-switch instructions at the top of the summary prompt
  telling the LLM not to use <observation> tags and that only <summary>
  output will be accepted
- Add a warning log in parseSummary() when <observation> tags are found
  in a response that has no <summary> block, making the issue visible
  in logs instead of silently discarding

Fixes #1312

Signed-off-by: umut-polat <52835619+umut-polat@users.noreply.github.com>
2026-03-12 19:57:52 -07:00
Alex Newman 3651a34e96 docs: update CHANGELOG.md for v10.5.5
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:02:35 -07:00
Alex Newman 79bc3c85b3 chore: bump version to 10.5.5
Publish to npm / publish (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:02:03 -07:00
Alex Newman 6581d2ef45 fix: unify mode type/concept loading to always use mode definition (#1316)
* fix: unify mode type/concept loading to always use mode definition

Code mode previously read observation types/concepts from settings.json
while non-code modes read from their mode JSON definition. This caused
stale filters to persist when switching between modes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: remove dead observation type/concept settings constants

CLAUDE_MEM_CONTEXT_OBSERVATION_TYPES and OBSERVATION_CONCEPTS are no
longer read by ContextConfigLoader since all modes now use their mode
definition. Removes the constants, defaults, UI controls, and the
now-empty observation-metadata.ts file.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:00:20 -07:00
Alex Newman 39db5c4882 docs: update CHANGELOG.md for v10.5.4
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 21:02:40 -07:00
Alex Newman 3af68b7dfe chore: bump version to 10.5.4
Publish to npm / publish (push) Has been cancelled
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 21:00:15 -07:00
Alex Newman e9b4f75fb2 fix: restore modes to plugin/modes/ from erroneous plugin/hooks/modes/ location
Modes were incorrectly moved to plugin/hooks/modes/ in v10.5.3, breaking
the release. This restores them to the correct plugin/modes/ directory.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 20:52:24 -07:00
Alex Newman 2af37422da docs: update CHANGELOG.md for v10.5.3 2026-03-08 19:36:20 -07:00
Alex Newman a32151a166 chore: bump version to 10.5.3
Publish to npm / publish (push) Has been cancelled
2026-03-08 19:35:56 -07:00
Alex Newman 97ea9e45fc feat: add law-study mode for law students (#1305)
Adds a new claude-mem mode tailored for law school study sessions, with
observation types for case holdings, issue patterns, prof frameworks,
doctrine/rule synthesis, argument structures, and cross-case connections.
Includes a chill variant and a CLAUDE.md template for use as a legal
study partner.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 19:35:21 -07:00
Alex Newman ecb09df420 docs: update CHANGELOG.md for v10.5.2
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 22:23:45 -05:00
Alex Newman 6c7acfbc1c chore: bump version to 10.5.2
Publish to npm / publish (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 22:23:15 -05:00
Alex Newman 44a7b2fcb9 docs: add smart-explore benchmark report and update skill with benchmark data
Add public benchmark report documenting the A/B comparison between Smart
Explore and the standard Explore agent (17.8x cheaper discovery, 19.4x
cheaper targeted reads). Update SKILL.md with benchmark-accurate token
economics, completeness guarantee, map-first principle, and Explore agent
escalation guidance.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 22:22:41 -05:00
Alex Newman 7015301d8f docs: update CHANGELOG.md for v10.5.1
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 21:08:33 -05:00
Alex Newman a5e86ad4ab chore: bump version to 10.5.1
Publish to npm / publish (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 21:08:19 -05:00
Alex Newman d93bde059e fix: restore hooks.json to pre-smart-explore configuration
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 21:08:01 -05:00
Alex Newman d60ae14a9b docs: update CHANGELOG.md for v10.5.0
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 21:03:27 -05:00
Alex Newman 272391ec9d chore: bump version to 10.5.0
Publish to npm / publish (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 21:02:55 -05:00
Alex Newman 0e502dbd21 feat: add smart-explore AST-based code navigation (#1244)
* feat: add smart-file-read module for token-optimized semantic code search

- Created package.json for the smart-file-read module with dependencies and scripts.
- Implemented parser.ts for code structure parsing using tree-sitter, supporting multiple languages.
- Developed search.ts for searching code files and symbols with grep-style and structural matching.
- Added test-run.mjs for testing search and outline functionalities.
- Configured TypeScript with tsconfig.json for strict type checking and module resolution.

* fix: update .gitignore to include _tree-sitter and remove unused subproject

* feat: add preliminary results and skill recommendation for smart-explore module

* chore: remove outdated plan.md file detailing session start hook issues

* feat: update Smart File Read integration plan and skill documentation for smart-explore

* feat: migrate Smart File Read to web-tree-sitter WASM for cross-platform compatibility

* refactor: switch to tree-sitter CLI for parsing and enhance search functionality

- Updated `parser.ts` to utilize the tree-sitter CLI for AST extraction instead of native bindings, improving compatibility and performance.
- Removed grammar loading logic and replaced it with a path resolution for grammar packages.
- Implemented batch parsing in `parseFilesBatch` to handle multiple files in a single CLI call, enhancing search speed.
- Refactored `searchCodebase` to collect files and parse them in batches, streamlining the search process.
- Adjusted symbol extraction logic to accommodate the new parsing method and ensure accurate symbol matching.

* feat: update Smart File Read integration plan to utilize tree-sitter CLI for improved performance and cross-platform compatibility

* feat: add smart-file-read parser and search to src/services

Copy validated tree-sitter CLI-based parser and search modules from
smart-file-read prototype into the claude-mem source tree for MCP
tool integration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: register smart_search, smart_unfold, smart_outline MCP tools

Add 3 tree-sitter AST-based code exploration tools to the MCP server.
Direct execution (no HTTP delegation) — they call parser/search
functions directly for sub-second response times.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add tree-sitter CLI deps to build system and plugin runtime

Externalize tree-sitter packages in esbuild MCP server build. Add
10 grammar packages + CLI to plugin package.json for runtime install.
Remove unused @chroma-core/default-embed from plugin deps.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: create smart-explore skill with 3-layer workflow docs

Progressive disclosure workflow: search -> outline -> unfold.
Documents all 3 MCP tools with parameters and token economics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add comprehensive documentation for the smart-explore feature

- Introduced a detailed technical reference covering the architecture, parser, search engine, and tool registration for the smart-explore feature in claude-mem.
- Documented the three-layer workflow: search, outline, and unfold, along with their respective MCP tools.
- Explained the parsing process using tree-sitter, including language support, query patterns, and symbol extraction.
- Outlined the search module's functionality, including file discovery, batch parsing, and relevance scoring.
- Provided insights into build system integration and token economics for efficient code exploration.

* chore: remove experiment artifacts, prototypes, and plan files

Remove A/B test docs, prototype smart-file-read directory, and
implementation plans. Keep only production code.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: simplify hooks configuration and remove setup script

* fix: use execFileSync to prevent command injection in tree-sitter parser

Replaces execSync shell string with execFileSync + argument array,
eliminating shell interpretation of file paths. Also corrects
file_pattern description from "Glob pattern" to "Substring filter".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 21:00:26 -05:00
Alex Newman 9ab119932a docs: update CHANGELOG.md for v10.4.4
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 19:56:05 -05:00
Alex Newman 50d1dfb7ee chore: bump version to 10.4.4
Publish to npm / publish (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 19:55:44 -05:00
Alex Newman 0b034af98b fix: remove save_observation from MCP tool surface
save_observation is an internal API-only feature and should not be
exposed as an MCP tool to Claude.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 19:55:12 -05:00
Alex Newman b43ad00f8b docs: update CHANGELOG.md for v10.4.3
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 19:34:56 -05:00
Alex Newman dd1b812443 chore: bump version to 10.4.3
Publish to npm / publish (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 19:34:28 -05:00
Alex Newman ad3d236cec fix: resolve hook crashes and CLAUDE_PLUGIN_ROOT fallback (#1215, #1220) (#1229)
* fix: resolve PostToolUse hook crashes and 5s latency (#1220)

Three compounding bugs caused hook failures:

1. Missing break statements in worker-service.ts switch — if async
   code threw before process.exit(), execution fell through to
   subsequent cases. Added break to all 7 cases missing them.

2. Unhandled promise rejection on main() — added .catch() that logs
   the error and exits 0 (per project exit code strategy: don't block
   Claude Code or leave Windows Terminal tabs open).

3. Redundant start commands in hooks.json — PostToolUse,
   UserPromptSubmit, and Stop groups each had a standalone start
   command that was redundant (the hook case already calls
   ensureWorkerStarted internally). The redundant start also caused
   5s latency via bun-runner.js collectStdin() timeout since Claude
   Code never closes stdin.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add CLAUDE_PLUGIN_ROOT fallback for Stop hooks (#1215)

Upstream Claude Code bug (anthropics/claude-code#24529) leaves
CLAUDE_PLUGIN_ROOT unset for Stop hooks on macOS and ALL hooks
on Linux. Two-layer defense:

1. Shell-level: hooks.json commands now use inline fallback
   _R="${CLAUDE_PLUGIN_ROOT}"; [ -z "$_R" ] && _R="$HOME/...";
   falling back to the known marketplace install path.

2. Script-level: bun-runner.js self-resolves plugin root from
   its own filesystem location via import.meta.url, and fixes
   broken /scripts/... paths that result from empty expansion.

Added test to verify all hook commands include the fallback path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 19:31:26 -05:00
Alex Newman 494f681cbf docs: update CHANGELOG.md for v10.4.1 2026-02-23 22:13:35 -05:00
Alex Newman 4144010264 chore: bump version to 10.4.1
Publish to npm / publish (push) Has been cancelled
2026-02-23 22:13:11 -05:00
Alex Newman d482f3ed76 chore: bump version to 10.4.1 2026-02-23 22:10:38 -05:00
Alex Newman 3c4486e69e feat: convert make-plan and do commands to skills (#1216) 2026-02-23 22:08:21 -05:00
alan e0fec4bad7 feat: add terminal output control for SessionStart context (#1143)
* feat: add terminal output control for SessionStart context

Add CLAUDE_MEM_CONTEXT_SHOW_TERMINAL_OUTPUT setting to control whether
context is displayed in the terminal at SessionStart.

When set to "false", the terminal remains clean at startup while
context is still injected into Claude's system prompt. This allows
users who find the context output verbose to disable it without
losing the automatic context injection.

Defaults to "true" for backward compatibility.

Changes:
- Add CLAUDE_MEM_CONTEXT_SHOW_TERMINAL_OUTPUT to SettingsDefaultsManager
- Check setting in context handler before setting systemMessage
- Update settings file format to include new option

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: use USER_SETTINGS_PATH and skip color fetch when disabled

Address PR feedback from automated review:

1. Use shared USER_SETTINGS_PATH constant instead of hardcoded path
   - Respects custom CLAUDE_MEM_DATA_DIR override
   - Consistent with other handlers (session-init, observation)

2. Skip color fetch when terminal output disabled
   - Check setting before making HTTP requests
   - Saves network round-trip on every session start

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Alan Dong <adong@Alans-MacBook-Pro.local>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-23 21:05:05 -05:00
Alex Newman f5a873ffdc docs: update CHANGELOG.md for v10.4.0 2026-02-23 19:39:59 -05:00
Alex Newman 23f30d35b9 chore: bump version to 10.4.0
Publish to npm / publish (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 19:39:06 -05:00
Alex Newman c6f932988a Fix 30+ root-cause bugs across 10 triage phases (#1214)
* MAESTRO: fix ChromaDB core issues — Python pinning, Windows paths, disable toggle, metadata sanitization, transport errors

- Add --python version pinning to uvx args in both local and remote mode (fixes #1196, #1206, #1208)
- Convert backslash paths to forward slashes for --data-dir on Windows (fixes #1199)
- Add CLAUDE_MEM_CHROMA_ENABLED setting for SQLite-only fallback mode (fixes #707)
- Sanitize metadata in addDocuments() to filter null/undefined/empty values (fixes #1183, #1188)
- Wrap callTool() in try/catch for transport errors with auto-reconnect (fixes #1162)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix data integrity — content-hash deduplication, project name collision, empty project guard, stuck isProcessing

- Add SHA-256 content-hash deduplication to observations INSERT (store.ts, transactions.ts, SessionStore.ts)
- Add content_hash column via migration 22 with backfill and index
- Fix project name collision: getCurrentProjectName() now returns parent/basename
- Guard against empty project string with cwd-derived fallback
- Fix stuck isProcessing: hasAnyPendingWork() resets processing messages older than 5 minutes
- Add 12 new tests covering all four fixes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix hook lifecycle — stderr suppression, output isolation, conversation pollution prevention

- Suppress process.stderr.write in hookCommand() to prevent Claude Code showing diagnostic
  output as error UI (#1181). Restores stderr in finally block for worker-continues case.
- Convert console.error() to logger.warn()/error() in hook-command.ts and handlers/index.ts
  so all diagnostics route to log file instead of stderr.
- Verified all 7 handlers return suppressOutput: true (prevents conversation pollution #598, #784).
- Verified session-complete is a recognized event type (fixes #984).
- Verified unknown event types return no-op handler with exit 0 (graceful degradation).
- Added 10 new tests in tests/hook-lifecycle.test.ts covering event dispatch, adapter defaults,
  stderr suppression, and standard response constants.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix worker lifecycle — restart loop coordination, stale transport retry, ENOENT shutdown race

- Add PID file mtime guard to prevent concurrent restart storms (#1145):
  isPidFileRecent() + touchPidFile() coordinate across sessions
- Add transparent retry in ChromaMcpManager.callTool() on transport
  error — reconnects and retries once instead of failing (#1131)
- Wrap getInstalledPluginVersion() with ENOENT/EBUSY handling (#1042)
- Verified ChromaMcpManager.stop() already called on all shutdown paths

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix Windows platform support — uvx.cmd spawn, PowerShell $_ elimination, windowsHide, FTS5 fallback

- Route uvx spawn through cmd.exe /c on Windows since MCP SDK lacks shell:true (#1190, #1192, #1199)
- Replace all PowerShell Where-Object {$_} pipelines with WQL -Filter server-side filtering (#1024, #1062)
- Add windowsHide: true to all exec/spawn calls missing it to prevent console popups (#1048)
- Add FTS5 runtime probe with graceful fallback when unavailable on Windows (#791)
- Guard FTS5 table creation in migrations, SessionSearch, and SessionStore with try/catch

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix skills/ distribution — build-time verification and regression tests (#1187)

Add post-build verification in build-hooks.js that fails if critical
distribution files (skills, hooks, plugin manifest) are missing. Add
10 regression tests covering skill file presence, YAML frontmatter,
hooks.json integrity, and package.json files field.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix MigrationRunner schema initialization (#979) — version conflict between parallel migration systems

Root cause: old DatabaseManager migrations 1-7 shared schema_versions table with
MigrationRunner's 4-22, causing version number collisions (5=drop tables vs add column,
6=FTS5 vs prompt tracking, 7=discovery_tokens vs remove UNIQUE).  initializeSchema()
was gated behind maxApplied===0, so core tables were never created when old versions
were present.

Fixes:
- initializeSchema() always creates core tables via CREATE TABLE IF NOT EXISTS
- Migrations 5-7 check actual DB state (columns/constraints) not just version tracking
- Crash-safe temp table rebuilds (DROP IF EXISTS _new before CREATE)
- Added missing migration 21 (ON UPDATE CASCADE) to MigrationRunner
- Added ON UPDATE CASCADE to FK definitions in initializeSchema()
- All changes applied to both runner.ts and SessionStore.ts

Tests: 13 new tests in migration-runner.test.ts covering fresh DB, idempotency,
version conflicts, crash recovery, FK constraints, and data integrity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix 21 test failures — stale mocks, outdated assertions, missing OpenClaw guards

Server tests (12): Added missing workerPath and getAiStatus to ServerOptions
mocks after interface expansion. ChromaSync tests (3): Updated to verify
transport cleanup in ChromaMcpManager after architecture refactor. OpenClaw (2):
Added memory_ tool skipping and response truncation to prevent recursive loops
and oversized payloads. MarkdownFormatter (2): Updated assertions to match
current output. SettingsDefaultsManager (1): Used correct default key for
getBool test. Logger standards (1): Excluded CLI transcript command from
background service check.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix Codex CLI compatibility (#744) — session_id fallbacks, unknown platform tolerance, undefined guard

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix Cursor IDE integration (#838, #1049) — adapter field fallbacks, tolerant session-init validation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix /api/logs OOM (#1203) — tail-read replaces full-file readFileSync

Replace readFileSync (loads entire file into memory) with readLastLines()
that reads only from the end of the file in expanding chunks (64KB → 10MB cap).
Prevents OOM on large log files while preserving the same API response shape.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix Settings CORS error (#1029) — explicit methods and allowedHeaders in CORS config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: add session custom_title for agent attribution (#1213) — migration 23, endpoint + store support

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: prevent CLAUDE.md/AGENTS.md writes inside .git/ directories (#1165)

Add .git path guard to all 4 write sites to prevent ref corruption when
paths resolve inside .git internals.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix plugin disabled state not respected (#781) — early exit check in all hook entry points

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix UserPromptSubmit context re-injection on every turn (#1079) — contextInjected session flag

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* MAESTRO: fix stale AbortController queue stall (#1099) — lastGeneratorActivity tracking + 30s timeout

Three-layer fix:
1. Added lastGeneratorActivity timestamp to ActiveSession, updated by
   processAgentResponse (all agents), getMessageIterator (queue yields),
   and startGeneratorWithProvider (generator launch)
2. Added stale generator detection in ensureGeneratorRunning — if no
   activity for >30s, aborts stale controller, resets state, restarts
3. Added AbortSignal.timeout(30000) in deleteSession to prevent
   indefinite hang when awaiting a stuck generator promise

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 19:34:35 -05:00
Alex Newman d9a30cc7d4 MAESTRO: exclude transcript CLI from logger-usage-standards test
src/services/transcripts/cli.ts is a CLI command with user-visible
console output, not a background service — console.log is intentional.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 18:55:50 -05:00
Alex Newman 50eeed97e7 MAESTRO: fix cache-based install — resolveRoot, post-install verification, CLI path
Replace hardcoded marketplace path in plugin/scripts/smart-install.js with
resolveRoot() that uses CLAUDE_PLUGIN_ROOT env var (set by Claude Code for
all hooks), with fallback to script location and legacy paths. Fixes #1128,
#1166 where cache installs couldn't find or install node_modules.

Also fixes installCLI() path (ROOT/plugin/scripts/ → ROOT/scripts/) and adds
verifyCriticalModules() post-install check with npm fallback on failure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 17:28:51 -05:00
Alex Newman 5f28550551 MAESTRO: fix MCP type coercion for batch endpoints, add defensive observation error handling
Add string-to-array coercion for ids and memorySessionIds in DataRoutes.ts
batch endpoints so MCP clients sending "[1,2,3]" or "1,2,3" instead of
native arrays no longer get 400 errors. Wrap observation storage path in
SessionRoutes.ts with try/catch returning 200 on recoverable errors instead
of 500, preventing hook breakage.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 16:41:28 -05:00
Alex Newman a6a843f871 docs: update CHANGELOG.md for v10.3.3 2026-02-23 03:48:22 -05:00
Alex Newman 2db9d0e383 chore: bump version to 10.3.3
Publish to npm / publish (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 03:47:48 -05:00
Alex Newman 0a26bb18bf fix: update footer text to reference claude-mem skill instead of MCP search tools
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 03:47:02 -05:00
Alex Newman bd11ccf12e docs: update CHANGELOG.md for v10.3.2
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 03:32:51 -05:00
146 changed files with 10707 additions and 2895 deletions
+1 -1
View File
@@ -10,7 +10,7 @@
"plugins": [
{
"name": "claude-mem",
"version": "10.3.2",
"version": "10.6.0",
"source": "./plugin",
"description": "Persistent memory system for Claude Code - context compression across sessions"
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "claude-mem",
"version": "10.2.5",
"version": "10.4.1",
"description": "Persistent memory system for Claude Code - seamlessly preserve context across sessions",
"author": {
"name": "Alex Newman"
+6 -8
View File
@@ -2,6 +2,7 @@ datasets/
node_modules/
dist/
!installer/dist/
**/_tree-sitter/
*.log
.DS_Store
.env
@@ -19,7 +20,6 @@ plugin/data.backup/
package-lock.json
bun.lock
private/
datasets/
Auto Run Docs/
# Generated UI files (built from viewer-template.html)
@@ -29,12 +29,10 @@ src/ui/viewer.html
.mcp.json
.cursor/
# Prevent literal tilde directories (path validation bug artifacts)
~*/
# Prevent other malformed path directories
http*/
https*/
# Ignore WebStorm project files (for dinosaur IDE users)
.idea/
.claude-octopus/
.claude/session-intent.md
.claude/session-plan.md
.octo/
+3
View File
@@ -0,0 +1,3 @@
{
"MD013": false
}
+736
View File
@@ -0,0 +1,736 @@
# Plan: NPX Distribution + Universal IDE/CLI Coverage for claude-mem
## Problem
1. **Installation is slow and fragile**: Current install clones the full git repo, runs `npm install`, and builds from source. The npm package already ships pre-built artifacts.
2. **IDE coverage is limited**: claude-mem only supports Claude Code (plugin) and Cursor (hooks installer). The AI coding tools landscape has exploded — Gemini CLI (95k stars), OpenCode (110k stars), Windsurf (~1M users), Codex CLI, Antigravity, Goose, Crush, Copilot CLI, and more all support extensibility.
## Key Insights
- **npm package already has everything**: `plugin/` directory ships pre-built. No git clone or build needed.
- **Transcript watcher already exists**: `src/services/transcripts/` has a fully built schema-based JSONL tailer. It just needs schemas for more tools.
- **3 integration tiers exist**: (1) Hook/plugin-based (Claude Code, Gemini CLI, OpenCode, Windsurf, Codex CLI, OpenClaw), (2) MCP-based (Cursor, Copilot CLI, Antigravity, Goose, Crush, Roo Code), (3) Transcript-based (anything with structured log files).
- **OpenClaw plugin already built**: Full plugin at `openclaw/src/index.ts` (1000+ lines). Needs to be wired into the npx installer.
- **Gemini CLI is architecturally near-identical to Claude Code**: 11 lifecycle hooks, JSON via stdin/stdout, exit code 0/2 convention, `GEMINI.md` context files, `~/.gemini/settings.json`. This is the easiest high-value integration.
- **OpenCode has the richest plugin system**: 20+ hook events across 12 categories, JS/TS plugin modules, custom tool creation, MCP support. 110k stars — largest open-source AI CLI.
- **`npx skills` by Vercel supports 41 agents** — proving the multi-IDE installer UX works. Their agent detection pattern (check if config dir exists) is the right model.
- **All IDEs share a single worker on port 37777**: One worker serves all integrations. Session source (which IDE) is tracked via the `source` field in hook payloads. No per-IDE worker instances.
- **This npx CLI fully replaces the old `claude-mem-installer`**: Not a supplement — the complete replacement.
## Solution
`npx claude-mem` becomes a unified CLI: install, configure any IDE, manage the worker, search memory.
```
npx claude-mem # Interactive install + IDE selection
npx claude-mem install # Same as above
npx claude-mem install --ide windsurf # Direct IDE setup
npx claude-mem start / stop / status # Worker management
npx claude-mem search <query> # Search memory from terminal
npx claude-mem transcript watch # Start transcript watcher
```
## Platform Support
**Windows, macOS, and Linux are all first-class targets.** Platform-specific considerations:
- **Config paths**: Use `os.homedir()` and `path.join()` everywhere — never hardcode `/` or `~`
- **Shebangs**: `#!/usr/bin/env node` for the CLI entry point (cross-platform via Node)
- **Bun detection**: Check `PATH`, common install locations per platform (`%USERPROFILE%\.bun\bin\bun.exe` on Windows, `~/.bun/bin/bun` on Unix)
- **File permissions**: `fs.chmod` is a no-op on Windows; don't gate on it
- **Process management**: Worker start/stop uses signals on Unix, taskkill on Windows — match existing `worker-service.ts` patterns
- **VS Code paths**: `~/Library/Application Support/Code/` (macOS), `~/.config/Code/` (Linux), `%APPDATA%/Code/` (Windows)
- **Shell config**: `.bashrc`/`.zshrc` on Unix, PowerShell profile on Windows (for PATH modifications if needed)
---
## Phase 0: Research Findings
### IDE Integration Tiers
**Tier 1 — Native Hook/Plugin Systems** (highest fidelity, real-time capture):
| Tool | Hooks | Config Location | Context Injection | Stars/Users |
|------|-------|----------------|-------------------|-------------|
| Claude Code | 5 lifecycle hooks | `~/.claude/settings.json` | CLAUDE.md, plugins | ~25% market |
| Gemini CLI | 11 lifecycle hooks | `~/.gemini/settings.json` | GEMINI.md | ~95k stars |
| OpenCode | 20+ event hooks + plugin SDK | `~/.config/opencode/opencode.json` | AGENTS.md + rules dirs | ~110k stars |
| Windsurf | 11 Cascade hooks | `.windsurf/hooks.json` | `.windsurf/rules/*.md` | ~1M users |
| Codex CLI | `notify` hook | `~/.codex/config.toml` | `.codex/AGENTS.md`, MCP | Growing (OpenAI) |
| OpenClaw | 8 event hooks + plugin SDK | `~/.openclaw/openclaw.json` | MEMORY.md sync | ~196k stars |
**Tier 2 — MCP Integration** (tool-based, search + context injection):
| Tool | MCP Support | Config Location | Context Injection |
|------|------------|----------------|-------------------|
| Cursor | First-class | `.cursor/mcp.json` | `.cursor/rules/*.mdc` |
| Copilot CLI | First-class (default MCP) | `~/.copilot/config` | `.github/copilot-instructions.md` |
| Antigravity | First-class + MCP Store | `~/.gemini/antigravity/mcp_config.json` | `.agent/rules/`, GEMINI.md |
| Goose | Native MCP (co-developed protocol) | `~/.config/goose/config.yaml` | MCP context |
| Crush | MCP + Skills | JSON config (charm.land schema) | Skills system |
| Roo Code | First-class | `.roo/` | `.roo/rules/*.md`, `AGENTS.md` |
| Warp | MCP + Warp Drive | `WARP.md` + Warp Drive UI | `WARP.md` |
**Tier 3 — Transcript File Watching** (passive, file-based):
| Tool | Transcript Location | Format |
|------|-------------------|--------|
| Claude Code | `~/.claude/projects/<proj>/<session>.jsonl` | JSONL |
| Codex CLI | `~/.codex/sessions/**/*.jsonl` | JSONL |
| Gemini CLI | `~/.gemini/tmp/<hash>/chats/` | JSON |
| OpenCode | `.opencode/` (SQLite) | SQLite — needs export |
### What claude-mem Already Has
| Component | Status | Location |
|-----------|--------|----------|
| Claude Code plugin | Complete | `plugin/hooks/hooks.json` |
| Cursor hooks installer | Complete | `src/services/integrations/CursorHooksInstaller.ts` |
| Platform adapters | Claude Code + Cursor + raw | `src/cli/adapters/` |
| Transcript watcher | Complete (schema-based JSONL) | `src/services/transcripts/` |
| Codex transcript schema | Sample exists | `src/services/transcripts/config.ts` |
| OpenClaw plugin | Complete (1000+ lines) | `openclaw/src/index.ts` |
| MCP server | Complete | `plugin/scripts/mcp-server.cjs` |
| Gemini CLI support | Not started | — |
| OpenCode support | Not started | — |
| Windsurf support | Not started | — |
### Patterns to Copy
- **Agent detection from `npx skills`** (`vercel-labs/skills/src/agents.ts`): Check if config directory exists
- **Existing installer logic** (`installer/src/steps/install.ts:29-83`): registerMarketplace, registerPlugin, enablePluginInClaudeSettings — **extract shared logic** from existing installer into reusable modules (DRY with the new CLI)
- **Bun resolution** (`plugin/scripts/bun-runner.js`): PATH lookup + common locations per platform
- **CursorHooksInstaller** (`src/services/integrations/CursorHooksInstaller.ts`): Reference implementation for IDE hooks installation
---
## Phase 1: NPX CLI Entry Point
### What to implement
1. **Add `bin` field to `package.json`**:
```json
"bin": {
"claude-mem": "./dist/cli/index.js"
}
```
2. **Create `src/npx-cli/index.ts`** — a Node.js CLI router (NOT Bun) with command categories:
**Install commands** (pure Node.js, no Bun required):
- `npx claude-mem` or `npx claude-mem install` → interactive install (IDE multi-select)
- `npx claude-mem install --ide <name>` → direct IDE setup (only for implemented IDEs; unimplemented ones error with "Support for <name> coming soon")
- `npx claude-mem update` → update to latest version
- `npx claude-mem uninstall` → remove plugin and IDE configs
- `npx claude-mem version` → print version
**Runtime commands** (delegate to Bun via installed plugin):
- `npx claude-mem start` → spawns `bun worker-service.cjs start`
- `npx claude-mem stop` → spawns `bun worker-service.cjs stop`
- `npx claude-mem restart` → spawns `bun worker-service.cjs restart`
- `npx claude-mem status` → spawns `bun worker-service.cjs status`
- `npx claude-mem search <query>` → hits `GET http://localhost:37777/api/search?q=<query>`
- `npx claude-mem transcript watch` → starts transcript watcher
**Runtime commands must check for installation first**: If plugin directory doesn't exist at `~/.claude/plugins/marketplaces/thedotmack/`, print "claude-mem is not installed. Run: npx claude-mem install" and exit.
3. **The install flow** (fully replaces git clone + build):
- Detect the npm package's own location (`import.meta.url` or `__dirname`)
- Copy `plugin/` from the npm package to `~/.claude/plugins/marketplaces/thedotmack/`
- Copy `plugin/` to `~/.claude/plugins/cache/thedotmack/claude-mem/<version>/`
- Register marketplace in `~/.claude/plugins/known_marketplaces.json`
- Register plugin in `~/.claude/plugins/installed_plugins.json`
- Enable in `~/.claude/settings.json`
- Run `npm install` in the marketplace dir (for `@chroma-core/default-embed` — native ONNX binaries, can't be bundled)
- Trigger smart-install.js for Bun/uv setup
- Run IDE-specific setup for each selected IDE
4. **Interactive IDE selection** (auto-detect + prompt):
- Auto-detect installed IDEs by checking config directories
- Present multi-select with detected IDEs pre-selected
- Detection map:
- Claude Code: `~/.claude/` exists
- Gemini CLI: `~/.gemini/` exists
- OpenCode: `~/.config/opencode/` exists OR `opencode` in PATH
- OpenClaw: `~/.openclaw/` exists
- Windsurf: `~/.codeium/windsurf/` exists
- Codex CLI: `~/.codex/` exists
- Cursor: `~/.cursor/` exists
- Copilot CLI: `copilot` in PATH (it's a CLI tool, not a config dir)
- Antigravity: `~/.gemini/antigravity/` exists
- Goose: `~/.config/goose/` exists OR `goose` in PATH
- Crush: `crush` in PATH
- Roo Code: check for VS Code extension directory containing `roo-code`
- Warp: `~/.warp/` exists OR `warp` in PATH
5. **The runtime command routing**:
- Locate the installed plugin directory
- Find Bun binary (same logic as `bun-runner.js`, platform-aware)
- Spawn `bun worker-service.cjs <command>` and pipe stdio through
- For `search`: HTTP request to running worker
### Patterns to follow
- `installer/src/steps/install.ts:29-83` for marketplace registration — **extract to shared module**
- `plugin/scripts/bun-runner.js` for Bun resolution
- `vercel-labs/skills/src/agents.ts` for IDE auto-detection pattern
### Verification
- `npx claude-mem install` copies plugin to correct directories on macOS, Linux, and Windows
- Auto-detection finds installed IDEs
- `npx claude-mem start/stop/status` work after install
- `npx claude-mem search "test"` returns results
- `npx claude-mem start` before install prints helpful error message
- `npx claude-mem update` and `npx claude-mem uninstall` work correctly
- `npx claude-mem version` prints version
### Anti-patterns
- Do NOT require Bun for install commands — pure Node.js
- Do NOT clone the git repo
- Do NOT build from source at install time
- Do NOT depend on `bun:sqlite` in the CLI entry point
---
## Phase 2: Build Pipeline Integration
### What to implement
1. **Add CLI build step to `scripts/build-hooks.js`**:
- Compile `src/npx-cli/index.ts` → `dist/cli/index.js`
- Bundle `@clack/prompts` and `picocolors` into the output (self-contained)
- Shebang: `#!/usr/bin/env node`
- Set executable permissions (no-op on Windows, that's fine)
2. **Move `@clack/prompts` and `picocolors`** to main package.json as dev dependencies (bundled by esbuild into dist/cli/index.js)
3. **Verify `package.json` `files` field**: Currently `["dist", "plugin"]`. `dist/cli/index.js` is already included since it's under `dist/`. No change needed.
4. **Update `prepublishOnly`** to ensure CLI is built before npm publish (already covered — `npm run build` calls `build-hooks.js`)
5. **Pre-build OpenClaw plugin**: Add an esbuild step that compiles `openclaw/src/index.ts` → `openclaw/dist/index.js` so it ships ready-to-use. No `tsc` at install time.
6. **Add `openclaw/dist/` to `package.json` `files` field** (or add `openclaw` if the whole directory should ship)
### Verification
- `npm run build` produces `dist/cli/index.js` with correct shebang
- `npm run build` produces `openclaw/dist/index.js` pre-built
- `npm pack` includes both `dist/cli/index.js` and `openclaw/dist/`
- `node dist/cli/index.js --help` works without Bun
- Package size is reasonable (check with `npm pack --dry-run`)
---
## Phase 3: Gemini CLI Integration (Tier 1 — Hook-Based)
**Why first among new IDEs**: Near-identical architecture to Claude Code. 11 lifecycle hooks with JSON stdin/stdout, same exit code conventions (0=success, 2=block), `GEMINI.md` context files. 95k GitHub stars. Lowest effort, highest confidence.
### Gemini CLI Hook Events
| Event | Map to claude-mem | Use |
|-------|-------------------|-----|
| `SessionStart` | `session-init` | Start tracking session |
| `BeforeAgent` | `user-prompt` | Capture user prompt |
| `AfterAgent` | `observation` | Capture full agent response |
| `BeforeTool` | — | Skip (pre-execution, no result yet) |
| `AfterTool` | `observation` | Capture tool name + input + response |
| `BeforeModel` | — | Skip (too low-level, LLM request details) |
| `AfterModel` | — | Skip (raw LLM response, redundant with AfterAgent) |
| `BeforeToolSelection` | — | Skip (internal planning step) |
| `PreCompress` | `summary` | Trigger summary before context compression |
| `Notification` | — | Skip (system alerts, not session data) |
| `SessionEnd` | `session-end` | Finalize session |
**Mapped**: 5 of 11 events. **Skipped**: 6 events that are either too low-level (BeforeModel/AfterModel), pre-execution (BeforeTool, BeforeToolSelection), or system-level (Notification).
### Verified Stdin Payload Schemas (from `packages/core/src/hooks/types.ts`)
**Base input (all hooks receive):**
```typescript
{ session_id: string, transcript_path: string, cwd: string, hook_event_name: string, timestamp: string }
```
**Event-specific fields:**
| Event | Additional Fields |
|-------|-------------------|
| `SessionStart` | `source: "startup" \| "resume" \| "clear"` |
| `SessionEnd` | `reason: "exit" \| "clear" \| "logout" \| "prompt_input_exit" \| "other"` |
| `BeforeAgent` | `prompt: string` |
| `AfterAgent` | `prompt: string, prompt_response: string, stop_hook_active: boolean` |
| `BeforeTool` | `tool_name: string, tool_input: Record<string, unknown>, mcp_context?: McpToolContext, original_request_name?: string` |
| `AfterTool` | `tool_name: string, tool_input: Record<string, unknown>, tool_response: Record<string, unknown>, mcp_context?: McpToolContext` |
| `PreCompress` | `trigger: "auto" \| "manual"` |
| `Notification` | `notification_type: "ToolPermission", message: string, details: Record<string, unknown>` |
**Output (all hooks can return):**
```typescript
{ continue?: boolean, stopReason?: string, suppressOutput?: boolean, systemMessage?: string, decision?: "allow" | "deny" | "block" | "approve" | "ask", reason?: string, hookSpecificOutput?: Record<string, unknown> }
```
**Advisory (non-blocking) hooks:** SessionStart, SessionEnd, PreCompress, Notification — `continue` and `decision` fields are ignored.
**Environment variables provided:** `GEMINI_PROJECT_DIR`, `GEMINI_SESSION_ID`, `GEMINI_CWD`, `CLAUDE_PROJECT_DIR` (compat alias)
### What to implement
1. **Create Gemini CLI platform adapter** at `src/cli/adapters/gemini-cli.ts`:
- Normalize Gemini CLI's hook JSON to `NormalizedHookInput`
- Base fields always present: `session_id`, `transcript_path`, `cwd`, `hook_event_name`, `timestamp`
- Map per event:
- `SessionStart`: `source` → session init metadata
- `BeforeAgent`: `prompt` → user prompt text
- `AfterAgent`: `prompt` + `prompt_response` → full conversation turn
- `AfterTool`: `tool_name` + `tool_input` + `tool_response` → observation
- `PreCompress`: `trigger` → summary trigger
- `SessionEnd`: `reason` → session finalization
2. **Create Gemini CLI hooks installer** at `src/services/integrations/GeminiCliHooksInstaller.ts`:
- Write hooks to `~/.gemini/settings.json` under the `hooks` key
- Must **merge** with existing settings (read → parse → deep merge → write)
- Hook config format (verified against official docs):
```json
{
"hooks": {
"AfterTool": [{
"matcher": "*",
"hooks": [{ "name": "claude-mem", "type": "command", "command": "<path-to-hook-script>", "timeout": 5000 }]
}]
}
}
```
- Note: `matcher` uses regex for tool events, exact string for lifecycle events. `"*"` or `""` matches all.
- Hook groups support `sequential: boolean` (default false = parallel execution)
- Security: Project-level hooks are fingerprinted — if name/command changes, user is warned
- Context injection via `~/.gemini/GEMINI.md` (append claude-mem section with `<claude-mem-context>` tags, same pattern as CLAUDE.md)
- Settings hierarchy: project `.gemini/settings.json` > user `~/.gemini/settings.json` > system `/etc/gemini-cli/settings.json`
3. **Register `gemini-cli` in `getPlatformAdapter()`** at `src/cli/adapters/index.ts`
4. **Add Gemini CLI to installer IDE selection**
### Verification
- `npx claude-mem install --ide gemini-cli` merges hooks into `~/.gemini/settings.json`
- Gemini CLI sessions are captured by the worker
- `AfterTool` events produce observations with correct `tool_name`, `tool_input`, `tool_response`
- `GEMINI.md` gets claude-mem context section
- Existing Gemini CLI settings are preserved (merge, not overwrite)
- Verify `session_id` from base input is used for session tracking
### Anti-patterns
- Do NOT overwrite `~/.gemini/settings.json` — must deep merge
- Do NOT map all 11 events — the 6 skipped events would produce noise, not signal
- Do NOT use `type: "runtime"` — that's for internal extensions only; use `type: "command"`
- Advisory hooks (SessionStart, SessionEnd, PreCompress, Notification) cannot block — don't set `decision` or `continue` fields on them
---
## Phase 4: OpenCode Integration (Tier 1 — Plugin-Based)
**Why next**: 110k stars, richest plugin ecosystem. OpenCode plugins are JS/TS modules auto-loaded from plugin directories. OpenCode also has a Claude Code compatibility fallback (reads `~/.claude/CLAUDE.md` if no global `AGENTS.md` exists, controllable via `OPENCODE_DISABLE_CLAUDE_CODE_PROMPT=1`).
### Verified Plugin API (from `packages/plugin/src/index.ts`)
**Plugin signature:**
```typescript
import { type Plugin, tool } from "@opencode-ai/plugin"
export const ClaudeMemPlugin: Plugin = async (ctx) => {
// ctx: { client, project, directory, worktree, serverUrl, $ }
return { /* hooks object */ }
}
```
**PluginInput type (6 properties, not 4):**
```typescript
type PluginInput = {
client: ReturnType<typeof createOpencodeClient> // OpenCode SDK client
project: Project // Current project info
directory: string // Current working directory
worktree: string // Git worktree path
serverUrl: URL // Server URL
$: BunShell // Bun shell API
}
```
**Two hook mechanisms (important distinction):**
1. **Direct interceptor hooks** — keys on the returned `Hooks` object, receive `(input, output)` allowing mutation:
- `tool.execute.before`: `(input: { tool, sessionID, callID }, output: { args })`
- `tool.execute.after`: `(input: { tool, sessionID, callID, args }, output: { title, output, metadata })`
- `shell.env`, `chat.message`, `chat.params`, `chat.headers`, `permission.ask`, `command.execute.before`
- Experimental: `experimental.session.compacting`, `experimental.chat.messages.transform`, `experimental.chat.system.transform`
2. **Bus event catch-all** — generic `event` hook, receives `{ event }` where `event.type` is the event name:
- `session.created`, `session.compacted`, `session.deleted`, `session.idle`, `session.error`, `session.status`, `session.updated`, `session.diff`
- `message.updated`, `message.part.updated`, `message.part.removed`, `message.removed`
- `file.edited`, `file.watcher.updated`
- `command.executed`, `todo.updated`, `installation.updated`, `server.connected`
- `permission.asked`, `permission.replied`
- `lsp.client.diagnostics`, `lsp.updated`
- `tui.prompt.append`, `tui.command.execute`, `tui.toast.show`
- Total: **27 bus events** across **12 categories**
**Custom tool registration (CORRECTED — name is the key, not positional arg):**
```typescript
return {
tool: {
claude_mem_search: tool({
description: "Search claude-mem memory database",
args: { query: tool.schema.string() },
async execute(args, context) {
// context: { sessionID, messageID, agent, directory, worktree, abort, metadata, ask }
const response = await fetch(`http://localhost:37777/api/search?q=${encodeURIComponent(args.query)}`)
return await response.text()
},
}),
},
}
```
### What to implement
1. **Create OpenCode plugin** at `src/integrations/opencode-plugin/index.ts`:
- Export a `Plugin` function receiving full `PluginInput` context
- Use **direct interceptor** `tool.execute.after` for tool observation capture (gives `tool`, `args`, `output`)
- Use **bus event catch-all** `event` for session lifecycle:
| Mechanism | Event | Map to claude-mem |
|-----------|-------|-------------------|
| interceptor | `tool.execute.after` | `observation` (tool name + args + output) |
| bus event | `session.created` | `session-init` |
| bus event | `message.updated` | `observation` (assistant messages) |
| bus event | `session.compacted` | `summary` |
| bus event | `file.edited` | `observation` (file changes) |
| bus event | `session.deleted` | `session-end` |
- Register `claude_mem_search` custom tool using correct `tool({ description, args, execute })` API
- Hit `localhost:37777` API endpoints from the plugin
2. **Build the plugin** in the esbuild pipeline → `dist/opencode-plugin/index.js`
3. **Create OpenCode setup in installer** (two options, prefer file-based):
- **Option A (file-based):** Copy plugin to `~/.config/opencode/plugins/claude-mem.ts` (auto-loaded at startup)
- **Option B (npm-based):** Add to `~/.config/opencode/opencode.json` under `"plugin"` array: `["claude-mem"]`
- Config also supports JSONC (`opencode.jsonc`) and legacy `config.json`
- Context injection: Append to `~/.config/opencode/AGENTS.md` (or create it) with `<claude-mem-context>` tags
- Additional context via `"instructions"` config key (supports file paths, globs, remote URLs)
4. **Add OpenCode to installer IDE selection**
### OpenCode Verification
- `npx claude-mem install --ide opencode` registers the plugin (file or npm)
- OpenCode loads the plugin on next session
- `tool.execute.after` interceptor produces observations with `tool`, `args`, `output`
- Bus events (`session.created`, `session.deleted`) handle session lifecycle
- `claude_mem_search` custom tool works in OpenCode sessions
- Context is injected via AGENTS.md
### OpenCode Anti-patterns
- Do NOT try to use OpenCode's `session.diff` for full capture — it's a summary diff, not raw data
- Do NOT use `tool('name', schema, handler)` — wrong signature. Name is the key in the `tool:{}` map
- Do NOT assume bus events have the same `(input, output)` mutation pattern — they only receive `{ event }`
- OpenCode plugins run in Bun — the plugin CAN use Bun APIs (unlike the npx CLI itself)
- Do NOT hardcode `~/.config/opencode/` — respect `OPENCODE_CONFIG_DIR` env var if set
---
## Phase 5: Windsurf Integration (Tier 1 — Hook-Based)
**Why next**: 11 Cascade hooks, ~1M users. Hook architecture uses JSON stdin with a consistent envelope format.
### Verified Windsurf Hook Events (from docs.windsurf.com/windsurf/cascade/hooks)
**Naming pattern**: `pre_`/`post_` prefix + 5 action categories, plus 2 standalone post-only events.
| Event | Can Block? | Map to claude-mem | Use |
|-------|-----------|-------------------|-----|
| `pre_user_prompt` | Yes | `session-init` + `context` | Start session, inject context |
| `pre_read_code` | Yes | — | Skip (pre-execution, can block file reads) |
| `post_read_code` | No | — | Skip (too noisy, file reads are frequent) |
| `pre_write_code` | Yes | — | Skip (pre-execution, can block writes) |
| `post_write_code` | No | `observation` | Code generation |
| `pre_run_command` | Yes | — | Skip (pre-execution, can block commands) |
| `post_run_command` | No | `observation` | Shell command execution |
| `pre_mcp_tool_use` | Yes | — | Skip (pre-execution, can block MCP calls) |
| `post_mcp_tool_use` | No | `observation` | MCP tool results |
| `post_cascade_response` | No | `observation` | Full AI response |
| `post_setup_worktree` | No | — | Skip (informational) |
**Mapped**: 5 of 11 events (all post-action). **Skipped**: 4 pre-hooks (blocking-capable, pre-execution) + 2 low-value post-hooks.
### Verified Stdin Payload Schema
**Common envelope (all hooks):**
```json
{
"agent_action_name": "string",
"trajectory_id": "string",
"execution_id": "string",
"timestamp": "ISO 8601 string",
"tool_info": { /* event-specific payload */ }
}
```
**Event-specific `tool_info` payloads:**
| Event | `tool_info` fields |
|-------|-------------------|
| `pre_user_prompt` | `{ user_prompt: string }` |
| `pre_read_code` / `post_read_code` | `{ file_path: string }` |
| `pre_write_code` / `post_write_code` | `{ file_path: string, edits: [{ old_string: string, new_string: string }] }` |
| `pre_run_command` / `post_run_command` | `{ command_line: string, cwd: string }` |
| `pre_mcp_tool_use` | `{ mcp_server_name: string, mcp_tool_name: string, mcp_tool_arguments: {} }` |
| `post_mcp_tool_use` | `{ mcp_server_name: string, mcp_tool_name: string, mcp_tool_arguments: {}, mcp_result: string }` |
| `post_cascade_response` | `{ response: string }` (markdown) |
| `post_setup_worktree` | `{ worktree_path: string, root_workspace_path: string }` |
**Exit codes:** `0` = success, `2` = block (pre-hooks only; stderr shown to agent), any other = non-blocking warning.
### What to implement
1. **Create Windsurf platform adapter** at `src/cli/adapters/windsurf.ts`:
- Normalize Windsurf's hook input format to `NormalizedHookInput`
- Common envelope: `agent_action_name`, `trajectory_id`, `execution_id`, `timestamp`, `tool_info`
- Map: `trajectory_id` → `sessionId`, `tool_info` fields per event type
- For `post_write_code`: `tool_info.file_path` + `tool_info.edits` → file change observation
- For `post_run_command`: `tool_info.command_line` + `tool_info.cwd` → command observation
- For `post_mcp_tool_use`: `tool_info.mcp_tool_name` + `tool_info.mcp_tool_arguments` + `tool_info.mcp_result` → tool observation
- For `post_cascade_response`: `tool_info.response` → full AI response observation
2. **Create Windsurf hooks installer** at `src/services/integrations/WindsurfHooksInstaller.ts`:
- Write hooks to `~/.codeium/windsurf/hooks.json` (user-level, for global coverage)
- Per-workspace override at `.windsurf/hooks.json` if user chooses workspace-level install
- Config format (verified):
```json
{
"hooks": {
"post_write_code": [{
"command": "<path-to-hook-script>",
"show_output": false,
"working_directory": "<optional>"
}]
}
}
```
- Note: Tilde expansion (`~`) is NOT supported in `working_directory` — use absolute paths
- Merge order: cloud → system → user → workspace (all hooks at all levels execute)
- Context injection via `.windsurf/rules/claude-mem-context.md` (workspace-level; Windsurf rules are workspace-scoped)
- Rule limits: 6,000 chars per file, 12,000 chars total across all rules
3. **Register `windsurf` in `getPlatformAdapter()`** at `src/cli/adapters/index.ts`
4. **Add Windsurf to installer IDE selection**
### Windsurf Verification
- `npx claude-mem install --ide windsurf` creates hooks config at `~/.codeium/windsurf/hooks.json`
- Windsurf sessions are captured by the worker via post-action hooks
- `trajectory_id` is used as session identifier
- Context is injected via `.windsurf/rules/claude-mem-context.md` (under 6K char limit)
- Existing hooks.json is preserved (merge, not overwrite)
### Windsurf Anti-patterns
- Do NOT use fabricated event names (`post_search_code`, `post_lint_code`, `on_error`, `pre_tool_execution`) — they don't exist
- Do NOT assume Windsurf's stdin JSON matches Claude Code's — it uses `tool_info` envelope, not flat fields
- Do NOT use tilde (`~`) in `working_directory` — not supported, use absolute paths
- Do NOT exceed 6K chars in the context rule file — Windsurf truncates beyond that
- Pre-hooks can block actions (exit 2) — only use post-hooks for observation capture
---
## Phase 6: Codex CLI Integration (Tier 1 — Hook + Transcript)
### Dedup strategy
Codex has both a `notify` hook (real-time) and transcript files (complete history). Use **transcript watching only** — it's more complete and avoids the complexity of dual capture paths. The `notify` hook is a simpler mechanism that doesn't provide enough granularity to justify maintaining two integration paths. If transcript watching proves insufficient, add the notify hook later.
### What to implement
1. **Create Codex transcript schema** — the sample in `src/services/transcripts/config.ts` is already production-quality. Verify against current Codex CLI JSONL format and update if needed.
2. **Create Codex setup in installer**:
- Write transcript-watch config to `~/.claude-mem/transcript-watch.json`
- Set up watch for `~/.codex/sessions/**/*.jsonl` using existing CODEX_SAMPLE_SCHEMA
- Context injection via `.codex/AGENTS.md` (Codex reads this natively)
- Must merge with existing `config.toml` if it exists (read → parse → merge → write)
3. **Add Codex CLI to installer IDE selection**
### Verification
- `npx claude-mem install --ide codex` creates transcript watch config
- Codex sessions appear in claude-mem database
- `AGENTS.md` updated with context after sessions
- Existing `config.toml` is preserved
---
## Phase 7: OpenClaw Integration (Tier 1 — Plugin-Based)
**Plugin is already fully built** at `openclaw/src/index.ts` (~1000 lines). Has event hooks, SSE observation feed, MEMORY.md sync, slash commands. Only wiring into the installer is needed.
### What to implement
1. **Wire OpenClaw into the npx installer**:
- Detect `~/.openclaw/` directory
- Copy pre-built plugin from `openclaw/dist/` (built in Phase 2) to OpenClaw plugins location
- Register in `~/.openclaw/openclaw.json` under `plugins.claude-mem`
- Configure worker port, project name, syncMemoryFile
- Optionally prompt for observation feed setup (channel type + target ID)
2. **Add OpenClaw to IDE selection TUI** with hint about messaging channel support
### Verification
- `npx claude-mem install --ide openclaw` registers the plugin
- OpenClaw gateway loads the plugin on restart
- Observations are recorded from OpenClaw sessions
- MEMORY.md syncs to agent workspaces
### Anti-patterns
- Do NOT rebuild the OpenClaw plugin from source at install time — it ships pre-built from Phase 2
- Do NOT modify the plugin's event handling — it's battle-tested
---
## Phase 8: MCP-Based Integrations (Tier 2)
**These get the MCP server for free** — it already exists at `plugin/scripts/mcp-server.cjs`. The installer just needs to write the right config files per IDE.
MCP-only integrations provide: search tools + context injection. They do NOT capture transcripts or tool usage in real-time.
### What to implement
1. **Copilot CLI MCP setup**:
- Write MCP config to `~/.copilot/config` (merge, not overwrite)
- Context injection: `.github/copilot-instructions.md`
- Detection: `copilot` command in PATH
2. **Antigravity MCP setup**:
- Write MCP config to `~/.gemini/antigravity/mcp_config.json` (merge, not overwrite)
- Context injection: `~/.gemini/GEMINI.md` (shared with Gemini CLI) and/or `.agent/rules/claude-mem-context.md`
- Detection: `~/.gemini/antigravity/` exists
- Note: Antigravity has NO hook system — MCP is the only integration path
3. **Goose MCP setup**:
- Write MCP config to `~/.config/goose/config.yaml` (YAML merge — use a lightweight YAML parser or write the block manually if config doesn't exist)
- Detection: `~/.config/goose/` exists OR `goose` in PATH
- Note: Goose co-developed MCP with Anthropic, so MCP support is excellent
4. **Crush MCP setup**:
- Write MCP config to Crush's JSON config
- Detection: `crush` in PATH
5. **Roo Code MCP setup**:
- Write MCP config to `.roo/` or workspace settings
- Context injection: `.roo/rules/claude-mem-context.md`
- Detection: Check for VS Code extension directory containing `roo-code`
6. **Warp MCP setup**:
- Warp uses `WARP.md` in project root for context injection (similar to CLAUDE.md)
- MCP servers configured via Warp Drive UI, but also via config files
- Detection: `~/.warp/` exists OR `warp` in PATH
- Note: Warp is a terminal replacement (~26k stars), not just a CLI tool — multi-agent orchestration with management UI
7. **For each**: Add to installer IDE detection and selection
### Config merging strategy
JSON configs: Read → parse → deep merge → write back. YAML configs (Goose): If file exists, read and append the MCP block. If not, create from template. Avoid pulling in a full YAML parser library — write the MCP block as a string append with proper indentation if the format is predictable.
### Verification
- Each IDE can search claude-mem via MCP tools
- Context files are written to IDE-specific locations
- Existing configs are preserved
### Anti-patterns
- MCP-only integrations do NOT capture transcripts — don't claim "full integration"
- Do NOT overwrite existing config files — always merge
- Do NOT add a heavy YAML parser dependency for one integration
---
## Phase 9: Remove Old Installer
This is a **full replacement**, not a deprecation.
### What to implement
1. Remove `claude-mem-installer` npm package (unpublish or mark deprecated with message pointing to `npx claude-mem`)
2. Update `install/public/install.sh` → redirect to `npx claude-mem`
3. Remove `installer/` directory from the repository (it's replaced by `src/npx-cli/`)
4. Update docs site to reflect the new install command
5. Update README.md install instructions
---
## Phase 10: Final Verification
### All platforms (macOS, Linux, Windows)
1. `npm run build` succeeds, produces `dist/cli/index.js` and `openclaw/dist/index.js`
2. `node dist/cli/index.js install` works clean (no prior install)
3. Auto-detects installed IDEs correctly per platform
4. `npx claude-mem start/stop/status/search` all work
5. `npx claude-mem update` updates correctly
6. `npx claude-mem uninstall` cleans up all IDE configs
7. `npx claude-mem version` prints version
8. `npx claude-mem start` before install shows helpful error
9. No Bun dependency at install time
### Per-integration verification
| Integration | Type | Captures Sessions | Search via MCP | Context Injection |
|-------------|------|-------------------|----------------|-------------------|
| Claude Code | Plugin | Yes (hooks) | Yes | CLAUDE.md |
| Gemini CLI | Hooks | Yes (AfterTool, AfterAgent) | Yes (via hook) | GEMINI.md |
| OpenCode | Plugin | Yes (tool.execute.after, message.updated) | Yes (custom tool) | AGENTS.md / rules |
| Windsurf | Hooks | Yes (post_cascade_response, etc.) | Yes (via hook) | .windsurf/rules/ |
| Codex CLI | Transcript | Yes (JSONL watcher) | No (passive only) | .codex/AGENTS.md |
| OpenClaw | Plugin | Yes (event hooks) | Yes (slash commands) | MEMORY.md |
| Copilot CLI | MCP | No | Yes | copilot-instructions.md |
| Antigravity | MCP | No | Yes | .agent/rules/ |
| Goose | MCP | No | Yes | MCP context |
| Crush | MCP | No | Yes | Skills |
| Roo Code | MCP | No | Yes | .roo/rules/ |
| Warp | MCP | No | Yes | WARP.md |
---
## Priority Order & Impact
| Phase | IDE/Tool | Integration Type | Stars/Users | Effort |
|-------|----------|-----------------|-------------|--------|
| 1-2 | (infrastructure) | npx CLI + build pipeline | All users | Medium |
| 3 | Gemini CLI | Hooks (Tier 1) | ~95k stars | Medium (near-identical to Claude Code) |
| 4 | OpenCode | Plugin (Tier 1) | ~110k stars | Medium (rich plugin SDK) |
| 5 | Windsurf | Hooks (Tier 1) | ~1M users | Medium |
| 6 | Codex CLI | Transcript (Tier 3) | Growing (OpenAI) | Low (schema already exists) |
| 7 | OpenClaw | Plugin (Tier 1) — pre-built | ~196k stars | Low (wire into installer) |
| 8 | Copilot CLI, Antigravity, Goose, Crush, Warp, Roo Code | MCP (Tier 2) | 20M+ combined | Low per IDE |
| 9 | (remove old installer) | — | — | Low |
| 10 | (final verification) | — | — | Low |
## Out of Scope
- **Removing Bun as runtime dependency**: Worker still requires Bun for `bun:sqlite`. Runtime commands delegate to Bun; install commands don't need it.
- **JetBrains plugin**: Requires Kotlin/Java development — different ecosystem entirely.
- **Zed extension**: WASM sandbox limits feasibility.
- **Neovim/Emacs plugins**: Niche audiences, complex plugin ecosystems (Lua/Elisp). Could be added later via MCP (gptel supports it).
- **Amazon Q / Kiro**: Amazon Q Developer CLI has been sunsetted in favor of Kiro (proprietary, no public extensibility API yet). Revisit when Kiro opens up.
- **Aider**: Niche audience, writes Markdown transcripts (not JSONL), would require a markdown parser mode in the watcher. Add if demand materializes.
- **Continue.dev**: Small user base relative to other MCP tools. Can be added as a Tier 2 MCP integration later if requested.
- **Toad / Qwen Code / Oh-my-pi**: Too early-stage or too niche. Monitor for growth.
- **OpenClaw plugin development**: The plugin is already complete. Only installer wiring is in scope.
+241 -558
View File
@@ -2,6 +2,247 @@
All notable changes to claude-mem.
## [v10.5.6] - 2026-03-16
## Patch: Process Supervisor Hardening & Logging Cleanup
### Fixes
- **Downgrade HTTP request/response logging from INFO to DEBUG** — eliminates noisy per-request log spam from the viewer UI polling
- **Fix `isPidAlive(0)` returning true** — PID 0 is the kernel scheduler, not a valid child process
- **Fix signal handler race condition** — added `shutdownInitiated` flag to prevent duplicate shutdown cascades when signals arrive before `stopPromise` is set
- **Remove unused `dataDir` parameter** from `ShutdownCascadeOptions`
- **Export and reuse env sanitizer constants** — `Server.ts` now imports `ENV_PREFIXES`/`ENV_EXACT_MATCHES` from `env-sanitizer.ts` instead of duplicating them
- **Rename `zombiePidFiles` to `deadProcessPids`** — now returns actual PID array instead of a boolean
- **Use `buildWorkerUrl` helper** in `workerHttpRequest` instead of inline URL construction
- **Remove unused `getWorkerPort` imports** from observation and session-init handlers
- **Upgrade `reapSession` failure log** from debug to warn level
- **Clean up `.gitignore`** — remove stale `~*/`, `http*/`, `https*/` patterns and duplicate `datasets/` entry
### Tests
- Rewrote supervisor index tests to use temp directories instead of relying on real `~/.claude-mem/worker.pid`
- Added deterministic test cases for missing, invalid, stale, and alive PID file states
- Removed unused `dataDir` from shutdown test fixtures
## [v10.5.5] - 2026-03-09
### Bug Fix
- **Fixed empty context queries after mode switching**: Switching from a non-code mode (e.g., law-study) back to code mode left stale observation type/concept filters in `settings.json`, causing all context queries to return empty results. All modes now read types/concepts from their mode JSON definition uniformly.
### Cleanup
- Removed dead `CLAUDE_MEM_CONTEXT_OBSERVATION_TYPES` and `CLAUDE_MEM_CONTEXT_OBSERVATION_CONCEPTS` settings constants
- Deleted `src/constants/observation-metadata.ts` (no longer needed)
- Removed observation type/concept filter UI controls from the viewer's Context Settings modal
## [v10.5.4] - 2026-03-09
## Bug Fixes
- **fix: restore modes to correct location** — All modes (`code`, code language variants, `email-investigation`) were erroneously moved from `plugin/modes/` to `plugin/hooks/modes/` during the v10.5.3 release, breaking mode loading. This patch restores them to `plugin/modes/` where they belong.
## [v10.5.3] - 2026-03-09
## What's New
### Law Study Mode
Adds `law-study` — a purpose-built claude-mem mode for law students.
**Observation Types:**
- **Case Holding** — 2-3 sentence brief with extracted legal rule
- **Issue Pattern** — exam trigger or fact pattern that signals a legal issue
- **Prof Framework** — professor's analytical lens and emphasis for a topic
- **Doctrine / Rule** — legal test or standard synthesized from cases/statutes
- **Argument Structure** — legal argument or counter-argument worked through analytically
- **Cross-Case Connection** — insight linking cases or doctrines to reveal a deeper principle
**Concepts (cross-cutting tags):**
`exam-relevant` · `minority-position` · `gotcha` · `unsettled-law` · `policy-rationale` · `course-theme`
**Chill Variant**`law-study--chill` records only high-signal items: issue patterns, gotchas, and professor frameworks. Skips routine case holdings unless the result is counterintuitive.
**CLAUDE.md Template**`law-study-CLAUDE.md` is a drop-in template for any law study project directory. It configures Claude as a Socratic legal study partner: precise case briefs, critical document analysis, issue spotting, and doctrine synthesis — without writing exam answers for the student.
Activate with: `/mode law-study` or `/mode law-study--chill`
## [v10.5.2] - 2026-02-26
## Smart Explore Benchmark Docs & Skill Update
### Documentation
- Published smart-explore benchmark report to public docs — full A/B comparison with methodology, raw data tables, quality assessment, and decision framework
- Added benchmark report to docs.json navigation under Best Practices
### Smart Explore Skill
- Updated token economics with benchmark-accurate data (11-18x savings on exploration, 4-8x on file understanding)
- Added "map first" core principle as decision heuristic for tool selection
- Added AST completeness guarantee to smart_unfold documentation (never truncates, unlike Explore agents)
- Added Explore agent escalation guidance for multi-file synthesis tasks
- Updated smart_unfold token range from ~1-7k to ~400-2,100 based on measurements
- Updated Explore agent token range from ~20-40k to ~39-59k based on measurements
## [v10.5.1] - 2026-02-26
### Bug Fix
- Restored hooks.json to pre-smart-explore configuration (re-adds Setup hook, separate worker start command, PostToolUse matcher)
## [v10.5.0] - 2026-02-26
## Smart Explore: AST-Powered Code Navigation
This release introduces **Smart Explore**, a token-optimized structural code search system built on tree-sitter AST parsing. It applies the same progressive disclosure pattern used in human-readable code outlines — but programmatically, for AI agents.
### Why This Matters
The standard exploration cycle (Glob → Grep → Read) forces agents to consume entire files to understand code structure. A typical 800-line file costs ~12,000 tokens to read. Smart Explore replaces this with a 3-layer progressive disclosure workflow that delivers the same understanding at **6-12x lower token cost**.
### 3 New MCP Tools
- **`smart_search`** — Walks directories, parses all code files via tree-sitter, and returns ranked symbols with signatures and line numbers. Replaces the Glob → Grep discovery cycle in a single call (~2-6k tokens).
- **`smart_outline`** — Returns the complete structural skeleton of a file: all functions, classes, methods, properties, imports (~1-2k tokens vs ~12k for a full Read).
- **`smart_unfold`** — Expands a single symbol to its full source code including JSDoc, decorators, and implementation (~1-7k tokens).
### Token Economics
| Approach | Tokens | Savings |
|----------|--------|---------|
| smart_outline + smart_unfold | ~3,100 | 8x vs Read |
| smart_search (cross-file) | ~2,000-6,000 | 6-12x vs Explore agent |
| Read (full file) | ~12,000+ | baseline |
| Explore agent | ~20,000-40,000 | baseline |
### Language Support
10 languages via tree-sitter grammars: TypeScript, JavaScript, Python, Rust, Go, Java, C, C++, Ruby, PHP.
### Other Changes
- Simplified hooks configuration
- Removed legacy setup.sh script
- Security fix: replaced `execSync` with `execFileSync` to prevent command injection in file path handling
## [v10.4.4] - 2026-02-26
## Fix
- **Remove `save_observation` from MCP tool surface** — This tool was exposed as an MCP tool available to Claude, but it's an internal API-only feature. Removing it from the MCP server prevents unintended tool invocation and keeps the tool surface clean.
## [v10.4.3] - 2026-02-25
## Bug Fixes
- **Fix PostToolUse hook crashes and 5-second latency (#1220)**: Added missing `break` statements to all 7 switch cases in `worker-service.ts` preventing fall-through execution, added `.catch()` on `main()` to handle unhandled promise rejections, and removed redundant `start` commands from hook groups that triggered the 5-second `collectStdin()` timeout
- **Fix CLAUDE_PLUGIN_ROOT fallback for Stop hooks (#1215)**: Added POSIX shell-level `CLAUDE_PLUGIN_ROOT` fallback in `hooks.json` for environments where the variable isn't injected, added script-level self-resolution via `import.meta.url` in `bun-runner.js`, and regression test added in `plugin-distribution.test.ts`
## Maintenance
- Synced all version files (plugin.json was stuck at 10.4.0)
## [v10.4.2] - 2026-02-25
## Bug Fixes
- **Fix PostToolUse hook crashes and 5-second latency (#1220)**: Added missing `break` statements to all 7 switch cases in `worker-service.ts` preventing fall-through execution, added `.catch()` on `main()` to handle unhandled promise rejections, and removed redundant `start` commands from hook groups that triggered the 5-second `collectStdin()` timeout
- **Fix CLAUDE_PLUGIN_ROOT fallback for Stop hooks (#1215)**: Added POSIX shell-level `CLAUDE_PLUGIN_ROOT` fallback in `hooks.json` for environments where the variable isn't injected, added script-level self-resolution via `import.meta.url` in `bun-runner.js`, and regression test added in `plugin-distribution.test.ts`
- **Sync plugin.json version**: Fixed `plugin.json` being stuck at 10.4.0 while other version files were at 10.4.1
## [v10.4.1] - 2026-02-24
### Refactor
- **Skills Conversion**: Converted `/make-plan` and `/do` commands into first-class skills in `plugin/skills/`.
- **Organization**: Centralized planning and execution instructions alongside `mem-search`.
- **Compatibility**: Added symlinks for `openclaw/skills/` to ensure seamless integration with OpenClaw.
### Chore
- **Version Bump**: Aligned all package and plugin manifests to v10.4.1.
## [v10.4.0] - 2026-02-24
## v10.4.0 — Stability & Platform Hardening
Massive reliability release: 30+ root-cause bug fixes across 10 triage phases, plus new features for agent attribution, Chroma control, and broader platform support.
### New Features
- **Session custom titles** — Agents can now set `custom_title` on sessions for attribution (migration 23, new endpoint)
- **Chroma toggle** — `CLAUDE_MEM_CHROMA_ENABLED` setting allows SQLite-only fallback mode (#707)
- **Plugin disabled state** — Early exit check in all hook entry points when plugin is disabled (#781)
- **Context re-injection guard** — `contextInjected` session flag prevents re-injecting context on every UserPromptSubmit turn (#1079)
### Bug Fixes
#### Data Integrity
- SHA-256 content-hash deduplication on observation INSERT (migration 22 with backfill + index)
- Project name collision fix: `getCurrentProjectName()` now returns `parent/basename`
- Empty project string guard with cwd-derived fallback
- Stuck `isProcessing` reset: pending work older than 5 minutes auto-clears
#### ChromaDB
- Python version pinning in uvx args for both local and remote mode (#1196, #1206, #1208)
- Windows backslash-to-forward-slash path conversion for `--data-dir` (#1199)
- Metadata sanitization: filter null/undefined/empty values in `addDocuments()` (#1183, #1188)
- Transport error auto-reconnect in `callTool()` (#1162)
- Stale transport retry with transparent reconnect (#1131)
#### Hook Lifecycle
- Suppress `process.stderr.write` in `hookCommand()` to prevent diagnostic output showing as error UI (#1181)
- Route all `console.error()` through logger instead of stderr
- Verified all 7 handlers return `suppressOutput: true` (#598, #784)
#### Worker Lifecycle
- PID file mtime guard prevents concurrent restart storms (#1145)
- `getInstalledPluginVersion()` ENOENT/EBUSY handling (#1042)
#### SQLite Migrations
- Schema initialization always creates core tables via `CREATE TABLE IF NOT EXISTS`
- Migrations 5-7 check actual DB state instead of version tracking (fixes version collision between old/new migration systems, #979)
- Crash-safe temp table rebuilds
#### Platform Support
- **Windows**: `cmd.exe /c` uvx spawn, PowerShell `$_` elimination with WQL filtering, `windowsHide: true`, FTS5 runtime probe with fallback (#1190, #1192, #1199, #1024, #1062, #1048, #791)
- **Cursor IDE**: Adapter field fallbacks, tolerant session-init validation (#838, #1049)
- **Codex CLI**: `session_id` fallbacks, unknown platform tolerance, undefined guard (#744)
#### API & Infrastructure
- `/api/logs` OOM fix: tail-read replaces full-file `readFileSync` (64KB expanding chunks, 10MB cap, #1203)
- CORS: explicit methods and allowedHeaders (#1029)
- MCP type coercion for batch endpoints: string-to-array for `ids` and `memorySessionIds`
- Defensive observation error handling returns 200 on recoverable errors instead of 500
- `.git/` directory write guard on all 4 CLAUDE.md/AGENTS.md write sites (#1165)
#### Stale AbortController Fix
- `lastGeneratorActivity` timestamp tracking with 30s timeout (#1099)
- Stale generator detection + abort + restart in `ensureGeneratorRunning`
- `AbortSignal.timeout(30000)` in `deleteSession` prevents indefinite hang
### Installation
- `resolveRoot()` replaces hardcoded marketplace path using `CLAUDE_PLUGIN_ROOT` env var (#1128, #1166)
- `installCLI()` path correction and `verifyCriticalModules()` post-install check
- Build-time distribution verification for skills, hooks, and plugin manifest (#1187)
### Testing
- 50+ new tests across hook lifecycle, context re-injection, plugin distribution, migration runner, data integrity, stale abort controller, logs tail-read, CORS, MCP type coercion, and smart-install
- 68 files changed, ~4200 insertions, ~900 deletions
## [v10.3.3] - 2026-02-23
### Bug Fixes
- Fixed session context footer to reference the claude-mem skill instead of MCP search tools for accessing memories
## [v10.3.2] - 2026-02-23
## Bug Fixes
- **Worker startup readiness**: Worker startup hook now waits for full DB/search readiness before proceeding, fixing the race condition where hooks would fire before the worker was initialized on first start (#1210)
- **MCP tool naming**: Renamed `save_memory` to `save_observation` for consistency with the observation-based data model (#1210)
- **MCP search instructions**: Updated MCP server tool descriptions to accurately reflect the 3-layer search workflow (#1210)
- **Installer hosting**: Serve installer JS from install.cmem.ai instead of GitHub raw URLs for reliability
- **Installer routing**: Added rewrite rule so install.cmem.ai root path correctly serves the install script
- **Installer build**: Added compiled installer dist so CLI installation works out of the box
## [v10.3.1] - 2026-02-19
## Fix: Prevent Duplicate Worker Daemons and Zombie Processes
@@ -893,561 +1134,3 @@ Fixed an issue where the worker service startup wasn't producing proper JSON sta
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v9.0.0...v9.0.1
## [v9.0.0] - 2026-01-06
## 🚀 Live Context System
Version 9.0.0 introduces the **Live Context System** - a major new capability that provides folder-level activity context through auto-generated CLAUDE.md files.
### ✨ New Features
#### Live Context System
- **Folder CLAUDE.md Files**: Each directory now gets an auto-generated CLAUDE.md file containing a chronological timeline of recent development activity
- **Activity Timelines**: Tables show observation ID, time, type, title, and estimated token cost for relevant work in each folder
- **Worktree Support**: Proper detection of git worktrees with project-aware filtering to show only relevant observations per worktree
- **Configurable Limits**: Control observation count via `CLAUDE_MEM_CONTEXT_OBSERVATIONS` setting
#### Modular Architecture Refactor
- **Service Layer Decomposition**: Major refactoring from monolithic worker-service to modular domain services
- **SQLite Module Extraction**: Database operations split into dedicated modules (observations, sessions, summaries, prompts, timeline)
- **Context Builder System**: New modular context generation with TimelineRenderer, FooterRenderer, and ObservationCompiler
- **Error Handler Centralization**: Unified Express error handling via ErrorHandler module
#### SDK Agent Improvements
- **Session Resume**: Memory sessions can now resume across Claude conversations using SDK session IDs
- **Memory Session ID Tracking**: Proper separation of content session IDs from memory session IDs
- **Response Processor Refactor**: Cleaner message handling and observation extraction
### 🔧 Improvements
#### Windows Stability
- Fixed Windows PowerShell variable escaping in hook execution
- Improved IPC detection for Windows managed mode
- Better PATH handling for Bun and uv on Windows
#### Settings & Configuration
- **Auto-Creation**: Settings file automatically created with defaults on first run
- **Worker Host Configuration**: `CLAUDE_MEM_WORKER_HOST` setting for custom worker endpoints
- Settings validation with helpful error messages
#### MCP Tools
- Standardized naming: "MCP tools" terminology instead of "mem-search skill"
- Improved tool descriptions for better Claude integration
- Context injection API now supports worktree parameter
### 📚 Documentation
- New **Folder Context Files** documentation page
- **Worktree Support** section explaining git worktree behavior
- Updated architecture documentation reflecting modular refactor
- v9.0 release notes in introduction page
### 🐛 Bug Fixes
- Fixed stale session resume crash when SDK session is orphaned
- Fixed logger serialization bug causing silent ChromaSync failures
- Fixed CLAUDE.md path resolution in worktree environments
- Fixed date preservation in folder timeline generation
- Fixed foreign key constraint issues in observation storage
- Resolved multiple TypeScript type errors across codebase
### 🗑️ Removed
- Deprecated context-generator.ts (functionality moved to modular system)
- Obsolete queue analysis documents
- Legacy worker wrapper scripts
---
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.5.10...v9.0.0
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## [v8.5.10] - 2026-01-06
## Bug Fixes
- **#545**: Fixed `formatTool` crash when parsing non-JSON tool inputs (e.g., raw Bash commands)
- **#544**: Fixed terminology in context hints - changed "mem-search skill" to "MCP tools"
- **#557**: Settings file now auto-creates with defaults on first run (no more "module loader" errors)
- **#543**: Fixed hook execution by switching runtime from `node` to `bun` (resolves `bun:sqlite` issues)
## Code Quality
- Fixed circular dependency between Logger and SettingsDefaultsManager
- Added 72 integration tests for critical coverage gaps
- Cleaned up mock-heavy tests causing module cache pollution
## Full Changelog
See PR #558 for complete details and diagnostic reports.
## [v8.5.9] - 2026-01-04
## What's New
### Context Header Timestamp
The context injection header now displays the current date and time, making it easier to understand when context was generated.
**Example:** `[claude-mem] recent context, 2026-01-04 2:46am EST`
This appears in both terminal (colored) output and markdown format, including empty state messages.
---
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.5.8...v8.5.9
## [v8.5.8] - 2026-01-04
## Bug Fixes
- **#511**: Add `gemini-3-flash` model to GeminiAgent with proper rate limits and validation
- **#517**: Fix Windows process management by replacing PowerShell with WMIC (fixes Git Bash/WSL compatibility)
- **#527**: Add Apple Silicon Homebrew paths (`/opt/homebrew/bin`) for `bun` and `uv` detection
- **#531**: Remove duplicate type definitions from `export-memories.ts` using shared bridge file
## Tests
- Added regression tests for PR #542 covering Gemini model support, WMIC parsing, Apple Silicon paths, and export type refactoring
## Documentation
- Added detailed analysis reports for GitHub issues #511, #514, #517, #520, #527, #531, #532
## [v8.5.7] - 2026-01-04
## Modular Architecture Refactor
This release refactors the monolithic service architecture into focused, single-responsibility modules with comprehensive test coverage.
### Architecture Improvements
- **SQLite Repositories** (`src/services/sqlite/`) - Modular repositories for sessions, observations, prompts, summaries, and timeline
- **Worker Agents** (`src/services/worker/agents/`) - Extracted response processing, error handling, and session cleanup
- **Search Strategies** (`src/services/worker/search/`) - Modular search with Chroma, SQLite, and Hybrid strategies plus orchestrator
- **Context Generation** (`src/services/context/`) - Separated context building, token calculation, formatters, and renderers
- **Infrastructure** (`src/services/infrastructure/`) - Graceful shutdown, health monitoring, and process management
- **Server** (`src/services/server/`) - Express server setup, middleware, and error handling
### Test Coverage
- **595 tests** across 36 test files
- **1,120 expect() assertions**
- Coverage for SQLite repos, worker agents, search, context, infrastructure, and server modules
### Session ID Refactor
- Aligned tests with NULL-based memory session initialization pattern
- Updated `SESSION_ID_ARCHITECTURE.md` documentation
### Other Improvements
- Added missing logger imports to 34 files for better observability
- Updated esbuild and MCP SDK to latest versions
- Removed `bun.lock` from version control
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.5.6...v8.5.7
## [v8.5.6] - 2026-01-04
## Major Architectural Refactoring
Decomposes monolithic services into modular, maintainable components:
### Worker Service
Extracted infrastructure (GracefulShutdown, HealthMonitor, ProcessManager), server layer (ErrorHandler, Middleware, Server), and integrations (CursorHooksInstaller)
### Context Generator
Split into ContextBuilder, ContextConfigLoader, ObservationCompiler, TokenCalculator, formatters (Color/Markdown), and section renderers (Header/Footer/Summary/Timeline)
### Search System
Extracted SearchOrchestrator, ResultFormatter, TimelineBuilder, and strategy pattern (Chroma/SQLite/Hybrid search strategies) with dedicated filters (Date/Project/Type)
### Agent System
Extracted shared logic into ResponseProcessor, ObservationBroadcaster, FallbackErrorHandler, and SessionCleanupHelper
### SQLite Layer
Decomposed SessionStore into domain modules (observations, prompts, sessions, summaries, timeline) with proper type exports
## Bug Fixes
- Fixed duplicate observation storage bug (observations stored multiple times when messages were batched)
- Added duplicate observation cleanup script for production database remediation
- Fixed FOREIGN KEY constraint and missing `failed_at_epoch` column issues
## Coming Next
Comprehensive test suite in a new PR, targeting **v8.6.0**
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## [v8.5.5] - 2026-01-03
## Improved Error Handling and Logging
This patch release enhances error handling and logging across all worker services for better debugging and reliability.
### Changes
- **Enhanced Error Logging**: Improved error context across SessionStore, SearchManager, SDKAgent, GeminiAgent, and OpenRouterAgent
- **SearchManager**: Restored error handling for Chroma calls with improved logging
- **SessionStore**: Enhanced error logging throughout database operations
- **Bug Fix**: Fixed critical bug where `memory_session_id` could incorrectly equal `content_session_id`
- **Hooks**: Streamlined error handling and loading states for better maintainability
### Investigation Reports
- Added detailed analysis documents for generator failures and observation duplication regressions
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.5.4...v8.5.5
## [v8.5.4] - 2026-01-02
## Bug Fixes
### Chroma Connection Error Handling
Fixed a critical bug in ChromaSync where connection-related errors were misinterpreted as missing collections. The `ensureCollection()` method previously caught ALL errors and assumed they meant the collection doesn't exist, which caused connection errors to trigger unnecessary collection creation attempts. Now connection-related errors like "Not connected" are properly distinguished and re-thrown immediately, preventing false error handling paths and inappropriate fallback behavior.
### Removed Dead last_user_message Code
Cleaned up dead code related to `last_user_message` handling in the summary flow. This field was being extracted from transcripts but never used anywhere - in Claude Code transcripts, "user" type messages are mostly tool_results rather than actual user input, and the user's original request is already stored in the user_prompts table. Removing this unused field eliminates confusing warnings like "Missing last_user_message when queueing summary". Changes span summary-hook, SessionRoutes, SessionManager, interface definitions, and all agent implementations.
## Improvements
### Enhanced Error Handling Across Services
Comprehensive improvement to error handling across 8 core services:
- **BranchManager** - Now logs recovery checkout failures
- **PaginationHelper** - Logs when file paths are plain strings instead of valid JSON
- **SDKAgent** - Enhanced logging for Claude executable detection failures
- **SearchManager** - Logs plain string handling for files read and edited
- **paths.ts** - Improved logging for git root detection failures
- **timeline-formatting** - Enhanced JSON parsing errors with input previews
- **transcript-parser** - Logs summary of parse errors after processing
- **ChromaSync** - Logs full error context before attempting collection creation
### Error Handling Documentation & Tooling
- Created `error-handling-baseline.txt` establishing baseline error handling practices
- Documented error handling anti-pattern rules in CLAUDE.md
- Added `detect-error-handling-antipatterns.ts` script to identify empty catch blocks, improper logging practices, and oversized try-catch blocks
## New Features
### Console Filter Bar with Log Parsing
Implemented interactive log filtering in the viewer UI:
- **Structured Log Parsing** - Extracts timestamp, level, component, correlation ID, and message content using regex pattern matching
- **Level Filtering** - Toggle visibility for DEBUG, INFO, WARN, ERROR log levels
- **Component Filtering** - Filter by 9 component types: HOOK, WORKER, SDK, PARSER, DB, SYSTEM, HTTP, SESSION, CHROMA
- **Color-Coded Rendering** - Visual distinction with component-specific icons and log level colors
- **Special Message Detection** - Recognizes markers like → (dataIn), ← (dataOut), ✓ (success), ✗ (failure), ⏱ (timing), [HAPPY-PATH]
- **Smart Auto-Scroll** - Maintains scroll position when reviewing older logs
- **Responsive Design** - Filter bar adapts to smaller screens
## [v8.5.3] - 2026-01-02
# 🛡️ Error Handling Hardening & Developer Tools
Version 8.5.3 introduces comprehensive error handling improvements that prevent silent failures and reduce debugging time from hours to minutes. This release also adds new developer tools for queue management and log monitoring.
---
## 🔴 Critical Error Handling Improvements
### The Problem
A single overly-broad try-catch block caused a **10-hour debugging session** by silently swallowing errors. This pattern was pervasive throughout the codebase, creating invisible failure modes.
### The Solution
**Automated Anti-Pattern Detection** (`scripts/detect-error-handling-antipatterns.ts`)
- Detects 7 categories of error handling anti-patterns
- Enforces zero-tolerance policy for empty catch blocks
- Identifies large try-catch blocks (>10 lines) that mask specific errors
- Flags missing error logging that causes silent failures
- Supports approved overrides with justification comments
- Exit code 1 if critical issues detected (enforceable in CI)
**New Error Handling Standards** (Added to `CLAUDE.md`)
- **5-Question Pre-Flight Checklist**: Required before writing any try-catch
1. What SPECIFIC error am I catching?
2. Show documentation proving this error can occur
3. Why can't this error be prevented?
4. What will the catch block DO?
5. Why shouldn't this error propagate?
- **Forbidden Patterns**: Empty catch, catch without logging, large try blocks, promise catch without handlers
- **Allowed Patterns**: Specific errors, logged failures, minimal scope, explicit recovery
- **Meta-Rule**: Uncertainty triggers research, NOT try-catch
### Fixes Applied
**Wave 1: Empty Catch Blocks** (5 files)
- `import-xml-observations.ts` - Log skipped invalid JSON
- `bun-path.ts` - Log when bun not in PATH
- `cursor-utils.ts` - Log failed registry reads & corrupt MCP config
- `worker-utils.ts` - Log failed health checks
**Wave 2: Promise Catches on Critical Paths** (8 locations)
- `worker-service.ts` - Background initialization failures
- `SDKAgent.ts` - Session processor errors (2 locations)
- `GeminiAgent.ts` - Finalization failures (2 locations)
- `OpenRouterAgent.ts` - Finalization failures (2 locations)
- `SessionManager.ts` - Generator promise failures
**Wave 3: Comprehensive Audit** (29 catch blocks)
- Added logging to 16 catch blocks (UI, servers, worker, routes, services)
- Documented 13 intentional exceptions with justification comments
- All patterns now follow error handling guidelines with appropriate log levels
### Approved Override System
For justified exceptions (performance-critical paths, expected failures), use:
```typescript
// [APPROVED OVERRIDE]: Brief technical justification
try {
// code
} catch {
// allowed exception
}
```
**Progress**: 163 anti-patterns → 26 approved overrides (84% reduction in silent failures)
---
## 🗂️ Queue Management Features
**New Commands**
- `npm run queue:clear` - Interactive removal of failed messages
- `npm run queue:clear -- --all` - Clear all messages (pending, processing, failed)
- `npm run queue:clear -- --force` - Non-interactive mode
**HTTP API Endpoints**
- `DELETE /api/pending-queue/failed` - Remove failed messages
- `DELETE /api/pending-queue/all` - Complete queue reset
Failed messages exceed max retry count and remain for debugging. These commands provide clean queue maintenance.
---
## 🪵 Developer Console (Chrome DevTools Style)
**UI Improvements**
- Bottom drawer console (slides up from bottom-left corner)
- Draggable resize handle for height adjustment
- Auto-refresh toggle (2s interval)
- Clear logs button with confirmation
- Monospace font (SF Mono/Monaco/Consolas)
- Minimum height: 150px, adjustable to window height - 100px
**API Endpoints**
- `GET /api/logs` - Fetch last 1000 lines of current day's log
- `DELETE /api/logs` - Clear current log file
Logs viewer accessible via floating console button in UI.
---
## 📚 Architecture Documentation
**Session ID Architecture** (`docs/SESSION_ID_ARCHITECTURE.md`)
- Comprehensive documentation of 1:1 session mapping guarantees
- 19 validation tests proving UNIQUE constraints and resume consistency
- Documents single-transition vulnerability (application-level enforcement)
- Complete reference for session lifecycle management
---
## 📊 Impact Summary
- **Debugging Time**: 10 hours → minutes (proper error visibility)
- **Test Coverage**: +19 critical architecture validation tests
- **Silent Failures**: 84% reduction (163 → 26 approved exceptions)
- **Protection**: Automated detection prevents regression
- **Developer UX**: Console logs, queue management, comprehensive docs
---
## 🔧 Technical Details
**Files Changed**: 25+ files across error handling, queue management, UI, and documentation
**Critical Path Protection**
These files now have strict error propagation (no catch-and-continue):
- `SDKAgent.ts`
- `GeminiAgent.ts`
- `OpenRouterAgent.ts`
- `SessionStore.ts`
- `worker-service.ts`
**Build Verification**: All changes tested, build successful
---
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.5.2...v8.5.3
## [v8.5.2] - 2025-12-31
## Bug Fixes
### Fixed SDK Agent Memory Leak (#499)
Fixed a critical memory leak where Claude SDK child processes were never terminated after sessions completed. Over extended usage, this caused hundreds of orphaned processes consuming 40GB+ of RAM.
**Root Cause:**
- When the SDK agent generator completed naturally (no more messages to process), the `AbortController` was never aborted
- Child processes spawned by the Agent SDK remained running indefinitely
- Sessions stayed in memory (by design for future events) but underlying processes were never cleaned up
**Fix:**
- Added proper cleanup to SessionRoutes finally block
- Now calls `abortController.abort()` when generator completes with no pending work
- Creates new `AbortController` when crash recovery restarts generators
- Ensures cleanup happens even if recovery logic fails
**Impact:**
- Prevents orphaned `claude` processes from accumulating
- Eliminates multi-gigabyte memory leaks during normal usage
- Maintains crash recovery functionality with proper resource cleanup
Thanks to @yonnock for the detailed bug report and investigation in #499!
## [v8.5.1] - 2025-12-30
## Bug Fix
**Fixed**: Migration 17 column rename failing for databases in intermediate states (#481)
### Problem
Migration 17 renamed session ID columns but used a single check to determine if ALL tables were migrated. This caused errors for databases in partial migration states:
- `no such column: sdk_session_id` (when columns already renamed)
- `table observations has no column named memory_session_id` (when not renamed)
### Solution
- Rewrote migration 17 to check **each table individually** before renaming
- Added `safeRenameColumn()` helper that handles all edge cases gracefully
- Handles all database states: fresh, old, and partially migrated
### Who was affected
- Users upgrading from pre-v8.2.6 versions
- Users whose migration was interrupted (crash, restart, etc.)
- Users who restored database from backup
---
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## [v8.5.0] - 2025-12-30
# Cursor Support Now Available 🎉
This is a major release introducing **full Cursor IDE support**. Claude-mem now works with Cursor, bringing persistent AI memory to Cursor users with or without a Claude Code subscription.
## Highlights
**Give Cursor persistent memory.** Every Cursor session starts fresh - your AI doesn't remember what it worked on yesterday. Claude-mem changes that. Your agent builds cumulative knowledge about your codebase, decisions, and patterns over time.
### Works Without Claude Code
You can now use claude-mem with Cursor using free AI providers:
- **Gemini** (recommended): 1,500 free requests/day, no credit card required
- **OpenRouter**: Access to 100+ models including free options
- **Claude SDK**: For Claude Code subscribers
### Cross-Platform Support
Full support for all major platforms:
- **macOS**: Bash scripts with `jq` and `curl`
- **Linux**: Same toolchain as macOS
- **Windows**: Native PowerShell scripts, no WSL required
## New Features
### Interactive Setup Wizard (`bun run cursor:setup`)
A guided installer that:
- Detects your environment (Claude Code present or not)
- Helps you choose and configure an AI provider
- Installs Cursor hooks automatically
- Starts the worker service
- Verifies everything is working
### Cursor Lifecycle Hooks
Complete hook integration with Cursor's native hook system:
- `session-init.sh/.ps1` - Session start with context injection
- `user-message.sh/.ps1` - User prompt capture
- `save-observation.sh/.ps1` - Tool usage logging
- `save-file-edit.sh/.ps1` - File edit tracking
- `session-summary.sh/.ps1` - Session end summary
- `context-inject.sh/.ps1` - Load relevant history
### Context Injection via `.cursor/rules`
Relevant past context is automatically injected into Cursor sessions via the `.cursor/rules/claude-mem-context.mdc` file, giving your AI immediate awareness of prior work.
### Project Registry
Multi-project support with automatic project detection:
- Projects registered in `~/.claude-mem/cursor-projects.json`
- Context automatically scoped to current project
- Works across multiple workspaces simultaneously
### MCP Search Tools
Full MCP server integration for Cursor:
- `search` - Find observations by query, date, type
- `timeline` - Get context around specific observations
- `get_observations` - Fetch full details for filtered IDs
## New Commands
| Command | Description |
|---------|-------------|
| `bun run cursor:setup` | Interactive setup wizard |
| `bun run cursor:install` | Install Cursor hooks |
| `bun run cursor:uninstall` | Remove Cursor hooks |
| `bun run cursor:status` | Check hook installation status |
## Documentation
Full documentation available at [docs.claude-mem.ai/cursor](https://docs.claude-mem.ai/cursor):
- Cursor Integration Overview
- Gemini Setup Guide (free tier)
- OpenRouter Setup Guide
- Troubleshooting
## Getting Started
### For Cursor-Only Users (No Claude Code)
```bash
git clone https://github.com/thedotmack/claude-mem.git
cd claude-mem && bun install && bun run build
bun run cursor:setup
```
### For Claude Code Users
```bash
/plugin marketplace add thedotmack/claude-mem
/plugin install claude-mem
claude-mem cursor install
```
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.2.10...v8.5.0
## [v8.2.10] - 2025-12-30
## Bug Fixes
- **Auto-restart worker on version mismatch** (#484): When the plugin updates but the worker was already running on the old version, the worker now automatically restarts instead of failing with 400 errors.
### Changes
- `/api/version` endpoint now returns the built-in version (compiled at build time) instead of reading from disk
- `worker-service start` command checks for version mismatch and auto-restarts if needed
- Downgraded hook version mismatch warning to debug logging (now handled by auto-restart)
Thanks @yungweng for the detailed bug report!
## [v8.2.9] - 2025-12-29
## Bug Fixes
- **Worker Service**: Remove file-based locking and improve Windows stability
- Replaced file-based locking with health-check-first approach for cleaner mutual exclusion
- Removed AbortSignal.timeout() calls to reduce Bun libuv assertion errors on Windows
- Added 500ms shutdown delays on Windows to prevent zombie ports
- Reduced hook timeout values for improved responsiveness
- Increased worker readiness polling duration from 5s to 15s
## Internal Changes
- Updated worker CLI scripts to reference worker-service.cjs directly
- Simplified hook command configurations
+4
View File
@@ -14,6 +14,10 @@ Claude-mem is a Claude Code plugin providing persistent memory across sessions.
**Search Skill** (`plugin/skills/mem-search/SKILL.md`) - HTTP API for searching past work, auto-invoked when users ask about history
**Planning Skill** (`plugin/skills/make-plan/SKILL.md`) - Orchestrator instructions for creating phased implementation plans with documentation discovery
**Execution Skill** (`plugin/skills/do/SKILL.md`) - Orchestrator instructions for executing phased plans using subagents
**Chroma** (`src/services/sync/ChromaSync.ts`) - Vector embeddings for semantic search
**Viewer UI** (`src/ui/viewer/`) - React interface at http://localhost:37777, built to `plugin/ui/viewer.html`
+40 -17
View File
@@ -1,13 +1,3 @@
<p align="center">
Official $CMEM Links:
<a href="https://bags.fm/2TsmuYUrsctE57VLckZBYEEzdokUF8j8e1GavekWBAGS">Bags.fm</a> •
<a href="https://jup.ag/tokens/2TsmuYUrsctE57VLckZBYEEzdokUF8j8e1GavekWBAGS">Jupiter</a> •
<a href="https://photon-sol.tinyastro.io/en/lp/6MzFAkWnac6GSK1EdFX93dZeukGfzrFq4UHWarhGSQyd">Photon</a> •
<a href="https://dexscreener.com/solana/6mzfakwnac6gsk1edfx93dzeukgfzrfq4uhwarhgsqyd">DEXScreener</a>
</p>
<p align="center">Official CA: 2TsmuYUrsctE57VLckZBYEEzdokUF8j8e1GavekWBAGS (on Solana)</p>
<h1 align="center">
<br>
<a href="https://github.com/thedotmack/claude-mem">
@@ -84,13 +74,40 @@
<br>
<p align="center">
<a href="https://github.com/thedotmack/claude-mem">
<picture>
<img src="https://raw.githubusercontent.com/thedotmack/claude-mem/main/docs/public/cm-preview.gif" alt="Claude-Mem Preview" width="800">
</picture>
</a>
</p>
<table align="center">
<tr>
<td align="center">
<a href="https://github.com/thedotmack/claude-mem">
<picture>
<img
src="https://raw.githubusercontent.com/thedotmack/claude-mem/main/docs/public/cm-preview.gif"
alt="Claude-Mem Preview"
width="500"
>
</picture>
</a>
</td>
<td align="center">
<a href="https://www.star-history.com/#thedotmack/claude-mem&Date">
<picture>
<source
media="(prefers-color-scheme: dark)"
srcset="https://api.star-history.com/image?repos=thedotmack/claude-mem&type=date&theme=dark&legend=top-left"
/>
<source
media="(prefers-color-scheme: light)"
srcset="https://api.star-history.com/image?repos=thedotmack/claude-mem&type=date&legend=top-left"
/>
<img
alt="Star History Chart"
src="https://api.star-history.com/image?repos=thedotmack/claude-mem&type=date&legend=top-left"
width="500"
/>
</picture>
</a>
</td>
</tr>
</table>
<p align="center">
<a href="#quick-start">Quick Start</a> •
@@ -340,3 +357,9 @@ See the [LICENSE](LICENSE) file for full details.
---
**Built with Claude Agent SDK** | **Powered by Claude Code** | **Made with TypeScript**
---
### What About $CMEM?
$CMEM is a solana token created by a 3rd party without Claude-Mem's prior consent, but officially embraced by the creator of Claude-Mem (Alex Newman, @thedotmack). The token acts as a community catalyst for growth and a vehicle for bringing real-time agent data to the developers and knowledge workers that need it most. $CMEM: 2TsmuYUrsctE57VLckZBYEEzdokUF8j8e1GavekWBAGS
+2 -1
View File
@@ -62,7 +62,8 @@
"icon": "lightbulb",
"pages": [
"context-engineering",
"progressive-disclosure"
"progressive-disclosure",
"smart-explore-benchmark"
]
},
{
+25 -29
View File
@@ -1,6 +1,6 @@
---
title: OpenClaw Integration
description: Persistent memory for OpenClaw agents — observation recording, MEMORY.md live sync, and real-time observation feeds
description: Persistent memory for OpenClaw agents — observation recording, system prompt context injection, and real-time observation feeds
icon: dragon
---
@@ -9,7 +9,7 @@ icon: dragon
The OpenClaw plugin gives claude-mem persistent memory to agents running on the [OpenClaw](https://openclaw.ai) gateway. It handles three things:
1. **Observation recording** — Captures tool usage from OpenClaw's embedded runner and sends it to the claude-mem worker for AI processing
2. **MEMORY.md live sync** — Writes a continuously-updated timeline to each agent's workspace so agents always have context from previous sessions
2. **System prompt context injection** — Injects the observation timeline into each agent's system prompt via the `before_prompt_build` hook, keeping `MEMORY.md` free for agent-curated memory
3. **Observation feed** — Streams new observations to messaging channels (Telegram, Discord, Slack, etc.) in real-time via SSE
<Info>
@@ -21,10 +21,11 @@ OpenClaw's embedded runner (`pi-embedded`) calls the Anthropic API directly with
```plaintext
OpenClaw Gateway
├── before_agent_start ──→ Sync MEMORY.md + Init session
├── tool_result_persist ──→ Record observation + Re-sync MEMORY.md
├── before_agent_start ──→ Init session
├── before_prompt_build ──→ Inject context into system prompt
├── tool_result_persist ──→ Record observation
├── agent_end ────────────→ Summarize + Complete session
└── gateway_start ────────→ Reset session tracking
└── gateway_start ────────→ Reset session tracking + context cache
Claude-Mem Worker (localhost:37777)
@@ -32,7 +33,7 @@ OpenClaw Gateway
├── POST /api/sessions/observations
├── POST /api/sessions/summarize
├── POST /api/sessions/complete
├── GET /api/context/inject ──→ MEMORY.md content
├── GET /api/context/inject ──→ System prompt context
└── GET /stream ─────────────→ SSE → Messaging channels
```
@@ -40,21 +41,15 @@ OpenClaw Gateway
<Steps>
<Step title="Agent starts (before_agent_start)">
When an OpenClaw agent starts, the plugin does two things:
When an OpenClaw agent starts, the plugin initializes a session by sending the user prompt to `POST /api/sessions/init` so the worker can create a new session and start processing.
</Step>
<Step title="Context injected (before_prompt_build)">
Before each LLM call, the plugin fetches the observation timeline from the worker's `/api/context/inject` endpoint and returns it as `appendSystemContext`. This injects cross-session context directly into the agent's system prompt without writing any files.
1. **Syncs MEMORY.md** — Fetches the latest timeline from the worker's `/api/context/inject` endpoint and writes it to `MEMORY.md` in the agent's workspace directory. This gives the agent context from all previous sessions before it starts working.
2. **Initializes a session** — Sends the user prompt to `POST /api/sessions/init` so the worker can create a new session and start processing.
Short prompts (under 10 characters) skip session init but still sync MEMORY.md.
The context is cached for 60 seconds to avoid re-fetching on every LLM turn within a session.
</Step>
<Step title="Tool use recorded (tool_result_persist)">
Every time the agent uses a tool (Read, Write, Bash, etc.), the plugin:
1. **Sends the observation** to `POST /api/sessions/observations` with the tool name, input, and truncated response (max 1000 chars)
2. **Re-syncs MEMORY.md** with the latest timeline from the worker
Both operations are fire-and-forget — they don't block the agent from continuing work. The MEMORY.md file gets progressively richer as the session continues.
Every time the agent uses a tool (Read, Write, Bash, etc.), the plugin sends the observation to `POST /api/sessions/observations` with the tool name, input, and truncated response (max 1000 chars). This is fire-and-forget — it doesn't block the agent from continuing work.
Tools prefixed with `memory_` are skipped to avoid recursive recording.
</Step>
@@ -62,21 +57,18 @@ OpenClaw Gateway
When the agent completes, the plugin extracts the last assistant message and sends it to `POST /api/sessions/summarize`, then calls `POST /api/sessions/complete` to close the session. Both are fire-and-forget.
</Step>
<Step title="Gateway restarts (gateway_start)">
Clears all session tracking (session IDs, workspace directory mappings) so agents get fresh state after a gateway restart.
Clears all session tracking (session IDs, context cache) so agents get fresh state after a gateway restart.
</Step>
</Steps>
### MEMORY.md Live Sync
### System Prompt Context Injection
The plugin writes a `MEMORY.md` file to each agent's workspace directory containing the full timeline of observations and summaries from previous sessions. This file is updated:
The plugin injects cross-session observation context into each agent's system prompt via OpenClaw's `before_prompt_build` hook. The content comes from the worker's `GET /api/context/inject?projects=<project>` endpoint, which generates a formatted markdown timeline from the SQLite database.
- On every `before_agent_start` event (agent gets fresh context before starting)
- On every `tool_result_persist` event (context stays current during the session)
The content comes from the worker's `GET /api/context/inject?projects=<project>` endpoint, which generates a formatted markdown timeline from the SQLite database.
This approach keeps `MEMORY.md` under the agent's control for curated long-term memory (decisions, preferences, durable facts), while the observation timeline is delivered through the system prompt where it belongs.
<Info>
MEMORY.md updates are fire-and-forget. They run in the background without blocking the agent. The file reflects whatever the worker has processed so far — it doesn't wait for the current observation to be fully processed before writing.
Context is cached for 60 seconds per project to avoid re-fetching on every LLM turn. The cache is cleared on gateway restart. Use `syncMemoryFileExclude` to opt specific agents out of context injection entirely.
</Info>
### Observation Feed (SSE → Messaging)
@@ -319,7 +311,11 @@ The claude-mem worker service must be running on the same machine as the OpenCla
</ParamField>
<ParamField body="syncMemoryFile" type="boolean" default={true}>
Enable automatic MEMORY.md sync to agent workspaces. Set to `false` if you don't want the plugin writing files to workspace directories.
Inject observation context into the agent system prompt via `before_prompt_build` hook. When `true`, agents receive cross-session context automatically. Set to `false` to disable context injection entirely (observations are still recorded).
</ParamField>
<ParamField body="syncMemoryFileExclude" type="string[]" default={[]}>
Agent IDs excluded from automatic context injection. Useful for agents that curate their own memory and don't need the observation timeline (e.g., `["snarf", "debugger"]`). Observations are still recorded for excluded agents — only the context injection is skipped.
</ParamField>
<ParamField body="workerPort" type="number" default={37777}>
@@ -374,9 +370,9 @@ The plugin uses HTTP calls to the already-running claude-mem worker service rath
Each OpenClaw agent session gets a unique `contentSessionId` (format: `openclaw-<sessionKey>-<timestamp>`) that maps to a claude-mem session in the worker. The plugin tracks:
- `sessionIds` — Maps OpenClaw session keys to content session IDs
- `workspaceDirsBySessionKey` — Maps session keys to workspace directories so `tool_result_persist` events can sync MEMORY.md even when the event context doesn't include `workspaceDir`
- `contextCache` — TTL cache (60s) for context injection responses, keyed by project
Both maps are cleared on `gateway_start`.
Both are cleared on `gateway_start`.
## Requirements
+196
View File
@@ -0,0 +1,196 @@
---
title: "Smart Explore Benchmark"
description: "Token efficiency comparison between AST-based and traditional code exploration"
---
# Smart Explore Benchmark
Smart Explore uses tree-sitter AST parsing to provide structural code navigation through three MCP tools: `smart_search`, `smart_outline`, and `smart_unfold`. This report documents a rigorous A/B comparison against the standard Explore agent (which uses Glob, Grep, and Read tools) to quantify the token savings and quality trade-offs.
## Executive Summary
| Metric | Smart Explore | Explore Agent | Advantage |
|--------|:---:|:---:|---|
| Discovery (cross-file search) | ~14,200 tokens | ~252,500 tokens | **17.8x cheaper** |
| Targeted reads (specific symbols) | ~5,650 tokens | ~109,400 tokens | **19.4x cheaper** |
| End-to-end (search + read) | ~4,200 tokens | ~45,000 tokens | **10-12x cheaper** |
| Completeness | 5/5 full source returned | 4/5 (truncated longest method) | Smart Explore more reliable |
| Speed | Under 2s per call | 5-66s per call | **10-30x faster** |
## Methodology
### Test Environment
- **Codebase**: claude-mem (`src/` directory, 194 TypeScript files, 1,206 parsed symbols)
- **Model**: Claude Opus 4.6 for both approaches
- **Measurement**: Token counts from tool response metadata (`total_tokens` for Explore agents, self-reported `~N tokens for folded view` for Smart Explore)
### Controls
The Explore agents were explicitly instructed: *"Do NOT use smart_search, smart_outline, or smart_unfold tools. Only use Glob, Grep, and Read tools."* This was verified necessary after an initial round where agents opportunistically used the Smart Explore tools, invalidating the comparison.
### Queries
Five queries were selected to represent common exploration tasks:
1. **"session processing"** -- Cross-cutting feature spanning multiple services
2. **"shutdown"** -- Infrastructure concern touching 6+ files
3. **"hook registration"** -- Architecture question about plugin system
4. **"sqlite database"** -- Technology-specific search across the data layer
5. **"worker-service.ts outline"** -- Single large file (1,225 lines) structural understanding
## Round 1: Discovery
*"What exists and where is it?"* -- Finding relevant files and symbols across the codebase.
### Results
| Query | Smart Explore | Explore Agent | Ratio | Explore Tool Calls |
|-------|:---:|:---:|:---:|:---:|
| session processing | ~4,391 t | 51,659 t | **11.8x** | 15 |
| shutdown | ~3,852 t | 51,523 t | **13.4x** | 18 |
| hook registration | ~1,930 t | 51,688 t | **26.8x** | 37 |
| sqlite database | ~2,543 t | 58,633 t | **23.1x** | 16 |
| worker-service outline | ~1,500 t | 38,973 t | **26.0x** | 15 |
| **Total** | **~14,216 t** | **252,476 t** | **17.8x** | **101** |
### What Each Returned
**Smart Explore** (1 tool call each): 10 ranked symbols with signatures, line numbers, and JSDoc summaries, plus folded structural views of all matching files showing every function/class/interface with bodies collapsed.
**Explore Agent** (15-37 tool calls each): Synthesized narrative reports with architecture diagrams, design pattern analysis, data flow explanations, complete interface dumps, and file structure maps. Significantly more explanatory prose.
### Analysis
The token gap is widest for narrowly-scoped queries ("hook registration" at 26.8x) because the Explore agent reads multiple full files to find relatively few relevant symbols. For broad queries ("session processing" at 11.8x), more of the file content is relevant, narrowing the ratio.
Smart Explore's consistent 1-tool-call pattern means its cost is predictable. The Explore agent's cost varies with how many files it reads and how much it synthesizes -- ranging from 15 to 37 tool calls for comparable scope.
## Round 2: Targeted Reads
*"Show me this specific function."* -- Reading the implementation of a known symbol after discovery.
Based on the Round 1 results, five specific symbols were selected as natural drill-down targets:
| Target Symbol | File | Lines |
|---------------|------|:---:|
| `SessionManager.initializeSession` | services/worker/SessionManager.ts | 135 |
| `performGracefulShutdown` | services/infrastructure/GracefulShutdown.ts | 48 |
| `hookCommand` | cli/hook-command.ts | 45 |
| `DatabaseManager.initialize` | services/sqlite/Database.ts | 27 |
| `WorkerService.startSessionProcessor` | services/worker-service.ts | 158 |
### Results
| Symbol | Smart Unfold | Explore Agent | Ratio | Completeness |
|--------|:---:|:---:|:---:|---|
| initializeSession (135 lines) | ~1,800 t | 27,816 t | **15.5x** | Both returned full source |
| performGracefulShutdown (48 lines) | ~700 t | 19,621 t | **28.0x** | Both returned full source |
| hookCommand (45 lines) | ~650 t | 18,680 t | **28.7x** | Both returned full source |
| DatabaseManager.initialize (27 lines) | ~400 t | 22,334 t | **55.8x** | Both returned full source |
| startSessionProcessor (158 lines) | ~2,100 t | 20,906 t | **10.0x** | Smart Unfold: complete. Explore: **truncated** |
| **Total** | **~5,650 t** | **109,357 t** | **19.4x** | |
### Analysis
**The ratio scales inversely with symbol size.** The smallest function (`initialize`, 27 lines) shows the biggest gap at 55.8x because the Explore agent still reads the entire 235-line file to extract 27 lines. The largest method (`startSessionProcessor`, 158 lines) narrows to 10x since more of the file is "useful."
**Smart Unfold returned more complete code.** For the longest method (158 lines), the Explore agent truncated the error handling section with "... error handling continues ...", while `smart_unfold` returned the complete implementation. This is because smart_unfold extracts by AST node boundaries, guaranteeing completeness regardless of symbol size.
**Explore agents add zero unique information for targeted reads.** When you already know the file path and symbol name, the agent's overhead is pure waste -- it reads the file, locates the function, and echoes it back. The only addition is a brief explanatory paragraph.
## Combined Workflow
The realistic workflow is discovery followed by targeted reading. Here is the end-to-end cost comparison for understanding a single function:
### Smart Explore: search + unfold
```
smart_search("shutdown", path="./src") ~3,852 tokens
smart_unfold("GracefulShutdown.ts", "performGracefulShutdown") ~700 tokens
────────────────────────────────────────────────────────────────
Total: ~4,552 tokens (2 tool calls, under 3 seconds)
```
### Explore Agent: single query
```
"Find and explain the shutdown logic" ~51,523 tokens
────────────────────────────────────────────────────────────────
Total: ~51,523 tokens (18 tool calls, ~43 seconds)
```
**End-to-end ratio: 11.3x** -- and the Smart Explore workflow gives you the actual source code, while the Explore agent gives you a prose summary that may paraphrase or truncate.
## Quality Assessment
Neither approach is universally better. They optimize for different outcomes.
### Smart Explore Strengths
- **Predictable cost**: 1 tool call per operation, consistent token ranges
- **Complete source code**: AST-based extraction guarantees full symbol bodies
- **Structural context**: Folded views show every symbol in matching files
- **Speed**: Sub-second responses enable rapid iteration
- **Composability**: Search, outline, and unfold chain naturally
### Explore Agent Strengths
- **Synthesized understanding**: Produces architecture narratives, data flow diagrams, and design pattern analysis
- **Cross-cutting explanation**: Connects concepts across files that individual symbol reads cannot
- **Onboarding quality**: Output reads like documentation, not raw code
- **Error handling insight**: Identifies edge cases and design decisions that require reading multiple related functions
- **No prior knowledge needed**: Can answer open-ended questions without knowing file paths or symbol names
### Quality by Task Type
| Task | Better Tool | Why |
|------|-------------|-----|
| "Where is X defined?" | Smart Explore | One call, exact answer |
| "What functions are in this file?" | Smart Explore | Outline returns complete structural map |
| "Show me this function" | Smart Explore | Unfold returns exact source, never truncates |
| "How does feature X work end-to-end?" | Explore Agent | Reads multiple files and synthesizes narrative |
| "What design patterns are used here?" | Explore Agent | Requires reading and interpreting, not just extracting |
| "Help me understand this codebase" | Explore Agent | Produces onboarding-quality documentation |
## When to Use Which
**Use Smart Explore when:**
- You know what you are looking for (function name, concept, file)
- You need source code, not explanation
- You are iterating quickly (read, modify, read again)
- Token budget matters (large codebases, long sessions)
- You need file structure at a glance
**Use the Explore Agent when:**
- You need synthesized cross-cutting understanding
- The question is open-ended ("how does this system work?")
- You are writing documentation or architecture reviews
- You need to understand *why*, not just *what*
- You are onboarding to an unfamiliar codebase
**Use both when:**
- Start with Smart Explore for discovery and navigation
- Escalate to Explore Agent only for deep analysis that requires multi-file synthesis
- This hybrid approach captures most of the token savings while preserving access to deep understanding when needed
## Token Economics Reference
| Operation | Tokens | Use Case |
|-----------|:---:|----------|
| `smart_search` | 2,000-6,000 | Cross-file symbol discovery |
| `smart_outline` | 1,000-2,000 | Single file structural map |
| `smart_unfold` | 400-2,100 | Single symbol full source |
| `smart_search` + `smart_unfold` | 3,000-8,000 | End-to-end: find and read |
| Explore Agent (targeted) | 18,000-28,000 | Single function with explanation |
| Explore Agent (cross-cutting) | 39,000-59,000 | Architecture-level understanding |
| Read (full file) | 8,000-15,000+ | Complete file contents |
### Savings by Workflow
| Workflow | Smart Explore | Traditional | Savings |
|----------|:---:|:---:|:---:|
| Understand one file | outline + unfold (~3,100 t) | Read full file (~12,000 t) | **4x** |
| Find a function across codebase | search (~3,500 t) | Explore agent (~50,000 t) | **14x** |
| Find and read a specific function | search + unfold (~4,500 t) | Explore agent (~50,000 t) | **11x** |
| Navigate a 1,200-line file | outline (~1,500 t) | Read full file (~12,000 t) | **8x** |
+24 -20
View File
@@ -1,6 +1,6 @@
# Claude-Mem OpenClaw Plugin — Setup Guide
This guide walks through setting up the claude-mem plugin on an OpenClaw gateway. By the end, your agents will have persistent memory across sessions, a live-updating MEMORY.md in their workspace, and optionally a real-time observation feed streaming to a messaging channel.
This guide walks through setting up the claude-mem plugin on an OpenClaw gateway. By the end, your agents will have persistent memory across sessions via system prompt context injection, and optionally a real-time observation feed streaming to a messaging channel.
## Quick Install (Recommended)
@@ -138,7 +138,9 @@ Add the `claude-mem` plugin to your OpenClaw gateway configuration:
- **`project`** (string, default: `"openclaw"`) — The project name that scopes all observations in the memory database. Use a unique name per gateway/use-case so observations don't mix. For example, if this gateway runs a coding bot, use `"coding-bot"`.
- **`syncMemoryFile`** (boolean, default: `true`) — When enabled, the plugin writes a `MEMORY.md` file to each agent's workspace directory. This file contains the full timeline of observations and summaries from previous sessions, and it updates on every tool use so agents always have fresh context. Set to `false` only if you don't want the plugin writing files to agent workspaces.
- **`syncMemoryFile`** (boolean, default: `true`) — When enabled, the plugin injects the observation timeline into each agent's system prompt via the `before_prompt_build` hook. This gives agents cross-session context without writing to MEMORY.md. Set to `false` to disable context injection entirely (observations are still recorded).
- **`syncMemoryFileExclude`** (string[], default: `[]`) — Agent IDs excluded from automatic context injection. Useful for agents that curate their own memory. Observations are still recorded for excluded agents.
- **`workerPort`** (number, default: `37777`) — The port where the claude-mem worker service is listening. Only change this if you configured the worker to use a different port.
@@ -168,13 +170,14 @@ The observation feed shows `disconnected` because we haven't configured it yet.
Have an agent do some work. The plugin automatically records observations through these OpenClaw events:
1. **`before_agent_start`** — Initializes a claude-mem session when the agent starts, syncs MEMORY.md to the workspace
2. **`tool_result_persist`** — Records each tool use (Read, Write, Bash, etc.) as an observation, re-syncs MEMORY.md
3. **`agent_end`** — Summarizes the session and marks it complete
1. **`before_agent_start`** — Initializes a claude-mem session when the agent starts
2. **`before_prompt_build`** — Injects the observation timeline into the agent's system prompt (cached for 60s)
3. **`tool_result_persist`** — Records each tool use (Read, Write, Bash, etc.) as an observation
4. **`agent_end`** — Summarizes the session and marks it complete
All of this happens automatically. No additional configuration needed.
To verify it's working, check the agent's workspace directory for a `MEMORY.md` file after the agent runs. It should contain a formatted timeline of observations.
To verify it's working, check the worker's viewer UI at http://localhost:37777 to see observations appearing after the agent runs.
You can also check the worker's viewer UI at http://localhost:37777 to see observations appearing in real time.
@@ -372,10 +375,11 @@ Shows observation feed status. Accepts optional `on`/`off` argument.
```
OpenClaw Gateway
├── before_agent_start ──→ Sync MEMORY.md + Init session
├── tool_result_persist ──→ Record observation + Re-sync MEMORY.md
├── before_agent_start ──→ Init session
├── before_prompt_build ──→ Inject context into system prompt
├── tool_result_persist ──→ Record observation
├── agent_end ────────────→ Summarize + Complete session
└── gateway_start ────────→ Reset session tracking
└── gateway_start ────────→ Reset session tracking + context cache
Claude-Mem Worker (localhost:37777)
@@ -383,17 +387,15 @@ OpenClaw Gateway
├── POST /api/sessions/observations
├── POST /api/sessions/summarize
├── POST /api/sessions/complete
├── GET /api/context/inject ──→ MEMORY.md content
├── GET /api/context/inject ──→ System prompt context
└── GET /stream ─────────────→ SSE → Messaging channels
```
### MEMORY.md live sync
### System prompt context injection
The plugin writes `MEMORY.md` to each agent's workspace with the full observation timeline. It updates:
- On every `before_agent_start` — agent gets fresh context before starting
- On every `tool_result_persist` — context stays current as the agent works
The plugin injects the observation timeline into each agent's system prompt via the `before_prompt_build` hook. The content comes from the worker's `GET /api/context/inject` endpoint. Context is cached for 60 seconds per project to avoid re-fetching on every LLM turn. The cache is cleared on gateway restart.
Updates are fire-and-forget (non-blocking). The agent is never held up waiting for MEMORY.md to write.
This keeps MEMORY.md under the agent's control for curated long-term memory, while the observation timeline is delivered through the system prompt.
### Observation recording
@@ -401,10 +403,11 @@ Every tool use (Read, Write, Bash, etc.) is sent to the claude-mem worker as an
### Session lifecycle
- **`before_agent_start`** — Creates a session in the worker, syncs MEMORY.md. Short prompts (under 10 chars) skip session init but still sync.
- **`tool_result_persist`** — Records observation (fire-and-forget), re-syncs MEMORY.md (fire-and-forget). Tool responses are truncated to 1000 characters.
- **`before_agent_start`** — Creates a session in the worker.
- **`before_prompt_build`** — Fetches the observation timeline and returns it as `appendSystemContext`. Cached for 60s.
- **`tool_result_persist`** — Records observation (fire-and-forget). Tool responses are truncated to 1000 characters.
- **`agent_end`** — Sends the last assistant message for summarization, then completes the session. Both fire-and-forget.
- **`gateway_start`** — Clears all session tracking (session IDs, workspace mappings) so agents start fresh.
- **`gateway_start`** — Clears all session tracking (session IDs, context cache) so agents start fresh.
### Observation feed
@@ -417,7 +420,7 @@ A background service connects to the worker's SSE stream and forwards `new_obser
| Worker health check fails | Is bun installed? (`bun --version`). Is something else on port 37777? (`lsof -i :37777`). Try running directly: `bun plugin/scripts/worker-service.cjs start` |
| Worker started from Claude Code install but not responding | Check `cd ~/.claude/plugins/marketplaces/thedotmack && npm run worker:status`. May need `npm run worker:restart`. |
| Worker started from cloned repo but not responding | Check `cd /path/to/claude-mem && npm run worker:status`. Make sure you ran `npm install && npm run build` first. |
| No MEMORY.md appearing | Check that `syncMemoryFile` is not set to `false`. Verify the agent's event context includes `workspaceDir`. |
| No context in agent system prompt | Check that `syncMemoryFile` is not set to `false`. Check that the agent's ID is not in `syncMemoryFileExclude`. Verify the worker is running and has observations. |
| Observations not being recorded | Check gateway logs for `[claude-mem]` messages. The worker must be running and reachable on localhost:37777. |
| Feed shows `disconnected` | Worker's `/stream` endpoint not reachable. Check `workerPort` matches the actual worker port. |
| Feed shows `reconnecting` | Connection dropped. The plugin auto-reconnects — wait up to 30 seconds. |
@@ -451,7 +454,8 @@ A background service connects to the worker's SSE stream and forwards `new_obser
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `project` | string | `"openclaw"` | Project name scoping observations in the database |
| `syncMemoryFile` | boolean | `true` | Write MEMORY.md to agent workspaces |
| `syncMemoryFile` | boolean | `true` | Inject observation context into agent system prompt |
| `syncMemoryFileExclude` | string[] | `[]` | Agent IDs excluded from context injection |
| `workerPort` | number | `37777` | Claude-mem worker service port |
| `observationFeed.enabled` | boolean | `false` | Stream observations to a messaging channel |
| `observationFeed.channel` | string | — | Channel type: `telegram`, `discord`, `slack`, `signal`, `whatsapp`, `line` |
+9 -3
View File
@@ -3,10 +3,10 @@
"name": "Claude-Mem (Persistent Memory)",
"description": "Official OpenClaw plugin for Claude-Mem. Records observations from embedded runner sessions and streams them to messaging channels.",
"kind": "memory",
"version": "1.0.0",
"version": "10.4.1",
"author": "thedotmack",
"homepage": "https://claude-mem.com",
"skills": ["skills/make-plan", "skills/do-plan"],
"skills": ["skills/make-plan", "skills/do"],
"configSchema": {
"type": "object",
"additionalProperties": false,
@@ -14,7 +14,13 @@
"syncMemoryFile": {
"type": "boolean",
"default": true,
"description": "Automatically sync MEMORY.md on session start"
"description": "Inject observation context into the agent system prompt via before_prompt_build hook. When true, agents receive cross-session context without MEMORY.md being overwritten."
},
"syncMemoryFileExclude": {
"type": "array",
"items": { "type": "string" },
"default": [],
"description": "Agent IDs excluded from automatic context injection (observations are still recorded, only prompt injection is skipped)"
},
"workerPort": {
"type": "number",
+1
View File
@@ -0,0 +1 @@
../../../plugin/skills/do/SKILL.md
-63
View File
@@ -1,63 +0,0 @@
---
name: make-plan
description: Create a detailed, phased implementation plan with documentation discovery. Use when asked to plan a feature, task, or multi-step implementation — especially before executing with do-plan.
---
# Make Plan
You are an ORCHESTRATOR. Create an LLM-friendly plan in phases that can be executed consecutively in new chat contexts.
## Delegation Model
Use subagents for *fact gathering and extraction* (docs, examples, signatures, grep results). Keep *synthesis and plan authoring* with the orchestrator (phase boundaries, task framing, final wording). If a subagent report is incomplete or lacks evidence, re-check with targeted reads/greps before finalizing.
### Subagent Reporting Contract (MANDATORY)
Each subagent response must include:
1. Sources consulted (files/URLs) and what was read
2. Concrete findings (exact API names/signatures; exact file paths/locations)
3. Copy-ready snippet locations (example files/sections to copy)
4. "Confidence" note + known gaps (what might still be missing)
Reject and redeploy the subagent if it reports conclusions without sources.
## Plan Structure
### Phase 0: Documentation Discovery (ALWAYS FIRST)
Before planning implementation, deploy "Documentation Discovery" subagents to:
1. Search for and read relevant documentation, examples, and existing patterns
2. Identify the actual APIs, methods, and signatures available (not assumed)
3. Create a brief "Allowed APIs" list citing specific documentation sources
4. Note any anti-patterns to avoid (methods that DON'T exist, deprecated parameters)
The orchestrator consolidates findings into a single Phase 0 output.
### Each Implementation Phase Must Include
1. **What to implement** — Frame tasks to COPY from docs, not transform existing code
- Good: "Copy the V2 session pattern from docs/examples.ts:45-60"
- Bad: "Migrate the existing code to V2"
2. **Documentation references** — Cite specific files/lines for patterns to follow
3. **Verification checklist** — How to prove this phase worked (tests, grep checks)
4. **Anti-pattern guards** — What NOT to do (invented APIs, undocumented params)
### Final Phase: Verification
1. Verify all implementations match documentation
2. Check for anti-patterns (grep for known bad patterns)
3. Run tests to confirm functionality
## Key Principles
- Documentation Availability ≠ Usage: Explicitly require reading docs
- Task Framing Matters: Direct agents to docs, not just outcomes
- Verify > Assume: Require proof, not assumptions about APIs
- Session Boundaries: Each phase should be self-contained with its own doc references
## Anti-Patterns to Prevent
- Inventing API methods that "should" exist
- Adding parameters not in documentation
- Skipping verification steps
- Assuming structure without checking examples
+1
View File
@@ -0,0 +1 @@
../../../plugin/skills/make-plan/SKILL.md
+115 -96
View File
@@ -87,9 +87,11 @@ function createMockApi(pluginConfigOverride: Record<string, any> = {}) {
getEventHandlers: (event: string) => eventHandlers.get(event) || [],
fireEvent: async (event: string, data: any, ctx: any = {}) => {
const handlers = eventHandlers.get(event) || [];
let lastResult: any;
for (const handler of handlers) {
await handler(data, ctx);
lastResult = await handler(data, ctx);
}
return lastResult;
},
};
}
@@ -106,6 +108,7 @@ describe("claudeMemPlugin", () => {
assert.ok(getEventHandlers("session_start").length > 0, "session_start handler registered");
assert.ok(getEventHandlers("after_compaction").length > 0, "after_compaction handler registered");
assert.ok(getEventHandlers("before_agent_start").length > 0, "before_agent_start handler registered");
assert.ok(getEventHandlers("before_prompt_build").length > 0, "before_prompt_build handler registered");
assert.ok(getEventHandlers("tool_result_persist").length > 0, "tool_result_persist handler registered");
assert.ok(getEventHandlers("agent_end").length > 0, "agent_end handler registered");
assert.ok(getEventHandlers("gateway_start").length > 0, "gateway_start handler registered");
@@ -535,11 +538,10 @@ describe("Observation I/O event handlers", () => {
});
});
describe("MEMORY.md context sync", () => {
describe("before_prompt_build context injection", () => {
let workerServer: Server;
let workerPort: number;
let receivedRequests: Array<{ method: string; url: string; body: any }> = [];
let tmpDir: string;
let contextResponse = "# Claude-Mem Context\n\n## Timeline\n- Session 1: Did some work";
function startWorkerMock(): Promise<number> {
@@ -586,21 +588,20 @@ describe("MEMORY.md context sync", () => {
receivedRequests = [];
contextResponse = "# Claude-Mem Context\n\n## Timeline\n- Session 1: Did some work";
workerPort = await startWorkerMock();
tmpDir = await mkdtemp(join(tmpdir(), "claude-mem-test-"));
});
afterEach(async () => {
workerServer?.close();
await rm(tmpDir, { recursive: true, force: true });
});
it("writes MEMORY.md to workspace on before_agent_start", async () => {
it("returns appendSystemContext from before_prompt_build", async () => {
const { api, logs, fireEvent } = createMockApi({ workerPort });
claudeMemPlugin(api);
await fireEvent("before_agent_start", {
const result = await fireEvent("before_prompt_build", {
prompt: "Help me write a function",
}, { sessionKey: "sync-test", workspaceDir: tmpDir });
messages: [],
}, { agentId: "main" });
await new Promise((resolve) => setTimeout(resolve, 200));
@@ -608,142 +609,143 @@ describe("MEMORY.md context sync", () => {
assert.ok(contextRequest, "should request context from worker");
assert.ok(contextRequest!.url!.includes("projects=openclaw"));
const memoryContent = await readFile(join(tmpDir, "MEMORY.md"), "utf-8");
assert.ok(memoryContent.includes("Claude-Mem Context"), "MEMORY.md should contain context");
assert.ok(memoryContent.includes("Session 1"), "MEMORY.md should contain timeline");
assert.ok(logs.some((l) => l.includes("MEMORY.md synced")));
assert.ok(result, "should return a result");
assert.ok(result.appendSystemContext, "should return appendSystemContext");
assert.ok(result.appendSystemContext.includes("Claude-Mem Context"), "should contain context");
assert.ok(result.appendSystemContext.includes("Session 1"), "should contain timeline");
assert.ok(logs.some((l) => l.includes("Context injected via system prompt")));
});
it("syncs MEMORY.md on every before_agent_start call", async () => {
const { api, fireEvent } = createMockApi({ workerPort });
claudeMemPlugin(api);
it("does not write MEMORY.md on before_agent_start", async () => {
const tmpDir = await mkdtemp(join(tmpdir(), "claude-mem-test-"));
try {
const { api, fireEvent } = createMockApi({ workerPort });
claudeMemPlugin(api);
await fireEvent("before_agent_start", {
prompt: "First prompt for this agent",
}, { sessionKey: "agent-a", workspaceDir: tmpDir });
await fireEvent("before_agent_start", {
prompt: "Help me write a function",
}, { sessionKey: "sync-test", workspaceDir: tmpDir });
await new Promise((resolve) => setTimeout(resolve, 200));
await new Promise((resolve) => setTimeout(resolve, 200));
const firstContextRequests = receivedRequests.filter((r) => r.url?.startsWith("/api/context/inject"));
assert.equal(firstContextRequests.length, 1, "first call should fetch context");
await fireEvent("before_agent_start", {
prompt: "Second prompt for same agent",
}, { sessionKey: "agent-a", workspaceDir: tmpDir });
await new Promise((resolve) => setTimeout(resolve, 200));
const allContextRequests = receivedRequests.filter((r) => r.url?.startsWith("/api/context/inject"));
assert.equal(allContextRequests.length, 2, "should re-fetch context on every call");
let memoryExists = true;
try {
await readFile(join(tmpDir, "MEMORY.md"), "utf-8");
} catch {
memoryExists = false;
}
assert.ok(!memoryExists, "MEMORY.md should not be created by before_agent_start");
} finally {
await rm(tmpDir, { recursive: true, force: true });
}
});
it("syncs MEMORY.md on tool_result_persist via fire-and-forget", async () => {
const { api, fireEvent } = createMockApi({ workerPort });
claudeMemPlugin(api);
it("does not sync MEMORY.md on tool_result_persist", async () => {
const tmpDir = await mkdtemp(join(tmpdir(), "claude-mem-test-"));
try {
const { api, fireEvent } = createMockApi({ workerPort });
claudeMemPlugin(api);
// Init session to register workspace dir
await fireEvent("before_agent_start", {
prompt: "Help me write a function",
}, { sessionKey: "tool-sync", workspaceDir: tmpDir });
await fireEvent("before_agent_start", {
prompt: "Help me write a function",
}, { sessionKey: "tool-sync", workspaceDir: tmpDir });
await new Promise((resolve) => setTimeout(resolve, 200));
await new Promise((resolve) => setTimeout(resolve, 200));
const preToolContextRequests = receivedRequests.filter((r) => r.url?.startsWith("/api/context/inject"));
assert.equal(preToolContextRequests.length, 1, "before_agent_start should sync once");
await fireEvent("tool_result_persist", {
toolName: "Read",
params: { file_path: "/src/app.ts" },
message: { content: [{ type: "text", text: "file contents" }] },
}, { sessionKey: "tool-sync" });
// Fire tool result — should trigger another MEMORY.md sync
await fireEvent("tool_result_persist", {
toolName: "Read",
params: { file_path: "/src/app.ts" },
message: { content: [{ type: "text", text: "file contents" }] },
}, { sessionKey: "tool-sync" });
await new Promise((resolve) => setTimeout(resolve, 200));
await new Promise((resolve) => setTimeout(resolve, 200));
const contextRequests = receivedRequests.filter((r) => r.url?.startsWith("/api/context/inject"));
assert.equal(contextRequests.length, 0, "tool_result_persist should not fetch context");
const postToolContextRequests = receivedRequests.filter((r) => r.url?.startsWith("/api/context/inject"));
assert.equal(postToolContextRequests.length, 2, "tool_result_persist should trigger another sync");
const memoryContent = await readFile(join(tmpDir, "MEMORY.md"), "utf-8");
assert.ok(memoryContent.includes("Claude-Mem Context"), "MEMORY.md should be updated");
let memoryExists = true;
try {
await readFile(join(tmpDir, "MEMORY.md"), "utf-8");
} catch {
memoryExists = false;
}
assert.ok(!memoryExists, "MEMORY.md should not be written by tool_result_persist");
} finally {
await rm(tmpDir, { recursive: true, force: true });
}
});
it("skips MEMORY.md sync when syncMemoryFile is false", async () => {
it("skips context injection when syncMemoryFile is false", async () => {
const { api, fireEvent } = createMockApi({ workerPort, syncMemoryFile: false });
claudeMemPlugin(api);
await fireEvent("before_agent_start", {
const result = await fireEvent("before_prompt_build", {
prompt: "Help me write a function",
}, { sessionKey: "no-sync", workspaceDir: tmpDir });
messages: [],
}, { agentId: "main" });
await new Promise((resolve) => setTimeout(resolve, 200));
const contextRequest = receivedRequests.find((r) => r.url?.startsWith("/api/context/inject"));
assert.ok(!contextRequest, "should not fetch context when sync disabled");
assert.ok(!contextRequest, "should not fetch context when injection disabled");
assert.equal(result, undefined, "should return undefined when injection disabled");
});
it("skips MEMORY.md sync when no workspaceDir in context", async () => {
const { api, fireEvent } = createMockApi({ workerPort });
it("skips context injection for excluded agents", async () => {
const { api, fireEvent } = createMockApi({ workerPort, syncMemoryFileExclude: ["snarf"] });
claudeMemPlugin(api);
await fireEvent("before_agent_start", {
prompt: "Help me write a function",
}, { sessionKey: "no-workspace" });
const result = await fireEvent("before_prompt_build", {
prompt: "Help me",
messages: [],
}, { agentId: "snarf" });
await new Promise((resolve) => setTimeout(resolve, 200));
const contextRequest = receivedRequests.find((r) => r.url?.startsWith("/api/context/inject"));
assert.ok(!contextRequest, "should not fetch context without workspaceDir");
assert.ok(!contextRequest, "should not fetch context for excluded agent");
assert.equal(result, undefined, "should return undefined for excluded agent");
});
it("skips writing MEMORY.md when context is empty", async () => {
it("injects context for non-excluded agents", async () => {
const { api, fireEvent } = createMockApi({ workerPort, syncMemoryFileExclude: ["snarf"] });
claudeMemPlugin(api);
const result = await fireEvent("before_prompt_build", {
prompt: "Help me",
messages: [],
}, { agentId: "main" });
await new Promise((resolve) => setTimeout(resolve, 200));
assert.ok(result, "should return a result for non-excluded agent");
assert.ok(result.appendSystemContext, "should inject context for non-excluded agent");
});
it("returns undefined when context is empty", async () => {
contextResponse = " ";
const { api, logs, fireEvent } = createMockApi({ workerPort });
claudeMemPlugin(api);
await fireEvent("before_agent_start", {
const result = await fireEvent("before_prompt_build", {
prompt: "Help me write a function",
}, { sessionKey: "empty-ctx", workspaceDir: tmpDir });
messages: [],
}, { agentId: "main" });
await new Promise((resolve) => setTimeout(resolve, 200));
assert.ok(!logs.some((l) => l.includes("MEMORY.md synced")), "should not log sync for empty context");
});
it("gateway_start resets sync tracking so next agent re-syncs", async () => {
const { api, fireEvent } = createMockApi({ workerPort });
claudeMemPlugin(api);
// First sync
await fireEvent("before_agent_start", {
prompt: "Help me write a function",
}, { sessionKey: "agent-1", workspaceDir: tmpDir });
await new Promise((resolve) => setTimeout(resolve, 200));
const firstContextRequests = receivedRequests.filter((r) => r.url?.startsWith("/api/context/inject"));
assert.equal(firstContextRequests.length, 1);
// Gateway restart
await fireEvent("gateway_start", {}, {});
// Second sync after gateway restart — same workspace should re-sync
await fireEvent("before_agent_start", {
prompt: "Help me after gateway restart",
}, { sessionKey: "agent-1", workspaceDir: tmpDir });
await new Promise((resolve) => setTimeout(resolve, 200));
const allContextRequests = receivedRequests.filter((r) => r.url?.startsWith("/api/context/inject"));
assert.equal(allContextRequests.length, 2, "should re-fetch context after gateway restart");
assert.equal(result, undefined, "should return undefined for empty context");
assert.ok(!logs.some((l) => l.includes("Context injected")), "should not log injection for empty context");
});
it("uses custom project name in context inject URL", async () => {
const { api, fireEvent } = createMockApi({ workerPort, project: "my-bot" });
claudeMemPlugin(api);
await fireEvent("before_agent_start", {
await fireEvent("before_prompt_build", {
prompt: "Help me write a function",
}, { sessionKey: "proj-test", workspaceDir: tmpDir });
messages: [],
}, { agentId: "main" });
await new Promise((resolve) => setTimeout(resolve, 200));
@@ -751,6 +753,23 @@ describe("MEMORY.md context sync", () => {
assert.ok(contextRequest, "should request context");
assert.ok(contextRequest!.url!.includes("projects=my-bot"), "should use custom project name");
});
it("includes agent-scoped project in context request", async () => {
const { api, fireEvent } = createMockApi({ workerPort });
claudeMemPlugin(api);
await fireEvent("before_prompt_build", {
prompt: "Help me",
messages: [],
}, { agentId: "debugger" });
await new Promise((resolve) => setTimeout(resolve, 200));
const contextRequest = receivedRequests.find((r) => r.url?.startsWith("/api/context/inject"));
assert.ok(contextRequest, "should request context");
const url = decodeURIComponent(contextRequest!.url!);
assert.ok(url.includes("openclaw,openclaw-debugger"), "should include both base and agent-scoped projects");
});
});
describe("SSE stream integration", () => {
+75 -31
View File
@@ -1,5 +1,5 @@
import { writeFile } from "fs/promises";
import { join } from "path";
// No file-system imports needed — context is injected via system prompt hook,
// not by writing to MEMORY.md.
// Minimal type declarations for the OpenClaw Plugin SDK.
// These match the real OpenClawPluginApi provided by the gateway at runtime.
@@ -35,6 +35,18 @@ interface BeforeAgentStartEvent {
prompt?: string;
}
interface BeforePromptBuildEvent {
prompt: string;
messages: unknown[];
}
interface BeforePromptBuildResult {
systemPrompt?: string;
prependContext?: string;
prependSystemContext?: string;
appendSystemContext?: string;
}
interface ToolResultPersistEvent {
toolName?: string;
params?: Record<string, unknown>;
@@ -87,6 +99,7 @@ interface MessageContext {
}
type EventCallback<T> = (event: T, ctx: EventContext) => void | Promise<void>;
type PromptBuildCallback = (event: BeforePromptBuildEvent, ctx: EventContext) => BeforePromptBuildResult | Promise<BeforePromptBuildResult | void> | void;
type MessageEventCallback<T> = (event: T, ctx: MessageContext) => void | Promise<void>;
interface OpenClawPluginApi {
@@ -109,7 +122,8 @@ interface OpenClawPluginApi {
requireAuth?: boolean;
handler: (ctx: PluginCommandContext) => PluginCommandResult | Promise<PluginCommandResult>;
}) => void;
on: ((event: "before_agent_start", callback: EventCallback<BeforeAgentStartEvent>) => void) &
on: ((event: "before_prompt_build", callback: PromptBuildCallback) => void) &
((event: "before_agent_start", callback: EventCallback<BeforeAgentStartEvent>) => void) &
((event: "tool_result_persist", callback: EventCallback<ToolResultPersistEvent>) => void) &
((event: "agent_end", callback: EventCallback<AgentEndEvent>) => void) &
((event: "session_start", callback: EventCallback<SessionStartEvent>) => void) &
@@ -166,6 +180,7 @@ interface FeedEmojiConfig {
interface ClaudeMemPluginConfig {
syncMemoryFile?: boolean;
syncMemoryFileExclude?: string[];
project?: string;
workerPort?: number;
observationFeed?: {
@@ -532,8 +547,8 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void {
// Session tracking for observation I/O
// ------------------------------------------------------------------
const sessionIds = new Map<string, string>();
const workspaceDirsBySessionKey = new Map<string, string>();
const syncMemoryFile = userConfig.syncMemoryFile !== false; // default true
const syncMemoryFileExclude = new Set(userConfig.syncMemoryFileExclude || []);
function getContentSessionId(sessionKey?: string): string {
const key = sessionKey || "default";
@@ -543,27 +558,45 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void {
return sessionIds.get(key)!;
}
async function syncMemoryToWorkspace(workspaceDir: string, ctx?: EventContext): Promise<void> {
function shouldInjectContext(ctx?: EventContext): boolean {
if (!syncMemoryFile) return false;
const agentId = ctx?.agentId;
if (agentId && syncMemoryFileExclude.has(agentId)) return false;
return true;
}
// TTL cache for context injection to avoid re-fetching on every LLM turn.
// before_prompt_build fires on every turn; caching for 60s keeps the worker
// load manageable while still picking up new observations reasonably quickly.
const CONTEXT_CACHE_TTL_MS = 60_000;
const contextCache = new Map<string, { text: string; fetchedAt: number }>();
async function getContextForPrompt(ctx?: EventContext): Promise<string | null> {
// Include both the base project and agent-scoped project (e.g. "openclaw" + "openclaw-main")
const projects = [baseProjectName];
const agentProject = ctx ? getProjectName(ctx) : null;
if (agentProject && agentProject !== baseProjectName) {
projects.push(agentProject);
}
const cacheKey = projects.join(",");
// Return cached context if still fresh
const cached = contextCache.get(cacheKey);
if (cached && Date.now() - cached.fetchedAt < CONTEXT_CACHE_TTL_MS) {
return cached.text;
}
const contextText = await workerGetText(
workerPort,
`/api/context/inject?projects=${encodeURIComponent(projects.join(","))}`,
`/api/context/inject?projects=${encodeURIComponent(cacheKey)}`,
api.logger
);
if (contextText && contextText.trim().length > 0) {
try {
await writeFile(join(workspaceDir, "MEMORY.md"), contextText, "utf-8");
api.logger.info(`[claude-mem] MEMORY.md synced to ${workspaceDir}`);
} catch (writeError: unknown) {
const msg = writeError instanceof Error ? writeError.message : String(writeError);
api.logger.warn(`[claude-mem] Failed to write MEMORY.md: ${msg}`);
}
const trimmed = contextText.trim();
contextCache.set(cacheKey, { text: trimmed, fetchedAt: Date.now() });
return trimmed;
}
return null;
}
// ------------------------------------------------------------------
@@ -611,14 +644,9 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void {
});
// ------------------------------------------------------------------
// Event: before_agent_start — init session + sync MEMORY.md + track workspace
// Event: before_agent_start — init session
// ------------------------------------------------------------------
api.on("before_agent_start", async (event, ctx) => {
// Track workspace dir so tool_result_persist can sync MEMORY.md later
if (ctx.workspaceDir) {
workspaceDirsBySessionKey.set(ctx.sessionKey || "default", ctx.workspaceDir);
}
// Initialize session in the worker so observations are not skipped
// (the privacy check requires a stored user prompt to exist)
const contentSessionId = getContentSessionId(ctx.sessionKey);
@@ -627,21 +655,37 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void {
project: getProjectName(ctx),
prompt: event.prompt || "agent run",
}, api.logger);
});
// Sync MEMORY.md before agent runs (provides context to agent)
if (syncMemoryFile && ctx.workspaceDir) {
await syncMemoryToWorkspace(ctx.workspaceDir, ctx);
// ------------------------------------------------------------------
// Event: before_prompt_build — inject context into system prompt
//
// Instead of writing to MEMORY.md (which conflicts with agent-curated
// memory), inject the observation timeline via appendSystemContext.
// This keeps MEMORY.md under the agent's control while still providing
// cross-session context to the LLM.
// ------------------------------------------------------------------
api.on("before_prompt_build", async (_event, ctx) => {
if (!shouldInjectContext(ctx)) return;
const contextText = await getContextForPrompt(ctx);
if (contextText) {
api.logger.info(`[claude-mem] Context injected via system prompt for agent=${ctx.agentId ?? "unknown"}`);
return { appendSystemContext: contextText };
}
});
// ------------------------------------------------------------------
// Event: tool_result_persist — record tool observations + sync MEMORY.md
// Event: tool_result_persist — record tool observations
// ------------------------------------------------------------------
api.on("tool_result_persist", (event, ctx) => {
api.logger.info(`[claude-mem] tool_result_persist fired: tool=${event.toolName ?? "unknown"} agent=${ctx.agentId ?? "none"} session=${ctx.sessionKey ?? "none"}`);
const toolName = event.toolName;
if (!toolName) return;
// Skip memory_ tools to prevent recursive observation loops
if (toolName.startsWith("memory_")) return;
const contentSessionId = getContentSessionId(ctx.sessionKey);
// Extract result text from all content blocks
@@ -654,7 +698,13 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void {
.join("\n");
}
// Fire-and-forget: send observation + sync MEMORY.md in parallel
// Truncate long responses to prevent oversized payloads
const MAX_TOOL_RESPONSE_LENGTH = 1000;
if (toolResponseText.length > MAX_TOOL_RESPONSE_LENGTH) {
toolResponseText = toolResponseText.slice(0, MAX_TOOL_RESPONSE_LENGTH);
}
// Fire-and-forget: send observation to worker
workerPostFireAndForget(workerPort, "/api/sessions/observations", {
contentSessionId,
tool_name: toolName,
@@ -662,11 +712,6 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void {
tool_response: toolResponseText,
cwd: "",
}, api.logger);
const workspaceDir = ctx.workspaceDir || workspaceDirsBySessionKey.get(ctx.sessionKey || "default");
if (syncMemoryFile && workspaceDir) {
syncMemoryToWorkspace(workspaceDir, ctx);
}
});
// ------------------------------------------------------------------
@@ -713,15 +758,14 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void {
api.on("session_end", async (_event, ctx) => {
const key = ctx.sessionKey || "default";
sessionIds.delete(key);
workspaceDirsBySessionKey.delete(key);
});
// ------------------------------------------------------------------
// Event: gateway_start — clear session tracking for fresh start
// ------------------------------------------------------------------
api.on("gateway_start", async () => {
workspaceDirsBySessionKey.clear();
sessionIds.clear();
contextCache.clear();
api.logger.info("[claude-mem] Gateway started — session tracking reset");
});
+14 -1
View File
@@ -1,6 +1,6 @@
{
"name": "claude-mem",
"version": "10.3.2",
"version": "10.6.0",
"description": "Memory compression system for Claude Code - persist context across sessions",
"keywords": [
"claude",
@@ -117,7 +117,20 @@
"@types/react-dom": "^18.3.0",
"esbuild": "^0.27.2",
"np": "^11.0.2",
"tree-sitter-c": "^0.24.1",
"tree-sitter-cli": "^0.26.5",
"tree-sitter-cpp": "^0.23.4",
"tree-sitter-go": "^0.25.0",
"tree-sitter-java": "^0.23.5",
"tree-sitter-javascript": "^0.25.0",
"tree-sitter-python": "^0.25.0",
"tree-sitter-ruby": "^0.23.1",
"tree-sitter-rust": "^0.24.0",
"tree-sitter-typescript": "^0.23.2",
"tsx": "^4.20.6",
"typescript": "^5.3.0"
},
"optionalDependencies": {
"tree-kill": "^1.2.2"
}
}
-52
View File
@@ -1,52 +0,0 @@
# Fix: SessionStart Hook "startup hook error" — Worker Not Waiting
## Root Cause
The **installed plugin** (`~/.claude/plugins/marketplaces/thedotmack/`) is version **10.2.5** and has **none** of the recent fixes:
| Fix | Repo Status | Installed Status |
|-----|-------------|-----------------|
| Hook group split (smart-install isolated from worker start) | In `plugin/hooks/hooks.json` | **Missing** — all 3 hooks in one group, smart-install failure blocks worker |
| `waitForReadiness()` after spawn | In `src/services/infrastructure/HealthMonitor.ts` | **Missing** — 0 occurrences in installed `worker-service.cjs` |
| Early `initializationCompleteFlag` (after DB+search, not MCP) | In `src/services/worker-service.ts` | **Missing** — flag set after MCP connection (5+ minute wait) |
The changes exist in source code but were **never built and synced** to the installed location.
---
## Phase 1: Build and Sync
```bash
npm run build-and-sync
```
### Verification
```bash
# 1. Confirm waitForReadiness exists in installed build
grep -c "waitForReadiness" ~/.claude/plugins/marketplaces/thedotmack/plugin/scripts/worker-service.cjs
# Expected: > 0
# 2. Confirm hooks.json has two SessionStart groups (the split)
python3 -c "import json; d=json.load(open('$(echo $HOME)/.claude/plugins/marketplaces/thedotmack/plugin/hooks/hooks.json')); print('SessionStart groups:', len(d['hooks']['SessionStart']))"
# Expected: 2
# 3. Confirm initializationCompleteFlag is set before MCP connection
grep -n "Core initialization complete" ~/.claude/plugins/marketplaces/thedotmack/plugin/scripts/worker-service.cjs | head -1
# Expected: appears BEFORE "MCP server connected"
```
## Phase 2: Restart Worker and Test
```bash
# Stop existing worker
bun plugin/scripts/worker-service.cjs stop
# Verify stopped
curl -s http://127.0.0.1:37777/api/health && echo "STILL RUNNING" || echo "STOPPED"
```
Then start a new Claude Code session and verify:
- No "SessionStart:startup hook error" messages
- Worker is running: `curl http://127.0.0.1:37777/api/health`
- Readiness endpoint works: `curl http://127.0.0.1:37777/api/readiness`
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "claude-mem",
"version": "10.3.2",
"version": "10.6.0",
"description": "Persistent memory system for Claude Code - seamlessly preserve context across sessions",
"author": {
"name": "Alex Newman"
-43
View File
@@ -1,43 +0,0 @@
---
description: "Execute a plan using subagents for implementation"
argument-hint: "[task or plan reference]"
---
You are an ORCHESTRATOR.
Primary instruction: deploy subagents to execute *all* work for #$ARGUMENTS.
Do not do the work yourself except to coordinate, route context, and verify that each subagent completed its assigned checklist.
Deploy subagents to execute each phase of #$ARGUMENTS independently and consecutively. For every checklist item below, explicitly deploy (or reuse) a subagent responsible for that item and record its outcome before proceeding.
## Execution Protocol (Orchestrator-Driven)
Orchestrator rules:
- Each phase uses fresh subagents where noted (or when context is large/unclear).
- The orchestrator assigns one clear objective per subagent and requires evidence (commands run, outputs, files changed).
- Do not advance to the next step until the assigned subagent reports completion and the orchestrator confirms it matches the plan.
### During Each Phase:
Deploy an "Implementation" subagent to:
1. Execute the implementation as specified
2. COPY patterns from documentation, don't invent
3. Cite documentation sources in code comments when using unfamiliar APIs
4. If an API seems missing, STOP and verify - don't assume it exists
### After Each Phase:
Deploy subagents for each post-phase responsibility:
1. **Run verification checklist** - Deploy a "Verification" subagent to prove the phase worked
2. **Anti-pattern check** - Deploy an "Anti-pattern" subagent to grep for known bad patterns from the plan
3. **Code quality review** - Deploy a "Code Quality" subagent to review changes
4. **Commit only if verified** - Deploy a "Commit" subagent *only after* verification passes; otherwise, do not commit
### Between Phases:
Deploy a "Branch/Sync" subagent to:
- Push to working branch after each verified phase
- Prepare the next phase handoff so the next phase's subagents start fresh but have plan context
## Failure Modes to Prevent
- Don't invent APIs that "should" exist - verify against docs
- Don't add undocumented parameters - copy exact signatures
- Don't skip verification - deploy a verification subagent and run the checklist
- Don't commit before verification passes (or without explicit orchestrator approval)
-66
View File
@@ -1,66 +0,0 @@
---
description: "Create an implementation plan with documentation discovery"
argument-hint: "[feature or task description]"
---
You are an ORCHESTRATOR.
Create an LLM-friendly plan in phases that can be executed consecutively in new chat contexts.
Delegation model (because subagents can under-report):
- Use subagents for *fact gathering and extraction* (docs, examples, signatures, grep results).
- Keep *synthesis and plan authoring* with the orchestrator (phase boundaries, task framing, final wording).
- If a subagent report is incomplete or lacks evidence, the orchestrator must re-check with targeted reads/greps before finalizing the plan.
Subagent reporting contract (MANDATORY):
- Each subagent response must include:
1) Sources consulted (files/URLs) and what was read
2) Concrete findings (exact API names/signatures; exact file paths/locations)
3) Copy-ready snippet locations (example files/sections to copy)
4) "Confidence" note + known gaps (what might still be missing)
- Reject and redeploy the subagent if it reports conclusions without sources.
## Plan Structure Requirements
### Phase 0: Documentation Discovery (ALWAYS FIRST)
Before planning implementation, you MUST:
Deploy one or more "Documentation Discovery" subagents to:
1. Search for and read relevant documentation, examples, and existing patterns
2. Identify the actual APIs, methods, and signatures available (not assumed)
3. Create a brief "Allowed APIs" list citing specific documentation sources
4. Note any anti-patterns to avoid (methods that DON'T exist, deprecated parameters)
Then the orchestrator consolidates their findings into a single Phase 0 output.
### Each Implementation Phase Must Include:
1. **What to implement** - Frame tasks to COPY from docs, not transform existing code
- Good: "Copy the V2 session pattern from docs/examples.ts:45-60"
- Bad: "Migrate the existing code to V2"
2. **Documentation references** - Cite specific files/lines for patterns to follow
3. **Verification checklist** - How to prove this phase worked (tests, grep checks)
4. **Anti-pattern guards** - What NOT to do (invented APIs, undocumented params)
Subagent-friendly split:
- Subagents can propose candidate doc references and verification commands.
- The orchestrator must write the final phase text, ensuring tasks are copy-based, scoped, and independently executable.
### Final Phase: Verification
1. Verify all implementations match documentation
2. Check for anti-patterns (grep for known bad patterns)
3. Run tests to confirm functionality
Delegation guidance:
- Deploy a "Verification" subagent to draft the checklist and commands.
- The orchestrator must review the checklist for completeness and ensure it maps to earlier phase outputs.
## Key Principles
- Documentation Availability ≠ Usage: Explicitly require reading docs
- Task Framing Matters: Direct agents to docs, not just outcomes
- Verify > Assume: Require proof, not assumptions about APIs
- Session Boundaries: Each phase should be self-contained with its own doc references
## Anti-Patterns to Prevent
- Inventing API methods that "should" exist
- Adding parameters not in documentation
- Skipping verification steps
- Assuming structure without checking examples
+16 -30
View File
@@ -7,7 +7,7 @@
"hooks": [
{
"type": "command",
"command": "${CLAUDE_PLUGIN_ROOT}/scripts/setup.sh",
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; \"$_R/scripts/setup.sh\"",
"timeout": 300
}
]
@@ -19,22 +19,17 @@
"hooks": [
{
"type": "command",
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/smart-install.js\"",
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/smart-install.js\"",
"timeout": 300
}
]
},
{
"matcher": "startup|clear|compact",
"hooks": [
},
{
"type": "command",
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" start",
"timeout": 60
},
{
"type": "command",
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" hook claude-code context",
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code context",
"timeout": 60
}
]
@@ -45,12 +40,7 @@
"hooks": [
{
"type": "command",
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
"timeout": 60
},
{
"type": "command",
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" hook claude-code session-init",
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code session-init",
"timeout": 60
}
]
@@ -62,12 +52,7 @@
"hooks": [
{
"type": "command",
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
"timeout": 60
},
{
"type": "command",
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" hook claude-code observation",
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code observation",
"timeout": 120
}
]
@@ -78,17 +63,18 @@
"hooks": [
{
"type": "command",
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
"timeout": 60
},
{
"type": "command",
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" hook claude-code summarize",
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code summarize",
"timeout": 120
},
}
]
}
],
"SessionEnd": [
{
"hooks": [
{
"type": "command",
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" hook claude-code session-complete",
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code session-complete",
"timeout": 30
}
]
+7
View File
@@ -0,0 +1,7 @@
{
"name": "Law Study (Chill)",
"prompts": {
"recording_focus": "WHAT TO RECORD (HIGH SIGNAL ONLY)\n----------------------------------\nOnly record what would be painful to reconstruct later:\n- Issue-spotting triggers: specific fact patterns that signal a testable issue\n- Professor's explicit emphasis, frameworks, or exam tips\n- Counterintuitive holdings or gotchas that contradict intuition\n- Cross-case connections that reframe how a doctrine works\n- A synthesized rule only if it distills something non-obvious from multiple sources\n\nSkip anything that could be looked up in a casebook in under 60 seconds.\n\nUse verbs like: held, established, revealed, distinguished, flagged",
"skip_guidance": "WHEN TO SKIP (LIBERAL — WHEN IN DOUBT, SKIP)\n---------------------------------------------\nSkip freely:\n- All case briefs, even condensed ones, unless the holding is counterintuitive\n- Any rule or doctrine stated plainly in the casebook without nuance\n- Definitions of standard legal terms\n- Procedural history\n- Any fact pattern or case that wasn't specifically emphasized by the professor\n- Anything you could find again in under 60 seconds\n- **No output necessary if skipping.**"
}
}
+85
View File
@@ -0,0 +1,85 @@
# Legal Study Assistant
You are a rigorous legal study partner for a law student. Your job is to help them understand the law deeply enough to reason through novel fact patterns independently on exams and in practice.
---
## Your Role
- Help the student read, analyze, and extract meaning from legal documents
- Ask questions that surface the student's reasoning, not just answers
- Flag what matters for exams and what professors tend to emphasize
- Push back when the student's analysis is imprecise or incomplete
- Never write their exam answers — teach them to write their own
---
## Reading Cases Together
When the student shares a case or document:
1. Read it fully before saying anything. No skimming.
2. Identify the procedural posture, then the issue, then the holding, then the reasoning.
3. Separate holding from dicta explicitly — this distinction is always fair game.
4. Surface ambiguity when the court was evasive. That ambiguity is often the exam question.
5. Ask: "Which facts were outcome-determinative? What if those facts changed?"
**Case briefs are always 3 sentences max:**
> [Key facts that triggered the issue]. The court held [holding + extracted rule]. [Why this rule exists or how it fits the doctrine — only if non-obvious.]
---
## Critical Questions to Drive Analysis
After reading any legal material, push the student to answer:
- What is the rule stated as elements?
- What did the dissent argue and why does it matter?
- How does this fit — or conflict with — earlier cases?
- What fact pattern on an exam triggers this rule?
- What does the professor emphasize about this? Their framing is the exam framing.
- Is the law settled or contested here?
---
## Issue Spotting
When working through a fact pattern:
1. Read the entire hypo before naming any issues.
2. List every potential claim and defense — err toward inclusion.
3. For each issue: rule → application to these specific facts → where the argument turns.
4. Treat "irrelevant" facts as planted triggers. Nothing in an exam hypo is accidental.
5. Calibrate to the professor's emphasis — they wrote the exam.
---
## Synthesizing Doctrine
When pulling together multiple cases or a whole doctrine:
1. Find the common principle across all the cases.
2. Build the rule as a spectrum or taxonomy when cases represent different scenarios.
3. State the limiting principle — where does this rule stop and why.
4. Majority rule first, then minority positions with their rationale.
5. Identify the live tension — what the courts haven't resolved yet.
---
## Tone and Pace
- Be direct. Law school trains precision — model it.
- When the student is vague, say so and ask them to be specific.
- Celebrate when they spot something sharp. Legal reasoning is hard.
- Match the student's pace — deep dive when they want to go deep, quick synthesis when they're reviewing.
---
## Starting a Session
The student should tell you:
- Which course this is for
- What material they're working through (cases, statute, doctrine, hypo practice)
- What kind of help they want: deep analysis, synthesis, issue spotting, or exam review
Example: *"Contracts — working through consideration doctrine. Here are four cases. Help me find the through-line and identify what patterns trigger the issue on an exam."*
+120
View File
@@ -0,0 +1,120 @@
{
"name": "Law Study",
"description": "Legal study and exam preparation for law students",
"version": "1.0.0",
"observation_types": [
{
"id": "case-holding",
"label": "Case Holding",
"description": "Case brief (2-3 sentences: key facts + holding) with extracted legal rule",
"emoji": "⚖️",
"work_emoji": "📖"
},
{
"id": "issue-pattern",
"label": "Issue Pattern",
"description": "Exam trigger or fact pattern that signals a legal issue to spot",
"emoji": "🎯",
"work_emoji": "🔍"
},
{
"id": "prof-framework",
"label": "Prof Framework",
"description": "Professor's analytical lens, emphasis, or approach to a topic or doctrine",
"emoji": "🧑‍🏫",
"work_emoji": "📝"
},
{
"id": "doctrine-rule",
"label": "Doctrine / Rule",
"description": "Legal test, standard, or doctrine synthesized from cases, statutes, or restatements",
"emoji": "📜",
"work_emoji": "🔍"
},
{
"id": "argument-structure",
"label": "Argument Structure",
"description": "Legal argument or counter-argument worked through with analytical steps",
"emoji": "🗣️",
"work_emoji": "⚖️"
},
{
"id": "cross-case-connection",
"label": "Cross-Case Connection",
"description": "Insight linking multiple cases, doctrines, or topics that reveals a deeper principle",
"emoji": "🔗",
"work_emoji": "🔍"
}
],
"observation_concepts": [
{
"id": "exam-relevant",
"label": "Exam Relevant",
"description": "Flagged by professor or likely to appear on exams based on emphasis"
},
{
"id": "minority-position",
"label": "Minority Position",
"description": "Dissent, minority rule, or alternative jurisdictional approach worth knowing"
},
{
"id": "gotcha",
"label": "Gotcha",
"description": "Subtle nuance, counterintuitive result, or common mistake students get wrong"
},
{
"id": "unsettled-law",
"label": "Unsettled Law",
"description": "Circuit split, open question, or evolving area of law"
},
{
"id": "policy-rationale",
"label": "Policy Rationale",
"description": "Normative or policy argument underlying a rule or holding"
},
{
"id": "course-theme",
"label": "Course Theme",
"description": "How this case or rule connects to the overarching narrative or theory of the course"
}
],
"prompts": {
"system_identity": "You are Claude-Mem, a specialized observer tool for creating searchable memory FOR FUTURE SESSIONS.\n\nCRITICAL: Record what was READ, ANALYZED, SYNTHESIZED, or LEARNED about the law, not what you (the observer) are doing.\n\nYou do not have access to tools. All information you need is provided in <observed_from_primary_session> messages. Create observations from what you observe - no investigation needed.",
"spatial_awareness": "SPATIAL AWARENESS: Tool executions include the working directory (tool_cwd) to help you understand:\n- Which repository/project is being worked on\n- Where files are located relative to the project root\n- How to match requested paths to actual execution paths",
"observer_role": "Your job is to monitor a different Claude Code session happening RIGHT NOW, with the goal of creating observations and progress summaries as legal study is being done LIVE by the user. You are NOT the one doing the work - you are ONLY observing and recording what is being read, analyzed, briefed, or synthesized in the other session.",
"recording_focus": "WHAT TO RECORD\n--------------\nFocus on legal knowledge and exam-ready insights:\n- Case holdings distilled to 2-3 sentences (key facts + holding + rule)\n- Legal tests, elements, and standards extracted from cases or statutes\n- Issue-spotting triggers: what fact patterns signal which legal issues\n- Professor's framing, emphasis, or analytical approach to a doctrine\n- Arguments and counter-arguments worked through\n- Connections across cases or doctrines that reveal underlying principles\n\nUse verbs like: held, established, synthesized, identified, distinguished, analyzed, revealed, connected\n\n✅ GOOD EXAMPLES (describes what was learned about the law):\n- \"Palsgraf established proximate cause requires the harm be foreseeable to the defendant at the time of conduct\"\n- \"Prof frames consideration doctrine around the bargain theory, not benefit-detriment — exam answers should reflect this\"\n- \"When fact pattern shows concurrent causation, issue-spot both but-for AND substantial factor tests\"\n\n❌ BAD EXAMPLES (describes observation process - DO NOT DO THIS):\n- \"Analyzed the case and recorded findings about proximate cause\"\n- \"Tracked professor's comments and stored the framework\"\n- \"Monitored discussion of consideration and noted the approach\"",
"skip_guidance": "WHEN TO SKIP\n------------\nSkip these — not worth recording:\n- Full case briefs (only record the 2-3 sentence distilled version with the rule)\n- Re-reading the same case or passage without new insight\n- Definitions of basic terms the student already knows\n- Routine case brief formatting with no analytical content\n- Simple fact summaries that don't extract a rule or pattern\n- Procedural history details not relevant to the legal rule\n- **No output necessary if skipping.**",
"type_guidance": "**type**: MUST be EXACTLY one of these 6 options (no other values allowed):\n - case-holding: case brief (2-3 sentences: key facts + holding) with extracted legal rule\n - issue-pattern: exam trigger or fact pattern that signals a legal issue to spot\n - prof-framework: professor's analytical lens, emphasis, or approach to a topic or doctrine\n - doctrine-rule: legal test, standard, or doctrine synthesized from cases, statutes, or restatements\n - argument-structure: legal argument or counter-argument worked through with analytical steps\n - cross-case-connection: insight linking multiple cases, doctrines, or topics that reveals a deeper principle",
"concept_guidance": "**concepts**: 2-5 knowledge-type categories. MUST use ONLY these exact keywords:\n - exam-relevant: flagged by professor or likely to appear on exams\n - minority-position: dissent, minority rule, or alternative jurisdictional approach\n - gotcha: subtle nuance, counterintuitive result, or common mistake\n - unsettled-law: circuit split, open question, or evolving area\n - policy-rationale: normative or policy argument underlying a rule\n - course-theme: connects to the overarching narrative or theory of the course\n\n IMPORTANT: Do NOT include the observation type (case-holding/issue-pattern/etc.) as a concept.\n Types and concepts are separate dimensions.",
"field_guidance": "**facts**: Concise, self-contained statements\nEach fact is ONE piece of information\n No pronouns - each fact must stand alone\n Include specific details: case names, rule elements, test names, jurisdiction\n\n**files**: All files or documents read (full paths from project root)",
"output_format_header": "OUTPUT FORMAT\n-------------\nOutput observations using this XML structure:",
"format_examples": "",
"footer": "IMPORTANT! DO NOT do any work right now other than generating this OBSERVATIONS from tool use messages - and remember that you are a memory agent designed to summarize a DIFFERENT claude code session, not this one.\n\nNever reference yourself or your own actions. Do not output anything other than the observation content formatted in the XML structure above. All other output is ignored by the system, and the system has been designed to be smart about token usage. Please spend your tokens wisely on useful observations.\n\nRemember that we record these observations as a way of helping us stay on track with our progress, and to help us keep important decisions and changes at the forefront of our minds! :) Thank you so much for your help!",
"xml_title_placeholder": "[**title**: Case name, doctrine name, or short description of the legal insight]",
"xml_subtitle_placeholder": "[**subtitle**: One sentence capturing the core legal rule or exam relevance (max 24 words)]",
"xml_fact_placeholder": "[Concise, self-contained legal fact — include case names, rule elements, test names]",
"xml_narrative_placeholder": "[**narrative**: Full legal context: what the case held or rule requires, how it connects to other doctrine, why it matters for exams or practice]",
"xml_concept_placeholder": "[exam-relevant | minority-position | gotcha | unsettled-law | policy-rationale | course-theme]",
"xml_file_placeholder": "[path/to/document]",
"xml_summary_request_placeholder": "[Short title capturing the legal topic studied AND what was analyzed or synthesized]",
"xml_summary_investigated_placeholder": "[What cases, statutes, or doctrines were read or examined in this session?]",
"xml_summary_learned_placeholder": "[What legal rules, patterns, or frameworks were extracted and understood?]",
"xml_summary_completed_placeholder": "[What study work was completed? Which cases briefed, which doctrines synthesized, which issue patterns identified?]",
"xml_summary_next_steps_placeholder": "[What topics, cases, or doctrines are being studied next in this session?]",
"xml_summary_notes_placeholder": "[Additional insights about exam strategy, professor emphasis, or cross-topic connections observed in this session]",
"header_memory_start": "LAW STUDY MEMORY START\n=======================",
"header_memory_continued": "LAW STUDY MEMORY CONTINUED\n===========================",
"header_summary_checkpoint": "LAW STUDY SUMMARY CHECKPOINT\n============================",
"continuation_greeting": "Hello memory agent, you are continuing to observe the primary Claude session doing legal study and case analysis.",
"continuation_instruction": "IMPORTANT: Continue generating observations from tool use messages using the XML structure below.",
"summary_instruction": "Write progress notes of what legal material was studied, what rules and patterns were extracted, and what's next. This is a checkpoint to capture study progress so far. The session is ongoing - more cases or doctrines may be analyzed after this summary. Write \"next_steps\" as the current study trajectory (what topics or cases are actively being worked through), not as post-session plans. Always write at least a minimal summary explaining current progress, even if study is still early, so that users see a summary output tied to each study block.",
"summary_context_label": "Claude's Full Response to User:",
"summary_format_instruction": "Respond in this XML format:",
"summary_footer": "IMPORTANT! DO NOT do any work right now other than generating this next PROGRESS SUMMARY - and remember that you are a memory agent designed to summarize a DIFFERENT claude code session, not this one.\n\nNever reference yourself or your own actions. Do not output anything other than the summary content formatted in the XML structure above. All other output is ignored by the system, and the system has been designed to be smart about token usage. Please spend your tokens wisely on useful summary content.\n\nThank you, this summary will be very useful for keeping track of legal study progress!"
}
}
+11 -2
View File
@@ -1,11 +1,20 @@
{
"name": "claude-mem-plugin",
"version": "10.3.2",
"version": "10.6.0",
"private": true,
"description": "Runtime dependencies for claude-mem bundled hooks",
"type": "module",
"dependencies": {
"@chroma-core/default-embed": "^0.1.9"
"tree-sitter-cli": "^0.26.5",
"tree-sitter-c": "^0.24.1",
"tree-sitter-cpp": "^0.23.4",
"tree-sitter-go": "^0.25.0",
"tree-sitter-java": "^0.23.5",
"tree-sitter-javascript": "^0.25.0",
"tree-sitter-python": "^0.25.0",
"tree-sitter-ruby": "^0.23.1",
"tree-sitter-rust": "^0.24.0",
"tree-sitter-typescript": "^0.23.2"
},
"engines": {
"node": ">=18.0.0",
+2
View File
@@ -1,3 +1,5 @@
Never read built source files in this directory. These are compiled outputs — read the source files in `src/` instead.
<claude-mem-context>
# Recent Activity
+48 -2
View File
@@ -12,12 +12,37 @@
* Fixes #818: Worker fails to start on fresh install
*/
import { spawnSync, spawn } from 'child_process';
import { existsSync } from 'fs';
import { join } from 'path';
import { existsSync, readFileSync } from 'fs';
import { join, dirname, resolve } from 'path';
import { homedir } from 'os';
import { fileURLToPath } from 'url';
const IS_WINDOWS = process.platform === 'win32';
// Self-resolve plugin root when CLAUDE_PLUGIN_ROOT is not set by Claude Code.
// Upstream bug: anthropics/claude-code#24529 — Stop hooks (and on Linux, all hooks)
// don't receive CLAUDE_PLUGIN_ROOT, causing script paths to resolve to /scripts/...
// which doesn't exist. This fallback derives the plugin root from bun-runner.js's
// own filesystem location (this file lives in <plugin-root>/scripts/).
const __bun_runner_dirname = dirname(fileURLToPath(import.meta.url));
const RESOLVED_PLUGIN_ROOT = process.env.CLAUDE_PLUGIN_ROOT || resolve(__bun_runner_dirname, '..');
/**
* Fix script path arguments that were broken by empty CLAUDE_PLUGIN_ROOT.
* When CLAUDE_PLUGIN_ROOT is empty, "${CLAUDE_PLUGIN_ROOT}/scripts/foo.cjs"
* expands to "/scripts/foo.cjs" which doesn't exist. Detect this and rewrite
* the path using our self-resolved plugin root.
*/
function fixBrokenScriptPath(argPath) {
if (argPath.startsWith('/scripts/') && !existsSync(argPath)) {
const fixedPath = join(RESOLVED_PLUGIN_ROOT, argPath);
if (existsSync(fixedPath)) {
return fixedPath;
}
}
return argPath;
}
/**
* Find Bun executable - checks PATH first, then common install locations
*/
@@ -54,6 +79,24 @@ function findBun() {
return null;
}
// Early exit if plugin is disabled in Claude Code settings (#781).
// Sync read + JSON parse — fastest possible check before spawning Bun.
function isPluginDisabledInClaudeSettings() {
try {
const configDir = process.env.CLAUDE_CONFIG_DIR || join(homedir(), '.claude');
const settingsPath = join(configDir, 'settings.json');
if (!existsSync(settingsPath)) return false;
const settings = JSON.parse(readFileSync(settingsPath, 'utf-8'));
return settings?.enabledPlugins?.['claude-mem@thedotmack'] === false;
} catch {
return false;
}
}
if (isPluginDisabledInClaudeSettings()) {
process.exit(0);
}
// Get args: node bun-runner.js <script> [args...]
const args = process.argv.slice(2);
@@ -62,6 +105,9 @@ if (args.length === 0) {
process.exit(1);
}
// Fix broken script paths caused by empty CLAUDE_PLUGIN_ROOT (#1215)
args[0] = fixBrokenScriptPath(args[0]);
const bunPath = findBun();
if (!bunPath) {
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
-228
View File
@@ -1,228 +0,0 @@
#!/usr/bin/env bash
#
# claude-mem Setup Hook
# Ensures dependencies are installed before plugin runs
#
set -euo pipefail
# Use CLAUDE_PLUGIN_ROOT if available, otherwise detect from script location
if [[ -z "${CLAUDE_PLUGIN_ROOT:-}" ]]; then
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(dirname "$SCRIPT_DIR")"
else
ROOT="$CLAUDE_PLUGIN_ROOT"
fi
MARKER="$ROOT/.install-version"
PKG_JSON="$ROOT/package.json"
# Colors (when terminal supports it)
if [[ -t 2 ]]; then
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
else
RED='' GREEN='' YELLOW='' BLUE='' NC=''
fi
log_info() { echo -e "${BLUE}${NC} $*" >&2; }
log_ok() { echo -e "${GREEN}${NC} $*" >&2; }
log_warn() { echo -e "${YELLOW}${NC} $*" >&2; }
log_error() { echo -e "${RED}${NC} $*" >&2; }
#
# Detect Bun - check PATH and common locations
#
find_bun() {
# Try PATH first
if command -v bun &>/dev/null; then
echo "bun"
return 0
fi
# Check common install locations
local paths=(
"$HOME/.bun/bin/bun"
"/usr/local/bin/bun"
"/opt/homebrew/bin/bun"
)
for p in "${paths[@]}"; do
if [[ -x "$p" ]]; then
echo "$p"
return 0
fi
done
return 1
}
#
# Detect uv - check PATH and common locations
#
find_uv() {
# Try PATH first
if command -v uv &>/dev/null; then
echo "uv"
return 0
fi
# Check common install locations
local paths=(
"$HOME/.local/bin/uv"
"$HOME/.cargo/bin/uv"
"/usr/local/bin/uv"
"/opt/homebrew/bin/uv"
)
for p in "${paths[@]}"; do
if [[ -x "$p" ]]; then
echo "$p"
return 0
fi
done
return 1
}
#
# Get package.json version
#
get_pkg_version() {
if [[ -f "$PKG_JSON" ]]; then
# Simple grep-based extraction (no jq dependency)
grep -o '"version"[[:space:]]*:[[:space:]]*"[^"]*"' "$PKG_JSON" | head -1 | sed 's/.*"\([^"]*\)"$/\1/'
fi
}
#
# Get marker version (if exists)
#
get_marker_version() {
if [[ -f "$MARKER" ]]; then
grep -o '"version"[[:space:]]*:[[:space:]]*"[^"]*"' "$MARKER" | head -1 | sed 's/.*"\([^"]*\)"$/\1/'
fi
}
#
# Get marker's recorded bun version
#
get_marker_bun() {
if [[ -f "$MARKER" ]]; then
grep -o '"bun"[[:space:]]*:[[:space:]]*"[^"]*"' "$MARKER" | head -1 | sed 's/.*"\([^"]*\)"$/\1/'
fi
}
#
# Check if install is needed
#
needs_install() {
# No node_modules? Definitely need install
if [[ ! -d "$ROOT/node_modules" ]]; then
return 0
fi
# No marker? Need install
if [[ ! -f "$MARKER" ]]; then
return 0
fi
local pkg_ver marker_ver bun_ver marker_bun
pkg_ver=$(get_pkg_version)
marker_ver=$(get_marker_version)
# Version mismatch? Need install
if [[ "$pkg_ver" != "$marker_ver" ]]; then
return 0
fi
# Bun version changed? Need install
if BUN_PATH=$(find_bun); then
bun_ver=$("$BUN_PATH" --version 2>/dev/null || echo "")
marker_bun=$(get_marker_bun)
if [[ -n "$bun_ver" && "$bun_ver" != "$marker_bun" ]]; then
return 0
fi
fi
# All good, no install needed
return 1
}
#
# Write version marker after successful install
#
write_marker() {
local bun_ver uv_ver pkg_ver
pkg_ver=$(get_pkg_version)
bun_ver=$("$BUN_PATH" --version 2>/dev/null || echo "unknown")
if UV_PATH=$(find_uv); then
uv_ver=$("$UV_PATH" --version 2>/dev/null | head -1 || echo "unknown")
else
uv_ver="not-installed"
fi
cat > "$MARKER" <<EOF
{
"version": "$pkg_ver",
"bun": "$bun_ver",
"uv": "$uv_ver",
"installedAt": "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
}
EOF
}
#
# Main
#
# 1. Check for Bun
BUN_PATH=$(find_bun) || true
if [[ -z "$BUN_PATH" ]]; then
log_error "Bun runtime not found!"
echo "" >&2
echo "claude-mem requires Bun to run. Please install it:" >&2
echo "" >&2
echo " curl -fsSL https://bun.sh/install | bash" >&2
echo "" >&2
echo "Or on macOS with Homebrew:" >&2
echo "" >&2
echo " brew install oven-sh/bun/bun" >&2
echo "" >&2
echo "Then restart your terminal and try again." >&2
exit 1
fi
BUN_VERSION=$("$BUN_PATH" --version 2>/dev/null || echo "unknown")
log_ok "Bun $BUN_VERSION found at $BUN_PATH"
# 2. Check for uv (optional - for Python/Chroma support)
UV_PATH=$(find_uv) || true
if [[ -z "$UV_PATH" ]]; then
log_warn "uv not found (optional - needed for Python/Chroma vector search)"
echo " To install: curl -LsSf https://astral.sh/uv/install.sh | sh" >&2
else
UV_VERSION=$("$UV_PATH" --version 2>/dev/null | head -1 || echo "unknown")
log_ok "uv $UV_VERSION found"
fi
# 3. Install dependencies if needed
if needs_install; then
log_info "Installing dependencies with Bun..."
if ! "$BUN_PATH" install --cwd "$ROOT"; then
log_error "Failed to install dependencies"
exit 1
fi
write_marker
log_ok "Dependencies installed ($(get_pkg_version))"
else
log_ok "Dependencies up to date ($(get_marker_version))"
fi
exit 0
+117 -12
View File
@@ -4,16 +4,72 @@
*
* Ensures Bun runtime and uv (Python package manager) are installed
* (auto-installs if missing) and handles dependency installation when needed.
*
* Resolves the install directory from CLAUDE_PLUGIN_ROOT (set by Claude Code
* for both cache and marketplace installs), falling back to script location
* and legacy paths.
*/
import { existsSync, readFileSync, writeFileSync } from 'fs';
import { execSync, spawnSync } from 'child_process';
import { join } from 'path';
import { join, dirname } from 'path';
import { homedir } from 'os';
import { fileURLToPath } from 'url';
const ROOT = join(homedir(), '.claude', 'plugins', 'marketplaces', 'thedotmack');
const MARKER = join(ROOT, '.install-version');
// Early exit if plugin is disabled in Claude Code settings (#781)
function isPluginDisabledInClaudeSettings() {
try {
const configDir = process.env.CLAUDE_CONFIG_DIR || join(homedir(), '.claude');
const settingsPath = join(configDir, 'settings.json');
if (!existsSync(settingsPath)) return false;
const settings = JSON.parse(readFileSync(settingsPath, 'utf-8'));
return settings?.enabledPlugins?.['claude-mem@thedotmack'] === false;
} catch {
return false;
}
}
if (isPluginDisabledInClaudeSettings()) {
process.exit(0);
}
const IS_WINDOWS = process.platform === 'win32';
/**
* Resolve the plugin root directory where dependencies should be installed.
*
* Priority:
* 1. CLAUDE_PLUGIN_ROOT env var (set by Claude Code for hooks works for
* both cache-based and marketplace installs)
* 2. Script location (dirname of this file, up one level from scripts/)
* 3. XDG path (~/.config/claude/plugins/marketplaces/thedotmack)
* 4. Legacy path (~/.claude/plugins/marketplaces/thedotmack)
*/
function resolveRoot() {
// CLAUDE_PLUGIN_ROOT is the authoritative location set by Claude Code
if (process.env.CLAUDE_PLUGIN_ROOT) {
const root = process.env.CLAUDE_PLUGIN_ROOT;
if (existsSync(join(root, 'package.json'))) return root;
}
// Derive from script location (this file is in <root>/scripts/)
try {
const scriptDir = dirname(fileURLToPath(import.meta.url));
const candidate = dirname(scriptDir);
if (existsSync(join(candidate, 'package.json'))) return candidate;
} catch {
// import.meta.url not available
}
// Probe XDG path, then legacy
const marketplaceRel = join('plugins', 'marketplaces', 'thedotmack');
const xdg = join(homedir(), '.config', 'claude', marketplaceRel);
if (existsSync(join(xdg, 'package.json'))) return xdg;
return join(homedir(), '.claude', marketplaceRel);
}
const ROOT = resolveRoot();
const MARKER = join(ROOT, '.install-version');
/**
* Check if Bun is installed and accessible
*/
@@ -164,14 +220,14 @@ function installBun() {
// Windows: Use PowerShell installer
console.error(' Installing via PowerShell...');
execSync('powershell -c "irm bun.sh/install.ps1 | iex"', {
stdio: 'inherit',
stdio: ['pipe', 'pipe', 'inherit'],
shell: true
});
} else {
// Unix/macOS: Use curl installer
console.error(' Installing via curl...');
execSync('curl -fsSL https://bun.sh/install | bash', {
stdio: 'inherit',
stdio: ['pipe', 'pipe', 'inherit'],
shell: true
});
}
@@ -229,14 +285,14 @@ function installUv() {
// Windows: Use PowerShell installer
console.error(' Installing via PowerShell...');
execSync('powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"', {
stdio: 'inherit',
stdio: ['pipe', 'pipe', 'inherit'],
shell: true
});
} else {
// Unix/macOS: Use curl installer
console.error(' Installing via curl...');
execSync('curl -LsSf https://astral.sh/uv/install.sh | sh', {
stdio: 'inherit',
stdio: ['pipe', 'pipe', 'inherit'],
shell: true
});
}
@@ -287,7 +343,7 @@ function installUv() {
* Add shell alias for claude-mem command
*/
function installCLI() {
const WORKER_CLI = join(ROOT, 'plugin', 'scripts', 'worker-service.cjs');
const WORKER_CLI = join(ROOT, 'scripts', 'worker-service.cjs');
const bunPath = getBunPath() || 'bun';
const aliasLine = `alias claude-mem='${bunPath} "${WORKER_CLI}"'`;
const markerPath = join(ROOT, '.cli-installed');
@@ -370,14 +426,18 @@ function installDeps() {
// Quote path for Windows paths with spaces
const bunCmd = IS_WINDOWS && bunPath.includes(' ') ? `"${bunPath}"` : bunPath;
// Use pipe for stdout to prevent non-JSON output leaking to Claude Code hooks.
// stderr is inherited so progress/errors are still visible to the user.
const installStdio = ['pipe', 'pipe', 'inherit'];
let bunSucceeded = false;
try {
execSync(`${bunCmd} install`, { cwd: ROOT, stdio: 'inherit', shell: IS_WINDOWS });
execSync(`${bunCmd} install`, { cwd: ROOT, stdio: installStdio, shell: IS_WINDOWS });
bunSucceeded = true;
} catch {
// First attempt failed, try with force flag
try {
execSync(`${bunCmd} install --force`, { cwd: ROOT, stdio: 'inherit', shell: IS_WINDOWS });
execSync(`${bunCmd} install --force`, { cwd: ROOT, stdio: installStdio, shell: IS_WINDOWS });
bunSucceeded = true;
} catch {
// Bun failed completely, will try npm fallback
@@ -389,7 +449,7 @@ function installDeps() {
console.error('⚠️ Bun install failed, falling back to npm...');
console.error(' (This can happen with npm alias packages like *-cjs)');
try {
execSync('npm install', { cwd: ROOT, stdio: 'inherit', shell: IS_WINDOWS });
execSync('npm install', { cwd: ROOT, stdio: installStdio, shell: IS_WINDOWS });
} catch (npmError) {
throw new Error('Both bun and npm install failed: ' + npmError.message);
}
@@ -405,6 +465,31 @@ function installDeps() {
}));
}
/**
* Verify that critical runtime modules are resolvable from the install directory.
* Returns true if all critical modules exist, false otherwise.
*/
function verifyCriticalModules() {
const pkg = JSON.parse(readFileSync(join(ROOT, 'package.json'), 'utf-8'));
const dependencies = Object.keys(pkg.dependencies || {});
const missing = [];
for (const dep of dependencies) {
// Check that the module directory exists in node_modules
const modulePath = join(ROOT, 'node_modules', ...dep.split('/'));
if (!existsSync(modulePath)) {
missing.push(dep);
}
}
if (missing.length > 0) {
console.error(`❌ Post-install check failed: missing modules: ${missing.join(', ')}`);
return false;
}
return true;
}
// Main execution
try {
// Step 1: Ensure Bun is installed and meets minimum version (REQUIRED)
@@ -425,7 +510,7 @@ try {
console.error(`⚠️ Bun ${currentVersion} is outdated. Minimum required: ${MIN_BUN_VERSION}`);
console.error(' Upgrading bun...');
try {
execSync('bun upgrade', { stdio: 'inherit', shell: IS_WINDOWS });
execSync('bun upgrade', { stdio: ['pipe', 'pipe', 'inherit'], shell: IS_WINDOWS });
if (!isBunVersionSufficient()) {
console.error(`❌ Bun upgrade failed. Please manually upgrade: bun upgrade`);
process.exit(1);
@@ -456,6 +541,21 @@ try {
const newVersion = pkg.version;
installDeps();
// Verify critical modules are resolvable
if (!verifyCriticalModules()) {
console.error('⚠️ Retrying install with npm...');
try {
execSync('npm install --production', { cwd: ROOT, stdio: ['pipe', 'pipe', 'inherit'], shell: IS_WINDOWS });
} catch {
// npm also failed
}
if (!verifyCriticalModules()) {
console.error('❌ Dependencies could not be installed. Plugin may not work correctly.');
process.exit(1);
}
}
console.error('✅ Dependencies installed');
// Auto-restart worker to pick up new code
@@ -481,7 +581,12 @@ try {
// Step 4: Install CLI to PATH
installCLI();
// Output valid JSON for Claude Code hook contract
console.log(JSON.stringify({ continue: true, suppressOutput: true }));
} catch (e) {
console.error('❌ Installation failed:', e.message);
// Still output valid JSON so Claude Code doesn't show a confusing error
console.log(JSON.stringify({ continue: true, suppressOutput: true }));
process.exit(1);
}
File diff suppressed because one or more lines are too long
@@ -1,5 +1,5 @@
---
name: do-plan
name: do
description: Execute a phased implementation plan using subagents. Use when asked to execute, run, or carry out a plan — especially one created by make-plan.
---
+63
View File
@@ -0,0 +1,63 @@
---
name: make-plan
description: Create a detailed, phased implementation plan with documentation discovery. Use when asked to plan a feature, task, or multi-step implementation — especially before executing with do.
---
# Make Plan
You are an ORCHESTRATOR. Create an LLM-friendly plan in phases that can be executed consecutively in new chat contexts.
## Delegation Model
Use subagents for *fact gathering and extraction* (docs, examples, signatures, grep results). Keep *synthesis and plan authoring* with the orchestrator (phase boundaries, task framing, final wording). If a subagent report is incomplete or lacks evidence, re-check with targeted reads/greps before finalizing.
### Subagent Reporting Contract (MANDATORY)
Each subagent response must include:
1. Sources consulted (files/URLs) and what was read
2. Concrete findings (exact API names/signatures; exact file paths/locations)
3. Copy-ready snippet locations (example files/sections to copy)
4. "Confidence" note + known gaps (what might still be missing)
Reject and redeploy the subagent if it reports conclusions without sources.
## Plan Structure
### Phase 0: Documentation Discovery (ALWAYS FIRST)
Before planning implementation, deploy "Documentation Discovery" subagents to:
1. Search for and read relevant documentation, examples, and existing patterns
2. Identify the actual APIs, methods, and signatures available (not assumed)
3. Create a brief "Allowed APIs" list citing specific documentation sources
4. Note any anti-patterns to avoid (methods that DON'T exist, deprecated parameters)
The orchestrator consolidates findings into a single Phase 0 output.
### Each Implementation Phase Must Include
1. **What to implement** — Frame tasks to COPY from docs, not transform existing code
- Good: "Copy the V2 session pattern from docs/examples.ts:45-60"
- Bad: "Migrate the existing code to V2"
2. **Documentation references** — Cite specific files/lines for patterns to follow
3. **Verification checklist** — How to prove this phase worked (tests, grep checks)
4. **Anti-pattern guards** — What NOT to do (invented APIs, undocumented params)
### Final Phase: Verification
1. Verify all implementations match documentation
2. Check for anti-patterns (grep for known bad patterns)
3. Run tests to confirm functionality
## Key Principles
- Documentation Availability ≠ Usage: Explicitly require reading docs
- Task Framing Matters: Direct agents to docs, not just outcomes
- Verify > Assume: Require proof, not assumptions about APIs
- Session Boundaries: Each phase should be self-contained with its own doc references
## Anti-Patterns to Prevent
- Inventing API methods that "should" exist
- Adding parameters not in documentation
- Skipping verification steps
- Assuming structure without checking examples
+145
View File
@@ -0,0 +1,145 @@
---
name: smart-explore
description: Token-optimized structural code search using tree-sitter AST parsing. Use instead of reading full files when you need to understand code structure, find functions, or explore a codebase efficiently.
---
# Smart Explore
Structural code exploration using AST parsing. **This skill overrides your default exploration behavior.** While this skill is active, use smart_search/smart_outline/smart_unfold as your primary tools instead of Read, Grep, and Glob.
**Core principle:** Index first, fetch on demand. Give yourself a map of the code before loading implementation details. The question before every file read should be: "do I need to see all of this, or can I get a structural overview first?" The answer is almost always: get the map.
## Your Next Tool Call
This skill only loads instructions. You must call the MCP tools yourself. Your next action should be one of:
```
smart_search(query="<topic>", path="./src") -- discover files + symbols across a directory
smart_outline(file_path="<file>") -- structural skeleton of one file
smart_unfold(file_path="<file>", symbol_name="<name>") -- full source of one symbol
```
Do NOT run Grep, Glob, Read, or find to discover files first. `smart_search` walks directories, parses all code files, and returns ranked symbols in one call. It replaces the Glob → Grep → Read discovery cycle.
## 3-Layer Workflow
### Step 1: Search -- Discover Files and Symbols
```
smart_search(query="shutdown", path="./src", max_results=15)
```
**Returns:** Ranked symbols with signatures, line numbers, match reasons, plus folded file views (~2-6k tokens)
```
-- Matching Symbols --
function performGracefulShutdown (services/infrastructure/GracefulShutdown.ts:56)
function httpShutdown (services/infrastructure/HealthMonitor.ts:92)
method WorkerService.shutdown (services/worker-service.ts:846)
-- Folded File Views --
services/infrastructure/GracefulShutdown.ts (7 symbols)
services/worker-service.ts (12 symbols)
```
This is your discovery tool. It finds relevant files AND shows their structure. No Glob/find pre-scan needed.
**Parameters:**
- `query` (string, required) -- What to search for (function name, concept, class name)
- `path` (string) -- Root directory to search (defaults to cwd)
- `max_results` (number) -- Max matching symbols, default 20, max 50
- `file_pattern` (string, optional) -- Filter to specific files/paths
### Step 2: Outline -- Get File Structure
```
smart_outline(file_path="services/worker-service.ts")
```
**Returns:** Complete structural skeleton -- all functions, classes, methods, properties, imports (~1-2k tokens per file)
**Skip this step** when Step 1's folded file views already provide enough structure. Most useful for files not covered by the search results.
**Parameters:**
- `file_path` (string, required) -- Path to the file
### Step 3: Unfold -- See Implementation
Review symbols from Steps 1-2. Pick the ones you need. Unfold only those:
```
smart_unfold(file_path="services/worker-service.ts", symbol_name="shutdown")
```
**Returns:** Full source code of the specified symbol including JSDoc, decorators, and complete implementation (~400-2,100 tokens depending on symbol size). AST node boundaries guarantee completeness regardless of symbol size — unlike Read + agent summarization, which may truncate long methods.
**Parameters:**
- `file_path` (string, required) -- Path to the file (as returned by search/outline)
- `symbol_name` (string, required) -- Name of the function/class/method to expand
## When to Use Standard Tools Instead
Use these only when smart_* tools are the wrong fit:
- **Grep:** Exact string/regex search ("find all TODO comments", "where is `ensureWorkerStarted` defined?")
- **Read:** Small files under ~100 lines, non-code files (JSON, markdown, config)
- **Glob:** File path patterns ("find all test files")
- **Explore agent:** When you need synthesized understanding across 6+ files, architecture narratives, or answers to open-ended questions like "how does this entire system work end-to-end?" Smart-explore is a scalpel — it answers "where is this?" and "show me that." It doesn't synthesize cross-file data flows, design decisions, or edge cases across an entire feature.
For code files over ~100 lines, prefer smart_outline + smart_unfold over Read.
## Workflow Examples
**Discover how a feature works (cross-cutting):**
```
1. smart_search(query="shutdown", path="./src")
-> 14 symbols across 7 files, full picture in one call
2. smart_unfold(file_path="services/infrastructure/GracefulShutdown.ts", symbol_name="performGracefulShutdown")
-> See the core implementation
```
**Navigate a large file:**
```
1. smart_outline(file_path="services/worker-service.ts")
-> 1,466 tokens: 12 functions, WorkerService class with 24 members
2. smart_unfold(file_path="services/worker-service.ts", symbol_name="startSessionProcessor")
-> 1,610 tokens: the specific method you need
Total: ~3,076 tokens vs ~12,000 to Read the full file
```
**Write documentation about code (hybrid workflow):**
```
1. smart_search(query="feature name", path="./src") -- discover all relevant files and symbols
2. smart_outline on key files -- understand structure
3. smart_unfold on important functions -- get implementation details
4. Read on small config/markdown/plan files -- get non-code context
```
Use smart_* tools for code exploration, Read for non-code files. Mix freely.
**Exploration then precision:**
```
1. smart_search(query="session", path="./src", max_results=10)
-> 10 ranked symbols: SessionMetadata, SessionQueueProcessor, SessionSummary...
2. Pick the relevant one, unfold it
```
## Token Economics
| Approach | Tokens | Use Case |
|----------|--------|----------|
| smart_outline | ~1,000-2,000 | "What's in this file?" |
| smart_unfold | ~400-2,100 | "Show me this function" |
| smart_search | ~2,000-6,000 | "Find all X across the codebase" |
| search + unfold | ~3,000-8,000 | End-to-end: find and read (the primary workflow) |
| Read (full file) | ~12,000+ | When you truly need everything |
| Explore agent | ~39,000-59,000 | Cross-file synthesis with narrative |
**4-8x savings** on file understanding (outline + unfold vs Read). **11-18x savings** on codebase exploration vs Explore agent. The narrower the query, the wider the gap — a 27-line function costs 55x less to read via unfold than via an Explore agent, because the agent still reads the entire file.
File diff suppressed because one or more lines are too long
+1
View File
@@ -0,0 +1 @@
Never read built source files in this directory. These are compiled outputs — read the source files in `src/` instead.
+38 -3
View File
@@ -59,8 +59,16 @@ async function buildHooks() {
description: 'Runtime dependencies for claude-mem bundled hooks',
type: 'module',
dependencies: {
// Chroma embedding function with native ONNX binaries (can't be bundled)
'@chroma-core/default-embed': '^0.1.9'
'tree-sitter-cli': '^0.26.5',
'tree-sitter-c': '^0.24.1',
'tree-sitter-cpp': '^0.23.4',
'tree-sitter-go': '^0.25.0',
'tree-sitter-java': '^0.23.5',
'tree-sitter-javascript': '^0.25.0',
'tree-sitter-python': '^0.25.0',
'tree-sitter-ruby': '^0.23.1',
'tree-sitter-rust': '^0.24.0',
'tree-sitter-typescript': '^0.23.2',
},
engines: {
node: '>=18.0.0',
@@ -128,7 +136,19 @@ async function buildHooks() {
outfile: `${hooksDir}/${MCP_SERVER.name}.cjs`,
minify: true,
logLevel: 'error',
external: ['bun:sqlite'],
external: [
'bun:sqlite',
'tree-sitter-cli',
'tree-sitter-javascript',
'tree-sitter-typescript',
'tree-sitter-python',
'tree-sitter-go',
'tree-sitter-rust',
'tree-sitter-ruby',
'tree-sitter-java',
'tree-sitter-c',
'tree-sitter-cpp',
],
define: {
'__DEFAULT_PACKAGE_VERSION__': `"${version}"`
},
@@ -162,6 +182,21 @@ async function buildHooks() {
const contextGenStats = fs.statSync(`${hooksDir}/${CONTEXT_GENERATOR.name}.cjs`);
console.log(`✓ context-generator built (${(contextGenStats.size / 1024).toFixed(2)} KB)`);
// Verify critical distribution files exist (skills are source files, not build outputs)
console.log('\n📋 Verifying distribution files...');
const requiredDistributionFiles = [
'plugin/skills/mem-search/SKILL.md',
'plugin/skills/smart-explore/SKILL.md',
'plugin/hooks/hooks.json',
'plugin/.claude-plugin/plugin.json',
];
for (const filePath of requiredDistributionFiles) {
if (!fs.existsSync(filePath)) {
throw new Error(`Missing required distribution file: ${filePath}`);
}
}
console.log('✓ All required distribution files present');
console.log('\n✅ Worker service, MCP server, and context generator built successfully!');
console.log(` Output: ${hooksDir}/`);
console.log(` - Worker: worker-service.cjs`);
+5
View File
@@ -279,6 +279,11 @@ function formatObservationsForClaudeMd(observations: ObservationRow[], folderPat
* which only writes to existing folders.
*/
function writeClaudeMdToFolderForRegenerate(folderPath: string, newContent: string): void {
const resolvedPath = path.resolve(folderPath);
// Never write inside .git directories — corrupts refs (#1165)
if (resolvedPath.includes('/.git/') || resolvedPath.includes('\\.git\\') || resolvedPath.endsWith('/.git') || resolvedPath.endsWith('\\.git')) return;
const claudeMdPath = path.join(folderPath, 'CLAUDE.md');
const tempFile = `${claudeMdPath}.tmp`;
+53 -17
View File
@@ -7,34 +7,40 @@
*/
import { existsSync, readFileSync, writeFileSync } from 'fs';
import { execSync, spawnSync } from 'child_process';
import { join } from 'path';
import { join, dirname } from 'path';
import { homedir } from 'os';
import { fileURLToPath } from 'url';
const IS_WINDOWS = process.platform === 'win32';
/**
* Resolve the marketplace root directory.
* Resolve the plugin root directory where dependencies should be installed.
*
* Claude Code may store plugins under either `~/.claude/plugins/` (legacy) or
* `~/.config/claude/plugins/` (XDG-compliant, e.g. Nix-managed installs).
* When `CLAUDE_PLUGIN_ROOT` is set we derive the base from it; otherwise we
* probe both candidate paths and fall back to the legacy location.
* Priority:
* 1. CLAUDE_PLUGIN_ROOT env var (set by Claude Code for hooks works for
* both cache-based and marketplace installs)
* 2. Script location (dirname of this file, up one level from scripts/)
* 3. XDG path (~/.config/claude/plugins/marketplaces/thedotmack)
* 4. Legacy path (~/.claude/plugins/marketplaces/thedotmack)
*/
function resolveRoot() {
const marketplaceRel = join('plugins', 'marketplaces', 'thedotmack');
// Derive from CLAUDE_PLUGIN_ROOT (e.g. .../plugins/cache/thedotmack/claude-mem/<ver>)
// CLAUDE_PLUGIN_ROOT is the authoritative location set by Claude Code
if (process.env.CLAUDE_PLUGIN_ROOT) {
let dir = process.env.CLAUDE_PLUGIN_ROOT;
const cacheIndex = dir.indexOf(join('plugins', 'cache'));
if (cacheIndex !== -1) {
const base = dir.substring(0, cacheIndex);
const candidate = join(base, marketplaceRel);
if (existsSync(join(candidate, 'package.json'))) return candidate;
}
const root = process.env.CLAUDE_PLUGIN_ROOT;
if (existsSync(join(root, 'package.json'))) return root;
}
// Probe XDG path first, then legacy
// Derive from script location (this file is in <root>/scripts/)
try {
const scriptDir = dirname(fileURLToPath(import.meta.url));
const candidate = dirname(scriptDir);
if (existsSync(join(candidate, 'package.json'))) return candidate;
} catch {
// import.meta.url not available
}
// Probe XDG path, then legacy
const marketplaceRel = join('plugins', 'marketplaces', 'thedotmack');
const xdg = join(homedir(), '.config', 'claude', marketplaceRel);
if (existsSync(join(xdg, 'package.json'))) return xdg;
@@ -275,12 +281,42 @@ function installDeps() {
}));
}
/**
* Verify that critical runtime modules are resolvable from the install directory.
* Returns true if all critical modules exist, false otherwise.
*/
function verifyCriticalModules() {
const pkg = JSON.parse(readFileSync(join(ROOT, 'package.json'), 'utf-8'));
const dependencies = Object.keys(pkg.dependencies || {});
const missing = [];
for (const dep of dependencies) {
const modulePath = join(ROOT, 'node_modules', ...dep.split('/'));
if (!existsSync(modulePath)) {
missing.push(dep);
}
}
if (missing.length > 0) {
console.error(`❌ Post-install check failed: missing modules: ${missing.join(', ')}`);
return false;
}
return true;
}
// Main execution
try {
if (!isBunInstalled()) installBun();
if (!isUvInstalled()) installUv();
if (needsInstall()) {
installDeps();
if (!verifyCriticalModules()) {
console.error('❌ Dependencies could not be installed. Plugin may not work correctly.');
process.exit(1);
}
console.error('✅ Dependencies installed');
}
} catch (e) {
+1 -1
View File
@@ -76,7 +76,7 @@ try {
const gitignoreExcludes = getGitignoreExcludes(rootDir);
execSync(
`rsync -av --delete --exclude=.git --exclude=/.mcp.json --exclude=bun.lock --exclude=package-lock.json ${gitignoreExcludes} ./ ~/.claude/plugins/marketplaces/thedotmack/`,
`rsync -av --delete --exclude=.git --exclude=bun.lock --exclude=package-lock.json ${gitignoreExcludes} ./ ~/.claude/plugins/marketplaces/thedotmack/`,
{ stdio: 'inherit' }
);
+19
View File
@@ -0,0 +1,19 @@
#!/usr/bin/env node
/**
* Wipes the Chroma data directory so backfillAllProjects rebuilds it on next worker start.
* Chroma is always rebuildable from SQLite this is safe.
*/
const fs = require('fs');
const path = require('path');
const os = require('os');
const chromaDir = path.join(os.homedir(), '.claude-mem', 'chroma');
if (fs.existsSync(chromaDir)) {
const before = fs.readdirSync(chromaDir);
console.log(`Wiping ${chromaDir} (${before.length} items)...`);
fs.rmSync(chromaDir, { recursive: true, force: true });
console.log('Done. Chroma will rebuild from SQLite on next worker restart.');
} else {
console.log('Chroma directory does not exist, nothing to wipe.');
}
+12 -5
View File
@@ -6,7 +6,7 @@ export const claudeCodeAdapter: PlatformAdapter = {
normalizeInput(raw) {
const r = (raw ?? {}) as any;
return {
sessionId: r.session_id,
sessionId: r.session_id ?? r.id ?? r.sessionId,
cwd: r.cwd ?? process.cwd(),
prompt: r.prompt,
toolName: r.tool_name,
@@ -16,13 +16,20 @@ export const claudeCodeAdapter: PlatformAdapter = {
};
},
formatOutput(result) {
if (result.hookSpecificOutput) {
const r = result ?? ({} as HookResult);
if (r.hookSpecificOutput) {
const output: Record<string, unknown> = { hookSpecificOutput: result.hookSpecificOutput };
if (result.systemMessage) {
output.systemMessage = result.systemMessage;
if (r.systemMessage) {
output.systemMessage = r.systemMessage;
}
return output;
}
return { continue: result.continue ?? true, suppressOutput: result.suppressOutput ?? true };
// Only emit fields in the Claude Code hook contract — unrecognized fields
// cause "JSON validation failed" in Stop hooks.
const output: Record<string, unknown> = {};
if (r.systemMessage) {
output.systemMessage = r.systemMessage;
}
return output;
}
};
+8 -3
View File
@@ -3,15 +3,20 @@ import type { PlatformAdapter, NormalizedHookInput, HookResult } from '../types.
// Maps Cursor stdin format - field names differ from Claude Code
// Cursor uses: conversation_id, workspace_roots[], result_json, command/output
// Handle undefined input gracefully for hooks that don't receive stdin
//
// Cursor payload variations (#838, #1049):
// Session ID: conversation_id, generation_id, or id
// Prompt: prompt, query, input, or message (varies by Cursor version/hook type)
// CWD: workspace_roots[0] or cwd
export const cursorAdapter: PlatformAdapter = {
normalizeInput(raw) {
const r = (raw ?? {}) as any;
// Cursor-specific: shell commands come as command/output instead of tool_name/input/response
const isShellCommand = !!r.command && !r.tool_name;
return {
sessionId: r.conversation_id || r.generation_id, // conversation_id preferred
cwd: r.workspace_roots?.[0] ?? process.cwd(), // First workspace root
prompt: r.prompt,
sessionId: r.conversation_id || r.generation_id || r.id,
cwd: r.workspace_roots?.[0] ?? r.cwd ?? process.cwd(),
prompt: r.prompt ?? r.query ?? r.input ?? r.message,
toolName: isShellCommand ? 'Bash' : r.tool_name,
toolInput: isShellCommand ? { command: r.command } : r.tool_input,
toolResponse: isShellCommand ? { output: r.output } : r.result_json, // result_json not tool_response
+2 -1
View File
@@ -8,7 +8,8 @@ export function getPlatformAdapter(platform: string): PlatformAdapter {
case 'claude-code': return claudeCodeAdapter;
case 'cursor': return cursorAdapter;
case 'raw': return rawAdapter;
default: throw new Error(`Unknown platform: ${platform}`);
// Codex CLI and other compatible platforms use the raw adapter (accepts both camelCase and snake_case fields)
default: return rawAdapter;
}
}
+5
View File
@@ -264,6 +264,11 @@ function formatObservationsForClaudeMd(observations: ObservationRow[], folderPat
* Only writes to folders that exist never creates directories.
*/
function writeClaudeMdToFolder(folderPath: string, newContent: string): void {
const resolvedPath = path.resolve(folderPath);
// Never write inside .git directories — corrupts refs (#1165)
if (resolvedPath.includes('/.git/') || resolvedPath.includes('\\.git\\') || resolvedPath.endsWith('/.git') || resolvedPath.endsWith('\\.git')) return;
const claudeMdPath = path.join(folderPath, 'CLAUDE.md');
const tempFile = `${claudeMdPath}.tmp`;
+14 -7
View File
@@ -6,10 +6,12 @@
*/
import type { EventHandler, NormalizedHookInput, HookResult } from '../types.js';
import { ensureWorkerRunning, getWorkerPort } from '../../shared/worker-utils.js';
import { ensureWorkerRunning, getWorkerPort, workerHttpRequest } from '../../shared/worker-utils.js';
import { getProjectContext } from '../../utils/project-name.js';
import { HOOK_EXIT_CODES } from '../../shared/hook-constants.js';
import { logger } from '../../utils/logger.js';
import { SettingsDefaultsManager } from '../../shared/SettingsDefaultsManager.js';
import { USER_SETTINGS_PATH } from '../../shared/paths.js';
export const contextHandler: EventHandler = {
async execute(input: NormalizedHookInput): Promise<HookResult> {
@@ -30,18 +32,22 @@ export const contextHandler: EventHandler = {
const context = getProjectContext(cwd);
const port = getWorkerPort();
// Check if terminal output should be shown (load settings early)
const settings = SettingsDefaultsManager.loadFromFile(USER_SETTINGS_PATH);
const showTerminalOutput = settings.CLAUDE_MEM_CONTEXT_SHOW_TERMINAL_OUTPUT === 'true';
// Pass all projects (parent + worktree if applicable) for unified timeline
const projectsParam = context.allProjects.join(',');
const url = `http://127.0.0.1:${port}/api/context/inject?projects=${encodeURIComponent(projectsParam)}`;
const apiPath = `/api/context/inject?projects=${encodeURIComponent(projectsParam)}`;
const colorApiPath = `${apiPath}&colors=true`;
// Note: Removed AbortSignal.timeout due to Windows Bun cleanup issue (libuv assertion)
// Worker service has its own timeouts, so client-side timeout is redundant
try {
// Fetch both markdown (for Claude context) and colored (for user display) truly in parallel
const colorUrl = `${url}&colors=true`;
// Fetch markdown (for Claude context) and optionally colored (for user display)
const [response, colorResponse] = await Promise.all([
fetch(url),
fetch(colorUrl).catch(() => null)
workerHttpRequest(apiPath),
showTerminalOutput ? workerHttpRequest(colorApiPath).catch(() => null) : Promise.resolve(null)
]);
if (!response.ok) {
@@ -60,7 +66,8 @@ export const contextHandler: EventHandler = {
const additionalContext = contextResult.trim();
const coloredTimeline = colorResult.trim();
const systemMessage = coloredTimeline
const systemMessage = showTerminalOutput && coloredTimeline
? `${coloredTimeline}\n\nView Observations Live @ http://localhost:${port}`
: undefined;
+2 -6
View File
@@ -6,7 +6,7 @@
*/
import type { EventHandler, NormalizedHookInput, HookResult } from '../types.js';
import { ensureWorkerRunning, getWorkerPort } from '../../shared/worker-utils.js';
import { ensureWorkerRunning, workerHttpRequest } from '../../shared/worker-utils.js';
import { logger } from '../../utils/logger.js';
import { HOOK_EXIT_CODES } from '../../shared/hook-constants.js';
@@ -25,10 +25,7 @@ export const fileEditHandler: EventHandler = {
throw new Error('fileEditHandler requires filePath');
}
const port = getWorkerPort();
logger.dataIn('HOOK', `FileEdit: ${filePath}`, {
workerPort: port,
editCount: edits?.length ?? 0
});
@@ -40,7 +37,7 @@ export const fileEditHandler: EventHandler = {
// Send to worker as an observation with file edit metadata
// The observation handler on the worker will process this appropriately
try {
const response = await fetch(`http://127.0.0.1:${port}/api/sessions/observations`, {
const response = await workerHttpRequest('/api/sessions/observations', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
@@ -50,7 +47,6 @@ export const fileEditHandler: EventHandler = {
tool_response: { success: true },
cwd
})
// Note: Removed signal to avoid Windows Bun cleanup issue (libuv assertion)
});
if (!response.ok) {
+2 -1
View File
@@ -6,6 +6,7 @@
import type { EventHandler } from '../types.js';
import { HOOK_EXIT_CODES } from '../../shared/hook-constants.js';
import { logger } from '../../utils/logger.js';
import { contextHandler } from './context.js';
import { sessionInitHandler } from './session-init.js';
import { observationHandler } from './observation.js';
@@ -46,7 +47,7 @@ const handlers: Record<EventType, EventHandler> = {
export function getEventHandler(eventType: string): EventHandler {
const handler = handlers[eventType as EventType];
if (!handler) {
console.error(`[claude-mem] Unknown event type: ${eventType}, returning no-op`);
logger.warn('HOOK', `Unknown event type: ${eventType}, returning no-op`);
return {
async execute() {
return { continue: true, suppressOutput: true, exitCode: HOOK_EXIT_CODES.SUCCESS };
+3 -8
View File
@@ -5,7 +5,7 @@
*/
import type { EventHandler, NormalizedHookInput, HookResult } from '../types.js';
import { ensureWorkerRunning, getWorkerPort } from '../../shared/worker-utils.js';
import { ensureWorkerRunning, workerHttpRequest } from '../../shared/worker-utils.js';
import { logger } from '../../utils/logger.js';
import { HOOK_EXIT_CODES } from '../../shared/hook-constants.js';
import { isProjectExcluded } from '../../utils/project-filter.js';
@@ -28,13 +28,9 @@ export const observationHandler: EventHandler = {
return { continue: true, suppressOutput: true, exitCode: HOOK_EXIT_CODES.SUCCESS };
}
const port = getWorkerPort();
const toolStr = logger.formatTool(toolName, toolInput);
logger.dataIn('HOOK', `PostToolUse: ${toolStr}`, {
workerPort: port
});
logger.dataIn('HOOK', `PostToolUse: ${toolStr}`, {});
// Validate required fields before sending to worker
if (!cwd) {
@@ -50,7 +46,7 @@ export const observationHandler: EventHandler = {
// Send to worker - worker handles privacy check and database operations
try {
const response = await fetch(`http://127.0.0.1:${port}/api/sessions/observations`, {
const response = await workerHttpRequest('/api/sessions/observations', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
@@ -60,7 +56,6 @@ export const observationHandler: EventHandler = {
tool_response: toolResponse,
cwd
})
// Note: Removed signal to avoid Windows Bun cleanup issue (libuv assertion)
});
if (!response.ok) {
+2 -4
View File
@@ -10,7 +10,7 @@
*/
import type { EventHandler, NormalizedHookInput, HookResult } from '../types.js';
import { ensureWorkerRunning, getWorkerPort } from '../../shared/worker-utils.js';
import { ensureWorkerRunning, workerHttpRequest } from '../../shared/worker-utils.js';
import { logger } from '../../utils/logger.js';
export const sessionCompleteHandler: EventHandler = {
@@ -23,7 +23,6 @@ export const sessionCompleteHandler: EventHandler = {
}
const { sessionId } = input;
const port = getWorkerPort();
if (!sessionId) {
logger.warn('HOOK', 'session-complete: Missing sessionId, skipping');
@@ -31,13 +30,12 @@ export const sessionCompleteHandler: EventHandler = {
}
logger.info('HOOK', '→ session-complete: Removing session from active map', {
workerPort: port,
contentSessionId: sessionId
});
try {
// Call the session complete endpoint by contentSessionId
const response = await fetch(`http://127.0.0.1:${port}/api/sessions/complete`, {
const response = await workerHttpRequest('/api/sessions/complete', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
+21 -7
View File
@@ -5,7 +5,7 @@
*/
import type { EventHandler, NormalizedHookInput, HookResult } from '../types.js';
import { ensureWorkerRunning, getWorkerPort } from '../../shared/worker-utils.js';
import { ensureWorkerRunning, workerHttpRequest } from '../../shared/worker-utils.js';
import { getProjectName } from '../../utils/project-name.js';
import { logger } from '../../utils/logger.js';
import { HOOK_EXIT_CODES } from '../../shared/hook-constants.js';
@@ -24,6 +24,12 @@ export const sessionInitHandler: EventHandler = {
const { sessionId, cwd, prompt: rawPrompt } = input;
// Guard: Codex CLI and other platforms may not provide a session_id (#744)
if (!sessionId) {
logger.warn('HOOK', 'session-init: No sessionId provided, skipping (Codex CLI or unknown platform)');
return { continue: true, suppressOutput: true, exitCode: HOOK_EXIT_CODES.SUCCESS };
}
// Check if project is excluded from tracking
const settings = SettingsDefaultsManager.loadFromFile(USER_SETTINGS_PATH);
if (cwd && isProjectExcluded(cwd, settings.CLAUDE_MEM_EXCLUDED_PROJECTS)) {
@@ -36,12 +42,11 @@ export const sessionInitHandler: EventHandler = {
const prompt = (!rawPrompt || !rawPrompt.trim()) ? '[media prompt]' : rawPrompt;
const project = getProjectName(cwd);
const port = getWorkerPort();
logger.debug('HOOK', 'session-init: Calling /api/sessions/init', { contentSessionId: sessionId, project });
// Initialize session via HTTP - handles DB operations and privacy checks
const initResponse = await fetch(`http://127.0.0.1:${port}/api/sessions/init`, {
const initResponse = await workerHttpRequest('/api/sessions/init', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
@@ -49,7 +54,6 @@ export const sessionInitHandler: EventHandler = {
project,
prompt
})
// Note: Removed signal to avoid Windows Bun cleanup issue (libuv assertion)
});
if (!initResponse.ok) {
@@ -63,11 +67,12 @@ export const sessionInitHandler: EventHandler = {
promptNumber: number;
skipped?: boolean;
reason?: string;
contextInjected?: boolean;
};
const sessionDbId = initResult.sessionDbId;
const promptNumber = initResult.promptNumber;
logger.debug('HOOK', 'session-init: Received from /api/sessions/init', { sessionDbId, promptNumber, skipped: initResult.skipped });
logger.debug('HOOK', 'session-init: Received from /api/sessions/init', { sessionDbId, promptNumber, skipped: initResult.skipped, contextInjected: initResult.contextInjected });
// Debug-level alignment log for detailed tracing
logger.debug('HOOK', `[ALIGNMENT] Hook Entry | contentSessionId=${sessionId} | prompt#=${promptNumber} | sessionDbId=${sessionDbId}`);
@@ -80,6 +85,16 @@ export const sessionInitHandler: EventHandler = {
return { continue: true, suppressOutput: true };
}
// Skip SDK agent re-initialization if context was already injected for this session (#1079)
// The prompt was already saved to the database by /api/sessions/init above —
// no need to re-start the SDK agent on every turn
if (initResult.contextInjected) {
logger.info('HOOK', `INIT_COMPLETE | sessionDbId=${sessionDbId} | promptNumber=${promptNumber} | skipped_agent_init=true | reason=context_already_injected`, {
sessionId: sessionDbId
});
return { continue: true, suppressOutput: true };
}
// Only initialize SDK agent for Claude Code (not Cursor)
// Cursor doesn't use the SDK agent - it only needs session/observation storage
if (input.platform !== 'cursor' && sessionDbId) {
@@ -90,11 +105,10 @@ export const sessionInitHandler: EventHandler = {
logger.debug('HOOK', 'session-init: Calling /sessions/{sessionDbId}/init', { sessionDbId, promptNumber });
// Initialize SDK agent session via HTTP (starts the agent!)
const response = await fetch(`http://127.0.0.1:${port}/sessions/${sessionDbId}/init`, {
const response = await workerHttpRequest(`/sessions/${sessionDbId}/init`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ userPrompt: cleanedPrompt, promptNumber })
// Note: Removed signal to avoid Windows Bun cleanup issue (libuv assertion)
});
if (!response.ok) {
+10 -16
View File
@@ -7,7 +7,7 @@
*/
import type { EventHandler, NormalizedHookInput, HookResult } from '../types.js';
import { ensureWorkerRunning, getWorkerPort, fetchWithTimeout } from '../../shared/worker-utils.js';
import { ensureWorkerRunning, workerHttpRequest } from '../../shared/worker-utils.js';
import { logger } from '../../utils/logger.js';
import { extractLastMessage } from '../../shared/transcript-parser.js';
import { HOOK_EXIT_CODES, HOOK_TIMEOUTS, getTimeout } from '../../shared/hook-constants.js';
@@ -25,8 +25,6 @@ export const summarizeHandler: EventHandler = {
const { sessionId, transcriptPath } = input;
const port = getWorkerPort();
// Validate required fields before processing
if (!transcriptPath) {
// No transcript available - skip summary gracefully (not an error)
@@ -40,23 +38,19 @@ export const summarizeHandler: EventHandler = {
const lastAssistantMessage = extractLastMessage(transcriptPath, 'assistant', true);
logger.dataIn('HOOK', 'Stop: Requesting summary', {
workerPort: port,
hasLastAssistantMessage: !!lastAssistantMessage
});
// Send to worker - worker handles privacy check and database operations
const response = await fetchWithTimeout(
`http://127.0.0.1:${port}/api/sessions/summarize`,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
contentSessionId: sessionId,
last_assistant_message: lastAssistantMessage
}),
},
SUMMARIZE_TIMEOUT_MS
);
const response = await workerHttpRequest('/api/sessions/summarize', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
contentSessionId: sessionId,
last_assistant_message: lastAssistantMessage
}),
timeoutMs: SUMMARIZE_TIMEOUT_MS
});
if (!response.ok) {
// Return standard response even on failure (matches original behavior)
+3 -5
View File
@@ -7,7 +7,7 @@
import { basename } from 'path';
import type { EventHandler, NormalizedHookInput, HookResult } from '../types.js';
import { ensureWorkerRunning, getWorkerPort } from '../../shared/worker-utils.js';
import { ensureWorkerRunning, getWorkerPort, workerHttpRequest } from '../../shared/worker-utils.js';
import { HOOK_EXIT_CODES } from '../../shared/hook-constants.js';
export const userMessageHandler: EventHandler = {
@@ -23,11 +23,9 @@ export const userMessageHandler: EventHandler = {
const project = basename(input.cwd ?? process.cwd());
// Fetch formatted context directly from worker API
// Note: Removed AbortSignal.timeout to avoid Windows Bun cleanup issue (libuv assertion)
try {
const response = await fetch(
`http://127.0.0.1:${port}/api/context/inject?project=${encodeURIComponent(project)}&colors=true`,
{ method: 'GET' }
const response = await workerHttpRequest(
`/api/context/inject?project=${encodeURIComponent(project)}&colors=true`
);
if (!response.ok) {
+14 -3
View File
@@ -2,6 +2,7 @@ import { readJsonFromStdin } from './stdin-reader.js';
import { getPlatformAdapter } from './adapters/index.js';
import { getEventHandler } from './handlers/index.js';
import { HOOK_EXIT_CODES } from '../shared/hook-constants.js';
import { logger } from '../utils/logger.js';
export interface HookCommandOptions {
/** If true, don't call process.exit() - let caller handle process lifecycle */
@@ -65,6 +66,12 @@ export function isWorkerUnavailableError(error: unknown): boolean {
}
export async function hookCommand(platform: string, event: string, options: HookCommandOptions = {}): Promise<number> {
// Suppress stderr in hook context — Claude Code shows stderr as error UI (#1181)
// Exit 1: stderr shown to user. Exit 2: stderr fed to Claude for processing.
// All diagnostics go to log file via logger; stderr must stay clean.
const originalStderrWrite = process.stderr.write.bind(process.stderr);
process.stderr.write = (() => true) as typeof process.stderr.write;
try {
const adapter = getPlatformAdapter(platform);
const handler = getEventHandler(event);
@@ -84,18 +91,22 @@ export async function hookCommand(platform: string, event: string, options: Hook
} catch (error) {
if (isWorkerUnavailableError(error)) {
// Worker unavailable — degrade gracefully, don't block the user
console.error(`[claude-mem] Worker unavailable, skipping hook: ${error instanceof Error ? error.message : error}`);
// Log to file instead of stderr (#1181)
logger.warn('HOOK', `Worker unavailable, skipping hook: ${error instanceof Error ? error.message : error}`);
if (!options.skipExit) {
process.exit(HOOK_EXIT_CODES.SUCCESS); // = 0 (graceful)
}
return HOOK_EXIT_CODES.SUCCESS;
}
// Handler/client bug — show as blocking error so developers see it
console.error(`Hook error: ${error}`);
// Handler/client bug — log to file instead of stderr (#1181)
logger.error('HOOK', `Hook error: ${error instanceof Error ? error.message : error}`, {}, error instanceof Error ? error : undefined);
if (!options.skipExit) {
process.exit(HOOK_EXIT_CODES.BLOCKING_ERROR); // = 2
}
return HOOK_EXIT_CODES.BLOCKING_ERROR;
} finally {
// Restore stderr for non-hook code paths (e.g., when skipExit is true and process continues as worker)
process.stderr.write = originalStderrWrite;
}
}
-19
View File
@@ -1,19 +0,0 @@
/**
* Observation metadata constants
* Shared across hooks, worker service, and UI components
*
* Note: These are fallback defaults for the code mode.
* Actual observation types and concepts are defined per-mode in the modes/ directory.
*/
/**
* Default observation types (comma-separated string for settings)
* Uses code mode defaults as fallback
*/
export const DEFAULT_OBSERVATION_TYPES_STRING = 'bugfix,feature,refactor,discovery,decision,change';
/**
* Default observation concepts (comma-separated string for settings)
* Uses code mode defaults as fallback
*/
export const DEFAULT_OBSERVATION_CONCEPTS_STRING = 'how-it-works,why-it-exists,what-changed,problem-solution,gotcha,pattern,trade-off';
+5
View File
@@ -120,6 +120,11 @@ export function parseSummary(text: string, sessionId?: number): ParsedSummary |
const summaryMatch = summaryRegex.exec(text);
if (!summaryMatch) {
// Log when the response contains <observation> instead of <summary>
// to help diagnose prompt conditioning issues (see #1312)
if (/<observation>/.test(text)) {
logger.warn('PARSER', 'Summary response contained <observation> tags instead of <summary> — prompt conditioning may need strengthening', { sessionId });
}
return null;
}
+5 -1
View File
@@ -130,7 +130,11 @@ export function buildSummaryPrompt(session: SDKSession, mode: ModeConfig): strin
return '';
})();
return `${mode.prompts.header_summary_checkpoint}
return `--- MODE SWITCH: PROGRESS SUMMARY ---
Do NOT output <observation> tags. This is a summary request, not an observation request.
Your response MUST use <summary> tags ONLY. Any <observation> output will be discarded.
${mode.prompts.header_summary_checkpoint}
${mode.prompts.summary_instruction}
${mode.prompts.summary_context_label}
+110 -29
View File
@@ -27,14 +27,11 @@ import {
CallToolRequestSchema,
ListToolsRequestSchema,
} from '@modelcontextprotocol/sdk/types.js';
import { getWorkerPort, getWorkerHost } from '../shared/worker-utils.js';
/**
* Worker HTTP API configuration
*/
const WORKER_PORT = getWorkerPort();
const WORKER_HOST = getWorkerHost();
const WORKER_BASE_URL = `http://${WORKER_HOST}:${WORKER_PORT}`;
import { workerHttpRequest } from '../shared/worker-utils.js';
import { searchCodebase, formatSearchResults } from '../services/smart-file-read/search.js';
import { parseFile, formatFoldedView, unfoldSymbol } from '../services/smart-file-read/parser.js';
import { readFile } from 'node:fs/promises';
import { resolve } from 'node:path';
/**
* Map tool names to Worker HTTP endpoints
@@ -45,7 +42,7 @@ const TOOL_ENDPOINT_MAP: Record<string, string> = {
};
/**
* Call Worker HTTP API endpoint
* Call Worker HTTP API endpoint (uses socket or TCP automatically)
*/
async function callWorkerAPI(
endpoint: string,
@@ -63,8 +60,8 @@ async function callWorkerAPI(
}
}
const url = `${WORKER_BASE_URL}${endpoint}?${searchParams}`;
const response = await fetch(url);
const apiPath = `${endpoint}?${searchParams}`;
const response = await workerHttpRequest(apiPath);
if (!response.ok) {
const errorText = await response.text();
@@ -99,12 +96,9 @@ async function callWorkerAPIPost(
logger.debug('HTTP', 'Worker API request (POST)', undefined, { endpoint });
try {
const url = `${WORKER_BASE_URL}${endpoint}`;
const response = await fetch(url, {
const response = await workerHttpRequest(endpoint, {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body)
});
@@ -141,7 +135,7 @@ async function callWorkerAPIPost(
*/
async function verifyWorkerConnection(): Promise<boolean> {
try {
const response = await fetch(`${WORKER_BASE_URL}/api/health`);
const response = await workerHttpRequest('/api/health');
return response.ok;
} catch (error) {
// Expected during worker startup or if worker is down
@@ -235,28 +229,115 @@ NEVER fetch full details without filtering first. 10x token savings.`,
}
},
{
name: 'save_observation',
description: 'Save an observation to the database. Params: text (required), title, project',
name: 'smart_search',
description: 'Search codebase for symbols, functions, classes using tree-sitter AST parsing. Returns folded structural views with token counts. Use path parameter to scope the search.',
inputSchema: {
type: 'object',
properties: {
text: {
query: {
type: 'string',
description: 'Content to remember (required)'
description: 'Search term — matches against symbol names, file names, and file content'
},
title: {
path: {
type: 'string',
description: 'Short title (auto-generated from text if omitted)'
description: 'Root directory to search (default: current working directory)'
},
project: {
max_results: {
type: 'number',
description: 'Maximum results to return (default: 20)'
},
file_pattern: {
type: 'string',
description: 'Project name (uses "claude-mem" if omitted)'
description: 'Substring filter for file paths (e.g. ".ts", "src/services")'
}
},
required: ['text']
required: ['query']
},
handler: async (args: any) => {
return await callWorkerAPIPost('/api/memory/save', args);
const rootDir = resolve(args.path || process.cwd());
const result = await searchCodebase(rootDir, args.query, {
maxResults: args.max_results || 20,
filePattern: args.file_pattern
});
const formatted = formatSearchResults(result, args.query);
return {
content: [{ type: 'text' as const, text: formatted }]
};
}
},
{
name: 'smart_unfold',
description: 'Expand a specific symbol (function, class, method) from a file. Returns the full source code of just that symbol. Use after smart_search or smart_outline to read specific code.',
inputSchema: {
type: 'object',
properties: {
file_path: {
type: 'string',
description: 'Path to the source file'
},
symbol_name: {
type: 'string',
description: 'Name of the symbol to unfold (function, class, method, etc.)'
}
},
required: ['file_path', 'symbol_name']
},
handler: async (args: any) => {
const filePath = resolve(args.file_path);
const content = await readFile(filePath, 'utf-8');
const unfolded = unfoldSymbol(content, filePath, args.symbol_name);
if (unfolded) {
return {
content: [{ type: 'text' as const, text: unfolded }]
};
}
// Symbol not found — show available symbols
const parsed = parseFile(content, filePath);
if (parsed.symbols.length > 0) {
const available = parsed.symbols.map(s => ` - ${s.name} (${s.kind})`).join('\n');
return {
content: [{
type: 'text' as const,
text: `Symbol "${args.symbol_name}" not found in ${args.file_path}.\n\nAvailable symbols:\n${available}`
}]
};
}
return {
content: [{
type: 'text' as const,
text: `Could not parse ${args.file_path}. File may be unsupported or empty.`
}]
};
}
},
{
name: 'smart_outline',
description: 'Get structural outline of a file — shows all symbols (functions, classes, methods, types) with signatures but bodies folded. Much cheaper than reading the full file.',
inputSchema: {
type: 'object',
properties: {
file_path: {
type: 'string',
description: 'Path to the source file'
}
},
required: ['file_path']
},
handler: async (args: any) => {
const filePath = resolve(args.file_path);
const content = await readFile(filePath, 'utf-8');
const parsed = parseFile(content, filePath);
if (parsed.symbols.length > 0) {
return {
content: [{ type: 'text' as const, text: formatFoldedView(parsed) }]
};
}
return {
content: [{
type: 'text' as const,
text: `Could not parse ${args.file_path}. File may use an unsupported language or be empty.`
}]
};
}
}
];
@@ -357,11 +438,11 @@ async function main() {
setTimeout(async () => {
const workerAvailable = await verifyWorkerConnection();
if (!workerAvailable) {
logger.error('SYSTEM', 'Worker not available', undefined, { workerUrl: WORKER_BASE_URL });
logger.error('SYSTEM', 'Worker not available', undefined, {});
logger.error('SYSTEM', 'Tools will fail until Worker is started');
logger.error('SYSTEM', 'Start Worker with: npm run worker:restart');
} else {
logger.info('SYSTEM', 'Worker available', undefined, { workerUrl: WORKER_BASE_URL });
logger.info('SYSTEM', 'Worker available', undefined, {});
}
}, 0);
}
+4 -21
View File
@@ -18,27 +18,10 @@ export function loadContextConfig(): ContextConfig {
const settingsPath = path.join(homedir(), '.claude-mem', 'settings.json');
const settings = SettingsDefaultsManager.loadFromFile(settingsPath);
// For non-code modes, use all types/concepts from active mode instead of settings
const modeId = settings.CLAUDE_MEM_MODE;
const isCodeMode = modeId === 'code' || modeId.startsWith('code--');
let observationTypes: Set<string>;
let observationConcepts: Set<string>;
if (isCodeMode) {
// Code mode: use settings-based filtering
observationTypes = new Set(
settings.CLAUDE_MEM_CONTEXT_OBSERVATION_TYPES.split(',').map((t: string) => t.trim()).filter(Boolean)
);
observationConcepts = new Set(
settings.CLAUDE_MEM_CONTEXT_OBSERVATION_CONCEPTS.split(',').map((c: string) => c.trim()).filter(Boolean)
);
} else {
// Non-code modes: use all types/concepts from active mode
const mode = ModeManager.getInstance().getActiveMode();
observationTypes = new Set(mode.observation_types.map(t => t.id));
observationConcepts = new Set(mode.observation_concepts.map(c => c.id));
}
// Always read types/concepts from the active mode definition
const mode = ModeManager.getInstance().getActiveMode();
const observationTypes = new Set(mode.observation_types.map(t => t.id));
const observationConcepts = new Set(mode.observation_concepts.map(c => c.id));
return {
totalObservationCount: parseInt(settings.CLAUDE_MEM_CONTEXT_OBSERVATIONS, 10),
@@ -226,7 +226,7 @@ export function renderColorFooter(totalDiscoveryTokens: number, totalReadTokens:
const workTokensK = Math.round(totalDiscoveryTokens / 1000);
return [
'',
`${colors.dim}Access ${workTokensK}k tokens of past research & decisions for just ${totalReadTokens.toLocaleString()}t. Use MCP search tools to access memories by ID.${colors.reset}`
`${colors.dim}Access ${workTokensK}k tokens of past research & decisions for just ${totalReadTokens.toLocaleString()}t. Use the claude-mem skill to access memories by ID.${colors.reset}`
];
}
@@ -229,7 +229,7 @@ export function renderMarkdownFooter(totalDiscoveryTokens: number, totalReadToke
const workTokensK = Math.round(totalDiscoveryTokens / 1000);
return [
'',
`Access ${workTokensK}k tokens of past research & decisions for just ${totalReadTokens.toLocaleString()}t. Use MCP search tools to access memories by ID.`
`Access ${workTokensK}k tokens of past research & decisions for just ${totalReadTokens.toLocaleString()}t. Use the claude-mem skill to access memories by ID.`
];
}
@@ -10,12 +10,7 @@
import http from 'http';
import { logger } from '../../utils/logger.js';
import {
getChildProcesses,
forceKillProcess,
waitForProcessesExit,
removePidFile
} from './ProcessManager.js';
import { stopSupervisor } from '../../supervisor/index.js';
export interface ShutdownableService {
shutdownAll(): Promise<void>;
@@ -57,49 +52,35 @@ export interface GracefulShutdownConfig {
export async function performGracefulShutdown(config: GracefulShutdownConfig): Promise<void> {
logger.info('SYSTEM', 'Shutdown initiated');
// Clean up PID file on shutdown
removePidFile();
// STEP 1: Enumerate all child processes BEFORE we start closing things
const childPids = await getChildProcesses(process.pid);
logger.info('SYSTEM', 'Found child processes', { count: childPids.length, pids: childPids });
// STEP 2: Close HTTP server first
// STEP 1: Close HTTP server first
if (config.server) {
await closeHttpServer(config.server);
logger.info('SYSTEM', 'HTTP server closed');
}
// STEP 3: Shutdown active sessions
// STEP 2: Shutdown active sessions
await config.sessionManager.shutdownAll();
// STEP 4: Close MCP client connection (signals child to exit gracefully)
// STEP 3: Close MCP client connection (signals child to exit gracefully)
if (config.mcpClient) {
await config.mcpClient.close();
logger.info('SYSTEM', 'MCP client closed');
}
// STEP 5: Stop Chroma MCP connection
// STEP 4: Stop Chroma MCP connection
if (config.chromaMcpManager) {
logger.info('SHUTDOWN', 'Stopping Chroma MCP connection...');
await config.chromaMcpManager.stop();
logger.info('SHUTDOWN', 'Chroma MCP connection stopped');
}
// STEP 6: Close database connection (includes ChromaSync cleanup)
// STEP 5: Close database connection (includes ChromaSync cleanup)
if (config.dbManager) {
await config.dbManager.close();
}
// STEP 7: Force kill any remaining child processes (Windows zombie port fix)
if (childPids.length > 0) {
logger.info('SYSTEM', 'Force killing remaining children');
for (const pid of childPids) {
await forceKillProcess(pid);
}
// Wait for children to fully exit
await waitForProcessesExit(childPids, 5000);
}
// STEP 6: Supervisor handles tracked child termination, PID cleanup, and stale sockets.
await stopSupervisor();
logger.info('SYSTEM', 'Worker shutdown complete');
}
+50 -25
View File
@@ -14,6 +14,26 @@ import { readFileSync } from 'fs';
import { logger } from '../../utils/logger.js';
import { MARKETPLACE_ROOT } from '../../shared/paths.js';
/**
* Make an HTTP request to the worker via TCP.
* Returns { ok, statusCode, body } or throws on transport error.
*/
async function httpRequestToWorker(
port: number,
endpointPath: string,
method: string = 'GET'
): Promise<{ ok: boolean; statusCode: number; body: string }> {
const response = await fetch(`http://127.0.0.1:${port}${endpointPath}`, { method });
// Gracefully handle cases where response body isn't available (e.g., test mocks)
let body = '';
try {
body = await response.text();
} catch {
// Body unavailable — health/readiness checks only need .ok
}
return { ok: response.ok, statusCode: response.status, body };
}
/**
* Check if a port is in use by querying the health endpoint
*/
@@ -29,7 +49,7 @@ export async function isPortInUse(port: number): Promise<boolean> {
}
/**
* Poll a localhost endpoint until it returns 200 OK or timeout.
* Poll a worker endpoint until it returns 200 OK or timeout.
* Shared implementation for liveness and readiness checks.
*/
async function pollEndpointUntilOk(
@@ -41,12 +61,11 @@ async function pollEndpointUntilOk(
const start = Date.now();
while (Date.now() - start < timeoutMs) {
try {
// Note: Removed AbortSignal.timeout to avoid Windows Bun cleanup issue (libuv assertion)
const response = await fetch(`http://127.0.0.1:${port}${endpointPath}`);
if (response.ok) return true;
const result = await httpRequestToWorker(port, endpointPath);
if (result.ok) return true;
} catch (error) {
// [ANTI-PATTERN IGNORED]: Retry loop - expected failures during startup, will retry
logger.debug('SYSTEM', retryLogMessage, { port }, error as Error);
logger.debug('SYSTEM', retryLogMessage, {}, error as Error);
}
await new Promise(r => setTimeout(r, 500));
}
@@ -87,55 +106,61 @@ export async function waitForPortFree(port: number, timeoutMs: number = 10000):
/**
* Send HTTP shutdown request to a running worker
* @param port Worker port
* @returns true if shutdown request was acknowledged, false otherwise
*/
export async function httpShutdown(port: number): Promise<boolean> {
try {
// Note: Removed AbortSignal.timeout to avoid Windows Bun cleanup issue (libuv assertion)
const response = await fetch(`http://127.0.0.1:${port}/api/admin/shutdown`, {
method: 'POST'
});
if (!response.ok) {
logger.warn('SYSTEM', 'Shutdown request returned error', { port, status: response.status });
const result = await httpRequestToWorker(port, '/api/admin/shutdown', 'POST');
if (!result.ok) {
logger.warn('SYSTEM', 'Shutdown request returned error', { status: result.statusCode });
return false;
}
return true;
} catch (error) {
// Connection refused is expected if worker already stopped
if (error instanceof Error && error.message?.includes('ECONNREFUSED')) {
logger.debug('SYSTEM', 'Worker already stopped', { port }, error);
logger.debug('SYSTEM', 'Worker already stopped', {}, error);
return false;
}
// Unexpected error - log full details
logger.error('SYSTEM', 'Shutdown request failed unexpectedly', { port }, error as Error);
logger.error('SYSTEM', 'Shutdown request failed unexpectedly', {}, error as Error);
return false;
}
}
/**
* Get the plugin version from the installed marketplace package.json
* This is the "expected" version that should be running
* This is the "expected" version that should be running.
* Returns 'unknown' on ENOENT/EBUSY (shutdown race condition, fix #1042).
*/
export function getInstalledPluginVersion(): string {
const packageJsonPath = path.join(MARKETPLACE_ROOT, 'package.json');
const packageJson = JSON.parse(readFileSync(packageJsonPath, 'utf-8'));
return packageJson.version;
try {
const packageJsonPath = path.join(MARKETPLACE_ROOT, 'package.json');
const packageJson = JSON.parse(readFileSync(packageJsonPath, 'utf-8'));
return packageJson.version;
} catch (error: unknown) {
const code = (error as NodeJS.ErrnoException).code;
if (code === 'ENOENT' || code === 'EBUSY') {
logger.debug('SYSTEM', 'Could not read plugin version (shutdown race)', { code });
return 'unknown';
}
throw error;
}
}
/**
* Get the running worker's version via API
* This is the "actual" version currently running
* This is the "actual" version currently running.
*/
export async function getRunningWorkerVersion(port: number): Promise<string | null> {
try {
const response = await fetch(`http://127.0.0.1:${port}/api/version`);
if (!response.ok) return null;
const data = await response.json() as { version: string };
const result = await httpRequestToWorker(port, '/api/version');
if (!result.ok) return null;
const data = JSON.parse(result.body) as { version: string };
return data.version;
} catch {
// Expected: worker not running or version endpoint unavailable
logger.debug('SYSTEM', 'Could not fetch worker version', { port });
logger.debug('SYSTEM', 'Could not fetch worker version', {});
return null;
}
}
@@ -155,8 +180,8 @@ export async function checkVersionMatch(port: number): Promise<VersionCheckResul
const pluginVersion = getInstalledPluginVersion();
const workerVersion = await getRunningWorkerVersion(port);
// If we can't get worker version, assume it matches (graceful degradation)
if (!workerVersion) {
// If either version is unknown/null, assume match (graceful degradation, fix #1042)
if (!workerVersion || pluginVersion === 'unknown') {
return { matches: true, pluginVersion, workerVersion };
}
+67 -36
View File
@@ -10,11 +10,13 @@
import path from 'path';
import { homedir } from 'os';
import { existsSync, writeFileSync, readFileSync, unlinkSync, mkdirSync, rmSync } from 'fs';
import { existsSync, writeFileSync, readFileSync, unlinkSync, mkdirSync, rmSync, statSync, utimesSync } from 'fs';
import { exec, execSync, spawn } from 'child_process';
import { promisify } from 'util';
import { logger } from '../../utils/logger.js';
import { HOOK_TIMEOUTS } from '../../shared/hook-constants.js';
import { sanitizeEnv } from '../../supervisor/env-sanitizer.js';
import { getSupervisor, validateWorkerPidFile, type ValidateWorkerPidStatus } from '../../supervisor/index.js';
const execAsync = promisify(exec);
@@ -54,7 +56,8 @@ function lookupBinaryInPath(binaryName: string, platform: NodeJS.Platform): stri
try {
const output = execSync(command, {
stdio: ['ignore', 'pipe', 'ignore'],
encoding: 'utf-8'
encoding: 'utf-8',
windowsHide: true
});
const firstMatch = output
@@ -191,10 +194,10 @@ export async function getChildProcesses(parentPid: number): Promise<number[]> {
}
try {
// PowerShell Get-Process instead of WMIC (deprecated in Windows 11)
const cmd = `powershell -NoProfile -NonInteractive -Command "Get-Process | Where-Object { $_.ParentProcessId -eq ${parentPid} } | Select-Object -ExpandProperty Id"`;
const { stdout } = await execAsync(cmd, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND });
// PowerShell outputs just numbers (one per line), simpler than WMIC's "ProcessId=1234" format
// Use WQL -Filter to avoid $_ pipeline syntax that breaks in Git Bash (#1062, #1024).
// Get-CimInstance with server-side filtering is also more efficient than piping through Where-Object.
const cmd = `powershell -NoProfile -NonInteractive -Command "Get-CimInstance Win32_Process -Filter 'ParentProcessId=${parentPid}' | Select-Object -ExpandProperty ProcessId"`;
const { stdout } = await execAsync(cmd, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, windowsHide: true });
return stdout
.split('\n')
.map(line => line.trim())
@@ -223,7 +226,7 @@ export async function forceKillProcess(pid: number): Promise<void> {
try {
if (process.platform === 'win32') {
// /T kills entire process tree, /F forces termination
await execAsync(`taskkill /PID ${pid} /T /F`, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND });
await execAsync(`taskkill /PID ${pid} /T /F`, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, windowsHide: true });
} else {
process.kill(pid, 'SIGKILL');
}
@@ -315,13 +318,14 @@ export async function cleanupOrphanedProcesses(): Promise<void> {
try {
if (isWindows) {
// Windows: Use PowerShell Get-CimInstance with JSON output for age filtering
const patternConditions = ORPHAN_PROCESS_PATTERNS
.map(p => `$_.CommandLine -like '*${p}*'`)
.join(' -or ');
// Windows: Use WQL -Filter for server-side filtering (no $_ pipeline syntax).
// Avoids Git Bash $_ interpretation (#1062) and PowerShell syntax errors (#1024).
const wqlPatternConditions = ORPHAN_PROCESS_PATTERNS
.map(p => `CommandLine LIKE '%${p}%'`)
.join(' OR ');
const cmd = `powershell -NoProfile -NonInteractive -Command "Get-CimInstance Win32_Process | Where-Object { (${patternConditions}) -and $_.ProcessId -ne ${currentPid} } | Select-Object ProcessId, CreationDate | ConvertTo-Json"`;
const { stdout } = await execAsync(cmd, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND });
const cmd = `powershell -NoProfile -NonInteractive -Command "Get-CimInstance Win32_Process -Filter '(${wqlPatternConditions}) AND ProcessId != ${currentPid}' | Select-Object ProcessId, CreationDate | ConvertTo-Json"`;
const { stdout } = await execAsync(cmd, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, windowsHide: true });
if (!stdout.trim() || stdout.trim() === 'null') {
logger.debug('SYSTEM', 'No orphaned claude-mem processes found (Windows)');
@@ -406,7 +410,7 @@ export async function cleanupOrphanedProcesses(): Promise<void> {
continue;
}
try {
execSync(`taskkill /PID ${pid} /T /F`, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, stdio: 'ignore' });
execSync(`taskkill /PID ${pid} /T /F`, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, stdio: 'ignore', windowsHide: true });
} catch (error) {
// [ANTI-PATTERN IGNORED]: Cleanup loop - process may have exited, continue to next PID
logger.debug('SYSTEM', 'Failed to kill process, may have already exited', { pid }, error as Error);
@@ -451,12 +455,14 @@ export async function aggressiveStartupCleanup(): Promise<void> {
try {
if (isWindows) {
const patternConditions = allPatterns
.map(p => `$_.CommandLine -like '*${p}*'`)
.join(' -or ');
// Use WQL -Filter for server-side filtering (no $_ pipeline syntax).
// Avoids Git Bash $_ interpretation (#1062) and PowerShell syntax errors (#1024).
const wqlPatternConditions = allPatterns
.map(p => `CommandLine LIKE '%${p}%'`)
.join(' OR ');
const cmd = `powershell -NoProfile -NonInteractive -Command "Get-CimInstance Win32_Process | Where-Object { (${patternConditions}) -and $_.ProcessId -ne ${currentPid} } | Select-Object ProcessId, CommandLine, CreationDate | ConvertTo-Json"`;
const { stdout } = await execAsync(cmd, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND });
const cmd = `powershell -NoProfile -NonInteractive -Command "Get-CimInstance Win32_Process -Filter '(${wqlPatternConditions}) AND ProcessId != ${currentPid}' | Select-Object ProcessId, CommandLine, CreationDate | ConvertTo-Json"`;
const { stdout } = await execAsync(cmd, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, windowsHide: true });
if (!stdout.trim() || stdout.trim() === 'null') {
logger.debug('SYSTEM', 'No orphaned claude-mem processes found (Windows)');
@@ -549,7 +555,7 @@ export async function aggressiveStartupCleanup(): Promise<void> {
for (const pid of pidsToKill) {
if (!Number.isInteger(pid) || pid <= 0) continue;
try {
execSync(`taskkill /PID ${pid} /T /F`, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, stdio: 'ignore' });
execSync(`taskkill /PID ${pid} /T /F`, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, stdio: 'ignore', windowsHide: true });
} catch (error) {
logger.debug('SYSTEM', 'Failed to kill process, may have already exited', { pid }, error as Error);
}
@@ -621,11 +627,13 @@ export function spawnDaemon(
extraEnv: Record<string, string> = {}
): number | undefined {
const isWindows = process.platform === 'win32';
const env = {
getSupervisor().assertCanSpawn('worker daemon');
const env = sanitizeEnv({
...process.env,
CLAUDE_MEM_WORKER_PORT: String(port),
...extraEnv
};
});
if (isWindows) {
// Use PowerShell Start-Process to spawn a hidden, independent process
@@ -699,10 +707,10 @@ export function spawnDaemon(
*
* EPERM is treated as "alive" because it means the process exists but
* belongs to a different user/session (common in multi-user setups).
* PID 0 (Windows WMIC sentinel for unknown PID) is treated as alive.
* PID 0 (Windows sentinel for unknown PID) is treated as alive.
*/
export function isProcessAlive(pid: number): boolean {
// PID 0 is the Windows WMIC sentinel value — process was spawned but PID unknown
// PID 0 is the Windows sentinel value — process was spawned but PID unknown
if (pid === 0) return true;
// Invalid PIDs are not alive
@@ -720,6 +728,39 @@ export function isProcessAlive(pid: number): boolean {
}
}
/**
* Check if the PID file was written recently (within thresholdMs).
*
* Used to coordinate restarts across concurrent sessions: if the PID file
* was recently written, another session likely just restarted the worker.
* Callers should poll /api/health instead of attempting their own restart.
*
* @param thresholdMs - Maximum age in ms to consider "recent" (default: 15000)
* @returns true if the PID file exists and was modified within thresholdMs
*/
export function isPidFileRecent(thresholdMs: number = 15000): boolean {
try {
const stats = statSync(PID_FILE);
return (Date.now() - stats.mtimeMs) < thresholdMs;
} catch {
return false;
}
}
/**
* Touch the PID file to update its mtime without changing contents.
* Used after a restart to signal other sessions that a restart just completed.
*/
export function touchPidFile(): void {
try {
if (!existsSync(PID_FILE)) return;
const now = new Date();
utimesSync(PID_FILE, now, now);
} catch {
// Best-effort — failure to touch doesn't affect correctness
}
}
/**
* Read the PID file and remove it if the recorded process is dead (stale).
*
@@ -727,18 +768,8 @@ export function isProcessAlive(pid: number): boolean {
* Called at the top of ensureWorkerStarted() to clean up after WSL2
* hibernate, OOM kills, or other ungraceful worker deaths.
*/
export function cleanStalePidFile(): void {
const pidInfo = readPidFile();
if (!pidInfo) return;
if (!isProcessAlive(pidInfo.pid)) {
logger.info('SYSTEM', 'Removing stale PID file (worker process is dead)', {
pid: pidInfo.pid,
port: pidInfo.port,
startedAt: pidInfo.startedAt
});
removePidFile();
}
export function cleanStalePidFile(): ValidateWorkerPidStatus {
return validateWorkerPidFile({ logAlive: false });
}
/**
@@ -15,7 +15,7 @@ import { existsSync, readFileSync, writeFileSync, unlinkSync, mkdirSync } from '
import { exec } from 'child_process';
import { promisify } from 'util';
import { logger } from '../../utils/logger.js';
import { getWorkerPort } from '../../shared/worker-utils.js';
import { getWorkerPort, workerHttpRequest } from '../../shared/worker-utils.js';
import { DATA_DIR, MARKETPLACE_ROOT, CLAUDE_CONFIG_DIR } from '../../shared/paths.js';
import {
readCursorRegistry as readCursorRegistryFromFile,
@@ -95,16 +95,16 @@ export function unregisterCursorProject(projectName: string): void {
* Update Cursor context files for all registered projects matching this project name.
* Called by SDK agents after saving a summary.
*/
export async function updateCursorContextForProject(projectName: string, port: number): Promise<void> {
export async function updateCursorContextForProject(projectName: string, _port: number): Promise<void> {
const registry = readCursorRegistry();
const entry = registry[projectName];
if (!entry) return; // Project doesn't have Cursor hooks installed
try {
// Fetch fresh context from worker
const response = await fetch(
`http://127.0.0.1:${port}/api/context/inject?project=${encodeURIComponent(projectName)}`
// Fetch fresh context from worker (uses socket or TCP automatically)
const response = await workerHttpRequest(
`/api/context/inject?project=${encodeURIComponent(projectName)}`
);
if (!response.ok) return;
@@ -398,19 +398,18 @@ async function setupProjectContext(targetDir: string, workspaceRoot: string): Pr
const rulesDir = path.join(targetDir, 'rules');
mkdirSync(rulesDir, { recursive: true });
const port = getWorkerPort();
const projectName = path.basename(workspaceRoot);
let contextGenerated = false;
console.log(` Generating initial context...`);
try {
// Check if worker is running
const healthResponse = await fetch(`http://127.0.0.1:${port}/api/readiness`);
// Check if worker is running (uses socket or TCP automatically)
const healthResponse = await workerHttpRequest('/api/readiness');
if (healthResponse.ok) {
// Fetch context
const contextResponse = await fetch(
`http://127.0.0.1:${port}/api/context/inject?project=${encodeURIComponent(projectName)}`
const contextResponse = await workerHttpRequest(
`/api/context/inject?project=${encodeURIComponent(projectName)}`
);
if (contextResponse.ok) {
const context = await contextResponse.text();
+47
View File
@@ -17,6 +17,9 @@ import { ALLOWED_OPERATIONS, ALLOWED_TOPICS } from './allowed-constants.js';
import { logger } from '../../utils/logger.js';
import { createMiddleware, summarizeRequestBody, requireLocalhost } from './Middleware.js';
import { errorHandler, notFoundHandler } from './ErrorHandler.js';
import { getSupervisor } from '../../supervisor/index.js';
import { isPidAlive } from '../../supervisor/process-registry.js';
import { ENV_PREFIXES, ENV_EXACT_MATCHES } from '../../supervisor/env-sanitizer.js';
// Build-time injected version constant (set by esbuild define)
declare const __DEFAULT_PACKAGE_VERSION__: string;
@@ -285,6 +288,50 @@ export class Server {
}, 100);
}
});
// Doctor endpoint - diagnostic view of supervisor, processes, and health
this.app.get('/api/admin/doctor', requireLocalhost, (_req: Request, res: Response) => {
const supervisor = getSupervisor();
const registry = supervisor.getRegistry();
const allRecords = registry.getAll();
// Check each process liveness
const processes = allRecords.map(record => ({
id: record.id,
pid: record.pid,
type: record.type,
status: isPidAlive(record.pid) ? 'alive' as const : 'dead' as const,
startedAt: record.startedAt,
}));
// Check for dead processes still in registry
const deadProcessPids = processes.filter(p => p.status === 'dead').map(p => p.pid);
// Check if CLAUDECODE_* env vars are leaking into this process
const envClean = !Object.keys(process.env).some(key =>
ENV_EXACT_MATCHES.has(key) || ENV_PREFIXES.some(prefix => key.startsWith(prefix))
);
// Format uptime
const uptimeMs = Date.now() - this.startTime;
const uptimeSeconds = Math.floor(uptimeMs / 1000);
const hours = Math.floor(uptimeSeconds / 3600);
const minutes = Math.floor((uptimeSeconds % 3600) / 60);
const formattedUptime = hours > 0 ? `${hours}h ${minutes}m` : `${minutes}m`;
res.json({
supervisor: {
running: true,
pid: process.pid,
uptime: formattedUptime,
},
processes,
health: {
deadProcessPids,
envClean,
},
});
});
}
/**
+666
View File
@@ -0,0 +1,666 @@
/**
* Code structure parser shells out to tree-sitter CLI for AST-based extraction.
*
* No native bindings. No WASM. Just the CLI binary + query patterns.
*
* Supported: JS, TS, Python, Go, Rust, Ruby, Java, C, C++
*
* by Copter Labs
*/
import { execFileSync } from "node:child_process";
import { writeFileSync, mkdtempSync, rmSync, existsSync } from "node:fs";
import { join, dirname } from "node:path";
import { tmpdir } from "node:os";
import { createRequire } from "node:module";
// CJS-safe require for resolving external packages at runtime.
// In ESM: import.meta.url works. In CJS bundle (esbuild): __filename works.
// typeof check avoids ReferenceError in ESM where __filename doesn't exist.
const _require = typeof __filename !== 'undefined'
? createRequire(__filename)
: createRequire(import.meta.url);
// --- Types ---
export interface CodeSymbol {
name: string;
kind: "function" | "class" | "method" | "interface" | "type" | "const" | "variable" | "export" | "struct" | "enum" | "trait" | "impl" | "property" | "getter" | "setter";
signature: string;
jsdoc?: string;
lineStart: number;
lineEnd: number;
parent?: string;
exported: boolean;
children?: CodeSymbol[];
}
export interface FoldedFile {
filePath: string;
language: string;
symbols: CodeSymbol[];
imports: string[];
totalLines: number;
foldedTokenEstimate: number;
}
// --- Language detection ---
const LANG_MAP: Record<string, string> = {
".js": "javascript",
".mjs": "javascript",
".cjs": "javascript",
".jsx": "tsx",
".ts": "typescript",
".tsx": "tsx",
".py": "python",
".pyw": "python",
".go": "go",
".rs": "rust",
".rb": "ruby",
".java": "java",
".c": "c",
".h": "c",
".cpp": "cpp",
".cc": "cpp",
".cxx": "cpp",
".hpp": "cpp",
".hh": "cpp",
};
export function detectLanguage(filePath: string): string {
const ext = filePath.slice(filePath.lastIndexOf("."));
return LANG_MAP[ext] || "unknown";
}
// --- Grammar path resolution ---
const GRAMMAR_PACKAGES: Record<string, string> = {
javascript: "tree-sitter-javascript",
typescript: "tree-sitter-typescript/typescript",
tsx: "tree-sitter-typescript/tsx",
python: "tree-sitter-python",
go: "tree-sitter-go",
rust: "tree-sitter-rust",
ruby: "tree-sitter-ruby",
java: "tree-sitter-java",
c: "tree-sitter-c",
cpp: "tree-sitter-cpp",
};
function resolveGrammarPath(language: string): string | null {
const pkg = GRAMMAR_PACKAGES[language];
if (!pkg) return null;
try {
const packageJsonPath = _require.resolve(pkg + "/package.json");
return dirname(packageJsonPath);
} catch {
return null;
}
}
// --- Query patterns (declarative symbol extraction) ---
const QUERIES: Record<string, string> = {
jsts: `
(function_declaration name: (identifier) @name) @func
(lexical_declaration (variable_declarator name: (identifier) @name value: [(arrow_function) (function_expression)])) @const_func
(class_declaration name: (type_identifier) @name) @cls
(method_definition name: (property_identifier) @name) @method
(interface_declaration name: (type_identifier) @name) @iface
(type_alias_declaration name: (type_identifier) @name) @tdef
(enum_declaration name: (identifier) @name) @enm
(import_statement) @imp
(export_statement) @exp
`,
python: `
(function_definition name: (identifier) @name) @func
(class_definition name: (identifier) @name) @cls
(import_statement) @imp
(import_from_statement) @imp
`,
go: `
(function_declaration name: (identifier) @name) @func
(method_declaration name: (field_identifier) @name) @method
(type_declaration (type_spec name: (type_identifier) @name)) @tdef
(import_declaration) @imp
`,
rust: `
(function_item name: (identifier) @name) @func
(struct_item name: (type_identifier) @name) @struct_def
(enum_item name: (type_identifier) @name) @enm
(trait_item name: (type_identifier) @name) @trait_def
(impl_item type: (type_identifier) @name) @impl_def
(use_declaration) @imp
`,
ruby: `
(method name: (identifier) @name) @func
(class name: (constant) @name) @cls
(module name: (constant) @name) @cls
(call method: (identifier) @name) @imp
`,
java: `
(method_declaration name: (identifier) @name) @method
(class_declaration name: (identifier) @name) @cls
(interface_declaration name: (identifier) @name) @iface
(enum_declaration name: (identifier) @name) @enm
(import_declaration) @imp
`,
generic: `
(function_declaration name: (identifier) @name) @func
(function_definition name: (identifier) @name) @func
(class_declaration name: (identifier) @name) @cls
(class_definition name: (identifier) @name) @cls
(import_statement) @imp
(import_declaration) @imp
`,
};
function getQueryKey(language: string): string {
switch (language) {
case "javascript":
case "typescript":
case "tsx":
return "jsts";
case "python": return "python";
case "go": return "go";
case "rust": return "rust";
case "ruby": return "ruby";
case "java": return "java";
default: return "generic";
}
}
// --- Temp file management ---
let queryTmpDir: string | null = null;
const queryFileCache = new Map<string, string>();
function getQueryFile(queryKey: string): string {
if (queryFileCache.has(queryKey)) return queryFileCache.get(queryKey)!;
if (!queryTmpDir) {
queryTmpDir = mkdtempSync(join(tmpdir(), "smart-read-queries-"));
}
const filePath = join(queryTmpDir, `${queryKey}.scm`);
writeFileSync(filePath, QUERIES[queryKey]);
queryFileCache.set(queryKey, filePath);
return filePath;
}
// --- CLI execution ---
let cachedBinPath: string | null = null;
function getTreeSitterBin(): string {
if (cachedBinPath) return cachedBinPath;
// Try direct binary from tree-sitter-cli package
try {
const pkgPath = _require.resolve("tree-sitter-cli/package.json");
const binPath = join(dirname(pkgPath), "tree-sitter");
if (existsSync(binPath)) {
cachedBinPath = binPath;
return binPath;
}
} catch { /* fall through */ }
// Fallback: assume it's on PATH
cachedBinPath = "tree-sitter";
return cachedBinPath;
}
interface RawCapture {
tag: string;
startRow: number;
startCol: number;
endRow: number;
endCol: number;
text?: string;
}
interface RawMatch {
pattern: number;
captures: RawCapture[];
}
function runQuery(queryFile: string, sourceFile: string, grammarPath: string): RawMatch[] {
const result = runBatchQuery(queryFile, [sourceFile], grammarPath);
return result.get(sourceFile) || [];
}
function runBatchQuery(queryFile: string, sourceFiles: string[], grammarPath: string): Map<string, RawMatch[]> {
if (sourceFiles.length === 0) return new Map();
const bin = getTreeSitterBin();
const execArgs = ["query", "-p", grammarPath, queryFile, ...sourceFiles];
let output: string;
try {
output = execFileSync(bin, execArgs, { encoding: "utf-8", timeout: 30000, stdio: ["pipe", "pipe", "pipe"] });
} catch {
return new Map();
}
return parseMultiFileQueryOutput(output);
}
function parseMultiFileQueryOutput(output: string): Map<string, RawMatch[]> {
const fileMatches = new Map<string, RawMatch[]>();
let currentFile: string | null = null;
let currentMatch: RawMatch | null = null;
for (const line of output.split("\n")) {
// File header: a line that doesn't start with whitespace and isn't empty
if (line.length > 0 && !line.startsWith(" ") && !line.startsWith("\t")) {
currentFile = line.trim();
if (!fileMatches.has(currentFile)) {
fileMatches.set(currentFile, []);
}
currentMatch = null;
continue;
}
if (!currentFile) continue;
const patternMatch = line.match(/^\s+pattern:\s+(\d+)/);
if (patternMatch) {
currentMatch = { pattern: parseInt(patternMatch[1]), captures: [] };
fileMatches.get(currentFile)!.push(currentMatch);
continue;
}
const captureMatch = line.match(
/^\s+capture:\s+(?:\d+\s*-\s*)?(\w+),\s*start:\s*\((\d+),\s*(\d+)\),\s*end:\s*\((\d+),\s*(\d+)\)(?:,\s*text:\s*`([^`]*)`)?/
);
if (captureMatch && currentMatch) {
currentMatch.captures.push({
tag: captureMatch[1],
startRow: parseInt(captureMatch[2]),
startCol: parseInt(captureMatch[3]),
endRow: parseInt(captureMatch[4]),
endCol: parseInt(captureMatch[5]),
text: captureMatch[6],
});
}
}
return fileMatches;
}
// --- Symbol building ---
const KIND_MAP: Record<string, CodeSymbol["kind"]> = {
func: "function",
const_func: "function",
cls: "class",
method: "method",
iface: "interface",
tdef: "type",
enm: "enum",
struct_def: "struct",
trait_def: "trait",
impl_def: "impl",
};
const CONTAINER_KINDS = new Set(["class", "struct", "impl", "trait"]);
function extractSignatureFromLines(lines: string[], startRow: number, endRow: number, maxLen: number = 200): string {
const firstLine = lines[startRow] || "";
let sig = firstLine;
if (!sig.trimEnd().endsWith("{") && !sig.trimEnd().endsWith(":")) {
const chunk = lines.slice(startRow, Math.min(startRow + 10, endRow + 1)).join("\n");
const braceIdx = chunk.indexOf("{");
if (braceIdx !== -1 && braceIdx < 500) {
sig = chunk.slice(0, braceIdx).replace(/\n/g, " ").replace(/\s+/g, " ").trim();
}
}
sig = sig.replace(/\s*[{:]\s*$/, "").trim();
if (sig.length > maxLen) sig = sig.slice(0, maxLen - 3) + "...";
return sig;
}
function findCommentAbove(lines: string[], startRow: number): string | undefined {
const commentLines: string[] = [];
let foundComment = false;
for (let i = startRow - 1; i >= 0; i--) {
const trimmed = lines[i].trim();
if (trimmed === "") {
if (foundComment) break;
continue;
}
if (trimmed.startsWith("/**") || trimmed.startsWith("*") || trimmed.startsWith("*/") ||
trimmed.startsWith("//") || trimmed.startsWith("///") || trimmed.startsWith("//!") ||
trimmed.startsWith("#") || trimmed.startsWith("@")) {
commentLines.unshift(lines[i]);
foundComment = true;
} else {
break;
}
}
return commentLines.length > 0 ? commentLines.join("\n").trim() : undefined;
}
function findPythonDocstringFromLines(lines: string[], startRow: number, endRow: number): string | undefined {
for (let i = startRow + 1; i <= Math.min(startRow + 3, endRow); i++) {
const trimmed = lines[i]?.trim();
if (!trimmed) continue;
if (trimmed.startsWith('"""') || trimmed.startsWith("'''")) return trimmed;
break;
}
return undefined;
}
function isExported(
name: string, startRow: number, endRow: number,
exportRanges: Array<{ startRow: number; endRow: number }>,
lines: string[], language: string
): boolean {
switch (language) {
case "javascript":
case "typescript":
case "tsx":
return exportRanges.some(r => startRow >= r.startRow && endRow <= r.endRow);
case "python":
return !name.startsWith("_");
case "go":
return name.length > 0 && name[0] === name[0].toUpperCase() && name[0] !== name[0].toLowerCase();
case "rust":
return lines[startRow]?.trimStart().startsWith("pub") ?? false;
default:
return true;
}
}
function buildSymbols(matches: RawMatch[], lines: string[], language: string): { symbols: CodeSymbol[]; imports: string[] } {
const symbols: CodeSymbol[] = [];
const imports: string[] = [];
const exportRanges: Array<{ startRow: number; endRow: number }> = [];
const containers: Array<{ sym: CodeSymbol; startRow: number; endRow: number }> = [];
// Collect exports and imports
for (const match of matches) {
for (const cap of match.captures) {
if (cap.tag === "exp") {
exportRanges.push({ startRow: cap.startRow, endRow: cap.endRow });
}
if (cap.tag === "imp") {
imports.push(cap.text || lines[cap.startRow]?.trim() || "");
}
}
}
// Build symbols
for (const match of matches) {
const kindCapture = match.captures.find(c => KIND_MAP[c.tag]);
const nameCapture = match.captures.find(c => c.tag === "name");
if (!kindCapture) continue;
const name = nameCapture?.text || "anonymous";
const startRow = kindCapture.startRow;
const endRow = kindCapture.endRow;
const kind = KIND_MAP[kindCapture.tag];
const comment = findCommentAbove(lines, startRow);
const docstring = language === "python" ? findPythonDocstringFromLines(lines, startRow, endRow) : undefined;
const sym: CodeSymbol = {
name,
kind,
signature: extractSignatureFromLines(lines, startRow, endRow),
jsdoc: comment || docstring,
lineStart: startRow,
lineEnd: endRow,
exported: isExported(name, startRow, endRow, exportRanges, lines, language),
};
if (CONTAINER_KINDS.has(kind)) {
sym.children = [];
containers.push({ sym, startRow, endRow });
}
symbols.push(sym);
}
// Nest methods inside containers
const nested = new Set<CodeSymbol>();
for (const container of containers) {
for (const sym of symbols) {
if (sym === container.sym) continue;
if (sym.lineStart > container.startRow && sym.lineEnd <= container.endRow) {
if (sym.kind === "function") sym.kind = "method";
container.sym.children!.push(sym);
nested.add(sym);
}
}
}
return { symbols: symbols.filter(s => !nested.has(s)), imports };
}
// --- Main parse functions ---
export function parseFile(content: string, filePath: string): FoldedFile {
const language = detectLanguage(filePath);
const lines = content.split("\n");
const grammarPath = resolveGrammarPath(language);
if (!grammarPath) {
return {
filePath, language, symbols: [], imports: [],
totalLines: lines.length, foldedTokenEstimate: 50,
};
}
const queryKey = getQueryKey(language);
const queryFile = getQueryFile(queryKey);
// Write content to temp file with correct extension for language detection
const ext = filePath.slice(filePath.lastIndexOf(".")) || ".txt";
const tmpDir = mkdtempSync(join(tmpdir(), "smart-src-"));
const tmpFile = join(tmpDir, `source${ext}`);
writeFileSync(tmpFile, content);
try {
const matches = runQuery(queryFile, tmpFile, grammarPath);
const result = buildSymbols(matches, lines, language);
const folded = formatFoldedView({
filePath, language,
symbols: result.symbols, imports: result.imports,
totalLines: lines.length, foldedTokenEstimate: 0,
});
return {
filePath, language,
symbols: result.symbols, imports: result.imports,
totalLines: lines.length,
foldedTokenEstimate: Math.ceil(folded.length / 4),
};
} finally {
rmSync(tmpDir, { recursive: true, force: true });
}
}
/**
* Batch parse multiple on-disk files. Groups by language for one CLI call per language.
* Much faster than calling parseFile() per file (one process spawn per language vs per file).
*/
export function parseFilesBatch(
files: Array<{ absolutePath: string; relativePath: string; content: string }>
): Map<string, FoldedFile> {
const results = new Map<string, FoldedFile>();
// Group files by language (and thus by query + grammar)
const languageGroups = new Map<string, typeof files>();
for (const file of files) {
const language = detectLanguage(file.relativePath);
if (!languageGroups.has(language)) languageGroups.set(language, []);
languageGroups.get(language)!.push(file);
}
for (const [language, groupFiles] of languageGroups) {
const grammarPath = resolveGrammarPath(language);
if (!grammarPath) {
// No grammar — return empty results for these files
for (const file of groupFiles) {
const lines = file.content.split("\n");
results.set(file.relativePath, {
filePath: file.relativePath, language, symbols: [], imports: [],
totalLines: lines.length, foldedTokenEstimate: 50,
});
}
continue;
}
const queryKey = getQueryKey(language);
const queryFile = getQueryFile(queryKey);
// Run one batch query for all files of this language
const absolutePaths = groupFiles.map(f => f.absolutePath);
const batchResults = runBatchQuery(queryFile, absolutePaths, grammarPath);
// Build FoldedFile for each file using the batch results
for (const file of groupFiles) {
const lines = file.content.split("\n");
const matches = batchResults.get(file.absolutePath) || [];
const symbolResult = buildSymbols(matches, lines, language);
const folded = formatFoldedView({
filePath: file.relativePath, language,
symbols: symbolResult.symbols, imports: symbolResult.imports,
totalLines: lines.length, foldedTokenEstimate: 0,
});
results.set(file.relativePath, {
filePath: file.relativePath, language,
symbols: symbolResult.symbols, imports: symbolResult.imports,
totalLines: lines.length,
foldedTokenEstimate: Math.ceil(folded.length / 4),
});
}
}
return results;
}
// --- Formatting ---
export function formatFoldedView(file: FoldedFile): string {
const parts: string[] = [];
parts.push(`📁 ${file.filePath} (${file.language}, ${file.totalLines} lines)`);
parts.push("");
if (file.imports.length > 0) {
parts.push(` 📦 Imports: ${file.imports.length} statements`);
for (const imp of file.imports.slice(0, 10)) {
parts.push(` ${imp}`);
}
if (file.imports.length > 10) {
parts.push(` ... +${file.imports.length - 10} more`);
}
parts.push("");
}
for (const sym of file.symbols) {
parts.push(formatSymbol(sym, " "));
}
return parts.join("\n");
}
function formatSymbol(sym: CodeSymbol, indent: string): string {
const parts: string[] = [];
const icon = getSymbolIcon(sym.kind);
const exportTag = sym.exported ? " [exported]" : "";
const lineRange = sym.lineStart === sym.lineEnd
? `L${sym.lineStart + 1}`
: `L${sym.lineStart + 1}-${sym.lineEnd + 1}`;
parts.push(`${indent}${icon} ${sym.name}${exportTag} (${lineRange})`);
parts.push(`${indent} ${sym.signature}`);
if (sym.jsdoc) {
const jsdocLines = sym.jsdoc.split("\n");
const firstLine = jsdocLines.find(l => {
const t = l.replace(/^[\s*/]+/, "").replace(/^['"`]{3}/, "").trim();
return t.length > 0 && !t.startsWith("/**");
});
if (firstLine) {
const cleaned = firstLine.replace(/^[\s*/]+/, "").replace(/^['"`]{3}/, "").replace(/['"`]{3}$/, "").trim();
if (cleaned) {
parts.push(`${indent} 💬 ${cleaned}`);
}
}
}
if (sym.children && sym.children.length > 0) {
for (const child of sym.children) {
parts.push(formatSymbol(child, indent + " "));
}
}
return parts.join("\n");
}
function getSymbolIcon(kind: CodeSymbol["kind"]): string {
const icons: Record<string, string> = {
function: "ƒ", method: "ƒ", class: "◆", interface: "◇",
type: "◇", const: "●", variable: "○", export: "→",
struct: "◆", enum: "▣", trait: "◇", impl: "◈",
property: "○", getter: "⇢", setter: "⇠",
};
return icons[kind] || "·";
}
// --- Unfold ---
export function unfoldSymbol(content: string, filePath: string, symbolName: string): string | null {
const file = parseFile(content, filePath);
const findSymbol = (symbols: CodeSymbol[]): CodeSymbol | null => {
for (const sym of symbols) {
if (sym.name === symbolName) return sym;
if (sym.children) {
const found = findSymbol(sym.children);
if (found) return found;
}
}
return null;
};
const symbol = findSymbol(file.symbols);
if (!symbol) return null;
const lines = content.split("\n");
// Include preceding comments/decorators
let start = symbol.lineStart;
for (let i = symbol.lineStart - 1; i >= 0; i--) {
const trimmed = lines[i].trim();
if (trimmed === "" || trimmed.startsWith("*") || trimmed.startsWith("/**") ||
trimmed.startsWith("///") || trimmed.startsWith("//") ||
trimmed.startsWith("#") || trimmed.startsWith("@") ||
trimmed === "*/") {
start = i;
} else {
break;
}
}
const extracted = lines.slice(start, symbol.lineEnd + 1).join("\n");
return `// 📍 ${filePath} L${start + 1}-${symbol.lineEnd + 1}\n${extracted}`;
}
+316
View File
@@ -0,0 +1,316 @@
/**
* Search module finds code files and symbols matching a query.
*
* Two search modes:
* 1. Grep-style: find files/lines containing the query string
* 2. Structural: parse files and match against symbol names/signatures
*
* Both return folded views, not raw content.
*
* Uses batch parsing (one CLI call per language) for fast multi-file search.
*/
import { readFile, readdir, stat } from "node:fs/promises";
import { join, relative } from "node:path";
import { parseFilesBatch, formatFoldedView, type FoldedFile } from "./parser.js";
const CODE_EXTENSIONS = new Set([
".js", ".jsx", ".ts", ".tsx", ".mjs", ".cjs",
".py", ".pyw",
".go",
".rs",
".rb",
".java",
".cs",
".cpp", ".c", ".h", ".hpp",
".swift",
".kt",
".php",
".vue", ".svelte",
]);
const IGNORE_DIRS = new Set([
"node_modules", ".git", "dist", "build", ".next", "__pycache__",
".venv", "venv", "env", ".env", "target", "vendor",
".cache", ".turbo", "coverage", ".nyc_output",
".claude", ".smart-file-read",
]);
const MAX_FILE_SIZE = 512 * 1024; // 512KB — skip huge files
export interface SearchResult {
foldedFiles: FoldedFile[];
matchingSymbols: SymbolMatch[];
totalFilesScanned: number;
totalSymbolsFound: number;
tokenEstimate: number;
}
export interface SymbolMatch {
filePath: string;
symbolName: string;
kind: string;
signature: string;
jsdoc?: string;
lineStart: number;
lineEnd: number;
matchReason: string; // why this matched
}
/**
* Walk a directory recursively, yielding file paths.
*/
async function* walkDir(dir: string, rootDir: string, maxDepth: number = 20): AsyncGenerator<string> {
if (maxDepth <= 0) return;
let entries;
try {
entries = await readdir(dir, { withFileTypes: true });
} catch {
return; // permission denied, etc.
}
for (const entry of entries) {
if (entry.name.startsWith(".") && entry.name !== ".") continue;
if (IGNORE_DIRS.has(entry.name)) continue;
const fullPath = join(dir, entry.name);
if (entry.isDirectory()) {
yield* walkDir(fullPath, rootDir, maxDepth - 1);
} else if (entry.isFile()) {
const ext = entry.name.slice(entry.name.lastIndexOf("."));
if (CODE_EXTENSIONS.has(ext)) {
yield fullPath;
}
}
}
}
/**
* Read a file safely, skipping if too large or binary.
*/
async function safeReadFile(filePath: string): Promise<string | null> {
try {
const stats = await stat(filePath);
if (stats.size > MAX_FILE_SIZE) return null;
if (stats.size === 0) return null;
const content = await readFile(filePath, "utf-8");
// Quick binary check — if first 1000 chars have null bytes, skip
if (content.slice(0, 1000).includes("\0")) return null;
return content;
} catch {
return null;
}
}
/**
* Search a codebase for symbols matching a query.
*
* Phase 1: Collect files and read content
* Phase 2: Batch parse all files (one CLI call per language)
* Phase 3: Match query against parsed symbols
*/
export async function searchCodebase(
rootDir: string,
query: string,
options: {
maxResults?: number;
includeImports?: boolean;
filePattern?: string;
} = {}
): Promise<SearchResult> {
const maxResults = options.maxResults || 20;
const queryLower = query.toLowerCase();
const queryParts = queryLower.split(/[\s_\-./]+/).filter(p => p.length > 0);
// Phase 1: Collect files
const filesToParse: Array<{ absolutePath: string; relativePath: string; content: string }> = [];
for await (const filePath of walkDir(rootDir, rootDir)) {
if (options.filePattern) {
const relPath = relative(rootDir, filePath);
if (!relPath.toLowerCase().includes(options.filePattern.toLowerCase())) continue;
}
const content = await safeReadFile(filePath);
if (!content) continue;
filesToParse.push({
absolutePath: filePath,
relativePath: relative(rootDir, filePath),
content,
});
}
// Phase 2: Batch parse (one CLI call per language)
const parsedFiles = parseFilesBatch(filesToParse);
// Phase 3: Match query against symbols
const foldedFiles: FoldedFile[] = [];
const matchingSymbols: SymbolMatch[] = [];
let totalSymbolsFound = 0;
for (const [relPath, parsed] of parsedFiles) {
totalSymbolsFound += countSymbols(parsed);
const pathMatch = matchScore(relPath.toLowerCase(), queryParts);
let fileHasMatch = pathMatch > 0;
const fileSymbolMatches: SymbolMatch[] = [];
const checkSymbols = (symbols: typeof parsed.symbols, parent?: string) => {
for (const sym of symbols) {
let score = 0;
let reason = "";
const nameScore = matchScore(sym.name.toLowerCase(), queryParts);
if (nameScore > 0) {
score += nameScore * 3;
reason = "name match";
}
if (sym.signature.toLowerCase().includes(queryLower)) {
score += 2;
reason = reason ? `${reason} + signature` : "signature match";
}
if (sym.jsdoc && sym.jsdoc.toLowerCase().includes(queryLower)) {
score += 1;
reason = reason ? `${reason} + jsdoc` : "jsdoc match";
}
if (score > 0) {
fileHasMatch = true;
fileSymbolMatches.push({
filePath: relPath,
symbolName: parent ? `${parent}.${sym.name}` : sym.name,
kind: sym.kind,
signature: sym.signature,
jsdoc: sym.jsdoc,
lineStart: sym.lineStart,
lineEnd: sym.lineEnd,
matchReason: reason,
});
}
if (sym.children) {
checkSymbols(sym.children, sym.name);
}
}
};
checkSymbols(parsed.symbols);
if (fileHasMatch) {
foldedFiles.push(parsed);
matchingSymbols.push(...fileSymbolMatches);
}
}
// Sort by relevance and trim
matchingSymbols.sort((a, b) => {
const aScore = matchScore(a.symbolName.toLowerCase(), queryParts);
const bScore = matchScore(b.symbolName.toLowerCase(), queryParts);
return bScore - aScore;
});
const trimmedSymbols = matchingSymbols.slice(0, maxResults);
const relevantFiles = new Set(trimmedSymbols.map(s => s.filePath));
const trimmedFiles = foldedFiles.filter(f => relevantFiles.has(f.filePath)).slice(0, maxResults);
const tokenEstimate = trimmedFiles.reduce((sum, f) => sum + f.foldedTokenEstimate, 0);
return {
foldedFiles: trimmedFiles,
matchingSymbols: trimmedSymbols,
totalFilesScanned: filesToParse.length,
totalSymbolsFound,
tokenEstimate,
};
}
/**
* Score how well query parts match a string.
* Returns 0 for no match, higher for better matches.
*/
function matchScore(text: string, queryParts: string[]): number {
let score = 0;
for (const part of queryParts) {
if (text === part) {
score += 10; // exact match
} else if (text.includes(part)) {
score += 5; // substring match
} else {
// Fuzzy: check if all chars appear in order
let ti = 0;
let matched = 0;
for (const ch of part) {
const idx = text.indexOf(ch, ti);
if (idx !== -1) {
matched++;
ti = idx + 1;
}
}
if (matched === part.length) {
score += 1; // loose fuzzy match
}
}
}
return score;
}
function countSymbols(file: FoldedFile): number {
let count = file.symbols.length;
for (const sym of file.symbols) {
if (sym.children) count += sym.children.length;
}
return count;
}
/**
* Format search results for LLM consumption.
*/
export function formatSearchResults(result: SearchResult, query: string): string {
const parts: string[] = [];
parts.push(`🔍 Smart Search: "${query}"`);
parts.push(` Scanned ${result.totalFilesScanned} files, found ${result.totalSymbolsFound} symbols`);
parts.push(` ${result.matchingSymbols.length} matches across ${result.foldedFiles.length} files (~${result.tokenEstimate} tokens for folded view)`);
parts.push("");
if (result.matchingSymbols.length === 0) {
parts.push(" No matching symbols found.");
return parts.join("\n");
}
// Show matching symbols first (compact)
parts.push("── Matching Symbols ──");
parts.push("");
for (const match of result.matchingSymbols) {
parts.push(` ${match.kind} ${match.symbolName} (${match.filePath}:${match.lineStart + 1})`);
parts.push(` ${match.signature}`);
if (match.jsdoc) {
const firstLine = match.jsdoc.split("\n").find(l => l.replace(/^[\s*/]+/, "").trim().length > 0);
if (firstLine) {
parts.push(` 💬 ${firstLine.replace(/^[\s*/]+/, "").trim()}`);
}
}
parts.push("");
}
// Show folded file views
parts.push("── Folded File Views ──");
parts.push("");
for (const file of result.foldedFiles) {
parts.push(formatFoldedView(file));
parts.push("");
}
parts.push("── Actions ──");
parts.push(' To see full implementation: use smart_unfold with file path and symbol name');
return parts.join("\n");
}
+125
View File
@@ -1,4 +1,8 @@
import { Database } from 'bun:sqlite';
import { execFileSync } from 'child_process';
import { existsSync, unlinkSync, writeFileSync } from 'fs';
import { tmpdir } from 'os';
import { join } from 'path';
import { DATA_DIR, DB_PATH, ensureDir } from '../../shared/paths.js';
import { logger } from '../../utils/logger.js';
import { MigrationRunner } from './migrations/runner.js';
@@ -15,6 +19,118 @@ export interface Migration {
let dbInstance: Database | null = null;
/**
* Repair malformed database schema before migrations run.
*
* This handles the case where a database is synced between machines running
* different claude-mem versions. A newer version may have added columns and
* indexes that an older version (or even the same version on a fresh install)
* cannot process. SQLite throws "malformed database schema" when it encounters
* an index referencing a non-existent column, which prevents ALL queries
* including the migrations that would fix the schema.
*
* The fix: use Python's sqlite3 module (which supports writable_schema) to
* drop the orphaned schema objects, then let the migration system recreate
* them properly. bun:sqlite doesn't allow DELETE FROM sqlite_master even
* with writable_schema = ON.
*/
function repairMalformedSchema(db: Database): void {
try {
// Quick test: if we can query sqlite_master, the schema is fine
db.query('SELECT name FROM sqlite_master WHERE type = "table" LIMIT 1').all();
return;
} catch (error: unknown) {
const message = error instanceof Error ? error.message : String(error);
if (!message.includes('malformed database schema')) {
throw error;
}
logger.warn('DB', 'Detected malformed database schema, attempting repair', { error: message });
// Extract the problematic object name from the error message
// Format: "malformed database schema (object_name) - details"
const match = message.match(/malformed database schema \(([^)]+)\)/);
if (!match) {
logger.error('DB', 'Could not parse malformed schema error, cannot auto-repair', { error: message });
throw error;
}
const objectName = match[1];
logger.info('DB', `Dropping malformed schema object: ${objectName}`);
// Get the DB file path. For file-based DBs, we can use Python to repair.
// For in-memory DBs, we can't shell out — just re-throw.
const dbPath = db.filename;
if (!dbPath || dbPath === ':memory:' || dbPath === '') {
logger.error('DB', 'Cannot auto-repair in-memory database');
throw error;
}
// Close the connection so Python can safely modify the file
db.close();
// Use Python's sqlite3 module to drop the orphaned object and reset
// related migration versions so they re-run and recreate things properly.
// bun:sqlite doesn't support DELETE FROM sqlite_master even with writable_schema.
//
// We write a temp script rather than using -c to avoid shell escaping issues
// with paths containing spaces or special characters. execFileSync passes
// args directly without a shell, so dbPath and objectName are safe.
const scriptPath = join(tmpdir(), `claude-mem-repair-${Date.now()}.py`);
try {
writeFileSync(scriptPath, `
import sqlite3, sys
db_path = sys.argv[1]
obj_name = sys.argv[2]
c = sqlite3.connect(db_path)
c.execute('PRAGMA writable_schema = ON')
c.execute('DELETE FROM sqlite_master WHERE name = ?', (obj_name,))
c.execute('PRAGMA writable_schema = OFF')
# Reset migration versions so affected migrations re-run.
# Guard with existence check: schema_versions may not exist on a very fresh DB.
has_sv = c.execute(
"SELECT count(*) FROM sqlite_master WHERE type='table' AND name='schema_versions'"
).fetchone()[0]
if has_sv:
c.execute('DELETE FROM schema_versions')
c.commit()
c.close()
`);
execFileSync('python3', [scriptPath, dbPath, objectName], { timeout: 10000 });
logger.info('DB', `Dropped orphaned schema object "${objectName}" and reset migration versions via Python sqlite3. All migrations will re-run (they are idempotent).`);
} catch (pyError: unknown) {
const pyMessage = pyError instanceof Error ? pyError.message : String(pyError);
logger.error('DB', 'Python sqlite3 repair failed', { error: pyMessage });
throw new Error(`Schema repair failed: ${message}. Python repair error: ${pyMessage}`);
} finally {
if (existsSync(scriptPath)) unlinkSync(scriptPath);
}
}
}
/**
* Wrapper that handles the close/reopen cycle needed for schema repair.
* Returns a (possibly new) Database connection.
*/
function repairMalformedSchemaWithReopen(dbPath: string, db: Database): Database {
try {
db.query('SELECT name FROM sqlite_master WHERE type = "table" LIMIT 1').all();
return db;
} catch (error: unknown) {
const message = error instanceof Error ? error.message : String(error);
if (!message.includes('malformed database schema')) {
throw error;
}
// repairMalformedSchema closes the DB internally for Python access
repairMalformedSchema(db);
// Reopen and check for additional malformed objects
const newDb = new Database(dbPath, { create: true, readwrite: true });
return repairMalformedSchemaWithReopen(dbPath, newDb);
}
}
/**
* ClaudeMemDatabase - New entry point for the sqlite module
*
@@ -38,6 +154,11 @@ export class ClaudeMemDatabase {
// Create database connection
this.db = new Database(dbPath, { create: true, readwrite: true });
// Repair any malformed schema before applying settings or running migrations.
// Must happen first — even PRAGMA calls can fail on a corrupted schema.
// This may close and reopen the connection if repair is needed.
this.db = repairMalformedSchemaWithReopen(dbPath, this.db);
// Apply optimized SQLite settings
this.db.run('PRAGMA journal_mode = WAL');
this.db.run('PRAGMA synchronous = NORMAL');
@@ -97,6 +218,10 @@ export class DatabaseManager {
this.db = new Database(DB_PATH, { create: true, readwrite: true });
// Repair any malformed schema before applying settings or running migrations.
// Must happen first — even PRAGMA calls can fail on a corrupted schema.
this.db = repairMalformedSchemaWithReopen(DB_PATH, this.db);
// Apply optimized SQLite settings
this.db.run('PRAGMA journal_mode = WAL');
this.db.run('PRAGMA synchronous = NORMAL');
+14 -1
View File
@@ -398,9 +398,22 @@ export class PendingMessageStore {
}
/**
* Check if any session has pending work
* Check if any session has pending work.
* Excludes 'processing' messages stuck for >5 minutes (resets them to 'pending' as a side effect).
*/
hasAnyPendingWork(): boolean {
// Reset stuck 'processing' messages older than 5 minutes before checking
const stuckCutoff = Date.now() - (5 * 60 * 1000);
const resetStmt = this.db.prepare(`
UPDATE pending_messages
SET status = 'pending', started_processing_at_epoch = NULL
WHERE status = 'processing' AND started_processing_at_epoch < ?
`);
const resetResult = resetStmt.run(stuckCutoff);
if (resetResult.changes > 0) {
logger.info('QUEUE', `STUCK_RESET | hasAnyPendingWork reset ${resetResult.changes} stuck processing message(s) older than 5 minutes`);
}
const stmt = this.db.prepare(`
SELECT COUNT(*) as count FROM pending_messages
WHERE status IN ('pending', 'processing')
+102 -72
View File
@@ -46,6 +46,10 @@ export class SessionSearch {
* - Tables maintained but search paths removed
* - Triggers still fire to keep tables synchronized
*
* FTS5 may be unavailable on some platforms (e.g., Bun on Windows #791).
* When unavailable, we skip FTS table creation search falls back to
* ChromaDB (vector) and LIKE queries (structured filters) which are unaffected.
*
* TODO: Remove FTS5 infrastructure in future major version (v7.0.0)
*/
private ensureFTSTables(): void {
@@ -58,91 +62,117 @@ export class SessionSearch {
return;
}
// Runtime check: verify FTS5 is available before attempting to create tables.
// bun:sqlite on Windows may not include the FTS5 extension (#791).
if (!this.isFts5Available()) {
logger.warn('DB', 'FTS5 not available on this platform — skipping FTS table creation (search uses ChromaDB)');
return;
}
logger.info('DB', 'Creating FTS5 tables');
// Create observations_fts virtual table
this.db.run(`
CREATE VIRTUAL TABLE IF NOT EXISTS observations_fts USING fts5(
title,
subtitle,
narrative,
text,
facts,
concepts,
content='observations',
content_rowid='id'
);
`);
try {
// Create observations_fts virtual table
this.db.run(`
CREATE VIRTUAL TABLE IF NOT EXISTS observations_fts USING fts5(
title,
subtitle,
narrative,
text,
facts,
concepts,
content='observations',
content_rowid='id'
);
`);
// Populate with existing data
this.db.run(`
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
SELECT id, title, subtitle, narrative, text, facts, concepts
FROM observations;
`);
// Create triggers for observations
this.db.run(`
CREATE TRIGGER IF NOT EXISTS observations_ai AFTER INSERT ON observations BEGIN
// Populate with existing data
this.db.run(`
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
END;
SELECT id, title, subtitle, narrative, text, facts, concepts
FROM observations;
`);
CREATE TRIGGER IF NOT EXISTS observations_ad AFTER DELETE ON observations BEGIN
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
END;
// Create triggers for observations
this.db.run(`
CREATE TRIGGER IF NOT EXISTS observations_ai AFTER INSERT ON observations BEGIN
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
END;
CREATE TRIGGER IF NOT EXISTS observations_au AFTER UPDATE ON observations BEGIN
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
END;
`);
CREATE TRIGGER IF NOT EXISTS observations_ad AFTER DELETE ON observations BEGIN
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
END;
// Create session_summaries_fts virtual table
this.db.run(`
CREATE VIRTUAL TABLE IF NOT EXISTS session_summaries_fts USING fts5(
request,
investigated,
learned,
completed,
next_steps,
notes,
content='session_summaries',
content_rowid='id'
);
`);
CREATE TRIGGER IF NOT EXISTS observations_au AFTER UPDATE ON observations BEGIN
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
END;
`);
// Populate with existing data
this.db.run(`
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
SELECT id, request, investigated, learned, completed, next_steps, notes
FROM session_summaries;
`);
// Create session_summaries_fts virtual table
this.db.run(`
CREATE VIRTUAL TABLE IF NOT EXISTS session_summaries_fts USING fts5(
request,
investigated,
learned,
completed,
next_steps,
notes,
content='session_summaries',
content_rowid='id'
);
`);
// Create triggers for session_summaries
this.db.run(`
CREATE TRIGGER IF NOT EXISTS session_summaries_ai AFTER INSERT ON session_summaries BEGIN
// Populate with existing data
this.db.run(`
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
END;
SELECT id, request, investigated, learned, completed, next_steps, notes
FROM session_summaries;
`);
CREATE TRIGGER IF NOT EXISTS session_summaries_ad AFTER DELETE ON session_summaries BEGIN
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
END;
// Create triggers for session_summaries
this.db.run(`
CREATE TRIGGER IF NOT EXISTS session_summaries_ai AFTER INSERT ON session_summaries BEGIN
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
END;
CREATE TRIGGER IF NOT EXISTS session_summaries_au AFTER UPDATE ON session_summaries BEGIN
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
END;
`);
CREATE TRIGGER IF NOT EXISTS session_summaries_ad AFTER DELETE ON session_summaries BEGIN
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
END;
logger.info('DB', 'FTS5 tables created successfully');
CREATE TRIGGER IF NOT EXISTS session_summaries_au AFTER UPDATE ON session_summaries BEGIN
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
END;
`);
logger.info('DB', 'FTS5 tables created successfully');
} catch (error) {
// FTS5 creation failed at runtime despite probe succeeding — degrade gracefully
logger.warn('DB', 'FTS5 table creation failed — search will use ChromaDB and LIKE queries', {}, error as Error);
}
}
/**
* Probe whether the FTS5 extension is available in the current SQLite build.
* Creates and immediately drops a temporary FTS5 table.
*/
private isFts5Available(): boolean {
try {
this.db.run('CREATE VIRTUAL TABLE _fts5_probe USING fts5(test_column)');
this.db.run('DROP TABLE _fts5_probe');
return true;
} catch {
return false;
}
}
+209 -123
View File
@@ -13,6 +13,7 @@ import {
LatestPromptResult
} from '../../types/database.js';
import type { PendingMessageStore } from './PendingMessageStore.js';
import { computeObservationContentHash, findDuplicateObservation } from './observations/store.js';
/**
* Session data store for SDK sessions, observations, and summaries
@@ -48,11 +49,17 @@ export class SessionStore {
this.repairSessionIdColumnRename();
this.addFailedAtEpochColumn();
this.addOnUpdateCascadeToForeignKeys();
this.addObservationContentHashColumn();
this.addSessionCustomTitleColumn();
}
/**
* Initialize database schema using migrations (migration004)
* This runs the core SDK tables migration if no tables exist
* Initialize database schema (migration004)
*
* ALWAYS creates core tables using CREATE TABLE IF NOT EXISTS safe to run
* regardless of schema_versions state. This fixes issue #979 where the old
* DatabaseManager migration system (versions 1-7) shared the schema_versions
* table, causing maxApplied > 0 and skipping core table creation entirely.
*/
private initializeSchema(): void {
// Create schema_versions table if it doesn't exist
@@ -64,90 +71,77 @@ export class SessionStore {
)
`);
// Get applied migrations
const appliedVersions = this.db.prepare('SELECT version FROM schema_versions ORDER BY version').all() as SchemaVersion[];
const maxApplied = appliedVersions.length > 0 ? Math.max(...appliedVersions.map(v => v.version)) : 0;
// Always create core tables — IF NOT EXISTS makes this idempotent
this.db.run(`
CREATE TABLE IF NOT EXISTS sdk_sessions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
content_session_id TEXT UNIQUE NOT NULL,
memory_session_id TEXT UNIQUE,
project TEXT NOT NULL,
user_prompt TEXT,
started_at TEXT NOT NULL,
started_at_epoch INTEGER NOT NULL,
completed_at TEXT,
completed_at_epoch INTEGER,
status TEXT CHECK(status IN ('active', 'completed', 'failed')) NOT NULL DEFAULT 'active'
);
// Only run migration004 if no migrations have been applied
// This creates the sdk_sessions, observations, and session_summaries tables
if (maxApplied === 0) {
logger.info('DB', 'Initializing fresh database with migration004');
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_claude_id ON sdk_sessions(content_session_id);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_sdk_id ON sdk_sessions(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_project ON sdk_sessions(project);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_status ON sdk_sessions(status);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_started ON sdk_sessions(started_at_epoch DESC);
// Migration004: SDK agent architecture tables
this.db.run(`
CREATE TABLE IF NOT EXISTS sdk_sessions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
content_session_id TEXT UNIQUE NOT NULL,
memory_session_id TEXT UNIQUE,
project TEXT NOT NULL,
user_prompt TEXT,
started_at TEXT NOT NULL,
started_at_epoch INTEGER NOT NULL,
completed_at TEXT,
completed_at_epoch INTEGER,
status TEXT CHECK(status IN ('active', 'completed', 'failed')) NOT NULL DEFAULT 'active'
);
CREATE TABLE IF NOT EXISTS observations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
memory_session_id TEXT NOT NULL,
project TEXT NOT NULL,
text TEXT NOT NULL,
type TEXT NOT NULL,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_claude_id ON sdk_sessions(content_session_id);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_sdk_id ON sdk_sessions(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_project ON sdk_sessions(project);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_status ON sdk_sessions(status);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_started ON sdk_sessions(started_at_epoch DESC);
CREATE INDEX IF NOT EXISTS idx_observations_sdk_session ON observations(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_observations_project ON observations(project);
CREATE INDEX IF NOT EXISTS idx_observations_type ON observations(type);
CREATE INDEX IF NOT EXISTS idx_observations_created ON observations(created_at_epoch DESC);
CREATE TABLE IF NOT EXISTS observations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
memory_session_id TEXT NOT NULL,
project TEXT NOT NULL,
text TEXT NOT NULL,
type TEXT NOT NULL,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE TABLE IF NOT EXISTS session_summaries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
memory_session_id TEXT UNIQUE NOT NULL,
project TEXT NOT NULL,
request TEXT,
investigated TEXT,
learned TEXT,
completed TEXT,
next_steps TEXT,
files_read TEXT,
files_edited TEXT,
notes TEXT,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_observations_sdk_session ON observations(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_observations_project ON observations(project);
CREATE INDEX IF NOT EXISTS idx_observations_type ON observations(type);
CREATE INDEX IF NOT EXISTS idx_observations_created ON observations(created_at_epoch DESC);
CREATE INDEX IF NOT EXISTS idx_session_summaries_sdk_session ON session_summaries(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_session_summaries_project ON session_summaries(project);
CREATE INDEX IF NOT EXISTS idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
`);
CREATE TABLE IF NOT EXISTS session_summaries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
memory_session_id TEXT UNIQUE NOT NULL,
project TEXT NOT NULL,
request TEXT,
investigated TEXT,
learned TEXT,
completed TEXT,
next_steps TEXT,
files_read TEXT,
files_edited TEXT,
notes TEXT,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_session_summaries_sdk_session ON session_summaries(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_session_summaries_project ON session_summaries(project);
CREATE INDEX IF NOT EXISTS idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
`);
// Record migration004 as applied
this.db.prepare('INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)').run(4, new Date().toISOString());
logger.info('DB', 'Migration004 applied successfully');
}
// Record migration004 as applied (OR IGNORE handles re-runs safely)
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(4, new Date().toISOString());
}
/**
* Ensure worker_port column exists (migration 5)
*
* NOTE: Version 5 conflicts with old DatabaseManager migration005 (which drops orphaned tables).
* We check actual column state rather than relying solely on version tracking.
*/
private ensureWorkerPortColumn(): void {
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(5) as SchemaVersion | undefined;
if (applied) return;
// Check if column exists
// Check actual column existence — don't rely on version tracking alone (issue #979)
const tableInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
const hasWorkerPort = tableInfo.some(col => col.name === 'worker_port');
@@ -162,12 +156,12 @@ export class SessionStore {
/**
* Ensure prompt tracking columns exist (migration 6)
*
* NOTE: Version 6 conflicts with old DatabaseManager migration006 (which creates FTS5 tables).
* We check actual column state rather than relying solely on version tracking.
*/
private ensurePromptTrackingColumns(): void {
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(6) as SchemaVersion | undefined;
if (applied) return;
// Check actual column existence — don't rely on version tracking alone (issue #979)
// Check sdk_sessions for prompt_counter
const sessionsInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
const hasPromptCounter = sessionsInfo.some(col => col.name === 'prompt_counter');
@@ -201,13 +195,12 @@ export class SessionStore {
/**
* Remove UNIQUE constraint from session_summaries.memory_session_id (migration 7)
*
* NOTE: Version 7 conflicts with old DatabaseManager migration007 (which adds discovery_tokens).
* We check actual constraint state rather than relying solely on version tracking.
*/
private removeSessionSummariesUniqueConstraint(): void {
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(7) as SchemaVersion | undefined;
if (applied) return;
// Check if UNIQUE constraint exists
// Check actual constraint state — don't rely on version tracking alone (issue #979)
const summariesIndexes = this.db.query('PRAGMA index_list(session_summaries)').all() as IndexInfo[];
const hasUniqueConstraint = summariesIndexes.some(idx => idx.unique === 1);
@@ -222,6 +215,9 @@ export class SessionStore {
// Begin transaction
this.db.run('BEGIN TRANSACTION');
// Clean up leftover temp table from a previously-crashed run
this.db.run('DROP TABLE IF EXISTS session_summaries_new');
// Create new table without UNIQUE constraint
this.db.run(`
CREATE TABLE session_summaries_new (
@@ -335,6 +331,9 @@ export class SessionStore {
// Begin transaction
this.db.run('BEGIN TRANSACTION');
// Clean up leftover temp table from a previously-crashed run
this.db.run('DROP TABLE IF EXISTS observations_new');
// Create new table with text as nullable
this.db.run(`
CREATE TABLE observations_new (
@@ -428,34 +427,39 @@ export class SessionStore {
CREATE INDEX idx_user_prompts_lookup ON user_prompts(content_session_id, prompt_number);
`);
// Create FTS5 virtual table
this.db.run(`
CREATE VIRTUAL TABLE user_prompts_fts USING fts5(
prompt_text,
content='user_prompts',
content_rowid='id'
);
`);
// Create FTS5 virtual table — skip if FTS5 is unavailable (e.g., Bun on Windows #791).
// The user_prompts table itself is still created; only FTS indexing is skipped.
try {
this.db.run(`
CREATE VIRTUAL TABLE user_prompts_fts USING fts5(
prompt_text,
content='user_prompts',
content_rowid='id'
);
`);
// Create triggers to sync FTS5
this.db.run(`
CREATE TRIGGER user_prompts_ai AFTER INSERT ON user_prompts BEGIN
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
// Create triggers to sync FTS5
this.db.run(`
CREATE TRIGGER user_prompts_ai AFTER INSERT ON user_prompts BEGIN
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
CREATE TRIGGER user_prompts_ad AFTER DELETE ON user_prompts BEGIN
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
VALUES('delete', old.id, old.prompt_text);
END;
CREATE TRIGGER user_prompts_ad AFTER DELETE ON user_prompts BEGIN
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
VALUES('delete', old.id, old.prompt_text);
END;
CREATE TRIGGER user_prompts_au AFTER UPDATE ON user_prompts BEGIN
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
VALUES('delete', old.id, old.prompt_text);
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
`);
CREATE TRIGGER user_prompts_au AFTER UPDATE ON user_prompts BEGIN
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
VALUES('delete', old.id, old.prompt_text);
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
`);
} catch (ftsError) {
logger.warn('DB', 'FTS5 not available — user_prompts_fts skipped (search uses ChromaDB)', {}, ftsError as Error);
}
// Commit transaction
this.db.run('COMMIT');
@@ -463,7 +467,7 @@ export class SessionStore {
// Record migration
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(10, new Date().toISOString());
logger.debug('DB', 'Successfully created user_prompts table with FTS5 support');
logger.debug('DB', 'Successfully created user_prompts table');
}
/**
@@ -675,6 +679,9 @@ export class SessionStore {
this.db.run('DROP TRIGGER IF EXISTS observations_ad');
this.db.run('DROP TRIGGER IF EXISTS observations_au');
// Clean up leftover temp table from a previously-crashed run
this.db.run('DROP TABLE IF EXISTS observations_new');
this.db.run(`
CREATE TABLE observations_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
@@ -744,6 +751,9 @@ export class SessionStore {
// 2. Recreate session_summaries table
// ==========================================
// Clean up leftover temp table from a previously-crashed run
this.db.run('DROP TABLE IF EXISTS session_summaries_new');
this.db.run(`
CREATE TABLE session_summaries_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
@@ -825,6 +835,46 @@ export class SessionStore {
}
}
/**
* Add content_hash column to observations for deduplication (migration 22)
*/
private addObservationContentHashColumn(): void {
// Check actual schema first — cross-machine DB sync can leave schema_versions
// claiming this migration ran while the column is actually missing.
const tableInfo = this.db.query('PRAGMA table_info(observations)').all() as TableColumnInfo[];
const hasColumn = tableInfo.some(col => col.name === 'content_hash');
if (hasColumn) {
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(22, new Date().toISOString());
return;
}
this.db.run('ALTER TABLE observations ADD COLUMN content_hash TEXT');
this.db.run("UPDATE observations SET content_hash = substr(hex(randomblob(8)), 1, 16) WHERE content_hash IS NULL");
this.db.run('CREATE INDEX IF NOT EXISTS idx_observations_content_hash ON observations(content_hash, created_at_epoch)');
logger.debug('DB', 'Added content_hash column to observations table with backfill and index');
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(22, new Date().toISOString());
}
/**
* Add custom_title column to sdk_sessions for agent attribution (migration 23)
*/
private addSessionCustomTitleColumn(): void {
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(23) as SchemaVersion | undefined;
if (applied) return;
const tableInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
const hasColumn = tableInfo.some(col => col.name === 'custom_title');
if (!hasColumn) {
this.db.run('ALTER TABLE sdk_sessions ADD COLUMN custom_title TEXT');
logger.debug('DB', 'Added custom_title column to sdk_sessions table');
}
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(23, new Date().toISOString());
}
/**
* Update the memory session ID for a session
* Called by SDKAgent when it captures the session ID from the first SDK message
@@ -1290,9 +1340,10 @@ export class SessionStore {
memory_session_id: string | null;
project: string;
user_prompt: string;
custom_title: string | null;
} | null {
const stmt = this.db.prepare(`
SELECT id, content_session_id, memory_session_id, project, user_prompt
SELECT id, content_session_id, memory_session_id, project, user_prompt, custom_title
FROM sdk_sessions
WHERE id = ?
LIMIT 1
@@ -1311,6 +1362,7 @@ export class SessionStore {
memory_session_id: string;
project: string;
user_prompt: string;
custom_title: string | null;
started_at: string;
started_at_epoch: number;
completed_at: string | null;
@@ -1321,7 +1373,7 @@ export class SessionStore {
const placeholders = memorySessionIds.map(() => '?').join(',');
const stmt = this.db.prepare(`
SELECT id, content_session_id, memory_session_id, project, user_prompt,
SELECT id, content_session_id, memory_session_id, project, user_prompt, custom_title,
started_at, started_at_epoch, completed_at, completed_at_epoch, status
FROM sdk_sessions
WHERE memory_session_id IN (${placeholders})
@@ -1366,7 +1418,7 @@ export class SessionStore {
* Pure get-or-create: never modifies memory_session_id.
* Multi-terminal isolation is handled by ON UPDATE CASCADE at the schema level.
*/
createSDKSession(contentSessionId: string, project: string, userPrompt: string): number {
createSDKSession(contentSessionId: string, project: string, userPrompt: string, customTitle?: string): number {
const now = new Date();
const nowEpoch = now.getTime();
@@ -1383,6 +1435,13 @@ export class SessionStore {
WHERE content_session_id = ? AND (project IS NULL OR project = '')
`).run(project, contentSessionId);
}
// Backfill custom_title if provided and not yet set
if (customTitle) {
this.db.prepare(`
UPDATE sdk_sessions SET custom_title = ?
WHERE content_session_id = ? AND custom_title IS NULL
`).run(customTitle, contentSessionId);
}
return existing.id;
}
@@ -1392,9 +1451,9 @@ export class SessionStore {
// must NEVER equal contentSessionId - that would inject memory messages into the user's transcript!
this.db.prepare(`
INSERT INTO sdk_sessions
(content_session_id, memory_session_id, project, user_prompt, started_at, started_at_epoch, status)
VALUES (?, NULL, ?, ?, ?, ?, 'active')
`).run(contentSessionId, project, userPrompt, now.toISOString(), nowEpoch);
(content_session_id, memory_session_id, project, user_prompt, custom_title, started_at, started_at_epoch, status)
VALUES (?, NULL, ?, ?, ?, ?, ?, 'active')
`).run(contentSessionId, project, userPrompt, customTitle || null, now.toISOString(), nowEpoch);
// Return new ID
const row = this.db.prepare('SELECT id FROM sdk_sessions WHERE content_session_id = ?')
@@ -1441,6 +1500,7 @@ export class SessionStore {
/**
* Store an observation (from SDK parsing)
* Assumes session already exists (created by hook)
* Performs content-hash deduplication: skips INSERT if an identical observation exists within 30s
*/
storeObservation(
memorySessionId: string,
@@ -1463,11 +1523,18 @@ export class SessionStore {
const timestampEpoch = overrideTimestampEpoch ?? Date.now();
const timestampIso = new Date(timestampEpoch).toISOString();
// Content-hash deduplication
const contentHash = computeObservationContentHash(memorySessionId, observation.title, observation.narrative);
const existing = findDuplicateObservation(this.db, contentHash, timestampEpoch);
if (existing) {
return { id: existing.id, createdAtEpoch: existing.created_at_epoch };
}
const stmt = this.db.prepare(`
INSERT INTO observations
(memory_session_id, project, type, title, subtitle, facts, narrative, concepts,
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
files_read, files_modified, prompt_number, discovery_tokens, content_hash, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`);
const result = stmt.run(
@@ -1483,6 +1550,7 @@ export class SessionStore {
JSON.stringify(observation.files_modified),
promptNumber || null,
discoveryTokens,
contentHash,
timestampIso,
timestampEpoch
);
@@ -1593,15 +1661,23 @@ export class SessionStore {
const storeTx = this.db.transaction(() => {
const observationIds: number[] = [];
// 1. Store all observations
// 1. Store all observations (with content-hash deduplication)
const obsStmt = this.db.prepare(`
INSERT INTO observations
(memory_session_id, project, type, title, subtitle, facts, narrative, concepts,
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
files_read, files_modified, prompt_number, discovery_tokens, content_hash, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`);
for (const observation of observations) {
// Content-hash deduplication (same logic as storeObservation singular)
const contentHash = computeObservationContentHash(memorySessionId, observation.title, observation.narrative);
const existing = findDuplicateObservation(this.db, contentHash, timestampEpoch);
if (existing) {
observationIds.push(existing.id);
continue;
}
const result = obsStmt.run(
memorySessionId,
project,
@@ -1615,6 +1691,7 @@ export class SessionStore {
JSON.stringify(observation.files_modified),
promptNumber || null,
discoveryTokens,
contentHash,
timestampIso,
timestampEpoch
);
@@ -1713,15 +1790,23 @@ export class SessionStore {
const storeAndMarkTx = this.db.transaction(() => {
const observationIds: number[] = [];
// 1. Store all observations
// 1. Store all observations (with content-hash deduplication)
const obsStmt = this.db.prepare(`
INSERT INTO observations
(memory_session_id, project, type, title, subtitle, facts, narrative, concepts,
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
files_read, files_modified, prompt_number, discovery_tokens, content_hash, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`);
for (const observation of observations) {
// Content-hash deduplication (same logic as storeObservation singular)
const contentHash = computeObservationContentHash(memorySessionId, observation.title, observation.narrative);
const existing = findDuplicateObservation(this.db, contentHash, timestampEpoch);
if (existing) {
observationIds.push(existing.id);
continue;
}
const result = obsStmt.run(
memorySessionId,
project,
@@ -1735,6 +1820,7 @@ export class SessionStore {
JSON.stringify(observation.files_modified),
promptNumber || null,
discoveryTokens,
contentHash,
timestampIso,
timestampEpoch
);
+10
View File
@@ -372,6 +372,16 @@ export const migration005: Migration = {
export const migration006: Migration = {
version: 6,
up: (db: Database) => {
// FTS5 may be unavailable on some platforms (e.g., Bun on Windows #791).
// Probe before creating tables — search falls back to ChromaDB when unavailable.
try {
db.run('CREATE VIRTUAL TABLE _fts5_probe USING fts5(test_column)');
db.run('DROP TABLE _fts5_probe');
} catch {
console.log('⚠️ FTS5 not available on this platform — skipping FTS migration (search uses ChromaDB)');
return;
}
// FTS5 virtual table for observations
// Note: This assumes the hierarchical fields (title, subtitle, etc.) already exist
// from the inline migrations in SessionStore constructor
+344 -109
View File
@@ -31,11 +31,18 @@ export class MigrationRunner {
this.renameSessionIdColumns();
this.repairSessionIdColumnRename();
this.addFailedAtEpochColumn();
this.addOnUpdateCascadeToForeignKeys();
this.addObservationContentHashColumn();
this.addSessionCustomTitleColumn();
}
/**
* Initialize database schema using migrations (migration004)
* This runs the core SDK tables migration if no tables exist
* Initialize database schema (migration004)
*
* ALWAYS creates core tables using CREATE TABLE IF NOT EXISTS safe to run
* regardless of schema_versions state. This fixes issue #979 where the old
* DatabaseManager migration system (versions 1-7) shared the schema_versions
* table, causing maxApplied > 0 and skipping core table creation entirely.
*/
private initializeSchema(): void {
// Create schema_versions table if it doesn't exist
@@ -47,90 +54,77 @@ export class MigrationRunner {
)
`);
// Get applied migrations
const appliedVersions = this.db.prepare('SELECT version FROM schema_versions ORDER BY version').all() as SchemaVersion[];
const maxApplied = appliedVersions.length > 0 ? Math.max(...appliedVersions.map(v => v.version)) : 0;
// Always create core tables — IF NOT EXISTS makes this idempotent
this.db.run(`
CREATE TABLE IF NOT EXISTS sdk_sessions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
content_session_id TEXT UNIQUE NOT NULL,
memory_session_id TEXT UNIQUE,
project TEXT NOT NULL,
user_prompt TEXT,
started_at TEXT NOT NULL,
started_at_epoch INTEGER NOT NULL,
completed_at TEXT,
completed_at_epoch INTEGER,
status TEXT CHECK(status IN ('active', 'completed', 'failed')) NOT NULL DEFAULT 'active'
);
// Only run migration004 if no migrations have been applied
// This creates the sdk_sessions, observations, and session_summaries tables
if (maxApplied === 0) {
logger.info('DB', 'Initializing fresh database with migration004');
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_claude_id ON sdk_sessions(content_session_id);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_sdk_id ON sdk_sessions(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_project ON sdk_sessions(project);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_status ON sdk_sessions(status);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_started ON sdk_sessions(started_at_epoch DESC);
// Migration004: SDK agent architecture tables
this.db.run(`
CREATE TABLE IF NOT EXISTS sdk_sessions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
content_session_id TEXT UNIQUE NOT NULL,
memory_session_id TEXT UNIQUE,
project TEXT NOT NULL,
user_prompt TEXT,
started_at TEXT NOT NULL,
started_at_epoch INTEGER NOT NULL,
completed_at TEXT,
completed_at_epoch INTEGER,
status TEXT CHECK(status IN ('active', 'completed', 'failed')) NOT NULL DEFAULT 'active'
);
CREATE TABLE IF NOT EXISTS observations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
memory_session_id TEXT NOT NULL,
project TEXT NOT NULL,
text TEXT NOT NULL,
type TEXT NOT NULL,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_claude_id ON sdk_sessions(content_session_id);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_sdk_id ON sdk_sessions(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_project ON sdk_sessions(project);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_status ON sdk_sessions(status);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_started ON sdk_sessions(started_at_epoch DESC);
CREATE INDEX IF NOT EXISTS idx_observations_sdk_session ON observations(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_observations_project ON observations(project);
CREATE INDEX IF NOT EXISTS idx_observations_type ON observations(type);
CREATE INDEX IF NOT EXISTS idx_observations_created ON observations(created_at_epoch DESC);
CREATE TABLE IF NOT EXISTS observations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
memory_session_id TEXT NOT NULL,
project TEXT NOT NULL,
text TEXT NOT NULL,
type TEXT NOT NULL,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE
);
CREATE TABLE IF NOT EXISTS session_summaries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
memory_session_id TEXT UNIQUE NOT NULL,
project TEXT NOT NULL,
request TEXT,
investigated TEXT,
learned TEXT,
completed TEXT,
next_steps TEXT,
files_read TEXT,
files_edited TEXT,
notes TEXT,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_observations_sdk_session ON observations(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_observations_project ON observations(project);
CREATE INDEX IF NOT EXISTS idx_observations_type ON observations(type);
CREATE INDEX IF NOT EXISTS idx_observations_created ON observations(created_at_epoch DESC);
CREATE INDEX IF NOT EXISTS idx_session_summaries_sdk_session ON session_summaries(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_session_summaries_project ON session_summaries(project);
CREATE INDEX IF NOT EXISTS idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
`);
CREATE TABLE IF NOT EXISTS session_summaries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
memory_session_id TEXT UNIQUE NOT NULL,
project TEXT NOT NULL,
request TEXT,
investigated TEXT,
learned TEXT,
completed TEXT,
next_steps TEXT,
files_read TEXT,
files_edited TEXT,
notes TEXT,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_session_summaries_sdk_session ON session_summaries(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_session_summaries_project ON session_summaries(project);
CREATE INDEX IF NOT EXISTS idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
`);
// Record migration004 as applied
this.db.prepare('INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)').run(4, new Date().toISOString());
logger.info('DB', 'Migration004 applied successfully');
}
// Record migration004 as applied (OR IGNORE handles re-runs safely)
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(4, new Date().toISOString());
}
/**
* Ensure worker_port column exists (migration 5)
*
* NOTE: Version 5 conflicts with old DatabaseManager migration005 (which drops orphaned tables).
* We check actual column state rather than relying solely on version tracking.
*/
private ensureWorkerPortColumn(): void {
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(5) as SchemaVersion | undefined;
if (applied) return;
// Check if column exists
// Check actual column existence — don't rely on version tracking alone (issue #979)
const tableInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
const hasWorkerPort = tableInfo.some(col => col.name === 'worker_port');
@@ -145,12 +139,12 @@ export class MigrationRunner {
/**
* Ensure prompt tracking columns exist (migration 6)
*
* NOTE: Version 6 conflicts with old DatabaseManager migration006 (which creates FTS5 tables).
* We check actual column state rather than relying solely on version tracking.
*/
private ensurePromptTrackingColumns(): void {
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(6) as SchemaVersion | undefined;
if (applied) return;
// Check actual column existence — don't rely on version tracking alone (issue #979)
// Check sdk_sessions for prompt_counter
const sessionsInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
const hasPromptCounter = sessionsInfo.some(col => col.name === 'prompt_counter');
@@ -184,13 +178,12 @@ export class MigrationRunner {
/**
* Remove UNIQUE constraint from session_summaries.memory_session_id (migration 7)
*
* NOTE: Version 7 conflicts with old DatabaseManager migration007 (which adds discovery_tokens).
* We check actual constraint state rather than relying solely on version tracking.
*/
private removeSessionSummariesUniqueConstraint(): void {
// Check if migration already applied
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(7) as SchemaVersion | undefined;
if (applied) return;
// Check if UNIQUE constraint exists
// Check actual constraint state — don't rely on version tracking alone (issue #979)
const summariesIndexes = this.db.query('PRAGMA index_list(session_summaries)').all() as IndexInfo[];
const hasUniqueConstraint = summariesIndexes.some(idx => idx.unique === 1);
@@ -205,6 +198,9 @@ export class MigrationRunner {
// Begin transaction
this.db.run('BEGIN TRANSACTION');
// Clean up leftover temp table from a previously-crashed run
this.db.run('DROP TABLE IF EXISTS session_summaries_new');
// Create new table without UNIQUE constraint
this.db.run(`
CREATE TABLE session_summaries_new (
@@ -318,6 +314,9 @@ export class MigrationRunner {
// Begin transaction
this.db.run('BEGIN TRANSACTION');
// Clean up leftover temp table from a previously-crashed run
this.db.run('DROP TABLE IF EXISTS observations_new');
// Create new table with text as nullable
this.db.run(`
CREATE TABLE observations_new (
@@ -411,34 +410,39 @@ export class MigrationRunner {
CREATE INDEX idx_user_prompts_lookup ON user_prompts(content_session_id, prompt_number);
`);
// Create FTS5 virtual table
this.db.run(`
CREATE VIRTUAL TABLE user_prompts_fts USING fts5(
prompt_text,
content='user_prompts',
content_rowid='id'
);
`);
// Create FTS5 virtual table — skip if FTS5 is unavailable (e.g., Bun on Windows #791).
// The user_prompts table itself is still created; only FTS indexing is skipped.
try {
this.db.run(`
CREATE VIRTUAL TABLE user_prompts_fts USING fts5(
prompt_text,
content='user_prompts',
content_rowid='id'
);
`);
// Create triggers to sync FTS5
this.db.run(`
CREATE TRIGGER user_prompts_ai AFTER INSERT ON user_prompts BEGIN
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
// Create triggers to sync FTS5
this.db.run(`
CREATE TRIGGER user_prompts_ai AFTER INSERT ON user_prompts BEGIN
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
CREATE TRIGGER user_prompts_ad AFTER DELETE ON user_prompts BEGIN
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
VALUES('delete', old.id, old.prompt_text);
END;
CREATE TRIGGER user_prompts_ad AFTER DELETE ON user_prompts BEGIN
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
VALUES('delete', old.id, old.prompt_text);
END;
CREATE TRIGGER user_prompts_au AFTER UPDATE ON user_prompts BEGIN
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
VALUES('delete', old.id, old.prompt_text);
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
`);
CREATE TRIGGER user_prompts_au AFTER UPDATE ON user_prompts BEGIN
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
VALUES('delete', old.id, old.prompt_text);
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
`);
} catch (ftsError) {
logger.warn('DB', 'FTS5 not available — user_prompts_fts skipped (search uses ChromaDB)', {}, ftsError as Error);
}
// Commit transaction
this.db.run('COMMIT');
@@ -446,7 +450,7 @@ export class MigrationRunner {
// Record migration
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(10, new Date().toISOString());
logger.debug('DB', 'Successfully created user_prompts table with FTS5 support');
logger.debug('DB', 'Successfully created user_prompts table');
}
/**
@@ -628,4 +632,235 @@ export class MigrationRunner {
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(20, new Date().toISOString());
}
/**
* Add ON UPDATE CASCADE to FK constraints on observations and session_summaries (migration 21)
*
* Both tables have FK(memory_session_id) -> sdk_sessions(memory_session_id) with ON DELETE CASCADE
* but missing ON UPDATE CASCADE. This causes FK constraint violations when code updates
* sdk_sessions.memory_session_id while child rows still reference the old value.
*
* SQLite doesn't support ALTER TABLE for FK changes, so we recreate both tables.
*/
private addOnUpdateCascadeToForeignKeys(): void {
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(21) as SchemaVersion | undefined;
if (applied) return;
logger.debug('DB', 'Adding ON UPDATE CASCADE to FK constraints on observations and session_summaries');
// PRAGMA foreign_keys must be set outside a transaction
this.db.run('PRAGMA foreign_keys = OFF');
this.db.run('BEGIN TRANSACTION');
try {
// ==========================================
// 1. Recreate observations table
// ==========================================
// Drop FTS triggers first (they reference the observations table)
this.db.run('DROP TRIGGER IF EXISTS observations_ai');
this.db.run('DROP TRIGGER IF EXISTS observations_ad');
this.db.run('DROP TRIGGER IF EXISTS observations_au');
// Clean up leftover temp table from a previously-crashed run
this.db.run('DROP TABLE IF EXISTS observations_new');
this.db.run(`
CREATE TABLE observations_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
memory_session_id TEXT NOT NULL,
project TEXT NOT NULL,
text TEXT,
type TEXT NOT NULL,
title TEXT,
subtitle TEXT,
facts TEXT,
narrative TEXT,
concepts TEXT,
files_read TEXT,
files_modified TEXT,
prompt_number INTEGER,
discovery_tokens INTEGER DEFAULT 0,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
)
`);
this.db.run(`
INSERT INTO observations_new
SELECT id, memory_session_id, project, text, type, title, subtitle, facts,
narrative, concepts, files_read, files_modified, prompt_number,
discovery_tokens, created_at, created_at_epoch
FROM observations
`);
this.db.run('DROP TABLE observations');
this.db.run('ALTER TABLE observations_new RENAME TO observations');
// Recreate indexes
this.db.run(`
CREATE INDEX idx_observations_sdk_session ON observations(memory_session_id);
CREATE INDEX idx_observations_project ON observations(project);
CREATE INDEX idx_observations_type ON observations(type);
CREATE INDEX idx_observations_created ON observations(created_at_epoch DESC);
`);
// Recreate FTS triggers only if observations_fts exists
const hasFTS = (this.db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name='observations_fts'").all() as { name: string }[]).length > 0;
if (hasFTS) {
this.db.run(`
CREATE TRIGGER IF NOT EXISTS observations_ai AFTER INSERT ON observations BEGIN
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
END;
CREATE TRIGGER IF NOT EXISTS observations_ad AFTER DELETE ON observations BEGIN
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
END;
CREATE TRIGGER IF NOT EXISTS observations_au AFTER UPDATE ON observations BEGIN
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
END;
`);
}
// ==========================================
// 2. Recreate session_summaries table
// ==========================================
// Clean up leftover temp table from a previously-crashed run
this.db.run('DROP TABLE IF EXISTS session_summaries_new');
this.db.run(`
CREATE TABLE session_summaries_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
memory_session_id TEXT NOT NULL,
project TEXT NOT NULL,
request TEXT,
investigated TEXT,
learned TEXT,
completed TEXT,
next_steps TEXT,
files_read TEXT,
files_edited TEXT,
notes TEXT,
prompt_number INTEGER,
discovery_tokens INTEGER DEFAULT 0,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
)
`);
this.db.run(`
INSERT INTO session_summaries_new
SELECT id, memory_session_id, project, request, investigated, learned,
completed, next_steps, files_read, files_edited, notes,
prompt_number, discovery_tokens, created_at, created_at_epoch
FROM session_summaries
`);
// Drop session_summaries FTS triggers before dropping the table
this.db.run('DROP TRIGGER IF EXISTS session_summaries_ai');
this.db.run('DROP TRIGGER IF EXISTS session_summaries_ad');
this.db.run('DROP TRIGGER IF EXISTS session_summaries_au');
this.db.run('DROP TABLE session_summaries');
this.db.run('ALTER TABLE session_summaries_new RENAME TO session_summaries');
// Recreate indexes
this.db.run(`
CREATE INDEX idx_session_summaries_sdk_session ON session_summaries(memory_session_id);
CREATE INDEX idx_session_summaries_project ON session_summaries(project);
CREATE INDEX idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
`);
// Recreate session_summaries FTS triggers if FTS table exists
const hasSummariesFTS = (this.db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name='session_summaries_fts'").all() as { name: string }[]).length > 0;
if (hasSummariesFTS) {
this.db.run(`
CREATE TRIGGER IF NOT EXISTS session_summaries_ai AFTER INSERT ON session_summaries BEGIN
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
END;
CREATE TRIGGER IF NOT EXISTS session_summaries_ad AFTER DELETE ON session_summaries BEGIN
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
END;
CREATE TRIGGER IF NOT EXISTS session_summaries_au AFTER UPDATE ON session_summaries BEGIN
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
END;
`);
}
// Record migration
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(21, new Date().toISOString());
this.db.run('COMMIT');
this.db.run('PRAGMA foreign_keys = ON');
logger.debug('DB', 'Successfully added ON UPDATE CASCADE to FK constraints');
} catch (error) {
this.db.run('ROLLBACK');
this.db.run('PRAGMA foreign_keys = ON');
throw error;
}
}
/**
* Add content_hash column to observations for deduplication (migration 22)
* Prevents duplicate observations from being stored when the same content is processed multiple times.
* Backfills existing rows with unique random hashes so they don't block new inserts.
*/
private addObservationContentHashColumn(): void {
// Check actual schema first — cross-machine DB sync can leave schema_versions
// claiming this migration ran while the column is actually missing (e.g. migration 21
// recreated the table without content_hash on the synced machine).
const tableInfo = this.db.query('PRAGMA table_info(observations)').all() as TableColumnInfo[];
const hasColumn = tableInfo.some(col => col.name === 'content_hash');
if (hasColumn) {
// Column exists — just ensure version record is present
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(22, new Date().toISOString());
return;
}
this.db.run('ALTER TABLE observations ADD COLUMN content_hash TEXT');
// Backfill existing rows with unique random hashes
this.db.run("UPDATE observations SET content_hash = substr(hex(randomblob(8)), 1, 16) WHERE content_hash IS NULL");
// Index for fast dedup lookups
this.db.run('CREATE INDEX IF NOT EXISTS idx_observations_content_hash ON observations(content_hash, created_at_epoch)');
logger.debug('DB', 'Added content_hash column to observations table with backfill and index');
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(22, new Date().toISOString());
}
/**
* Add custom_title column to sdk_sessions for agent attribution (migration 23)
* Allows callers (e.g. Maestro agents) to label sessions with a human-readable name.
*/
private addSessionCustomTitleColumn(): void {
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(23) as SchemaVersion | undefined;
if (applied) return;
const tableInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
const hasColumn = tableInfo.some(col => col.name === 'custom_title');
if (!hasColumn) {
this.db.run('ALTER TABLE sdk_sessions ADD COLUMN custom_title TEXT');
logger.debug('DB', 'Added custom_title column to sdk_sessions table');
}
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(23, new Date().toISOString());
}
}
+52 -3
View File
@@ -3,13 +3,50 @@
* Extracted from SessionStore.ts for modular organization
*/
import { createHash } from 'crypto';
import { Database } from 'bun:sqlite';
import { logger } from '../../../utils/logger.js';
import { getCurrentProjectName } from '../../../shared/paths.js';
import type { ObservationInput, StoreObservationResult } from './types.js';
/** Deduplication window: observations with the same content hash within this window are skipped */
const DEDUP_WINDOW_MS = 30_000;
/**
* Compute a short content hash for deduplication.
* Uses (memory_session_id, title, narrative) as the semantic identity of an observation.
*/
export function computeObservationContentHash(
memorySessionId: string,
title: string | null,
narrative: string | null
): string {
return createHash('sha256')
.update((memorySessionId || '') + (title || '') + (narrative || ''))
.digest('hex')
.slice(0, 16);
}
/**
* Check if a duplicate observation exists within the dedup window.
* Returns the existing observation's id and timestamp if found, null otherwise.
*/
export function findDuplicateObservation(
db: Database,
contentHash: string,
timestampEpoch: number
): { id: number; created_at_epoch: number } | null {
const windowStart = timestampEpoch - DEDUP_WINDOW_MS;
const stmt = db.prepare(
'SELECT id, created_at_epoch FROM observations WHERE content_hash = ? AND created_at_epoch > ?'
);
return (stmt.get(contentHash, windowStart) as { id: number; created_at_epoch: number } | null);
}
/**
* Store an observation (from SDK parsing)
* Assumes session already exists (created by hook)
* Performs content-hash deduplication: skips INSERT if an identical observation exists within 30s
*/
export function storeObservation(
db: Database,
@@ -24,16 +61,27 @@ export function storeObservation(
const timestampEpoch = overrideTimestampEpoch ?? Date.now();
const timestampIso = new Date(timestampEpoch).toISOString();
// Guard against empty project string (race condition where project isn't set yet)
const resolvedProject = project || getCurrentProjectName();
// Content-hash deduplication
const contentHash = computeObservationContentHash(memorySessionId, observation.title, observation.narrative);
const existing = findDuplicateObservation(db, contentHash, timestampEpoch);
if (existing) {
logger.debug('DEDUP', `Skipped duplicate observation | contentHash=${contentHash} | existingId=${existing.id}`);
return { id: existing.id, createdAtEpoch: existing.created_at_epoch };
}
const stmt = db.prepare(`
INSERT INTO observations
(memory_session_id, project, type, title, subtitle, facts, narrative, concepts,
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
files_read, files_modified, prompt_number, discovery_tokens, content_hash, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`);
const result = stmt.run(
memorySessionId,
project,
resolvedProject,
observation.type,
observation.title,
observation.subtitle,
@@ -44,6 +92,7 @@ export function storeObservation(
JSON.stringify(observation.files_modified),
promptNumber || null,
discoveryTokens,
contentHash,
timestampIso,
timestampEpoch
);
+12 -4
View File
@@ -21,7 +21,8 @@ export function createSDKSession(
db: Database,
contentSessionId: string,
project: string,
userPrompt: string
userPrompt: string,
customTitle?: string
): number {
const now = new Date();
const nowEpoch = now.getTime();
@@ -39,6 +40,13 @@ export function createSDKSession(
WHERE content_session_id = ? AND (project IS NULL OR project = '')
`).run(project, contentSessionId);
}
// Backfill custom_title if provided and not yet set
if (customTitle) {
db.prepare(`
UPDATE sdk_sessions SET custom_title = ?
WHERE content_session_id = ? AND custom_title IS NULL
`).run(customTitle, contentSessionId);
}
return existing.id;
}
@@ -48,9 +56,9 @@ export function createSDKSession(
// must NEVER equal contentSessionId - that would inject memory messages into the user's transcript!
db.prepare(`
INSERT INTO sdk_sessions
(content_session_id, memory_session_id, project, user_prompt, started_at, started_at_epoch, status)
VALUES (?, NULL, ?, ?, ?, ?, 'active')
`).run(contentSessionId, project, userPrompt, now.toISOString(), nowEpoch);
(content_session_id, memory_session_id, project, user_prompt, custom_title, started_at, started_at_epoch, status)
VALUES (?, NULL, ?, ?, ?, ?, ?, 'active')
`).run(contentSessionId, project, userPrompt, customTitle || null, now.toISOString(), nowEpoch);
// Return new ID
const row = db.prepare('SELECT id FROM sdk_sessions WHERE content_session_id = ?')
+2 -2
View File
@@ -17,7 +17,7 @@ import type {
*/
export function getSessionById(db: Database, id: number): SessionBasic | null {
const stmt = db.prepare(`
SELECT id, content_session_id, memory_session_id, project, user_prompt
SELECT id, content_session_id, memory_session_id, project, user_prompt, custom_title
FROM sdk_sessions
WHERE id = ?
LIMIT 1
@@ -38,7 +38,7 @@ export function getSdkSessionsBySessionIds(
const placeholders = memorySessionIds.map(() => '?').join(',');
const stmt = db.prepare(`
SELECT id, content_session_id, memory_session_id, project, user_prompt,
SELECT id, content_session_id, memory_session_id, project, user_prompt, custom_title,
started_at, started_at_epoch, completed_at, completed_at_epoch, status
FROM sdk_sessions
WHERE memory_session_id IN (${placeholders})
+2
View File
@@ -13,6 +13,7 @@ export interface SessionBasic {
memory_session_id: string | null;
project: string;
user_prompt: string;
custom_title: string | null;
}
/**
@@ -24,6 +25,7 @@ export interface SessionFull {
memory_session_id: string;
project: string;
user_prompt: string;
custom_title: string | null;
started_at: string;
started_at_epoch: number;
completed_at: string | null;
+23 -6
View File
@@ -10,6 +10,7 @@ import { Database } from 'bun:sqlite';
import { logger } from '../../utils/logger.js';
import type { ObservationInput } from './observations/types.js';
import type { SummaryInput } from './summaries/types.js';
import { computeObservationContentHash, findDuplicateObservation } from './observations/store.js';
/**
* Result from storeObservations / storeObservationsAndMarkComplete transaction
@@ -63,15 +64,22 @@ export function storeObservationsAndMarkComplete(
const storeAndMarkTx = db.transaction(() => {
const observationIds: number[] = [];
// 1. Store all observations
// 1. Store all observations (with content-hash deduplication)
const obsStmt = db.prepare(`
INSERT INTO observations
(memory_session_id, project, type, title, subtitle, facts, narrative, concepts,
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
files_read, files_modified, prompt_number, discovery_tokens, content_hash, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`);
for (const observation of observations) {
const contentHash = computeObservationContentHash(memorySessionId, observation.title, observation.narrative);
const existing = findDuplicateObservation(db, contentHash, timestampEpoch);
if (existing) {
observationIds.push(existing.id);
continue;
}
const result = obsStmt.run(
memorySessionId,
project,
@@ -85,6 +93,7 @@ export function storeObservationsAndMarkComplete(
JSON.stringify(observation.files_modified),
promptNumber || null,
discoveryTokens,
contentHash,
timestampIso,
timestampEpoch
);
@@ -174,15 +183,22 @@ export function storeObservations(
const storeTx = db.transaction(() => {
const observationIds: number[] = [];
// 1. Store all observations
// 1. Store all observations (with content-hash deduplication)
const obsStmt = db.prepare(`
INSERT INTO observations
(memory_session_id, project, type, title, subtitle, facts, narrative, concepts,
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
files_read, files_modified, prompt_number, discovery_tokens, content_hash, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`);
for (const observation of observations) {
const contentHash = computeObservationContentHash(memorySessionId, observation.title, observation.narrative);
const existing = findDuplicateObservation(db, contentHash, timestampEpoch);
if (existing) {
observationIds.push(existing.id);
continue;
}
const result = obsStmt.run(
memorySessionId,
project,
@@ -196,6 +212,7 @@ export function storeObservations(
JSON.stringify(observation.files_modified),
promptNumber || null,
discoveryTokens,
contentHash,
timestampIso,
timestampEpoch
);
+70 -14
View File
@@ -21,12 +21,15 @@ import fs from 'fs';
import { logger } from '../../utils/logger.js';
import { SettingsDefaultsManager } from '../../shared/SettingsDefaultsManager.js';
import { USER_SETTINGS_PATH } from '../../shared/paths.js';
import { sanitizeEnv } from '../../supervisor/env-sanitizer.js';
import { getSupervisor } from '../../supervisor/index.js';
const CHROMA_MCP_CLIENT_NAME = 'claude-mem-chroma';
const CHROMA_MCP_CLIENT_VERSION = '1.0.0';
const MCP_CONNECTION_TIMEOUT_MS = 30_000;
const RECONNECT_BACKOFF_MS = 10_000; // Don't retry connections faster than this after failure
const DEFAULT_CHROMA_DATA_DIR = path.join(os.homedir(), '.claude-mem', 'chroma');
const CHROMA_SUPERVISOR_ID = 'chroma-mcp';
export class ChromaMcpManager {
private static instance: ChromaMcpManager | null = null;
@@ -101,18 +104,25 @@ export class ChromaMcpManager {
const commandArgs = this.buildCommandArgs();
const spawnEnvironment = this.getSpawnEnv();
getSupervisor().assertCanSpawn('chroma mcp');
// On Windows, .cmd files require shell resolution. Since MCP SDK's
// StdioClientTransport doesn't support `shell: true`, route through
// cmd.exe which resolves .cmd/.bat extensions and PATH automatically.
// This also fixes Git Bash compatibility (#1062) since cmd.exe handles
// Windows-native command resolution regardless of the calling shell.
const isWindows = process.platform === 'win32';
const uvxCommand = isWindows ? 'uvx.cmd' : 'uvx';
const uvxSpawnCommand = isWindows ? (process.env.ComSpec || 'cmd.exe') : 'uvx';
const uvxSpawnArgs = isWindows ? ['/c', 'uvx', ...commandArgs] : commandArgs;
logger.info('CHROMA_MCP', 'Connecting to chroma-mcp via MCP stdio', {
command: uvxCommand,
args: commandArgs.join(' ')
command: uvxSpawnCommand,
args: uvxSpawnArgs.join(' ')
});
this.transport = new StdioClientTransport({
command: uvxCommand,
args: commandArgs,
command: uvxSpawnCommand,
args: uvxSpawnArgs,
env: spawnEnvironment,
stderr: 'pipe'
});
@@ -149,6 +159,7 @@ export class ChromaMcpManager {
clearTimeout(timeoutId!);
this.connected = true;
this.registerManagedProcess();
logger.info('CHROMA_MCP', 'Connected to chroma-mcp successfully');
@@ -163,6 +174,7 @@ export class ChromaMcpManager {
}
logger.warn('CHROMA_MCP', 'chroma-mcp subprocess closed unexpectedly, applying reconnect backoff');
this.connected = false;
getSupervisor().unregisterProcess(CHROMA_SUPERVISOR_ID);
this.client = null;
this.transport = null;
this.lastConnectionFailureTimestamp = Date.now();
@@ -177,6 +189,7 @@ export class ChromaMcpManager {
private buildCommandArgs(): string[] {
const settings = SettingsDefaultsManager.loadFromFile(USER_SETTINGS_PATH);
const chromaMode = settings.CLAUDE_MEM_CHROMA_MODE || 'local';
const pythonVersion = process.env.CLAUDE_MEM_PYTHON_VERSION || settings.CLAUDE_MEM_PYTHON_VERSION || '3.13';
if (chromaMode === 'remote') {
const chromaHost = settings.CLAUDE_MEM_CHROMA_HOST || '127.0.0.1';
@@ -187,15 +200,14 @@ export class ChromaMcpManager {
const chromaApiKey = settings.CLAUDE_MEM_CHROMA_API_KEY || '';
const args = [
'--python', pythonVersion,
'chroma-mcp',
'--client-type', 'http',
'--host', chromaHost,
'--port', chromaPort
];
if (chromaSsl) {
args.push('--ssl');
}
args.push('--ssl', chromaSsl ? 'true' : 'false');
if (chromaTenant !== 'default_tenant') {
args.push('--tenant', chromaTenant);
@@ -214,9 +226,10 @@ export class ChromaMcpManager {
// Local mode: persistent client with data directory
return [
'--python', pythonVersion,
'chroma-mcp',
'--client-type', 'persistent',
'--data-dir', DEFAULT_CHROMA_DATA_DIR
'--data-dir', DEFAULT_CHROMA_DATA_DIR.replace(/\\/g, '/')
];
}
@@ -235,10 +248,35 @@ export class ChromaMcpManager {
arguments: JSON.stringify(toolArguments).slice(0, 200)
});
const result = await this.client!.callTool({
name: toolName,
arguments: toolArguments
});
let result;
try {
result = await this.client!.callTool({
name: toolName,
arguments: toolArguments
});
} catch (transportError) {
// Transport error: chroma-mcp subprocess likely died (e.g., killed by orphan reaper,
// HNSW index corruption). Mark connection dead and retry once after reconnect (#1131).
// Without this retry, callers see a one-shot error even though reconnect would succeed.
this.connected = false;
this.client = null;
this.transport = null;
logger.warn('CHROMA_MCP', `Transport error during "${toolName}", reconnecting and retrying once`, {
error: transportError instanceof Error ? transportError.message : String(transportError)
});
try {
await this.ensureConnected();
result = await this.client!.callTool({
name: toolName,
arguments: toolArguments
});
} catch (retryError) {
this.connected = false;
throw new Error(`chroma-mcp transport error during "${toolName}" (retry failed): ${retryError instanceof Error ? retryError.message : String(retryError)}`);
}
}
// MCP tools signal errors via isError flag on the CallToolResult
if (result.isError) {
@@ -301,6 +339,7 @@ export class ChromaMcpManager {
logger.debug('CHROMA_MCP', 'Error during client close (subprocess may already be dead)', {}, error as Error);
}
getSupervisor().unregisterProcess(CHROMA_SUPERVISOR_ID);
this.client = null;
this.transport = null;
this.connected = false;
@@ -396,7 +435,7 @@ export class ChromaMcpManager {
*/
private getSpawnEnv(): Record<string, string> {
const baseEnv: Record<string, string> = {};
for (const [key, value] of Object.entries(process.env)) {
for (const [key, value] of Object.entries(sanitizeEnv(process.env))) {
if (value !== undefined) {
baseEnv[key] = value;
}
@@ -419,4 +458,21 @@ export class ChromaMcpManager {
NODE_EXTRA_CA_CERTS: combinedCertPath
};
}
private registerManagedProcess(): void {
const chromaProcess = (this.transport as unknown as { _process?: import('child_process').ChildProcess })._process;
if (!chromaProcess?.pid) {
return;
}
getSupervisor().registerProcess(CHROMA_SUPERVISOR_ID, {
pid: chromaProcess.pid,
type: 'chroma',
startedAt: new Date().toISOString()
}, chromaProcess);
chromaProcess.once('exit', () => {
getSupervisor().unregisterProcess(CHROMA_SUPERVISOR_ID);
});
}
}
+22 -6
View File
@@ -267,12 +267,28 @@ export class ChromaSync {
for (let i = 0; i < documents.length; i += this.BATCH_SIZE) {
const batch = documents.slice(i, i + this.BATCH_SIZE);
await chromaMcp.callTool('chroma_add_documents', {
collection_name: this.collectionName,
ids: batch.map(d => d.id),
documents: batch.map(d => d.document),
metadatas: batch.map(d => d.metadata)
});
// Sanitize metadata: filter out null, undefined, and empty string values
// that chroma-mcp may reject (e.g., null subtitle from raw SQLite rows)
const cleanMetadatas = batch.map(d =>
Object.fromEntries(
Object.entries(d.metadata).filter(([_, v]) => v !== null && v !== undefined && v !== '')
)
);
try {
await chromaMcp.callTool('chroma_add_documents', {
collection_name: this.collectionName,
ids: batch.map(d => d.id),
documents: batch.map(d => d.document),
metadatas: cleanMetadatas
});
} catch (error) {
logger.error('CHROMA_SYNC', 'Batch add failed, continuing with remaining batches', {
collection: this.collectionName,
batchStart: i,
batchSize: batch.length
}, error as Error);
}
}
logger.debug('CHROMA_SYNC', 'Documents added', {
+4 -6
View File
@@ -2,7 +2,7 @@ import { sessionInitHandler } from '../../cli/handlers/session-init.js';
import { observationHandler } from '../../cli/handlers/observation.js';
import { fileEditHandler } from '../../cli/handlers/file-edit.js';
import { sessionCompleteHandler } from '../../cli/handlers/session-complete.js';
import { ensureWorkerRunning, getWorkerPort } from '../../shared/worker-utils.js';
import { ensureWorkerRunning, workerHttpRequest } from '../../shared/worker-utils.js';
import { logger } from '../../utils/logger.js';
import { getProjectContext, getProjectName } from '../../utils/project-name.js';
import { writeAgentsMd } from '../../utils/agents-md-utils.js';
@@ -317,11 +317,10 @@ export class TranscriptEventProcessor {
const workerReady = await ensureWorkerRunning();
if (!workerReady) return;
const port = getWorkerPort();
const lastAssistantMessage = session.lastAssistantMessage ?? '';
try {
await fetch(`http://127.0.0.1:${port}/api/sessions/summarize`, {
await workerHttpRequest('/api/sessions/summarize', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
@@ -348,11 +347,10 @@ export class TranscriptEventProcessor {
const context = getProjectContext(cwd);
const projectsParam = context.allProjects.join(',');
const port = getWorkerPort();
try {
const response = await fetch(
`http://127.0.0.1:${port}/api/context/inject?projects=${encodeURIComponent(projectsParam)}`
const response = await workerHttpRequest(
`/api/context/inject?projects=${encodeURIComponent(projectsParam)}`
);
if (!response.ok) return;
+157 -78
View File
@@ -20,6 +20,8 @@ import { getAuthMethodDescription } from '../shared/EnvManager.js';
import { logger } from '../utils/logger.js';
import { ChromaMcpManager } from './sync/ChromaMcpManager.js';
import { ChromaSync } from './sync/ChromaSync.js';
import { configureSupervisorSignalHandlers, getSupervisor, startSupervisor } from '../supervisor/index.js';
import { sanitizeEnv } from '../supervisor/env-sanitizer.js';
// Windows: avoid repeated spawn popups when startup fails (issue #921)
const WINDOWS_SPAWN_COOLDOWN_MS = 2 * 60 * 1000;
@@ -59,6 +61,10 @@ function clearWorkerSpawnAttempted(): void {
}
}
// Re-export for backward compatibility — canonical implementation in shared/plugin-state.ts
export { isPluginDisabledInClaudeSettings } from '../shared/plugin-state.js';
import { isPluginDisabledInClaudeSettings } from '../shared/plugin-state.js';
// Version injected at build time by esbuild define
declare const __DEFAULT_PACKAGE_VERSION__: string;
const packageVersion = typeof __DEFAULT_PACKAGE_VERSION__ !== 'undefined' ? __DEFAULT_PACKAGE_VERSION__ : '0.0.0-dev';
@@ -74,7 +80,8 @@ import {
cleanStalePidFile,
isProcessAlive,
spawnDaemon,
createSignalHandler
isPidFileRecent,
touchPidFile
} from './infrastructure/ProcessManager.js';
import {
isPortInUse,
@@ -257,33 +264,10 @@ export class WorkerService {
* Register signal handlers for graceful shutdown
*/
private registerSignalHandlers(): void {
const shutdownRef = { value: this.isShuttingDown };
const handler = createSignalHandler(() => this.shutdown(), shutdownRef);
process.on('SIGTERM', () => {
this.isShuttingDown = shutdownRef.value;
handler('SIGTERM');
configureSupervisorSignalHandlers(async () => {
this.isShuttingDown = true;
await this.shutdown();
});
process.on('SIGINT', () => {
this.isShuttingDown = shutdownRef.value;
handler('SIGINT');
});
// SIGHUP: sent by kernel when controlling terminal closes.
// Daemon mode: ignore it (survive parent shell exit).
// Interactive mode: treat like SIGTERM (graceful shutdown).
if (process.platform !== 'win32') {
if (process.argv.includes('--daemon')) {
process.on('SIGHUP', () => {
logger.debug('SYSTEM', 'Ignoring SIGHUP in daemon mode');
});
} else {
process.on('SIGHUP', () => {
this.isShuttingDown = shutdownRef.value;
handler('SIGHUP');
});
}
}
}
/**
@@ -345,7 +329,9 @@ export class WorkerService {
const port = getWorkerPort();
const host = getWorkerHost();
// Start HTTP server FIRST - make port available immediately
await startSupervisor();
// Start HTTP server FIRST - make it available immediately
await this.server.listen(port, host);
// Worker writes its own PID - reliable on all platforms
@@ -357,6 +343,12 @@ export class WorkerService {
startedAt: new Date().toISOString()
});
getSupervisor().registerProcess('worker', {
pid: process.pid,
type: 'worker',
startedAt: new Date().toISOString()
});
logger.info('SYSTEM', 'Worker started', { host, port, pid: process.pid });
// Do slow initialization in background (non-blocking)
@@ -385,9 +377,14 @@ export class WorkerService {
runOneTimeChromaMigration();
}
// Initialize ChromaMcpManager (lazy - connects on first use via ChromaSync)
this.chromaMcpManager = ChromaMcpManager.getInstance();
logger.info('SYSTEM', 'ChromaMcpManager initialized (lazy - connects on first use)');
// Initialize ChromaMcpManager only if Chroma is enabled
const chromaEnabled = settings.CLAUDE_MEM_CHROMA_ENABLED !== 'false';
if (chromaEnabled) {
this.chromaMcpManager = ChromaMcpManager.getInstance();
logger.info('SYSTEM', 'ChromaMcpManager initialized (lazy - connects on first use)');
} else {
logger.info('SYSTEM', 'Chroma disabled via CLAUDE_MEM_CHROMA_ENABLED=false, skipping ChromaMcpManager');
}
const modeId = settings.CLAUDE_MEM_MODE;
ModeManager.getInstance().loadMode(modeId);
@@ -435,19 +432,50 @@ export class WorkerService {
// Connect to MCP server
const mcpServerPath = path.join(__dirname, 'mcp-server.cjs');
getSupervisor().assertCanSpawn('mcp server');
const transport = new StdioClientTransport({
command: 'node',
args: [mcpServerPath],
env: process.env
env: sanitizeEnv(process.env)
});
const MCP_INIT_TIMEOUT_MS = 300000;
const mcpConnectionPromise = this.mcpClient.connect(transport);
const timeoutPromise = new Promise<never>((_, reject) =>
setTimeout(() => reject(new Error('MCP connection timeout after 5 minutes')), MCP_INIT_TIMEOUT_MS)
);
let timeoutId: ReturnType<typeof setTimeout>;
const timeoutPromise = new Promise<never>((_, reject) => {
timeoutId = setTimeout(
() => reject(new Error('MCP connection timeout after 5 minutes')),
MCP_INIT_TIMEOUT_MS
);
});
await Promise.race([mcpConnectionPromise, timeoutPromise]);
try {
await Promise.race([mcpConnectionPromise, timeoutPromise]);
} catch (connectionError) {
clearTimeout(timeoutId!);
logger.warn('WORKER', 'MCP server connection failed, cleaning up subprocess', {
error: connectionError instanceof Error ? connectionError.message : String(connectionError)
});
try {
await transport.close();
} catch {
// Best effort: the supervisor handles later process cleanup for survivors.
}
throw connectionError;
}
clearTimeout(timeoutId!);
const mcpProcess = (transport as unknown as { _process?: import('child_process').ChildProcess })._process;
if (mcpProcess?.pid) {
getSupervisor().registerProcess('mcp-server', {
pid: mcpProcess.pid,
type: 'mcp',
startedAt: new Date().toISOString()
}, mcpProcess);
mcpProcess.once('exit', () => {
getSupervisor().unregisterProcess('mcp-server');
});
}
this.mcpReady = true;
logger.success('WORKER', 'MCP server connected');
@@ -459,7 +487,7 @@ export class WorkerService {
}
return activeIds;
});
logger.info('SYSTEM', 'Started orphan reaper (runs every 5 minutes)');
logger.info('SYSTEM', 'Started orphan reaper (runs every 30 seconds)');
// Reap stale sessions to unblock orphan process cleanup (Issue #1168)
this.staleSessionReaperInterval = setInterval(async () => {
@@ -535,6 +563,9 @@ export class WorkerService {
logger.info('SYSTEM', `Starting generator (${source}) using ${providerName}`, { sessionId: sid });
// Track generator activity for stale detection (Issue #1099)
session.lastGeneratorActivity = Date.now();
session.generatorPromise = agent.startSession(session, this)
.catch(async (error: unknown) => {
const errorMessage = (error as Error)?.message || '';
@@ -547,6 +578,7 @@ export class WorkerService {
'ENOENT',
'spawn',
'Invalid API key',
'FOREIGN KEY constraint failed',
];
if (unrecoverablePatterns.some(pattern => errorMessage.includes(pattern))) {
hadUnrecoverableError = true;
@@ -604,7 +636,7 @@ export class WorkerService {
.finally(async () => {
// CRITICAL: Verify subprocess exit to prevent zombie accumulation (Issue #1168)
const trackedProcess = getProcessBySession(session.sessionDbId);
if (trackedProcess && !trackedProcess.process.killed && trackedProcess.process.exitCode === null) {
if (trackedProcess && trackedProcess.process.exitCode === null) {
await ensureProcessExit(trackedProcess, 5000);
}
@@ -645,16 +677,35 @@ export class WorkerService {
// Check if there's pending work that needs processing with a fresh AbortController
const pendingCount = pendingStore.getPendingCount(session.sessionDbId);
const MAX_PENDING_RESTARTS = 3;
if (pendingCount > 0) {
// Track consecutive pending-work restarts to prevent infinite loops (e.g. FK errors)
session.consecutiveRestarts = (session.consecutiveRestarts || 0) + 1;
if (session.consecutiveRestarts > MAX_PENDING_RESTARTS) {
logger.error('SYSTEM', 'Exceeded max pending-work restarts, stopping to prevent infinite loop', {
sessionId: session.sessionDbId,
pendingCount,
consecutiveRestarts: session.consecutiveRestarts
});
session.consecutiveRestarts = 0;
this.broadcastProcessingStatus();
return;
}
logger.info('SYSTEM', 'Pending work remains after generator exit, restarting with fresh AbortController', {
sessionId: session.sessionDbId,
pendingCount
pendingCount,
attempt: session.consecutiveRestarts
});
// Reset AbortController for restart
session.abortController = new AbortController();
// Restart processor
this.startSessionProcessor(session, 'pending-work-restart');
} else {
// Successful completion with no pending work — reset counter
session.consecutiveRestarts = 0;
}
this.broadcastProcessingStatus();
@@ -882,17 +933,44 @@ export class WorkerService {
* Ensures the worker is started and healthy.
* This function can be called by both 'start' and 'hook' commands.
*
* @param port - The port the worker should run on
* @param port - The TCP port (used for port-in-use checks and daemon spawn)
* @returns true if worker is healthy (existing or newly started), false on failure
*/
async function ensureWorkerStarted(port: number): Promise<boolean> {
// Clean stale PID file first (cheap: 1 fs read + 1 signal-0 check)
cleanStalePidFile();
const pidFileStatus = cleanStalePidFile();
if (pidFileStatus === 'alive') {
logger.info('SYSTEM', 'Worker PID file points to a live process, skipping duplicate spawn');
const healthy = await waitForHealth(port, getPlatformTimeout(HOOK_TIMEOUTS.PORT_IN_USE_WAIT));
if (healthy) {
logger.info('SYSTEM', 'Worker became healthy while waiting on live PID');
return true;
}
logger.warn('SYSTEM', 'Live PID detected but worker did not become healthy before timeout');
return false;
}
// Check if worker is already running and healthy
if (await waitForHealth(port, 1000)) {
const versionCheck = await checkVersionMatch(port);
if (!versionCheck.matches) {
// Guard: If PID file was written recently, another session is likely already
// restarting the worker. Poll health instead of starting a concurrent restart.
// This prevents the "100 sessions all restart simultaneously" storm (#1145).
const RESTART_COORDINATION_THRESHOLD_MS = 15000;
if (isPidFileRecent(RESTART_COORDINATION_THRESHOLD_MS)) {
logger.info('SYSTEM', 'Version mismatch detected but PID file is recent — another restart likely in progress, polling health', {
pluginVersion: versionCheck.pluginVersion,
workerVersion: versionCheck.workerVersion
});
const healthy = await waitForHealth(port, RESTART_COORDINATION_THRESHOLD_MS);
if (healthy) {
logger.info('SYSTEM', 'Worker became healthy after waiting for concurrent restart');
return true;
}
logger.warn('SYSTEM', 'Worker did not become healthy after waiting — proceeding with own restart');
}
logger.info('SYSTEM', 'Worker version mismatch detected - auto-restarting', {
pluginVersion: versionCheck.pluginVersion,
workerVersion: versionCheck.workerVersion
@@ -957,6 +1035,9 @@ async function ensureWorkerStarted(port: number): Promise<boolean> {
}
clearWorkerSpawnAttempted();
// Touch PID file to signal other sessions that a restart just completed.
// Other sessions checking isPidFileRecent() will see this and skip their own restart.
touchPidFile();
logger.info('SYSTEM', 'Worker started successfully');
return true;
}
@@ -967,6 +1048,14 @@ async function ensureWorkerStarted(port: number): Promise<boolean> {
async function main() {
const command = process.argv[2];
// Early exit if plugin is disabled in Claude Code settings (#781).
// Only gate hook-initiated commands; CLI management (stop/status) still works.
const hookInitiatedCommands = ['start', 'hook', 'restart', '--daemon'];
if ((hookInitiatedCommands.includes(command) || command === undefined) && isPluginDisabledInClaudeSettings()) {
process.exit(0);
}
const port = getWorkerPort();
// Helper for JSON status output in 'start' command
@@ -985,6 +1074,7 @@ async function main() {
} else {
exitWithStatus('error', 'Failed to start worker');
}
break;
}
case 'stop': {
@@ -996,16 +1086,15 @@ async function main() {
removePidFile();
logger.info('SYSTEM', 'Worker stopped successfully');
process.exit(0);
break;
}
case 'restart': {
logger.info('SYSTEM', 'Restarting worker');
await httpShutdown(port);
const freed = await waitForPortFree(port, getPlatformTimeout(15000));
if (!freed) {
const restartFreed = await waitForPortFree(port, getPlatformTimeout(15000));
if (!restartFreed) {
logger.error('SYSTEM', 'Port did not free up after shutdown, aborting restart', { port });
// Exit gracefully: Windows Terminal won't keep tab open on exit 0
// The wrapper/plugin will handle restart logic if needed
process.exit(0);
}
removePidFile();
@@ -1032,12 +1121,13 @@ async function main() {
logger.info('SYSTEM', 'Worker restarted successfully');
process.exit(0);
break;
}
case 'status': {
const running = await isPortInUse(port);
const portInUse = await isPortInUse(port);
const pidInfo = readPidFile();
if (running && pidInfo) {
if (portInUse && pidInfo) {
console.log('Worker is running');
console.log(` PID: ${pidInfo.pid}`);
console.log(` Port: ${pidInfo.port}`);
@@ -1046,22 +1136,18 @@ async function main() {
console.log('Worker is not running');
}
process.exit(0);
break;
}
case 'cursor': {
const subcommand = process.argv[3];
const cursorResult = await handleCursorCommand(subcommand, process.argv.slice(4));
process.exit(cursorResult);
break;
}
case 'hook': {
// Auto-start worker if not running
const workerReady = await ensureWorkerStarted(port);
if (!workerReady) {
logger.warn('SYSTEM', 'Worker failed to start before hook, handler will retry');
}
// Existing logic unchanged
// Validate CLI args first (before any I/O)
const platform = process.argv[3];
const event = process.argv[4];
if (!platform || !event) {
@@ -1071,32 +1157,20 @@ async function main() {
process.exit(1);
}
// Check if worker is already running on port
const portInUse = await isPortInUse(port);
let startedWorkerInProcess = false;
if (!portInUse) {
// Port free - start worker IN THIS PROCESS (no spawn!)
// This process becomes the worker and stays alive
try {
logger.info('SYSTEM', 'Starting worker in-process for hook', { event });
const worker = new WorkerService();
await worker.start();
startedWorkerInProcess = true;
// Worker is now running in this process on the port
} catch (error) {
logger.failure('SYSTEM', 'Worker failed to start in hook', {}, error as Error);
removePidFile();
process.exit(0);
}
// Ensure worker is running as a detached daemon (#1249).
//
// IMPORTANT: The hook process MUST NOT become the worker. Starting the
// worker in-process makes it a grandchild of Claude Code, which the
// sandbox kills. Instead, ensureWorkerStarted() spawns a fully detached
// daemon (detached: true, stdio: 'ignore', child.unref()) that survives
// the hook process's exit and is invisible to Claude Code's sandbox.
const workerReady = await ensureWorkerStarted(port);
if (!workerReady) {
logger.warn('SYSTEM', 'Worker failed to start before hook, handler will proceed gracefully');
}
// If port in use, we'll use HTTP to the existing worker
const { hookCommand } = await import('../cli/hook-command.js');
// If we started the worker in this process, skip process.exit() so we stay alive as the worker
await hookCommand(platform, event, { skipExit: startedWorkerInProcess });
// Note: if we started worker in-process, this process stays alive as the worker
// The break allows the event loop to continue serving requests
await hookCommand(platform, event);
break;
}
@@ -1105,6 +1179,7 @@ async function main() {
const { generateClaudeMd } = await import('../cli/claude-md-commands.js');
const result = await generateClaudeMd(dryRun);
process.exit(result);
break;
}
case 'clean': {
@@ -1112,6 +1187,7 @@ async function main() {
const { cleanClaudeMd } = await import('../cli/claude-md-commands.js');
const result = await cleanClaudeMd(dryRun);
process.exit(result);
break;
}
case '--daemon':
@@ -1168,5 +1244,8 @@ const isMainModule = typeof require !== 'undefined' && typeof module !== 'undefi
: import.meta.url === `file://${process.argv[1]}` || process.argv[1]?.endsWith('worker-service');
if (isMainModule) {
main();
main().catch((error) => {
logger.error('SYSTEM', 'Fatal error in main', {}, error instanceof Error ? error : undefined);
process.exit(0); // Exit 0: don't block Claude Code, don't leave Windows Terminal tabs open
});
}
+1
View File
@@ -36,6 +36,7 @@ export interface ActiveSession {
consecutiveRestarts: number; // Track consecutive restart attempts to prevent infinite loops
forceInit?: boolean; // Force fresh SDK session (skip resume)
idleTimedOut?: boolean; // Set when session exits due to idle timeout (prevents restart loop)
lastGeneratorActivity: number; // Timestamp of last generator progress (for stale detection, Issue #1099)
// CLAIM-CONFIRM FIX: Track IDs of messages currently being processed
// These IDs will be confirmed (deleted) after successful storage
processingMessageIds: number[];
+12 -7
View File
@@ -11,6 +11,8 @@
import { SessionStore } from '../sqlite/SessionStore.js';
import { SessionSearch } from '../sqlite/SessionSearch.js';
import { ChromaSync } from '../sync/ChromaSync.js';
import { SettingsDefaultsManager } from '../../shared/SettingsDefaultsManager.js';
import { USER_SETTINGS_PATH } from '../../shared/paths.js';
import { logger } from '../../utils/logger.js';
import type { DBSession } from '../worker-types.js';
@@ -27,8 +29,14 @@ export class DatabaseManager {
this.sessionStore = new SessionStore();
this.sessionSearch = new SessionSearch();
// Initialize ChromaSync (lazy - connects on first search, not at startup)
this.chromaSync = new ChromaSync('claude-mem');
// Initialize ChromaSync only if Chroma is enabled (SQLite-only fallback when disabled)
const settings = SettingsDefaultsManager.loadFromFile(USER_SETTINGS_PATH);
const chromaEnabled = settings.CLAUDE_MEM_CHROMA_ENABLED !== 'false';
if (chromaEnabled) {
this.chromaSync = new ChromaSync('claude-mem');
} else {
logger.info('DB', 'Chroma disabled via CLAUDE_MEM_CHROMA_ENABLED=false, using SQLite-only search');
}
logger.info('DB', 'Database initialized');
}
@@ -75,12 +83,9 @@ export class DatabaseManager {
}
/**
* Get ChromaSync instance (throws if not initialized)
* Get ChromaSync instance (returns null if Chroma is disabled)
*/
getChromaSync(): ChromaSync {
if (!this.chromaSync) {
throw new Error('ChromaSync not initialized');
}
getChromaSync(): ChromaSync | null {
return this.chromaSync;
}
+82 -30
View File
@@ -19,6 +19,8 @@
import { spawn, exec, ChildProcess } from 'child_process';
import { promisify } from 'util';
import { logger } from '../../utils/logger.js';
import { sanitizeEnv } from '../../supervisor/env-sanitizer.js';
import { getSupervisor } from '../../supervisor/index.js';
const execAsync = promisify(exec);
@@ -29,14 +31,36 @@ interface TrackedProcess {
process: ChildProcess;
}
// PID Registry - tracks spawned Claude subprocesses
const processRegistry = new Map<number, TrackedProcess>();
function getTrackedProcesses(): TrackedProcess[] {
return getSupervisor().getRegistry()
.getAll()
.filter(record => record.type === 'sdk')
.map((record) => {
const processRef = getSupervisor().getRegistry().getRuntimeProcess(record.id);
if (!processRef) {
return null;
}
return {
pid: record.pid,
sessionDbId: Number(record.sessionId),
spawnedAt: Date.parse(record.startedAt),
process: processRef
};
})
.filter((value): value is TrackedProcess => value !== null);
}
/**
* Register a spawned process in the registry
*/
export function registerProcess(pid: number, sessionDbId: number, process: ChildProcess): void {
processRegistry.set(pid, { pid, sessionDbId, spawnedAt: Date.now(), process });
getSupervisor().registerProcess(`sdk:${sessionDbId}:${pid}`, {
pid,
type: 'sdk',
sessionId: sessionDbId,
startedAt: new Date().toISOString()
}, process);
logger.info('PROCESS', `Registered PID ${pid} for session ${sessionDbId}`, { pid, sessionDbId });
}
@@ -44,7 +68,11 @@ export function registerProcess(pid: number, sessionDbId: number, process: Child
* Unregister a process from the registry and notify pool waiters
*/
export function unregisterProcess(pid: number): void {
processRegistry.delete(pid);
for (const record of getSupervisor().getRegistry().getByPid(pid)) {
if (record.type === 'sdk') {
getSupervisor().unregisterProcess(record.id);
}
}
logger.debug('PROCESS', `Unregistered PID ${pid}`, { pid });
// Notify waiters that a pool slot may be available
notifySlotAvailable();
@@ -55,10 +83,7 @@ export function unregisterProcess(pid: number): void {
* Warns if multiple processes found (indicates race condition)
*/
export function getProcessBySession(sessionDbId: number): TrackedProcess | undefined {
const matches: TrackedProcess[] = [];
for (const [, info] of processRegistry) {
if (info.sessionDbId === sessionDbId) matches.push(info);
}
const matches = getTrackedProcesses().filter(info => info.sessionDbId === sessionDbId);
if (matches.length > 1) {
logger.warn('PROCESS', `Multiple processes found for session ${sessionDbId}`, {
count: matches.length,
@@ -72,7 +97,7 @@ export function getProcessBySession(sessionDbId: number): TrackedProcess | undef
* Get count of active processes in the registry
*/
export function getActiveCount(): number {
return processRegistry.size;
return getSupervisor().getRegistry().getAll().filter(record => record.type === 'sdk').length;
}
// Waiters for pool slots - resolved when a process exits and frees a slot
@@ -91,10 +116,18 @@ function notifySlotAvailable(): void {
* @param maxConcurrent Max number of concurrent agents
* @param timeoutMs Max time to wait before giving up
*/
export async function waitForSlot(maxConcurrent: number, timeoutMs: number = 60_000): Promise<void> {
if (processRegistry.size < maxConcurrent) return;
const TOTAL_PROCESS_HARD_CAP = 10;
logger.info('PROCESS', `Pool limit reached (${processRegistry.size}/${maxConcurrent}), waiting for slot...`);
export async function waitForSlot(maxConcurrent: number, timeoutMs: number = 60_000): Promise<void> {
// Hard cap: refuse to spawn if too many processes exist regardless of pool accounting
const activeCount = getActiveCount();
if (activeCount >= TOTAL_PROCESS_HARD_CAP) {
throw new Error(`Hard cap exceeded: ${activeCount} processes in registry (cap=${TOTAL_PROCESS_HARD_CAP}). Refusing to spawn more.`);
}
if (activeCount < maxConcurrent) return;
logger.info('PROCESS', `Pool limit reached (${activeCount}/${maxConcurrent}), waiting for slot...`);
return new Promise<void>((resolve, reject) => {
const timeout = setTimeout(() => {
@@ -105,7 +138,7 @@ export async function waitForSlot(maxConcurrent: number, timeoutMs: number = 60_
const onSlot = () => {
clearTimeout(timeout);
if (processRegistry.size < maxConcurrent) {
if (getActiveCount() < maxConcurrent) {
resolve();
} else {
// Still full, re-queue
@@ -122,7 +155,7 @@ export async function waitForSlot(maxConcurrent: number, timeoutMs: number = 60_
*/
export function getActiveProcesses(): Array<{ pid: number; sessionDbId: number; ageMs: number }> {
const now = Date.now();
return Array.from(processRegistry.values()).map(info => ({
return getTrackedProcesses().map(info => ({
pid: info.pid,
sessionDbId: info.sessionDbId,
ageMs: now - info.spawnedAt
@@ -136,8 +169,9 @@ export function getActiveProcesses(): Array<{ pid: number; sessionDbId: number;
export async function ensureProcessExit(tracked: TrackedProcess, timeoutMs: number = 5000): Promise<void> {
const { pid, process: proc } = tracked;
// Already exited?
if (proc.killed || proc.exitCode !== null) {
// Already exited? Only trust exitCode, NOT proc.killed
// proc.killed only means Node sent a signal — the process can still be alive
if (proc.exitCode !== null) {
unregisterProcess(pid);
return;
}
@@ -153,8 +187,8 @@ export async function ensureProcessExit(tracked: TrackedProcess, timeoutMs: numb
await Promise.race([exitPromise, timeoutPromise]);
// Check if exited gracefully
if (proc.killed || proc.exitCode !== null) {
// Check if exited gracefully — only trust exitCode
if (proc.exitCode !== null) {
unregisterProcess(pid);
return;
}
@@ -167,8 +201,14 @@ export async function ensureProcessExit(tracked: TrackedProcess, timeoutMs: numb
// Already dead
}
// Brief wait for SIGKILL to take effect
await new Promise(resolve => setTimeout(resolve, 200));
// Wait for SIGKILL to take effect — use exit event with 1s timeout instead of blind sleep
const sigkillExitPromise = new Promise<void>((resolve) => {
proc.once('exit', () => resolve());
});
const sigkillTimeout = new Promise<void>((resolve) => {
setTimeout(resolve, 1000);
});
await Promise.race([sigkillExitPromise, sigkillTimeout]);
unregisterProcess(pid);
}
@@ -234,8 +274,8 @@ async function killIdleDaemonChildren(): Promise<number> {
minutes = parseInt(minMatch[1], 10);
}
// Kill if idle for more than 2 minutes
if (minutes >= 2) {
// Kill if idle for more than 1 minute
if (minutes >= 1) {
logger.info('PROCESS', `Killing idle daemon child PID ${pid} (idle ${minutes}m)`, { pid, minutes });
try {
process.kill(pid, 'SIGKILL');
@@ -294,17 +334,26 @@ export async function reapOrphanedProcesses(activeSessionIds: Set<number>): Prom
let killed = 0;
// Registry-based: kill processes for dead sessions
for (const [pid, info] of processRegistry) {
if (activeSessionIds.has(info.sessionDbId)) continue; // Active = safe
for (const record of getSupervisor().getRegistry().getAll().filter(entry => entry.type === 'sdk')) {
const pid = record.pid;
const sessionDbId = Number(record.sessionId);
const processRef = getSupervisor().getRegistry().getRuntimeProcess(record.id);
logger.warn('PROCESS', `Killing orphan PID ${pid} (session ${info.sessionDbId} gone)`, { pid, sessionDbId: info.sessionDbId });
if (activeSessionIds.has(sessionDbId)) continue; // Active = safe
logger.warn('PROCESS', `Killing orphan PID ${pid} (session ${sessionDbId} gone)`, { pid, sessionDbId });
try {
info.process.kill('SIGKILL');
if (processRef) {
processRef.kill('SIGKILL');
} else {
process.kill(pid, 'SIGKILL');
}
killed++;
} catch {
// Already dead
}
unregisterProcess(pid);
getSupervisor().unregisterProcess(record.id);
notifySlotAvailable();
}
// System-level: find ppid=1 orphans
@@ -333,20 +382,23 @@ export function createPidCapturingSpawn(sessionDbId: number) {
env?: NodeJS.ProcessEnv;
signal?: AbortSignal;
}) => {
getSupervisor().assertCanSpawn('claude sdk');
// On Windows, use cmd.exe wrapper for .cmd files to properly handle paths with spaces
const useCmdWrapper = process.platform === 'win32' && spawnOptions.command.endsWith('.cmd');
const env = sanitizeEnv(spawnOptions.env ?? process.env);
const child = useCmdWrapper
? spawn('cmd.exe', ['/d', '/c', spawnOptions.command, ...spawnOptions.args], {
cwd: spawnOptions.cwd,
env: spawnOptions.env,
env,
stdio: ['pipe', 'pipe', 'pipe'],
signal: spawnOptions.signal,
windowsHide: true
})
: spawn(spawnOptions.command, spawnOptions.args, {
cwd: spawnOptions.cwd,
env: spawnOptions.env,
env,
stdio: ['pipe', 'pipe', 'pipe'],
signal: spawnOptions.signal, // CRITICAL: Pass signal for AbortController integration
windowsHide: true
@@ -393,7 +445,7 @@ export function createPidCapturingSpawn(sessionDbId: number) {
* Start the orphan reaper interval
* Returns cleanup function to stop the interval
*/
export function startOrphanReaper(getActiveSessionIds: () => Set<number>, intervalMs: number = 5 * 60 * 1000): () => void {
export function startOrphanReaper(getActiveSessionIds: () => Set<number>, intervalMs: number = 30 * 1000): () => void {
const interval = setInterval(async () => {
try {
const activeIds = getActiveSessionIds();
+3 -2
View File
@@ -22,6 +22,7 @@ import type { ActiveSession, SDKUserMessage } from '../worker-types.js';
import { ModeManager } from '../domain/ModeManager.js';
import { processAgentResponse, type WorkerRef } from './agents/index.js';
import { createPidCapturingSpawn, getProcessBySession, ensureProcessExit, waitForSlot } from './ProcessRegistry.js';
import { sanitizeEnv } from '../../supervisor/env-sanitizer.js';
// Import Agent SDK (assumes it's installed)
// @ts-ignore - Agent SDK types may not be available
@@ -96,7 +97,7 @@ export class SDKAgent {
// Build isolated environment from ~/.claude-mem/.env
// This prevents Issue #733: random ANTHROPIC_API_KEY from project .env files
// being used instead of the configured auth method (CLI subscription or explicit API key)
const isolatedEnv = buildIsolatedEnv();
const isolatedEnv = sanitizeEnv(buildIsolatedEnv());
const authMethod = getAuthMethodDescription();
logger.info('SDK', 'Starting SDK query', {
@@ -281,7 +282,7 @@ export class SDKAgent {
} finally {
// Ensure subprocess is terminated after query completes (or on error)
const tracked = getProcessBySession(session.sessionDbId);
if (tracked && !tracked.process.killed && tracked.process.exitCode === null) {
if (tracked && tracked.process.exitCode === null) {
await ensureProcessExit(tracked, 5000);
}
}
+30 -5
View File
@@ -39,7 +39,7 @@ export class SearchManager {
constructor(
private sessionSearch: SessionSearch,
private sessionStore: SessionStore,
private chromaSync: ChromaSync,
private chromaSync: ChromaSync | null,
private formatter: FormattingService,
private timelineService: TimelineService
) {
@@ -61,6 +61,9 @@ export class SearchManager {
limit: number,
whereFilter?: Record<string, any>
): Promise<{ ids: number[]; distances: number[]; metadatas: any[] }> {
if (!this.chromaSync) {
return { ids: [], distances: [], metadatas: [] };
}
return await this.chromaSync.queryChroma(query, limit, whereFilter);
}
@@ -180,15 +183,37 @@ export class SearchManager {
logger.debug('SEARCH', 'ChromaDB returned semantic matches', { matchCount: chromaResults.ids.length });
if (chromaResults.ids.length > 0) {
// Step 2: Filter by recency (90 days)
const ninetyDaysAgo = Date.now() - SEARCH_CONSTANTS.RECENCY_WINDOW_MS;
// Step 2: Filter by date range
// Use user-provided dateRange if available, otherwise fall back to 90-day recency window
const { dateRange } = options;
let startEpoch: number | undefined;
let endEpoch: number | undefined;
if (dateRange) {
if (dateRange.start) {
startEpoch = typeof dateRange.start === 'number'
? dateRange.start
: new Date(dateRange.start).getTime();
}
if (dateRange.end) {
endEpoch = typeof dateRange.end === 'number'
? dateRange.end
: new Date(dateRange.end).getTime();
}
} else {
// Default: 90-day recency window
startEpoch = Date.now() - SEARCH_CONSTANTS.RECENCY_WINDOW_MS;
}
const recentMetadata = chromaResults.metadatas.map((meta, idx) => ({
id: chromaResults.ids[idx],
meta,
isRecent: meta && meta.created_at_epoch > ninetyDaysAgo
isRecent: meta && meta.created_at_epoch != null
&& (!startEpoch || meta.created_at_epoch >= startEpoch)
&& (!endEpoch || meta.created_at_epoch <= endEpoch)
})).filter(item => item.isRecent);
logger.debug('SEARCH', 'Results within 90-day window', { count: recentMetadata.length });
logger.debug('SEARCH', dateRange ? 'Results within user date range' : 'Results within 90-day window', { count: recentMetadata.length });
// Step 3: Categorize IDs by document type
const obsIds: number[] = [];
+26 -4
View File
@@ -15,6 +15,7 @@ import type { ActiveSession, PendingMessage, PendingMessageWithId, ObservationDa
import { PendingMessageStore } from '../sqlite/PendingMessageStore.js';
import { SessionQueueProcessor } from '../queue/SessionQueueProcessor.js';
import { getProcessBySession, ensureProcessExit } from './ProcessRegistry.js';
import { getSupervisor } from '../../supervisor/index.js';
export class SessionManager {
private dbManager: DatabaseManager;
@@ -155,7 +156,8 @@ export class SessionManager {
conversationHistory: [], // Initialize empty - will be populated by agents
currentProvider: null, // Will be set when generator starts
consecutiveRestarts: 0, // Track consecutive restart attempts to prevent infinite loops
processingMessageIds: [] // CLAIM-CONFIRM: Track message IDs for confirmProcessed()
processingMessageIds: [], // CLAIM-CONFIRM: Track message IDs for confirmProcessed()
lastGeneratorActivity: Date.now() // Initialize for stale detection (Issue #1099)
};
logger.debug('SESSION', 'Creating new session object (memorySessionId cleared to prevent stale resume)', {
@@ -286,16 +288,22 @@ export class SessionManager {
// 1. Abort the SDK agent
session.abortController.abort();
// 2. Wait for generator to finish
// 2. Wait for generator to finish (with 30s timeout to prevent stale stall, Issue #1099)
if (session.generatorPromise) {
await session.generatorPromise.catch(() => {
const generatorDone = session.generatorPromise.catch(() => {
logger.debug('SYSTEM', 'Generator already failed, cleaning up', { sessionId: session.sessionDbId });
});
const timeoutDone = new Promise<void>(resolve => {
AbortSignal.timeout(30_000).addEventListener('abort', () => resolve(), { once: true });
});
await Promise.race([generatorDone, timeoutDone]).then(() => {}, () => {
logger.warn('SESSION', 'Generator did not exit within 30s after abort, forcing cleanup (#1099)', { sessionDbId });
});
}
// 3. Verify subprocess exit with 5s timeout (Issue #737 fix)
const tracked = getProcessBySession(sessionDbId);
if (tracked && !tracked.process.killed && tracked.process.exitCode === null) {
if (tracked && tracked.process.exitCode === null) {
logger.debug('SESSION', `Waiting for subprocess PID ${tracked.pid} to exit`, {
sessionId: sessionDbId,
pid: tracked.pid
@@ -303,6 +311,17 @@ export class SessionManager {
await ensureProcessExit(tracked, 5000);
}
// 3b. Reap all supervisor-tracked processes for this session (#1351)
// This catches MCP servers and other child processes not tracked by the
// in-memory ProcessRegistry (e.g. processes registered only in supervisor.json).
try {
await getSupervisor().getRegistry().reapSession(sessionDbId);
} catch (error) {
logger.warn('SESSION', 'Supervisor reapSession failed (non-blocking)', {
sessionId: sessionDbId
}, error as Error);
}
// 4. Cleanup
this.sessions.delete(sessionDbId);
this.sessionQueues.delete(sessionDbId);
@@ -468,6 +487,9 @@ export class SessionManager {
session.earliestPendingTimestamp = Math.min(session.earliestPendingTimestamp, message._originalTimestamp);
}
// Update generator activity for stale detection (Issue #1099)
session.lastGeneratorActivity = Date.now();
yield message;
}
}
@@ -56,6 +56,9 @@ export async function processAgentResponse(
agentName: string,
projectRoot?: string
): Promise<void> {
// Track generator activity for stale detection (Issue #1099)
session.lastGeneratorActivity = Date.now();
// Add assistant response to shared conversation history for provider interop
if (text) {
session.conversationHistory.push({ role: 'assistant', content: text });
@@ -189,8 +192,8 @@ async function syncAndBroadcastObservations(
const obs = observations[i];
const chromaStart = Date.now();
// Sync to Chroma (fire-and-forget)
dbManager.getChromaSync().syncObservation(
// Sync to Chroma (fire-and-forget, skipped if Chroma is disabled)
dbManager.getChromaSync()?.syncObservation(
obsId,
session.contentSessionId,
session.project,
@@ -282,8 +285,8 @@ async function syncAndBroadcastSummary(
const chromaStart = Date.now();
// Sync to Chroma (fire-and-forget)
dbManager.getChromaSync().syncSummary(
// Sync to Chroma (fire-and-forget, skipped if Chroma is disabled)
dbManager.getChromaSync()?.syncSummary(
result.summaryId,
session.contentSessionId,
session.project,
+4 -2
View File
@@ -37,6 +37,8 @@ export function createMiddleware(
callback(new Error('CORS not allowed'));
}
},
methods: ['GET', 'HEAD', 'POST', 'PUT', 'PATCH', 'DELETE'],
allowedHeaders: ['Content-Type', 'Authorization', 'X-Requested-With'],
credentials: false
}));
@@ -55,13 +57,13 @@ export function createMiddleware(
// Log incoming request with body summary
const bodySummary = summarizeRequestBody(req.method, req.path, req.body);
logger.info('HTTP', `${req.method} ${req.path}`, { requestId }, bodySummary);
logger.debug('HTTP', `${req.method} ${req.path}`, { requestId }, bodySummary);
// Capture response
const originalSend = res.send.bind(res);
res.send = function(body: any) {
const duration = Date.now() - start;
logger.info('HTTP', `${res.statusCode} ${req.path}`, { requestId, duration: `${duration}ms` });
logger.debug('HTTP', `${res.statusCode} ${req.path}`, { requestId, duration: `${duration}ms` });
return originalSend(body);
};
+12 -2
View File
@@ -114,7 +114,12 @@ export class DataRoutes extends BaseRouteHandler {
* Body: { ids: number[], orderBy?: 'date_desc' | 'date_asc', limit?: number, project?: string }
*/
private handleGetObservationsByIds = this.wrapHandler((req: Request, res: Response): void => {
const { ids, orderBy, limit, project } = req.body;
let { ids, orderBy, limit, project } = req.body;
// Coerce string-encoded arrays from MCP clients (e.g. "[1,2,3]" or "1,2,3")
if (typeof ids === 'string') {
try { ids = JSON.parse(ids); } catch { ids = ids.split(',').map(Number); }
}
if (!ids || !Array.isArray(ids)) {
this.badRequest(res, 'ids must be an array of numbers');
@@ -163,7 +168,12 @@ export class DataRoutes extends BaseRouteHandler {
* Body: { memorySessionIds: string[] }
*/
private handleGetSdkSessionsByIds = this.wrapHandler((req: Request, res: Response): void => {
const { memorySessionIds } = req.body;
let { memorySessionIds } = req.body;
// Coerce string-encoded arrays from MCP clients (e.g. '["a","b"]' or "a,b")
if (typeof memorySessionIds === 'string') {
try { memorySessionIds = JSON.parse(memorySessionIds); } catch { memorySessionIds = memorySessionIds.split(',').map((s: string) => s.trim()); }
}
if (!Array.isArray(memorySessionIds)) {
this.badRequest(res, 'memorySessionIds must be an array');
+78 -9
View File
@@ -5,12 +5,85 @@
*/
import express, { Request, Response } from 'express';
import { readFileSync, existsSync, writeFileSync, readdirSync } from 'fs';
import { openSync, fstatSync, readSync, closeSync, existsSync, writeFileSync } from 'fs';
import { join } from 'path';
import { logger } from '../../../../utils/logger.js';
import { SettingsDefaultsManager } from '../../../../shared/SettingsDefaultsManager.js';
import { BaseRouteHandler } from '../BaseRouteHandler.js';
/**
* Read the last N lines from a file without loading the entire file into memory.
* Reads backwards from the end of the file in chunks until enough lines are found.
*/
export function readLastLines(filePath: string, lineCount: number): { lines: string; totalEstimate: number } {
const fd = openSync(filePath, 'r');
try {
const stat = fstatSync(fd);
const fileSize = stat.size;
if (fileSize === 0) {
return { lines: '', totalEstimate: 0 };
}
// Start with a reasonable chunk size, expand if needed
const INITIAL_CHUNK_SIZE = 64 * 1024; // 64KB
const MAX_READ_SIZE = 10 * 1024 * 1024; // 10MB cap to prevent OOM on huge single-line files
let readSize = Math.min(INITIAL_CHUNK_SIZE, fileSize);
let content = '';
let newlineCount = 0;
while (readSize <= fileSize && readSize <= MAX_READ_SIZE) {
const startPosition = Math.max(0, fileSize - readSize);
const bytesToRead = fileSize - startPosition;
const buffer = Buffer.alloc(bytesToRead);
readSync(fd, buffer, 0, bytesToRead, startPosition);
content = buffer.toString('utf-8');
// Count newlines to see if we have enough
newlineCount = 0;
for (let i = 0; i < content.length; i++) {
if (content[i] === '\n') newlineCount++;
}
// We need lineCount newlines to get lineCount full lines (trailing newline)
if (newlineCount >= lineCount || startPosition === 0) {
break;
}
// Double the read size for next attempt
readSize = Math.min(readSize * 2, fileSize, MAX_READ_SIZE);
}
// Split and take the last N lines
const allLines = content.split('\n');
// Remove trailing empty element from final newline
if (allLines.length > 0 && allLines[allLines.length - 1] === '') {
allLines.pop();
}
const startIndex = Math.max(0, allLines.length - lineCount);
const resultLines = allLines.slice(startIndex);
// Estimate total lines: if we read the whole file, we know exactly; otherwise estimate
let totalEstimate: number;
if (fileSize <= readSize) {
totalEstimate = allLines.length;
} else {
// Rough estimate based on average line length in the chunk we read
const avgLineLength = content.length / Math.max(newlineCount, 1);
totalEstimate = Math.round(fileSize / avgLineLength);
}
return {
lines: resultLines.join('\n'),
totalEstimate,
};
} finally {
closeSync(fd);
}
}
export class LogsRoutes extends BaseRouteHandler {
private getLogFilePath(): string {
const dataDir = SettingsDefaultsManager.get('CLAUDE_MEM_DATA_DIR');
@@ -50,19 +123,15 @@ export class LogsRoutes extends BaseRouteHandler {
const requestedLines = parseInt(req.query.lines as string || '1000', 10);
const maxLines = Math.min(requestedLines, 10000); // Cap at 10k lines
const content = readFileSync(logFilePath, 'utf-8');
const lines = content.split('\n');
// Return the last N lines
const startIndex = Math.max(0, lines.length - maxLines);
const recentLines = lines.slice(startIndex).join('\n');
const { lines: recentLines, totalEstimate } = readLastLines(logFilePath, maxLines);
const returnedLines = recentLines === '' ? 0 : recentLines.split('\n').length;
res.json({
logs: recentLines,
path: logFilePath,
exists: true,
totalLines: lines.length,
returnedLines: lines.length - startIndex
totalLines: totalEstimate,
returnedLines,
});
});
@@ -90,6 +90,8 @@ export class SessionRoutes extends BaseRouteHandler {
* we let the current generator finish naturally (max 5s linger timeout).
* The next generator will use the new provider with shared conversationHistory.
*/
private static readonly STALE_GENERATOR_THRESHOLD_MS = 30_000; // 30 seconds (#1099)
private ensureGeneratorRunning(sessionDbId: number, source: string): void {
const session = this.sessionManager.getSession(sessionDbId);
if (!session) return;
@@ -109,6 +111,26 @@ export class SessionRoutes extends BaseRouteHandler {
return;
}
// Generator is running - check if stale (no activity for 30s) to prevent queue stall (#1099)
const timeSinceActivity = Date.now() - session.lastGeneratorActivity;
if (timeSinceActivity > SessionRoutes.STALE_GENERATOR_THRESHOLD_MS) {
logger.warn('SESSION', 'Stale generator detected, aborting to prevent queue stall (#1099)', {
sessionId: sessionDbId,
timeSinceActivityMs: timeSinceActivity,
thresholdMs: SessionRoutes.STALE_GENERATOR_THRESHOLD_MS,
source
});
// Abort the stale generator and reset state
session.abortController.abort();
session.generatorPromise = null;
session.abortController = new AbortController();
session.lastGeneratorActivity = Date.now();
// Start a fresh generator
this.spawnInProgress.set(sessionDbId, true);
this.startGeneratorWithProvider(session, selectedProvider, 'stale-recovery');
return;
}
// Generator is running - check if provider changed
if (session.currentProvider && session.currentProvider !== selectedProvider) {
logger.info('SESSION', `Provider changed, will switch after current generator finishes`, {
@@ -155,8 +177,9 @@ export class SessionRoutes extends BaseRouteHandler {
historyLength: session.conversationHistory.length
});
// Track which provider is running
// Track which provider is running and mark activity for stale detection (#1099)
session.currentProvider = provider;
session.lastGeneratorActivity = Date.now();
session.generatorPromise = agent.startSession(session, this.workerService)
.catch(error => {
@@ -333,7 +356,7 @@ export class SessionRoutes extends BaseRouteHandler {
// Sync user prompt to Chroma
const chromaStart = Date.now();
const promptText = latestPrompt.prompt_text;
this.dbManager.getChromaSync().syncUserPrompt(
this.dbManager.getChromaSync()?.syncUserPrompt(
latestPrompt.id,
latestPrompt.memory_session_id,
latestPrompt.project,
@@ -504,57 +527,63 @@ export class SessionRoutes extends BaseRouteHandler {
}
}
const store = this.dbManager.getSessionStore();
try {
const store = this.dbManager.getSessionStore();
// Get or create session
const sessionDbId = store.createSDKSession(contentSessionId, '', '');
const promptNumber = store.getPromptNumberFromUserPrompts(contentSessionId);
// Get or create session
const sessionDbId = store.createSDKSession(contentSessionId, '', '');
const promptNumber = store.getPromptNumberFromUserPrompts(contentSessionId);
// Privacy check: skip if user prompt was entirely private
const userPrompt = PrivacyCheckValidator.checkUserPromptPrivacy(
store,
contentSessionId,
promptNumber,
'observation',
sessionDbId,
{ tool_name }
);
if (!userPrompt) {
res.json({ status: 'skipped', reason: 'private' });
return;
// Privacy check: skip if user prompt was entirely private
const userPrompt = PrivacyCheckValidator.checkUserPromptPrivacy(
store,
contentSessionId,
promptNumber,
'observation',
sessionDbId,
{ tool_name }
);
if (!userPrompt) {
res.json({ status: 'skipped', reason: 'private' });
return;
}
// Strip memory tags from tool_input and tool_response
const cleanedToolInput = tool_input !== undefined
? stripMemoryTagsFromJson(JSON.stringify(tool_input))
: '{}';
const cleanedToolResponse = tool_response !== undefined
? stripMemoryTagsFromJson(JSON.stringify(tool_response))
: '{}';
// Queue observation
this.sessionManager.queueObservation(sessionDbId, {
tool_name,
tool_input: cleanedToolInput,
tool_response: cleanedToolResponse,
prompt_number: promptNumber,
cwd: cwd || (() => {
logger.error('SESSION', 'Missing cwd when queueing observation in SessionRoutes', {
sessionId: sessionDbId,
tool_name
});
return '';
})()
});
// Ensure SDK agent is running
this.ensureGeneratorRunning(sessionDbId, 'observation');
// Broadcast observation queued event
this.eventBroadcaster.broadcastObservationQueued(sessionDbId);
res.json({ status: 'queued' });
} catch (error) {
// Return 200 on recoverable errors so the hook doesn't break
logger.error('SESSION', 'Observation storage failed', { contentSessionId, tool_name }, error as Error);
res.json({ stored: false, reason: (error as Error).message });
}
// Strip memory tags from tool_input and tool_response
const cleanedToolInput = tool_input !== undefined
? stripMemoryTagsFromJson(JSON.stringify(tool_input))
: '{}';
const cleanedToolResponse = tool_response !== undefined
? stripMemoryTagsFromJson(JSON.stringify(tool_response))
: '{}';
// Queue observation
this.sessionManager.queueObservation(sessionDbId, {
tool_name,
tool_input: cleanedToolInput,
tool_response: cleanedToolResponse,
prompt_number: promptNumber,
cwd: cwd || (() => {
logger.error('SESSION', 'Missing cwd when queueing observation in SessionRoutes', {
sessionId: sessionDbId,
tool_name
});
return '';
})()
});
// Ensure SDK agent is running
this.ensureGeneratorRunning(sessionDbId, 'observation');
// Broadcast observation queued event
this.eventBroadcaster.broadcastObservationQueued(sessionDbId);
res.json({ status: 'queued' });
});
/**
@@ -663,23 +692,30 @@ export class SessionRoutes extends BaseRouteHandler {
* Returns: { sessionDbId, promptNumber, skipped: boolean, reason?: string }
*/
private handleSessionInitByClaudeId = this.wrapHandler((req: Request, res: Response): void => {
const { contentSessionId, project, prompt } = req.body;
const { contentSessionId } = req.body;
// Only contentSessionId is truly required — Cursor and other platforms
// may omit prompt/project in their payload (#838, #1049)
const project = req.body.project || 'unknown';
const prompt = req.body.prompt || '[media prompt]';
const customTitle = req.body.customTitle || undefined;
logger.info('HTTP', 'SessionRoutes: handleSessionInitByClaudeId called', {
contentSessionId,
project,
prompt_length: prompt?.length
prompt_length: prompt?.length,
customTitle
});
// Validate required parameters
if (!this.validateRequired(req, res, ['contentSessionId', 'project', 'prompt'])) {
if (!this.validateRequired(req, res, ['contentSessionId'])) {
return;
}
const store = this.dbManager.getSessionStore();
// Step 1: Create/get SDK session (idempotent INSERT OR IGNORE)
const sessionDbId = store.createSDKSession(contentSessionId, project, prompt);
const sessionDbId = store.createSDKSession(contentSessionId, project, prompt, customTitle);
// Verify session creation with DB lookup
const dbSession = store.getSessionById(sessionDbId);
@@ -723,16 +759,22 @@ export class SessionRoutes extends BaseRouteHandler {
// Step 5: Save cleaned user prompt
store.saveUserPrompt(contentSessionId, promptNumber, cleanedPrompt);
// Step 6: Check if SDK agent is already running for this session (#1079)
// If contextInjected is true, the hook should skip re-initializing the SDK agent
const contextInjected = this.sessionManager.getSession(sessionDbId) !== undefined;
// Debug-level log since CREATED already logged the key info
logger.debug('SESSION', 'User prompt saved', {
sessionId: sessionDbId,
promptNumber
promptNumber,
contextInjected
});
res.json({
sessionDbId,
promptNumber,
skipped: false
skipped: false,
contextInjected
});
});
}
+11 -9
View File
@@ -8,7 +8,6 @@
import { readFileSync, writeFileSync, existsSync, mkdirSync } from 'fs';
import { join, dirname } from 'path';
import { homedir } from 'os';
import { DEFAULT_OBSERVATION_TYPES_STRING, DEFAULT_OBSERVATION_CONCEPTS_STRING } from '../constants/observation-metadata.js';
// NOTE: Do NOT import logger here - it creates a circular dependency
// logger.ts depends on SettingsDefaultsManager for its initialization
@@ -41,9 +40,6 @@ export interface SettingsDefaults {
CLAUDE_MEM_CONTEXT_SHOW_WORK_TOKENS: string;
CLAUDE_MEM_CONTEXT_SHOW_SAVINGS_AMOUNT: string;
CLAUDE_MEM_CONTEXT_SHOW_SAVINGS_PERCENT: string;
// Observation Filtering
CLAUDE_MEM_CONTEXT_OBSERVATION_TYPES: string;
CLAUDE_MEM_CONTEXT_OBSERVATION_CONCEPTS: string;
// Display Configuration
CLAUDE_MEM_CONTEXT_FULL_COUNT: string;
CLAUDE_MEM_CONTEXT_FULL_FIELD: string;
@@ -51,6 +47,7 @@ export interface SettingsDefaults {
// Feature Toggles
CLAUDE_MEM_CONTEXT_SHOW_LAST_SUMMARY: string;
CLAUDE_MEM_CONTEXT_SHOW_LAST_MESSAGE: string;
CLAUDE_MEM_CONTEXT_SHOW_TERMINAL_OUTPUT: string;
CLAUDE_MEM_FOLDER_CLAUDEMD_ENABLED: string;
// Process Management
CLAUDE_MEM_MAX_CONCURRENT_AGENTS: string; // Max concurrent Claude SDK agent subprocesses (default: 2)
@@ -58,6 +55,7 @@ export interface SettingsDefaults {
CLAUDE_MEM_EXCLUDED_PROJECTS: string; // Comma-separated glob patterns for excluded project paths
CLAUDE_MEM_FOLDER_MD_EXCLUDE: string; // JSON array of folder paths to exclude from CLAUDE.md generation
// Chroma Vector Database Configuration
CLAUDE_MEM_CHROMA_ENABLED: string; // 'true' | 'false' - set to 'false' for SQLite-only mode
CLAUDE_MEM_CHROMA_MODE: string; // 'local' | 'remote'
CLAUDE_MEM_CHROMA_HOST: string;
CLAUDE_MEM_CHROMA_PORT: string;
@@ -101,9 +99,6 @@ export class SettingsDefaultsManager {
CLAUDE_MEM_CONTEXT_SHOW_WORK_TOKENS: 'false',
CLAUDE_MEM_CONTEXT_SHOW_SAVINGS_AMOUNT: 'false',
CLAUDE_MEM_CONTEXT_SHOW_SAVINGS_PERCENT: 'true',
// Observation Filtering
CLAUDE_MEM_CONTEXT_OBSERVATION_TYPES: DEFAULT_OBSERVATION_TYPES_STRING,
CLAUDE_MEM_CONTEXT_OBSERVATION_CONCEPTS: DEFAULT_OBSERVATION_CONCEPTS_STRING,
// Display Configuration
CLAUDE_MEM_CONTEXT_FULL_COUNT: '0',
CLAUDE_MEM_CONTEXT_FULL_FIELD: 'narrative',
@@ -111,6 +106,7 @@ export class SettingsDefaultsManager {
// Feature Toggles
CLAUDE_MEM_CONTEXT_SHOW_LAST_SUMMARY: 'true',
CLAUDE_MEM_CONTEXT_SHOW_LAST_MESSAGE: 'false',
CLAUDE_MEM_CONTEXT_SHOW_TERMINAL_OUTPUT: 'true',
CLAUDE_MEM_FOLDER_CLAUDEMD_ENABLED: 'false',
// Process Management
CLAUDE_MEM_MAX_CONCURRENT_AGENTS: '2', // Max concurrent Claude SDK agent subprocesses
@@ -118,6 +114,7 @@ export class SettingsDefaultsManager {
CLAUDE_MEM_EXCLUDED_PROJECTS: '', // Comma-separated glob patterns for excluded project paths
CLAUDE_MEM_FOLDER_MD_EXCLUDE: '[]', // JSON array of folder paths to exclude from CLAUDE.md generation
// Chroma Vector Database Configuration
CLAUDE_MEM_CHROMA_ENABLED: 'true', // Set to 'false' to disable Chroma and use SQLite-only search
CLAUDE_MEM_CHROMA_MODE: 'local', // 'local' uses persistent chroma-mcp via uvx, 'remote' connects to existing server
CLAUDE_MEM_CHROMA_HOST: '127.0.0.1',
CLAUDE_MEM_CHROMA_PORT: '8000',
@@ -136,10 +133,15 @@ export class SettingsDefaultsManager {
}
/**
* Get a default value from defaults (no environment variable override)
* Get a setting value with environment variable override.
* Priority: process.env > hardcoded default
*
* For full priority (env > settings file > default), use loadFromFile().
* This method is safe to call at module-load time (no file I/O) and still
* respects environment variable overrides that were previously ignored.
*/
static get(key: keyof SettingsDefaults): string {
return this.DEFAULTS[key];
return process.env[key] ?? this.DEFAULTS[key];
}
/**
+37 -4
View File
@@ -24,7 +24,37 @@ const _dirname = getDirname();
*/
// Base directories
export const DATA_DIR = SettingsDefaultsManager.get('CLAUDE_MEM_DATA_DIR');
// Resolve DATA_DIR with full priority: env var > settings.json > default.
// SettingsDefaultsManager.get() handles env > default. For settings file
// support, we do a one-time synchronous read of the default settings path
// to check if the user configured a custom DATA_DIR there.
function resolveDataDir(): string {
// 1. Environment variable (highest priority) — already handled by get()
if (process.env.CLAUDE_MEM_DATA_DIR) {
return process.env.CLAUDE_MEM_DATA_DIR;
}
// 2. Settings file at the default location
const defaultDataDir = join(homedir(), '.claude-mem');
const settingsPath = join(defaultDataDir, 'settings.json');
try {
if (existsSync(settingsPath)) {
const { readFileSync } = require('fs');
const raw = JSON.parse(readFileSync(settingsPath, 'utf-8'));
const settings = raw.env ?? raw; // handle legacy nested schema
if (settings.CLAUDE_MEM_DATA_DIR) {
return settings.CLAUDE_MEM_DATA_DIR;
}
}
} catch {
// settings file missing or corrupt — fall through to default
}
// 3. Hardcoded default
return defaultDataDir;
}
export const DATA_DIR = resolveDataDir();
// Note: CLAUDE_CONFIG_DIR is a Claude Code setting, not claude-mem, so leave as env var
export const CLAUDE_CONFIG_DIR = process.env.CLAUDE_CONFIG_DIR || join(homedir(), '.claude');
@@ -99,7 +129,9 @@ export function ensureAllClaudeDirs(): void {
}
/**
* Get current project name from git root or cwd
* Get current project name from git root or cwd.
* Includes parent directory to avoid collisions when repos share a folder name
* (e.g., ~/work/monorepo "work/monorepo" vs ~/personal/monorepo "personal/monorepo").
*/
export function getCurrentProjectName(): string {
try {
@@ -109,12 +141,13 @@ export function getCurrentProjectName(): string {
stdio: ['pipe', 'pipe', 'ignore'],
windowsHide: true
}).trim();
return basename(gitRoot);
return basename(dirname(gitRoot)) + '/' + basename(gitRoot);
} catch (error) {
logger.debug('SYSTEM', 'Git root detection failed, using cwd basename', {
cwd: process.cwd()
}, error as Error);
return basename(process.cwd());
const cwd = process.cwd();
return basename(dirname(cwd)) + '/' + basename(cwd);
}
}

Some files were not shown because too many files have changed in this diff Show More