Compare commits

...

27 Commits

Author SHA1 Message Date
Alex Newman 18aa5dc4e7 chore: bump version to 11.0.1
Publish to npm / publish (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 21:08:32 -07:00
Alex Newman 6cb74c6183 Merge pull request #1622 from thedotmack/disable-semantic-inject-default
fix: disable semantic inject by default
2026-04-05 21:07:23 -07:00
Alex Newman 0f9745535a fix: disable semantic inject by default — experimental feature not ready for all users
The per-prompt Chroma vector search injection on UserPromptSubmit adds latency
and context noise. Disable by default while we iterate on a more precise
file-context approach. Users can still opt in via CLAUDE_MEM_SEMANTIC_INJECT=true.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 21:05:03 -07:00
Alex Newman 76a27296f0 fix: wire up Cursor integration in installer (#1605)
* fix: wire up Cursor integration in installer — was incorrectly marked "coming soon"

CursorHooksInstaller.ts was fully built but never connected to the
installer. Set supported: true in IDE detection and call installCursorHooks
in the setup flow, matching the pattern used by other integrations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: wire up Cursor MCP configuration during install

PR review flagged that the hint says "hooks + MCP integration" but
configureCursorMcp() was never called during install. Now invoked
after hooks install with graceful fallback if MCP setup fails.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 22:44:49 -07:00
Alex Newman e2d4babae8 docs: regenerate CHANGELOG.md with comprehensive v11.0.0 release notes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 19:48:13 -07:00
Alex Newman 00ab61b46e docs: update CHANGELOG.md for v11.0.0
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 19:42:18 -07:00
Alex Newman a7ebc35ee0 chore: bump version to 11.0.0
Publish to npm / publish (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 19:39:28 -07:00
Alex Newman 9063c5d8a7 fix: block memory agent prose-skip responses at prompt and runtime levels
Observer prompt now explicitly requires XML observation blocks or empty
responses — prose explanations like "Skipping" are discarded. ResponseProcessor
logs a warning when non-XML content is received. Recording focus expanded to
include concrete debugging findings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 19:39:01 -07:00
Alex Newman 3b34feb779 chore: rebuild plugin artifacts for v10.7.2 with Alessandro's stability PRs (#1607)
Rebuilt worker-service, mcp-server, and viewer-bundle to include:
- SIGTERM drain for orphaned pending messages (#1567)
- Multi-machine sync script (#1570)
- 3 upstream bug fixes: summarize loop, ChromaSync duplicates, TOCTOU port check (#1566)
- Semantic context injection via Chroma (#1568)
- Tier routing by queue complexity (#1569)
- Architecture overview + production guide docs (#1574)

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 19:36:32 -07:00
Alex Newman ad58fdf8fc docs: update CHANGELOG.md for v10.7.2
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 19:25:32 -07:00
Alex Newman b385570884 chore: bump version to 10.7.2
Publish to npm / publish (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 19:22:50 -07:00
Alex Newman 29ef3f5603 fix: downgrade concept-type cleanup log from error to debug (#1606)
The parser correctly strips observation types from concepts arrays when the
LLM ignores the prompt instruction. This is routine data normalization, not
an error — downgrade to debug to reduce log noise.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 19:21:38 -07:00
Alex Newman f7a088c6d9 docs: update CHANGELOG.md for v10.7.1 2026-04-04 19:01:19 -07:00
Alex Newman 538ada9ec4 docs: update CHANGELOG.md for v10.7.1 2026-04-04 19:00:04 -07:00
Alex Newman bedca129ac chore: bump version to 10.7.1
Publish to npm / publish (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 18:59:00 -07:00
Alex Newman 70a8edc5b1 fix: restore full interactive installer — Claude Code CLI delegation was Claude-Code-only
The install simplification in 21b10b46 over-applied scope: it replaced the
entire runInstallCommand (interactive IDE multi-select, --ide flag, 13 IDE
setup dispatchers) with just two `claude` CLI commands. The intent was to
simplify the Claude Code path only.

Now: Claude Code uses `claude plugin marketplace add` + `claude plugin install`.
All other IDEs get the full installer flow (file copy, registration, IDE-specific
setup). Interactive multi-select and --ide flag are restored.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 18:57:53 -07:00
Alex Newman 811c94da36 fix: correct content hash description, update merged PR references, fix ChromaSync desc 2026-04-04 15:20:30 -07:00
Alessandro Costa af6bfda2d8 fix: address CodeRabbit review on PR #1574
- architecture-overview: add 'text' language to all fenced code blocks (MD040)
- architecture-overview: split 'Stop' lifecycle into 'Summary' + 'SessionEnd'
  to match canonical 5-hook naming
- production-guide: reference PR numbers for proposed settings not yet upstream

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 15:19:52 -07:00
Alessandro Costa bf8b7dbd9f docs: add architecture overview and production guide
Architecture overview covers the 4-layer system design, hook lifecycle,
data flow, and key patterns (CLAIM-CONFIRM, circuit-breaker, graceful
degradation, deduplication, dual session IDs).

Production guide provides recommended settings, health monitoring
metrics and thresholds, quick health check commands, multi-machine
sync setup, growth expectations, common issues with solutions, and
log analysis tips.

Based on 23 days of production usage with 3,400+ observations
across two physical servers and 8 projects.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 15:19:52 -07:00
Alex Newman 76207fb8d6 Merge branch 'feat/tier-routing-feedback' into thedotmack/merge-alessandro-prs 2026-04-04 15:18:34 -07:00
Alessandro Costa 42cc863bf2 fix: address CodeRabbit review on PR #1569
Critical:
- migrations: change version 8 → 25 to avoid collision with
  MigrationRunner.addObservationHierarchicalFields (uses version 8)
- SessionRoutes: remove duplicate imports that prevent compilation

Major:
- SessionRoutes: call applyTierRouting() before every generator spawn
  (stale-recovery and crash-recovery paths were missing it)
- applyTierRouting: clear session.modelOverride at top before re-evaluating
  to prevent stale tier from persisting across spawns

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 15:18:13 -07:00
Alessandro Costa 0fcc078873 feat: tier routing by queue complexity + observation feedback table
Tier Routing:
- Inspect pending queue before starting generator
- Summarize messages → CLAUDE_MEM_TIER_SUMMARY_MODEL (e.g., Opus)
- All simple tools (Read, Glob, Grep, LS) → CLAUDE_MEM_TIER_SIMPLE_MODEL (Haiku)
- Mixed/complex → default model (no override)
- session.modelOverride in ActiveSession, used by SDKAgent.getModelId()
- peekPendingTypes() in PendingMessageStore for non-claiming inspection
- Configurable via CLAUDE_MEM_TIER_ROUTING_ENABLED (default: true)

Feedback Collection (schema only):
- New observation_feedback table via MigrationRunner (schema version 24)
- Tracks signal_type (semantic_inject_hit, search_accessed, etc.)
- Indexes on observation_id and signal_type
- Foundation for future Thompson Sampling optimization

Production data (24h tier routing test):
- 36 Haiku observations in 4 min, quality indistinguishable from Sonnet
- Estimated ~52% cost reduction on SDK Agent usage
- 835 → 6,695 feedback signals collected over 13 days

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 15:18:13 -07:00
Alex Newman d11c0821bb fix: correct semantic endpoint doc comment GET→POST, clamp limit 1-20
Follow-up to PR #1568: fix stale doc comment that still said GET, and add
limit parameter validation (default 5, clamped to 1-20 range).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 15:17:11 -07:00
Alessandro Costa 876cc4d837 feat: semantic context injection via Chroma on UserPromptSubmit (#1568)
* feat: semantic context injection via Chroma on every UserPromptSubmit

On each prompt, queries ChromaDB for the top-N most relevant past
observations and injects them as additionalContext. Replaces the
recency-based "last N observations" approach with relevance-based
semantic search.

Changes:
- session-init.ts: After session init, query /api/context/semantic
  with user's prompt text. If results found, return as
  hookSpecificOutput with hookEventName 'UserPromptSubmit'.
- SearchRoutes.ts: New GET /api/context/semantic endpoint that queries
  SearchManager with format='json' and formats results as markdown.
- SettingsDefaultsManager.ts: New settings CLAUDE_MEM_SEMANTIC_INJECT
  (default: true) and CLAUDE_MEM_SEMANTIC_INJECT_LIMIT (default: 5).

Key behaviors:
- Fires on every UserPromptSubmit (not just SessionStart)
- Minimum prompt length: 20 chars (skips "ok", "yes", etc.)
- Skips media-only prompts
- Graceful degradation: if worker/Chroma unavailable, no injection
- Survives /clear: re-injects on next prompt (not session-bound)
- Uses workerHttpRequest (v10.6.3 API, not raw fetch)

Production data (23 days, 3,400+ observations):
- Before: 8 most recent observations (often irrelevant to current topic)
- After: 5 most relevant observations (semantic match)
- Token cost: ~1800 → ~800-1200 per injection

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address CodeRabbit review on PR #1568

- session-init: don't skip semantic injection when contextInjected=true
  (only skip agent re-init, semantic lookup must run every prompt)
- session-init: normalize SEMANTIC_INJECT toggle via String().toLowerCase()
- semantic endpoint: change from GET to POST to avoid URL-length limits
  and prompt exposure in access logs. Handler accepts both body and query
  for backwards compatibility.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Alessandro Costa <alessandro@claudio.dev>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 15:16:46 -07:00
Alessandro Costa 64cce2bf10 fix: resolve 3 upstream bugs (summarize, ChromaSync, HealthMonitor) (#1566)
* fix: resolve 3 upstream bugs in summarize, ChromaSync, and HealthMonitor

1. summarize.ts: Skip summary when transcript has no assistant message.
   Prevents error loop where empty transcripts cause repeated failed
   summarize attempts (~30 errors/day observed in production).

2. ChromaSync.ts: Fallback to chroma_update_documents when add fails
   with "IDs already exist". Handles partial writes after MCP timeout
   without waiting for next backfill cycle.

3. HealthMonitor.ts: Replace HTTP-based isPortInUse with atomic socket
   bind on Unix. Eliminates TOCTOU race when two sessions start
   simultaneously (HTTP check is non-atomic — both see "port free"
   before either completes listen()). Updated tests accordingly.

All three bugs are pre-existing in v10.5.5. Confirmed via log analysis
of 543K lines over 17 days of production usage across two servers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: add CONTRIB_NOTES.md to gitignore

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address CodeRabbit review on PR #1566

- HealthMonitor: add APPROVED OVERRIDE annotation for Win32 HTTP fallback
- ChromaSync: replace chroma_update_documents with delete+add for proper
  upsert (update only modifies existing IDs, silently ignores missing ones)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Alessandro Costa <alessandro@claudio.dev>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 15:15:08 -07:00
Alessandro Costa 5a27420809 feat: add claude-mem-sync for multi-machine observation synchronization (#1570)
Bidirectional sync of observations and session summaries between
machines via SSH/SCP. Exports to JSON, transfers, imports with
deduplication by (created_at, title).

Commands:
  claude-mem-sync push <remote-host>    # local → remote
  claude-mem-sync pull <remote-host>    # remote → local
  claude-mem-sync sync <remote-host>    # bidirectional
  claude-mem-sync status <remote-host>  # compare counts

Features:
- Deduplication prevents duplicates on repeated runs
- Configurable paths via CLAUDE_MEM_DB / CLAUDE_MEM_REMOTE_DB
- Automatic temp file cleanup
- Requires only Python 3 + SSH on both machines

Tested syncing 3,400+ observations between two physical servers.
After sync, a session on the remote server used the transferred
memory to deliver a real feature PR — proving productive
cross-machine workflows.

Co-authored-by: Alessandro Costa <alessandro@claudio.dev>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 15:14:31 -07:00
Alessandro Costa 8958c3335d feat: drain orphaned pending messages on SIGTERM session completion (#1567)
* feat: drain orphaned pending messages on session completion (SIGTERM)

When deleteSession() aborts the SDK agent via SIGTERM, pending messages
in the queue are never processed. Without drain, they remain in
'pending' status forever — no future generator picks them up because
the session is already completed.

Adds markAllSessionMessagesAbandoned() call after deleteSession() in
completeByDbId(). This reuses the existing PendingMessageStore method
already used by worker-service.ts terminateSession().

Production evidence: 15 orphaned summarize messages found across
completed sessions (ages 3h to 3 days) before this fix. After fix:
0 orphaned messages over 23 days of operation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: document best-effort drain limitation per CodeRabbit review #1567

Add comment noting the rare race condition when generators outlive the
30s SIGTERM timeout. Practical risk is negligible (0 orphans over 23 days).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Alessandro Costa <alessandro@claudio.dev>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 15:14:25 -07:00
35 changed files with 6120 additions and 423 deletions
+1 -1
View File
@@ -10,7 +10,7 @@
"plugins": [
{
"name": "claude-mem",
"version": "10.7.0",
"version": "11.0.1",
"source": "./plugin",
"description": "Persistent memory system for Claude Code - context compression across sessions"
}
+4 -1
View File
@@ -34,4 +34,7 @@ src/ui/viewer.html
.claude-octopus/
.claude/session-intent.md
.claude/session-plan.md
.octo/
.octo/
# Local contribution analysis (not part of upstream)
CONTRIB_NOTES.md
+2 -65
View File
@@ -1,68 +1,5 @@
# Memory Context from Past Sessions
The following context is from claude-mem, a persistent memory system that tracks your coding sessions.
*No context yet. Complete your first session and context will appear here.*
# $CMEM claude-mem 2026-04-03 6:48pm PDT
Legend: 🎯session 🔴bugfix 🟣feature 🔄refactor ✅change 🔵discovery ⚖️decision
Format: ID TIME TYPE TITLE
Fetch details: get_observations([IDs]) | Search: mem-search skill
Stats: 50 obs (18,868t read) | 401,168t work | 95% savings
### Apr 3, 2026
62994 1:47p 🔴 Merge Commit Finalized on thedotmack/npx-gemini-cli Branch
62995 1:48p 🔵 Worker Running but Health Endpoint Doesn't Accept POST
62996 " 🔵 Worker Health Endpoint Returns Detailed Status via GET
62997 1:49p 🔵 Worker Service Timeout and Shutdown Behavior in worker-service.ts
62998 " 🔵 claude-mem Hook Architecture Defined in plugin/hooks/hooks.json
62999 " 🔵 Session Idle Timeout Architecture: Two-Tier System in claude-mem
63000 " 🔵 Orphan Reaper Runs Every 30 Seconds; Sessions Orphaned After 6 Hours
63001 1:51p 🔵 POST /api/sessions/complete Removes Sessions from Active Map to Unblock Orphan Reaper
63002 1:52p 🔵 Stop Hook Summarize Flow: Extracts Last Assistant Message from Transcript
63004 " 🔵 POST /api/sessions/summarize: Privacy Check Before Queuing SDK Agent
63005 " 🔵 SessionManager.deleteSession Verifies Subprocess Exit to Prevent Zombies
63007 " 🔵 deleteSession: 4-Step Teardown with Generator and Subprocess Timeouts
63008 1:53p 🔵 Queue Depth Always Read from Database; Generator Restarts Capped at 3
63009 " 🔴 Fixed Lost Summaries: session-complete Now Waits for Pending Work Before Deleting Session
63010 1:54p 🔴 SessionEnd Hook Timeout Increased to 180s
63014 2:00p 🔵 claude-mem Hook Architecture and Exit Code System
63015 2:01p 🔵 SessionEnd Hook Has a 1.5s Default Timeout Controlled by Environment Variable
63016 2:02p 🔴 Stop Hook Now Owns Full Session Lifecycle: Summarize → Poll → Complete
63017 " 🔵 Missing /api/sessions/status Route — Only DB-ID Variant Exists
63018 2:03p 🔴 Added /api/sessions/status Route Registration to SessionRoutes
63020 " 🟣 Added handleStatusByClaudeId Handler for GET /api/sessions/status
63022 " 🔄 Removed Pending-Work Polling from /api/sessions/complete — Moved to Stop Hook
63024 " 🔄 SessionEnd Hook Reverted to Fast Fire-and-Forget (2s Timeout)
63026 2:04p 🔵 claude-mem hooks.json Full Hook Lifecycle Configuration
63027 2:05p ✅ Push to Pull Request
63028 " 🔵 Pre-Push State: claude-mem Repository Changes
63029 " 🔴 Fix Lost Summaries: Move Summary Wait into Stop Hook
63035 2:11p ✅ Testing Plan Created for tmux-cli npx Installation Flows
63036 2:12p 🔵 claude-mem Supports 13 npx Installation Flows Across IDE Integrations
63037 " 🔵 Detailed Integration Strategies for All 13 claude-mem npx Installation Flows
63038 2:13p ✅ NPX Install Flow Test Plan Document Created
63039 " ✅ 12 TODO Tasks Created for npx Install Flow Testing
63040 2:19p 🟣 Comprehensive Test Suite Requested for Claude-Mem CLI
63041 2:20p 🔵 NPX Install Flow Test Plan Exists for 12 IDE Integrations
63042 " 🟣 Phase 2 E2E Runtime Testing Added to NPX Install Test Plan
63043 " ✅ Test Tasks Updated with Phase 2 E2E Runtime Steps for 5 IDE Flows
63044 " ✅ All Remaining Test Tasks (612) Updated with Phase 2 E2E Runtime Steps
63079 6:31p ⚖️ Test Execution via Subagents Using /do Command
63080 6:32p 🔵 IDE Auto-Detection Module in claude-mem
63081 " 🔵 Install Command Architecture with Multi-IDE Dispatch
63082 " 🔵 MCP Integrations Module for 6 IDEs
63083 " 🔵 Cursor, Windsurf, and Gemini CLI Hook-Based Integrations
63084 " 🔵 OpenCode, OpenClaw, and Codex CLI Installers
63085 6:33p 🔵 tmux-cli Available for Automated Testing
63086 " 🔵 NPX Install Flow Test Plan — 12 IDE Flows
63087 6:34p 🟣 Detailed Test Execution Plan Created for NPX Install Flows
63103 6:47p 🔵 NPX Install Fails for Windsurf IDE with Missing rxjs Dependency
63104 " 🔵 Windsurf Install Failure Was a Dependency Ordering Race
63105 " 🟣 claude-mem Gemini CLI Integration: 8 Hooks Registered
63106 " 🟣 claude-mem OpenCode Integration: Plugin File + AGENTS.md Context
Access 401k tokens of past work via get_observations([IDs]) or mem-search skill.
---
*Auto-updated by claude-mem after each session. Use MCP search tools for detailed queries.*
Use claude-mem's MCP search tools for manual memory queries.
+4397 -64
View File
File diff suppressed because it is too large Load Diff
+140
View File
@@ -0,0 +1,140 @@
# claude-mem Architecture Overview
## System Layers
```text
+-----------------------------------------------------------+
| Claude Code (host) |
| +-- Hook System (5 events) |
| +-- MCP Client (search tools) |
+-----------------------------------------------------------+
| CLI Layer (Bun) |
| +-- bun-runner.js (Node->Bun bridge) |
| +-- hook-command.ts (orchestrator) |
| +-- handlers/ (context, session-init, observation, |
| summarize, session-complete) |
+-----------------------------------------------------------+
| Worker Daemon (Express, port 37777) |
| +-- SessionManager (session lifecycle) |
| +-- SDKAgent (Claude Agent SDK) |
| +-- SearchManager (search orchestration) |
| +-- ProcessRegistry (subprocess management) |
| +-- ChromaSync (embedding synchronization) |
+-----------------------------------------------------------+
| Storage Layer |
| +-- SQLite (claude-mem.db) -- structured data |
| +-- ChromaDB (chroma.sqlite3) -- vector embeddings |
| +-- MCP Server (interface for Claude Code) |
+-----------------------------------------------------------+
```
## Hook Lifecycle
| Event | Handler | What it does | Timeout |
|-------|---------|-------------|---------|
| Setup | setup.sh | Install system dependencies | 300s |
| SessionStart | smart-install.js + context | Install deps + start worker + inject context | 60s |
| UserPromptSubmit | session-init | Register session + start SDK agent + semantic injection | 60s |
| PostToolUse | observation | Capture tool usage -> enqueue in worker | 120s |
| Summary | summarize | Request session summary from SDK agent | 120s |
| SessionEnd | session-complete | End session + drain pending messages | 30s |
## Data Flow
```text
User prompt -> session-init -> /api/sessions/init + /api/context/semantic
|
Tool use -> observation -> /api/sessions/observations
| |
| PendingMessageStore.enqueue()
| |
| SDKAgent.startSession()
| |
| Claude Agent SDK -> ResponseProcessor
| |
| +-- storeObservations() -> SQLite
| +-- chromaSync.sync() -> ChromaDB
| +-- broadcastObservation() -> SSE/UI
|
Stop -> summarize -> /api/sessions/summarize
-> session-complete -> /api/sessions/complete + drain
```
## Key Patterns
### CLAIM-CONFIRM (PendingMessageStore)
```text
enqueue() -> INSERT status='pending'
claimNextMessage() -> UPDATE status='processing' (atomic)
confirmProcessed() -> DELETE (success)
markFailed() -> UPDATE status='failed' (retry < 3)
Self-healing: messages in 'processing' for >60s reset to 'pending'
```
### Circuit-Breaker (SessionRoutes)
```text
Generator crash -> retry 1 (1s) -> retry 2 (2s) -> retry 3 (4s)
-> consecutiveRestarts > 3 -> CIRCUIT-BREAKER
-> markAllSessionMessagesAbandoned(sessionDbId)
-> Stop. No infinite loop.
```
Counter resets to 0 when generator completes work naturally.
### Graceful Degradation (hook-command.ts)
```text
Transport errors (ECONNREFUSED, timeout, 5xx) -> exit 0 (never block Claude Code)
Client bugs (4xx, TypeError, ReferenceError) -> exit 2 (blocking, needs fix)
```
The worker being unavailable NEVER blocks the user's Claude Code session.
### Deduplication (observations)
```text
SHA256(memory_session_id + title + narrative)[:16] -> content_hash (16 hex chars)
If hash exists within 30s window -> return existing ID (no insert)
```
### Two Types of Session ID
- `contentSessionId` — from Claude Code, invariant during the session
- `memorySessionId` — from SDK Agent, changes on each worker restart
The conversion between them is handled by SessionStore and is critical for FK constraints.
## Storage
### SQLite (claude-mem.db)
| Table | Key fields | Purpose |
|-------|-----------|---------|
| sdk_sessions | content_session_id, memory_session_id, status | Session lifecycle |
| observations | memory_session_id, type, title, narrative, content_hash | Tool usage observations |
| session_summaries | memory_session_id, request, learned, completed | Session summaries |
| user_prompts | content_session_id, prompt_text | User prompt history |
| pending_messages | session_db_id, status, message_type | CLAIM-CONFIRM queue |
| observation_feedback | observation_id, signal_type | Usage tracking |
### ChromaDB (chroma.sqlite3)
Vector embeddings for semantic search. Each observation generates multiple documents:
```text
obs_{id}_narrative -> main text
obs_{id}_fact_0 -> first fact
obs_{id}_fact_1 -> second fact
...
```
Accessed via chroma-mcp (MCP process), communication over stdio.
## Process Management
- **ProcessRegistry:** Tracks all Claude SDK subprocesses, manages PID lifecycle
- **Orphan Reaper (5min):** Kills processes with no active session
- **GracefulShutdown:** 7-step shutdown (PID file, children, HTTP server, sessions, MCP, DB, force-kill)
+111
View File
@@ -0,0 +1,111 @@
# claude-mem Production Guide
Practical guide based on 23 days of production usage with 3,400+ observations across two physical servers and 8 projects.
## Recommended Settings
| Setting | Default | Recommended | Why |
|---------|---------|-------------|-----|
| CLAUDE_MEM_MAX_CONCURRENT_AGENTS | 2 | 3 | Better throughput without overload |
| CLAUDE_MEM_SEMANTIC_INJECT | true | true | Relevant context >> recent context |
| CLAUDE_MEM_SEMANTIC_INJECT_LIMIT | 5 | 5 | Sweet spot for token cost vs coverage |
| CLAUDE_MEM_TIER_ROUTING_ENABLED | true | true | ~52% cost savings, no quality loss |
## Health Monitoring
### Key metrics to watch
| Metric | Healthy | Warning | Action |
|--------|---------|---------|--------|
| pending_messages (pending) | 0-5 | >10 | Check worker logs, may need restart |
| pending_messages (failed) | 0 | >0 growing | Circuit-breaker may be tripping |
| sdk_sessions (active) | 0-3 | >5 stuck | Orphan sessions, worker restart |
| WAL size | <10 MB | >20 MB | Run `PRAGMA wal_checkpoint(TRUNCATE)` |
| Chroma size | Growing slowly | Sudden jump | Check for sync loops |
| Errors/day in logs | 0-2 | >10 | Investigate log patterns |
### Quick health check
```bash
# Check worker status
curl -s http://127.0.0.1:37777/api/health | python3 -m json.tool
# Check database stats
sqlite3 ~/.claude-mem/claude-mem.db "
SELECT 'observations' as metric, COUNT(*) as value FROM observations
UNION ALL SELECT 'summaries', COUNT(*) FROM session_summaries
UNION ALL SELECT 'pending', COUNT(*) FROM pending_messages WHERE status='pending'
UNION ALL SELECT 'active_sessions', COUNT(*) FROM sdk_sessions WHERE status='active';
"
```
## Multi-Machine Setup
If running claude-mem on multiple machines, use `claude-mem-sync` to keep observations in sync:
```bash
claude-mem-sync push <remote-host> # local -> remote
claude-mem-sync pull <remote-host> # remote -> local
claude-mem-sync sync <remote-host> # bidirectional
claude-mem-sync status <remote-host> # compare counts
```
Deduplication is by `(created_at, title)` — safe to run repeatedly.
## Growth Expectations
Based on active daily development usage:
| Metric | Per day | Per month | Notes |
|--------|---------|-----------|-------|
| Observations | ~120 | ~3,600 | Varies with coding activity |
| Summaries | ~40 | ~1,200 | One per session |
| SQLite | ~0.8 MB | ~24 MB | ~5 KB per observation |
| Chroma | ~4 MB | ~120 MB | ~50 KB per observation (embeddings) |
## Common Issues and Solutions
### Summarize error loop
**Symptom:** Repeated `[ERROR] Missing last_assistant_message` in logs.
**Cause:** Transcript with no assistant messages triggers summary attempt that fails repeatedly.
**Fix:** PR #1566 — skip summary when transcript is empty.
### Chroma sync failures
**Symptom:** `[ERROR] Batch add failed... IDs already exist`
**Cause:** MCP timeout during add leaves partial writes; retry fails on existing IDs.
**Fix:** PR #1566 — fallback to delete+add reconciliation.
### Port conflict on startup
**Symptom:** `Worker failed to start... Is port 37777 in use?`
**Cause:** Two sessions starting simultaneously — HTTP check is non-atomic (TOCTOU race).
**Fix:** PR #1566 — atomic socket bind on Unix.
### Orphaned pending messages
**Symptom:** `pending_messages` table growing with old entries for completed sessions.
**Cause:** SIGTERM kills generator before queue is drained.
**Fix:** PR #1567 — drain after deleteSession().
### Context not relevant to current topic
**Symptom:** Claude receives observations about CSS when you're asking about authentication.
**Cause:** Default recency-based injection selects most recent, not most relevant.
**Fix:** PR #1568 — semantic injection via Chroma on every prompt.
## Log Analysis Tips
```bash
# Count errors by day
grep '\[ERROR\]' ~/.claude-mem/logs/claude-mem-*.log | \
sed 's/\[20[0-9][0-9]-[0-9][0-9]-/\n&/g' | \
grep -oP '^\[20\d{2}-\d{2}-\d{2}' | sort | uniq -c
# Find circuit-breaker trips
grep 'circuit\|Circuit\|ABANDONED\|abandoned' ~/.claude-mem/logs/claude-mem-*.log
# Check pending message health
grep 'CLAIMED\|CONFIRMED\|FAILED\|ABANDONED' ~/.claude-mem/logs/claude-mem-$(date +%Y-%m-%d).log | tail -20
```
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "claude-mem",
"version": "10.7.0",
"version": "11.0.1",
"description": "Memory compression system for Claude Code - persist context across sessions",
"keywords": [
"claude",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "claude-mem",
"version": "10.7.0",
"version": "11.0.1",
"description": "Persistent memory system for Claude Code - seamlessly preserve context across sessions",
"author": {
"name": "Alex Newman"
+3 -3
View File
@@ -87,8 +87,8 @@
"system_identity": "You are a Claude-Mem, a specialized observer tool for creating searchable memory FOR FUTURE SESSIONS.\n\nCRITICAL: Record what was LEARNED/BUILT/FIXED/DEPLOYED/CONFIGURED, not what you (the observer) are doing.\n\nYou do not have access to tools. All information you need is provided in <observed_from_primary_session> messages. Create observations from what you observe - no investigation needed.",
"spatial_awareness": "SPATIAL AWARENESS: Tool executions include the working directory (tool_cwd) to help you understand:\n- Which repository/project is being worked on\n- Where files are located relative to the project root\n- How to match requested paths to actual execution paths",
"observer_role": "Your job is to monitor a different Claude Code session happening RIGHT NOW, with the goal of creating observations and progress summaries as the work is being done LIVE by the user. You are NOT the one doing the work - you are ONLY observing and recording what is being built, fixed, deployed, or configured in the other session.",
"recording_focus": "WHAT TO RECORD\n--------------\nFocus on deliverables and capabilities:\n- What the system NOW DOES differently (new capabilities)\n- What shipped to users/production (features, fixes, configs, docs)\n- Changes in technical domains (auth, data, UI, infra, DevOps, docs)\n\nUse verbs like: implemented, fixed, deployed, configured, migrated, optimized, added, refactored\n\n✅ GOOD EXAMPLES (describes what was built):\n- \"Authentication now supports OAuth2 with PKCE flow\"\n- \"Deployment pipeline runs canary releases with auto-rollback\"\n- \"Database indexes optimized for common query patterns\"\n\n❌ BAD EXAMPLES (describes observation process - DO NOT DO THIS):\n- \"Analyzed authentication implementation and stored findings\"\n- \"Tracked deployment steps and logged outcomes\"\n- \"Monitored database performance and recorded metrics\"",
"skip_guidance": "WHEN TO SKIP\n------------\nSkip routine operations:\n- Empty status checks\n- Package installations with no errors\n- Simple file listings\n- Repetitive operations you've already documented\n- If file related research comes back as empty or not found\n- **No output necessary if skipping.**",
"recording_focus": "WHAT TO RECORD\n--------------\nFocus on durable technical signal:\n- What the system NOW DOES differently (new capabilities)\n- What shipped to users/production (features, fixes, configs, docs)\n- Changes in technical domains (auth, data, UI, infra, DevOps, docs)\n- Concrete debugging or investigative findings from logs, traces, queue state, database rows, and code-path inspection\n\nUse verbs like: implemented, fixed, deployed, configured, migrated, optimized, added, refactored, discovered, confirmed, traced\n\n✅ GOOD EXAMPLES (describes what was built or learned):\n- \"Authentication now supports OAuth2 with PKCE flow\"\n- \"Deployment pipeline runs canary releases with auto-rollback\"\n- \"Database indexes optimized for common query patterns\"\n- \"Observation queue for claude-mem session timed out waiting for an agent pool slot\"\n- \"Fallback processing abandoned pending messages after Gemini and OpenRouter returned 404\"\n\n❌ BAD EXAMPLES (describes observation process - DO NOT DO THIS):\n- \"Analyzed authentication implementation and stored findings\"\n- \"Tracked deployment steps and logged outcomes\"\n- \"Monitored database performance and recorded metrics\"",
"skip_guidance": "WHEN TO SKIP\n------------\nSkip routine operations:\n- Empty status checks\n- Package installations with no errors\n- Simple file listings with no follow-on finding\n- Repetitive operations you've already documented\n- File related research that comes back empty or not found\n\nIf skipping, return an empty response only. Do not explain the skip in prose.",
"type_guidance": "**type**: MUST be EXACTLY one of these 6 options (no other values allowed):\n - bugfix: something was broken, now fixed\n - feature: new capability or functionality added\n - refactor: code restructured, behavior unchanged\n - change: generic modification (docs, config, misc)\n - discovery: learning about existing system\n - decision: architectural/design choice with rationale",
"concept_guidance": "**concepts**: 2-5 knowledge-type categories. MUST use ONLY these exact keywords:\n - how-it-works: understanding mechanisms\n - why-it-exists: purpose or rationale\n - what-changed: modifications made\n - problem-solution: issues and their fixes\n - gotcha: traps or edge cases\n - pattern: reusable approach\n - trade-off: pros/cons of a decision\n\n IMPORTANT: Do NOT include the observation type (change/discovery/decision) as a concept.\n Types and concepts are separate dimensions.",
"field_guidance": "**facts**: Concise, self-contained statements\nEach fact is ONE piece of information\n No pronouns - each fact must stand alone\n Include specific details: filenames, functions, values\n\n**files**: All files touched (full paths from project root)",
@@ -122,4 +122,4 @@
"summary_format_instruction": "Respond in this XML format:",
"summary_footer": "IMPORTANT! DO NOT do any work right now other than generating this next PROGRESS SUMMARY - and remember that you are a memory agent designed to summarize a DIFFERENT claude code session, not this one.\n\nNever reference yourself or your own actions. Do not output anything other than the summary content formatted in the XML structure above. All other output is ignored by the system, and the system has been designed to be smart about token usage. Please spend your tokens wisely on useful summary content.\n\nThank you, this summary will be very useful for keeping track of our progress!"
}
}
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "claude-mem-plugin",
"version": "10.7.0",
"version": "11.0.1",
"private": true,
"description": "Runtime dependencies for claude-mem bundled hooks",
"type": "module",
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+181
View File
@@ -0,0 +1,181 @@
#!/bin/bash
# claude-mem-sync — Synchronize claude-mem observations between machines
#
# Usage:
# claude-mem-sync push <remote-host> # local → remote
# claude-mem-sync pull <remote-host> # remote → local
# claude-mem-sync sync <remote-host> # bidirectional (push + pull)
# claude-mem-sync status <remote-host> # compare counts
#
# Prerequisites:
# - SSH access to remote host (key-based auth recommended)
# - Python 3 on both machines
# - claude-mem installed on both machines (~/.claude-mem/claude-mem.db)
#
# Environment variables:
# CLAUDE_MEM_DB Local database path (default: ~/.claude-mem/claude-mem.db)
# CLAUDE_MEM_REMOTE_DB Remote database path (default: ~/.claude-mem/claude-mem.db)
set -euo pipefail
LOCAL_DB="${CLAUDE_MEM_DB:-$HOME/.claude-mem/claude-mem.db}"
COMMAND="${1:?Usage: claude-mem-sync <push|pull|sync|status> <remote-host>}"
REMOTE_HOST="${2:?Missing remote host. Usage: claude-mem-sync $COMMAND <remote-host>}"
REMOTE_DB="${CLAUDE_MEM_REMOTE_DB:-\$HOME/.claude-mem/claude-mem.db}"
TMPDIR="/tmp/claude-mem-sync-$$"
mkdir -p "$TMPDIR"
trap "rm -rf $TMPDIR" EXIT
# Column lists for observations and session_summaries
OBS_COLS="memory_session_id,project,text,type,title,subtitle,facts,narrative,concepts,files_read,files_modified,prompt_number,discovery_tokens,created_at,created_at_epoch"
SUM_COLS="memory_session_id,project,request,investigated,learned,completed,next_steps,files_read,files_edited,notes,prompt_number,discovery_tokens,created_at,created_at_epoch"
export_obs() {
local db="$1" output="$2"
python3 -c "
import sqlite3, json, sys
conn = sqlite3.connect('$db')
cur = conn.cursor()
cur.execute('''SELECT $OBS_COLS FROM observations ORDER BY created_at''')
cols = '$OBS_COLS'.split(',')
rows = [dict(zip(cols, r)) for r in cur.fetchall()]
cur.execute('''SELECT $SUM_COLS FROM session_summaries ORDER BY created_at''')
cols2 = '$SUM_COLS'.split(',')
sums = [dict(zip(cols2, r)) for r in cur.fetchall()]
json.dump({'observations': rows, 'summaries': sums}, open('$output', 'w'))
print(f'{len(rows)} obs, {len(sums)} sums exported', file=sys.stderr)
conn.close()
"
}
import_obs() {
local db="$1" input="$2"
python3 -c "
import sqlite3, json, sys
conn = sqlite3.connect('$db')
cur = conn.cursor()
cur.execute('SELECT created_at, title FROM observations')
existing = set((r[0],r[1]) for r in cur.fetchall())
cur.execute('SELECT created_at, request FROM session_summaries')
existing_s = set((r[0],r[1]) for r in cur.fetchall())
data = json.load(open('$input'))
oi, si = 0, 0
obs_cols = '$OBS_COLS'.split(',')
sum_cols = '$SUM_COLS'.split(',')
obs_placeholders = ','.join(['?'] * len(obs_cols))
sum_placeholders = ','.join(['?'] * len(sum_cols))
for o in data['observations']:
if (o['created_at'], o['title']) not in existing:
cur.execute(f'INSERT INTO observations ($OBS_COLS) VALUES ({obs_placeholders})',
tuple(o[k] for k in obs_cols))
oi += 1
for s in data['summaries']:
if (s['created_at'], s['request']) not in existing_s:
cur.execute(f'INSERT INTO session_summaries ($SUM_COLS) VALUES ({sum_placeholders})',
tuple(s[k] for k in sum_cols))
si += 1
conn.commit()
print(f'{oi} new obs, {si} new sums imported', file=sys.stderr)
conn.close()
"
}
count_db() {
local db="$1"
python3 -c "
import sqlite3
conn = sqlite3.connect('$db')
cur = conn.cursor()
cur.execute('SELECT COUNT(*) FROM observations')
obs = cur.fetchone()[0]
cur.execute('SELECT COUNT(*) FROM session_summaries')
sums = cur.fetchone()[0]
cur.execute('SELECT MAX(created_at) FROM observations')
last = cur.fetchone()[0] or 'empty'
print(f'{obs} obs, {sums} sums (last: {last[:19]})')
conn.close()
"
}
case "$COMMAND" in
push)
echo "=== Push: local → $REMOTE_HOST ==="
export_obs "$LOCAL_DB" "$TMPDIR/export.json"
scp -q "$TMPDIR/export.json" "$REMOTE_HOST:/tmp/mem-import.json"
# Run import on remote
ssh "$REMOTE_HOST" "python3 -c \"
import sqlite3, json, sys
conn = sqlite3.connect('$REMOTE_DB')
cur = conn.cursor()
cur.execute('SELECT created_at, title FROM observations')
existing = set((r[0],r[1]) for r in cur.fetchall())
cur.execute('SELECT created_at, request FROM session_summaries')
existing_s = set((r[0],r[1]) for r in cur.fetchall())
data = json.load(open('/tmp/mem-import.json'))
obs_cols = '$OBS_COLS'.split(',')
sum_cols = '$SUM_COLS'.split(',')
obs_ph = ','.join(['?'] * len(obs_cols))
sum_ph = ','.join(['?'] * len(sum_cols))
oi, si = 0, 0
for o in data['observations']:
if (o['created_at'], o['title']) not in existing:
cur.execute(f'INSERT INTO observations ($OBS_COLS) VALUES ({obs_ph})', tuple(o[k] for k in obs_cols))
oi += 1
for s in data['summaries']:
if (s['created_at'], s['request']) not in existing_s:
cur.execute(f'INSERT INTO session_summaries ($SUM_COLS) VALUES ({sum_ph})', tuple(s[k] for k in sum_cols))
si += 1
conn.commit()
print(f'Remote: {oi} new obs, {si} new sums imported', file=sys.stderr)
conn.close()
\""
;;
pull)
echo "=== Pull: $REMOTE_HOST → local ==="
ssh "$REMOTE_HOST" "python3 -c \"
import sqlite3, json
conn = sqlite3.connect('$REMOTE_DB')
cur = conn.cursor()
cur.execute('SELECT $OBS_COLS FROM observations ORDER BY created_at')
cols = '$OBS_COLS'.split(',')
obs = [dict(zip(cols, r)) for r in cur.fetchall()]
cur.execute('SELECT $SUM_COLS FROM session_summaries ORDER BY created_at')
cols2 = '$SUM_COLS'.split(',')
sums = [dict(zip(cols2, r)) for r in cur.fetchall()]
json.dump({'observations': obs, 'summaries': sums}, open('/tmp/mem-export.json', 'w'))
print(f'{len(obs)} obs, {len(sums)} sums exported')
conn.close()
\""
scp -q "$REMOTE_HOST:/tmp/mem-export.json" "$TMPDIR/import.json"
import_obs "$LOCAL_DB" "$TMPDIR/import.json"
;;
sync)
echo "=== Bidirectional sync with $REMOTE_HOST ==="
"$0" push "$REMOTE_HOST"
"$0" pull "$REMOTE_HOST"
"$0" status "$REMOTE_HOST"
;;
status)
echo "=== Local ==="
count_db "$LOCAL_DB"
echo "=== Remote ($REMOTE_HOST) ==="
ssh "$REMOTE_HOST" "python3 -c \"
import sqlite3
conn = sqlite3.connect('$REMOTE_DB')
cur = conn.cursor()
cur.execute('SELECT COUNT(*) FROM observations')
obs = cur.fetchone()[0]
cur.execute('SELECT COUNT(*) FROM session_summaries')
sums = cur.fetchone()[0]
cur.execute('SELECT MAX(created_at) FROM observations')
last = cur.fetchone()[0] or 'empty'
print(f'{obs} obs, {sums} sums (last: {last[:19]})')
conn.close()
\""
;;
*)
echo "Usage: claude-mem-sync <push|pull|sync|status> <remote-host>"
exit 1
;;
esac
+50 -5
View File
@@ -87,17 +87,18 @@ export const sessionInitHandler: EventHandler = {
// Skip SDK agent re-initialization if context was already injected for this session (#1079)
// The prompt was already saved to the database by /api/sessions/init above —
// no need to re-start the SDK agent on every turn
if (initResult.contextInjected) {
// no need to re-start the SDK agent on every turn.
// Note: we do NOT return here — semantic injection below must run on every prompt.
const skipAgentInit = Boolean(initResult.contextInjected);
if (skipAgentInit) {
logger.info('HOOK', `INIT_COMPLETE | sessionDbId=${sessionDbId} | promptNumber=${promptNumber} | skipped_agent_init=true | reason=context_already_injected`, {
sessionId: sessionDbId
});
return { continue: true, suppressOutput: true };
}
// Only initialize SDK agent for Claude Code (not Cursor)
// Cursor doesn't use the SDK agent - it only needs session/observation storage
if (input.platform !== 'cursor' && sessionDbId) {
if (!skipAgentInit && input.platform !== 'cursor' && sessionDbId) {
// Strip leading slash from commands for memory agent
// /review 101 -> review 101 (more semantic for observations)
const cleanedPrompt = prompt.startsWith('/') ? prompt.substring(1) : prompt;
@@ -115,14 +116,58 @@ export const sessionInitHandler: EventHandler = {
// Log but don't throw - SDK agent failure should not block the user's prompt
logger.failure('HOOK', `SDK agent start failed: ${response.status}`, { sessionDbId, promptNumber });
}
} else if (input.platform === 'cursor') {
} else if (!skipAgentInit && input.platform === 'cursor') {
logger.debug('HOOK', 'session-init: Skipping SDK agent init for Cursor platform', { sessionDbId, promptNumber });
}
// Semantic context injection: query Chroma for relevant past observations
// and inject as additionalContext so Claude receives relevant memory each prompt.
// Controlled by CLAUDE_MEM_SEMANTIC_INJECT setting (default: true).
const semanticInject =
String(settings.CLAUDE_MEM_SEMANTIC_INJECT).toLowerCase() === 'true';
let additionalContext = '';
if (semanticInject && prompt && prompt.length >= 20 && prompt !== '[media prompt]') {
try {
const limit = settings.CLAUDE_MEM_SEMANTIC_INJECT_LIMIT || '5';
const semanticRes = await workerHttpRequest('/api/context/semantic', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ q: prompt, project, limit })
});
if (semanticRes.ok) {
const data = await semanticRes.json() as { context: string; count: number };
if (data.context) {
additionalContext = data.context;
logger.debug('HOOK', `Semantic injection: ${data.count} observations for prompt`, {
sessionId: sessionDbId, count: data.count
});
}
}
} catch (e) {
// Graceful degradation — semantic injection is optional
logger.debug('HOOK', 'Semantic injection unavailable', {
error: e instanceof Error ? e.message : String(e)
});
}
}
logger.info('HOOK', `INIT_COMPLETE | sessionDbId=${sessionDbId} | promptNumber=${promptNumber} | project=${project}`, {
sessionId: sessionDbId
});
// Return with semantic context if available
if (additionalContext) {
return {
continue: true,
suppressOutput: true,
hookSpecificOutput: {
hookEventName: 'UserPromptSubmit',
additionalContext
}
};
}
return { continue: true, suppressOutput: true };
}
};
+10
View File
@@ -52,6 +52,16 @@ export const summarizeHandler: EventHandler = {
return { continue: true, suppressOutput: true, exitCode: HOOK_EXIT_CODES.SUCCESS };
}
// Skip summary if transcript has no assistant message (prevents repeated
// empty summarize requests that pollute logs — upstream bug)
if (!lastAssistantMessage || !lastAssistantMessage.trim()) {
logger.debug('HOOK', 'No assistant message in transcript - skipping summary', {
sessionId,
transcriptPath
});
return { continue: true, suppressOutput: true, exitCode: HOOK_EXIT_CODES.SUCCESS };
}
logger.dataIn('HOOK', 'Stop: Requesting summary', {
hasLastAssistantMessage: !!lastAssistantMessage
});
+2 -1
View File
@@ -116,7 +116,8 @@ export function detectInstalledIDEs(): IDEInfo[] {
id: 'cursor',
label: 'Cursor',
detected: existsSync(join(home, '.cursor')),
supported: false,
supported: true,
hint: 'hooks + MCP integration',
},
{
id: 'copilot-cli',
+548 -21
View File
@@ -1,37 +1,564 @@
/**
* Install command for `npx claude-mem install`.
*
* Delegates to Claude Code's native plugin system two commands handle
* marketplace registration, plugin installation, dependency setup, and
* settings enablement.
* Replaces the git-clone + build workflow. The npm package already ships
* a pre-built `plugin/` directory; this command copies it into the right
* locations and registers it with Claude Code.
*
* Pure Node.js no Bun APIs used.
*/
import { execSync } from 'child_process';
import * as p from '@clack/prompts';
import pc from 'picocolors';
import { execSync } from 'child_process';
import { cpSync, existsSync, readFileSync, rmSync } from 'fs';
import { join } from 'path';
// Non-TTY detection: @clack/prompts crashes with ENOENT in non-TTY environments
const isInteractive = process.stdin.isTTY === true;
/** Run a list of tasks, falling back to plain console.log when non-TTY */
interface TaskDescriptor {
title: string;
task: (message: (msg: string) => void) => Promise<string>;
}
async function runTasks(tasks: TaskDescriptor[]): Promise<void> {
if (isInteractive) {
await p.tasks(tasks);
} else {
for (const t of tasks) {
const result = await t.task((msg: string) => console.log(` ${msg}`));
console.log(` ${result}`);
}
}
}
/** Log helpers that fall back to console.log in non-TTY */
const log = {
info: (msg: string) => isInteractive ? p.log.info(msg) : console.log(` ${msg}`),
success: (msg: string) => isInteractive ? p.log.success(msg) : console.log(` ${msg}`),
warn: (msg: string) => isInteractive ? p.log.warn(msg) : console.warn(` ${msg}`),
error: (msg: string) => isInteractive ? p.log.error(msg) : console.error(` ${msg}`),
};
import {
claudeSettingsPath,
ensureDirectoryExists,
installedPluginsPath,
IS_WINDOWS,
knownMarketplacesPath,
marketplaceDirectory,
npmPackagePluginDirectory,
npmPackageRootDirectory,
pluginCacheDirectory,
pluginsDirectory,
readPluginVersion,
writeJsonFileAtomic,
} from '../utils/paths.js';
import { readJsonSafe } from '../../utils/json-utils.js';
import { detectInstalledIDEs } from './ide-detection.js';
// ---------------------------------------------------------------------------
// Registration helpers
// ---------------------------------------------------------------------------
function registerMarketplace(): void {
const knownMarketplaces = readJsonSafe<Record<string, any>>(knownMarketplacesPath(), {});
knownMarketplaces['thedotmack'] = {
source: {
source: 'github',
repo: 'thedotmack/claude-mem',
},
installLocation: marketplaceDirectory(),
lastUpdated: new Date().toISOString(),
autoUpdate: true,
};
ensureDirectoryExists(pluginsDirectory());
writeJsonFileAtomic(knownMarketplacesPath(), knownMarketplaces);
}
function registerPlugin(version: string): void {
const installedPlugins = readJsonSafe<Record<string, any>>(installedPluginsPath(), {});
if (!installedPlugins.version) installedPlugins.version = 2;
if (!installedPlugins.plugins) installedPlugins.plugins = {};
const cachePath = pluginCacheDirectory(version);
const now = new Date().toISOString();
installedPlugins.plugins['claude-mem@thedotmack'] = [
{
scope: 'user',
installPath: cachePath,
version,
installedAt: now,
lastUpdated: now,
},
];
writeJsonFileAtomic(installedPluginsPath(), installedPlugins);
}
function enablePluginInClaudeSettings(): void {
const settings = readJsonSafe<Record<string, any>>(claudeSettingsPath(), {});
if (!settings.enabledPlugins) settings.enabledPlugins = {};
settings.enabledPlugins['claude-mem@thedotmack'] = true;
writeJsonFileAtomic(claudeSettingsPath(), settings);
}
// ---------------------------------------------------------------------------
// IDE setup dispatcher
// ---------------------------------------------------------------------------
/** Returns a list of IDE IDs that failed setup. */
async function setupIDEs(selectedIDEs: string[]): Promise<string[]> {
const failedIDEs: string[] = [];
for (const ideId of selectedIDEs) {
switch (ideId) {
case 'claude-code': {
// Claude Code uses its native plugin CLI — two commands handle
// marketplace registration, plugin installation, and enablement.
try {
execSync(
'claude plugin marketplace add thedotmack/claude-mem && claude plugin install claude-mem',
{ stdio: 'inherit' },
);
log.success('Claude Code: plugin installed via CLI.');
} catch {
log.error('Claude Code: plugin install failed. Is `claude` CLI on your PATH?');
failedIDEs.push(ideId);
}
break;
}
case 'cursor': {
const { installCursorHooks, configureCursorMcp } = await import('../../services/integrations/CursorHooksInstaller.js');
const cursorResult = await installCursorHooks('user');
if (cursorResult === 0) {
const mcpResult = configureCursorMcp('user');
if (mcpResult === 0) {
log.success('Cursor: hooks + MCP installed.');
} else {
log.success('Cursor: hooks installed (MCP setup failed — run `npx claude-mem cursor mcp` to retry).');
}
} else {
log.error('Cursor: hook installation failed.');
failedIDEs.push(ideId);
}
break;
}
case 'gemini-cli': {
const { installGeminiCliHooks } = await import('../../services/integrations/GeminiCliHooksInstaller.js');
const geminiResult = await installGeminiCliHooks();
if (geminiResult === 0) {
log.success('Gemini CLI: hooks installed.');
} else {
log.error('Gemini CLI: hook installation failed.');
failedIDEs.push(ideId);
}
break;
}
case 'opencode': {
const { installOpenCodeIntegration } = await import('../../services/integrations/OpenCodeInstaller.js');
const openCodeResult = await installOpenCodeIntegration();
if (openCodeResult === 0) {
log.success('OpenCode: plugin installed.');
} else {
log.error('OpenCode: plugin installation failed.');
failedIDEs.push(ideId);
}
break;
}
case 'windsurf': {
const { installWindsurfHooks } = await import('../../services/integrations/WindsurfHooksInstaller.js');
const windsurfResult = await installWindsurfHooks();
if (windsurfResult === 0) {
log.success('Windsurf: hooks installed.');
} else {
log.error('Windsurf: hook installation failed.');
failedIDEs.push(ideId);
}
break;
}
case 'openclaw': {
const { installOpenClawIntegration } = await import('../../services/integrations/OpenClawInstaller.js');
const openClawResult = await installOpenClawIntegration();
if (openClawResult === 0) {
log.success('OpenClaw: plugin installed.');
} else {
log.error('OpenClaw: plugin installation failed.');
failedIDEs.push(ideId);
}
break;
}
case 'codex-cli': {
const { installCodexCli } = await import('../../services/integrations/CodexCliInstaller.js');
const codexResult = await installCodexCli();
if (codexResult === 0) {
log.success('Codex CLI: transcript watching configured.');
} else {
log.error('Codex CLI: integration setup failed.');
failedIDEs.push(ideId);
}
break;
}
case 'copilot-cli':
case 'antigravity':
case 'goose':
case 'crush':
case 'roo-code':
case 'warp': {
const { MCP_IDE_INSTALLERS } = await import('../../services/integrations/McpIntegrations.js');
const mcpInstaller = MCP_IDE_INSTALLERS[ideId];
if (mcpInstaller) {
const mcpResult = await mcpInstaller();
const allIDEs = detectInstalledIDEs();
const ideInfo = allIDEs.find((i) => i.id === ideId);
const ideLabel = ideInfo?.label ?? ideId;
if (mcpResult === 0) {
log.success(`${ideLabel}: MCP integration installed.`);
} else {
log.error(`${ideLabel}: MCP integration failed.`);
failedIDEs.push(ideId);
}
}
break;
}
default: {
const allIDEs = detectInstalledIDEs();
const ide = allIDEs.find((i) => i.id === ideId);
if (ide && !ide.supported) {
log.warn(`Support for ${ide.label} coming soon.`);
}
break;
}
}
}
return failedIDEs;
}
// ---------------------------------------------------------------------------
// Interactive IDE selection
// ---------------------------------------------------------------------------
async function promptForIDESelection(): Promise<string[]> {
const detectedIDEs = detectInstalledIDEs();
const detected = detectedIDEs.filter((ide) => ide.detected);
if (detected.length === 0) {
log.warn('No supported IDEs detected. Installing for Claude Code by default.');
return ['claude-code'];
}
const options = detected.map((ide) => ({
value: ide.id,
label: ide.label,
hint: ide.supported ? ide.hint : 'coming soon',
}));
const result = await p.multiselect({
message: 'Which IDEs do you use?',
options,
initialValues: detected
.filter((ide) => ide.supported)
.map((ide) => ide.id),
required: true,
});
if (p.isCancel(result)) {
p.cancel('Installation cancelled.');
process.exit(0);
}
return result as string[];
}
// ---------------------------------------------------------------------------
// Core copy logic
// ---------------------------------------------------------------------------
function copyPluginToMarketplace(): void {
const marketplaceDir = marketplaceDirectory();
const packageRoot = npmPackageRootDirectory();
ensureDirectoryExists(marketplaceDir);
// Only copy directories/files that are actually needed at runtime.
// The npm package ships plugin/, package.json, node_modules/, openclaw/, dist/.
// When running from a dev checkout, the root contains many extra dirs
// (.claude, .agents, src, docs, etc.) that must NOT be copied.
const allowedTopLevelEntries = [
'plugin',
'package.json',
'package-lock.json',
'node_modules',
'openclaw',
'dist',
'LICENSE',
'README.md',
'CHANGELOG.md',
];
for (const entry of allowedTopLevelEntries) {
const sourcePath = join(packageRoot, entry);
const destPath = join(marketplaceDir, entry);
if (!existsSync(sourcePath)) continue;
// Clean replace: remove stale files from previous installs before copying
if (existsSync(destPath)) {
rmSync(destPath, { recursive: true, force: true });
}
cpSync(sourcePath, destPath, {
recursive: true,
force: true,
});
}
}
function copyPluginToCache(version: string): void {
const sourcePluginDirectory = npmPackagePluginDirectory();
const cachePath = pluginCacheDirectory(version);
// Clean replace: remove stale cache before copying
rmSync(cachePath, { recursive: true, force: true });
ensureDirectoryExists(cachePath);
cpSync(sourcePluginDirectory, cachePath, { recursive: true, force: true });
}
// ---------------------------------------------------------------------------
// npm install in marketplace dir
// ---------------------------------------------------------------------------
function runNpmInstallInMarketplace(): void {
const marketplaceDir = marketplaceDirectory();
const packageJsonPath = join(marketplaceDir, 'package.json');
if (!existsSync(packageJsonPath)) return;
execSync('npm install --production', {
cwd: marketplaceDir,
stdio: 'pipe',
...(IS_WINDOWS ? { shell: true as const } : {}),
});
}
// ---------------------------------------------------------------------------
// Trigger smart-install for Bun / uv
// ---------------------------------------------------------------------------
function runSmartInstall(): boolean {
const smartInstallPath = join(marketplaceDirectory(), 'plugin', 'scripts', 'smart-install.js');
if (!existsSync(smartInstallPath)) {
log.warn('smart-install.js not found — skipping Bun/uv auto-install.');
return false;
}
try {
execSync(`node "${smartInstallPath}"`, {
stdio: 'inherit',
...(IS_WINDOWS ? { shell: true as const } : {}),
});
return true;
} catch {
log.warn('smart-install encountered an issue. You may need to install Bun/uv manually.');
return false;
}
}
// ---------------------------------------------------------------------------
// Public API
// ---------------------------------------------------------------------------
export interface InstallOptions {
/** Unused — kept for CLI compat. IDE integrations are separate. */
/** When provided, skip the interactive IDE multi-select and use this IDE. */
ide?: string;
}
export async function runInstallCommand(_options: InstallOptions = {}): Promise<void> {
console.log(pc.bold('claude-mem install'));
console.log();
export async function runInstallCommand(options: InstallOptions = {}): Promise<void> {
const version = readPluginVersion();
try {
execSync(
'claude plugin marketplace add thedotmack/claude-mem && claude plugin install claude-mem',
{ stdio: 'inherit' },
);
} catch (error: any) {
console.error(pc.red('Installation failed.'));
console.error('Make sure Claude Code CLI is installed and on your PATH.');
process.exit(1);
if (isInteractive) {
p.intro(pc.bgCyan(pc.black(' claude-mem install ')));
} else {
console.log('claude-mem install');
}
log.info(`Version: ${pc.cyan(version)}`);
log.info(`Platform: ${process.platform} (${process.arch})`);
// Check for existing installation
const marketplaceDir = marketplaceDirectory();
const alreadyInstalled = existsSync(join(marketplaceDir, 'plugin', '.claude-plugin', 'plugin.json'));
if (alreadyInstalled) {
// Read existing version
try {
const existingPluginJson = JSON.parse(
readFileSync(join(marketplaceDir, 'plugin', '.claude-plugin', 'plugin.json'), 'utf-8'),
);
log.warn(`Existing installation detected (v${existingPluginJson.version ?? 'unknown'}).`);
} catch {
log.warn('Existing installation detected.');
}
if (process.stdin.isTTY) {
const shouldContinue = await p.confirm({
message: 'Overwrite existing installation?',
initialValue: true,
});
if (p.isCancel(shouldContinue) || !shouldContinue) {
p.cancel('Installation cancelled.');
process.exit(0);
}
}
}
console.log();
console.log(pc.green('claude-mem installed successfully!'));
console.log();
console.log('Open Claude Code and start a conversation — memory is automatic.');
// IDE selection
let selectedIDEs: string[];
if (options.ide) {
selectedIDEs = [options.ide];
const allIDEs = detectInstalledIDEs();
const match = allIDEs.find((i) => i.id === options.ide);
if (match && !match.supported) {
log.error(`Support for ${match.label} coming soon.`);
process.exit(1);
}
if (!match) {
log.error(`Unknown IDE: ${options.ide}`);
log.info(`Available IDEs: ${allIDEs.map((i) => i.id).join(', ')}`);
process.exit(1);
}
} else if (process.stdin.isTTY) {
selectedIDEs = await promptForIDESelection();
} else {
// Non-interactive: default to claude-code
selectedIDEs = ['claude-code'];
}
// Non-Claude-Code IDEs need the manual file copy / registration flow.
// Claude Code handles its own installation via `claude plugin install`.
const needsManualInstall = selectedIDEs.some((id) => id !== 'claude-code');
if (needsManualInstall) {
await runTasks([
{
title: 'Copying plugin files',
task: async (message) => {
message('Copying to marketplace directory...');
copyPluginToMarketplace();
return `Plugin files copied ${pc.green('OK')}`;
},
},
{
title: 'Caching plugin version',
task: async (message) => {
message(`Caching v${version}...`);
copyPluginToCache(version);
return `Plugin cached (v${version}) ${pc.green('OK')}`;
},
},
{
title: 'Registering marketplace',
task: async () => {
registerMarketplace();
return `Marketplace registered ${pc.green('OK')}`;
},
},
{
title: 'Registering plugin',
task: async () => {
registerPlugin(version);
return `Plugin registered ${pc.green('OK')}`;
},
},
{
title: 'Enabling plugin in Claude settings',
task: async () => {
enablePluginInClaudeSettings();
return `Plugin enabled ${pc.green('OK')}`;
},
},
{
title: 'Installing dependencies',
task: async (message) => {
message('Running npm install...');
try {
runNpmInstallInMarketplace();
return `Dependencies installed ${pc.green('OK')}`;
} catch {
return `Dependencies may need manual install ${pc.yellow('!')}`;
}
},
},
{
title: 'Setting up Bun and uv',
task: async (message) => {
message('Running smart-install...');
return runSmartInstall()
? `Runtime dependencies ready ${pc.green('OK')}`
: `Runtime setup may need attention ${pc.yellow('!')}`;
},
},
]);
}
// IDE-specific setup
const failedIDEs = await setupIDEs(selectedIDEs);
// Summary
const installStatus = failedIDEs.length > 0 ? 'Installation Partial' : 'Installation Complete';
const summaryLines = [
`Version: ${pc.cyan(version)}`,
`Plugin dir: ${pc.cyan(marketplaceDir)}`,
`IDEs: ${pc.cyan(selectedIDEs.join(', '))}`,
];
if (failedIDEs.length > 0) {
summaryLines.push(`Failed: ${pc.red(failedIDEs.join(', '))}`);
}
if (isInteractive) {
p.note(summaryLines.join('\n'), installStatus);
} else {
console.log(`\n ${installStatus}`);
summaryLines.forEach(l => console.log(` ${l}`));
}
const workerPort = process.env.CLAUDE_MEM_WORKER_PORT || '37777';
const nextSteps = [
'Open Claude Code and start a conversation -- memory is automatic!',
`View your memories: ${pc.underline(`http://localhost:${workerPort}`)}`,
`Search past work: use ${pc.bold('/mem-search')} in Claude Code`,
`Start worker: ${pc.bold('npx claude-mem start')}`,
];
if (isInteractive) {
p.note(nextSteps.join('\n'), 'Next Steps');
if (failedIDEs.length > 0) {
p.outro(pc.yellow('claude-mem installed with some IDE setup failures.'));
} else {
p.outro(pc.green('claude-mem installed successfully!'));
}
} else {
console.log('\n Next Steps');
nextSteps.forEach(l => console.log(` ${l}`));
if (failedIDEs.length > 0) {
console.log('\nclaude-mem installed with some IDE setup failures.');
process.exitCode = 1;
} else {
console.log('\nclaude-mem installed successfully!');
}
}
}
+1 -1
View File
@@ -75,7 +75,7 @@ export function parseObservations(text: string, correlationId?: string): ParsedO
const cleanedConcepts = concepts.filter(c => c !== finalType);
if (cleanedConcepts.length !== concepts.length) {
logger.error('PARSER', 'Removed observation type from concepts array', {
logger.debug('PARSER', 'Removed observation type from concepts array', {
correlationId,
type: finalType,
originalConcepts: concepts,
+6 -2
View File
@@ -116,7 +116,11 @@ export function buildObservationPrompt(obs: Observation): string {
<occurred_at>${new Date(obs.created_at_epoch).toISOString()}</occurred_at>${obs.cwd ? `\n <working_directory>${obs.cwd}</working_directory>` : ''}
<parameters>${JSON.stringify(toolInput, null, 2)}</parameters>
<outcome>${JSON.stringify(toolOutput, null, 2)}</outcome>
</observed_from_primary_session>`;
</observed_from_primary_session>
Return either one or more <observation>...</observation> blocks, or an empty response if this tool use should be skipped.
Concrete debugging findings from logs, queue state, database rows, session routing, or code-path inspection count as durable discoveries and should be recorded.
Never reply with prose such as "Skipping", "No substantive tool executions", or any explanation outside XML. Non-XML text is discarded.`;
}
/**
@@ -235,4 +239,4 @@ ${mode.prompts.format_examples}
${mode.prompts.footer}
${mode.prompts.header_memory_continued}`;
}
}
+35 -8
View File
@@ -10,6 +10,7 @@
*/
import path from 'path';
import net from 'net';
import { readFileSync } from 'fs';
import { logger } from '../../utils/logger.js';
import { MARKETPLACE_ROOT } from '../../shared/paths.js';
@@ -35,17 +36,43 @@ async function httpRequestToWorker(
}
/**
* Check if a port is in use by querying the health endpoint
* Check if a port is in use by attempting an atomic socket bind.
* More reliable than HTTP health check for daemon spawn guards
* prevents TOCTOU race where two daemons both see "port free" via
* HTTP and then both try to listen() (upstream bug workaround).
*
* Falls back to HTTP health check on Windows where socket bind
* behavior differs.
*/
export async function isPortInUse(port: number): Promise<boolean> {
try {
// Note: Removed AbortSignal.timeout to avoid Windows Bun cleanup issue (libuv assertion)
const response = await fetch(`http://127.0.0.1:${port}/api/health`);
return response.ok;
} catch (error) {
// [ANTI-PATTERN IGNORED]: Health check polls every 500ms, logging would flood
return false;
if (process.platform === 'win32') {
// APPROVED OVERRIDE: Windows keeps HTTP health check because socket bind
// semantics differ (SO_REUSEADDR defaults, firewall prompts). The TOCTOU
// race remains on Windows but is an accepted limitation — the atomic
// socket approach would cause false positives or UAC popups.
try {
const response = await fetch(`http://127.0.0.1:${port}/api/health`);
return response.ok;
} catch {
return false;
}
}
// Unix: atomic socket bind check — no TOCTOU race
return new Promise((resolve) => {
const server = net.createServer();
server.once('error', (err: NodeJS.ErrnoException) => {
if (err.code === 'EADDRINUSE') {
resolve(true);
} else {
resolve(false);
}
});
server.once('listening', () => {
server.close(() => resolve(false));
});
server.listen(port, '127.0.0.1');
});
}
/**
@@ -397,6 +397,19 @@ export class PendingMessageStore {
return result.count;
}
/**
* Peek at pending message types for a session (for tier routing).
* Returns list of { message_type, tool_name } without claiming.
*/
peekPendingTypes(sessionDbId: number): Array<{ message_type: string; tool_name: string | null }> {
const stmt = this.db.prepare(`
SELECT message_type, tool_name FROM pending_messages
WHERE session_db_id = ? AND status IN ('pending', 'processing')
ORDER BY id ASC
`);
return stmt.all(sessionDbId) as Array<{ message_type: string; tool_name: string | null }>;
}
/**
* Check if any session has pending work.
* Excludes 'processing' messages stuck for >5 minutes (resets them to 'pending' as a side effect).
+34 -1
View File
@@ -509,6 +509,38 @@ export const migration007: Migration = {
};
/**
* All migrations in order
*/
/**
* Migration 008: Observation feedback table for tracking observation usage
*
* Tracks how observations are used (semantic injection hits, search access,
* explicit retrieval). Foundation for future Thompson Sampling optimization.
*/
export const migration008: Migration = {
version: 25,
up: (db: Database) => {
db.run(`
CREATE TABLE IF NOT EXISTS observation_feedback (
id INTEGER PRIMARY KEY AUTOINCREMENT,
observation_id INTEGER NOT NULL,
signal_type TEXT NOT NULL,
session_db_id INTEGER,
created_at_epoch INTEGER NOT NULL,
metadata TEXT,
FOREIGN KEY (observation_id) REFERENCES observations(id) ON DELETE CASCADE
)
`);
db.run(`CREATE INDEX IF NOT EXISTS idx_feedback_observation ON observation_feedback(observation_id)`);
db.run(`CREATE INDEX IF NOT EXISTS idx_feedback_signal ON observation_feedback(signal_type)`);
console.log('✅ Created observation_feedback table for usage tracking');
},
down: (db: Database) => {
db.run(`DROP TABLE IF EXISTS observation_feedback`);
}
};
/**
* All migrations in order
*/
@@ -519,5 +551,6 @@ export const migrations: Migration[] = [
migration004,
migration005,
migration006,
migration007
migration007,
migration008
];
+28
View File
@@ -34,6 +34,7 @@ export class MigrationRunner {
this.addOnUpdateCascadeToForeignKeys();
this.addObservationContentHashColumn();
this.addSessionCustomTitleColumn();
this.createObservationFeedbackTable();
}
/**
@@ -863,4 +864,31 @@ export class MigrationRunner {
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(23, new Date().toISOString());
}
/**
* Create observation_feedback table for tracking observation usage signals.
* Foundation for tier routing optimization and future Thompson Sampling.
* Schema version 24.
*/
private createObservationFeedbackTable(): void {
const applied = this.db.query('SELECT 1 FROM schema_versions WHERE version = 24').get();
if (applied) return;
this.db.run(`
CREATE TABLE IF NOT EXISTS observation_feedback (
id INTEGER PRIMARY KEY AUTOINCREMENT,
observation_id INTEGER NOT NULL,
signal_type TEXT NOT NULL,
session_db_id INTEGER,
created_at_epoch INTEGER NOT NULL,
metadata TEXT,
FOREIGN KEY (observation_id) REFERENCES observations(id) ON DELETE CASCADE
)
`);
this.db.run('CREATE INDEX IF NOT EXISTS idx_feedback_observation ON observation_feedback(observation_id)');
this.db.run('CREATE INDEX IF NOT EXISTS idx_feedback_signal ON observation_feedback(signal_type)');
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(24, new Date().toISOString());
logger.debug('DB', 'Created observation_feedback table for usage tracking');
}
}
+35 -5
View File
@@ -283,11 +283,41 @@ export class ChromaSync {
metadatas: cleanMetadatas
});
} catch (error) {
logger.error('CHROMA_SYNC', 'Batch add failed, continuing with remaining batches', {
collection: this.collectionName,
batchStart: i,
batchSize: batch.length
}, error as Error);
const errMsg = error instanceof Error ? error.message : String(error);
// APPROVED OVERRIDE: Duplicate IDs from partial write before timeout/crash.
// chroma_update_documents only updates *existing* IDs — it silently ignores
// missing ones. So we delete-then-add to guarantee all IDs are written.
if (errMsg.includes('already exist')) {
try {
await chromaMcp.callTool('chroma_delete_documents', {
collection_name: this.collectionName,
ids: batch.map(d => d.id)
});
await chromaMcp.callTool('chroma_add_documents', {
collection_name: this.collectionName,
ids: batch.map(d => d.id),
documents: batch.map(d => d.document),
metadatas: cleanMetadatas
});
logger.info('CHROMA_SYNC', 'Batch reconciled via delete+add after duplicate conflict', {
collection: this.collectionName,
batchStart: i,
batchSize: batch.length
});
} catch (reconcileError) {
logger.error('CHROMA_SYNC', 'Batch reconcile (delete+add) failed', {
collection: this.collectionName,
batchStart: i,
batchSize: batch.length
}, reconcileError as Error);
}
} else {
logger.error('CHROMA_SYNC', 'Batch add failed, continuing with remaining batches', {
collection: this.collectionName,
batchStart: i,
batchSize: batch.length
}, error as Error);
}
}
}
+2
View File
@@ -40,6 +40,8 @@ export interface ActiveSession {
// CLAIM-CONFIRM FIX: Track IDs of messages currently being processed
// These IDs will be confirmed (deleted) after successful storage
processingMessageIds: number[];
// Tier routing: model override per session based on queue complexity
modelOverride?: string;
}
export interface PendingMessage {
+2 -2
View File
@@ -49,8 +49,8 @@ export class SDKAgent {
// Find Claude executable
const claudePath = this.findClaudeExecutable();
// Get model ID and disallowed tools
const modelId = this.getModelId();
// Get model ID (tier routing override takes precedence)
const modelId = session.modelOverride || this.getModelId();
// Memory agent is OBSERVER ONLY - no tools allowed
const disallowedTools = [
'Bash', // Prevent infinite loops
@@ -68,6 +68,19 @@ export async function processAgentResponse(
const observations = parseObservations(text, session.contentSessionId);
const summary = parseSummary(text, session.sessionDbId);
if (
text.trim() &&
observations.length === 0 &&
!summary &&
!/<observation>|<summary>|<skip_summary\b/.test(text)
) {
const preview = text.length > 200 ? `${text.slice(0, 200)}...` : text;
logger.warn('PARSER', `${agentName} returned non-XML response; observation content was discarded`, {
sessionId: session.sessionDbId,
preview
});
}
// Convert nullable fields to empty strings for storeSummary (if summary exists)
const summaryForStore = normalizeSummaryForStorage(summary);
@@ -38,6 +38,7 @@ export class SearchRoutes extends BaseRouteHandler {
app.get('/api/context/timeline', this.handleGetContextTimeline.bind(this));
app.get('/api/context/preview', this.handleContextPreview.bind(this));
app.get('/api/context/inject', this.handleContextInject.bind(this));
app.post('/api/context/semantic', this.handleSemanticContext.bind(this));
// Timeline and help endpoints
app.get('/api/timeline/by-query', this.handleGetTimelineByQuery.bind(this));
@@ -246,6 +247,54 @@ export class SearchRoutes extends BaseRouteHandler {
res.send(contextText);
});
/**
* Semantic context search for per-prompt injection
* POST /api/context/semantic { q, project?, limit? }
*
* Queries Chroma for observations semantically similar to the user's prompt.
* Returns compact markdown for injection as additionalContext.
*/
private handleSemanticContext = this.wrapHandler(async (req: Request, res: Response): Promise<void> => {
const query = (req.body?.q || req.query.q) as string;
const project = (req.body?.project || req.query.project) as string;
const limit = Math.min(Math.max(parseInt(String(req.body?.limit || req.query.limit || '5'), 10) || 5, 1), 20);
if (!query || query.length < 20) {
res.json({ context: '', count: 0 });
return;
}
try {
const result = await this.searchManager.search({
query,
type: 'observations',
project,
limit: String(limit),
format: 'json'
});
const observations = (result as any)?.observations || [];
if (!observations.length) {
res.json({ context: '', count: 0 });
return;
}
// Format as compact markdown for context injection
const lines: string[] = ['## Relevant Past Work (semantic match)\n'];
for (const obs of observations.slice(0, limit)) {
const date = obs.created_at?.slice(0, 10) || '';
lines.push(`### ${obs.title || 'Observation'} (${date})`);
if (obs.narrative) lines.push(obs.narrative);
lines.push('');
}
res.json({ context: lines.join('\n'), count: observations.length });
} catch (error) {
logger.error('SEARCH', 'Semantic context query failed', {}, error as Error);
res.json({ context: '', count: 0 });
}
});
/**
* Get timeline by query (search first, then get timeline around best match)
* GET /api/timeline/by-query?query=...&mode=auto&depth_before=10&depth_after=10
@@ -106,6 +106,8 @@ export class SessionRoutes extends BaseRouteHandler {
// Start generator if not running
if (!session.generatorPromise) {
// Apply tier routing before starting the generator
this.applyTierRouting(session);
this.spawnInProgress.set(sessionDbId, true);
this.startGeneratorWithProvider(session, selectedProvider, source);
return;
@@ -126,6 +128,7 @@ export class SessionRoutes extends BaseRouteHandler {
session.abortController = new AbortController();
session.lastGeneratorActivity = Date.now();
// Start a fresh generator
this.applyTierRouting(session);
this.spawnInProgress.set(sessionDbId, true);
this.startGeneratorWithProvider(session, selectedProvider, 'stale-recovery');
return;
@@ -283,6 +286,7 @@ export class SessionRoutes extends BaseRouteHandler {
this.crashRecoveryScheduled.delete(sessionDbId);
const stillExists = this.sessionManager.getSession(sessionDbId);
if (stillExists && !stillExists.generatorPromise) {
this.applyTierRouting(stillExists);
this.startGeneratorWithProvider(stillExists, this.getSelectedProvider(), 'crash-recovery');
}
}, backoffMs);
@@ -813,4 +817,60 @@ export class SessionRoutes extends BaseRouteHandler {
contextInjected
});
});
// Simple tool names that produce low-complexity observations
private static readonly SIMPLE_TOOLS = new Set([
'Read', 'Glob', 'Grep', 'LS', 'ListMcpResourcesTool'
]);
/**
* Apply tier routing: select model based on pending queue complexity.
* - Summarize in queue summary model (e.g., Opus)
* - All simple tools simple model (e.g., Haiku)
* - Otherwise default model (no override)
*/
private applyTierRouting(session: NonNullable<ReturnType<typeof this.sessionManager.getSession>>): void {
const settings = SettingsDefaultsManager.loadFromFile(USER_SETTINGS_PATH);
if (settings.CLAUDE_MEM_TIER_ROUTING_ENABLED === 'false') {
session.modelOverride = undefined;
return;
}
// Clear stale override before re-evaluating — prevents previous tier
// from persisting when queue composition changes between spawns.
session.modelOverride = undefined;
const pendingStore = this.sessionManager.getPendingMessageStore();
const pending = pendingStore.peekPendingTypes(session.sessionDbId);
if (pending.length === 0) {
session.modelOverride = undefined;
return;
}
const hasSummarize = pending.some(m => m.message_type === 'summarize');
const allSimple = pending.every(m =>
m.message_type === 'observation' && m.tool_name && SessionRoutes.SIMPLE_TOOLS.has(m.tool_name)
);
if (hasSummarize) {
const summaryModel = settings.CLAUDE_MEM_TIER_SUMMARY_MODEL;
if (summaryModel) {
session.modelOverride = summaryModel;
logger.debug('SESSION', `Tier routing: summary model`, {
sessionId: session.sessionDbId, model: summaryModel
});
}
} else if (allSimple) {
const simpleModel = settings.CLAUDE_MEM_TIER_SIMPLE_MODEL;
if (simpleModel) {
session.modelOverride = simpleModel;
logger.debug('SESSION', `Tier routing: simple model`, {
sessionId: session.sessionDbId, model: simpleModel
});
}
} else {
session.modelOverride = undefined;
}
}
}
@@ -24,9 +24,30 @@ export class SessionCompletionHandler {
* Used by DELETE /api/sessions/:id and POST /api/sessions/:id/complete
*/
async completeByDbId(sessionDbId: number): Promise<void> {
// Delete from session manager (aborts SDK agent)
// Delete from session manager (aborts SDK agent via SIGTERM)
await this.sessionManager.deleteSession(sessionDbId);
// Drain orphaned pending messages left by SIGTERM.
// When deleteSession() aborts the generator, pending messages in the queue
// are never processed. Without drain, they stay in 'pending' status forever
// since no future generator will pick them up for a completed session.
// Note: this is best-effort — if a generator outlives the 30s SIGTERM timeout
// (SessionManager.deleteSession), it may enqueue messages after this drain.
// In practice this race is rare (zero orphans over 23 days, 3400+ observations).
try {
const pendingStore = this.sessionManager.getPendingMessageStore();
const drainedCount = pendingStore.markAllSessionMessagesAbandoned(sessionDbId);
if (drainedCount > 0) {
logger.warn('SESSION', `Drained ${drainedCount} orphaned pending messages on session completion`, {
sessionId: sessionDbId, drainedCount
});
}
} catch (e) {
logger.debug('SESSION', 'Failed to drain pending queue on session completion', {
sessionId: sessionDbId, error: e instanceof Error ? e.message : String(e)
});
}
// Broadcast session completed event
this.eventBroadcaster.broadcastSessionCompleted(sessionDbId);
}
+14
View File
@@ -54,6 +54,13 @@ export interface SettingsDefaults {
// Exclusion Settings
CLAUDE_MEM_EXCLUDED_PROJECTS: string; // Comma-separated glob patterns for excluded project paths
CLAUDE_MEM_FOLDER_MD_EXCLUDE: string; // JSON array of folder paths to exclude from CLAUDE.md generation
// Semantic Context Injection (per-prompt via Chroma)
CLAUDE_MEM_SEMANTIC_INJECT: string; // 'true' | 'false' - inject relevant observations on each prompt
CLAUDE_MEM_SEMANTIC_INJECT_LIMIT: string; // Max observations to inject per prompt
// Tier Routing (model selection by queue complexity)
CLAUDE_MEM_TIER_ROUTING_ENABLED: string; // 'true' | 'false' - enable model tier routing
CLAUDE_MEM_TIER_SIMPLE_MODEL: string; // Tier alias or model ID for simple tool observations (Read, Glob, Grep)
CLAUDE_MEM_TIER_SUMMARY_MODEL: string; // Tier alias or model ID for session summaries
// Chroma Vector Database Configuration
CLAUDE_MEM_CHROMA_ENABLED: string; // 'true' | 'false' - set to 'false' for SQLite-only mode
CLAUDE_MEM_CHROMA_MODE: string; // 'local' | 'remote'
@@ -113,6 +120,13 @@ export class SettingsDefaultsManager {
// Exclusion Settings
CLAUDE_MEM_EXCLUDED_PROJECTS: '', // Comma-separated glob patterns for excluded project paths
CLAUDE_MEM_FOLDER_MD_EXCLUDE: '[]', // JSON array of folder paths to exclude from CLAUDE.md generation
// Semantic Context Injection (per-prompt via Chroma vector search)
CLAUDE_MEM_SEMANTIC_INJECT: 'false', // Inject relevant past observations on every UserPromptSubmit (experimental, disabled by default)
CLAUDE_MEM_SEMANTIC_INJECT_LIMIT: '5', // Top-N most relevant observations to inject per prompt
// Tier Routing (model selection by queue complexity)
CLAUDE_MEM_TIER_ROUTING_ENABLED: 'true', // Route observations to models by complexity
CLAUDE_MEM_TIER_SIMPLE_MODEL: 'haiku', // Portable tier alias — works across Direct API, Bedrock, Vertex, Azure (see #1463)
CLAUDE_MEM_TIER_SUMMARY_MODEL: '', // Empty = use default model for summaries
// Chroma Vector Database Configuration
CLAUDE_MEM_CHROMA_ENABLED: 'true', // Set to 'false' to disable Chroma and use SQLite-only search
CLAUDE_MEM_CHROMA_MODE: 'local', // 'local' uses persistent chroma-mcp via uvx, 'remote' connects to existing server
+93 -38
View File
@@ -1,4 +1,5 @@
import { describe, it, expect, beforeEach, afterEach, mock } from 'bun:test';
import { describe, it, expect, beforeEach, afterEach, mock, spyOn } from 'bun:test';
import net from 'net';
import {
isPortInUse,
waitForHealth,
@@ -15,45 +16,73 @@ describe('HealthMonitor', () => {
});
describe('isPortInUse', () => {
it('should return true for occupied port (health check succeeds)', async () => {
global.fetch = mock(() => Promise.resolve({ ok: true } as Response));
// Note: Since we are on Linux (as per session_context), isPortInUse uses 'net'
// instead of 'fetch'. We need to mock 'net.createServer().listen()'
it('should return true for occupied port (EADDRINUSE)', async () => {
// Create a specific mock for this test
const createServerMock = mock(() => ({
once: mock((event: string, cb: Function) => {
if (event === 'error') {
// Trigger EADDRINUSE immediately
setTimeout(() => cb({ code: 'EADDRINUSE' }), 0);
}
}),
listen: mock(() => {})
}));
const spy = spyOn(net, 'createServer').mockImplementation(createServerMock as any);
const result = await isPortInUse(37777);
expect(result).toBe(true);
expect(global.fetch).toHaveBeenCalledWith('http://127.0.0.1:37777/api/health');
expect(net.createServer).toHaveBeenCalled();
spy.mockRestore();
});
it('should return false for free port (connection refused)', async () => {
global.fetch = mock(() => Promise.reject(new Error('ECONNREFUSED')));
it('should return false for free port (listening succeeds)', async () => {
const closeMock = mock((cb: Function) => cb());
const createServerMock = mock(() => ({
once: mock((event: string, cb: Function) => {
if (event === 'listening') {
// Trigger listening success
setTimeout(() => cb(), 0);
}
}),
listen: mock(() => {}),
close: closeMock
}));
const spy = spyOn(net, 'createServer').mockImplementation(createServerMock as any);
const result = await isPortInUse(39999);
expect(result).toBe(false);
expect(net.createServer).toHaveBeenCalled();
expect(closeMock).toHaveBeenCalled();
spy.mockRestore();
});
it('should return false when health check returns non-ok', async () => {
global.fetch = mock(() => Promise.resolve({ ok: false, status: 503 } as Response));
const result = await isPortInUse(37777);
expect(result).toBe(false);
});
it('should return false on network timeout', async () => {
global.fetch = mock(() => Promise.reject(new Error('ETIMEDOUT')));
const result = await isPortInUse(37777);
expect(result).toBe(false);
});
it('should return false on fetch failed error', async () => {
global.fetch = mock(() => Promise.reject(new Error('fetch failed')));
it('should return false for other socket errors', async () => {
const createServerMock = mock(() => ({
once: mock((event: string, cb: Function) => {
if (event === 'error') {
// Trigger other error (e.g., EACCES)
setTimeout(() => cb({ code: 'EACCES' }), 0);
}
}),
listen: mock(() => {})
}));
const spy = spyOn(net, 'createServer').mockImplementation(createServerMock as any);
const result = await isPortInUse(37777);
expect(result).toBe(false);
spy.mockRestore();
});
});
@@ -203,54 +232,80 @@ describe('HealthMonitor', () => {
describe('waitForPortFree', () => {
it('should return true immediately when port is already free', async () => {
global.fetch = mock(() => Promise.reject(new Error('ECONNREFUSED')));
const createServerMock = mock(() => ({
once: mock((event: string, cb: Function) => {
if (event === 'listening') setTimeout(() => cb(), 0);
}),
listen: mock(() => {}),
close: mock((cb: Function) => cb())
}));
const spy = spyOn(net, 'createServer').mockImplementation(createServerMock as any);
const start = Date.now();
const result = await waitForPortFree(39999, 5000);
const elapsed = Date.now() - start;
expect(result).toBe(true);
// Should return quickly
expect(elapsed).toBeLessThan(1000);
spy.mockRestore();
});
it('should timeout when port remains occupied', async () => {
global.fetch = mock(() => Promise.resolve({ ok: true } as Response));
const createServerMock = mock(() => ({
once: mock((event: string, cb: Function) => {
if (event === 'error') setTimeout(() => cb({ code: 'EADDRINUSE' }), 0);
}),
listen: mock(() => {})
}));
const spy = spyOn(net, 'createServer').mockImplementation(createServerMock as any);
const start = Date.now();
const result = await waitForPortFree(37777, 1500);
const elapsed = Date.now() - start;
expect(result).toBe(false);
// Should take close to timeout duration
expect(elapsed).toBeGreaterThanOrEqual(1400);
expect(elapsed).toBeLessThan(2500);
spy.mockRestore();
});
it('should succeed when port becomes free', async () => {
let callCount = 0;
global.fetch = mock(() => {
callCount++;
// Port occupied for first 2 checks, then free
if (callCount < 3) {
return Promise.resolve({ ok: true } as Response);
}
return Promise.reject(new Error('ECONNREFUSED'));
});
const spy = spyOn(net, 'createServer').mockImplementation(() => ({
once: mock((event: string, cb: Function) => {
callCount++;
// Port occupied for first 2 checks, then free
if (callCount < 3) {
if (event === 'error') setTimeout(() => cb({ code: 'EADDRINUSE' }), 0);
} else {
if (event === 'listening') setTimeout(() => cb(), 0);
}
}),
listen: mock(() => {}),
close: mock((cb: Function) => cb())
} as any));
const result = await waitForPortFree(37777, 5000);
expect(result).toBe(true);
expect(callCount).toBeGreaterThanOrEqual(3);
spy.mockRestore();
});
it('should use default timeout when not specified', async () => {
global.fetch = mock(() => Promise.reject(new Error('ECONNREFUSED')));
const createServerMock = mock(() => ({
once: mock((event: string, cb: Function) => {
if (event === 'listening') setTimeout(() => cb(), 0);
}),
listen: mock(() => {}),
close: mock((cb: Function) => cb())
}));
const spy = spyOn(net, 'createServer').mockImplementation(createServerMock as any);
// Just verify it doesn't throw and returns quickly
const result = await waitForPortFree(39999);
expect(result).toBe(true);
spy.mockRestore();
});
});
});
+20
View File
@@ -0,0 +1,20 @@
import { describe, expect, it } from 'bun:test';
import { buildObservationPrompt } from '../../src/sdk/prompts.js';
describe('buildObservationPrompt', () => {
it('instructs the observer to avoid prose skip responses', () => {
const prompt = buildObservationPrompt({
id: 1,
tool_name: 'exec_command',
tool_input: JSON.stringify({ cmd: 'pwd' }),
tool_output: JSON.stringify({ output: '/repo' }),
created_at_epoch: Date.now(),
cwd: '/repo',
});
expect(prompt).toContain('Return either one or more <observation>...</observation> blocks, or an empty response');
expect(prompt).toContain('Concrete debugging findings from logs, queue state, database rows, session routing, or code-path inspection');
expect(prompt).toContain('Never reply with prose such as "Skipping", "No substantive tool executions"');
});
});
@@ -212,6 +212,36 @@ describe('ResponseProcessor', () => {
});
});
describe('non-XML observer responses', () => {
it('warns when the observer returns prose that will be discarded', async () => {
const session = createMockSession();
const responseText = 'Skipping — repeated log scan with no new findings.';
await processAgentResponse(
responseText,
session,
mockDbManager,
mockSessionManager,
mockWorker,
100,
null,
'TestAgent'
);
expect(logger.warn).toHaveBeenCalledWith(
'PARSER',
'TestAgent returned non-XML response; observation content was discarded',
expect.objectContaining({
sessionId: 1,
preview: responseText
})
);
const [, , observations, summary] = mockStoreObservations.mock.calls[0];
expect(observations).toHaveLength(0);
expect(summary).toBeNull();
});
});
describe('parsing summary from XML response', () => {
it('should parse summary from response', async () => {
const session = createMockSession();