Compare commits
48 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 50d1dfb7ee | |||
| 0b034af98b | |||
| b43ad00f8b | |||
| dd1b812443 | |||
| ad3d236cec | |||
| 494f681cbf | |||
| 4144010264 | |||
| d482f3ed76 | |||
| 3c4486e69e | |||
| e0fec4bad7 | |||
| f5a873ffdc | |||
| 23f30d35b9 | |||
| c6f932988a | |||
| d9a30cc7d4 | |||
| 50eeed97e7 | |||
| 5f28550551 | |||
| a6a843f871 | |||
| 2db9d0e383 | |||
| 0a26bb18bf | |||
| bd11ccf12e | |||
| c2c3e3069c | |||
| 7966c6cba9 | |||
| e4e735d3ff | |||
| 780cc3894e | |||
| 8d46c00dd8 | |||
| 4ab601fc9f | |||
| 097035de6c | |||
| e788fd3676 | |||
| 44cdbec173 | |||
| 91b48a6481 | |||
| 40daf8f3fa | |||
| 7e57b6e02d | |||
| ea683a4e6c | |||
| 5d79bb7a7a | |||
| 2180d31ee6 | |||
| 75dd8e3174 | |||
| 149f548667 | |||
| b88251bc8b | |||
| b2e3a7e668 | |||
| b1cfc85333 | |||
| ca8421611c | |||
| eea4f599c0 | |||
| b446f2630e | |||
| 224567f980 | |||
| 2b31792f06 | |||
| 613f0e9795 | |||
| f1162ed4a4 | |||
| 8039ada222 |
@@ -10,7 +10,7 @@
|
||||
"plugins": [
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "10.2.1",
|
||||
"version": "10.4.4",
|
||||
"source": "./plugin",
|
||||
"description": "Persistent memory system for Claude Code - context compression across sessions"
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "9.0.6",
|
||||
"version": "10.4.1",
|
||||
"description": "Persistent memory system for Claude Code - seamlessly preserve context across sessions",
|
||||
"author": {
|
||||
"name": "Alex Newman"
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
datasets/
|
||||
node_modules/
|
||||
dist/
|
||||
!installer/dist/
|
||||
*.log
|
||||
.DS_Store
|
||||
.env
|
||||
|
||||
@@ -0,0 +1,736 @@
|
||||
# Plan: NPX Distribution + Universal IDE/CLI Coverage for claude-mem
|
||||
|
||||
## Problem
|
||||
|
||||
1. **Installation is slow and fragile**: Current install clones the full git repo, runs `npm install`, and builds from source. The npm package already ships pre-built artifacts.
|
||||
|
||||
2. **IDE coverage is limited**: claude-mem only supports Claude Code (plugin) and Cursor (hooks installer). The AI coding tools landscape has exploded — Gemini CLI (95k stars), OpenCode (110k stars), Windsurf (~1M users), Codex CLI, Antigravity, Goose, Crush, Copilot CLI, and more all support extensibility.
|
||||
|
||||
## Key Insights
|
||||
|
||||
- **npm package already has everything**: `plugin/` directory ships pre-built. No git clone or build needed.
|
||||
- **Transcript watcher already exists**: `src/services/transcripts/` has a fully built schema-based JSONL tailer. It just needs schemas for more tools.
|
||||
- **3 integration tiers exist**: (1) Hook/plugin-based (Claude Code, Gemini CLI, OpenCode, Windsurf, Codex CLI, OpenClaw), (2) MCP-based (Cursor, Copilot CLI, Antigravity, Goose, Crush, Roo Code), (3) Transcript-based (anything with structured log files).
|
||||
- **OpenClaw plugin already built**: Full plugin at `openclaw/src/index.ts` (1000+ lines). Needs to be wired into the npx installer.
|
||||
- **Gemini CLI is architecturally near-identical to Claude Code**: 11 lifecycle hooks, JSON via stdin/stdout, exit code 0/2 convention, `GEMINI.md` context files, `~/.gemini/settings.json`. This is the easiest high-value integration.
|
||||
- **OpenCode has the richest plugin system**: 20+ hook events across 12 categories, JS/TS plugin modules, custom tool creation, MCP support. 110k stars — largest open-source AI CLI.
|
||||
- **`npx skills` by Vercel supports 41 agents** — proving the multi-IDE installer UX works. Their agent detection pattern (check if config dir exists) is the right model.
|
||||
- **All IDEs share a single worker on port 37777**: One worker serves all integrations. Session source (which IDE) is tracked via the `source` field in hook payloads. No per-IDE worker instances.
|
||||
- **This npx CLI fully replaces the old `claude-mem-installer`**: Not a supplement — the complete replacement.
|
||||
|
||||
## Solution
|
||||
|
||||
`npx claude-mem` becomes a unified CLI: install, configure any IDE, manage the worker, search memory.
|
||||
|
||||
```
|
||||
npx claude-mem # Interactive install + IDE selection
|
||||
npx claude-mem install # Same as above
|
||||
npx claude-mem install --ide windsurf # Direct IDE setup
|
||||
npx claude-mem start / stop / status # Worker management
|
||||
npx claude-mem search <query> # Search memory from terminal
|
||||
npx claude-mem transcript watch # Start transcript watcher
|
||||
```
|
||||
|
||||
## Platform Support
|
||||
|
||||
**Windows, macOS, and Linux are all first-class targets.** Platform-specific considerations:
|
||||
|
||||
- **Config paths**: Use `os.homedir()` and `path.join()` everywhere — never hardcode `/` or `~`
|
||||
- **Shebangs**: `#!/usr/bin/env node` for the CLI entry point (cross-platform via Node)
|
||||
- **Bun detection**: Check `PATH`, common install locations per platform (`%USERPROFILE%\.bun\bin\bun.exe` on Windows, `~/.bun/bin/bun` on Unix)
|
||||
- **File permissions**: `fs.chmod` is a no-op on Windows; don't gate on it
|
||||
- **Process management**: Worker start/stop uses signals on Unix, taskkill on Windows — match existing `worker-service.ts` patterns
|
||||
- **VS Code paths**: `~/Library/Application Support/Code/` (macOS), `~/.config/Code/` (Linux), `%APPDATA%/Code/` (Windows)
|
||||
- **Shell config**: `.bashrc`/`.zshrc` on Unix, PowerShell profile on Windows (for PATH modifications if needed)
|
||||
|
||||
---
|
||||
|
||||
## Phase 0: Research Findings
|
||||
|
||||
### IDE Integration Tiers
|
||||
|
||||
**Tier 1 — Native Hook/Plugin Systems** (highest fidelity, real-time capture):
|
||||
|
||||
| Tool | Hooks | Config Location | Context Injection | Stars/Users |
|
||||
|------|-------|----------------|-------------------|-------------|
|
||||
| Claude Code | 5 lifecycle hooks | `~/.claude/settings.json` | CLAUDE.md, plugins | ~25% market |
|
||||
| Gemini CLI | 11 lifecycle hooks | `~/.gemini/settings.json` | GEMINI.md | ~95k stars |
|
||||
| OpenCode | 20+ event hooks + plugin SDK | `~/.config/opencode/opencode.json` | AGENTS.md + rules dirs | ~110k stars |
|
||||
| Windsurf | 11 Cascade hooks | `.windsurf/hooks.json` | `.windsurf/rules/*.md` | ~1M users |
|
||||
| Codex CLI | `notify` hook | `~/.codex/config.toml` | `.codex/AGENTS.md`, MCP | Growing (OpenAI) |
|
||||
| OpenClaw | 8 event hooks + plugin SDK | `~/.openclaw/openclaw.json` | MEMORY.md sync | ~196k stars |
|
||||
|
||||
**Tier 2 — MCP Integration** (tool-based, search + context injection):
|
||||
|
||||
| Tool | MCP Support | Config Location | Context Injection |
|
||||
|------|------------|----------------|-------------------|
|
||||
| Cursor | First-class | `.cursor/mcp.json` | `.cursor/rules/*.mdc` |
|
||||
| Copilot CLI | First-class (default MCP) | `~/.copilot/config` | `.github/copilot-instructions.md` |
|
||||
| Antigravity | First-class + MCP Store | `~/.gemini/antigravity/mcp_config.json` | `.agent/rules/`, GEMINI.md |
|
||||
| Goose | Native MCP (co-developed protocol) | `~/.config/goose/config.yaml` | MCP context |
|
||||
| Crush | MCP + Skills | JSON config (charm.land schema) | Skills system |
|
||||
| Roo Code | First-class | `.roo/` | `.roo/rules/*.md`, `AGENTS.md` |
|
||||
| Warp | MCP + Warp Drive | `WARP.md` + Warp Drive UI | `WARP.md` |
|
||||
|
||||
**Tier 3 — Transcript File Watching** (passive, file-based):
|
||||
|
||||
| Tool | Transcript Location | Format |
|
||||
|------|-------------------|--------|
|
||||
| Claude Code | `~/.claude/projects/<proj>/<session>.jsonl` | JSONL |
|
||||
| Codex CLI | `~/.codex/sessions/**/*.jsonl` | JSONL |
|
||||
| Gemini CLI | `~/.gemini/tmp/<hash>/chats/` | JSON |
|
||||
| OpenCode | `.opencode/` (SQLite) | SQLite — needs export |
|
||||
|
||||
### What claude-mem Already Has
|
||||
|
||||
| Component | Status | Location |
|
||||
|-----------|--------|----------|
|
||||
| Claude Code plugin | Complete | `plugin/hooks/hooks.json` |
|
||||
| Cursor hooks installer | Complete | `src/services/integrations/CursorHooksInstaller.ts` |
|
||||
| Platform adapters | Claude Code + Cursor + raw | `src/cli/adapters/` |
|
||||
| Transcript watcher | Complete (schema-based JSONL) | `src/services/transcripts/` |
|
||||
| Codex transcript schema | Sample exists | `src/services/transcripts/config.ts` |
|
||||
| OpenClaw plugin | Complete (1000+ lines) | `openclaw/src/index.ts` |
|
||||
| MCP server | Complete | `plugin/scripts/mcp-server.cjs` |
|
||||
| Gemini CLI support | Not started | — |
|
||||
| OpenCode support | Not started | — |
|
||||
| Windsurf support | Not started | — |
|
||||
|
||||
### Patterns to Copy
|
||||
|
||||
- **Agent detection from `npx skills`** (`vercel-labs/skills/src/agents.ts`): Check if config directory exists
|
||||
- **Existing installer logic** (`installer/src/steps/install.ts:29-83`): registerMarketplace, registerPlugin, enablePluginInClaudeSettings — **extract shared logic** from existing installer into reusable modules (DRY with the new CLI)
|
||||
- **Bun resolution** (`plugin/scripts/bun-runner.js`): PATH lookup + common locations per platform
|
||||
- **CursorHooksInstaller** (`src/services/integrations/CursorHooksInstaller.ts`): Reference implementation for IDE hooks installation
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: NPX CLI Entry Point
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Add `bin` field to `package.json`**:
|
||||
```json
|
||||
"bin": {
|
||||
"claude-mem": "./dist/cli/index.js"
|
||||
}
|
||||
```
|
||||
|
||||
2. **Create `src/npx-cli/index.ts`** — a Node.js CLI router (NOT Bun) with command categories:
|
||||
|
||||
**Install commands** (pure Node.js, no Bun required):
|
||||
- `npx claude-mem` or `npx claude-mem install` → interactive install (IDE multi-select)
|
||||
- `npx claude-mem install --ide <name>` → direct IDE setup (only for implemented IDEs; unimplemented ones error with "Support for <name> coming soon")
|
||||
- `npx claude-mem update` → update to latest version
|
||||
- `npx claude-mem uninstall` → remove plugin and IDE configs
|
||||
- `npx claude-mem version` → print version
|
||||
|
||||
**Runtime commands** (delegate to Bun via installed plugin):
|
||||
- `npx claude-mem start` → spawns `bun worker-service.cjs start`
|
||||
- `npx claude-mem stop` → spawns `bun worker-service.cjs stop`
|
||||
- `npx claude-mem restart` → spawns `bun worker-service.cjs restart`
|
||||
- `npx claude-mem status` → spawns `bun worker-service.cjs status`
|
||||
- `npx claude-mem search <query>` → hits `GET http://localhost:37777/api/search?q=<query>`
|
||||
- `npx claude-mem transcript watch` → starts transcript watcher
|
||||
|
||||
**Runtime commands must check for installation first**: If plugin directory doesn't exist at `~/.claude/plugins/marketplaces/thedotmack/`, print "claude-mem is not installed. Run: npx claude-mem install" and exit.
|
||||
|
||||
3. **The install flow** (fully replaces git clone + build):
|
||||
- Detect the npm package's own location (`import.meta.url` or `__dirname`)
|
||||
- Copy `plugin/` from the npm package to `~/.claude/plugins/marketplaces/thedotmack/`
|
||||
- Copy `plugin/` to `~/.claude/plugins/cache/thedotmack/claude-mem/<version>/`
|
||||
- Register marketplace in `~/.claude/plugins/known_marketplaces.json`
|
||||
- Register plugin in `~/.claude/plugins/installed_plugins.json`
|
||||
- Enable in `~/.claude/settings.json`
|
||||
- Run `npm install` in the marketplace dir (for `@chroma-core/default-embed` — native ONNX binaries, can't be bundled)
|
||||
- Trigger smart-install.js for Bun/uv setup
|
||||
- Run IDE-specific setup for each selected IDE
|
||||
|
||||
4. **Interactive IDE selection** (auto-detect + prompt):
|
||||
- Auto-detect installed IDEs by checking config directories
|
||||
- Present multi-select with detected IDEs pre-selected
|
||||
- Detection map:
|
||||
- Claude Code: `~/.claude/` exists
|
||||
- Gemini CLI: `~/.gemini/` exists
|
||||
- OpenCode: `~/.config/opencode/` exists OR `opencode` in PATH
|
||||
- OpenClaw: `~/.openclaw/` exists
|
||||
- Windsurf: `~/.codeium/windsurf/` exists
|
||||
- Codex CLI: `~/.codex/` exists
|
||||
- Cursor: `~/.cursor/` exists
|
||||
- Copilot CLI: `copilot` in PATH (it's a CLI tool, not a config dir)
|
||||
- Antigravity: `~/.gemini/antigravity/` exists
|
||||
- Goose: `~/.config/goose/` exists OR `goose` in PATH
|
||||
- Crush: `crush` in PATH
|
||||
- Roo Code: check for VS Code extension directory containing `roo-code`
|
||||
- Warp: `~/.warp/` exists OR `warp` in PATH
|
||||
|
||||
5. **The runtime command routing**:
|
||||
- Locate the installed plugin directory
|
||||
- Find Bun binary (same logic as `bun-runner.js`, platform-aware)
|
||||
- Spawn `bun worker-service.cjs <command>` and pipe stdio through
|
||||
- For `search`: HTTP request to running worker
|
||||
|
||||
### Patterns to follow
|
||||
|
||||
- `installer/src/steps/install.ts:29-83` for marketplace registration — **extract to shared module**
|
||||
- `plugin/scripts/bun-runner.js` for Bun resolution
|
||||
- `vercel-labs/skills/src/agents.ts` for IDE auto-detection pattern
|
||||
|
||||
### Verification
|
||||
|
||||
- `npx claude-mem install` copies plugin to correct directories on macOS, Linux, and Windows
|
||||
- Auto-detection finds installed IDEs
|
||||
- `npx claude-mem start/stop/status` work after install
|
||||
- `npx claude-mem search "test"` returns results
|
||||
- `npx claude-mem start` before install prints helpful error message
|
||||
- `npx claude-mem update` and `npx claude-mem uninstall` work correctly
|
||||
- `npx claude-mem version` prints version
|
||||
|
||||
### Anti-patterns
|
||||
|
||||
- Do NOT require Bun for install commands — pure Node.js
|
||||
- Do NOT clone the git repo
|
||||
- Do NOT build from source at install time
|
||||
- Do NOT depend on `bun:sqlite` in the CLI entry point
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Build Pipeline Integration
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Add CLI build step to `scripts/build-hooks.js`**:
|
||||
- Compile `src/npx-cli/index.ts` → `dist/cli/index.js`
|
||||
- Bundle `@clack/prompts` and `picocolors` into the output (self-contained)
|
||||
- Shebang: `#!/usr/bin/env node`
|
||||
- Set executable permissions (no-op on Windows, that's fine)
|
||||
|
||||
2. **Move `@clack/prompts` and `picocolors`** to main package.json as dev dependencies (bundled by esbuild into dist/cli/index.js)
|
||||
|
||||
3. **Verify `package.json` `files` field**: Currently `["dist", "plugin"]`. `dist/cli/index.js` is already included since it's under `dist/`. No change needed.
|
||||
|
||||
4. **Update `prepublishOnly`** to ensure CLI is built before npm publish (already covered — `npm run build` calls `build-hooks.js`)
|
||||
|
||||
5. **Pre-build OpenClaw plugin**: Add an esbuild step that compiles `openclaw/src/index.ts` → `openclaw/dist/index.js` so it ships ready-to-use. No `tsc` at install time.
|
||||
|
||||
6. **Add `openclaw/dist/` to `package.json` `files` field** (or add `openclaw` if the whole directory should ship)
|
||||
|
||||
### Verification
|
||||
|
||||
- `npm run build` produces `dist/cli/index.js` with correct shebang
|
||||
- `npm run build` produces `openclaw/dist/index.js` pre-built
|
||||
- `npm pack` includes both `dist/cli/index.js` and `openclaw/dist/`
|
||||
- `node dist/cli/index.js --help` works without Bun
|
||||
- Package size is reasonable (check with `npm pack --dry-run`)
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Gemini CLI Integration (Tier 1 — Hook-Based)
|
||||
|
||||
**Why first among new IDEs**: Near-identical architecture to Claude Code. 11 lifecycle hooks with JSON stdin/stdout, same exit code conventions (0=success, 2=block), `GEMINI.md` context files. 95k GitHub stars. Lowest effort, highest confidence.
|
||||
|
||||
### Gemini CLI Hook Events
|
||||
|
||||
| Event | Map to claude-mem | Use |
|
||||
|-------|-------------------|-----|
|
||||
| `SessionStart` | `session-init` | Start tracking session |
|
||||
| `BeforeAgent` | `user-prompt` | Capture user prompt |
|
||||
| `AfterAgent` | `observation` | Capture full agent response |
|
||||
| `BeforeTool` | — | Skip (pre-execution, no result yet) |
|
||||
| `AfterTool` | `observation` | Capture tool name + input + response |
|
||||
| `BeforeModel` | — | Skip (too low-level, LLM request details) |
|
||||
| `AfterModel` | — | Skip (raw LLM response, redundant with AfterAgent) |
|
||||
| `BeforeToolSelection` | — | Skip (internal planning step) |
|
||||
| `PreCompress` | `summary` | Trigger summary before context compression |
|
||||
| `Notification` | — | Skip (system alerts, not session data) |
|
||||
| `SessionEnd` | `session-end` | Finalize session |
|
||||
|
||||
**Mapped**: 5 of 11 events. **Skipped**: 6 events that are either too low-level (BeforeModel/AfterModel), pre-execution (BeforeTool, BeforeToolSelection), or system-level (Notification).
|
||||
|
||||
### Verified Stdin Payload Schemas (from `packages/core/src/hooks/types.ts`)
|
||||
|
||||
**Base input (all hooks receive):**
|
||||
```typescript
|
||||
{ session_id: string, transcript_path: string, cwd: string, hook_event_name: string, timestamp: string }
|
||||
```
|
||||
|
||||
**Event-specific fields:**
|
||||
| Event | Additional Fields |
|
||||
|-------|-------------------|
|
||||
| `SessionStart` | `source: "startup" \| "resume" \| "clear"` |
|
||||
| `SessionEnd` | `reason: "exit" \| "clear" \| "logout" \| "prompt_input_exit" \| "other"` |
|
||||
| `BeforeAgent` | `prompt: string` |
|
||||
| `AfterAgent` | `prompt: string, prompt_response: string, stop_hook_active: boolean` |
|
||||
| `BeforeTool` | `tool_name: string, tool_input: Record<string, unknown>, mcp_context?: McpToolContext, original_request_name?: string` |
|
||||
| `AfterTool` | `tool_name: string, tool_input: Record<string, unknown>, tool_response: Record<string, unknown>, mcp_context?: McpToolContext` |
|
||||
| `PreCompress` | `trigger: "auto" \| "manual"` |
|
||||
| `Notification` | `notification_type: "ToolPermission", message: string, details: Record<string, unknown>` |
|
||||
|
||||
**Output (all hooks can return):**
|
||||
```typescript
|
||||
{ continue?: boolean, stopReason?: string, suppressOutput?: boolean, systemMessage?: string, decision?: "allow" | "deny" | "block" | "approve" | "ask", reason?: string, hookSpecificOutput?: Record<string, unknown> }
|
||||
```
|
||||
|
||||
**Advisory (non-blocking) hooks:** SessionStart, SessionEnd, PreCompress, Notification — `continue` and `decision` fields are ignored.
|
||||
|
||||
**Environment variables provided:** `GEMINI_PROJECT_DIR`, `GEMINI_SESSION_ID`, `GEMINI_CWD`, `CLAUDE_PROJECT_DIR` (compat alias)
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Create Gemini CLI platform adapter** at `src/cli/adapters/gemini-cli.ts`:
|
||||
- Normalize Gemini CLI's hook JSON to `NormalizedHookInput`
|
||||
- Base fields always present: `session_id`, `transcript_path`, `cwd`, `hook_event_name`, `timestamp`
|
||||
- Map per event:
|
||||
- `SessionStart`: `source` → session init metadata
|
||||
- `BeforeAgent`: `prompt` → user prompt text
|
||||
- `AfterAgent`: `prompt` + `prompt_response` → full conversation turn
|
||||
- `AfterTool`: `tool_name` + `tool_input` + `tool_response` → observation
|
||||
- `PreCompress`: `trigger` → summary trigger
|
||||
- `SessionEnd`: `reason` → session finalization
|
||||
|
||||
2. **Create Gemini CLI hooks installer** at `src/services/integrations/GeminiCliHooksInstaller.ts`:
|
||||
- Write hooks to `~/.gemini/settings.json` under the `hooks` key
|
||||
- Must **merge** with existing settings (read → parse → deep merge → write)
|
||||
- Hook config format (verified against official docs):
|
||||
```json
|
||||
{
|
||||
"hooks": {
|
||||
"AfterTool": [{
|
||||
"matcher": "*",
|
||||
"hooks": [{ "name": "claude-mem", "type": "command", "command": "<path-to-hook-script>", "timeout": 5000 }]
|
||||
}]
|
||||
}
|
||||
}
|
||||
```
|
||||
- Note: `matcher` uses regex for tool events, exact string for lifecycle events. `"*"` or `""` matches all.
|
||||
- Hook groups support `sequential: boolean` (default false = parallel execution)
|
||||
- Security: Project-level hooks are fingerprinted — if name/command changes, user is warned
|
||||
- Context injection via `~/.gemini/GEMINI.md` (append claude-mem section with `<claude-mem-context>` tags, same pattern as CLAUDE.md)
|
||||
- Settings hierarchy: project `.gemini/settings.json` > user `~/.gemini/settings.json` > system `/etc/gemini-cli/settings.json`
|
||||
|
||||
3. **Register `gemini-cli` in `getPlatformAdapter()`** at `src/cli/adapters/index.ts`
|
||||
|
||||
4. **Add Gemini CLI to installer IDE selection**
|
||||
|
||||
### Verification
|
||||
|
||||
- `npx claude-mem install --ide gemini-cli` merges hooks into `~/.gemini/settings.json`
|
||||
- Gemini CLI sessions are captured by the worker
|
||||
- `AfterTool` events produce observations with correct `tool_name`, `tool_input`, `tool_response`
|
||||
- `GEMINI.md` gets claude-mem context section
|
||||
- Existing Gemini CLI settings are preserved (merge, not overwrite)
|
||||
- Verify `session_id` from base input is used for session tracking
|
||||
|
||||
### Anti-patterns
|
||||
|
||||
- Do NOT overwrite `~/.gemini/settings.json` — must deep merge
|
||||
- Do NOT map all 11 events — the 6 skipped events would produce noise, not signal
|
||||
- Do NOT use `type: "runtime"` — that's for internal extensions only; use `type: "command"`
|
||||
- Advisory hooks (SessionStart, SessionEnd, PreCompress, Notification) cannot block — don't set `decision` or `continue` fields on them
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: OpenCode Integration (Tier 1 — Plugin-Based)
|
||||
|
||||
**Why next**: 110k stars, richest plugin ecosystem. OpenCode plugins are JS/TS modules auto-loaded from plugin directories. OpenCode also has a Claude Code compatibility fallback (reads `~/.claude/CLAUDE.md` if no global `AGENTS.md` exists, controllable via `OPENCODE_DISABLE_CLAUDE_CODE_PROMPT=1`).
|
||||
|
||||
### Verified Plugin API (from `packages/plugin/src/index.ts`)
|
||||
|
||||
**Plugin signature:**
|
||||
```typescript
|
||||
import { type Plugin, tool } from "@opencode-ai/plugin"
|
||||
|
||||
export const ClaudeMemPlugin: Plugin = async (ctx) => {
|
||||
// ctx: { client, project, directory, worktree, serverUrl, $ }
|
||||
return { /* hooks object */ }
|
||||
}
|
||||
```
|
||||
|
||||
**PluginInput type (6 properties, not 4):**
|
||||
```typescript
|
||||
type PluginInput = {
|
||||
client: ReturnType<typeof createOpencodeClient> // OpenCode SDK client
|
||||
project: Project // Current project info
|
||||
directory: string // Current working directory
|
||||
worktree: string // Git worktree path
|
||||
serverUrl: URL // Server URL
|
||||
$: BunShell // Bun shell API
|
||||
}
|
||||
```
|
||||
|
||||
**Two hook mechanisms (important distinction):**
|
||||
|
||||
1. **Direct interceptor hooks** — keys on the returned `Hooks` object, receive `(input, output)` allowing mutation:
|
||||
- `tool.execute.before`: `(input: { tool, sessionID, callID }, output: { args })`
|
||||
- `tool.execute.after`: `(input: { tool, sessionID, callID, args }, output: { title, output, metadata })`
|
||||
- `shell.env`, `chat.message`, `chat.params`, `chat.headers`, `permission.ask`, `command.execute.before`
|
||||
- Experimental: `experimental.session.compacting`, `experimental.chat.messages.transform`, `experimental.chat.system.transform`
|
||||
|
||||
2. **Bus event catch-all** — generic `event` hook, receives `{ event }` where `event.type` is the event name:
|
||||
- `session.created`, `session.compacted`, `session.deleted`, `session.idle`, `session.error`, `session.status`, `session.updated`, `session.diff`
|
||||
- `message.updated`, `message.part.updated`, `message.part.removed`, `message.removed`
|
||||
- `file.edited`, `file.watcher.updated`
|
||||
- `command.executed`, `todo.updated`, `installation.updated`, `server.connected`
|
||||
- `permission.asked`, `permission.replied`
|
||||
- `lsp.client.diagnostics`, `lsp.updated`
|
||||
- `tui.prompt.append`, `tui.command.execute`, `tui.toast.show`
|
||||
- Total: **27 bus events** across **12 categories**
|
||||
|
||||
**Custom tool registration (CORRECTED — name is the key, not positional arg):**
|
||||
```typescript
|
||||
return {
|
||||
tool: {
|
||||
claude_mem_search: tool({
|
||||
description: "Search claude-mem memory database",
|
||||
args: { query: tool.schema.string() },
|
||||
async execute(args, context) {
|
||||
// context: { sessionID, messageID, agent, directory, worktree, abort, metadata, ask }
|
||||
const response = await fetch(`http://localhost:37777/api/search?q=${encodeURIComponent(args.query)}`)
|
||||
return await response.text()
|
||||
},
|
||||
}),
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Create OpenCode plugin** at `src/integrations/opencode-plugin/index.ts`:
|
||||
- Export a `Plugin` function receiving full `PluginInput` context
|
||||
- Use **direct interceptor** `tool.execute.after` for tool observation capture (gives `tool`, `args`, `output`)
|
||||
- Use **bus event catch-all** `event` for session lifecycle:
|
||||
|
||||
| Mechanism | Event | Map to claude-mem |
|
||||
|-----------|-------|-------------------|
|
||||
| interceptor | `tool.execute.after` | `observation` (tool name + args + output) |
|
||||
| bus event | `session.created` | `session-init` |
|
||||
| bus event | `message.updated` | `observation` (assistant messages) |
|
||||
| bus event | `session.compacted` | `summary` |
|
||||
| bus event | `file.edited` | `observation` (file changes) |
|
||||
| bus event | `session.deleted` | `session-end` |
|
||||
|
||||
- Register `claude_mem_search` custom tool using correct `tool({ description, args, execute })` API
|
||||
- Hit `localhost:37777` API endpoints from the plugin
|
||||
|
||||
2. **Build the plugin** in the esbuild pipeline → `dist/opencode-plugin/index.js`
|
||||
|
||||
3. **Create OpenCode setup in installer** (two options, prefer file-based):
|
||||
- **Option A (file-based):** Copy plugin to `~/.config/opencode/plugins/claude-mem.ts` (auto-loaded at startup)
|
||||
- **Option B (npm-based):** Add to `~/.config/opencode/opencode.json` under `"plugin"` array: `["claude-mem"]`
|
||||
- Config also supports JSONC (`opencode.jsonc`) and legacy `config.json`
|
||||
- Context injection: Append to `~/.config/opencode/AGENTS.md` (or create it) with `<claude-mem-context>` tags
|
||||
- Additional context via `"instructions"` config key (supports file paths, globs, remote URLs)
|
||||
|
||||
4. **Add OpenCode to installer IDE selection**
|
||||
|
||||
### OpenCode Verification
|
||||
|
||||
- `npx claude-mem install --ide opencode` registers the plugin (file or npm)
|
||||
- OpenCode loads the plugin on next session
|
||||
- `tool.execute.after` interceptor produces observations with `tool`, `args`, `output`
|
||||
- Bus events (`session.created`, `session.deleted`) handle session lifecycle
|
||||
- `claude_mem_search` custom tool works in OpenCode sessions
|
||||
- Context is injected via AGENTS.md
|
||||
|
||||
### OpenCode Anti-patterns
|
||||
|
||||
- Do NOT try to use OpenCode's `session.diff` for full capture — it's a summary diff, not raw data
|
||||
- Do NOT use `tool('name', schema, handler)` — wrong signature. Name is the key in the `tool:{}` map
|
||||
- Do NOT assume bus events have the same `(input, output)` mutation pattern — they only receive `{ event }`
|
||||
- OpenCode plugins run in Bun — the plugin CAN use Bun APIs (unlike the npx CLI itself)
|
||||
- Do NOT hardcode `~/.config/opencode/` — respect `OPENCODE_CONFIG_DIR` env var if set
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Windsurf Integration (Tier 1 — Hook-Based)
|
||||
|
||||
**Why next**: 11 Cascade hooks, ~1M users. Hook architecture uses JSON stdin with a consistent envelope format.
|
||||
|
||||
### Verified Windsurf Hook Events (from docs.windsurf.com/windsurf/cascade/hooks)
|
||||
|
||||
**Naming pattern**: `pre_`/`post_` prefix + 5 action categories, plus 2 standalone post-only events.
|
||||
|
||||
| Event | Can Block? | Map to claude-mem | Use |
|
||||
|-------|-----------|-------------------|-----|
|
||||
| `pre_user_prompt` | Yes | `session-init` + `context` | Start session, inject context |
|
||||
| `pre_read_code` | Yes | — | Skip (pre-execution, can block file reads) |
|
||||
| `post_read_code` | No | — | Skip (too noisy, file reads are frequent) |
|
||||
| `pre_write_code` | Yes | — | Skip (pre-execution, can block writes) |
|
||||
| `post_write_code` | No | `observation` | Code generation |
|
||||
| `pre_run_command` | Yes | — | Skip (pre-execution, can block commands) |
|
||||
| `post_run_command` | No | `observation` | Shell command execution |
|
||||
| `pre_mcp_tool_use` | Yes | — | Skip (pre-execution, can block MCP calls) |
|
||||
| `post_mcp_tool_use` | No | `observation` | MCP tool results |
|
||||
| `post_cascade_response` | No | `observation` | Full AI response |
|
||||
| `post_setup_worktree` | No | — | Skip (informational) |
|
||||
|
||||
**Mapped**: 5 of 11 events (all post-action). **Skipped**: 4 pre-hooks (blocking-capable, pre-execution) + 2 low-value post-hooks.
|
||||
|
||||
### Verified Stdin Payload Schema
|
||||
|
||||
**Common envelope (all hooks):**
|
||||
```json
|
||||
{
|
||||
"agent_action_name": "string",
|
||||
"trajectory_id": "string",
|
||||
"execution_id": "string",
|
||||
"timestamp": "ISO 8601 string",
|
||||
"tool_info": { /* event-specific payload */ }
|
||||
}
|
||||
```
|
||||
|
||||
**Event-specific `tool_info` payloads:**
|
||||
|
||||
| Event | `tool_info` fields |
|
||||
|-------|-------------------|
|
||||
| `pre_user_prompt` | `{ user_prompt: string }` |
|
||||
| `pre_read_code` / `post_read_code` | `{ file_path: string }` |
|
||||
| `pre_write_code` / `post_write_code` | `{ file_path: string, edits: [{ old_string: string, new_string: string }] }` |
|
||||
| `pre_run_command` / `post_run_command` | `{ command_line: string, cwd: string }` |
|
||||
| `pre_mcp_tool_use` | `{ mcp_server_name: string, mcp_tool_name: string, mcp_tool_arguments: {} }` |
|
||||
| `post_mcp_tool_use` | `{ mcp_server_name: string, mcp_tool_name: string, mcp_tool_arguments: {}, mcp_result: string }` |
|
||||
| `post_cascade_response` | `{ response: string }` (markdown) |
|
||||
| `post_setup_worktree` | `{ worktree_path: string, root_workspace_path: string }` |
|
||||
|
||||
**Exit codes:** `0` = success, `2` = block (pre-hooks only; stderr shown to agent), any other = non-blocking warning.
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Create Windsurf platform adapter** at `src/cli/adapters/windsurf.ts`:
|
||||
- Normalize Windsurf's hook input format to `NormalizedHookInput`
|
||||
- Common envelope: `agent_action_name`, `trajectory_id`, `execution_id`, `timestamp`, `tool_info`
|
||||
- Map: `trajectory_id` → `sessionId`, `tool_info` fields per event type
|
||||
- For `post_write_code`: `tool_info.file_path` + `tool_info.edits` → file change observation
|
||||
- For `post_run_command`: `tool_info.command_line` + `tool_info.cwd` → command observation
|
||||
- For `post_mcp_tool_use`: `tool_info.mcp_tool_name` + `tool_info.mcp_tool_arguments` + `tool_info.mcp_result` → tool observation
|
||||
- For `post_cascade_response`: `tool_info.response` → full AI response observation
|
||||
|
||||
2. **Create Windsurf hooks installer** at `src/services/integrations/WindsurfHooksInstaller.ts`:
|
||||
- Write hooks to `~/.codeium/windsurf/hooks.json` (user-level, for global coverage)
|
||||
- Per-workspace override at `.windsurf/hooks.json` if user chooses workspace-level install
|
||||
- Config format (verified):
|
||||
```json
|
||||
{
|
||||
"hooks": {
|
||||
"post_write_code": [{
|
||||
"command": "<path-to-hook-script>",
|
||||
"show_output": false,
|
||||
"working_directory": "<optional>"
|
||||
}]
|
||||
}
|
||||
}
|
||||
```
|
||||
- Note: Tilde expansion (`~`) is NOT supported in `working_directory` — use absolute paths
|
||||
- Merge order: cloud → system → user → workspace (all hooks at all levels execute)
|
||||
- Context injection via `.windsurf/rules/claude-mem-context.md` (workspace-level; Windsurf rules are workspace-scoped)
|
||||
- Rule limits: 6,000 chars per file, 12,000 chars total across all rules
|
||||
|
||||
3. **Register `windsurf` in `getPlatformAdapter()`** at `src/cli/adapters/index.ts`
|
||||
|
||||
4. **Add Windsurf to installer IDE selection**
|
||||
|
||||
### Windsurf Verification
|
||||
|
||||
- `npx claude-mem install --ide windsurf` creates hooks config at `~/.codeium/windsurf/hooks.json`
|
||||
- Windsurf sessions are captured by the worker via post-action hooks
|
||||
- `trajectory_id` is used as session identifier
|
||||
- Context is injected via `.windsurf/rules/claude-mem-context.md` (under 6K char limit)
|
||||
- Existing hooks.json is preserved (merge, not overwrite)
|
||||
|
||||
### Windsurf Anti-patterns
|
||||
|
||||
- Do NOT use fabricated event names (`post_search_code`, `post_lint_code`, `on_error`, `pre_tool_execution`) — they don't exist
|
||||
- Do NOT assume Windsurf's stdin JSON matches Claude Code's — it uses `tool_info` envelope, not flat fields
|
||||
- Do NOT use tilde (`~`) in `working_directory` — not supported, use absolute paths
|
||||
- Do NOT exceed 6K chars in the context rule file — Windsurf truncates beyond that
|
||||
- Pre-hooks can block actions (exit 2) — only use post-hooks for observation capture
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Codex CLI Integration (Tier 1 — Hook + Transcript)
|
||||
|
||||
### Dedup strategy
|
||||
|
||||
Codex has both a `notify` hook (real-time) and transcript files (complete history). Use **transcript watching only** — it's more complete and avoids the complexity of dual capture paths. The `notify` hook is a simpler mechanism that doesn't provide enough granularity to justify maintaining two integration paths. If transcript watching proves insufficient, add the notify hook later.
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Create Codex transcript schema** — the sample in `src/services/transcripts/config.ts` is already production-quality. Verify against current Codex CLI JSONL format and update if needed.
|
||||
|
||||
2. **Create Codex setup in installer**:
|
||||
- Write transcript-watch config to `~/.claude-mem/transcript-watch.json`
|
||||
- Set up watch for `~/.codex/sessions/**/*.jsonl` using existing CODEX_SAMPLE_SCHEMA
|
||||
- Context injection via `.codex/AGENTS.md` (Codex reads this natively)
|
||||
- Must merge with existing `config.toml` if it exists (read → parse → merge → write)
|
||||
|
||||
3. **Add Codex CLI to installer IDE selection**
|
||||
|
||||
### Verification
|
||||
|
||||
- `npx claude-mem install --ide codex` creates transcript watch config
|
||||
- Codex sessions appear in claude-mem database
|
||||
- `AGENTS.md` updated with context after sessions
|
||||
- Existing `config.toml` is preserved
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: OpenClaw Integration (Tier 1 — Plugin-Based)
|
||||
|
||||
**Plugin is already fully built** at `openclaw/src/index.ts` (~1000 lines). Has event hooks, SSE observation feed, MEMORY.md sync, slash commands. Only wiring into the installer is needed.
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Wire OpenClaw into the npx installer**:
|
||||
- Detect `~/.openclaw/` directory
|
||||
- Copy pre-built plugin from `openclaw/dist/` (built in Phase 2) to OpenClaw plugins location
|
||||
- Register in `~/.openclaw/openclaw.json` under `plugins.claude-mem`
|
||||
- Configure worker port, project name, syncMemoryFile
|
||||
- Optionally prompt for observation feed setup (channel type + target ID)
|
||||
|
||||
2. **Add OpenClaw to IDE selection TUI** with hint about messaging channel support
|
||||
|
||||
### Verification
|
||||
|
||||
- `npx claude-mem install --ide openclaw` registers the plugin
|
||||
- OpenClaw gateway loads the plugin on restart
|
||||
- Observations are recorded from OpenClaw sessions
|
||||
- MEMORY.md syncs to agent workspaces
|
||||
|
||||
### Anti-patterns
|
||||
|
||||
- Do NOT rebuild the OpenClaw plugin from source at install time — it ships pre-built from Phase 2
|
||||
- Do NOT modify the plugin's event handling — it's battle-tested
|
||||
|
||||
---
|
||||
|
||||
## Phase 8: MCP-Based Integrations (Tier 2)
|
||||
|
||||
**These get the MCP server for free** — it already exists at `plugin/scripts/mcp-server.cjs`. The installer just needs to write the right config files per IDE.
|
||||
|
||||
MCP-only integrations provide: search tools + context injection. They do NOT capture transcripts or tool usage in real-time.
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Copilot CLI MCP setup**:
|
||||
- Write MCP config to `~/.copilot/config` (merge, not overwrite)
|
||||
- Context injection: `.github/copilot-instructions.md`
|
||||
- Detection: `copilot` command in PATH
|
||||
|
||||
2. **Antigravity MCP setup**:
|
||||
- Write MCP config to `~/.gemini/antigravity/mcp_config.json` (merge, not overwrite)
|
||||
- Context injection: `~/.gemini/GEMINI.md` (shared with Gemini CLI) and/or `.agent/rules/claude-mem-context.md`
|
||||
- Detection: `~/.gemini/antigravity/` exists
|
||||
- Note: Antigravity has NO hook system — MCP is the only integration path
|
||||
|
||||
3. **Goose MCP setup**:
|
||||
- Write MCP config to `~/.config/goose/config.yaml` (YAML merge — use a lightweight YAML parser or write the block manually if config doesn't exist)
|
||||
- Detection: `~/.config/goose/` exists OR `goose` in PATH
|
||||
- Note: Goose co-developed MCP with Anthropic, so MCP support is excellent
|
||||
|
||||
4. **Crush MCP setup**:
|
||||
- Write MCP config to Crush's JSON config
|
||||
- Detection: `crush` in PATH
|
||||
|
||||
5. **Roo Code MCP setup**:
|
||||
- Write MCP config to `.roo/` or workspace settings
|
||||
- Context injection: `.roo/rules/claude-mem-context.md`
|
||||
- Detection: Check for VS Code extension directory containing `roo-code`
|
||||
|
||||
6. **Warp MCP setup**:
|
||||
- Warp uses `WARP.md` in project root for context injection (similar to CLAUDE.md)
|
||||
- MCP servers configured via Warp Drive UI, but also via config files
|
||||
- Detection: `~/.warp/` exists OR `warp` in PATH
|
||||
- Note: Warp is a terminal replacement (~26k stars), not just a CLI tool — multi-agent orchestration with management UI
|
||||
|
||||
7. **For each**: Add to installer IDE detection and selection
|
||||
|
||||
### Config merging strategy
|
||||
|
||||
JSON configs: Read → parse → deep merge → write back. YAML configs (Goose): If file exists, read and append the MCP block. If not, create from template. Avoid pulling in a full YAML parser library — write the MCP block as a string append with proper indentation if the format is predictable.
|
||||
|
||||
### Verification
|
||||
|
||||
- Each IDE can search claude-mem via MCP tools
|
||||
- Context files are written to IDE-specific locations
|
||||
- Existing configs are preserved
|
||||
|
||||
### Anti-patterns
|
||||
|
||||
- MCP-only integrations do NOT capture transcripts — don't claim "full integration"
|
||||
- Do NOT overwrite existing config files — always merge
|
||||
- Do NOT add a heavy YAML parser dependency for one integration
|
||||
|
||||
---
|
||||
|
||||
## Phase 9: Remove Old Installer
|
||||
|
||||
This is a **full replacement**, not a deprecation.
|
||||
|
||||
### What to implement
|
||||
|
||||
1. Remove `claude-mem-installer` npm package (unpublish or mark deprecated with message pointing to `npx claude-mem`)
|
||||
2. Update `install/public/install.sh` → redirect to `npx claude-mem`
|
||||
3. Remove `installer/` directory from the repository (it's replaced by `src/npx-cli/`)
|
||||
4. Update docs site to reflect the new install command
|
||||
5. Update README.md install instructions
|
||||
|
||||
---
|
||||
|
||||
## Phase 10: Final Verification
|
||||
|
||||
### All platforms (macOS, Linux, Windows)
|
||||
|
||||
1. `npm run build` succeeds, produces `dist/cli/index.js` and `openclaw/dist/index.js`
|
||||
2. `node dist/cli/index.js install` works clean (no prior install)
|
||||
3. Auto-detects installed IDEs correctly per platform
|
||||
4. `npx claude-mem start/stop/status/search` all work
|
||||
5. `npx claude-mem update` updates correctly
|
||||
6. `npx claude-mem uninstall` cleans up all IDE configs
|
||||
7. `npx claude-mem version` prints version
|
||||
8. `npx claude-mem start` before install shows helpful error
|
||||
9. No Bun dependency at install time
|
||||
|
||||
### Per-integration verification
|
||||
|
||||
| Integration | Type | Captures Sessions | Search via MCP | Context Injection |
|
||||
|-------------|------|-------------------|----------------|-------------------|
|
||||
| Claude Code | Plugin | Yes (hooks) | Yes | CLAUDE.md |
|
||||
| Gemini CLI | Hooks | Yes (AfterTool, AfterAgent) | Yes (via hook) | GEMINI.md |
|
||||
| OpenCode | Plugin | Yes (tool.execute.after, message.updated) | Yes (custom tool) | AGENTS.md / rules |
|
||||
| Windsurf | Hooks | Yes (post_cascade_response, etc.) | Yes (via hook) | .windsurf/rules/ |
|
||||
| Codex CLI | Transcript | Yes (JSONL watcher) | No (passive only) | .codex/AGENTS.md |
|
||||
| OpenClaw | Plugin | Yes (event hooks) | Yes (slash commands) | MEMORY.md |
|
||||
| Copilot CLI | MCP | No | Yes | copilot-instructions.md |
|
||||
| Antigravity | MCP | No | Yes | .agent/rules/ |
|
||||
| Goose | MCP | No | Yes | MCP context |
|
||||
| Crush | MCP | No | Yes | Skills |
|
||||
| Roo Code | MCP | No | Yes | .roo/rules/ |
|
||||
| Warp | MCP | No | Yes | WARP.md |
|
||||
|
||||
---
|
||||
|
||||
## Priority Order & Impact
|
||||
|
||||
| Phase | IDE/Tool | Integration Type | Stars/Users | Effort |
|
||||
|-------|----------|-----------------|-------------|--------|
|
||||
| 1-2 | (infrastructure) | npx CLI + build pipeline | All users | Medium |
|
||||
| 3 | Gemini CLI | Hooks (Tier 1) | ~95k stars | Medium (near-identical to Claude Code) |
|
||||
| 4 | OpenCode | Plugin (Tier 1) | ~110k stars | Medium (rich plugin SDK) |
|
||||
| 5 | Windsurf | Hooks (Tier 1) | ~1M users | Medium |
|
||||
| 6 | Codex CLI | Transcript (Tier 3) | Growing (OpenAI) | Low (schema already exists) |
|
||||
| 7 | OpenClaw | Plugin (Tier 1) — pre-built | ~196k stars | Low (wire into installer) |
|
||||
| 8 | Copilot CLI, Antigravity, Goose, Crush, Warp, Roo Code | MCP (Tier 2) | 20M+ combined | Low per IDE |
|
||||
| 9 | (remove old installer) | — | — | Low |
|
||||
| 10 | (final verification) | — | — | Low |
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- **Removing Bun as runtime dependency**: Worker still requires Bun for `bun:sqlite`. Runtime commands delegate to Bun; install commands don't need it.
|
||||
- **JetBrains plugin**: Requires Kotlin/Java development — different ecosystem entirely.
|
||||
- **Zed extension**: WASM sandbox limits feasibility.
|
||||
- **Neovim/Emacs plugins**: Niche audiences, complex plugin ecosystems (Lua/Elisp). Could be added later via MCP (gptel supports it).
|
||||
- **Amazon Q / Kiro**: Amazon Q Developer CLI has been sunsetted in favor of Kiro (proprietary, no public extensibility API yet). Revisit when Kiro opens up.
|
||||
- **Aider**: Niche audience, writes Markdown transcripts (not JSONL), would require a markdown parser mode in the watcher. Add if demand materializes.
|
||||
- **Continue.dev**: Small user base relative to other MCP tools. Can be added as a Tier 2 MCP integration later if requested.
|
||||
- **Toad / Qwen Code / Oh-my-pi**: Too early-stage or too niche. Monitor for growth.
|
||||
- **OpenClaw plugin development**: The plugin is already complete. Only installer wiring is in scope.
|
||||
+248
-475
@@ -2,6 +2,254 @@
|
||||
|
||||
All notable changes to claude-mem.
|
||||
|
||||
## [v10.4.3] - 2026-02-25
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- **Fix PostToolUse hook crashes and 5-second latency (#1220)**: Added missing `break` statements to all 7 switch cases in `worker-service.ts` preventing fall-through execution, added `.catch()` on `main()` to handle unhandled promise rejections, and removed redundant `start` commands from hook groups that triggered the 5-second `collectStdin()` timeout
|
||||
- **Fix CLAUDE_PLUGIN_ROOT fallback for Stop hooks (#1215)**: Added POSIX shell-level `CLAUDE_PLUGIN_ROOT` fallback in `hooks.json` for environments where the variable isn't injected, added script-level self-resolution via `import.meta.url` in `bun-runner.js`, and regression test added in `plugin-distribution.test.ts`
|
||||
|
||||
## Maintenance
|
||||
|
||||
- Synced all version files (plugin.json was stuck at 10.4.0)
|
||||
|
||||
## [v10.4.2] - 2026-02-25
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- **Fix PostToolUse hook crashes and 5-second latency (#1220)**: Added missing `break` statements to all 7 switch cases in `worker-service.ts` preventing fall-through execution, added `.catch()` on `main()` to handle unhandled promise rejections, and removed redundant `start` commands from hook groups that triggered the 5-second `collectStdin()` timeout
|
||||
- **Fix CLAUDE_PLUGIN_ROOT fallback for Stop hooks (#1215)**: Added POSIX shell-level `CLAUDE_PLUGIN_ROOT` fallback in `hooks.json` for environments where the variable isn't injected, added script-level self-resolution via `import.meta.url` in `bun-runner.js`, and regression test added in `plugin-distribution.test.ts`
|
||||
- **Sync plugin.json version**: Fixed `plugin.json` being stuck at 10.4.0 while other version files were at 10.4.1
|
||||
|
||||
## [v10.4.1] - 2026-02-24
|
||||
|
||||
### Refactor
|
||||
- **Skills Conversion**: Converted `/make-plan` and `/do` commands into first-class skills in `plugin/skills/`.
|
||||
- **Organization**: Centralized planning and execution instructions alongside `mem-search`.
|
||||
- **Compatibility**: Added symlinks for `openclaw/skills/` to ensure seamless integration with OpenClaw.
|
||||
|
||||
### Chore
|
||||
- **Version Bump**: Aligned all package and plugin manifests to v10.4.1.
|
||||
|
||||
## [v10.4.0] - 2026-02-24
|
||||
|
||||
## v10.4.0 — Stability & Platform Hardening
|
||||
|
||||
Massive reliability release: 30+ root-cause bug fixes across 10 triage phases, plus new features for agent attribution, Chroma control, and broader platform support.
|
||||
|
||||
### New Features
|
||||
|
||||
- **Session custom titles** — Agents can now set `custom_title` on sessions for attribution (migration 23, new endpoint)
|
||||
- **Chroma toggle** — `CLAUDE_MEM_CHROMA_ENABLED` setting allows SQLite-only fallback mode (#707)
|
||||
- **Plugin disabled state** — Early exit check in all hook entry points when plugin is disabled (#781)
|
||||
- **Context re-injection guard** — `contextInjected` session flag prevents re-injecting context on every UserPromptSubmit turn (#1079)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
#### Data Integrity
|
||||
- SHA-256 content-hash deduplication on observation INSERT (migration 22 with backfill + index)
|
||||
- Project name collision fix: `getCurrentProjectName()` now returns `parent/basename`
|
||||
- Empty project string guard with cwd-derived fallback
|
||||
- Stuck `isProcessing` reset: pending work older than 5 minutes auto-clears
|
||||
|
||||
#### ChromaDB
|
||||
- Python version pinning in uvx args for both local and remote mode (#1196, #1206, #1208)
|
||||
- Windows backslash-to-forward-slash path conversion for `--data-dir` (#1199)
|
||||
- Metadata sanitization: filter null/undefined/empty values in `addDocuments()` (#1183, #1188)
|
||||
- Transport error auto-reconnect in `callTool()` (#1162)
|
||||
- Stale transport retry with transparent reconnect (#1131)
|
||||
|
||||
#### Hook Lifecycle
|
||||
- Suppress `process.stderr.write` in `hookCommand()` to prevent diagnostic output showing as error UI (#1181)
|
||||
- Route all `console.error()` through logger instead of stderr
|
||||
- Verified all 7 handlers return `suppressOutput: true` (#598, #784)
|
||||
|
||||
#### Worker Lifecycle
|
||||
- PID file mtime guard prevents concurrent restart storms (#1145)
|
||||
- `getInstalledPluginVersion()` ENOENT/EBUSY handling (#1042)
|
||||
|
||||
#### SQLite Migrations
|
||||
- Schema initialization always creates core tables via `CREATE TABLE IF NOT EXISTS`
|
||||
- Migrations 5-7 check actual DB state instead of version tracking (fixes version collision between old/new migration systems, #979)
|
||||
- Crash-safe temp table rebuilds
|
||||
|
||||
#### Platform Support
|
||||
- **Windows**: `cmd.exe /c` uvx spawn, PowerShell `$_` elimination with WQL filtering, `windowsHide: true`, FTS5 runtime probe with fallback (#1190, #1192, #1199, #1024, #1062, #1048, #791)
|
||||
- **Cursor IDE**: Adapter field fallbacks, tolerant session-init validation (#838, #1049)
|
||||
- **Codex CLI**: `session_id` fallbacks, unknown platform tolerance, undefined guard (#744)
|
||||
|
||||
#### API & Infrastructure
|
||||
- `/api/logs` OOM fix: tail-read replaces full-file `readFileSync` (64KB expanding chunks, 10MB cap, #1203)
|
||||
- CORS: explicit methods and allowedHeaders (#1029)
|
||||
- MCP type coercion for batch endpoints: string-to-array for `ids` and `memorySessionIds`
|
||||
- Defensive observation error handling returns 200 on recoverable errors instead of 500
|
||||
- `.git/` directory write guard on all 4 CLAUDE.md/AGENTS.md write sites (#1165)
|
||||
|
||||
#### Stale AbortController Fix
|
||||
- `lastGeneratorActivity` timestamp tracking with 30s timeout (#1099)
|
||||
- Stale generator detection + abort + restart in `ensureGeneratorRunning`
|
||||
- `AbortSignal.timeout(30000)` in `deleteSession` prevents indefinite hang
|
||||
|
||||
### Installation
|
||||
- `resolveRoot()` replaces hardcoded marketplace path using `CLAUDE_PLUGIN_ROOT` env var (#1128, #1166)
|
||||
- `installCLI()` path correction and `verifyCriticalModules()` post-install check
|
||||
- Build-time distribution verification for skills, hooks, and plugin manifest (#1187)
|
||||
|
||||
### Testing
|
||||
- 50+ new tests across hook lifecycle, context re-injection, plugin distribution, migration runner, data integrity, stale abort controller, logs tail-read, CORS, MCP type coercion, and smart-install
|
||||
- 68 files changed, ~4200 insertions, ~900 deletions
|
||||
|
||||
## [v10.3.3] - 2026-02-23
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Fixed session context footer to reference the claude-mem skill instead of MCP search tools for accessing memories
|
||||
|
||||
## [v10.3.2] - 2026-02-23
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- **Worker startup readiness**: Worker startup hook now waits for full DB/search readiness before proceeding, fixing the race condition where hooks would fire before the worker was initialized on first start (#1210)
|
||||
- **MCP tool naming**: Renamed `save_memory` to `save_observation` for consistency with the observation-based data model (#1210)
|
||||
- **MCP search instructions**: Updated MCP server tool descriptions to accurately reflect the 3-layer search workflow (#1210)
|
||||
- **Installer hosting**: Serve installer JS from install.cmem.ai instead of GitHub raw URLs for reliability
|
||||
- **Installer routing**: Added rewrite rule so install.cmem.ai root path correctly serves the install script
|
||||
- **Installer build**: Added compiled installer dist so CLI installation works out of the box
|
||||
|
||||
## [v10.3.1] - 2026-02-19
|
||||
|
||||
## Fix: Prevent Duplicate Worker Daemons and Zombie Processes
|
||||
|
||||
Three root causes of chroma-mcp timeouts identified and fixed:
|
||||
|
||||
### PID-based daemon guard
|
||||
Exit immediately on startup if PID file points to a live process. Prevents the race condition where hooks firing simultaneously could start multiple daemons before either wrote a PID file.
|
||||
|
||||
### Port-based daemon guard
|
||||
Exit if port 37777 is already bound — runs before WorkerService constructor registers keepalive signal handlers that previously prevented exit on EADDRINUSE.
|
||||
|
||||
### Guaranteed process.exit() after HTTP shutdown
|
||||
HTTP shutdown (POST /api/admin/shutdown) now calls `process.exit(0)` in a `try/finally` block. Previously, zombie workers stayed alive after shutdown, and background tasks reconnected to chroma-mcp, spawning duplicate subprocesses contending for the same data directory.
|
||||
|
||||
## [v10.3.0] - 2026-02-18
|
||||
|
||||
## Replace WASM Embeddings with Persistent chroma-mcp MCP Connection
|
||||
|
||||
### Highlights
|
||||
|
||||
- **New: ChromaMcpManager** — Singleton stdio MCP client communicating with chroma-mcp via `uvx`, replacing the previous ChromaServerManager (`npx chroma run` + `chromadb` npm + ONNX/WASM)
|
||||
- **Eliminates native binary issues** — No more segfaults, WASM embedding failures, or cross-platform install headaches
|
||||
- **Graceful subprocess lifecycle** — Wired into GracefulShutdown for clean teardown; zombie process prevention with kill-on-failure and stale `onclose` handler guards
|
||||
- **Connection backoff** — 10-second reconnect backoff prevents chroma-mcp spawn storms
|
||||
- **SQL injection guards** — Added parameterization to ChromaSync ID exclusion queries
|
||||
- **Simplified ChromaSync** — Reduced complexity by delegating embedding concerns to chroma-mcp
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
None — backward compatible. ChromaDB data is preserved; only the connection mechanism changed.
|
||||
|
||||
### Files Changed
|
||||
|
||||
- `src/services/sync/ChromaMcpManager.ts` (new) — MCP client singleton
|
||||
- `src/services/sync/ChromaServerManager.ts` (deleted) — Old WASM/native approach
|
||||
- `src/services/sync/ChromaSync.ts` — Simplified to use MCP client
|
||||
- `src/services/worker-service.ts` — Updated startup sequence
|
||||
- `src/services/infrastructure/GracefulShutdown.ts` — Subprocess cleanup integration
|
||||
|
||||
## [v10.2.6] - 2026-02-18
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
### Zombie Process Prevention (#1168, #1175)
|
||||
|
||||
Observer Claude CLI subprocesses were accumulating as zombies — processes that never exited after their session ended, causing massive resource leaks on long-running systems.
|
||||
|
||||
**Root cause:** When observer sessions ended (via idle timeout, abort, or error), the spawned Claude CLI subprocesses were not being reliably killed. The existing `ensureProcessExit()` in `SDKAgent` only covered the happy path; sessions terminated through `SessionRoutes` or `worker-service` bypassed process cleanup entirely.
|
||||
|
||||
**Fix — dual-layer approach:**
|
||||
|
||||
1. **Immediate cleanup:** Added `ensureProcessExit()` calls to the `finally` blocks in both `SessionRoutes.ts` and `worker-service.ts`, ensuring every session exit path kills its subprocess
|
||||
2. **Periodic reaping:** Added `reapStaleSessions()` to `SessionManager` — a background interval that scans `~/.claude-mem/observer-sessions/` for stale PID files, verifies the process is still running, and kills any orphans with SIGKILL escalation
|
||||
|
||||
This ensures no observer subprocess survives beyond its session lifetime, even in crash scenarios.
|
||||
|
||||
## [v10.2.5] - 2026-02-18
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **Self-healing message queue**: Renamed `claimAndDelete` → `claimNextMessage` with atomic self-healing — automatically resets stale processing messages (>60s) back to pending before claiming, eliminating stuck messages from generator crashes without external timers
|
||||
- **Removed redundant idle-timeout reset**: The `resetStaleProcessingMessages()` call during idle timeout in worker-service was removed (startup reset kept), since the atomic self-healing in `claimNextMessage` now handles recovery inline
|
||||
- **TypeScript diagnostic fix**: Added `QUEUE` to logger `Component` type
|
||||
|
||||
### Tests
|
||||
|
||||
- 5 new tests for self-healing behavior (stuck recovery, active protection, atomicity, empty queue, session isolation)
|
||||
- 1 new integration test for stuck recovery in zombie-prevention suite
|
||||
- All existing queue tests updated for renamed method
|
||||
|
||||
## [v10.2.4] - 2026-02-18
|
||||
|
||||
## Chroma Vector DB Backfill Fix
|
||||
|
||||
Fixes the Chroma backfill system to correctly sync all SQLite observations into the vector database on worker startup.
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **Backfill all projects on startup** — `backfillAllProjects()` now runs on worker startup, iterating all projects in SQLite and syncing missing observations to Chroma. Previously `ensureBackfilled()` existed but was never called, leaving Chroma with incomplete data after cache clears.
|
||||
|
||||
- **Fixed critical collection routing bug** — Backfill now uses the shared `cm__claude-mem` collection (matching how DatabaseManager and SearchManager operate) instead of creating per-project orphan collections that no search path reads from.
|
||||
|
||||
- **Hardened collection name sanitization** — Project names with special characters (e.g., "YC Stuff") are sanitized for Chroma's naming constraints, including stripping trailing non-alphanumeric characters.
|
||||
|
||||
- **Eliminated shared mutable state** — `ensureBackfilled()` and `getExistingChromaIds()` now accept project as a parameter instead of mutating instance state, keeping a single Chroma connection while avoiding fragile property mutation across iterations.
|
||||
|
||||
- **Chroma readiness guard** — Backfill waits for Chroma server readiness before running, preventing spurious error logs when Chroma fails to start.
|
||||
|
||||
### Changed Files
|
||||
|
||||
- `src/services/sync/ChromaSync.ts` — Core backfill logic, sanitization, parameter passing
|
||||
- `src/services/worker-service.ts` — Startup backfill trigger + readiness guard
|
||||
- `src/utils/logger.ts` — Added `CHROMA_SYNC` log component
|
||||
|
||||
## [v10.2.3] - 2026-02-17
|
||||
|
||||
## Fix Chroma ONNX Model Cache Corruption
|
||||
|
||||
Addresses the persistent embedding pipeline failures reported across #1104, #1105, #1110, and subsequent sessions. Three root causes identified and fixed:
|
||||
|
||||
### Changes
|
||||
|
||||
- **Removed nuclear `bun pm cache rm`** from both `smart-install.js` and `sync-marketplace.cjs`. This was added in v10.2.2 for the now-removed sharp dependency but destroyed all cached packages, breaking the ONNX resolution chain.
|
||||
- **Added `bun install` in plugin cache directory** after marketplace sync. The cache directory had a `package.json` with `@chroma-core/default-embed` as a dependency but never ran install, so the worker couldn't resolve it at runtime.
|
||||
- **Moved HuggingFace model cache to `~/.claude-mem/models/`** outside `node_modules`. The ~23MB ONNX model was stored inside `node_modules/@huggingface/transformers/.cache/`, so any reinstall or cache clear corrupted it.
|
||||
- **Added self-healing retry** for Protobuf parsing failures. If the downloaded model is corrupted, the cache is cleared and re-downloaded automatically on next use.
|
||||
|
||||
### Files Changed
|
||||
|
||||
- `scripts/smart-install.js` — removed `bun pm cache rm`
|
||||
- `scripts/sync-marketplace.cjs` — removed `bun pm cache rm`, added `bun install` in cache dir
|
||||
- `src/services/sync/ChromaSync.ts` — moved model cache, added corruption recovery
|
||||
|
||||
## [v10.2.2] - 2026-02-17
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- **Removed `node-addon-api` dev dependency** — was only needed for `sharp`, which was already removed in v10.2.1
|
||||
- **Simplified native module cache clearing** in `smart-install.js` and `sync-marketplace.cjs` — replaced targeted `@img/sharp` directory deletion and lockfile removal with `bun pm cache rm`
|
||||
- Reduced ~30 lines of brittle file system manipulation to a clean Bun CLI command
|
||||
|
||||
## [v10.2.1] - 2026-02-16
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- **Bun install & sharp native modules**: Fixed stale native module cache issues on Bun updates, added `node-addon-api` as a dev dependency required by sharp (#1140)
|
||||
- **PendingMessageStore consolidation**: Deduplicated PendingMessageStore initialization in worker-service; added session-scoped filtering to `resetStaleProcessingMessages` to prevent cross-session message resets (#1140)
|
||||
- **Gemini empty response handling**: Fixed silent message deletion when Gemini returns empty summary responses — now logs a warning and preserves the original message (#1138)
|
||||
- **Idle timeout session scoping**: Fixed idle timeout handler to only reset messages for the timed-out session instead of globally resetting all sessions (#1138)
|
||||
- **Shell injection in sync-marketplace**: Replaced `execSync` with `spawnSync` for rsync calls to eliminate command injection via gitignore patterns (#1138)
|
||||
- **Sharp cache invalidation**: Added cache clearing for sharp's native bindings when Bun version changes (#1138)
|
||||
- **Marketplace install**: Switched marketplace sync from npm to bun for package installation consistency (#1140)
|
||||
|
||||
## [v10.1.0] - 2026-02-16
|
||||
|
||||
## SessionStart System Message & Cleaner Defaults
|
||||
@@ -1001,478 +1249,3 @@ Implemented interactive log filtering in the viewer UI:
|
||||
- **Smart Auto-Scroll** - Maintains scroll position when reviewing older logs
|
||||
- **Responsive Design** - Filter bar adapts to smaller screens
|
||||
|
||||
## [v8.5.3] - 2026-01-02
|
||||
|
||||
# 🛡️ Error Handling Hardening & Developer Tools
|
||||
|
||||
Version 8.5.3 introduces comprehensive error handling improvements that prevent silent failures and reduce debugging time from hours to minutes. This release also adds new developer tools for queue management and log monitoring.
|
||||
|
||||
---
|
||||
|
||||
## 🔴 Critical Error Handling Improvements
|
||||
|
||||
### The Problem
|
||||
A single overly-broad try-catch block caused a **10-hour debugging session** by silently swallowing errors. This pattern was pervasive throughout the codebase, creating invisible failure modes.
|
||||
|
||||
### The Solution
|
||||
|
||||
**Automated Anti-Pattern Detection** (`scripts/detect-error-handling-antipatterns.ts`)
|
||||
- Detects 7 categories of error handling anti-patterns
|
||||
- Enforces zero-tolerance policy for empty catch blocks
|
||||
- Identifies large try-catch blocks (>10 lines) that mask specific errors
|
||||
- Flags missing error logging that causes silent failures
|
||||
- Supports approved overrides with justification comments
|
||||
- Exit code 1 if critical issues detected (enforceable in CI)
|
||||
|
||||
**New Error Handling Standards** (Added to `CLAUDE.md`)
|
||||
- **5-Question Pre-Flight Checklist**: Required before writing any try-catch
|
||||
1. What SPECIFIC error am I catching?
|
||||
2. Show documentation proving this error can occur
|
||||
3. Why can't this error be prevented?
|
||||
4. What will the catch block DO?
|
||||
5. Why shouldn't this error propagate?
|
||||
- **Forbidden Patterns**: Empty catch, catch without logging, large try blocks, promise catch without handlers
|
||||
- **Allowed Patterns**: Specific errors, logged failures, minimal scope, explicit recovery
|
||||
- **Meta-Rule**: Uncertainty triggers research, NOT try-catch
|
||||
|
||||
### Fixes Applied
|
||||
|
||||
**Wave 1: Empty Catch Blocks** (5 files)
|
||||
- `import-xml-observations.ts` - Log skipped invalid JSON
|
||||
- `bun-path.ts` - Log when bun not in PATH
|
||||
- `cursor-utils.ts` - Log failed registry reads & corrupt MCP config
|
||||
- `worker-utils.ts` - Log failed health checks
|
||||
|
||||
**Wave 2: Promise Catches on Critical Paths** (8 locations)
|
||||
- `worker-service.ts` - Background initialization failures
|
||||
- `SDKAgent.ts` - Session processor errors (2 locations)
|
||||
- `GeminiAgent.ts` - Finalization failures (2 locations)
|
||||
- `OpenRouterAgent.ts` - Finalization failures (2 locations)
|
||||
- `SessionManager.ts` - Generator promise failures
|
||||
|
||||
**Wave 3: Comprehensive Audit** (29 catch blocks)
|
||||
- Added logging to 16 catch blocks (UI, servers, worker, routes, services)
|
||||
- Documented 13 intentional exceptions with justification comments
|
||||
- All patterns now follow error handling guidelines with appropriate log levels
|
||||
|
||||
### Approved Override System
|
||||
|
||||
For justified exceptions (performance-critical paths, expected failures), use:
|
||||
```typescript
|
||||
// [APPROVED OVERRIDE]: Brief technical justification
|
||||
try {
|
||||
// code
|
||||
} catch {
|
||||
// allowed exception
|
||||
}
|
||||
```
|
||||
|
||||
**Progress**: 163 anti-patterns → 26 approved overrides (84% reduction in silent failures)
|
||||
|
||||
---
|
||||
|
||||
## 🗂️ Queue Management Features
|
||||
|
||||
**New Commands**
|
||||
- `npm run queue:clear` - Interactive removal of failed messages
|
||||
- `npm run queue:clear -- --all` - Clear all messages (pending, processing, failed)
|
||||
- `npm run queue:clear -- --force` - Non-interactive mode
|
||||
|
||||
**HTTP API Endpoints**
|
||||
- `DELETE /api/pending-queue/failed` - Remove failed messages
|
||||
- `DELETE /api/pending-queue/all` - Complete queue reset
|
||||
|
||||
Failed messages exceed max retry count and remain for debugging. These commands provide clean queue maintenance.
|
||||
|
||||
---
|
||||
|
||||
## 🪵 Developer Console (Chrome DevTools Style)
|
||||
|
||||
**UI Improvements**
|
||||
- Bottom drawer console (slides up from bottom-left corner)
|
||||
- Draggable resize handle for height adjustment
|
||||
- Auto-refresh toggle (2s interval)
|
||||
- Clear logs button with confirmation
|
||||
- Monospace font (SF Mono/Monaco/Consolas)
|
||||
- Minimum height: 150px, adjustable to window height - 100px
|
||||
|
||||
**API Endpoints**
|
||||
- `GET /api/logs` - Fetch last 1000 lines of current day's log
|
||||
- `DELETE /api/logs` - Clear current log file
|
||||
|
||||
Logs viewer accessible via floating console button in UI.
|
||||
|
||||
---
|
||||
|
||||
## 📚 Architecture Documentation
|
||||
|
||||
**Session ID Architecture** (`docs/SESSION_ID_ARCHITECTURE.md`)
|
||||
- Comprehensive documentation of 1:1 session mapping guarantees
|
||||
- 19 validation tests proving UNIQUE constraints and resume consistency
|
||||
- Documents single-transition vulnerability (application-level enforcement)
|
||||
- Complete reference for session lifecycle management
|
||||
|
||||
---
|
||||
|
||||
## 📊 Impact Summary
|
||||
|
||||
- **Debugging Time**: 10 hours → minutes (proper error visibility)
|
||||
- **Test Coverage**: +19 critical architecture validation tests
|
||||
- **Silent Failures**: 84% reduction (163 → 26 approved exceptions)
|
||||
- **Protection**: Automated detection prevents regression
|
||||
- **Developer UX**: Console logs, queue management, comprehensive docs
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Technical Details
|
||||
|
||||
**Files Changed**: 25+ files across error handling, queue management, UI, and documentation
|
||||
|
||||
**Critical Path Protection**
|
||||
These files now have strict error propagation (no catch-and-continue):
|
||||
- `SDKAgent.ts`
|
||||
- `GeminiAgent.ts`
|
||||
- `OpenRouterAgent.ts`
|
||||
- `SessionStore.ts`
|
||||
- `worker-service.ts`
|
||||
|
||||
**Build Verification**: All changes tested, build successful
|
||||
|
||||
---
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.5.2...v8.5.3
|
||||
|
||||
## [v8.5.2] - 2025-12-31
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
### Fixed SDK Agent Memory Leak (#499)
|
||||
|
||||
Fixed a critical memory leak where Claude SDK child processes were never terminated after sessions completed. Over extended usage, this caused hundreds of orphaned processes consuming 40GB+ of RAM.
|
||||
|
||||
**Root Cause:**
|
||||
- When the SDK agent generator completed naturally (no more messages to process), the `AbortController` was never aborted
|
||||
- Child processes spawned by the Agent SDK remained running indefinitely
|
||||
- Sessions stayed in memory (by design for future events) but underlying processes were never cleaned up
|
||||
|
||||
**Fix:**
|
||||
- Added proper cleanup to SessionRoutes finally block
|
||||
- Now calls `abortController.abort()` when generator completes with no pending work
|
||||
- Creates new `AbortController` when crash recovery restarts generators
|
||||
- Ensures cleanup happens even if recovery logic fails
|
||||
|
||||
**Impact:**
|
||||
- Prevents orphaned `claude` processes from accumulating
|
||||
- Eliminates multi-gigabyte memory leaks during normal usage
|
||||
- Maintains crash recovery functionality with proper resource cleanup
|
||||
|
||||
Thanks to @yonnock for the detailed bug report and investigation in #499!
|
||||
|
||||
## [v8.5.1] - 2025-12-30
|
||||
|
||||
## Bug Fix
|
||||
|
||||
**Fixed**: Migration 17 column rename failing for databases in intermediate states (#481)
|
||||
|
||||
### Problem
|
||||
Migration 17 renamed session ID columns but used a single check to determine if ALL tables were migrated. This caused errors for databases in partial migration states:
|
||||
- `no such column: sdk_session_id` (when columns already renamed)
|
||||
- `table observations has no column named memory_session_id` (when not renamed)
|
||||
|
||||
### Solution
|
||||
- Rewrote migration 17 to check **each table individually** before renaming
|
||||
- Added `safeRenameColumn()` helper that handles all edge cases gracefully
|
||||
- Handles all database states: fresh, old, and partially migrated
|
||||
|
||||
### Who was affected
|
||||
- Users upgrading from pre-v8.2.6 versions
|
||||
- Users whose migration was interrupted (crash, restart, etc.)
|
||||
- Users who restored database from backup
|
||||
|
||||
---
|
||||
|
||||
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||||
|
||||
## [v8.5.0] - 2025-12-30
|
||||
|
||||
# Cursor Support Now Available 🎉
|
||||
|
||||
This is a major release introducing **full Cursor IDE support**. Claude-mem now works with Cursor, bringing persistent AI memory to Cursor users with or without a Claude Code subscription.
|
||||
|
||||
## Highlights
|
||||
|
||||
**Give Cursor persistent memory.** Every Cursor session starts fresh - your AI doesn't remember what it worked on yesterday. Claude-mem changes that. Your agent builds cumulative knowledge about your codebase, decisions, and patterns over time.
|
||||
|
||||
### Works Without Claude Code
|
||||
|
||||
You can now use claude-mem with Cursor using free AI providers:
|
||||
- **Gemini** (recommended): 1,500 free requests/day, no credit card required
|
||||
- **OpenRouter**: Access to 100+ models including free options
|
||||
- **Claude SDK**: For Claude Code subscribers
|
||||
|
||||
### Cross-Platform Support
|
||||
|
||||
Full support for all major platforms:
|
||||
- **macOS**: Bash scripts with `jq` and `curl`
|
||||
- **Linux**: Same toolchain as macOS
|
||||
- **Windows**: Native PowerShell scripts, no WSL required
|
||||
|
||||
## New Features
|
||||
|
||||
### Interactive Setup Wizard (`bun run cursor:setup`)
|
||||
A guided installer that:
|
||||
- Detects your environment (Claude Code present or not)
|
||||
- Helps you choose and configure an AI provider
|
||||
- Installs Cursor hooks automatically
|
||||
- Starts the worker service
|
||||
- Verifies everything is working
|
||||
|
||||
### Cursor Lifecycle Hooks
|
||||
Complete hook integration with Cursor's native hook system:
|
||||
- `session-init.sh/.ps1` - Session start with context injection
|
||||
- `user-message.sh/.ps1` - User prompt capture
|
||||
- `save-observation.sh/.ps1` - Tool usage logging
|
||||
- `save-file-edit.sh/.ps1` - File edit tracking
|
||||
- `session-summary.sh/.ps1` - Session end summary
|
||||
- `context-inject.sh/.ps1` - Load relevant history
|
||||
|
||||
### Context Injection via `.cursor/rules`
|
||||
Relevant past context is automatically injected into Cursor sessions via the `.cursor/rules/claude-mem-context.mdc` file, giving your AI immediate awareness of prior work.
|
||||
|
||||
### Project Registry
|
||||
Multi-project support with automatic project detection:
|
||||
- Projects registered in `~/.claude-mem/cursor-projects.json`
|
||||
- Context automatically scoped to current project
|
||||
- Works across multiple workspaces simultaneously
|
||||
|
||||
### MCP Search Tools
|
||||
Full MCP server integration for Cursor:
|
||||
- `search` - Find observations by query, date, type
|
||||
- `timeline` - Get context around specific observations
|
||||
- `get_observations` - Fetch full details for filtered IDs
|
||||
|
||||
## New Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `bun run cursor:setup` | Interactive setup wizard |
|
||||
| `bun run cursor:install` | Install Cursor hooks |
|
||||
| `bun run cursor:uninstall` | Remove Cursor hooks |
|
||||
| `bun run cursor:status` | Check hook installation status |
|
||||
|
||||
## Documentation
|
||||
|
||||
Full documentation available at [docs.claude-mem.ai/cursor](https://docs.claude-mem.ai/cursor):
|
||||
- Cursor Integration Overview
|
||||
- Gemini Setup Guide (free tier)
|
||||
- OpenRouter Setup Guide
|
||||
- Troubleshooting
|
||||
|
||||
## Getting Started
|
||||
|
||||
### For Cursor-Only Users (No Claude Code)
|
||||
|
||||
```bash
|
||||
git clone https://github.com/thedotmack/claude-mem.git
|
||||
cd claude-mem && bun install && bun run build
|
||||
bun run cursor:setup
|
||||
```
|
||||
|
||||
### For Claude Code Users
|
||||
|
||||
```bash
|
||||
/plugin marketplace add thedotmack/claude-mem
|
||||
/plugin install claude-mem
|
||||
claude-mem cursor install
|
||||
```
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.2.10...v8.5.0
|
||||
|
||||
## [v8.2.10] - 2025-12-30
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- **Auto-restart worker on version mismatch** (#484): When the plugin updates but the worker was already running on the old version, the worker now automatically restarts instead of failing with 400 errors.
|
||||
|
||||
### Changes
|
||||
- `/api/version` endpoint now returns the built-in version (compiled at build time) instead of reading from disk
|
||||
- `worker-service start` command checks for version mismatch and auto-restarts if needed
|
||||
- Downgraded hook version mismatch warning to debug logging (now handled by auto-restart)
|
||||
|
||||
Thanks @yungweng for the detailed bug report!
|
||||
|
||||
## [v8.2.9] - 2025-12-29
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- **Worker Service**: Remove file-based locking and improve Windows stability
|
||||
- Replaced file-based locking with health-check-first approach for cleaner mutual exclusion
|
||||
- Removed AbortSignal.timeout() calls to reduce Bun libuv assertion errors on Windows
|
||||
- Added 500ms shutdown delays on Windows to prevent zombie ports
|
||||
- Reduced hook timeout values for improved responsiveness
|
||||
- Increased worker readiness polling duration from 5s to 15s
|
||||
|
||||
## Internal Changes
|
||||
|
||||
- Updated worker CLI scripts to reference worker-service.cjs directly
|
||||
- Simplified hook command configurations
|
||||
|
||||
## [v8.2.8] - 2025-12-29
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- Fixed orphaned chroma-mcp processes during shutdown (#489)
|
||||
- Added graceful shutdown handling with signal handlers registered early in WorkerService lifecycle
|
||||
- Ensures ChromaSync subprocess cleanup even when interrupted during initialization
|
||||
- Removes PID file during shutdown to prevent stale process tracking
|
||||
|
||||
## Technical Details
|
||||
|
||||
This patch release addresses a race condition where SIGTERM/SIGINT signals arriving during ChromaSync initialization could leave orphaned chroma-mcp processes. The fix moves signal handler registration from the start() method to the constructor, ensuring cleanup handlers exist throughout the entire initialization lifecycle.
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.2.7...v8.2.8
|
||||
|
||||
## [v8.2.7] - 2025-12-29
|
||||
|
||||
## What's Changed
|
||||
|
||||
### Token Optimizations
|
||||
- Simplified MCP server tool definitions for reduced token usage
|
||||
- Removed outdated troubleshooting and mem-search skill documentation
|
||||
- Enhanced search parameter descriptions for better clarity
|
||||
- Streamlined MCP workflows for improved efficiency
|
||||
|
||||
This release significantly reduces the token footprint of the plugin's MCP tools and documentation.
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.2.6...v8.2.7
|
||||
|
||||
## [v8.2.6] - 2025-12-29
|
||||
|
||||
## What's Changed
|
||||
|
||||
### Bug Fixes & Improvements
|
||||
- Session ID semantic renaming for clarity (content_session_id, memory_session_id)
|
||||
- Queue system simplification with unified processing logic
|
||||
- Memory session ID capture for agent resume functionality
|
||||
- Comprehensive test suite for session ID refactoring
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.2.5...v8.2.6
|
||||
|
||||
## [v8.2.5] - 2025-12-28
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- **Logger**: Enhanced Error object handling in debug mode to prevent empty JSON serialization
|
||||
- **ChromaSync**: Refactored DatabaseManager to initialize ChromaSync lazily, removing background backfill on startup
|
||||
- **SessionManager**: Simplified message handling and removed linger timeout that was blocking completion
|
||||
|
||||
## Technical Details
|
||||
|
||||
This patch release addresses several issues discovered after the session continuity fix:
|
||||
|
||||
1. Logger now properly serializes Error objects with stack traces in debug mode
|
||||
2. ChromaSync initialization is now lazy to prevent silent failures during startup
|
||||
3. Session linger timeout removed to eliminate artificial 5-second delays on session completion
|
||||
|
||||
Full changelog: https://github.com/thedotmack/claude-mem/compare/v8.2.4...v8.2.5
|
||||
|
||||
## [v8.2.4] - 2025-12-28
|
||||
|
||||
Patch release v8.2.4
|
||||
|
||||
## [v8.2.3] - 2025-12-27
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- Fix worker port environment variable in smart-install script
|
||||
- Implement file-based locking mechanism for worker operations to prevent race conditions
|
||||
- Fix restart command references in documentation (changed from `claude-mem restart` to `npm run worker:restart`)
|
||||
|
||||
## [v8.2.2] - 2025-12-27
|
||||
|
||||
## What's Changed
|
||||
|
||||
### Features
|
||||
- Add OpenRouter provider settings and documentation
|
||||
- Add modal footer with save button and status indicators
|
||||
- Implement self-spawn pattern for background worker execution
|
||||
|
||||
### Bug Fixes
|
||||
- Resolve critical error handling issues in worker lifecycle
|
||||
- Handle Windows/Unix kill errors in orphaned process cleanup
|
||||
- Validate spawn pid before writing PID file
|
||||
- Handle process exit in waitForProcessesExit filter
|
||||
- Use readiness endpoint for health checks instead of port check
|
||||
- Add missing OpenRouter and Gemini settings to settingKeys array
|
||||
|
||||
### Other Changes
|
||||
- Enhance error handling and validation in agents and routes
|
||||
- Delete obsolete process management files (ProcessManager, worker-wrapper, worker-cli)
|
||||
- Update hooks.json to use worker-service.cjs CLI
|
||||
- Add comprehensive tests for hook constants and worker spawn functionality
|
||||
|
||||
## [v8.2.1] - 2025-12-27
|
||||
|
||||
## 🔧 Worker Lifecycle Hardening
|
||||
|
||||
This patch release addresses critical bugs discovered during PR review of the self-spawn pattern introduced in 8.2.0. The worker daemon now handles edge cases robustly across both Unix and Windows platforms.
|
||||
|
||||
### 🐛 Critical Bug Fixes
|
||||
|
||||
#### Process Exit Detection Fixed
|
||||
The `waitForProcessesExit` function was crashing when processes exited during monitoring. The `process.kill(pid, 0)` call throws when a process no longer exists, which was not being caught. Now wrapped in try/catch to correctly identify exited processes.
|
||||
|
||||
#### Spawn PID Validation
|
||||
The worker daemon now validates that `spawn()` actually returned a valid PID before writing to the PID file. Previously, spawn failures could leave invalid PID files that broke subsequent lifecycle operations.
|
||||
|
||||
#### Cross-Platform Orphan Cleanup
|
||||
- **Unix**: Replaced single `kill` command with individual `process.kill()` calls wrapped in try/catch, so one already-exited process doesn't abort cleanup of remaining orphans
|
||||
- **Windows**: Wrapped `taskkill` calls in try/catch for the same reason
|
||||
|
||||
#### Health Check Reliability
|
||||
Changed `waitForHealth` to use the `/api/readiness` endpoint (returns 503 until fully initialized) instead of just checking if the port is in use. Callers now wait for *actual* worker readiness, not just network availability.
|
||||
|
||||
### 🔄 Refactoring
|
||||
|
||||
#### Code Consolidation (-580 lines)
|
||||
Deleted obsolete process management infrastructure that was replaced by the self-spawn pattern:
|
||||
- `src/services/process/ProcessManager.ts` (433 lines) - PID management now in worker-service
|
||||
- `src/cli/worker-cli.ts` (81 lines) - CLI handling now in worker-service
|
||||
- `src/services/worker-wrapper.ts` (157 lines) - Replaced by `--daemon` flag
|
||||
|
||||
#### Updated Hook Commands
|
||||
All hooks now use `worker-service.cjs` CLI directly instead of the deleted `worker-cli.js`.
|
||||
|
||||
### ⏱️ Timeout Adjustments
|
||||
|
||||
Increased timeouts throughout for compatibility with slow systems:
|
||||
|
||||
| Component | Before | After |
|
||||
|-----------|--------|-------|
|
||||
| Default hook timeout | 120s | 300s |
|
||||
| Health check timeout | 1s | 30s |
|
||||
| Health check retries | 15 | 300 |
|
||||
| Context initialization | 30s | 300s |
|
||||
| MCP connection | 15s | 300s |
|
||||
| PowerShell commands | 5s | 60s |
|
||||
| Git commands | 30s | 300s |
|
||||
| NPM install | 120s | 600s |
|
||||
| Hook worker commands | 30s | 180s |
|
||||
|
||||
### 🧪 Testing
|
||||
|
||||
Added comprehensive test suites:
|
||||
- `tests/hook-constants.test.ts` - Validates timeout configurations
|
||||
- `tests/worker-spawn.test.ts` - Tests worker CLI and health endpoints
|
||||
|
||||
### 🛡️ Additional Robustness
|
||||
|
||||
- PID validation in restart command (matches start command behavior)
|
||||
- Try/catch around `forceKillProcess()` for graceful shutdown
|
||||
- Try/catch around `getChildProcesses()` for Windows failures
|
||||
- Improved logging for PID file operations and HTTP shutdown
|
||||
|
||||
---
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.2.0...v8.2.1
|
||||
|
||||
|
||||
@@ -14,6 +14,10 @@ Claude-mem is a Claude Code plugin providing persistent memory across sessions.
|
||||
|
||||
**Search Skill** (`plugin/skills/mem-search/SKILL.md`) - HTTP API for searching past work, auto-invoked when users ask about history
|
||||
|
||||
**Planning Skill** (`plugin/skills/make-plan/SKILL.md`) - Orchestrator instructions for creating phased implementation plans with documentation discovery
|
||||
|
||||
**Execution Skill** (`plugin/skills/do/SKILL.md`) - Orchestrator instructions for executing phased plans using subagents
|
||||
|
||||
**Chroma** (`src/services/sync/ChromaSync.ts`) - Vector embeddings for semantic search
|
||||
|
||||
**Viewer UI** (`src/ui/viewer/`) - React interface at http://localhost:37777, built to `plugin/ui/viewer.html`
|
||||
|
||||
@@ -198,7 +198,7 @@ See [Architecture Overview](https://docs.claude-mem.ai/architecture/overview) fo
|
||||
|
||||
## MCP Search Tools
|
||||
|
||||
Claude-Mem provides intelligent memory search through **5 MCP tools** following a token-efficient **3-layer workflow pattern**:
|
||||
Claude-Mem provides intelligent memory search through **4 MCP tools** following a token-efficient **3-layer workflow pattern**:
|
||||
|
||||
**The 3-Layer Workflow:**
|
||||
|
||||
@@ -211,7 +211,6 @@ Claude-Mem provides intelligent memory search through **5 MCP tools** following
|
||||
- Start with `search` to get an index of results
|
||||
- Use `timeline` to see what was happening around specific observations
|
||||
- Use `get_observations` to fetch full details for relevant IDs
|
||||
- Use `save_memory` to manually store important information
|
||||
- **~10x token savings** by filtering before fetching details
|
||||
|
||||
**Available MCP Tools:**
|
||||
@@ -219,8 +218,6 @@ Claude-Mem provides intelligent memory search through **5 MCP tools** following
|
||||
1. **`search`** - Search memory index with full-text queries, filters by type/date/project
|
||||
2. **`timeline`** - Get chronological context around a specific observation or query
|
||||
3. **`get_observations`** - Fetch full observation details by IDs (always batch multiple IDs)
|
||||
4. **`save_memory`** - Manually save a memory/observation for semantic search
|
||||
5. **`__IMPORTANT`** - Workflow documentation (always visible to Claude)
|
||||
|
||||
**Example Usage:**
|
||||
|
||||
@@ -232,9 +229,6 @@ search(query="authentication bug", type="bugfix", limit=10)
|
||||
|
||||
// Step 3: Fetch full details
|
||||
get_observations(ids=[123, 456])
|
||||
|
||||
// Save important information manually
|
||||
save_memory(text="API requires auth header X-API-Key", title="API Auth")
|
||||
```
|
||||
|
||||
See [Search Tools Guide](https://docs.claude-mem.ai/usage/search-tools) for detailed examples.
|
||||
|
||||
@@ -5,7 +5,7 @@ set -euo pipefail
|
||||
# Usage: curl -fsSL https://install.cmem.ai | bash
|
||||
# or: curl -fsSL https://install.cmem.ai | bash -s -- --provider=gemini --api-key=YOUR_KEY
|
||||
|
||||
INSTALLER_URL="https://raw.githubusercontent.com/thedotmack/claude-mem/main/installer/dist/index.js"
|
||||
INSTALLER_URL="https://install.cmem.ai/installer.js"
|
||||
|
||||
# Colors
|
||||
RED='\033[0;31m'
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,5 +1,8 @@
|
||||
{
|
||||
"$schema": "https://openapi.vercel.sh/vercel.json",
|
||||
"rewrites": [
|
||||
{ "source": "/", "destination": "/install.sh" }
|
||||
],
|
||||
"headers": [
|
||||
{
|
||||
"source": "/(.*)\\.sh",
|
||||
|
||||
Vendored
+2107
File diff suppressed because it is too large
Load Diff
@@ -3,10 +3,10 @@
|
||||
"name": "Claude-Mem (Persistent Memory)",
|
||||
"description": "Official OpenClaw plugin for Claude-Mem. Records observations from embedded runner sessions and streams them to messaging channels.",
|
||||
"kind": "memory",
|
||||
"version": "1.0.0",
|
||||
"version": "10.4.1",
|
||||
"author": "thedotmack",
|
||||
"homepage": "https://claude-mem.com",
|
||||
"skills": ["skills/make-plan", "skills/do-plan"],
|
||||
"skills": ["skills/make-plan", "skills/do"],
|
||||
"configSchema": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
|
||||
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../plugin/skills/do/SKILL.md
|
||||
@@ -1,63 +0,0 @@
|
||||
---
|
||||
name: make-plan
|
||||
description: Create a detailed, phased implementation plan with documentation discovery. Use when asked to plan a feature, task, or multi-step implementation — especially before executing with do-plan.
|
||||
---
|
||||
|
||||
# Make Plan
|
||||
|
||||
You are an ORCHESTRATOR. Create an LLM-friendly plan in phases that can be executed consecutively in new chat contexts.
|
||||
|
||||
## Delegation Model
|
||||
|
||||
Use subagents for *fact gathering and extraction* (docs, examples, signatures, grep results). Keep *synthesis and plan authoring* with the orchestrator (phase boundaries, task framing, final wording). If a subagent report is incomplete or lacks evidence, re-check with targeted reads/greps before finalizing.
|
||||
|
||||
### Subagent Reporting Contract (MANDATORY)
|
||||
|
||||
Each subagent response must include:
|
||||
1. Sources consulted (files/URLs) and what was read
|
||||
2. Concrete findings (exact API names/signatures; exact file paths/locations)
|
||||
3. Copy-ready snippet locations (example files/sections to copy)
|
||||
4. "Confidence" note + known gaps (what might still be missing)
|
||||
|
||||
Reject and redeploy the subagent if it reports conclusions without sources.
|
||||
|
||||
## Plan Structure
|
||||
|
||||
### Phase 0: Documentation Discovery (ALWAYS FIRST)
|
||||
|
||||
Before planning implementation, deploy "Documentation Discovery" subagents to:
|
||||
1. Search for and read relevant documentation, examples, and existing patterns
|
||||
2. Identify the actual APIs, methods, and signatures available (not assumed)
|
||||
3. Create a brief "Allowed APIs" list citing specific documentation sources
|
||||
4. Note any anti-patterns to avoid (methods that DON'T exist, deprecated parameters)
|
||||
|
||||
The orchestrator consolidates findings into a single Phase 0 output.
|
||||
|
||||
### Each Implementation Phase Must Include
|
||||
|
||||
1. **What to implement** — Frame tasks to COPY from docs, not transform existing code
|
||||
- Good: "Copy the V2 session pattern from docs/examples.ts:45-60"
|
||||
- Bad: "Migrate the existing code to V2"
|
||||
2. **Documentation references** — Cite specific files/lines for patterns to follow
|
||||
3. **Verification checklist** — How to prove this phase worked (tests, grep checks)
|
||||
4. **Anti-pattern guards** — What NOT to do (invented APIs, undocumented params)
|
||||
|
||||
### Final Phase: Verification
|
||||
|
||||
1. Verify all implementations match documentation
|
||||
2. Check for anti-patterns (grep for known bad patterns)
|
||||
3. Run tests to confirm functionality
|
||||
|
||||
## Key Principles
|
||||
|
||||
- Documentation Availability ≠ Usage: Explicitly require reading docs
|
||||
- Task Framing Matters: Direct agents to docs, not just outcomes
|
||||
- Verify > Assume: Require proof, not assumptions about APIs
|
||||
- Session Boundaries: Each phase should be self-contained with its own doc references
|
||||
|
||||
## Anti-Patterns to Prevent
|
||||
|
||||
- Inventing API methods that "should" exist
|
||||
- Adding parameters not in documentation
|
||||
- Skipping verification steps
|
||||
- Assuming structure without checking examples
|
||||
+1
@@ -0,0 +1 @@
|
||||
../../../plugin/skills/make-plan/SKILL.md
|
||||
@@ -642,6 +642,9 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void {
|
||||
const toolName = event.toolName;
|
||||
if (!toolName) return;
|
||||
|
||||
// Skip memory_ tools to prevent recursive observation loops
|
||||
if (toolName.startsWith("memory_")) return;
|
||||
|
||||
const contentSessionId = getContentSessionId(ctx.sessionKey);
|
||||
|
||||
// Extract result text from all content blocks
|
||||
@@ -654,6 +657,12 @@ export default function claudeMemPlugin(api: OpenClawPluginApi): void {
|
||||
.join("\n");
|
||||
}
|
||||
|
||||
// Truncate long responses to prevent oversized payloads
|
||||
const MAX_TOOL_RESPONSE_LENGTH = 1000;
|
||||
if (toolResponseText.length > MAX_TOOL_RESPONSE_LENGTH) {
|
||||
toolResponseText = toolResponseText.slice(0, MAX_TOOL_RESPONSE_LENGTH);
|
||||
}
|
||||
|
||||
// Fire-and-forget: send observation + sync MEMORY.md in parallel
|
||||
workerPostFireAndForget(workerPort, "/api/sessions/observations", {
|
||||
contentSessionId,
|
||||
|
||||
+1
-4
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "10.2.1",
|
||||
"version": "10.4.4",
|
||||
"description": "Memory compression system for Claude Code - persist context across sessions",
|
||||
"keywords": [
|
||||
"claude",
|
||||
@@ -98,9 +98,7 @@
|
||||
"dependencies": {
|
||||
"@anthropic-ai/claude-agent-sdk": "^0.1.76",
|
||||
"@modelcontextprotocol/sdk": "^1.25.1",
|
||||
"@chroma-core/default-embed": "^0.1.9",
|
||||
"ansi-to-html": "^0.7.2",
|
||||
"chromadb": "^3.2.2",
|
||||
"dompurify": "^3.3.1",
|
||||
"express": "^4.18.2",
|
||||
"glob": "^11.0.3",
|
||||
@@ -118,7 +116,6 @@
|
||||
"@types/react": "^18.3.5",
|
||||
"@types/react-dom": "^18.3.0",
|
||||
"esbuild": "^0.27.2",
|
||||
"node-addon-api": "^8.5.0",
|
||||
"np": "^11.0.2",
|
||||
"tsx": "^4.20.6",
|
||||
"typescript": "^5.3.0"
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
# Fix: SessionStart Hook "startup hook error" — Worker Not Waiting
|
||||
|
||||
## Root Cause
|
||||
|
||||
The **installed plugin** (`~/.claude/plugins/marketplaces/thedotmack/`) is version **10.2.5** and has **none** of the recent fixes:
|
||||
|
||||
| Fix | Repo Status | Installed Status |
|
||||
|-----|-------------|-----------------|
|
||||
| Hook group split (smart-install isolated from worker start) | In `plugin/hooks/hooks.json` | **Missing** — all 3 hooks in one group, smart-install failure blocks worker |
|
||||
| `waitForReadiness()` after spawn | In `src/services/infrastructure/HealthMonitor.ts` | **Missing** — 0 occurrences in installed `worker-service.cjs` |
|
||||
| Early `initializationCompleteFlag` (after DB+search, not MCP) | In `src/services/worker-service.ts` | **Missing** — flag set after MCP connection (5+ minute wait) |
|
||||
|
||||
The changes exist in source code but were **never built and synced** to the installed location.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Build and Sync
|
||||
|
||||
```bash
|
||||
npm run build-and-sync
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
# 1. Confirm waitForReadiness exists in installed build
|
||||
grep -c "waitForReadiness" ~/.claude/plugins/marketplaces/thedotmack/plugin/scripts/worker-service.cjs
|
||||
# Expected: > 0
|
||||
|
||||
# 2. Confirm hooks.json has two SessionStart groups (the split)
|
||||
python3 -c "import json; d=json.load(open('$(echo $HOME)/.claude/plugins/marketplaces/thedotmack/plugin/hooks/hooks.json')); print('SessionStart groups:', len(d['hooks']['SessionStart']))"
|
||||
# Expected: 2
|
||||
|
||||
# 3. Confirm initializationCompleteFlag is set before MCP connection
|
||||
grep -n "Core initialization complete" ~/.claude/plugins/marketplaces/thedotmack/plugin/scripts/worker-service.cjs | head -1
|
||||
# Expected: appears BEFORE "MCP server connected"
|
||||
```
|
||||
|
||||
## Phase 2: Restart Worker and Test
|
||||
|
||||
```bash
|
||||
# Stop existing worker
|
||||
bun plugin/scripts/worker-service.cjs stop
|
||||
|
||||
# Verify stopped
|
||||
curl -s http://127.0.0.1:37777/api/health && echo "STILL RUNNING" || echo "STOPPED"
|
||||
```
|
||||
|
||||
Then start a new Claude Code session and verify:
|
||||
- No "SessionStart:startup hook error" messages
|
||||
- Worker is running: `curl http://127.0.0.1:37777/api/health`
|
||||
- Readiness endpoint works: `curl http://127.0.0.1:37777/api/readiness`
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "10.2.1",
|
||||
"version": "10.4.4",
|
||||
"description": "Persistent memory system for Claude Code - seamlessly preserve context across sessions",
|
||||
"author": {
|
||||
"name": "Alex Newman"
|
||||
|
||||
@@ -1,43 +0,0 @@
|
||||
---
|
||||
description: "Execute a plan using subagents for implementation"
|
||||
argument-hint: "[task or plan reference]"
|
||||
---
|
||||
|
||||
You are an ORCHESTRATOR.
|
||||
|
||||
Primary instruction: deploy subagents to execute *all* work for #$ARGUMENTS.
|
||||
Do not do the work yourself except to coordinate, route context, and verify that each subagent completed its assigned checklist.
|
||||
|
||||
Deploy subagents to execute each phase of #$ARGUMENTS independently and consecutively. For every checklist item below, explicitly deploy (or reuse) a subagent responsible for that item and record its outcome before proceeding.
|
||||
|
||||
## Execution Protocol (Orchestrator-Driven)
|
||||
|
||||
Orchestrator rules:
|
||||
- Each phase uses fresh subagents where noted (or when context is large/unclear).
|
||||
- The orchestrator assigns one clear objective per subagent and requires evidence (commands run, outputs, files changed).
|
||||
- Do not advance to the next step until the assigned subagent reports completion and the orchestrator confirms it matches the plan.
|
||||
|
||||
### During Each Phase:
|
||||
Deploy an "Implementation" subagent to:
|
||||
1. Execute the implementation as specified
|
||||
2. COPY patterns from documentation, don't invent
|
||||
3. Cite documentation sources in code comments when using unfamiliar APIs
|
||||
4. If an API seems missing, STOP and verify - don't assume it exists
|
||||
|
||||
### After Each Phase:
|
||||
Deploy subagents for each post-phase responsibility:
|
||||
1. **Run verification checklist** - Deploy a "Verification" subagent to prove the phase worked
|
||||
2. **Anti-pattern check** - Deploy an "Anti-pattern" subagent to grep for known bad patterns from the plan
|
||||
3. **Code quality review** - Deploy a "Code Quality" subagent to review changes
|
||||
4. **Commit only if verified** - Deploy a "Commit" subagent *only after* verification passes; otherwise, do not commit
|
||||
|
||||
### Between Phases:
|
||||
Deploy a "Branch/Sync" subagent to:
|
||||
- Push to working branch after each verified phase
|
||||
- Prepare the next phase handoff so the next phase's subagents start fresh but have plan context
|
||||
|
||||
## Failure Modes to Prevent
|
||||
- Don't invent APIs that "should" exist - verify against docs
|
||||
- Don't add undocumented parameters - copy exact signatures
|
||||
- Don't skip verification - deploy a verification subagent and run the checklist
|
||||
- Don't commit before verification passes (or without explicit orchestrator approval)
|
||||
@@ -1,66 +0,0 @@
|
||||
---
|
||||
description: "Create an implementation plan with documentation discovery"
|
||||
argument-hint: "[feature or task description]"
|
||||
---
|
||||
|
||||
You are an ORCHESTRATOR.
|
||||
|
||||
Create an LLM-friendly plan in phases that can be executed consecutively in new chat contexts.
|
||||
|
||||
Delegation model (because subagents can under-report):
|
||||
- Use subagents for *fact gathering and extraction* (docs, examples, signatures, grep results).
|
||||
- Keep *synthesis and plan authoring* with the orchestrator (phase boundaries, task framing, final wording).
|
||||
- If a subagent report is incomplete or lacks evidence, the orchestrator must re-check with targeted reads/greps before finalizing the plan.
|
||||
|
||||
Subagent reporting contract (MANDATORY):
|
||||
- Each subagent response must include:
|
||||
1) Sources consulted (files/URLs) and what was read
|
||||
2) Concrete findings (exact API names/signatures; exact file paths/locations)
|
||||
3) Copy-ready snippet locations (example files/sections to copy)
|
||||
4) "Confidence" note + known gaps (what might still be missing)
|
||||
- Reject and redeploy the subagent if it reports conclusions without sources.
|
||||
|
||||
## Plan Structure Requirements
|
||||
|
||||
### Phase 0: Documentation Discovery (ALWAYS FIRST)
|
||||
Before planning implementation, you MUST:
|
||||
Deploy one or more "Documentation Discovery" subagents to:
|
||||
1. Search for and read relevant documentation, examples, and existing patterns
|
||||
2. Identify the actual APIs, methods, and signatures available (not assumed)
|
||||
3. Create a brief "Allowed APIs" list citing specific documentation sources
|
||||
4. Note any anti-patterns to avoid (methods that DON'T exist, deprecated parameters)
|
||||
|
||||
Then the orchestrator consolidates their findings into a single Phase 0 output.
|
||||
|
||||
### Each Implementation Phase Must Include:
|
||||
1. **What to implement** - Frame tasks to COPY from docs, not transform existing code
|
||||
- Good: "Copy the V2 session pattern from docs/examples.ts:45-60"
|
||||
- Bad: "Migrate the existing code to V2"
|
||||
2. **Documentation references** - Cite specific files/lines for patterns to follow
|
||||
3. **Verification checklist** - How to prove this phase worked (tests, grep checks)
|
||||
4. **Anti-pattern guards** - What NOT to do (invented APIs, undocumented params)
|
||||
|
||||
Subagent-friendly split:
|
||||
- Subagents can propose candidate doc references and verification commands.
|
||||
- The orchestrator must write the final phase text, ensuring tasks are copy-based, scoped, and independently executable.
|
||||
|
||||
### Final Phase: Verification
|
||||
1. Verify all implementations match documentation
|
||||
2. Check for anti-patterns (grep for known bad patterns)
|
||||
3. Run tests to confirm functionality
|
||||
|
||||
Delegation guidance:
|
||||
- Deploy a "Verification" subagent to draft the checklist and commands.
|
||||
- The orchestrator must review the checklist for completeness and ensure it maps to earlier phase outputs.
|
||||
|
||||
## Key Principles
|
||||
- Documentation Availability ≠ Usage: Explicitly require reading docs
|
||||
- Task Framing Matters: Direct agents to docs, not just outcomes
|
||||
- Verify > Assume: Require proof, not assumptions about APIs
|
||||
- Session Boundaries: Each phase should be self-contained with its own doc references
|
||||
|
||||
## Anti-Patterns to Prevent
|
||||
- Inventing API methods that "should" exist
|
||||
- Adding parameters not in documentation
|
||||
- Skipping verification steps
|
||||
- Assuming structure without checking examples
|
||||
+15
-25
@@ -7,8 +7,8 @@
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "${CLAUDE_PLUGIN_ROOT}/scripts/setup.sh",
|
||||
"timeout": 120
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; \"$_R/scripts/setup.sh\"",
|
||||
"timeout": 300
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -19,17 +19,22 @@
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/smart-install.js\"",
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/smart-install.js\"",
|
||||
"timeout": 300
|
||||
},
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": "startup|clear|compact",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" start",
|
||||
"timeout": 60
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" hook claude-code context",
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code context",
|
||||
"timeout": 60
|
||||
}
|
||||
]
|
||||
@@ -40,12 +45,7 @@
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
|
||||
"timeout": 60
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" hook claude-code session-init",
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code session-init",
|
||||
"timeout": 60
|
||||
}
|
||||
]
|
||||
@@ -57,12 +57,7 @@
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
|
||||
"timeout": 60
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" hook claude-code observation",
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code observation",
|
||||
"timeout": 120
|
||||
}
|
||||
]
|
||||
@@ -73,17 +68,12 @@
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
|
||||
"timeout": 60
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" hook claude-code summarize",
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code summarize",
|
||||
"timeout": 120
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" hook claude-code session-complete",
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code session-complete",
|
||||
"timeout": 30
|
||||
}
|
||||
]
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-mem-plugin",
|
||||
"version": "10.2.1",
|
||||
"version": "10.4.4",
|
||||
"private": true,
|
||||
"description": "Runtime dependencies for claude-mem bundled hooks",
|
||||
"type": "module",
|
||||
|
||||
@@ -12,12 +12,37 @@
|
||||
* Fixes #818: Worker fails to start on fresh install
|
||||
*/
|
||||
import { spawnSync, spawn } from 'child_process';
|
||||
import { existsSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { existsSync, readFileSync } from 'fs';
|
||||
import { join, dirname, resolve } from 'path';
|
||||
import { homedir } from 'os';
|
||||
import { fileURLToPath } from 'url';
|
||||
|
||||
const IS_WINDOWS = process.platform === 'win32';
|
||||
|
||||
// Self-resolve plugin root when CLAUDE_PLUGIN_ROOT is not set by Claude Code.
|
||||
// Upstream bug: anthropics/claude-code#24529 — Stop hooks (and on Linux, all hooks)
|
||||
// don't receive CLAUDE_PLUGIN_ROOT, causing script paths to resolve to /scripts/...
|
||||
// which doesn't exist. This fallback derives the plugin root from bun-runner.js's
|
||||
// own filesystem location (this file lives in <plugin-root>/scripts/).
|
||||
const __bun_runner_dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const RESOLVED_PLUGIN_ROOT = process.env.CLAUDE_PLUGIN_ROOT || resolve(__bun_runner_dirname, '..');
|
||||
|
||||
/**
|
||||
* Fix script path arguments that were broken by empty CLAUDE_PLUGIN_ROOT.
|
||||
* When CLAUDE_PLUGIN_ROOT is empty, "${CLAUDE_PLUGIN_ROOT}/scripts/foo.cjs"
|
||||
* expands to "/scripts/foo.cjs" which doesn't exist. Detect this and rewrite
|
||||
* the path using our self-resolved plugin root.
|
||||
*/
|
||||
function fixBrokenScriptPath(argPath) {
|
||||
if (argPath.startsWith('/scripts/') && !existsSync(argPath)) {
|
||||
const fixedPath = join(RESOLVED_PLUGIN_ROOT, argPath);
|
||||
if (existsSync(fixedPath)) {
|
||||
return fixedPath;
|
||||
}
|
||||
}
|
||||
return argPath;
|
||||
}
|
||||
|
||||
/**
|
||||
* Find Bun executable - checks PATH first, then common install locations
|
||||
*/
|
||||
@@ -54,6 +79,24 @@ function findBun() {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Early exit if plugin is disabled in Claude Code settings (#781).
|
||||
// Sync read + JSON parse — fastest possible check before spawning Bun.
|
||||
function isPluginDisabledInClaudeSettings() {
|
||||
try {
|
||||
const configDir = process.env.CLAUDE_CONFIG_DIR || join(homedir(), '.claude');
|
||||
const settingsPath = join(configDir, 'settings.json');
|
||||
if (!existsSync(settingsPath)) return false;
|
||||
const settings = JSON.parse(readFileSync(settingsPath, 'utf-8'));
|
||||
return settings?.enabledPlugins?.['claude-mem@thedotmack'] === false;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (isPluginDisabledInClaudeSettings()) {
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Get args: node bun-runner.js <script> [args...]
|
||||
const args = process.argv.slice(2);
|
||||
|
||||
@@ -62,6 +105,9 @@ if (args.length === 0) {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Fix broken script paths caused by empty CLAUDE_PLUGIN_ROOT (#1215)
|
||||
args[0] = fixBrokenScriptPath(args[0]);
|
||||
|
||||
const bunPath = findBun();
|
||||
|
||||
if (!bunPath) {
|
||||
|
||||
Executable
BIN
Binary file not shown.
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -4,16 +4,72 @@
|
||||
*
|
||||
* Ensures Bun runtime and uv (Python package manager) are installed
|
||||
* (auto-installs if missing) and handles dependency installation when needed.
|
||||
*
|
||||
* Resolves the install directory from CLAUDE_PLUGIN_ROOT (set by Claude Code
|
||||
* for both cache and marketplace installs), falling back to script location
|
||||
* and legacy paths.
|
||||
*/
|
||||
import { existsSync, readFileSync, writeFileSync } from 'fs';
|
||||
import { execSync, spawnSync } from 'child_process';
|
||||
import { join } from 'path';
|
||||
import { join, dirname } from 'path';
|
||||
import { homedir } from 'os';
|
||||
import { fileURLToPath } from 'url';
|
||||
|
||||
const ROOT = join(homedir(), '.claude', 'plugins', 'marketplaces', 'thedotmack');
|
||||
const MARKER = join(ROOT, '.install-version');
|
||||
// Early exit if plugin is disabled in Claude Code settings (#781)
|
||||
function isPluginDisabledInClaudeSettings() {
|
||||
try {
|
||||
const configDir = process.env.CLAUDE_CONFIG_DIR || join(homedir(), '.claude');
|
||||
const settingsPath = join(configDir, 'settings.json');
|
||||
if (!existsSync(settingsPath)) return false;
|
||||
const settings = JSON.parse(readFileSync(settingsPath, 'utf-8'));
|
||||
return settings?.enabledPlugins?.['claude-mem@thedotmack'] === false;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (isPluginDisabledInClaudeSettings()) {
|
||||
process.exit(0);
|
||||
}
|
||||
const IS_WINDOWS = process.platform === 'win32';
|
||||
|
||||
/**
|
||||
* Resolve the plugin root directory where dependencies should be installed.
|
||||
*
|
||||
* Priority:
|
||||
* 1. CLAUDE_PLUGIN_ROOT env var (set by Claude Code for hooks — works for
|
||||
* both cache-based and marketplace installs)
|
||||
* 2. Script location (dirname of this file, up one level from scripts/)
|
||||
* 3. XDG path (~/.config/claude/plugins/marketplaces/thedotmack)
|
||||
* 4. Legacy path (~/.claude/plugins/marketplaces/thedotmack)
|
||||
*/
|
||||
function resolveRoot() {
|
||||
// CLAUDE_PLUGIN_ROOT is the authoritative location set by Claude Code
|
||||
if (process.env.CLAUDE_PLUGIN_ROOT) {
|
||||
const root = process.env.CLAUDE_PLUGIN_ROOT;
|
||||
if (existsSync(join(root, 'package.json'))) return root;
|
||||
}
|
||||
|
||||
// Derive from script location (this file is in <root>/scripts/)
|
||||
try {
|
||||
const scriptDir = dirname(fileURLToPath(import.meta.url));
|
||||
const candidate = dirname(scriptDir);
|
||||
if (existsSync(join(candidate, 'package.json'))) return candidate;
|
||||
} catch {
|
||||
// import.meta.url not available
|
||||
}
|
||||
|
||||
// Probe XDG path, then legacy
|
||||
const marketplaceRel = join('plugins', 'marketplaces', 'thedotmack');
|
||||
const xdg = join(homedir(), '.config', 'claude', marketplaceRel);
|
||||
if (existsSync(join(xdg, 'package.json'))) return xdg;
|
||||
|
||||
return join(homedir(), '.claude', marketplaceRel);
|
||||
}
|
||||
|
||||
const ROOT = resolveRoot();
|
||||
const MARKER = join(ROOT, '.install-version');
|
||||
|
||||
/**
|
||||
* Check if Bun is installed and accessible
|
||||
*/
|
||||
@@ -287,7 +343,7 @@ function installUv() {
|
||||
* Add shell alias for claude-mem command
|
||||
*/
|
||||
function installCLI() {
|
||||
const WORKER_CLI = join(ROOT, 'plugin', 'scripts', 'worker-service.cjs');
|
||||
const WORKER_CLI = join(ROOT, 'scripts', 'worker-service.cjs');
|
||||
const bunPath = getBunPath() || 'bun';
|
||||
const aliasLine = `alias claude-mem='${bunPath} "${WORKER_CLI}"'`;
|
||||
const markerPath = join(ROOT, '.cli-installed');
|
||||
@@ -405,6 +461,31 @@ function installDeps() {
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* Verify that critical runtime modules are resolvable from the install directory.
|
||||
* Returns true if all critical modules exist, false otherwise.
|
||||
*/
|
||||
function verifyCriticalModules() {
|
||||
const pkg = JSON.parse(readFileSync(join(ROOT, 'package.json'), 'utf-8'));
|
||||
const dependencies = Object.keys(pkg.dependencies || {});
|
||||
|
||||
const missing = [];
|
||||
for (const dep of dependencies) {
|
||||
// Check that the module directory exists in node_modules
|
||||
const modulePath = join(ROOT, 'node_modules', ...dep.split('/'));
|
||||
if (!existsSync(modulePath)) {
|
||||
missing.push(dep);
|
||||
}
|
||||
}
|
||||
|
||||
if (missing.length > 0) {
|
||||
console.error(`❌ Post-install check failed: missing modules: ${missing.join(', ')}`);
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
// Main execution
|
||||
try {
|
||||
// Step 1: Ensure Bun is installed and meets minimum version (REQUIRED)
|
||||
@@ -456,6 +537,21 @@ try {
|
||||
const newVersion = pkg.version;
|
||||
|
||||
installDeps();
|
||||
|
||||
// Verify critical modules are resolvable
|
||||
if (!verifyCriticalModules()) {
|
||||
console.error('⚠️ Retrying install with npm...');
|
||||
try {
|
||||
execSync('npm install --production', { cwd: ROOT, stdio: 'inherit', shell: IS_WINDOWS });
|
||||
} catch {
|
||||
// npm also failed
|
||||
}
|
||||
if (!verifyCriticalModules()) {
|
||||
console.error('❌ Dependencies could not be installed. Plugin may not work correctly.');
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
console.error('✅ Dependencies installed');
|
||||
|
||||
// Auto-restart worker to pick up new code
|
||||
|
||||
+479
-460
File diff suppressed because one or more lines are too long
@@ -1,5 +1,5 @@
|
||||
---
|
||||
name: do-plan
|
||||
name: do
|
||||
description: Execute a phased implementation plan using subagents. Use when asked to execute, run, or carry out a plan — especially one created by make-plan.
|
||||
---
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
---
|
||||
name: make-plan
|
||||
description: Create a detailed, phased implementation plan with documentation discovery. Use when asked to plan a feature, task, or multi-step implementation — especially before executing with do.
|
||||
---
|
||||
|
||||
# Make Plan
|
||||
|
||||
You are an ORCHESTRATOR. Create an LLM-friendly plan in phases that can be executed consecutively in new chat contexts.
|
||||
|
||||
## Delegation Model
|
||||
|
||||
Use subagents for *fact gathering and extraction* (docs, examples, signatures, grep results). Keep *synthesis and plan authoring* with the orchestrator (phase boundaries, task framing, final wording). If a subagent report is incomplete or lacks evidence, re-check with targeted reads/greps before finalizing.
|
||||
|
||||
### Subagent Reporting Contract (MANDATORY)
|
||||
|
||||
Each subagent response must include:
|
||||
1. Sources consulted (files/URLs) and what was read
|
||||
2. Concrete findings (exact API names/signatures; exact file paths/locations)
|
||||
3. Copy-ready snippet locations (example files/sections to copy)
|
||||
4. "Confidence" note + known gaps (what might still be missing)
|
||||
|
||||
Reject and redeploy the subagent if it reports conclusions without sources.
|
||||
|
||||
## Plan Structure
|
||||
|
||||
### Phase 0: Documentation Discovery (ALWAYS FIRST)
|
||||
|
||||
Before planning implementation, deploy "Documentation Discovery" subagents to:
|
||||
1. Search for and read relevant documentation, examples, and existing patterns
|
||||
2. Identify the actual APIs, methods, and signatures available (not assumed)
|
||||
3. Create a brief "Allowed APIs" list citing specific documentation sources
|
||||
4. Note any anti-patterns to avoid (methods that DON'T exist, deprecated parameters)
|
||||
|
||||
The orchestrator consolidates findings into a single Phase 0 output.
|
||||
|
||||
### Each Implementation Phase Must Include
|
||||
|
||||
1. **What to implement** — Frame tasks to COPY from docs, not transform existing code
|
||||
- Good: "Copy the V2 session pattern from docs/examples.ts:45-60"
|
||||
- Bad: "Migrate the existing code to V2"
|
||||
2. **Documentation references** — Cite specific files/lines for patterns to follow
|
||||
3. **Verification checklist** — How to prove this phase worked (tests, grep checks)
|
||||
4. **Anti-pattern guards** — What NOT to do (invented APIs, undocumented params)
|
||||
|
||||
### Final Phase: Verification
|
||||
|
||||
1. Verify all implementations match documentation
|
||||
2. Check for anti-patterns (grep for known bad patterns)
|
||||
3. Run tests to confirm functionality
|
||||
|
||||
## Key Principles
|
||||
|
||||
- Documentation Availability ≠ Usage: Explicitly require reading docs
|
||||
- Task Framing Matters: Direct agents to docs, not just outcomes
|
||||
- Verify > Assume: Require proof, not assumptions about APIs
|
||||
- Session Boundaries: Each phase should be self-contained with its own doc references
|
||||
|
||||
## Anti-Patterns to Prevent
|
||||
|
||||
- Inventing API methods that "should" exist
|
||||
- Adding parameters not in documentation
|
||||
- Skipping verification steps
|
||||
- Assuming structure without checking examples
|
||||
@@ -93,20 +93,6 @@ get_observations(ids=[11131, 10942])
|
||||
|
||||
**Returns:** Complete observation objects with title, subtitle, narrative, facts, concepts, files (~500-1000 tokens each)
|
||||
|
||||
## Saving Memories
|
||||
|
||||
Use the `save_memory` MCP tool to store manual observations:
|
||||
|
||||
```
|
||||
save_memory(text="Important discovery about the auth system", title="Auth Architecture", project="my-project")
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
|
||||
- `text` (string, required) - Content to remember
|
||||
- `title` (string, optional) - Short title, auto-generated if omitted
|
||||
- `project` (string, optional) - Project name, defaults to "claude-mem"
|
||||
|
||||
## Examples
|
||||
|
||||
**Find recent bug fixes:**
|
||||
|
||||
@@ -162,6 +162,20 @@ async function buildHooks() {
|
||||
const contextGenStats = fs.statSync(`${hooksDir}/${CONTEXT_GENERATOR.name}.cjs`);
|
||||
console.log(`✓ context-generator built (${(contextGenStats.size / 1024).toFixed(2)} KB)`);
|
||||
|
||||
// Verify critical distribution files exist (skills are source files, not build outputs)
|
||||
console.log('\n📋 Verifying distribution files...');
|
||||
const requiredDistributionFiles = [
|
||||
'plugin/skills/mem-search/SKILL.md',
|
||||
'plugin/hooks/hooks.json',
|
||||
'plugin/.claude-plugin/plugin.json',
|
||||
];
|
||||
for (const filePath of requiredDistributionFiles) {
|
||||
if (!fs.existsSync(filePath)) {
|
||||
throw new Error(`Missing required distribution file: ${filePath}`);
|
||||
}
|
||||
}
|
||||
console.log('✓ All required distribution files present');
|
||||
|
||||
console.log('\n✅ Worker service, MCP server, and context generator built successfully!');
|
||||
console.log(` Output: ${hooksDir}/`);
|
||||
console.log(` - Worker: worker-service.cjs`);
|
||||
|
||||
@@ -279,6 +279,11 @@ function formatObservationsForClaudeMd(observations: ObservationRow[], folderPat
|
||||
* which only writes to existing folders.
|
||||
*/
|
||||
function writeClaudeMdToFolderForRegenerate(folderPath: string, newContent: string): void {
|
||||
const resolvedPath = path.resolve(folderPath);
|
||||
|
||||
// Never write inside .git directories — corrupts refs (#1165)
|
||||
if (resolvedPath.includes('/.git/') || resolvedPath.includes('\\.git\\') || resolvedPath.endsWith('/.git') || resolvedPath.endsWith('\\.git')) return;
|
||||
|
||||
const claudeMdPath = path.join(folderPath, 'CLAUDE.md');
|
||||
const tempFile = `${claudeMdPath}.tmp`;
|
||||
|
||||
|
||||
+54
-27
@@ -5,36 +5,42 @@
|
||||
* Ensures Bun runtime and uv (Python package manager) are installed
|
||||
* (auto-installs if missing) and handles dependency installation when needed.
|
||||
*/
|
||||
import { existsSync, readFileSync, writeFileSync, rmSync } from 'fs';
|
||||
import { existsSync, readFileSync, writeFileSync } from 'fs';
|
||||
import { execSync, spawnSync } from 'child_process';
|
||||
import { join } from 'path';
|
||||
import { join, dirname } from 'path';
|
||||
import { homedir } from 'os';
|
||||
import { fileURLToPath } from 'url';
|
||||
|
||||
const IS_WINDOWS = process.platform === 'win32';
|
||||
|
||||
/**
|
||||
* Resolve the marketplace root directory.
|
||||
* Resolve the plugin root directory where dependencies should be installed.
|
||||
*
|
||||
* Claude Code may store plugins under either `~/.claude/plugins/` (legacy) or
|
||||
* `~/.config/claude/plugins/` (XDG-compliant, e.g. Nix-managed installs).
|
||||
* When `CLAUDE_PLUGIN_ROOT` is set we derive the base from it; otherwise we
|
||||
* probe both candidate paths and fall back to the legacy location.
|
||||
* Priority:
|
||||
* 1. CLAUDE_PLUGIN_ROOT env var (set by Claude Code for hooks — works for
|
||||
* both cache-based and marketplace installs)
|
||||
* 2. Script location (dirname of this file, up one level from scripts/)
|
||||
* 3. XDG path (~/.config/claude/plugins/marketplaces/thedotmack)
|
||||
* 4. Legacy path (~/.claude/plugins/marketplaces/thedotmack)
|
||||
*/
|
||||
function resolveRoot() {
|
||||
const marketplaceRel = join('plugins', 'marketplaces', 'thedotmack');
|
||||
|
||||
// Derive from CLAUDE_PLUGIN_ROOT (e.g. .../plugins/cache/thedotmack/claude-mem/<ver>)
|
||||
// CLAUDE_PLUGIN_ROOT is the authoritative location set by Claude Code
|
||||
if (process.env.CLAUDE_PLUGIN_ROOT) {
|
||||
let dir = process.env.CLAUDE_PLUGIN_ROOT;
|
||||
const cacheIndex = dir.indexOf(join('plugins', 'cache'));
|
||||
if (cacheIndex !== -1) {
|
||||
const base = dir.substring(0, cacheIndex);
|
||||
const candidate = join(base, marketplaceRel);
|
||||
if (existsSync(join(candidate, 'package.json'))) return candidate;
|
||||
}
|
||||
const root = process.env.CLAUDE_PLUGIN_ROOT;
|
||||
if (existsSync(join(root, 'package.json'))) return root;
|
||||
}
|
||||
|
||||
// Probe XDG path first, then legacy
|
||||
// Derive from script location (this file is in <root>/scripts/)
|
||||
try {
|
||||
const scriptDir = dirname(fileURLToPath(import.meta.url));
|
||||
const candidate = dirname(scriptDir);
|
||||
if (existsSync(join(candidate, 'package.json'))) return candidate;
|
||||
} catch {
|
||||
// import.meta.url not available
|
||||
}
|
||||
|
||||
// Probe XDG path, then legacy
|
||||
const marketplaceRel = join('plugins', 'marketplaces', 'thedotmack');
|
||||
const xdg = join(homedir(), '.config', 'claude', marketplaceRel);
|
||||
if (existsSync(join(xdg, 'package.json'))) return xdg;
|
||||
|
||||
@@ -260,15 +266,6 @@ function installDeps() {
|
||||
|
||||
console.error('📦 Installing dependencies with Bun...');
|
||||
|
||||
// Clear stale native module cache (sharp/libvips) to prevent broken dylib references.
|
||||
// Bun's cache can retain native binaries that reference companion libraries at
|
||||
// broken relative paths after version upgrades.
|
||||
const bunCacheImgDir = join(homedir(), '.bun', 'install', 'cache', '@img');
|
||||
if (existsSync(bunCacheImgDir)) {
|
||||
console.error(' Clearing stale native module cache (@img/sharp)...');
|
||||
rmSync(bunCacheImgDir, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
// Quote path for Windows paths with spaces
|
||||
const bunCmd = IS_WINDOWS && bunPath.includes(' ') ? `"${bunPath}"` : bunPath;
|
||||
|
||||
@@ -284,12 +281,42 @@ function installDeps() {
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* Verify that critical runtime modules are resolvable from the install directory.
|
||||
* Returns true if all critical modules exist, false otherwise.
|
||||
*/
|
||||
function verifyCriticalModules() {
|
||||
const pkg = JSON.parse(readFileSync(join(ROOT, 'package.json'), 'utf-8'));
|
||||
const dependencies = Object.keys(pkg.dependencies || {});
|
||||
|
||||
const missing = [];
|
||||
for (const dep of dependencies) {
|
||||
const modulePath = join(ROOT, 'node_modules', ...dep.split('/'));
|
||||
if (!existsSync(modulePath)) {
|
||||
missing.push(dep);
|
||||
}
|
||||
}
|
||||
|
||||
if (missing.length > 0) {
|
||||
console.error(`❌ Post-install check failed: missing modules: ${missing.join(', ')}`);
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
// Main execution
|
||||
try {
|
||||
if (!isBunInstalled()) installBun();
|
||||
if (!isUvInstalled()) installUv();
|
||||
if (needsInstall()) {
|
||||
installDeps();
|
||||
|
||||
if (!verifyCriticalModules()) {
|
||||
console.error('❌ Dependencies could not be installed. Plugin may not work correctly.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
console.error('✅ Dependencies installed');
|
||||
}
|
||||
} catch (e) {
|
||||
|
||||
@@ -80,24 +80,6 @@ try {
|
||||
{ stdio: 'inherit' }
|
||||
);
|
||||
|
||||
// Remove stale lockfiles before install — they pin old native dep versions
|
||||
const { unlinkSync, rmSync } = require('fs');
|
||||
for (const lockfile of ['package-lock.json', 'bun.lock']) {
|
||||
const lockpath = path.join(INSTALLED_PATH, lockfile);
|
||||
if (existsSync(lockpath)) {
|
||||
unlinkSync(lockpath);
|
||||
console.log(`Removed stale ${lockfile}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Clear stale native module cache (sharp/libvips) — Bun's cache can retain
|
||||
// native binaries that reference companion libraries at broken relative paths
|
||||
const bunCacheImgDir = path.join(os.homedir(), '.bun', 'install', 'cache', '@img');
|
||||
if (existsSync(bunCacheImgDir)) {
|
||||
rmSync(bunCacheImgDir, { recursive: true, force: true });
|
||||
console.log('Cleared stale native module cache (@img/sharp)');
|
||||
}
|
||||
|
||||
console.log('Running bun install in marketplace...');
|
||||
execSync(
|
||||
'cd ~/.claude/plugins/marketplaces/thedotmack/ && bun install',
|
||||
@@ -117,6 +99,10 @@ try {
|
||||
{ stdio: 'inherit' }
|
||||
);
|
||||
|
||||
// Install dependencies in cache directory so worker can resolve them
|
||||
console.log(`Running bun install in cache folder (version ${version})...`);
|
||||
execSync(`bun install`, { cwd: CACHE_VERSION_PATH, stdio: 'inherit' });
|
||||
|
||||
console.log('\x1b[32m%s\x1b[0m', 'Sync complete!');
|
||||
|
||||
// Trigger worker restart after file sync
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Wipes the Chroma data directory so backfillAllProjects rebuilds it on next worker start.
|
||||
* Chroma is always rebuildable from SQLite — this is safe.
|
||||
*/
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const os = require('os');
|
||||
|
||||
const chromaDir = path.join(os.homedir(), '.claude-mem', 'chroma');
|
||||
|
||||
if (fs.existsSync(chromaDir)) {
|
||||
const before = fs.readdirSync(chromaDir);
|
||||
console.log(`Wiping ${chromaDir} (${before.length} items)...`);
|
||||
fs.rmSync(chromaDir, { recursive: true, force: true });
|
||||
console.log('Done. Chroma will rebuild from SQLite on next worker restart.');
|
||||
} else {
|
||||
console.log('Chroma directory does not exist, nothing to wipe.');
|
||||
}
|
||||
@@ -6,7 +6,7 @@ export const claudeCodeAdapter: PlatformAdapter = {
|
||||
normalizeInput(raw) {
|
||||
const r = (raw ?? {}) as any;
|
||||
return {
|
||||
sessionId: r.session_id,
|
||||
sessionId: r.session_id ?? r.id ?? r.sessionId,
|
||||
cwd: r.cwd ?? process.cwd(),
|
||||
prompt: r.prompt,
|
||||
toolName: r.tool_name,
|
||||
|
||||
@@ -3,15 +3,20 @@ import type { PlatformAdapter, NormalizedHookInput, HookResult } from '../types.
|
||||
// Maps Cursor stdin format - field names differ from Claude Code
|
||||
// Cursor uses: conversation_id, workspace_roots[], result_json, command/output
|
||||
// Handle undefined input gracefully for hooks that don't receive stdin
|
||||
//
|
||||
// Cursor payload variations (#838, #1049):
|
||||
// Session ID: conversation_id, generation_id, or id
|
||||
// Prompt: prompt, query, input, or message (varies by Cursor version/hook type)
|
||||
// CWD: workspace_roots[0] or cwd
|
||||
export const cursorAdapter: PlatformAdapter = {
|
||||
normalizeInput(raw) {
|
||||
const r = (raw ?? {}) as any;
|
||||
// Cursor-specific: shell commands come as command/output instead of tool_name/input/response
|
||||
const isShellCommand = !!r.command && !r.tool_name;
|
||||
return {
|
||||
sessionId: r.conversation_id || r.generation_id, // conversation_id preferred
|
||||
cwd: r.workspace_roots?.[0] ?? process.cwd(), // First workspace root
|
||||
prompt: r.prompt,
|
||||
sessionId: r.conversation_id || r.generation_id || r.id,
|
||||
cwd: r.workspace_roots?.[0] ?? r.cwd ?? process.cwd(),
|
||||
prompt: r.prompt ?? r.query ?? r.input ?? r.message,
|
||||
toolName: isShellCommand ? 'Bash' : r.tool_name,
|
||||
toolInput: isShellCommand ? { command: r.command } : r.tool_input,
|
||||
toolResponse: isShellCommand ? { output: r.output } : r.result_json, // result_json not tool_response
|
||||
|
||||
@@ -8,7 +8,8 @@ export function getPlatformAdapter(platform: string): PlatformAdapter {
|
||||
case 'claude-code': return claudeCodeAdapter;
|
||||
case 'cursor': return cursorAdapter;
|
||||
case 'raw': return rawAdapter;
|
||||
default: throw new Error(`Unknown platform: ${platform}`);
|
||||
// Codex CLI and other compatible platforms use the raw adapter (accepts both camelCase and snake_case fields)
|
||||
default: return rawAdapter;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -264,6 +264,11 @@ function formatObservationsForClaudeMd(observations: ObservationRow[], folderPat
|
||||
* Only writes to folders that exist — never creates directories.
|
||||
*/
|
||||
function writeClaudeMdToFolder(folderPath: string, newContent: string): void {
|
||||
const resolvedPath = path.resolve(folderPath);
|
||||
|
||||
// Never write inside .git directories — corrupts refs (#1165)
|
||||
if (resolvedPath.includes('/.git/') || resolvedPath.includes('\\.git\\') || resolvedPath.endsWith('/.git') || resolvedPath.endsWith('\\.git')) return;
|
||||
|
||||
const claudeMdPath = path.join(folderPath, 'CLAUDE.md');
|
||||
const tempFile = `${claudeMdPath}.tmp`;
|
||||
|
||||
|
||||
@@ -10,6 +10,8 @@ import { ensureWorkerRunning, getWorkerPort } from '../../shared/worker-utils.js
|
||||
import { getProjectContext } from '../../utils/project-name.js';
|
||||
import { HOOK_EXIT_CODES } from '../../shared/hook-constants.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
import { SettingsDefaultsManager } from '../../shared/SettingsDefaultsManager.js';
|
||||
import { USER_SETTINGS_PATH } from '../../shared/paths.js';
|
||||
|
||||
export const contextHandler: EventHandler = {
|
||||
async execute(input: NormalizedHookInput): Promise<HookResult> {
|
||||
@@ -30,6 +32,10 @@ export const contextHandler: EventHandler = {
|
||||
const context = getProjectContext(cwd);
|
||||
const port = getWorkerPort();
|
||||
|
||||
// Check if terminal output should be shown (load settings early)
|
||||
const settings = SettingsDefaultsManager.loadFromFile(USER_SETTINGS_PATH);
|
||||
const showTerminalOutput = settings.CLAUDE_MEM_CONTEXT_SHOW_TERMINAL_OUTPUT === 'true';
|
||||
|
||||
// Pass all projects (parent + worktree if applicable) for unified timeline
|
||||
const projectsParam = context.allProjects.join(',');
|
||||
const url = `http://127.0.0.1:${port}/api/context/inject?projects=${encodeURIComponent(projectsParam)}`;
|
||||
@@ -37,11 +43,11 @@ export const contextHandler: EventHandler = {
|
||||
// Note: Removed AbortSignal.timeout due to Windows Bun cleanup issue (libuv assertion)
|
||||
// Worker service has its own timeouts, so client-side timeout is redundant
|
||||
try {
|
||||
// Fetch both markdown (for Claude context) and colored (for user display) truly in parallel
|
||||
// Fetch markdown (for Claude context) and optionally colored (for user display)
|
||||
const colorUrl = `${url}&colors=true`;
|
||||
const [response, colorResponse] = await Promise.all([
|
||||
fetch(url),
|
||||
fetch(colorUrl).catch(() => null)
|
||||
showTerminalOutput ? fetch(colorUrl).catch(() => null) : Promise.resolve(null)
|
||||
]);
|
||||
|
||||
if (!response.ok) {
|
||||
@@ -60,7 +66,8 @@ export const contextHandler: EventHandler = {
|
||||
|
||||
const additionalContext = contextResult.trim();
|
||||
const coloredTimeline = colorResult.trim();
|
||||
const systemMessage = coloredTimeline
|
||||
|
||||
const systemMessage = showTerminalOutput && coloredTimeline
|
||||
? `${coloredTimeline}\n\nView Observations Live @ http://localhost:${port}`
|
||||
: undefined;
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
|
||||
import type { EventHandler } from '../types.js';
|
||||
import { HOOK_EXIT_CODES } from '../../shared/hook-constants.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
import { contextHandler } from './context.js';
|
||||
import { sessionInitHandler } from './session-init.js';
|
||||
import { observationHandler } from './observation.js';
|
||||
@@ -46,7 +47,7 @@ const handlers: Record<EventType, EventHandler> = {
|
||||
export function getEventHandler(eventType: string): EventHandler {
|
||||
const handler = handlers[eventType as EventType];
|
||||
if (!handler) {
|
||||
console.error(`[claude-mem] Unknown event type: ${eventType}, returning no-op`);
|
||||
logger.warn('HOOK', `Unknown event type: ${eventType}, returning no-op`);
|
||||
return {
|
||||
async execute() {
|
||||
return { continue: true, suppressOutput: true, exitCode: HOOK_EXIT_CODES.SUCCESS };
|
||||
|
||||
@@ -24,6 +24,12 @@ export const sessionInitHandler: EventHandler = {
|
||||
|
||||
const { sessionId, cwd, prompt: rawPrompt } = input;
|
||||
|
||||
// Guard: Codex CLI and other platforms may not provide a session_id (#744)
|
||||
if (!sessionId) {
|
||||
logger.warn('HOOK', 'session-init: No sessionId provided, skipping (Codex CLI or unknown platform)');
|
||||
return { continue: true, suppressOutput: true, exitCode: HOOK_EXIT_CODES.SUCCESS };
|
||||
}
|
||||
|
||||
// Check if project is excluded from tracking
|
||||
const settings = SettingsDefaultsManager.loadFromFile(USER_SETTINGS_PATH);
|
||||
if (cwd && isProjectExcluded(cwd, settings.CLAUDE_MEM_EXCLUDED_PROJECTS)) {
|
||||
@@ -63,11 +69,12 @@ export const sessionInitHandler: EventHandler = {
|
||||
promptNumber: number;
|
||||
skipped?: boolean;
|
||||
reason?: string;
|
||||
contextInjected?: boolean;
|
||||
};
|
||||
const sessionDbId = initResult.sessionDbId;
|
||||
const promptNumber = initResult.promptNumber;
|
||||
|
||||
logger.debug('HOOK', 'session-init: Received from /api/sessions/init', { sessionDbId, promptNumber, skipped: initResult.skipped });
|
||||
logger.debug('HOOK', 'session-init: Received from /api/sessions/init', { sessionDbId, promptNumber, skipped: initResult.skipped, contextInjected: initResult.contextInjected });
|
||||
|
||||
// Debug-level alignment log for detailed tracing
|
||||
logger.debug('HOOK', `[ALIGNMENT] Hook Entry | contentSessionId=${sessionId} | prompt#=${promptNumber} | sessionDbId=${sessionDbId}`);
|
||||
@@ -80,6 +87,16 @@ export const sessionInitHandler: EventHandler = {
|
||||
return { continue: true, suppressOutput: true };
|
||||
}
|
||||
|
||||
// Skip SDK agent re-initialization if context was already injected for this session (#1079)
|
||||
// The prompt was already saved to the database by /api/sessions/init above —
|
||||
// no need to re-start the SDK agent on every turn
|
||||
if (initResult.contextInjected) {
|
||||
logger.info('HOOK', `INIT_COMPLETE | sessionDbId=${sessionDbId} | promptNumber=${promptNumber} | skipped_agent_init=true | reason=context_already_injected`, {
|
||||
sessionId: sessionDbId
|
||||
});
|
||||
return { continue: true, suppressOutput: true };
|
||||
}
|
||||
|
||||
// Only initialize SDK agent for Claude Code (not Cursor)
|
||||
// Cursor doesn't use the SDK agent - it only needs session/observation storage
|
||||
if (input.platform !== 'cursor' && sessionDbId) {
|
||||
|
||||
+14
-3
@@ -2,6 +2,7 @@ import { readJsonFromStdin } from './stdin-reader.js';
|
||||
import { getPlatformAdapter } from './adapters/index.js';
|
||||
import { getEventHandler } from './handlers/index.js';
|
||||
import { HOOK_EXIT_CODES } from '../shared/hook-constants.js';
|
||||
import { logger } from '../utils/logger.js';
|
||||
|
||||
export interface HookCommandOptions {
|
||||
/** If true, don't call process.exit() - let caller handle process lifecycle */
|
||||
@@ -65,6 +66,12 @@ export function isWorkerUnavailableError(error: unknown): boolean {
|
||||
}
|
||||
|
||||
export async function hookCommand(platform: string, event: string, options: HookCommandOptions = {}): Promise<number> {
|
||||
// Suppress stderr in hook context — Claude Code shows stderr as error UI (#1181)
|
||||
// Exit 1: stderr shown to user. Exit 2: stderr fed to Claude for processing.
|
||||
// All diagnostics go to log file via logger; stderr must stay clean.
|
||||
const originalStderrWrite = process.stderr.write.bind(process.stderr);
|
||||
process.stderr.write = (() => true) as typeof process.stderr.write;
|
||||
|
||||
try {
|
||||
const adapter = getPlatformAdapter(platform);
|
||||
const handler = getEventHandler(event);
|
||||
@@ -84,18 +91,22 @@ export async function hookCommand(platform: string, event: string, options: Hook
|
||||
} catch (error) {
|
||||
if (isWorkerUnavailableError(error)) {
|
||||
// Worker unavailable — degrade gracefully, don't block the user
|
||||
console.error(`[claude-mem] Worker unavailable, skipping hook: ${error instanceof Error ? error.message : error}`);
|
||||
// Log to file instead of stderr (#1181)
|
||||
logger.warn('HOOK', `Worker unavailable, skipping hook: ${error instanceof Error ? error.message : error}`);
|
||||
if (!options.skipExit) {
|
||||
process.exit(HOOK_EXIT_CODES.SUCCESS); // = 0 (graceful)
|
||||
}
|
||||
return HOOK_EXIT_CODES.SUCCESS;
|
||||
}
|
||||
|
||||
// Handler/client bug — show as blocking error so developers see it
|
||||
console.error(`Hook error: ${error}`);
|
||||
// Handler/client bug — log to file instead of stderr (#1181)
|
||||
logger.error('HOOK', `Hook error: ${error instanceof Error ? error.message : error}`, {}, error instanceof Error ? error : undefined);
|
||||
if (!options.skipExit) {
|
||||
process.exit(HOOK_EXIT_CODES.BLOCKING_ERROR); // = 2
|
||||
}
|
||||
return HOOK_EXIT_CODES.BLOCKING_ERROR;
|
||||
} finally {
|
||||
// Restore stderr for non-hook code paths (e.g., when skipExit is true and process continues as worker)
|
||||
process.stderr.write = originalStderrWrite;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -233,31 +233,6 @@ NEVER fetch full details without filtering first. 10x token savings.`,
|
||||
handler: async (args: any) => {
|
||||
return await callWorkerAPIPost('/api/observations/batch', args);
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'save_memory',
|
||||
description: 'Save a manual memory/observation for semantic search. Use this to remember important information.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
text: {
|
||||
type: 'string',
|
||||
description: 'Content to remember (required)'
|
||||
},
|
||||
title: {
|
||||
type: 'string',
|
||||
description: 'Short title (auto-generated from text if omitted)'
|
||||
},
|
||||
project: {
|
||||
type: 'string',
|
||||
description: 'Project name (uses "claude-mem" if omitted)'
|
||||
}
|
||||
},
|
||||
required: ['text']
|
||||
},
|
||||
handler: async (args: any) => {
|
||||
return await callWorkerAPIPost('/api/memory/save', args);
|
||||
}
|
||||
}
|
||||
];
|
||||
|
||||
|
||||
@@ -74,8 +74,8 @@ export function renderColorContextIndex(): string[] {
|
||||
`${colors.dim}Context Index: This semantic index (titles, types, files, tokens) is usually sufficient to understand past work.${colors.reset}`,
|
||||
'',
|
||||
`${colors.dim}When you need implementation details, rationale, or debugging context:${colors.reset}`,
|
||||
`${colors.dim} - Use MCP tools (search, get_observations) to fetch full observations on-demand${colors.reset}`,
|
||||
`${colors.dim} - Critical types ( bugfix, decision) often need detailed fetching${colors.reset}`,
|
||||
`${colors.dim} - Fetch by ID: get_observations([IDs]) for observations visible in this index${colors.reset}`,
|
||||
`${colors.dim} - Search history: Use the mem-search skill for past decisions, bugs, and deeper research${colors.reset}`,
|
||||
`${colors.dim} - Trust this index over re-reading code for past decisions and learnings${colors.reset}`,
|
||||
''
|
||||
];
|
||||
@@ -226,7 +226,7 @@ export function renderColorFooter(totalDiscoveryTokens: number, totalReadTokens:
|
||||
const workTokensK = Math.round(totalDiscoveryTokens / 1000);
|
||||
return [
|
||||
'',
|
||||
`${colors.dim}Access ${workTokensK}k tokens of past research & decisions for just ${totalReadTokens.toLocaleString()}t. Use MCP search tools to access memories by ID.${colors.reset}`
|
||||
`${colors.dim}Access ${workTokensK}k tokens of past research & decisions for just ${totalReadTokens.toLocaleString()}t. Use the claude-mem skill to access memories by ID.${colors.reset}`
|
||||
];
|
||||
}
|
||||
|
||||
|
||||
@@ -72,8 +72,8 @@ export function renderMarkdownContextIndex(): string[] {
|
||||
`**Context Index:** This semantic index (titles, types, files, tokens) is usually sufficient to understand past work.`,
|
||||
'',
|
||||
`When you need implementation details, rationale, or debugging context:`,
|
||||
`- Use MCP tools (search, get_observations) to fetch full observations on-demand`,
|
||||
`- Critical types ( bugfix, decision) often need detailed fetching`,
|
||||
`- Fetch by ID: get_observations([IDs]) for observations visible in this index`,
|
||||
`- Search history: Use the mem-search skill for past decisions, bugs, and deeper research`,
|
||||
`- Trust this index over re-reading code for past decisions and learnings`,
|
||||
''
|
||||
];
|
||||
@@ -229,7 +229,7 @@ export function renderMarkdownFooter(totalDiscoveryTokens: number, totalReadToke
|
||||
const workTokensK = Math.round(totalDiscoveryTokens / 1000);
|
||||
return [
|
||||
'',
|
||||
`Access ${workTokensK}k tokens of past research & decisions for just ${totalReadTokens.toLocaleString()}t. Use MCP search tools to access memories by ID.`
|
||||
`Access ${workTokensK}k tokens of past research & decisions for just ${totalReadTokens.toLocaleString()}t. Use the claude-mem skill to access memories by ID.`
|
||||
];
|
||||
}
|
||||
|
||||
|
||||
@@ -30,9 +30,9 @@ export interface CloseableDatabase {
|
||||
}
|
||||
|
||||
/**
|
||||
* Stoppable service interface for Chroma server
|
||||
* Stoppable service interface for ChromaMcpManager
|
||||
*/
|
||||
export interface StoppableServer {
|
||||
export interface StoppableService {
|
||||
stop(): Promise<void>;
|
||||
}
|
||||
|
||||
@@ -44,7 +44,7 @@ export interface GracefulShutdownConfig {
|
||||
sessionManager: ShutdownableService;
|
||||
mcpClient?: CloseableClient;
|
||||
dbManager?: CloseableDatabase;
|
||||
chromaServer?: StoppableServer;
|
||||
chromaMcpManager?: StoppableService;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -79,11 +79,11 @@ export async function performGracefulShutdown(config: GracefulShutdownConfig): P
|
||||
logger.info('SYSTEM', 'MCP client closed');
|
||||
}
|
||||
|
||||
// STEP 5: Stop Chroma server (local mode only)
|
||||
if (config.chromaServer) {
|
||||
logger.info('SHUTDOWN', 'Stopping Chroma server...');
|
||||
await config.chromaServer.stop();
|
||||
logger.info('SHUTDOWN', 'Chroma server stopped');
|
||||
// STEP 5: Stop Chroma MCP connection
|
||||
if (config.chromaMcpManager) {
|
||||
logger.info('SHUTDOWN', 'Stopping Chroma MCP connection...');
|
||||
await config.chromaMcpManager.stop();
|
||||
logger.info('SHUTDOWN', 'Chroma MCP connection stopped');
|
||||
}
|
||||
|
||||
// STEP 6: Close database connection (includes ChromaSync cleanup)
|
||||
|
||||
@@ -29,31 +29,49 @@ export async function isPortInUse(port: number): Promise<boolean> {
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait for the worker HTTP server to become responsive (liveness check)
|
||||
* Uses /api/health instead of /api/readiness because:
|
||||
* - /api/health returns 200 as soon as HTTP server is listening
|
||||
* - /api/readiness waits for full initialization (MCP connection can take 5+ minutes)
|
||||
* See: https://github.com/thedotmack/claude-mem/issues/811
|
||||
* @param port Worker port to check
|
||||
* @param timeoutMs Maximum time to wait in milliseconds
|
||||
* @returns true if worker became responsive, false if timeout
|
||||
* Poll a localhost endpoint until it returns 200 OK or timeout.
|
||||
* Shared implementation for liveness and readiness checks.
|
||||
*/
|
||||
export async function waitForHealth(port: number, timeoutMs: number = 30000): Promise<boolean> {
|
||||
async function pollEndpointUntilOk(
|
||||
port: number,
|
||||
endpointPath: string,
|
||||
timeoutMs: number,
|
||||
retryLogMessage: string
|
||||
): Promise<boolean> {
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
try {
|
||||
// Note: Removed AbortSignal.timeout to avoid Windows Bun cleanup issue (libuv assertion)
|
||||
const response = await fetch(`http://127.0.0.1:${port}/api/health`);
|
||||
const response = await fetch(`http://127.0.0.1:${port}${endpointPath}`);
|
||||
if (response.ok) return true;
|
||||
} catch (error) {
|
||||
// [ANTI-PATTERN IGNORED]: Retry loop - expected failures during startup, will retry
|
||||
logger.debug('SYSTEM', 'Service not ready yet, will retry', { port }, error as Error);
|
||||
logger.debug('SYSTEM', retryLogMessage, { port }, error as Error);
|
||||
}
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait for the worker HTTP server to become responsive (liveness check).
|
||||
* Uses /api/health which returns 200 as soon as the HTTP server is listening.
|
||||
* For full initialization (DB + search), use waitForReadiness() instead.
|
||||
*/
|
||||
export function waitForHealth(port: number, timeoutMs: number = 30000): Promise<boolean> {
|
||||
return pollEndpointUntilOk(port, '/api/health', timeoutMs, 'Service not ready yet, will retry');
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait for the worker to be fully initialized (DB + search ready).
|
||||
* Uses /api/readiness which returns 200 only after core initialization completes.
|
||||
* Now that initializationCompleteFlag is set after DB/search init (not MCP),
|
||||
* this typically completes in a few seconds.
|
||||
*/
|
||||
export function waitForReadiness(port: number, timeoutMs: number = 30000): Promise<boolean> {
|
||||
return pollEndpointUntilOk(port, '/api/readiness', timeoutMs, 'Worker not ready yet, will retry');
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait for a port to become free (no longer responding to health checks)
|
||||
* Used after shutdown to confirm the port is available for restart
|
||||
@@ -97,12 +115,22 @@ export async function httpShutdown(port: number): Promise<boolean> {
|
||||
|
||||
/**
|
||||
* Get the plugin version from the installed marketplace package.json
|
||||
* This is the "expected" version that should be running
|
||||
* This is the "expected" version that should be running.
|
||||
* Returns 'unknown' on ENOENT/EBUSY (shutdown race condition, fix #1042).
|
||||
*/
|
||||
export function getInstalledPluginVersion(): string {
|
||||
const packageJsonPath = path.join(MARKETPLACE_ROOT, 'package.json');
|
||||
const packageJson = JSON.parse(readFileSync(packageJsonPath, 'utf-8'));
|
||||
return packageJson.version;
|
||||
try {
|
||||
const packageJsonPath = path.join(MARKETPLACE_ROOT, 'package.json');
|
||||
const packageJson = JSON.parse(readFileSync(packageJsonPath, 'utf-8'));
|
||||
return packageJson.version;
|
||||
} catch (error: unknown) {
|
||||
const code = (error as NodeJS.ErrnoException).code;
|
||||
if (code === 'ENOENT' || code === 'EBUSY') {
|
||||
logger.debug('SYSTEM', 'Could not read plugin version (shutdown race)', { code });
|
||||
return 'unknown';
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -137,8 +165,8 @@ export async function checkVersionMatch(port: number): Promise<VersionCheckResul
|
||||
const pluginVersion = getInstalledPluginVersion();
|
||||
const workerVersion = await getRunningWorkerVersion(port);
|
||||
|
||||
// If we can't get worker version, assume it matches (graceful degradation)
|
||||
if (!workerVersion) {
|
||||
// If either version is unknown/null, assume match (graceful degradation, fix #1042)
|
||||
if (!workerVersion || pluginVersion === 'unknown') {
|
||||
return { matches: true, pluginVersion, workerVersion };
|
||||
}
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
|
||||
import path from 'path';
|
||||
import { homedir } from 'os';
|
||||
import { existsSync, writeFileSync, readFileSync, unlinkSync, mkdirSync } from 'fs';
|
||||
import { existsSync, writeFileSync, readFileSync, unlinkSync, mkdirSync, rmSync, statSync, utimesSync } from 'fs';
|
||||
import { exec, execSync, spawn } from 'child_process';
|
||||
import { promisify } from 'util';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
@@ -54,7 +54,8 @@ function lookupBinaryInPath(binaryName: string, platform: NodeJS.Platform): stri
|
||||
try {
|
||||
const output = execSync(command, {
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
encoding: 'utf-8'
|
||||
encoding: 'utf-8',
|
||||
windowsHide: true
|
||||
});
|
||||
|
||||
const firstMatch = output
|
||||
@@ -191,10 +192,10 @@ export async function getChildProcesses(parentPid: number): Promise<number[]> {
|
||||
}
|
||||
|
||||
try {
|
||||
// PowerShell Get-Process instead of WMIC (deprecated in Windows 11)
|
||||
const cmd = `powershell -NoProfile -NonInteractive -Command "Get-Process | Where-Object { $_.ParentProcessId -eq ${parentPid} } | Select-Object -ExpandProperty Id"`;
|
||||
const { stdout } = await execAsync(cmd, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND });
|
||||
// PowerShell outputs just numbers (one per line), simpler than WMIC's "ProcessId=1234" format
|
||||
// Use WQL -Filter to avoid $_ pipeline syntax that breaks in Git Bash (#1062, #1024).
|
||||
// Get-CimInstance with server-side filtering is also more efficient than piping through Where-Object.
|
||||
const cmd = `powershell -NoProfile -NonInteractive -Command "Get-CimInstance Win32_Process -Filter 'ParentProcessId=${parentPid}' | Select-Object -ExpandProperty ProcessId"`;
|
||||
const { stdout } = await execAsync(cmd, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, windowsHide: true });
|
||||
return stdout
|
||||
.split('\n')
|
||||
.map(line => line.trim())
|
||||
@@ -223,7 +224,7 @@ export async function forceKillProcess(pid: number): Promise<void> {
|
||||
try {
|
||||
if (process.platform === 'win32') {
|
||||
// /T kills entire process tree, /F forces termination
|
||||
await execAsync(`taskkill /PID ${pid} /T /F`, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND });
|
||||
await execAsync(`taskkill /PID ${pid} /T /F`, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, windowsHide: true });
|
||||
} else {
|
||||
process.kill(pid, 'SIGKILL');
|
||||
}
|
||||
@@ -315,13 +316,14 @@ export async function cleanupOrphanedProcesses(): Promise<void> {
|
||||
|
||||
try {
|
||||
if (isWindows) {
|
||||
// Windows: Use PowerShell Get-CimInstance with JSON output for age filtering
|
||||
const patternConditions = ORPHAN_PROCESS_PATTERNS
|
||||
.map(p => `$_.CommandLine -like '*${p}*'`)
|
||||
.join(' -or ');
|
||||
// Windows: Use WQL -Filter for server-side filtering (no $_ pipeline syntax).
|
||||
// Avoids Git Bash $_ interpretation (#1062) and PowerShell syntax errors (#1024).
|
||||
const wqlPatternConditions = ORPHAN_PROCESS_PATTERNS
|
||||
.map(p => `CommandLine LIKE '%${p}%'`)
|
||||
.join(' OR ');
|
||||
|
||||
const cmd = `powershell -NoProfile -NonInteractive -Command "Get-CimInstance Win32_Process | Where-Object { (${patternConditions}) -and $_.ProcessId -ne ${currentPid} } | Select-Object ProcessId, CreationDate | ConvertTo-Json"`;
|
||||
const { stdout } = await execAsync(cmd, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND });
|
||||
const cmd = `powershell -NoProfile -NonInteractive -Command "Get-CimInstance Win32_Process -Filter '(${wqlPatternConditions}) AND ProcessId != ${currentPid}' | Select-Object ProcessId, CreationDate | ConvertTo-Json"`;
|
||||
const { stdout } = await execAsync(cmd, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, windowsHide: true });
|
||||
|
||||
if (!stdout.trim() || stdout.trim() === 'null') {
|
||||
logger.debug('SYSTEM', 'No orphaned claude-mem processes found (Windows)');
|
||||
@@ -406,7 +408,7 @@ export async function cleanupOrphanedProcesses(): Promise<void> {
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
execSync(`taskkill /PID ${pid} /T /F`, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, stdio: 'ignore' });
|
||||
execSync(`taskkill /PID ${pid} /T /F`, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, stdio: 'ignore', windowsHide: true });
|
||||
} catch (error) {
|
||||
// [ANTI-PATTERN IGNORED]: Cleanup loop - process may have exited, continue to next PID
|
||||
logger.debug('SYSTEM', 'Failed to kill process, may have already exited', { pid }, error as Error);
|
||||
@@ -426,6 +428,184 @@ export async function cleanupOrphanedProcesses(): Promise<void> {
|
||||
logger.info('SYSTEM', 'Orphaned processes cleaned up', { count: pidsToKill.length });
|
||||
}
|
||||
|
||||
// Patterns that should be killed immediately at startup (no age gate)
|
||||
// These are child processes that should not outlive their parent worker
|
||||
const AGGRESSIVE_CLEANUP_PATTERNS = ['worker-service.cjs', 'chroma-mcp'];
|
||||
|
||||
// Patterns that keep the age-gated threshold (may be legitimately running)
|
||||
const AGE_GATED_CLEANUP_PATTERNS = ['mcp-server.cjs'];
|
||||
|
||||
/**
|
||||
* Aggressive startup cleanup for orphaned claude-mem processes.
|
||||
*
|
||||
* Unlike cleanupOrphanedProcesses() which age-gates everything at 30 minutes,
|
||||
* this function kills worker-service.cjs and chroma-mcp processes immediately
|
||||
* (they should not outlive their parent worker). Only mcp-server.cjs keeps
|
||||
* the age threshold since it may be legitimately running.
|
||||
*
|
||||
* Called once at daemon startup.
|
||||
*/
|
||||
export async function aggressiveStartupCleanup(): Promise<void> {
|
||||
const isWindows = process.platform === 'win32';
|
||||
const currentPid = process.pid;
|
||||
const pidsToKill: number[] = [];
|
||||
const allPatterns = [...AGGRESSIVE_CLEANUP_PATTERNS, ...AGE_GATED_CLEANUP_PATTERNS];
|
||||
|
||||
try {
|
||||
if (isWindows) {
|
||||
// Use WQL -Filter for server-side filtering (no $_ pipeline syntax).
|
||||
// Avoids Git Bash $_ interpretation (#1062) and PowerShell syntax errors (#1024).
|
||||
const wqlPatternConditions = allPatterns
|
||||
.map(p => `CommandLine LIKE '%${p}%'`)
|
||||
.join(' OR ');
|
||||
|
||||
const cmd = `powershell -NoProfile -NonInteractive -Command "Get-CimInstance Win32_Process -Filter '(${wqlPatternConditions}) AND ProcessId != ${currentPid}' | Select-Object ProcessId, CommandLine, CreationDate | ConvertTo-Json"`;
|
||||
const { stdout } = await execAsync(cmd, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, windowsHide: true });
|
||||
|
||||
if (!stdout.trim() || stdout.trim() === 'null') {
|
||||
logger.debug('SYSTEM', 'No orphaned claude-mem processes found (Windows)');
|
||||
return;
|
||||
}
|
||||
|
||||
const processes = JSON.parse(stdout);
|
||||
const processList = Array.isArray(processes) ? processes : [processes];
|
||||
const now = Date.now();
|
||||
|
||||
for (const proc of processList) {
|
||||
const pid = proc.ProcessId;
|
||||
if (!Number.isInteger(pid) || pid <= 0 || pid === currentPid) continue;
|
||||
|
||||
const commandLine = proc.CommandLine || '';
|
||||
const isAggressive = AGGRESSIVE_CLEANUP_PATTERNS.some(p => commandLine.includes(p));
|
||||
|
||||
if (isAggressive) {
|
||||
// Kill immediately — no age check
|
||||
pidsToKill.push(pid);
|
||||
logger.debug('SYSTEM', 'Found orphaned process (aggressive)', { pid, commandLine: commandLine.substring(0, 80) });
|
||||
} else {
|
||||
// Age-gated: only kill if older than threshold
|
||||
const creationMatch = proc.CreationDate?.match(/\/Date\((\d+)\)\//);
|
||||
if (creationMatch) {
|
||||
const creationTime = parseInt(creationMatch[1], 10);
|
||||
const ageMinutes = (now - creationTime) / (1000 * 60);
|
||||
if (ageMinutes >= ORPHAN_MAX_AGE_MINUTES) {
|
||||
pidsToKill.push(pid);
|
||||
logger.debug('SYSTEM', 'Found orphaned process (age-gated)', { pid, ageMinutes: Math.round(ageMinutes) });
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// Unix: Use ps with elapsed time
|
||||
const patternRegex = allPatterns.join('|');
|
||||
const { stdout } = await execAsync(
|
||||
`ps -eo pid,etime,command | grep -E "${patternRegex}" | grep -v grep || true`
|
||||
);
|
||||
|
||||
if (!stdout.trim()) {
|
||||
logger.debug('SYSTEM', 'No orphaned claude-mem processes found (Unix)');
|
||||
return;
|
||||
}
|
||||
|
||||
const lines = stdout.trim().split('\n');
|
||||
for (const line of lines) {
|
||||
const match = line.trim().match(/^(\d+)\s+(\S+)\s+(.*)$/);
|
||||
if (!match) continue;
|
||||
|
||||
const pid = parseInt(match[1], 10);
|
||||
const etime = match[2];
|
||||
const command = match[3];
|
||||
|
||||
if (!Number.isInteger(pid) || pid <= 0 || pid === currentPid) continue;
|
||||
|
||||
const isAggressive = AGGRESSIVE_CLEANUP_PATTERNS.some(p => command.includes(p));
|
||||
|
||||
if (isAggressive) {
|
||||
// Kill immediately — no age check
|
||||
pidsToKill.push(pid);
|
||||
logger.debug('SYSTEM', 'Found orphaned process (aggressive)', { pid, command: command.substring(0, 80) });
|
||||
} else {
|
||||
// Age-gated: only kill if older than threshold
|
||||
const ageMinutes = parseElapsedTime(etime);
|
||||
if (ageMinutes >= ORPHAN_MAX_AGE_MINUTES) {
|
||||
pidsToKill.push(pid);
|
||||
logger.debug('SYSTEM', 'Found orphaned process (age-gated)', { pid, ageMinutes, command: command.substring(0, 80) });
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error('SYSTEM', 'Failed to enumerate orphaned processes during aggressive cleanup', {}, error as Error);
|
||||
return;
|
||||
}
|
||||
|
||||
if (pidsToKill.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
logger.info('SYSTEM', 'Aggressive startup cleanup: killing orphaned processes', {
|
||||
platform: isWindows ? 'Windows' : 'Unix',
|
||||
count: pidsToKill.length,
|
||||
pids: pidsToKill
|
||||
});
|
||||
|
||||
if (isWindows) {
|
||||
for (const pid of pidsToKill) {
|
||||
if (!Number.isInteger(pid) || pid <= 0) continue;
|
||||
try {
|
||||
execSync(`taskkill /PID ${pid} /T /F`, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, stdio: 'ignore', windowsHide: true });
|
||||
} catch (error) {
|
||||
logger.debug('SYSTEM', 'Failed to kill process, may have already exited', { pid }, error as Error);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
for (const pid of pidsToKill) {
|
||||
try {
|
||||
process.kill(pid, 'SIGKILL');
|
||||
} catch (error) {
|
||||
logger.debug('SYSTEM', 'Process already exited', { pid }, error as Error);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
logger.info('SYSTEM', 'Aggressive startup cleanup complete', { count: pidsToKill.length });
|
||||
}
|
||||
|
||||
const CHROMA_MIGRATION_MARKER_FILENAME = '.chroma-cleaned-v10.3';
|
||||
|
||||
/**
|
||||
* One-time chroma data wipe for users upgrading from versions with duplicate
|
||||
* worker bugs that could corrupt chroma data. Since chroma is always rebuildable
|
||||
* from SQLite (via backfillAllProjects), this is safe.
|
||||
*
|
||||
* Checks for a marker file. If absent, wipes ~/.claude-mem/chroma/ and writes
|
||||
* the marker. If present, skips. Idempotent.
|
||||
*
|
||||
* @param dataDirectory - Override for DATA_DIR (used in tests)
|
||||
*/
|
||||
export function runOneTimeChromaMigration(dataDirectory?: string): void {
|
||||
const effectiveDataDir = dataDirectory ?? DATA_DIR;
|
||||
const markerPath = path.join(effectiveDataDir, CHROMA_MIGRATION_MARKER_FILENAME);
|
||||
const chromaDir = path.join(effectiveDataDir, 'chroma');
|
||||
|
||||
if (existsSync(markerPath)) {
|
||||
logger.debug('SYSTEM', 'Chroma migration marker exists, skipping wipe');
|
||||
return;
|
||||
}
|
||||
|
||||
logger.warn('SYSTEM', 'Running one-time chroma data wipe (upgrade from pre-v10.3)', { chromaDir });
|
||||
|
||||
if (existsSync(chromaDir)) {
|
||||
rmSync(chromaDir, { recursive: true, force: true });
|
||||
logger.info('SYSTEM', 'Chroma data directory removed', { chromaDir });
|
||||
}
|
||||
|
||||
// Write marker file to prevent future wipes
|
||||
mkdirSync(effectiveDataDir, { recursive: true });
|
||||
writeFileSync(markerPath, new Date().toISOString());
|
||||
logger.info('SYSTEM', 'Chroma migration marker written', { markerPath });
|
||||
}
|
||||
|
||||
/**
|
||||
* Spawn a detached daemon process
|
||||
* Returns the child PID or undefined if spawn failed
|
||||
@@ -523,10 +703,10 @@ export function spawnDaemon(
|
||||
*
|
||||
* EPERM is treated as "alive" because it means the process exists but
|
||||
* belongs to a different user/session (common in multi-user setups).
|
||||
* PID 0 (Windows WMIC sentinel for unknown PID) is treated as alive.
|
||||
* PID 0 (Windows sentinel for unknown PID) is treated as alive.
|
||||
*/
|
||||
export function isProcessAlive(pid: number): boolean {
|
||||
// PID 0 is the Windows WMIC sentinel value — process was spawned but PID unknown
|
||||
// PID 0 is the Windows sentinel value — process was spawned but PID unknown
|
||||
if (pid === 0) return true;
|
||||
|
||||
// Invalid PIDs are not alive
|
||||
@@ -544,6 +724,39 @@ export function isProcessAlive(pid: number): boolean {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if the PID file was written recently (within thresholdMs).
|
||||
*
|
||||
* Used to coordinate restarts across concurrent sessions: if the PID file
|
||||
* was recently written, another session likely just restarted the worker.
|
||||
* Callers should poll /api/health instead of attempting their own restart.
|
||||
*
|
||||
* @param thresholdMs - Maximum age in ms to consider "recent" (default: 15000)
|
||||
* @returns true if the PID file exists and was modified within thresholdMs
|
||||
*/
|
||||
export function isPidFileRecent(thresholdMs: number = 15000): boolean {
|
||||
try {
|
||||
const stats = statSync(PID_FILE);
|
||||
return (Date.now() - stats.mtimeMs) < thresholdMs;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Touch the PID file to update its mtime without changing contents.
|
||||
* Used after a restart to signal other sessions that a restart just completed.
|
||||
*/
|
||||
export function touchPidFile(): void {
|
||||
try {
|
||||
if (!existsSync(PID_FILE)) return;
|
||||
const now = new Date();
|
||||
utimesSync(PID_FILE, now, now);
|
||||
} catch {
|
||||
// Best-effort — failure to touch doesn't affect correctness
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the PID file and remove it if the recorded process is dead (stale).
|
||||
*
|
||||
|
||||
@@ -20,8 +20,9 @@ export class SessionQueueProcessor {
|
||||
|
||||
/**
|
||||
* Create an async iterator that yields messages as they become available.
|
||||
* Uses atomic claim-and-delete to prevent duplicates.
|
||||
* The queue is a pure buffer: claim it, delete it, process in memory.
|
||||
* Uses atomic claim-confirm to prevent duplicates.
|
||||
* Messages are claimed (marked processing) and stay in DB until confirmProcessed().
|
||||
* Self-heals stale processing messages before each claim.
|
||||
* Waits for 'message' event when queue is empty.
|
||||
*
|
||||
* CRITICAL: Calls onIdleTimeout callback after 3 minutes of inactivity.
|
||||
@@ -34,14 +35,14 @@ export class SessionQueueProcessor {
|
||||
|
||||
while (!signal.aborted) {
|
||||
try {
|
||||
// Atomically claim AND DELETE next message from DB
|
||||
// Message is now in memory only - no "processing" state tracking needed
|
||||
const persistentMessage = this.store.claimAndDelete(sessionDbId);
|
||||
// Atomically claim next pending message (marks as 'processing')
|
||||
// Self-heals any stale processing messages before claiming
|
||||
const persistentMessage = this.store.claimNextMessage(sessionDbId);
|
||||
|
||||
if (persistentMessage) {
|
||||
// Reset activity time when we successfully yield a message
|
||||
lastActivityTime = Date.now();
|
||||
// Yield the message for processing (it's already deleted from queue)
|
||||
// Yield the message for processing (it's marked as 'processing' in DB)
|
||||
yield this.toPendingMessageWithId(persistentMessage);
|
||||
} else {
|
||||
// Queue empty - wait for wake-up event or timeout
|
||||
|
||||
@@ -248,8 +248,14 @@ export class Server {
|
||||
process.send!({ type: 'restart' });
|
||||
} else {
|
||||
// Unix or standalone Windows - handle restart ourselves
|
||||
// The spawner (ensureWorkerStarted/restart command) handles spawning the new daemon.
|
||||
// This process just needs to shut down and exit.
|
||||
setTimeout(async () => {
|
||||
await this.options.onRestart();
|
||||
try {
|
||||
await this.options.onRestart();
|
||||
} finally {
|
||||
process.exit(0);
|
||||
}
|
||||
}, 100);
|
||||
}
|
||||
});
|
||||
@@ -268,7 +274,14 @@ export class Server {
|
||||
} else {
|
||||
// Unix or standalone Windows - handle shutdown ourselves
|
||||
setTimeout(async () => {
|
||||
await this.options.onShutdown();
|
||||
try {
|
||||
await this.options.onShutdown();
|
||||
} finally {
|
||||
// CRITICAL: Exit the process after shutdown completes (or fails).
|
||||
// Without this, the daemon stays alive as a zombie — background tasks
|
||||
// (backfill, reconnects) keep running and respawn chroma-mcp subprocesses.
|
||||
process.exit(0);
|
||||
}
|
||||
}, 100);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -2,6 +2,9 @@ import { Database } from './sqlite-compat.js';
|
||||
import type { PendingMessage } from '../worker-types.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
|
||||
/** Messages processing longer than this are considered stale and reset to pending by self-healing */
|
||||
const STALE_PROCESSING_THRESHOLD_MS = 60_000;
|
||||
|
||||
/**
|
||||
* Persistent pending message record from database
|
||||
*/
|
||||
@@ -26,12 +29,17 @@ export interface PersistentPendingMessage {
|
||||
/**
|
||||
* PendingMessageStore - Persistent work queue for SDK messages
|
||||
*
|
||||
* Messages are persisted before processing using a claim-and-delete pattern.
|
||||
* Messages are persisted before processing using a claim-confirm pattern.
|
||||
* This simplifies the lifecycle and eliminates duplicate processing bugs.
|
||||
*
|
||||
* Lifecycle:
|
||||
* 1. enqueue() - Message persisted with status 'pending'
|
||||
* 2. claimAndDelete() - Atomically claims and deletes message (process in memory)
|
||||
* 2. claimNextMessage() - Atomically claims next pending message (marks as 'processing')
|
||||
* 3. confirmProcessed() - Deletes message after successful processing
|
||||
*
|
||||
* Self-healing:
|
||||
* - claimNextMessage() resets stale 'processing' messages (>60s) back to 'pending' before claiming
|
||||
* - This eliminates stuck messages from generator crashes without external timers
|
||||
*
|
||||
* Recovery:
|
||||
* - getSessionsWithPendingMessages() - Find sessions that need recovery on startup
|
||||
@@ -78,13 +86,29 @@ export class PendingMessageStore {
|
||||
|
||||
/**
|
||||
* Atomically claim the next pending message by marking it as 'processing'.
|
||||
* CRITICAL FIX: Does NOT delete - message stays in DB until confirmProcessed() is called.
|
||||
* This prevents message loss if the generator crashes mid-processing.
|
||||
* Self-healing: resets any stale 'processing' messages (>60s) back to 'pending' first.
|
||||
* Message stays in DB until confirmProcessed() is called.
|
||||
* Uses a transaction to prevent race conditions.
|
||||
*/
|
||||
claimAndDelete(sessionDbId: number): PersistentPendingMessage | null {
|
||||
const now = Date.now();
|
||||
claimNextMessage(sessionDbId: number): PersistentPendingMessage | null {
|
||||
const claimTx = this.db.transaction((sessionId: number) => {
|
||||
// Capture time inside transaction so it's fresh if WAL contention causes retry
|
||||
const now = Date.now();
|
||||
// Self-healing: reset stale 'processing' messages back to 'pending'
|
||||
// This recovers from generator crashes without external timers
|
||||
// Note: strict < means messages must be OLDER than threshold to be reset
|
||||
const staleCutoff = now - STALE_PROCESSING_THRESHOLD_MS;
|
||||
const resetStmt = this.db.prepare(`
|
||||
UPDATE pending_messages
|
||||
SET status = 'pending', started_processing_at_epoch = NULL
|
||||
WHERE session_db_id = ? AND status = 'processing'
|
||||
AND started_processing_at_epoch < ?
|
||||
`);
|
||||
const resetResult = resetStmt.run(sessionId, staleCutoff);
|
||||
if (resetResult.changes > 0) {
|
||||
logger.info('QUEUE', `SELF_HEAL | sessionDbId=${sessionId} | recovered ${resetResult.changes} stale processing message(s)`);
|
||||
}
|
||||
|
||||
const peekStmt = this.db.prepare(`
|
||||
SELECT * FROM pending_messages
|
||||
WHERE session_db_id = ? AND status = 'pending'
|
||||
@@ -374,9 +398,22 @@ export class PendingMessageStore {
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if any session has pending work
|
||||
* Check if any session has pending work.
|
||||
* Excludes 'processing' messages stuck for >5 minutes (resets them to 'pending' as a side effect).
|
||||
*/
|
||||
hasAnyPendingWork(): boolean {
|
||||
// Reset stuck 'processing' messages older than 5 minutes before checking
|
||||
const stuckCutoff = Date.now() - (5 * 60 * 1000);
|
||||
const resetStmt = this.db.prepare(`
|
||||
UPDATE pending_messages
|
||||
SET status = 'pending', started_processing_at_epoch = NULL
|
||||
WHERE status = 'processing' AND started_processing_at_epoch < ?
|
||||
`);
|
||||
const resetResult = resetStmt.run(stuckCutoff);
|
||||
if (resetResult.changes > 0) {
|
||||
logger.info('QUEUE', `STUCK_RESET | hasAnyPendingWork reset ${resetResult.changes} stuck processing message(s) older than 5 minutes`);
|
||||
}
|
||||
|
||||
const stmt = this.db.prepare(`
|
||||
SELECT COUNT(*) as count FROM pending_messages
|
||||
WHERE status IN ('pending', 'processing')
|
||||
|
||||
@@ -46,6 +46,10 @@ export class SessionSearch {
|
||||
* - Tables maintained but search paths removed
|
||||
* - Triggers still fire to keep tables synchronized
|
||||
*
|
||||
* FTS5 may be unavailable on some platforms (e.g., Bun on Windows #791).
|
||||
* When unavailable, we skip FTS table creation — search falls back to
|
||||
* ChromaDB (vector) and LIKE queries (structured filters) which are unaffected.
|
||||
*
|
||||
* TODO: Remove FTS5 infrastructure in future major version (v7.0.0)
|
||||
*/
|
||||
private ensureFTSTables(): void {
|
||||
@@ -58,91 +62,117 @@ export class SessionSearch {
|
||||
return;
|
||||
}
|
||||
|
||||
// Runtime check: verify FTS5 is available before attempting to create tables.
|
||||
// bun:sqlite on Windows may not include the FTS5 extension (#791).
|
||||
if (!this.isFts5Available()) {
|
||||
logger.warn('DB', 'FTS5 not available on this platform — skipping FTS table creation (search uses ChromaDB)');
|
||||
return;
|
||||
}
|
||||
|
||||
logger.info('DB', 'Creating FTS5 tables');
|
||||
|
||||
// Create observations_fts virtual table
|
||||
this.db.run(`
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS observations_fts USING fts5(
|
||||
title,
|
||||
subtitle,
|
||||
narrative,
|
||||
text,
|
||||
facts,
|
||||
concepts,
|
||||
content='observations',
|
||||
content_rowid='id'
|
||||
);
|
||||
`);
|
||||
try {
|
||||
// Create observations_fts virtual table
|
||||
this.db.run(`
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS observations_fts USING fts5(
|
||||
title,
|
||||
subtitle,
|
||||
narrative,
|
||||
text,
|
||||
facts,
|
||||
concepts,
|
||||
content='observations',
|
||||
content_rowid='id'
|
||||
);
|
||||
`);
|
||||
|
||||
// Populate with existing data
|
||||
this.db.run(`
|
||||
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
SELECT id, title, subtitle, narrative, text, facts, concepts
|
||||
FROM observations;
|
||||
`);
|
||||
|
||||
// Create triggers for observations
|
||||
this.db.run(`
|
||||
CREATE TRIGGER IF NOT EXISTS observations_ai AFTER INSERT ON observations BEGIN
|
||||
// Populate with existing data
|
||||
this.db.run(`
|
||||
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
|
||||
END;
|
||||
SELECT id, title, subtitle, narrative, text, facts, concepts
|
||||
FROM observations;
|
||||
`);
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS observations_ad AFTER DELETE ON observations BEGIN
|
||||
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
|
||||
END;
|
||||
// Create triggers for observations
|
||||
this.db.run(`
|
||||
CREATE TRIGGER IF NOT EXISTS observations_ai AFTER INSERT ON observations BEGIN
|
||||
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS observations_au AFTER UPDATE ON observations BEGIN
|
||||
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
|
||||
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
|
||||
END;
|
||||
`);
|
||||
CREATE TRIGGER IF NOT EXISTS observations_ad AFTER DELETE ON observations BEGIN
|
||||
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
|
||||
END;
|
||||
|
||||
// Create session_summaries_fts virtual table
|
||||
this.db.run(`
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS session_summaries_fts USING fts5(
|
||||
request,
|
||||
investigated,
|
||||
learned,
|
||||
completed,
|
||||
next_steps,
|
||||
notes,
|
||||
content='session_summaries',
|
||||
content_rowid='id'
|
||||
);
|
||||
`);
|
||||
CREATE TRIGGER IF NOT EXISTS observations_au AFTER UPDATE ON observations BEGIN
|
||||
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
|
||||
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
|
||||
END;
|
||||
`);
|
||||
|
||||
// Populate with existing data
|
||||
this.db.run(`
|
||||
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
|
||||
SELECT id, request, investigated, learned, completed, next_steps, notes
|
||||
FROM session_summaries;
|
||||
`);
|
||||
// Create session_summaries_fts virtual table
|
||||
this.db.run(`
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS session_summaries_fts USING fts5(
|
||||
request,
|
||||
investigated,
|
||||
learned,
|
||||
completed,
|
||||
next_steps,
|
||||
notes,
|
||||
content='session_summaries',
|
||||
content_rowid='id'
|
||||
);
|
||||
`);
|
||||
|
||||
// Create triggers for session_summaries
|
||||
this.db.run(`
|
||||
CREATE TRIGGER IF NOT EXISTS session_summaries_ai AFTER INSERT ON session_summaries BEGIN
|
||||
// Populate with existing data
|
||||
this.db.run(`
|
||||
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
|
||||
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
|
||||
END;
|
||||
SELECT id, request, investigated, learned, completed, next_steps, notes
|
||||
FROM session_summaries;
|
||||
`);
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS session_summaries_ad AFTER DELETE ON session_summaries BEGIN
|
||||
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
|
||||
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
|
||||
END;
|
||||
// Create triggers for session_summaries
|
||||
this.db.run(`
|
||||
CREATE TRIGGER IF NOT EXISTS session_summaries_ai AFTER INSERT ON session_summaries BEGIN
|
||||
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
|
||||
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS session_summaries_au AFTER UPDATE ON session_summaries BEGIN
|
||||
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
|
||||
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
|
||||
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
|
||||
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
|
||||
END;
|
||||
`);
|
||||
CREATE TRIGGER IF NOT EXISTS session_summaries_ad AFTER DELETE ON session_summaries BEGIN
|
||||
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
|
||||
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
|
||||
END;
|
||||
|
||||
logger.info('DB', 'FTS5 tables created successfully');
|
||||
CREATE TRIGGER IF NOT EXISTS session_summaries_au AFTER UPDATE ON session_summaries BEGIN
|
||||
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
|
||||
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
|
||||
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
|
||||
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
|
||||
END;
|
||||
`);
|
||||
|
||||
logger.info('DB', 'FTS5 tables created successfully');
|
||||
} catch (error) {
|
||||
// FTS5 creation failed at runtime despite probe succeeding — degrade gracefully
|
||||
logger.warn('DB', 'FTS5 table creation failed — search will use ChromaDB and LIKE queries', {}, error as Error);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Probe whether the FTS5 extension is available in the current SQLite build.
|
||||
* Creates and immediately drops a temporary FTS5 table.
|
||||
*/
|
||||
private isFts5Available(): boolean {
|
||||
try {
|
||||
this.db.run('CREATE VIRTUAL TABLE _fts5_probe USING fts5(test_column)');
|
||||
this.db.run('DROP TABLE _fts5_probe');
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
+183
-117
@@ -13,6 +13,7 @@ import {
|
||||
LatestPromptResult
|
||||
} from '../../types/database.js';
|
||||
import type { PendingMessageStore } from './PendingMessageStore.js';
|
||||
import { computeObservationContentHash, findDuplicateObservation } from './observations/store.js';
|
||||
|
||||
/**
|
||||
* Session data store for SDK sessions, observations, and summaries
|
||||
@@ -48,11 +49,17 @@ export class SessionStore {
|
||||
this.repairSessionIdColumnRename();
|
||||
this.addFailedAtEpochColumn();
|
||||
this.addOnUpdateCascadeToForeignKeys();
|
||||
this.addObservationContentHashColumn();
|
||||
this.addSessionCustomTitleColumn();
|
||||
}
|
||||
|
||||
/**
|
||||
* Initialize database schema using migrations (migration004)
|
||||
* This runs the core SDK tables migration if no tables exist
|
||||
* Initialize database schema (migration004)
|
||||
*
|
||||
* ALWAYS creates core tables using CREATE TABLE IF NOT EXISTS — safe to run
|
||||
* regardless of schema_versions state. This fixes issue #979 where the old
|
||||
* DatabaseManager migration system (versions 1-7) shared the schema_versions
|
||||
* table, causing maxApplied > 0 and skipping core table creation entirely.
|
||||
*/
|
||||
private initializeSchema(): void {
|
||||
// Create schema_versions table if it doesn't exist
|
||||
@@ -64,90 +71,77 @@ export class SessionStore {
|
||||
)
|
||||
`);
|
||||
|
||||
// Get applied migrations
|
||||
const appliedVersions = this.db.prepare('SELECT version FROM schema_versions ORDER BY version').all() as SchemaVersion[];
|
||||
const maxApplied = appliedVersions.length > 0 ? Math.max(...appliedVersions.map(v => v.version)) : 0;
|
||||
// Always create core tables — IF NOT EXISTS makes this idempotent
|
||||
this.db.run(`
|
||||
CREATE TABLE IF NOT EXISTS sdk_sessions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
content_session_id TEXT UNIQUE NOT NULL,
|
||||
memory_session_id TEXT UNIQUE,
|
||||
project TEXT NOT NULL,
|
||||
user_prompt TEXT,
|
||||
started_at TEXT NOT NULL,
|
||||
started_at_epoch INTEGER NOT NULL,
|
||||
completed_at TEXT,
|
||||
completed_at_epoch INTEGER,
|
||||
status TEXT CHECK(status IN ('active', 'completed', 'failed')) NOT NULL DEFAULT 'active'
|
||||
);
|
||||
|
||||
// Only run migration004 if no migrations have been applied
|
||||
// This creates the sdk_sessions, observations, and session_summaries tables
|
||||
if (maxApplied === 0) {
|
||||
logger.info('DB', 'Initializing fresh database with migration004');
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_claude_id ON sdk_sessions(content_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_sdk_id ON sdk_sessions(memory_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_project ON sdk_sessions(project);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_status ON sdk_sessions(status);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_started ON sdk_sessions(started_at_epoch DESC);
|
||||
|
||||
// Migration004: SDK agent architecture tables
|
||||
this.db.run(`
|
||||
CREATE TABLE IF NOT EXISTS sdk_sessions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
content_session_id TEXT UNIQUE NOT NULL,
|
||||
memory_session_id TEXT UNIQUE,
|
||||
project TEXT NOT NULL,
|
||||
user_prompt TEXT,
|
||||
started_at TEXT NOT NULL,
|
||||
started_at_epoch INTEGER NOT NULL,
|
||||
completed_at TEXT,
|
||||
completed_at_epoch INTEGER,
|
||||
status TEXT CHECK(status IN ('active', 'completed', 'failed')) NOT NULL DEFAULT 'active'
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS observations (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
memory_session_id TEXT NOT NULL,
|
||||
project TEXT NOT NULL,
|
||||
text TEXT NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_claude_id ON sdk_sessions(content_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_sdk_id ON sdk_sessions(memory_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_project ON sdk_sessions(project);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_status ON sdk_sessions(status);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_started ON sdk_sessions(started_at_epoch DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_sdk_session ON observations(memory_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_project ON observations(project);
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_type ON observations(type);
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_created ON observations(created_at_epoch DESC);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS observations (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
memory_session_id TEXT NOT NULL,
|
||||
project TEXT NOT NULL,
|
||||
text TEXT NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS session_summaries (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
memory_session_id TEXT UNIQUE NOT NULL,
|
||||
project TEXT NOT NULL,
|
||||
request TEXT,
|
||||
investigated TEXT,
|
||||
learned TEXT,
|
||||
completed TEXT,
|
||||
next_steps TEXT,
|
||||
files_read TEXT,
|
||||
files_edited TEXT,
|
||||
notes TEXT,
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_sdk_session ON observations(memory_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_project ON observations(project);
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_type ON observations(type);
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_created ON observations(created_at_epoch DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_sdk_session ON session_summaries(memory_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_project ON session_summaries(project);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
|
||||
`);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS session_summaries (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
memory_session_id TEXT UNIQUE NOT NULL,
|
||||
project TEXT NOT NULL,
|
||||
request TEXT,
|
||||
investigated TEXT,
|
||||
learned TEXT,
|
||||
completed TEXT,
|
||||
next_steps TEXT,
|
||||
files_read TEXT,
|
||||
files_edited TEXT,
|
||||
notes TEXT,
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_sdk_session ON session_summaries(memory_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_project ON session_summaries(project);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
|
||||
`);
|
||||
|
||||
// Record migration004 as applied
|
||||
this.db.prepare('INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)').run(4, new Date().toISOString());
|
||||
|
||||
logger.info('DB', 'Migration004 applied successfully');
|
||||
}
|
||||
// Record migration004 as applied (OR IGNORE handles re-runs safely)
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(4, new Date().toISOString());
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure worker_port column exists (migration 5)
|
||||
*
|
||||
* NOTE: Version 5 conflicts with old DatabaseManager migration005 (which drops orphaned tables).
|
||||
* We check actual column state rather than relying solely on version tracking.
|
||||
*/
|
||||
private ensureWorkerPortColumn(): void {
|
||||
// Check if migration already applied
|
||||
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(5) as SchemaVersion | undefined;
|
||||
if (applied) return;
|
||||
|
||||
// Check if column exists
|
||||
// Check actual column existence — don't rely on version tracking alone (issue #979)
|
||||
const tableInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
|
||||
const hasWorkerPort = tableInfo.some(col => col.name === 'worker_port');
|
||||
|
||||
@@ -162,12 +156,12 @@ export class SessionStore {
|
||||
|
||||
/**
|
||||
* Ensure prompt tracking columns exist (migration 6)
|
||||
*
|
||||
* NOTE: Version 6 conflicts with old DatabaseManager migration006 (which creates FTS5 tables).
|
||||
* We check actual column state rather than relying solely on version tracking.
|
||||
*/
|
||||
private ensurePromptTrackingColumns(): void {
|
||||
// Check if migration already applied
|
||||
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(6) as SchemaVersion | undefined;
|
||||
if (applied) return;
|
||||
|
||||
// Check actual column existence — don't rely on version tracking alone (issue #979)
|
||||
// Check sdk_sessions for prompt_counter
|
||||
const sessionsInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
|
||||
const hasPromptCounter = sessionsInfo.some(col => col.name === 'prompt_counter');
|
||||
@@ -201,13 +195,12 @@ export class SessionStore {
|
||||
|
||||
/**
|
||||
* Remove UNIQUE constraint from session_summaries.memory_session_id (migration 7)
|
||||
*
|
||||
* NOTE: Version 7 conflicts with old DatabaseManager migration007 (which adds discovery_tokens).
|
||||
* We check actual constraint state rather than relying solely on version tracking.
|
||||
*/
|
||||
private removeSessionSummariesUniqueConstraint(): void {
|
||||
// Check if migration already applied
|
||||
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(7) as SchemaVersion | undefined;
|
||||
if (applied) return;
|
||||
|
||||
// Check if UNIQUE constraint exists
|
||||
// Check actual constraint state — don't rely on version tracking alone (issue #979)
|
||||
const summariesIndexes = this.db.query('PRAGMA index_list(session_summaries)').all() as IndexInfo[];
|
||||
const hasUniqueConstraint = summariesIndexes.some(idx => idx.unique === 1);
|
||||
|
||||
@@ -222,6 +215,9 @@ export class SessionStore {
|
||||
// Begin transaction
|
||||
this.db.run('BEGIN TRANSACTION');
|
||||
|
||||
// Clean up leftover temp table from a previously-crashed run
|
||||
this.db.run('DROP TABLE IF EXISTS session_summaries_new');
|
||||
|
||||
// Create new table without UNIQUE constraint
|
||||
this.db.run(`
|
||||
CREATE TABLE session_summaries_new (
|
||||
@@ -335,6 +331,9 @@ export class SessionStore {
|
||||
// Begin transaction
|
||||
this.db.run('BEGIN TRANSACTION');
|
||||
|
||||
// Clean up leftover temp table from a previously-crashed run
|
||||
this.db.run('DROP TABLE IF EXISTS observations_new');
|
||||
|
||||
// Create new table with text as nullable
|
||||
this.db.run(`
|
||||
CREATE TABLE observations_new (
|
||||
@@ -428,34 +427,39 @@ export class SessionStore {
|
||||
CREATE INDEX idx_user_prompts_lookup ON user_prompts(content_session_id, prompt_number);
|
||||
`);
|
||||
|
||||
// Create FTS5 virtual table
|
||||
this.db.run(`
|
||||
CREATE VIRTUAL TABLE user_prompts_fts USING fts5(
|
||||
prompt_text,
|
||||
content='user_prompts',
|
||||
content_rowid='id'
|
||||
);
|
||||
`);
|
||||
// Create FTS5 virtual table — skip if FTS5 is unavailable (e.g., Bun on Windows #791).
|
||||
// The user_prompts table itself is still created; only FTS indexing is skipped.
|
||||
try {
|
||||
this.db.run(`
|
||||
CREATE VIRTUAL TABLE user_prompts_fts USING fts5(
|
||||
prompt_text,
|
||||
content='user_prompts',
|
||||
content_rowid='id'
|
||||
);
|
||||
`);
|
||||
|
||||
// Create triggers to sync FTS5
|
||||
this.db.run(`
|
||||
CREATE TRIGGER user_prompts_ai AFTER INSERT ON user_prompts BEGIN
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
// Create triggers to sync FTS5
|
||||
this.db.run(`
|
||||
CREATE TRIGGER user_prompts_ai AFTER INSERT ON user_prompts BEGIN
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER user_prompts_ad AFTER DELETE ON user_prompts BEGIN
|
||||
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
|
||||
VALUES('delete', old.id, old.prompt_text);
|
||||
END;
|
||||
CREATE TRIGGER user_prompts_ad AFTER DELETE ON user_prompts BEGIN
|
||||
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
|
||||
VALUES('delete', old.id, old.prompt_text);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER user_prompts_au AFTER UPDATE ON user_prompts BEGIN
|
||||
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
|
||||
VALUES('delete', old.id, old.prompt_text);
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
`);
|
||||
CREATE TRIGGER user_prompts_au AFTER UPDATE ON user_prompts BEGIN
|
||||
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
|
||||
VALUES('delete', old.id, old.prompt_text);
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
`);
|
||||
} catch (ftsError) {
|
||||
logger.warn('DB', 'FTS5 not available — user_prompts_fts skipped (search uses ChromaDB)', {}, ftsError as Error);
|
||||
}
|
||||
|
||||
// Commit transaction
|
||||
this.db.run('COMMIT');
|
||||
@@ -463,7 +467,7 @@ export class SessionStore {
|
||||
// Record migration
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(10, new Date().toISOString());
|
||||
|
||||
logger.debug('DB', 'Successfully created user_prompts table with FTS5 support');
|
||||
logger.debug('DB', 'Successfully created user_prompts table');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -675,6 +679,9 @@ export class SessionStore {
|
||||
this.db.run('DROP TRIGGER IF EXISTS observations_ad');
|
||||
this.db.run('DROP TRIGGER IF EXISTS observations_au');
|
||||
|
||||
// Clean up leftover temp table from a previously-crashed run
|
||||
this.db.run('DROP TABLE IF EXISTS observations_new');
|
||||
|
||||
this.db.run(`
|
||||
CREATE TABLE observations_new (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
@@ -744,6 +751,9 @@ export class SessionStore {
|
||||
// 2. Recreate session_summaries table
|
||||
// ==========================================
|
||||
|
||||
// Clean up leftover temp table from a previously-crashed run
|
||||
this.db.run('DROP TABLE IF EXISTS session_summaries_new');
|
||||
|
||||
this.db.run(`
|
||||
CREATE TABLE session_summaries_new (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
@@ -825,6 +835,44 @@ export class SessionStore {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Add content_hash column to observations for deduplication (migration 22)
|
||||
*/
|
||||
private addObservationContentHashColumn(): void {
|
||||
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(22) as SchemaVersion | undefined;
|
||||
if (applied) return;
|
||||
|
||||
const tableInfo = this.db.query('PRAGMA table_info(observations)').all() as TableColumnInfo[];
|
||||
const hasColumn = tableInfo.some(col => col.name === 'content_hash');
|
||||
|
||||
if (!hasColumn) {
|
||||
this.db.run('ALTER TABLE observations ADD COLUMN content_hash TEXT');
|
||||
this.db.run("UPDATE observations SET content_hash = substr(hex(randomblob(8)), 1, 16) WHERE content_hash IS NULL");
|
||||
this.db.run('CREATE INDEX IF NOT EXISTS idx_observations_content_hash ON observations(content_hash, created_at_epoch)');
|
||||
logger.debug('DB', 'Added content_hash column to observations table with backfill and index');
|
||||
}
|
||||
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(22, new Date().toISOString());
|
||||
}
|
||||
|
||||
/**
|
||||
* Add custom_title column to sdk_sessions for agent attribution (migration 23)
|
||||
*/
|
||||
private addSessionCustomTitleColumn(): void {
|
||||
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(23) as SchemaVersion | undefined;
|
||||
if (applied) return;
|
||||
|
||||
const tableInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
|
||||
const hasColumn = tableInfo.some(col => col.name === 'custom_title');
|
||||
|
||||
if (!hasColumn) {
|
||||
this.db.run('ALTER TABLE sdk_sessions ADD COLUMN custom_title TEXT');
|
||||
logger.debug('DB', 'Added custom_title column to sdk_sessions table');
|
||||
}
|
||||
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(23, new Date().toISOString());
|
||||
}
|
||||
|
||||
/**
|
||||
* Update the memory session ID for a session
|
||||
* Called by SDKAgent when it captures the session ID from the first SDK message
|
||||
@@ -1290,9 +1338,10 @@ export class SessionStore {
|
||||
memory_session_id: string | null;
|
||||
project: string;
|
||||
user_prompt: string;
|
||||
custom_title: string | null;
|
||||
} | null {
|
||||
const stmt = this.db.prepare(`
|
||||
SELECT id, content_session_id, memory_session_id, project, user_prompt
|
||||
SELECT id, content_session_id, memory_session_id, project, user_prompt, custom_title
|
||||
FROM sdk_sessions
|
||||
WHERE id = ?
|
||||
LIMIT 1
|
||||
@@ -1311,6 +1360,7 @@ export class SessionStore {
|
||||
memory_session_id: string;
|
||||
project: string;
|
||||
user_prompt: string;
|
||||
custom_title: string | null;
|
||||
started_at: string;
|
||||
started_at_epoch: number;
|
||||
completed_at: string | null;
|
||||
@@ -1321,7 +1371,7 @@ export class SessionStore {
|
||||
|
||||
const placeholders = memorySessionIds.map(() => '?').join(',');
|
||||
const stmt = this.db.prepare(`
|
||||
SELECT id, content_session_id, memory_session_id, project, user_prompt,
|
||||
SELECT id, content_session_id, memory_session_id, project, user_prompt, custom_title,
|
||||
started_at, started_at_epoch, completed_at, completed_at_epoch, status
|
||||
FROM sdk_sessions
|
||||
WHERE memory_session_id IN (${placeholders})
|
||||
@@ -1366,7 +1416,7 @@ export class SessionStore {
|
||||
* Pure get-or-create: never modifies memory_session_id.
|
||||
* Multi-terminal isolation is handled by ON UPDATE CASCADE at the schema level.
|
||||
*/
|
||||
createSDKSession(contentSessionId: string, project: string, userPrompt: string): number {
|
||||
createSDKSession(contentSessionId: string, project: string, userPrompt: string, customTitle?: string): number {
|
||||
const now = new Date();
|
||||
const nowEpoch = now.getTime();
|
||||
|
||||
@@ -1383,6 +1433,13 @@ export class SessionStore {
|
||||
WHERE content_session_id = ? AND (project IS NULL OR project = '')
|
||||
`).run(project, contentSessionId);
|
||||
}
|
||||
// Backfill custom_title if provided and not yet set
|
||||
if (customTitle) {
|
||||
this.db.prepare(`
|
||||
UPDATE sdk_sessions SET custom_title = ?
|
||||
WHERE content_session_id = ? AND custom_title IS NULL
|
||||
`).run(customTitle, contentSessionId);
|
||||
}
|
||||
return existing.id;
|
||||
}
|
||||
|
||||
@@ -1392,9 +1449,9 @@ export class SessionStore {
|
||||
// must NEVER equal contentSessionId - that would inject memory messages into the user's transcript!
|
||||
this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(content_session_id, memory_session_id, project, user_prompt, started_at, started_at_epoch, status)
|
||||
VALUES (?, NULL, ?, ?, ?, ?, 'active')
|
||||
`).run(contentSessionId, project, userPrompt, now.toISOString(), nowEpoch);
|
||||
(content_session_id, memory_session_id, project, user_prompt, custom_title, started_at, started_at_epoch, status)
|
||||
VALUES (?, NULL, ?, ?, ?, ?, ?, 'active')
|
||||
`).run(contentSessionId, project, userPrompt, customTitle || null, now.toISOString(), nowEpoch);
|
||||
|
||||
// Return new ID
|
||||
const row = this.db.prepare('SELECT id FROM sdk_sessions WHERE content_session_id = ?')
|
||||
@@ -1441,6 +1498,7 @@ export class SessionStore {
|
||||
/**
|
||||
* Store an observation (from SDK parsing)
|
||||
* Assumes session already exists (created by hook)
|
||||
* Performs content-hash deduplication: skips INSERT if an identical observation exists within 30s
|
||||
*/
|
||||
storeObservation(
|
||||
memorySessionId: string,
|
||||
@@ -1463,11 +1521,18 @@ export class SessionStore {
|
||||
const timestampEpoch = overrideTimestampEpoch ?? Date.now();
|
||||
const timestampIso = new Date(timestampEpoch).toISOString();
|
||||
|
||||
// Content-hash deduplication
|
||||
const contentHash = computeObservationContentHash(memorySessionId, observation.title, observation.narrative);
|
||||
const existing = findDuplicateObservation(this.db, contentHash, timestampEpoch);
|
||||
if (existing) {
|
||||
return { id: existing.id, createdAtEpoch: existing.created_at_epoch };
|
||||
}
|
||||
|
||||
const stmt = this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
(memory_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
files_read, files_modified, prompt_number, discovery_tokens, content_hash, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
const result = stmt.run(
|
||||
@@ -1483,6 +1548,7 @@ export class SessionStore {
|
||||
JSON.stringify(observation.files_modified),
|
||||
promptNumber || null,
|
||||
discoveryTokens,
|
||||
contentHash,
|
||||
timestampIso,
|
||||
timestampEpoch
|
||||
);
|
||||
|
||||
@@ -372,6 +372,16 @@ export const migration005: Migration = {
|
||||
export const migration006: Migration = {
|
||||
version: 6,
|
||||
up: (db: Database) => {
|
||||
// FTS5 may be unavailable on some platforms (e.g., Bun on Windows #791).
|
||||
// Probe before creating tables — search falls back to ChromaDB when unavailable.
|
||||
try {
|
||||
db.run('CREATE VIRTUAL TABLE _fts5_probe USING fts5(test_column)');
|
||||
db.run('DROP TABLE _fts5_probe');
|
||||
} catch {
|
||||
console.log('⚠️ FTS5 not available on this platform — skipping FTS migration (search uses ChromaDB)');
|
||||
return;
|
||||
}
|
||||
|
||||
// FTS5 virtual table for observations
|
||||
// Note: This assumes the hierarchical fields (title, subtitle, etc.) already exist
|
||||
// from the inline migrations in SessionStore constructor
|
||||
|
||||
@@ -31,11 +31,18 @@ export class MigrationRunner {
|
||||
this.renameSessionIdColumns();
|
||||
this.repairSessionIdColumnRename();
|
||||
this.addFailedAtEpochColumn();
|
||||
this.addOnUpdateCascadeToForeignKeys();
|
||||
this.addObservationContentHashColumn();
|
||||
this.addSessionCustomTitleColumn();
|
||||
}
|
||||
|
||||
/**
|
||||
* Initialize database schema using migrations (migration004)
|
||||
* This runs the core SDK tables migration if no tables exist
|
||||
* Initialize database schema (migration004)
|
||||
*
|
||||
* ALWAYS creates core tables using CREATE TABLE IF NOT EXISTS — safe to run
|
||||
* regardless of schema_versions state. This fixes issue #979 where the old
|
||||
* DatabaseManager migration system (versions 1-7) shared the schema_versions
|
||||
* table, causing maxApplied > 0 and skipping core table creation entirely.
|
||||
*/
|
||||
private initializeSchema(): void {
|
||||
// Create schema_versions table if it doesn't exist
|
||||
@@ -47,90 +54,77 @@ export class MigrationRunner {
|
||||
)
|
||||
`);
|
||||
|
||||
// Get applied migrations
|
||||
const appliedVersions = this.db.prepare('SELECT version FROM schema_versions ORDER BY version').all() as SchemaVersion[];
|
||||
const maxApplied = appliedVersions.length > 0 ? Math.max(...appliedVersions.map(v => v.version)) : 0;
|
||||
// Always create core tables — IF NOT EXISTS makes this idempotent
|
||||
this.db.run(`
|
||||
CREATE TABLE IF NOT EXISTS sdk_sessions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
content_session_id TEXT UNIQUE NOT NULL,
|
||||
memory_session_id TEXT UNIQUE,
|
||||
project TEXT NOT NULL,
|
||||
user_prompt TEXT,
|
||||
started_at TEXT NOT NULL,
|
||||
started_at_epoch INTEGER NOT NULL,
|
||||
completed_at TEXT,
|
||||
completed_at_epoch INTEGER,
|
||||
status TEXT CHECK(status IN ('active', 'completed', 'failed')) NOT NULL DEFAULT 'active'
|
||||
);
|
||||
|
||||
// Only run migration004 if no migrations have been applied
|
||||
// This creates the sdk_sessions, observations, and session_summaries tables
|
||||
if (maxApplied === 0) {
|
||||
logger.info('DB', 'Initializing fresh database with migration004');
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_claude_id ON sdk_sessions(content_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_sdk_id ON sdk_sessions(memory_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_project ON sdk_sessions(project);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_status ON sdk_sessions(status);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_started ON sdk_sessions(started_at_epoch DESC);
|
||||
|
||||
// Migration004: SDK agent architecture tables
|
||||
this.db.run(`
|
||||
CREATE TABLE IF NOT EXISTS sdk_sessions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
content_session_id TEXT UNIQUE NOT NULL,
|
||||
memory_session_id TEXT UNIQUE,
|
||||
project TEXT NOT NULL,
|
||||
user_prompt TEXT,
|
||||
started_at TEXT NOT NULL,
|
||||
started_at_epoch INTEGER NOT NULL,
|
||||
completed_at TEXT,
|
||||
completed_at_epoch INTEGER,
|
||||
status TEXT CHECK(status IN ('active', 'completed', 'failed')) NOT NULL DEFAULT 'active'
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS observations (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
memory_session_id TEXT NOT NULL,
|
||||
project TEXT NOT NULL,
|
||||
text TEXT NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_claude_id ON sdk_sessions(content_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_sdk_id ON sdk_sessions(memory_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_project ON sdk_sessions(project);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_status ON sdk_sessions(status);
|
||||
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_started ON sdk_sessions(started_at_epoch DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_sdk_session ON observations(memory_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_project ON observations(project);
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_type ON observations(type);
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_created ON observations(created_at_epoch DESC);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS observations (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
memory_session_id TEXT NOT NULL,
|
||||
project TEXT NOT NULL,
|
||||
text TEXT NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS session_summaries (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
memory_session_id TEXT UNIQUE NOT NULL,
|
||||
project TEXT NOT NULL,
|
||||
request TEXT,
|
||||
investigated TEXT,
|
||||
learned TEXT,
|
||||
completed TEXT,
|
||||
next_steps TEXT,
|
||||
files_read TEXT,
|
||||
files_edited TEXT,
|
||||
notes TEXT,
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_sdk_session ON observations(memory_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_project ON observations(project);
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_type ON observations(type);
|
||||
CREATE INDEX IF NOT EXISTS idx_observations_created ON observations(created_at_epoch DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_sdk_session ON session_summaries(memory_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_project ON session_summaries(project);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
|
||||
`);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS session_summaries (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
memory_session_id TEXT UNIQUE NOT NULL,
|
||||
project TEXT NOT NULL,
|
||||
request TEXT,
|
||||
investigated TEXT,
|
||||
learned TEXT,
|
||||
completed TEXT,
|
||||
next_steps TEXT,
|
||||
files_read TEXT,
|
||||
files_edited TEXT,
|
||||
notes TEXT,
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_sdk_session ON session_summaries(memory_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_project ON session_summaries(project);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
|
||||
`);
|
||||
|
||||
// Record migration004 as applied
|
||||
this.db.prepare('INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)').run(4, new Date().toISOString());
|
||||
|
||||
logger.info('DB', 'Migration004 applied successfully');
|
||||
}
|
||||
// Record migration004 as applied (OR IGNORE handles re-runs safely)
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(4, new Date().toISOString());
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure worker_port column exists (migration 5)
|
||||
*
|
||||
* NOTE: Version 5 conflicts with old DatabaseManager migration005 (which drops orphaned tables).
|
||||
* We check actual column state rather than relying solely on version tracking.
|
||||
*/
|
||||
private ensureWorkerPortColumn(): void {
|
||||
// Check if migration already applied
|
||||
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(5) as SchemaVersion | undefined;
|
||||
if (applied) return;
|
||||
|
||||
// Check if column exists
|
||||
// Check actual column existence — don't rely on version tracking alone (issue #979)
|
||||
const tableInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
|
||||
const hasWorkerPort = tableInfo.some(col => col.name === 'worker_port');
|
||||
|
||||
@@ -145,12 +139,12 @@ export class MigrationRunner {
|
||||
|
||||
/**
|
||||
* Ensure prompt tracking columns exist (migration 6)
|
||||
*
|
||||
* NOTE: Version 6 conflicts with old DatabaseManager migration006 (which creates FTS5 tables).
|
||||
* We check actual column state rather than relying solely on version tracking.
|
||||
*/
|
||||
private ensurePromptTrackingColumns(): void {
|
||||
// Check if migration already applied
|
||||
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(6) as SchemaVersion | undefined;
|
||||
if (applied) return;
|
||||
|
||||
// Check actual column existence — don't rely on version tracking alone (issue #979)
|
||||
// Check sdk_sessions for prompt_counter
|
||||
const sessionsInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
|
||||
const hasPromptCounter = sessionsInfo.some(col => col.name === 'prompt_counter');
|
||||
@@ -184,13 +178,12 @@ export class MigrationRunner {
|
||||
|
||||
/**
|
||||
* Remove UNIQUE constraint from session_summaries.memory_session_id (migration 7)
|
||||
*
|
||||
* NOTE: Version 7 conflicts with old DatabaseManager migration007 (which adds discovery_tokens).
|
||||
* We check actual constraint state rather than relying solely on version tracking.
|
||||
*/
|
||||
private removeSessionSummariesUniqueConstraint(): void {
|
||||
// Check if migration already applied
|
||||
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(7) as SchemaVersion | undefined;
|
||||
if (applied) return;
|
||||
|
||||
// Check if UNIQUE constraint exists
|
||||
// Check actual constraint state — don't rely on version tracking alone (issue #979)
|
||||
const summariesIndexes = this.db.query('PRAGMA index_list(session_summaries)').all() as IndexInfo[];
|
||||
const hasUniqueConstraint = summariesIndexes.some(idx => idx.unique === 1);
|
||||
|
||||
@@ -205,6 +198,9 @@ export class MigrationRunner {
|
||||
// Begin transaction
|
||||
this.db.run('BEGIN TRANSACTION');
|
||||
|
||||
// Clean up leftover temp table from a previously-crashed run
|
||||
this.db.run('DROP TABLE IF EXISTS session_summaries_new');
|
||||
|
||||
// Create new table without UNIQUE constraint
|
||||
this.db.run(`
|
||||
CREATE TABLE session_summaries_new (
|
||||
@@ -318,6 +314,9 @@ export class MigrationRunner {
|
||||
// Begin transaction
|
||||
this.db.run('BEGIN TRANSACTION');
|
||||
|
||||
// Clean up leftover temp table from a previously-crashed run
|
||||
this.db.run('DROP TABLE IF EXISTS observations_new');
|
||||
|
||||
// Create new table with text as nullable
|
||||
this.db.run(`
|
||||
CREATE TABLE observations_new (
|
||||
@@ -411,34 +410,39 @@ export class MigrationRunner {
|
||||
CREATE INDEX idx_user_prompts_lookup ON user_prompts(content_session_id, prompt_number);
|
||||
`);
|
||||
|
||||
// Create FTS5 virtual table
|
||||
this.db.run(`
|
||||
CREATE VIRTUAL TABLE user_prompts_fts USING fts5(
|
||||
prompt_text,
|
||||
content='user_prompts',
|
||||
content_rowid='id'
|
||||
);
|
||||
`);
|
||||
// Create FTS5 virtual table — skip if FTS5 is unavailable (e.g., Bun on Windows #791).
|
||||
// The user_prompts table itself is still created; only FTS indexing is skipped.
|
||||
try {
|
||||
this.db.run(`
|
||||
CREATE VIRTUAL TABLE user_prompts_fts USING fts5(
|
||||
prompt_text,
|
||||
content='user_prompts',
|
||||
content_rowid='id'
|
||||
);
|
||||
`);
|
||||
|
||||
// Create triggers to sync FTS5
|
||||
this.db.run(`
|
||||
CREATE TRIGGER user_prompts_ai AFTER INSERT ON user_prompts BEGIN
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
// Create triggers to sync FTS5
|
||||
this.db.run(`
|
||||
CREATE TRIGGER user_prompts_ai AFTER INSERT ON user_prompts BEGIN
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER user_prompts_ad AFTER DELETE ON user_prompts BEGIN
|
||||
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
|
||||
VALUES('delete', old.id, old.prompt_text);
|
||||
END;
|
||||
CREATE TRIGGER user_prompts_ad AFTER DELETE ON user_prompts BEGIN
|
||||
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
|
||||
VALUES('delete', old.id, old.prompt_text);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER user_prompts_au AFTER UPDATE ON user_prompts BEGIN
|
||||
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
|
||||
VALUES('delete', old.id, old.prompt_text);
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
`);
|
||||
CREATE TRIGGER user_prompts_au AFTER UPDATE ON user_prompts BEGIN
|
||||
INSERT INTO user_prompts_fts(user_prompts_fts, rowid, prompt_text)
|
||||
VALUES('delete', old.id, old.prompt_text);
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
`);
|
||||
} catch (ftsError) {
|
||||
logger.warn('DB', 'FTS5 not available — user_prompts_fts skipped (search uses ChromaDB)', {}, ftsError as Error);
|
||||
}
|
||||
|
||||
// Commit transaction
|
||||
this.db.run('COMMIT');
|
||||
@@ -446,7 +450,7 @@ export class MigrationRunner {
|
||||
// Record migration
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(10, new Date().toISOString());
|
||||
|
||||
logger.debug('DB', 'Successfully created user_prompts table with FTS5 support');
|
||||
logger.debug('DB', 'Successfully created user_prompts table');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -628,4 +632,231 @@ export class MigrationRunner {
|
||||
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(20, new Date().toISOString());
|
||||
}
|
||||
|
||||
/**
|
||||
* Add ON UPDATE CASCADE to FK constraints on observations and session_summaries (migration 21)
|
||||
*
|
||||
* Both tables have FK(memory_session_id) -> sdk_sessions(memory_session_id) with ON DELETE CASCADE
|
||||
* but missing ON UPDATE CASCADE. This causes FK constraint violations when code updates
|
||||
* sdk_sessions.memory_session_id while child rows still reference the old value.
|
||||
*
|
||||
* SQLite doesn't support ALTER TABLE for FK changes, so we recreate both tables.
|
||||
*/
|
||||
private addOnUpdateCascadeToForeignKeys(): void {
|
||||
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(21) as SchemaVersion | undefined;
|
||||
if (applied) return;
|
||||
|
||||
logger.debug('DB', 'Adding ON UPDATE CASCADE to FK constraints on observations and session_summaries');
|
||||
|
||||
// PRAGMA foreign_keys must be set outside a transaction
|
||||
this.db.run('PRAGMA foreign_keys = OFF');
|
||||
this.db.run('BEGIN TRANSACTION');
|
||||
|
||||
try {
|
||||
// ==========================================
|
||||
// 1. Recreate observations table
|
||||
// ==========================================
|
||||
|
||||
// Drop FTS triggers first (they reference the observations table)
|
||||
this.db.run('DROP TRIGGER IF EXISTS observations_ai');
|
||||
this.db.run('DROP TRIGGER IF EXISTS observations_ad');
|
||||
this.db.run('DROP TRIGGER IF EXISTS observations_au');
|
||||
|
||||
// Clean up leftover temp table from a previously-crashed run
|
||||
this.db.run('DROP TABLE IF EXISTS observations_new');
|
||||
|
||||
this.db.run(`
|
||||
CREATE TABLE observations_new (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
memory_session_id TEXT NOT NULL,
|
||||
project TEXT NOT NULL,
|
||||
text TEXT,
|
||||
type TEXT NOT NULL,
|
||||
title TEXT,
|
||||
subtitle TEXT,
|
||||
facts TEXT,
|
||||
narrative TEXT,
|
||||
concepts TEXT,
|
||||
files_read TEXT,
|
||||
files_modified TEXT,
|
||||
prompt_number INTEGER,
|
||||
discovery_tokens INTEGER DEFAULT 0,
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
|
||||
)
|
||||
`);
|
||||
|
||||
this.db.run(`
|
||||
INSERT INTO observations_new
|
||||
SELECT id, memory_session_id, project, text, type, title, subtitle, facts,
|
||||
narrative, concepts, files_read, files_modified, prompt_number,
|
||||
discovery_tokens, created_at, created_at_epoch
|
||||
FROM observations
|
||||
`);
|
||||
|
||||
this.db.run('DROP TABLE observations');
|
||||
this.db.run('ALTER TABLE observations_new RENAME TO observations');
|
||||
|
||||
// Recreate indexes
|
||||
this.db.run(`
|
||||
CREATE INDEX idx_observations_sdk_session ON observations(memory_session_id);
|
||||
CREATE INDEX idx_observations_project ON observations(project);
|
||||
CREATE INDEX idx_observations_type ON observations(type);
|
||||
CREATE INDEX idx_observations_created ON observations(created_at_epoch DESC);
|
||||
`);
|
||||
|
||||
// Recreate FTS triggers only if observations_fts exists
|
||||
const hasFTS = (this.db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name='observations_fts'").all() as { name: string }[]).length > 0;
|
||||
if (hasFTS) {
|
||||
this.db.run(`
|
||||
CREATE TRIGGER IF NOT EXISTS observations_ai AFTER INSERT ON observations BEGIN
|
||||
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS observations_ad AFTER DELETE ON observations BEGIN
|
||||
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS observations_au AFTER UPDATE ON observations BEGIN
|
||||
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
|
||||
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
|
||||
END;
|
||||
`);
|
||||
}
|
||||
|
||||
// ==========================================
|
||||
// 2. Recreate session_summaries table
|
||||
// ==========================================
|
||||
|
||||
// Clean up leftover temp table from a previously-crashed run
|
||||
this.db.run('DROP TABLE IF EXISTS session_summaries_new');
|
||||
|
||||
this.db.run(`
|
||||
CREATE TABLE session_summaries_new (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
memory_session_id TEXT NOT NULL,
|
||||
project TEXT NOT NULL,
|
||||
request TEXT,
|
||||
investigated TEXT,
|
||||
learned TEXT,
|
||||
completed TEXT,
|
||||
next_steps TEXT,
|
||||
files_read TEXT,
|
||||
files_edited TEXT,
|
||||
notes TEXT,
|
||||
prompt_number INTEGER,
|
||||
discovery_tokens INTEGER DEFAULT 0,
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE ON UPDATE CASCADE
|
||||
)
|
||||
`);
|
||||
|
||||
this.db.run(`
|
||||
INSERT INTO session_summaries_new
|
||||
SELECT id, memory_session_id, project, request, investigated, learned,
|
||||
completed, next_steps, files_read, files_edited, notes,
|
||||
prompt_number, discovery_tokens, created_at, created_at_epoch
|
||||
FROM session_summaries
|
||||
`);
|
||||
|
||||
// Drop session_summaries FTS triggers before dropping the table
|
||||
this.db.run('DROP TRIGGER IF EXISTS session_summaries_ai');
|
||||
this.db.run('DROP TRIGGER IF EXISTS session_summaries_ad');
|
||||
this.db.run('DROP TRIGGER IF EXISTS session_summaries_au');
|
||||
|
||||
this.db.run('DROP TABLE session_summaries');
|
||||
this.db.run('ALTER TABLE session_summaries_new RENAME TO session_summaries');
|
||||
|
||||
// Recreate indexes
|
||||
this.db.run(`
|
||||
CREATE INDEX idx_session_summaries_sdk_session ON session_summaries(memory_session_id);
|
||||
CREATE INDEX idx_session_summaries_project ON session_summaries(project);
|
||||
CREATE INDEX idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
|
||||
`);
|
||||
|
||||
// Recreate session_summaries FTS triggers if FTS table exists
|
||||
const hasSummariesFTS = (this.db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name='session_summaries_fts'").all() as { name: string }[]).length > 0;
|
||||
if (hasSummariesFTS) {
|
||||
this.db.run(`
|
||||
CREATE TRIGGER IF NOT EXISTS session_summaries_ai AFTER INSERT ON session_summaries BEGIN
|
||||
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
|
||||
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS session_summaries_ad AFTER DELETE ON session_summaries BEGIN
|
||||
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
|
||||
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS session_summaries_au AFTER UPDATE ON session_summaries BEGIN
|
||||
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
|
||||
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
|
||||
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
|
||||
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
|
||||
END;
|
||||
`);
|
||||
}
|
||||
|
||||
// Record migration
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(21, new Date().toISOString());
|
||||
|
||||
this.db.run('COMMIT');
|
||||
this.db.run('PRAGMA foreign_keys = ON');
|
||||
|
||||
logger.debug('DB', 'Successfully added ON UPDATE CASCADE to FK constraints');
|
||||
} catch (error) {
|
||||
this.db.run('ROLLBACK');
|
||||
this.db.run('PRAGMA foreign_keys = ON');
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Add content_hash column to observations for deduplication (migration 22)
|
||||
* Prevents duplicate observations from being stored when the same content is processed multiple times.
|
||||
* Backfills existing rows with unique random hashes so they don't block new inserts.
|
||||
*/
|
||||
private addObservationContentHashColumn(): void {
|
||||
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(22) as SchemaVersion | undefined;
|
||||
if (applied) return;
|
||||
|
||||
const tableInfo = this.db.query('PRAGMA table_info(observations)').all() as TableColumnInfo[];
|
||||
const hasColumn = tableInfo.some(col => col.name === 'content_hash');
|
||||
|
||||
if (!hasColumn) {
|
||||
this.db.run('ALTER TABLE observations ADD COLUMN content_hash TEXT');
|
||||
// Backfill existing rows with unique random hashes
|
||||
this.db.run("UPDATE observations SET content_hash = substr(hex(randomblob(8)), 1, 16) WHERE content_hash IS NULL");
|
||||
// Index for fast dedup lookups
|
||||
this.db.run('CREATE INDEX IF NOT EXISTS idx_observations_content_hash ON observations(content_hash, created_at_epoch)');
|
||||
logger.debug('DB', 'Added content_hash column to observations table with backfill and index');
|
||||
}
|
||||
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(22, new Date().toISOString());
|
||||
}
|
||||
|
||||
/**
|
||||
* Add custom_title column to sdk_sessions for agent attribution (migration 23)
|
||||
* Allows callers (e.g. Maestro agents) to label sessions with a human-readable name.
|
||||
*/
|
||||
private addSessionCustomTitleColumn(): void {
|
||||
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(23) as SchemaVersion | undefined;
|
||||
if (applied) return;
|
||||
|
||||
const tableInfo = this.db.query('PRAGMA table_info(sdk_sessions)').all() as TableColumnInfo[];
|
||||
const hasColumn = tableInfo.some(col => col.name === 'custom_title');
|
||||
|
||||
if (!hasColumn) {
|
||||
this.db.run('ALTER TABLE sdk_sessions ADD COLUMN custom_title TEXT');
|
||||
logger.debug('DB', 'Added custom_title column to sdk_sessions table');
|
||||
}
|
||||
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(23, new Date().toISOString());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,13 +3,50 @@
|
||||
* Extracted from SessionStore.ts for modular organization
|
||||
*/
|
||||
|
||||
import { createHash } from 'crypto';
|
||||
import { Database } from 'bun:sqlite';
|
||||
import { logger } from '../../../utils/logger.js';
|
||||
import { getCurrentProjectName } from '../../../shared/paths.js';
|
||||
import type { ObservationInput, StoreObservationResult } from './types.js';
|
||||
|
||||
/** Deduplication window: observations with the same content hash within this window are skipped */
|
||||
const DEDUP_WINDOW_MS = 30_000;
|
||||
|
||||
/**
|
||||
* Compute a short content hash for deduplication.
|
||||
* Uses (memory_session_id, title, narrative) as the semantic identity of an observation.
|
||||
*/
|
||||
export function computeObservationContentHash(
|
||||
memorySessionId: string,
|
||||
title: string | null,
|
||||
narrative: string | null
|
||||
): string {
|
||||
return createHash('sha256')
|
||||
.update((memorySessionId || '') + (title || '') + (narrative || ''))
|
||||
.digest('hex')
|
||||
.slice(0, 16);
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a duplicate observation exists within the dedup window.
|
||||
* Returns the existing observation's id and timestamp if found, null otherwise.
|
||||
*/
|
||||
export function findDuplicateObservation(
|
||||
db: Database,
|
||||
contentHash: string,
|
||||
timestampEpoch: number
|
||||
): { id: number; created_at_epoch: number } | null {
|
||||
const windowStart = timestampEpoch - DEDUP_WINDOW_MS;
|
||||
const stmt = db.prepare(
|
||||
'SELECT id, created_at_epoch FROM observations WHERE content_hash = ? AND created_at_epoch > ?'
|
||||
);
|
||||
return (stmt.get(contentHash, windowStart) as { id: number; created_at_epoch: number } | null);
|
||||
}
|
||||
|
||||
/**
|
||||
* Store an observation (from SDK parsing)
|
||||
* Assumes session already exists (created by hook)
|
||||
* Performs content-hash deduplication: skips INSERT if an identical observation exists within 30s
|
||||
*/
|
||||
export function storeObservation(
|
||||
db: Database,
|
||||
@@ -24,16 +61,27 @@ export function storeObservation(
|
||||
const timestampEpoch = overrideTimestampEpoch ?? Date.now();
|
||||
const timestampIso = new Date(timestampEpoch).toISOString();
|
||||
|
||||
// Guard against empty project string (race condition where project isn't set yet)
|
||||
const resolvedProject = project || getCurrentProjectName();
|
||||
|
||||
// Content-hash deduplication
|
||||
const contentHash = computeObservationContentHash(memorySessionId, observation.title, observation.narrative);
|
||||
const existing = findDuplicateObservation(db, contentHash, timestampEpoch);
|
||||
if (existing) {
|
||||
logger.debug('DEDUP', `Skipped duplicate observation | contentHash=${contentHash} | existingId=${existing.id}`);
|
||||
return { id: existing.id, createdAtEpoch: existing.created_at_epoch };
|
||||
}
|
||||
|
||||
const stmt = db.prepare(`
|
||||
INSERT INTO observations
|
||||
(memory_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
files_read, files_modified, prompt_number, discovery_tokens, content_hash, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
const result = stmt.run(
|
||||
memorySessionId,
|
||||
project,
|
||||
resolvedProject,
|
||||
observation.type,
|
||||
observation.title,
|
||||
observation.subtitle,
|
||||
@@ -44,6 +92,7 @@ export function storeObservation(
|
||||
JSON.stringify(observation.files_modified),
|
||||
promptNumber || null,
|
||||
discoveryTokens,
|
||||
contentHash,
|
||||
timestampIso,
|
||||
timestampEpoch
|
||||
);
|
||||
|
||||
@@ -21,7 +21,8 @@ export function createSDKSession(
|
||||
db: Database,
|
||||
contentSessionId: string,
|
||||
project: string,
|
||||
userPrompt: string
|
||||
userPrompt: string,
|
||||
customTitle?: string
|
||||
): number {
|
||||
const now = new Date();
|
||||
const nowEpoch = now.getTime();
|
||||
@@ -39,6 +40,13 @@ export function createSDKSession(
|
||||
WHERE content_session_id = ? AND (project IS NULL OR project = '')
|
||||
`).run(project, contentSessionId);
|
||||
}
|
||||
// Backfill custom_title if provided and not yet set
|
||||
if (customTitle) {
|
||||
db.prepare(`
|
||||
UPDATE sdk_sessions SET custom_title = ?
|
||||
WHERE content_session_id = ? AND custom_title IS NULL
|
||||
`).run(customTitle, contentSessionId);
|
||||
}
|
||||
return existing.id;
|
||||
}
|
||||
|
||||
@@ -48,9 +56,9 @@ export function createSDKSession(
|
||||
// must NEVER equal contentSessionId - that would inject memory messages into the user's transcript!
|
||||
db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(content_session_id, memory_session_id, project, user_prompt, started_at, started_at_epoch, status)
|
||||
VALUES (?, NULL, ?, ?, ?, ?, 'active')
|
||||
`).run(contentSessionId, project, userPrompt, now.toISOString(), nowEpoch);
|
||||
(content_session_id, memory_session_id, project, user_prompt, custom_title, started_at, started_at_epoch, status)
|
||||
VALUES (?, NULL, ?, ?, ?, ?, ?, 'active')
|
||||
`).run(contentSessionId, project, userPrompt, customTitle || null, now.toISOString(), nowEpoch);
|
||||
|
||||
// Return new ID
|
||||
const row = db.prepare('SELECT id FROM sdk_sessions WHERE content_session_id = ?')
|
||||
|
||||
@@ -17,7 +17,7 @@ import type {
|
||||
*/
|
||||
export function getSessionById(db: Database, id: number): SessionBasic | null {
|
||||
const stmt = db.prepare(`
|
||||
SELECT id, content_session_id, memory_session_id, project, user_prompt
|
||||
SELECT id, content_session_id, memory_session_id, project, user_prompt, custom_title
|
||||
FROM sdk_sessions
|
||||
WHERE id = ?
|
||||
LIMIT 1
|
||||
@@ -38,7 +38,7 @@ export function getSdkSessionsBySessionIds(
|
||||
|
||||
const placeholders = memorySessionIds.map(() => '?').join(',');
|
||||
const stmt = db.prepare(`
|
||||
SELECT id, content_session_id, memory_session_id, project, user_prompt,
|
||||
SELECT id, content_session_id, memory_session_id, project, user_prompt, custom_title,
|
||||
started_at, started_at_epoch, completed_at, completed_at_epoch, status
|
||||
FROM sdk_sessions
|
||||
WHERE memory_session_id IN (${placeholders})
|
||||
|
||||
@@ -13,6 +13,7 @@ export interface SessionBasic {
|
||||
memory_session_id: string | null;
|
||||
project: string;
|
||||
user_prompt: string;
|
||||
custom_title: string | null;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -24,6 +25,7 @@ export interface SessionFull {
|
||||
memory_session_id: string;
|
||||
project: string;
|
||||
user_prompt: string;
|
||||
custom_title: string | null;
|
||||
started_at: string;
|
||||
started_at_epoch: number;
|
||||
completed_at: string | null;
|
||||
|
||||
@@ -10,6 +10,7 @@ import { Database } from 'bun:sqlite';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
import type { ObservationInput } from './observations/types.js';
|
||||
import type { SummaryInput } from './summaries/types.js';
|
||||
import { computeObservationContentHash, findDuplicateObservation } from './observations/store.js';
|
||||
|
||||
/**
|
||||
* Result from storeObservations / storeObservationsAndMarkComplete transaction
|
||||
@@ -63,15 +64,22 @@ export function storeObservationsAndMarkComplete(
|
||||
const storeAndMarkTx = db.transaction(() => {
|
||||
const observationIds: number[] = [];
|
||||
|
||||
// 1. Store all observations
|
||||
// 1. Store all observations (with content-hash deduplication)
|
||||
const obsStmt = db.prepare(`
|
||||
INSERT INTO observations
|
||||
(memory_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
files_read, files_modified, prompt_number, discovery_tokens, content_hash, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
for (const observation of observations) {
|
||||
const contentHash = computeObservationContentHash(memorySessionId, observation.title, observation.narrative);
|
||||
const existing = findDuplicateObservation(db, contentHash, timestampEpoch);
|
||||
if (existing) {
|
||||
observationIds.push(existing.id);
|
||||
continue;
|
||||
}
|
||||
|
||||
const result = obsStmt.run(
|
||||
memorySessionId,
|
||||
project,
|
||||
@@ -85,6 +93,7 @@ export function storeObservationsAndMarkComplete(
|
||||
JSON.stringify(observation.files_modified),
|
||||
promptNumber || null,
|
||||
discoveryTokens,
|
||||
contentHash,
|
||||
timestampIso,
|
||||
timestampEpoch
|
||||
);
|
||||
@@ -174,15 +183,22 @@ export function storeObservations(
|
||||
const storeTx = db.transaction(() => {
|
||||
const observationIds: number[] = [];
|
||||
|
||||
// 1. Store all observations
|
||||
// 1. Store all observations (with content-hash deduplication)
|
||||
const obsStmt = db.prepare(`
|
||||
INSERT INTO observations
|
||||
(memory_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
files_read, files_modified, prompt_number, discovery_tokens, content_hash, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
for (const observation of observations) {
|
||||
const contentHash = computeObservationContentHash(memorySessionId, observation.title, observation.narrative);
|
||||
const existing = findDuplicateObservation(db, contentHash, timestampEpoch);
|
||||
if (existing) {
|
||||
observationIds.push(existing.id);
|
||||
continue;
|
||||
}
|
||||
|
||||
const result = obsStmt.run(
|
||||
memorySessionId,
|
||||
project,
|
||||
@@ -196,6 +212,7 @@ export function storeObservations(
|
||||
JSON.stringify(observation.files_modified),
|
||||
promptNumber || null,
|
||||
discoveryTokens,
|
||||
contentHash,
|
||||
timestampIso,
|
||||
timestampEpoch
|
||||
);
|
||||
|
||||
@@ -0,0 +1,456 @@
|
||||
/**
|
||||
* ChromaMcpManager - Singleton managing a persistent MCP connection to chroma-mcp via uvx
|
||||
*
|
||||
* Replaces ChromaServerManager (which spawned `npx chroma run`) with a stdio-based
|
||||
* MCP client that communicates with chroma-mcp as a subprocess. The chroma-mcp server
|
||||
* handles its own embedding and persistent storage, eliminating the need for a separate
|
||||
* HTTP server, chromadb npm package, and ONNX/WASM embedding dependencies.
|
||||
*
|
||||
* Lifecycle: lazy-connects on first callTool() use, maintains a single persistent
|
||||
* connection per worker lifetime, and auto-reconnects if the subprocess dies.
|
||||
*
|
||||
* Cross-platform: Linux, macOS, Windows
|
||||
*/
|
||||
|
||||
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
|
||||
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';
|
||||
import { execSync } from 'child_process';
|
||||
import path from 'path';
|
||||
import os from 'os';
|
||||
import fs from 'fs';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
import { SettingsDefaultsManager } from '../../shared/SettingsDefaultsManager.js';
|
||||
import { USER_SETTINGS_PATH } from '../../shared/paths.js';
|
||||
|
||||
const CHROMA_MCP_CLIENT_NAME = 'claude-mem-chroma';
|
||||
const CHROMA_MCP_CLIENT_VERSION = '1.0.0';
|
||||
const MCP_CONNECTION_TIMEOUT_MS = 30_000;
|
||||
const RECONNECT_BACKOFF_MS = 10_000; // Don't retry connections faster than this after failure
|
||||
const DEFAULT_CHROMA_DATA_DIR = path.join(os.homedir(), '.claude-mem', 'chroma');
|
||||
|
||||
export class ChromaMcpManager {
|
||||
private static instance: ChromaMcpManager | null = null;
|
||||
private client: Client | null = null;
|
||||
private transport: StdioClientTransport | null = null;
|
||||
private connected: boolean = false;
|
||||
private lastConnectionFailureTimestamp: number = 0;
|
||||
private connecting: Promise<void> | null = null;
|
||||
|
||||
private constructor() {}
|
||||
|
||||
/**
|
||||
* Get or create the singleton instance
|
||||
*/
|
||||
static getInstance(): ChromaMcpManager {
|
||||
if (!ChromaMcpManager.instance) {
|
||||
ChromaMcpManager.instance = new ChromaMcpManager();
|
||||
}
|
||||
return ChromaMcpManager.instance;
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure the MCP client is connected to chroma-mcp.
|
||||
* Uses a connection lock to prevent concurrent connection attempts.
|
||||
* If the subprocess has died since the last use, reconnects transparently.
|
||||
*/
|
||||
private async ensureConnected(): Promise<void> {
|
||||
if (this.connected && this.client) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Backoff: don't retry connections too fast after a failure
|
||||
const timeSinceLastFailure = Date.now() - this.lastConnectionFailureTimestamp;
|
||||
if (this.lastConnectionFailureTimestamp > 0 && timeSinceLastFailure < RECONNECT_BACKOFF_MS) {
|
||||
throw new Error(`chroma-mcp connection in backoff (${Math.ceil((RECONNECT_BACKOFF_MS - timeSinceLastFailure) / 1000)}s remaining)`);
|
||||
}
|
||||
|
||||
// If another caller is already connecting, wait for that attempt
|
||||
if (this.connecting) {
|
||||
await this.connecting;
|
||||
return;
|
||||
}
|
||||
|
||||
this.connecting = this.connectInternal();
|
||||
try {
|
||||
await this.connecting;
|
||||
} catch (error) {
|
||||
this.lastConnectionFailureTimestamp = Date.now();
|
||||
throw error;
|
||||
} finally {
|
||||
this.connecting = null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Internal connection logic - spawns uvx chroma-mcp and performs MCP handshake.
|
||||
* Called behind the connection lock to ensure only one connection attempt at a time.
|
||||
*/
|
||||
private async connectInternal(): Promise<void> {
|
||||
// Clean up any stale client/transport from a dead subprocess.
|
||||
// Close transport first (kills subprocess via SIGTERM) before client
|
||||
// to avoid hanging on a stuck process.
|
||||
if (this.transport) {
|
||||
try { await this.transport.close(); } catch { /* already dead */ }
|
||||
}
|
||||
if (this.client) {
|
||||
try { await this.client.close(); } catch { /* already dead */ }
|
||||
}
|
||||
this.client = null;
|
||||
this.transport = null;
|
||||
this.connected = false;
|
||||
|
||||
const commandArgs = this.buildCommandArgs();
|
||||
const spawnEnvironment = this.getSpawnEnv();
|
||||
|
||||
// On Windows, .cmd files require shell resolution. Since MCP SDK's
|
||||
// StdioClientTransport doesn't support `shell: true`, route through
|
||||
// cmd.exe which resolves .cmd/.bat extensions and PATH automatically.
|
||||
// This also fixes Git Bash compatibility (#1062) since cmd.exe handles
|
||||
// Windows-native command resolution regardless of the calling shell.
|
||||
const isWindows = process.platform === 'win32';
|
||||
const uvxSpawnCommand = isWindows ? (process.env.ComSpec || 'cmd.exe') : 'uvx';
|
||||
const uvxSpawnArgs = isWindows ? ['/c', 'uvx', ...commandArgs] : commandArgs;
|
||||
|
||||
logger.info('CHROMA_MCP', 'Connecting to chroma-mcp via MCP stdio', {
|
||||
command: uvxSpawnCommand,
|
||||
args: uvxSpawnArgs.join(' ')
|
||||
});
|
||||
|
||||
this.transport = new StdioClientTransport({
|
||||
command: uvxSpawnCommand,
|
||||
args: uvxSpawnArgs,
|
||||
env: spawnEnvironment,
|
||||
stderr: 'pipe'
|
||||
});
|
||||
|
||||
this.client = new Client(
|
||||
{ name: CHROMA_MCP_CLIENT_NAME, version: CHROMA_MCP_CLIENT_VERSION },
|
||||
{ capabilities: {} }
|
||||
);
|
||||
|
||||
const mcpConnectionPromise = this.client.connect(this.transport);
|
||||
let timeoutId: ReturnType<typeof setTimeout>;
|
||||
const timeoutPromise = new Promise<never>((_, reject) => {
|
||||
timeoutId = setTimeout(
|
||||
() => reject(new Error(`MCP connection to chroma-mcp timed out after ${MCP_CONNECTION_TIMEOUT_MS}ms`)),
|
||||
MCP_CONNECTION_TIMEOUT_MS
|
||||
);
|
||||
});
|
||||
|
||||
try {
|
||||
await Promise.race([mcpConnectionPromise, timeoutPromise]);
|
||||
} catch (connectionError) {
|
||||
// Connection failed or timed out - kill the subprocess to prevent zombies
|
||||
clearTimeout(timeoutId!);
|
||||
logger.warn('CHROMA_MCP', 'Connection failed, killing subprocess to prevent zombie', {
|
||||
error: connectionError instanceof Error ? connectionError.message : String(connectionError)
|
||||
});
|
||||
try { await this.transport.close(); } catch { /* best effort */ }
|
||||
try { await this.client.close(); } catch { /* best effort */ }
|
||||
this.client = null;
|
||||
this.transport = null;
|
||||
this.connected = false;
|
||||
throw connectionError;
|
||||
}
|
||||
clearTimeout(timeoutId!);
|
||||
|
||||
this.connected = true;
|
||||
|
||||
logger.info('CHROMA_MCP', 'Connected to chroma-mcp successfully');
|
||||
|
||||
// Listen for transport close to mark connection as dead and apply backoff.
|
||||
// CRITICAL: Guard with reference check to prevent stale onclose handlers from
|
||||
// previous transports overwriting the current connection (race condition).
|
||||
const currentTransport = this.transport;
|
||||
this.transport.onclose = () => {
|
||||
if (this.transport !== currentTransport) {
|
||||
logger.debug('CHROMA_MCP', 'Ignoring stale onclose from previous transport');
|
||||
return;
|
||||
}
|
||||
logger.warn('CHROMA_MCP', 'chroma-mcp subprocess closed unexpectedly, applying reconnect backoff');
|
||||
this.connected = false;
|
||||
this.client = null;
|
||||
this.transport = null;
|
||||
this.lastConnectionFailureTimestamp = Date.now();
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the uvx command arguments based on current settings.
|
||||
* In local mode: uses persistent client with local data directory.
|
||||
* In remote mode: uses http client with configured host/port/auth.
|
||||
*/
|
||||
private buildCommandArgs(): string[] {
|
||||
const settings = SettingsDefaultsManager.loadFromFile(USER_SETTINGS_PATH);
|
||||
const chromaMode = settings.CLAUDE_MEM_CHROMA_MODE || 'local';
|
||||
const pythonVersion = process.env.CLAUDE_MEM_PYTHON_VERSION || settings.CLAUDE_MEM_PYTHON_VERSION || '3.13';
|
||||
|
||||
if (chromaMode === 'remote') {
|
||||
const chromaHost = settings.CLAUDE_MEM_CHROMA_HOST || '127.0.0.1';
|
||||
const chromaPort = settings.CLAUDE_MEM_CHROMA_PORT || '8000';
|
||||
const chromaSsl = settings.CLAUDE_MEM_CHROMA_SSL === 'true';
|
||||
const chromaTenant = settings.CLAUDE_MEM_CHROMA_TENANT || 'default_tenant';
|
||||
const chromaDatabase = settings.CLAUDE_MEM_CHROMA_DATABASE || 'default_database';
|
||||
const chromaApiKey = settings.CLAUDE_MEM_CHROMA_API_KEY || '';
|
||||
|
||||
const args = [
|
||||
'--python', pythonVersion,
|
||||
'chroma-mcp',
|
||||
'--client-type', 'http',
|
||||
'--host', chromaHost,
|
||||
'--port', chromaPort
|
||||
];
|
||||
|
||||
if (chromaSsl) {
|
||||
args.push('--ssl');
|
||||
}
|
||||
|
||||
if (chromaTenant !== 'default_tenant') {
|
||||
args.push('--tenant', chromaTenant);
|
||||
}
|
||||
|
||||
if (chromaDatabase !== 'default_database') {
|
||||
args.push('--database', chromaDatabase);
|
||||
}
|
||||
|
||||
if (chromaApiKey) {
|
||||
args.push('--api-key', chromaApiKey);
|
||||
}
|
||||
|
||||
return args;
|
||||
}
|
||||
|
||||
// Local mode: persistent client with data directory
|
||||
return [
|
||||
'--python', pythonVersion,
|
||||
'chroma-mcp',
|
||||
'--client-type', 'persistent',
|
||||
'--data-dir', DEFAULT_CHROMA_DATA_DIR.replace(/\\/g, '/')
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* Call a chroma-mcp tool by name with the given arguments.
|
||||
* Lazily connects on first call. Reconnects if the subprocess has died.
|
||||
*
|
||||
* @param toolName - The chroma-mcp tool name (e.g. 'chroma_query_documents')
|
||||
* @param toolArguments - The tool arguments as a plain object
|
||||
* @returns The parsed JSON result from the tool's text output
|
||||
*/
|
||||
async callTool(toolName: string, toolArguments: Record<string, unknown>): Promise<unknown> {
|
||||
await this.ensureConnected();
|
||||
|
||||
logger.debug('CHROMA_MCP', `Calling tool: ${toolName}`, {
|
||||
arguments: JSON.stringify(toolArguments).slice(0, 200)
|
||||
});
|
||||
|
||||
let result;
|
||||
try {
|
||||
result = await this.client!.callTool({
|
||||
name: toolName,
|
||||
arguments: toolArguments
|
||||
});
|
||||
} catch (transportError) {
|
||||
// Transport error: chroma-mcp subprocess likely died (e.g., killed by orphan reaper,
|
||||
// HNSW index corruption). Mark connection dead and retry once after reconnect (#1131).
|
||||
// Without this retry, callers see a one-shot error even though reconnect would succeed.
|
||||
this.connected = false;
|
||||
this.client = null;
|
||||
this.transport = null;
|
||||
|
||||
logger.warn('CHROMA_MCP', `Transport error during "${toolName}", reconnecting and retrying once`, {
|
||||
error: transportError instanceof Error ? transportError.message : String(transportError)
|
||||
});
|
||||
|
||||
try {
|
||||
await this.ensureConnected();
|
||||
result = await this.client!.callTool({
|
||||
name: toolName,
|
||||
arguments: toolArguments
|
||||
});
|
||||
} catch (retryError) {
|
||||
this.connected = false;
|
||||
throw new Error(`chroma-mcp transport error during "${toolName}" (retry failed): ${retryError instanceof Error ? retryError.message : String(retryError)}`);
|
||||
}
|
||||
}
|
||||
|
||||
// MCP tools signal errors via isError flag on the CallToolResult
|
||||
if (result.isError) {
|
||||
const errorText = (result.content as Array<{ type: string; text?: string }>)
|
||||
?.find(item => item.type === 'text')?.text || 'Unknown chroma-mcp error';
|
||||
throw new Error(`chroma-mcp tool "${toolName}" returned error: ${errorText}`);
|
||||
}
|
||||
|
||||
// Extract text from MCP CallToolResult: { content: Array<{ type, text? }> }
|
||||
const contentArray = result.content as Array<{ type: string; text?: string }>;
|
||||
if (!contentArray || contentArray.length === 0) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const firstTextContent = contentArray.find(item => item.type === 'text' && item.text);
|
||||
if (!firstTextContent || !firstTextContent.text) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// chroma-mcp returns JSON for query/get results, but plain text for
|
||||
// mutating operations (e.g. "Successfully created collection ...").
|
||||
// Try JSON parse first; if it fails, return the raw text for non-error responses.
|
||||
try {
|
||||
return JSON.parse(firstTextContent.text);
|
||||
} catch {
|
||||
// Plain text response (e.g. "Successfully created collection cm__foo")
|
||||
// Return null for void-like success messages, callers don't need the text
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if the MCP connection is alive by calling chroma_list_collections.
|
||||
* Returns true if the connection is healthy, false otherwise.
|
||||
*/
|
||||
async isHealthy(): Promise<boolean> {
|
||||
try {
|
||||
await this.callTool('chroma_list_collections', { limit: 1 });
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Gracefully stop the MCP connection and kill the chroma-mcp subprocess.
|
||||
* client.close() sends stdin close -> SIGTERM -> SIGKILL to the subprocess.
|
||||
*/
|
||||
async stop(): Promise<void> {
|
||||
if (!this.client) {
|
||||
logger.debug('CHROMA_MCP', 'No active MCP connection to stop');
|
||||
return;
|
||||
}
|
||||
|
||||
logger.info('CHROMA_MCP', 'Stopping chroma-mcp MCP connection');
|
||||
|
||||
try {
|
||||
await this.client.close();
|
||||
} catch (error) {
|
||||
logger.debug('CHROMA_MCP', 'Error during client close (subprocess may already be dead)', {}, error as Error);
|
||||
}
|
||||
|
||||
this.client = null;
|
||||
this.transport = null;
|
||||
this.connected = false;
|
||||
this.connecting = null;
|
||||
|
||||
logger.info('CHROMA_MCP', 'chroma-mcp MCP connection stopped');
|
||||
}
|
||||
|
||||
/**
|
||||
* Reset the singleton instance (for testing).
|
||||
* Awaits stop() to prevent dual subprocesses.
|
||||
*/
|
||||
static async reset(): Promise<void> {
|
||||
if (ChromaMcpManager.instance) {
|
||||
await ChromaMcpManager.instance.stop();
|
||||
}
|
||||
ChromaMcpManager.instance = null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get or create a combined SSL certificate bundle for Zscaler/corporate proxy environments.
|
||||
* On macOS, combines the Python certifi CA bundle with any Zscaler certificates from
|
||||
* the system keychain. Caches the result for 24 hours at ~/.claude-mem/combined_certs.pem.
|
||||
*
|
||||
* Returns the path to the combined cert file, or undefined if not needed/available.
|
||||
*/
|
||||
private getCombinedCertPath(): string | undefined {
|
||||
const combinedCertPath = path.join(os.homedir(), '.claude-mem', 'combined_certs.pem');
|
||||
|
||||
if (fs.existsSync(combinedCertPath)) {
|
||||
const stats = fs.statSync(combinedCertPath);
|
||||
const ageMs = Date.now() - stats.mtimeMs;
|
||||
if (ageMs < 24 * 60 * 60 * 1000) {
|
||||
return combinedCertPath;
|
||||
}
|
||||
}
|
||||
|
||||
if (process.platform !== 'darwin') {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
try {
|
||||
let certifiPath: string | undefined;
|
||||
try {
|
||||
certifiPath = execSync(
|
||||
'uvx --with certifi python -c "import certifi; print(certifi.where())"',
|
||||
{ encoding: 'utf8', stdio: ['pipe', 'pipe', 'pipe'], timeout: 10000 }
|
||||
).trim();
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
if (!certifiPath || !fs.existsSync(certifiPath)) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
let zscalerCert = '';
|
||||
try {
|
||||
zscalerCert = execSync(
|
||||
'security find-certificate -a -c "Zscaler" -p /Library/Keychains/System.keychain',
|
||||
{ encoding: 'utf8', stdio: ['pipe', 'pipe', 'pipe'], timeout: 5000 }
|
||||
);
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
if (!zscalerCert ||
|
||||
!zscalerCert.includes('-----BEGIN CERTIFICATE-----') ||
|
||||
!zscalerCert.includes('-----END CERTIFICATE-----')) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const certifiContent = fs.readFileSync(certifiPath, 'utf8');
|
||||
const tempPath = combinedCertPath + '.tmp';
|
||||
fs.writeFileSync(tempPath, certifiContent + '\n' + zscalerCert);
|
||||
fs.renameSync(tempPath, combinedCertPath);
|
||||
|
||||
logger.info('CHROMA_MCP', 'Created combined SSL certificate bundle for Zscaler', {
|
||||
path: combinedCertPath
|
||||
});
|
||||
|
||||
return combinedCertPath;
|
||||
} catch (error) {
|
||||
logger.debug('CHROMA_MCP', 'Could not create combined cert bundle', {}, error as Error);
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build subprocess environment with SSL certificate overrides for enterprise proxy compatibility.
|
||||
* If a combined cert bundle exists (Zscaler), injects SSL_CERT_FILE, REQUESTS_CA_BUNDLE, etc.
|
||||
* Otherwise returns a plain string-keyed copy of process.env.
|
||||
*/
|
||||
private getSpawnEnv(): Record<string, string> {
|
||||
const baseEnv: Record<string, string> = {};
|
||||
for (const [key, value] of Object.entries(process.env)) {
|
||||
if (value !== undefined) {
|
||||
baseEnv[key] = value;
|
||||
}
|
||||
}
|
||||
|
||||
const combinedCertPath = this.getCombinedCertPath();
|
||||
if (!combinedCertPath) {
|
||||
return baseEnv;
|
||||
}
|
||||
|
||||
logger.info('CHROMA_MCP', 'Using combined SSL certificates for enterprise compatibility', {
|
||||
certPath: combinedCertPath
|
||||
});
|
||||
|
||||
return {
|
||||
...baseEnv,
|
||||
SSL_CERT_FILE: combinedCertPath,
|
||||
REQUESTS_CA_BUNDLE: combinedCertPath,
|
||||
CURL_CA_BUNDLE: combinedCertPath,
|
||||
NODE_EXTRA_CA_CERTS: combinedCertPath
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -1,446 +0,0 @@
|
||||
/**
|
||||
* ChromaServerManager - Singleton managing local Chroma HTTP server lifecycle
|
||||
*
|
||||
* Starts a persistent Chroma server via `npx chroma run` at worker startup
|
||||
* and manages its lifecycle. In 'remote' mode, skips server start and connects
|
||||
* to an existing server (future cloud support).
|
||||
*
|
||||
* Cross-platform: Linux, macOS, Windows
|
||||
*/
|
||||
|
||||
import { spawn, ChildProcess, execSync } from 'child_process';
|
||||
import path from 'path';
|
||||
import os from 'os';
|
||||
import fs, { existsSync } from 'fs';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
|
||||
export interface ChromaServerConfig {
|
||||
dataDir: string;
|
||||
host: string;
|
||||
port: number;
|
||||
}
|
||||
|
||||
export class ChromaServerManager {
|
||||
private static instance: ChromaServerManager | null = null;
|
||||
private serverProcess: ChildProcess | null = null;
|
||||
private config: ChromaServerConfig;
|
||||
private starting: boolean = false;
|
||||
private ready: boolean = false;
|
||||
private startPromise: Promise<boolean> | null = null;
|
||||
|
||||
private constructor(config: ChromaServerConfig) {
|
||||
this.config = config;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get or create the singleton instance
|
||||
*/
|
||||
static getInstance(config?: ChromaServerConfig): ChromaServerManager {
|
||||
if (!ChromaServerManager.instance) {
|
||||
const defaultConfig: ChromaServerConfig = {
|
||||
dataDir: path.join(os.homedir(), '.claude-mem', 'vector-db'),
|
||||
host: '127.0.0.1',
|
||||
port: 8000
|
||||
};
|
||||
ChromaServerManager.instance = new ChromaServerManager(config || defaultConfig);
|
||||
}
|
||||
return ChromaServerManager.instance;
|
||||
}
|
||||
|
||||
/**
|
||||
* Start the Chroma HTTP server
|
||||
* Reuses in-flight startup if already starting
|
||||
* Spawns `npx chroma run` as a background process
|
||||
* If a server is already running (from previous worker), reuses it
|
||||
*/
|
||||
async start(timeoutMs: number = 60000): Promise<boolean> {
|
||||
if (this.ready) {
|
||||
logger.debug('CHROMA_SERVER', 'Server already started or starting', {
|
||||
ready: this.ready,
|
||||
starting: this.starting
|
||||
});
|
||||
return true;
|
||||
}
|
||||
|
||||
if (this.startPromise) {
|
||||
logger.debug('CHROMA_SERVER', 'Awaiting existing startup', {
|
||||
host: this.config.host,
|
||||
port: this.config.port
|
||||
});
|
||||
return this.startPromise;
|
||||
}
|
||||
|
||||
this.starting = true;
|
||||
this.startPromise = this.startInternal(timeoutMs);
|
||||
|
||||
try {
|
||||
return await this.startPromise;
|
||||
} finally {
|
||||
this.startPromise = null;
|
||||
if (!this.ready) {
|
||||
this.starting = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Internal startup path used behind a single shared startPromise lock
|
||||
*/
|
||||
private async startInternal(timeoutMs: number): Promise<boolean> {
|
||||
// Check if a server is already running (from previous worker or manual start)
|
||||
try {
|
||||
const response = await fetch(
|
||||
`http://${this.config.host}:${this.config.port}/api/v2/heartbeat`,
|
||||
{ signal: AbortSignal.timeout(3000) }
|
||||
);
|
||||
if (response.ok) {
|
||||
logger.info('CHROMA_SERVER', 'Existing server detected, reusing', {
|
||||
host: this.config.host,
|
||||
port: this.config.port
|
||||
});
|
||||
this.ready = true;
|
||||
this.starting = false;
|
||||
return true;
|
||||
}
|
||||
} catch {
|
||||
// No server running, proceed to start one
|
||||
}
|
||||
|
||||
// Cross-platform: use npx.cmd on Windows
|
||||
const isWindows = process.platform === 'win32';
|
||||
|
||||
// Resolve chroma binary absolutely — npx fails when spawned from cache dirs (#1120)
|
||||
let command: string;
|
||||
let args: string[];
|
||||
try {
|
||||
// chromadb package installs a 'chroma' bin entry
|
||||
const chromaBinDir = path.dirname(require.resolve('chromadb/package.json'));
|
||||
// Check project-level .bin first (most common npm/bun installation layout)
|
||||
const projectBin = path.join(chromaBinDir, '..', '.bin', isWindows ? 'chroma.cmd' : 'chroma');
|
||||
// Fallback: nested node_modules .bin (rare — pnpm or workspace hoisting)
|
||||
const nestedBin = path.join(chromaBinDir, 'node_modules', '.bin', isWindows ? 'chroma.cmd' : 'chroma');
|
||||
|
||||
if (existsSync(projectBin)) {
|
||||
command = projectBin;
|
||||
} else if (existsSync(nestedBin)) {
|
||||
command = nestedBin;
|
||||
} else {
|
||||
// Last resort: npx with explicit cwd
|
||||
command = isWindows ? 'npx.cmd' : 'npx';
|
||||
}
|
||||
} catch {
|
||||
command = isWindows ? 'npx.cmd' : 'npx';
|
||||
}
|
||||
|
||||
if (command.includes('npx')) {
|
||||
args = ['chroma', 'run', '--path', this.config.dataDir, '--host', this.config.host, '--port', String(this.config.port)];
|
||||
} else {
|
||||
args = ['run', '--path', this.config.dataDir, '--host', this.config.host, '--port', String(this.config.port)];
|
||||
}
|
||||
|
||||
logger.info('CHROMA_SERVER', 'Starting Chroma server', {
|
||||
command,
|
||||
args: args.join(' '),
|
||||
dataDir: this.config.dataDir
|
||||
});
|
||||
|
||||
const spawnEnv = this.getSpawnEnv();
|
||||
|
||||
// Resolve cwd for npx fallback — ensures node_modules is findable (#1120)
|
||||
let spawnCwd: string | undefined;
|
||||
try {
|
||||
spawnCwd = path.dirname(require.resolve('chromadb/package.json'));
|
||||
} catch {
|
||||
// If chromadb isn't resolvable, omit cwd and let npx handle it
|
||||
}
|
||||
|
||||
this.serverProcess = spawn(command, args, {
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
detached: !isWindows, // Don't detach on Windows (no process groups)
|
||||
windowsHide: true, // Hide console window on Windows
|
||||
env: spawnEnv,
|
||||
...(spawnCwd && { cwd: spawnCwd })
|
||||
});
|
||||
|
||||
// Log server output for debugging
|
||||
this.serverProcess.stdout?.on('data', (data) => {
|
||||
const msg = data.toString().trim();
|
||||
if (msg) {
|
||||
logger.debug('CHROMA_SERVER', msg);
|
||||
}
|
||||
});
|
||||
|
||||
this.serverProcess.stderr?.on('data', (data) => {
|
||||
const msg = data.toString().trim();
|
||||
if (msg) {
|
||||
// Filter out noisy startup messages
|
||||
if (!msg.includes('Chroma') || msg.includes('error') || msg.includes('Error')) {
|
||||
logger.debug('CHROMA_SERVER', msg);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
this.serverProcess.on('error', (err) => {
|
||||
logger.error('CHROMA_SERVER', 'Server process error', {}, err);
|
||||
this.ready = false;
|
||||
this.starting = false;
|
||||
});
|
||||
|
||||
this.serverProcess.on('exit', (code, signal) => {
|
||||
logger.info('CHROMA_SERVER', 'Server process exited', { code, signal });
|
||||
this.ready = false;
|
||||
this.starting = false;
|
||||
this.serverProcess = null;
|
||||
});
|
||||
|
||||
return this.waitForReady(timeoutMs);
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait for the server to become ready
|
||||
* Polls the heartbeat endpoint until success or timeout
|
||||
*/
|
||||
async waitForReady(timeoutMs: number = 60000): Promise<boolean> {
|
||||
if (this.ready) {
|
||||
return true;
|
||||
}
|
||||
|
||||
const startTime = Date.now();
|
||||
const checkInterval = 500;
|
||||
|
||||
logger.info('CHROMA_SERVER', 'Waiting for server to be ready', {
|
||||
host: this.config.host,
|
||||
port: this.config.port,
|
||||
timeoutMs
|
||||
});
|
||||
|
||||
while (Date.now() - startTime < timeoutMs) {
|
||||
try {
|
||||
const response = await fetch(
|
||||
`http://${this.config.host}:${this.config.port}/api/v2/heartbeat`
|
||||
);
|
||||
if (response.ok) {
|
||||
this.ready = true;
|
||||
this.starting = false;
|
||||
logger.info('CHROMA_SERVER', 'Server ready', {
|
||||
host: this.config.host,
|
||||
port: this.config.port,
|
||||
startupTimeMs: Date.now() - startTime
|
||||
});
|
||||
return true;
|
||||
}
|
||||
} catch {
|
||||
// Server not ready yet, continue polling
|
||||
}
|
||||
await new Promise(resolve => setTimeout(resolve, checkInterval));
|
||||
}
|
||||
|
||||
this.starting = false;
|
||||
logger.error('CHROMA_SERVER', 'Server failed to start within timeout', {
|
||||
timeoutMs,
|
||||
elapsedMs: Date.now() - startTime
|
||||
});
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if the server is running and ready
|
||||
* Returns true if we manage the process OR if a server is responding
|
||||
*/
|
||||
isRunning(): boolean {
|
||||
return this.ready;
|
||||
}
|
||||
|
||||
/**
|
||||
* Async check if server is running by pinging heartbeat
|
||||
* Use this when you need to verify server is actually reachable
|
||||
*/
|
||||
async isServerReachable(): Promise<boolean> {
|
||||
try {
|
||||
const response = await fetch(
|
||||
`http://${this.config.host}:${this.config.port}/api/v2/heartbeat`
|
||||
);
|
||||
if (response.ok) {
|
||||
this.ready = true;
|
||||
return true;
|
||||
}
|
||||
} catch {
|
||||
// Server not reachable
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the server URL for client connections
|
||||
*/
|
||||
getUrl(): string {
|
||||
return `http://${this.config.host}:${this.config.port}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the server configuration
|
||||
*/
|
||||
getConfig(): ChromaServerConfig {
|
||||
return { ...this.config };
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop the Chroma server
|
||||
* Gracefully terminates the server process
|
||||
*/
|
||||
async stop(): Promise<void> {
|
||||
if (!this.serverProcess) {
|
||||
logger.debug('CHROMA_SERVER', 'No server process to stop');
|
||||
return;
|
||||
}
|
||||
|
||||
logger.info('CHROMA_SERVER', 'Stopping server', { pid: this.serverProcess.pid });
|
||||
|
||||
return new Promise((resolve) => {
|
||||
const proc = this.serverProcess!;
|
||||
const pid = proc.pid;
|
||||
|
||||
const cleanup = () => {
|
||||
this.serverProcess = null;
|
||||
this.ready = false;
|
||||
this.starting = false;
|
||||
this.startPromise = null;
|
||||
logger.info('CHROMA_SERVER', 'Server stopped', { pid });
|
||||
resolve();
|
||||
};
|
||||
|
||||
// Set up exit handler
|
||||
proc.once('exit', cleanup);
|
||||
|
||||
// Cross-platform graceful shutdown
|
||||
if (process.platform === 'win32') {
|
||||
// Windows: just send SIGTERM
|
||||
proc.kill('SIGTERM');
|
||||
} else {
|
||||
// Unix: kill the process group to ensure all children are killed
|
||||
if (pid !== undefined) {
|
||||
try {
|
||||
process.kill(-pid, 'SIGTERM');
|
||||
} catch (err) {
|
||||
// Process group kill failed, try direct kill
|
||||
proc.kill('SIGTERM');
|
||||
}
|
||||
} else {
|
||||
proc.kill('SIGTERM');
|
||||
}
|
||||
}
|
||||
|
||||
// Force kill after timeout if still running
|
||||
setTimeout(() => {
|
||||
if (this.serverProcess) {
|
||||
logger.warn('CHROMA_SERVER', 'Force killing server after timeout', { pid });
|
||||
try {
|
||||
proc.kill('SIGKILL');
|
||||
} catch {
|
||||
// Already dead
|
||||
}
|
||||
cleanup();
|
||||
}
|
||||
}, 5000);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Get or create combined SSL certificate bundle for Zscaler/corporate proxy environments.
|
||||
* This ports previous MCP SSL handling so local `npx chroma run` works behind enterprise proxies.
|
||||
*/
|
||||
private getCombinedCertPath(): string | undefined {
|
||||
const combinedCertPath = path.join(os.homedir(), '.claude-mem', 'combined_certs.pem');
|
||||
|
||||
if (fs.existsSync(combinedCertPath)) {
|
||||
const stats = fs.statSync(combinedCertPath);
|
||||
const ageMs = Date.now() - stats.mtimeMs;
|
||||
if (ageMs < 24 * 60 * 60 * 1000) {
|
||||
return combinedCertPath;
|
||||
}
|
||||
}
|
||||
|
||||
if (process.platform !== 'darwin') {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
try {
|
||||
let certifiPath: string | undefined;
|
||||
try {
|
||||
certifiPath = execSync(
|
||||
'uvx --with certifi python -c "import certifi; print(certifi.where())"',
|
||||
{ encoding: 'utf8', stdio: ['pipe', 'pipe', 'pipe'], timeout: 10000 }
|
||||
).trim();
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
if (!certifiPath || !fs.existsSync(certifiPath)) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
let zscalerCert = '';
|
||||
try {
|
||||
zscalerCert = execSync(
|
||||
'security find-certificate -a -c "Zscaler" -p /Library/Keychains/System.keychain',
|
||||
{ encoding: 'utf8', stdio: ['pipe', 'pipe', 'pipe'], timeout: 5000 }
|
||||
);
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
if (!zscalerCert ||
|
||||
!zscalerCert.includes('-----BEGIN CERTIFICATE-----') ||
|
||||
!zscalerCert.includes('-----END CERTIFICATE-----')) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const certifiContent = fs.readFileSync(certifiPath, 'utf8');
|
||||
const tempPath = combinedCertPath + '.tmp';
|
||||
fs.writeFileSync(tempPath, certifiContent + '\n' + zscalerCert);
|
||||
fs.renameSync(tempPath, combinedCertPath);
|
||||
|
||||
logger.info('CHROMA_SERVER', 'Created combined SSL certificate bundle for Zscaler', {
|
||||
path: combinedCertPath
|
||||
});
|
||||
|
||||
return combinedCertPath;
|
||||
} catch (error) {
|
||||
logger.debug('CHROMA_SERVER', 'Could not create combined cert bundle', {}, error as Error);
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build subprocess env and preserve Zscaler compatibility from previous architecture.
|
||||
*/
|
||||
private getSpawnEnv(): NodeJS.ProcessEnv {
|
||||
const combinedCertPath = this.getCombinedCertPath();
|
||||
if (!combinedCertPath) {
|
||||
return process.env;
|
||||
}
|
||||
|
||||
logger.info('CHROMA_SERVER', 'Using combined SSL certificates for enterprise compatibility', {
|
||||
certPath: combinedCertPath
|
||||
});
|
||||
|
||||
return {
|
||||
...process.env,
|
||||
SSL_CERT_FILE: combinedCertPath,
|
||||
REQUESTS_CA_BUNDLE: combinedCertPath,
|
||||
CURL_CA_BUNDLE: combinedCertPath,
|
||||
NODE_EXTRA_CA_CERTS: combinedCertPath
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Reset the singleton instance (for testing)
|
||||
*/
|
||||
static reset(): void {
|
||||
if (ChromaServerManager.instance) {
|
||||
// Don't await - just trigger stop
|
||||
ChromaServerManager.instance.stop().catch(() => {});
|
||||
}
|
||||
ChromaServerManager.instance = null;
|
||||
}
|
||||
}
|
||||
+196
-245
@@ -1,26 +1,21 @@
|
||||
/**
|
||||
* ChromaSync Service
|
||||
*
|
||||
* Automatically syncs observations and session summaries to ChromaDB via HTTP.
|
||||
* Automatically syncs observations and session summaries to ChromaDB via MCP.
|
||||
* This service provides real-time semantic search capabilities by maintaining
|
||||
* a vector database synchronized with SQLite.
|
||||
*
|
||||
* Uses the chromadb npm package's built-in ChromaClient for HTTP connections.
|
||||
* Supports both local server (managed by ChromaServerManager) and remote/cloud
|
||||
* servers for future claude-mem pro features.
|
||||
* Uses ChromaMcpManager to communicate with chroma-mcp over stdio MCP protocol.
|
||||
* The chroma-mcp server handles its own embedding and persistent storage,
|
||||
* eliminating the need for chromadb npm package and ONNX/WASM dependencies.
|
||||
*
|
||||
* Design: Fail-fast with no fallbacks - if Chroma is unavailable, syncing fails.
|
||||
*/
|
||||
|
||||
import { ChromaClient, Collection } from 'chromadb';
|
||||
import { ChromaMcpManager } from './ChromaMcpManager.js';
|
||||
import { ParsedObservation, ParsedSummary } from '../../sdk/parser.js';
|
||||
import { SessionStore } from '../sqlite/SessionStore.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
import { SettingsDefaultsManager } from '../../shared/SettingsDefaultsManager.js';
|
||||
import { USER_SETTINGS_PATH } from '../../shared/paths.js';
|
||||
import { ChromaServerManager } from './ChromaServerManager.js';
|
||||
import path from 'path';
|
||||
import os from 'os';
|
||||
|
||||
interface ChromaDocument {
|
||||
id: string;
|
||||
@@ -75,138 +70,49 @@ interface StoredUserPrompt {
|
||||
}
|
||||
|
||||
export class ChromaSync {
|
||||
private chromaClient: ChromaClient | null = null;
|
||||
private collection: Collection | null = null;
|
||||
private project: string;
|
||||
private collectionName: string;
|
||||
private readonly VECTOR_DB_DIR: string;
|
||||
private collectionCreated = false;
|
||||
private readonly BATCH_SIZE = 100;
|
||||
|
||||
constructor(project: string) {
|
||||
this.project = project;
|
||||
this.collectionName = `cm__${project}`;
|
||||
this.VECTOR_DB_DIR = path.join(os.homedir(), '.claude-mem', 'vector-db');
|
||||
// Chroma collection names only allow [a-zA-Z0-9._-], 3-512 chars,
|
||||
// must start/end with [a-zA-Z0-9]
|
||||
const sanitized = project
|
||||
.replace(/[^a-zA-Z0-9._-]/g, '_')
|
||||
.replace(/[^a-zA-Z0-9]+$/, ''); // strip trailing non-alphanumeric
|
||||
this.collectionName = `cm__${sanitized || 'unknown'}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure HTTP client is connected to Chroma server
|
||||
* In local mode, verifies ChromaServerManager has started the server
|
||||
* In remote mode, connects directly to configured host
|
||||
* Throws error if connection fails
|
||||
* Ensure collection exists in Chroma via MCP.
|
||||
* chroma_create_collection is idempotent - safe to call multiple times.
|
||||
* Uses collectionCreated flag to avoid redundant calls within a session.
|
||||
*/
|
||||
private async ensureConnection(): Promise<void> {
|
||||
if (this.chromaClient) {
|
||||
private async ensureCollectionExists(): Promise<void> {
|
||||
if (this.collectionCreated) {
|
||||
return;
|
||||
}
|
||||
|
||||
logger.info('CHROMA_SYNC', 'Connecting to Chroma HTTP server...', { project: this.project });
|
||||
|
||||
const chromaMcp = ChromaMcpManager.getInstance();
|
||||
try {
|
||||
const settings = SettingsDefaultsManager.loadFromFile(USER_SETTINGS_PATH);
|
||||
const mode = settings.CLAUDE_MEM_CHROMA_MODE || 'local';
|
||||
const host = settings.CLAUDE_MEM_CHROMA_HOST || '127.0.0.1';
|
||||
const port = parseInt(settings.CLAUDE_MEM_CHROMA_PORT || '8000', 10);
|
||||
const ssl = settings.CLAUDE_MEM_CHROMA_SSL === 'true';
|
||||
|
||||
// Multi-tenancy settings (used in remote/pro mode)
|
||||
const tenant = settings.CLAUDE_MEM_CHROMA_TENANT || 'default_tenant';
|
||||
const database = settings.CLAUDE_MEM_CHROMA_DATABASE || 'default_database';
|
||||
const apiKey = settings.CLAUDE_MEM_CHROMA_API_KEY || '';
|
||||
|
||||
// In local mode, verify server is reachable
|
||||
if (mode === 'local') {
|
||||
const serverManager = ChromaServerManager.getInstance();
|
||||
const reachable = await serverManager.isServerReachable();
|
||||
if (!reachable) {
|
||||
throw new Error('Chroma server not reachable. Ensure worker started correctly.');
|
||||
}
|
||||
}
|
||||
|
||||
// Create HTTP client
|
||||
const protocol = ssl ? 'https' : 'http';
|
||||
const chromaPath = `${protocol}://${host}:${port}`;
|
||||
|
||||
// Build client options
|
||||
const clientOptions: { path: string; tenant?: string; database?: string; headers?: Record<string, string> } = {
|
||||
path: chromaPath
|
||||
};
|
||||
|
||||
// In remote mode, use tenant isolation for pro users
|
||||
if (mode === 'remote') {
|
||||
clientOptions.tenant = tenant;
|
||||
clientOptions.database = database;
|
||||
|
||||
// Add API key header if configured
|
||||
if (apiKey) {
|
||||
clientOptions.headers = {
|
||||
'Authorization': `Bearer ${apiKey}`
|
||||
};
|
||||
}
|
||||
|
||||
logger.info('CHROMA_SYNC', 'Connecting with tenant isolation', {
|
||||
tenant,
|
||||
database,
|
||||
hasApiKey: !!apiKey
|
||||
});
|
||||
}
|
||||
|
||||
this.chromaClient = new ChromaClient(clientOptions);
|
||||
|
||||
// Verify connection with heartbeat
|
||||
await this.chromaClient.heartbeat();
|
||||
|
||||
logger.info('CHROMA_SYNC', 'Connected to Chroma HTTP server', {
|
||||
project: this.project,
|
||||
host,
|
||||
port,
|
||||
ssl,
|
||||
mode,
|
||||
tenant: mode === 'remote' ? tenant : 'default_tenant'
|
||||
await chromaMcp.callTool('chroma_create_collection', {
|
||||
collection_name: this.collectionName
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('CHROMA_SYNC', 'Failed to connect to Chroma HTTP server', { project: this.project }, error as Error);
|
||||
this.chromaClient = null;
|
||||
throw new Error(`Chroma connection failed: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure collection exists, create if needed
|
||||
* Throws error if collection creation fails
|
||||
*/
|
||||
private async ensureCollection(): Promise<void> {
|
||||
await this.ensureConnection();
|
||||
|
||||
if (this.collection) {
|
||||
return;
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
if (!message.includes('already exists')) {
|
||||
throw error;
|
||||
}
|
||||
// Collection already exists - this is the expected path after first creation
|
||||
}
|
||||
|
||||
if (!this.chromaClient) {
|
||||
throw new Error(
|
||||
'Chroma client not initialized. Call ensureConnection() before using client methods.' +
|
||||
` Project: ${this.project}`
|
||||
);
|
||||
}
|
||||
this.collectionCreated = true;
|
||||
|
||||
try {
|
||||
// Use WASM backend to avoid native ONNX binary issues (#1104, #1105, #1110).
|
||||
// Same model (all-MiniLM-L6-v2), same embeddings, but runs in WASM —
|
||||
// no native binary loading, no segfaults, no ENOENT errors.
|
||||
const { DefaultEmbeddingFunction } = await import('@chroma-core/default-embed');
|
||||
const embeddingFunction = new DefaultEmbeddingFunction({ wasm: true });
|
||||
|
||||
this.collection = await this.chromaClient.getOrCreateCollection({
|
||||
name: this.collectionName,
|
||||
embeddingFunction
|
||||
});
|
||||
|
||||
logger.debug('CHROMA_SYNC', 'Collection ready', {
|
||||
collection: this.collectionName
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('CHROMA_SYNC', 'Failed to get/create collection', { collection: this.collectionName }, error as Error);
|
||||
throw new Error(`Collection setup failed: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
logger.debug('CHROMA_SYNC', 'Collection ready', {
|
||||
collection: this.collectionName
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -345,7 +251,7 @@ export class ChromaSync {
|
||||
}
|
||||
|
||||
/**
|
||||
* Add documents to Chroma in batch
|
||||
* Add documents to Chroma in batch via MCP
|
||||
* Throws error if batch add fails
|
||||
*/
|
||||
private async addDocuments(documents: ChromaDocument[]): Promise<void> {
|
||||
@@ -353,33 +259,42 @@ export class ChromaSync {
|
||||
return;
|
||||
}
|
||||
|
||||
await this.ensureCollection();
|
||||
await this.ensureCollectionExists();
|
||||
|
||||
if (!this.collection) {
|
||||
throw new Error(
|
||||
'Chroma collection not initialized. Call ensureCollection() before using collection methods.' +
|
||||
` Project: ${this.project}`
|
||||
const chromaMcp = ChromaMcpManager.getInstance();
|
||||
|
||||
// Add in batches
|
||||
for (let i = 0; i < documents.length; i += this.BATCH_SIZE) {
|
||||
const batch = documents.slice(i, i + this.BATCH_SIZE);
|
||||
|
||||
// Sanitize metadata: filter out null, undefined, and empty string values
|
||||
// that chroma-mcp may reject (e.g., null subtitle from raw SQLite rows)
|
||||
const cleanMetadatas = batch.map(d =>
|
||||
Object.fromEntries(
|
||||
Object.entries(d.metadata).filter(([_, v]) => v !== null && v !== undefined && v !== '')
|
||||
)
|
||||
);
|
||||
|
||||
try {
|
||||
await chromaMcp.callTool('chroma_add_documents', {
|
||||
collection_name: this.collectionName,
|
||||
ids: batch.map(d => d.id),
|
||||
documents: batch.map(d => d.document),
|
||||
metadatas: cleanMetadatas
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('CHROMA_SYNC', 'Batch add failed, continuing with remaining batches', {
|
||||
collection: this.collectionName,
|
||||
batchStart: i,
|
||||
batchSize: batch.length
|
||||
}, error as Error);
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
await this.collection.add({
|
||||
ids: documents.map(d => d.id),
|
||||
documents: documents.map(d => d.document),
|
||||
metadatas: documents.map(d => d.metadata)
|
||||
});
|
||||
|
||||
logger.debug('CHROMA_SYNC', 'Documents added', {
|
||||
collection: this.collectionName,
|
||||
count: documents.length
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('CHROMA_SYNC', 'Failed to add documents', {
|
||||
collection: this.collectionName,
|
||||
count: documents.length
|
||||
}, error as Error);
|
||||
throw new Error(`Document add failed: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
logger.debug('CHROMA_SYNC', 'Documents added', {
|
||||
collection: this.collectionName,
|
||||
count: documents.length
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -521,22 +436,18 @@ export class ChromaSync {
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch all existing document IDs from Chroma collection
|
||||
* Fetch all existing document IDs from Chroma collection via MCP
|
||||
* Returns Sets of SQLite IDs for observations, summaries, and prompts
|
||||
*/
|
||||
private async getExistingChromaIds(): Promise<{
|
||||
private async getExistingChromaIds(projectOverride?: string): Promise<{
|
||||
observations: Set<number>;
|
||||
summaries: Set<number>;
|
||||
prompts: Set<number>;
|
||||
}> {
|
||||
await this.ensureCollection();
|
||||
const targetProject = projectOverride ?? this.project;
|
||||
await this.ensureCollectionExists();
|
||||
|
||||
if (!this.collection) {
|
||||
throw new Error(
|
||||
'Chroma collection not initialized. Call ensureCollection() before using collection methods.' +
|
||||
` Project: ${this.project}`
|
||||
);
|
||||
}
|
||||
const chromaMcp = ChromaMcpManager.getInstance();
|
||||
|
||||
const observationIds = new Set<number>();
|
||||
const summaryIds = new Set<number>();
|
||||
@@ -545,52 +456,49 @@ export class ChromaSync {
|
||||
let offset = 0;
|
||||
const limit = 1000; // Large batches, metadata only = fast
|
||||
|
||||
logger.info('CHROMA_SYNC', 'Fetching existing Chroma document IDs...', { project: this.project });
|
||||
logger.info('CHROMA_SYNC', 'Fetching existing Chroma document IDs...', { project: targetProject });
|
||||
|
||||
while (true) {
|
||||
try {
|
||||
const result = await this.collection.get({
|
||||
limit,
|
||||
offset,
|
||||
where: { project: this.project },
|
||||
include: ['metadatas']
|
||||
});
|
||||
const result = await chromaMcp.callTool('chroma_get_documents', {
|
||||
collection_name: this.collectionName,
|
||||
limit: limit,
|
||||
offset: offset,
|
||||
where: { project: targetProject },
|
||||
include: ['metadatas']
|
||||
}) as any;
|
||||
|
||||
const metadatas = result.metadatas || [];
|
||||
// chroma_get_documents returns flat arrays: { ids, metadatas, documents }
|
||||
const metadatas = result?.metadatas || [];
|
||||
|
||||
if (metadatas.length === 0) {
|
||||
break; // No more documents
|
||||
}
|
||||
if (metadatas.length === 0) {
|
||||
break; // No more documents
|
||||
}
|
||||
|
||||
// Extract SQLite IDs from metadata
|
||||
for (const meta of metadatas) {
|
||||
if (meta && meta.sqlite_id) {
|
||||
const sqliteId = meta.sqlite_id as number;
|
||||
if (meta.doc_type === 'observation') {
|
||||
observationIds.add(sqliteId);
|
||||
} else if (meta.doc_type === 'session_summary') {
|
||||
summaryIds.add(sqliteId);
|
||||
} else if (meta.doc_type === 'user_prompt') {
|
||||
promptIds.add(sqliteId);
|
||||
}
|
||||
// Extract SQLite IDs from metadata
|
||||
for (const meta of metadatas) {
|
||||
if (meta && meta.sqlite_id) {
|
||||
const sqliteId = meta.sqlite_id as number;
|
||||
if (meta.doc_type === 'observation') {
|
||||
observationIds.add(sqliteId);
|
||||
} else if (meta.doc_type === 'session_summary') {
|
||||
summaryIds.add(sqliteId);
|
||||
} else if (meta.doc_type === 'user_prompt') {
|
||||
promptIds.add(sqliteId);
|
||||
}
|
||||
}
|
||||
|
||||
offset += limit;
|
||||
|
||||
logger.debug('CHROMA_SYNC', 'Fetched batch of existing IDs', {
|
||||
project: this.project,
|
||||
offset,
|
||||
batchSize: metadatas.length
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('CHROMA_SYNC', 'Failed to fetch existing IDs', { project: this.project }, error as Error);
|
||||
throw error;
|
||||
}
|
||||
|
||||
offset += limit;
|
||||
|
||||
logger.debug('CHROMA_SYNC', 'Fetched batch of existing IDs', {
|
||||
project: targetProject,
|
||||
offset,
|
||||
batchSize: metadatas.length
|
||||
});
|
||||
}
|
||||
|
||||
logger.info('CHROMA_SYNC', 'Existing IDs fetched', {
|
||||
project: this.project,
|
||||
project: targetProject,
|
||||
observations: observationIds.size,
|
||||
summaries: summaryIds.size,
|
||||
prompts: promptIds.size
|
||||
@@ -602,21 +510,25 @@ export class ChromaSync {
|
||||
/**
|
||||
* Backfill: Sync all observations missing from Chroma
|
||||
* Reads from SQLite and syncs in batches
|
||||
* @param projectOverride - If provided, backfill this project instead of this.project.
|
||||
* Used by backfillAllProjects() to iterate projects without mutating instance state.
|
||||
* Throws error if backfill fails
|
||||
*/
|
||||
async ensureBackfilled(): Promise<void> {
|
||||
logger.info('CHROMA_SYNC', 'Starting smart backfill', { project: this.project });
|
||||
async ensureBackfilled(projectOverride?: string): Promise<void> {
|
||||
const backfillProject = projectOverride ?? this.project;
|
||||
logger.info('CHROMA_SYNC', 'Starting smart backfill', { project: backfillProject });
|
||||
|
||||
await this.ensureCollection();
|
||||
await this.ensureCollectionExists();
|
||||
|
||||
// Fetch existing IDs from Chroma (fast, metadata only)
|
||||
const existing = await this.getExistingChromaIds();
|
||||
const existing = await this.getExistingChromaIds(backfillProject);
|
||||
|
||||
const db = new SessionStore();
|
||||
|
||||
try {
|
||||
// Build exclusion list for observations
|
||||
const existingObsIds = Array.from(existing.observations);
|
||||
// Filter to validated positive integers before interpolating into SQL
|
||||
const existingObsIds = Array.from(existing.observations).filter(id => Number.isInteger(id) && id > 0);
|
||||
const obsExclusionClause = existingObsIds.length > 0
|
||||
? `AND id NOT IN (${existingObsIds.join(',')})`
|
||||
: '';
|
||||
@@ -626,14 +538,14 @@ export class ChromaSync {
|
||||
SELECT * FROM observations
|
||||
WHERE project = ? ${obsExclusionClause}
|
||||
ORDER BY id ASC
|
||||
`).all(this.project) as StoredObservation[];
|
||||
`).all(backfillProject) as StoredObservation[];
|
||||
|
||||
const totalObsCount = db.db.prepare(`
|
||||
SELECT COUNT(*) as count FROM observations WHERE project = ?
|
||||
`).get(this.project) as { count: number };
|
||||
`).get(backfillProject) as { count: number };
|
||||
|
||||
logger.info('CHROMA_SYNC', 'Backfilling observations', {
|
||||
project: this.project,
|
||||
project: backfillProject,
|
||||
missing: observations.length,
|
||||
existing: existing.observations.size,
|
||||
total: totalObsCount.count
|
||||
@@ -651,13 +563,13 @@ export class ChromaSync {
|
||||
await this.addDocuments(batch);
|
||||
|
||||
logger.debug('CHROMA_SYNC', 'Backfill progress', {
|
||||
project: this.project,
|
||||
project: backfillProject,
|
||||
progress: `${Math.min(i + this.BATCH_SIZE, allDocs.length)}/${allDocs.length}`
|
||||
});
|
||||
}
|
||||
|
||||
// Build exclusion list for summaries
|
||||
const existingSummaryIds = Array.from(existing.summaries);
|
||||
const existingSummaryIds = Array.from(existing.summaries).filter(id => Number.isInteger(id) && id > 0);
|
||||
const summaryExclusionClause = existingSummaryIds.length > 0
|
||||
? `AND id NOT IN (${existingSummaryIds.join(',')})`
|
||||
: '';
|
||||
@@ -667,14 +579,14 @@ export class ChromaSync {
|
||||
SELECT * FROM session_summaries
|
||||
WHERE project = ? ${summaryExclusionClause}
|
||||
ORDER BY id ASC
|
||||
`).all(this.project) as StoredSummary[];
|
||||
`).all(backfillProject) as StoredSummary[];
|
||||
|
||||
const totalSummaryCount = db.db.prepare(`
|
||||
SELECT COUNT(*) as count FROM session_summaries WHERE project = ?
|
||||
`).get(this.project) as { count: number };
|
||||
`).get(backfillProject) as { count: number };
|
||||
|
||||
logger.info('CHROMA_SYNC', 'Backfilling summaries', {
|
||||
project: this.project,
|
||||
project: backfillProject,
|
||||
missing: summaries.length,
|
||||
existing: existing.summaries.size,
|
||||
total: totalSummaryCount.count
|
||||
@@ -692,13 +604,13 @@ export class ChromaSync {
|
||||
await this.addDocuments(batch);
|
||||
|
||||
logger.debug('CHROMA_SYNC', 'Backfill progress', {
|
||||
project: this.project,
|
||||
project: backfillProject,
|
||||
progress: `${Math.min(i + this.BATCH_SIZE, summaryDocs.length)}/${summaryDocs.length}`
|
||||
});
|
||||
}
|
||||
|
||||
// Build exclusion list for prompts
|
||||
const existingPromptIds = Array.from(existing.prompts);
|
||||
const existingPromptIds = Array.from(existing.prompts).filter(id => Number.isInteger(id) && id > 0);
|
||||
const promptExclusionClause = existingPromptIds.length > 0
|
||||
? `AND up.id NOT IN (${existingPromptIds.join(',')})`
|
||||
: '';
|
||||
@@ -713,17 +625,17 @@ export class ChromaSync {
|
||||
JOIN sdk_sessions s ON up.content_session_id = s.content_session_id
|
||||
WHERE s.project = ? ${promptExclusionClause}
|
||||
ORDER BY up.id ASC
|
||||
`).all(this.project) as StoredUserPrompt[];
|
||||
`).all(backfillProject) as StoredUserPrompt[];
|
||||
|
||||
const totalPromptCount = db.db.prepare(`
|
||||
SELECT COUNT(*) as count
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.content_session_id = s.content_session_id
|
||||
WHERE s.project = ?
|
||||
`).get(this.project) as { count: number };
|
||||
`).get(backfillProject) as { count: number };
|
||||
|
||||
logger.info('CHROMA_SYNC', 'Backfilling user prompts', {
|
||||
project: this.project,
|
||||
project: backfillProject,
|
||||
missing: prompts.length,
|
||||
existing: existing.prompts.size,
|
||||
total: totalPromptCount.count
|
||||
@@ -741,13 +653,13 @@ export class ChromaSync {
|
||||
await this.addDocuments(batch);
|
||||
|
||||
logger.debug('CHROMA_SYNC', 'Backfill progress', {
|
||||
project: this.project,
|
||||
project: backfillProject,
|
||||
progress: `${Math.min(i + this.BATCH_SIZE, promptDocs.length)}/${promptDocs.length}`
|
||||
});
|
||||
}
|
||||
|
||||
logger.info('CHROMA_SYNC', 'Smart backfill complete', {
|
||||
project: this.project,
|
||||
project: backfillProject,
|
||||
synced: {
|
||||
observationDocs: allDocs.length,
|
||||
summaryDocs: summaryDocs.length,
|
||||
@@ -761,7 +673,7 @@ export class ChromaSync {
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
logger.error('CHROMA_SYNC', 'Backfill failed', { project: this.project }, error as Error);
|
||||
logger.error('CHROMA_SYNC', 'Backfill failed', { project: backfillProject }, error as Error);
|
||||
throw new Error(`Backfill failed: ${error instanceof Error ? error.message : String(error)}`);
|
||||
} finally {
|
||||
db.close();
|
||||
@@ -769,7 +681,7 @@ export class ChromaSync {
|
||||
}
|
||||
|
||||
/**
|
||||
* Query Chroma collection for semantic search
|
||||
* Query Chroma collection for semantic search via MCP
|
||||
* Used by SearchManager for vector-based search
|
||||
*/
|
||||
async queryChroma(
|
||||
@@ -777,27 +689,34 @@ export class ChromaSync {
|
||||
limit: number,
|
||||
whereFilter?: Record<string, any>
|
||||
): Promise<{ ids: number[]; distances: number[]; metadatas: any[] }> {
|
||||
await this.ensureCollection();
|
||||
|
||||
if (!this.collection) {
|
||||
throw new Error(
|
||||
'Chroma collection not initialized. Call ensureCollection() before using collection methods.' +
|
||||
` Project: ${this.project}`
|
||||
);
|
||||
}
|
||||
await this.ensureCollectionExists();
|
||||
|
||||
try {
|
||||
const results = await this.collection.query({
|
||||
queryTexts: [query],
|
||||
nResults: limit,
|
||||
where: whereFilter,
|
||||
const chromaMcp = ChromaMcpManager.getInstance();
|
||||
const results = await chromaMcp.callTool('chroma_query_documents', {
|
||||
collection_name: this.collectionName,
|
||||
query_texts: [query],
|
||||
n_results: limit,
|
||||
...(whereFilter && { where: whereFilter }),
|
||||
include: ['documents', 'metadatas', 'distances']
|
||||
});
|
||||
}) as any;
|
||||
|
||||
// Extract unique SQLite IDs from document IDs
|
||||
// chroma_query_documents returns nested arrays (one per query text)
|
||||
// We always pass a single query text, so we access [0]
|
||||
const ids: number[] = [];
|
||||
const docIds = results.ids?.[0] || [];
|
||||
for (const docId of docIds) {
|
||||
const seen = new Set<number>();
|
||||
const docIds = results?.ids?.[0] || [];
|
||||
const rawMetadatas = results?.metadatas?.[0] || [];
|
||||
const rawDistances = results?.distances?.[0] || [];
|
||||
|
||||
// Build deduplicated arrays that stay index-aligned:
|
||||
// Multiple Chroma docs map to the same SQLite ID (one per field).
|
||||
// Keep the first (best-ranked) distance and metadata per SQLite ID.
|
||||
const metadatas: any[] = [];
|
||||
const distances: number[] = [];
|
||||
|
||||
for (let i = 0; i < docIds.length; i++) {
|
||||
const docId = docIds[i];
|
||||
// Extract sqlite_id from document ID (supports three formats):
|
||||
// - obs_{id}_narrative, obs_{id}_fact_0, etc (observations)
|
||||
// - summary_{id}_request, summary_{id}_learned, etc (session summaries)
|
||||
@@ -815,16 +734,15 @@ export class ChromaSync {
|
||||
sqliteId = parseInt(promptMatch[1], 10);
|
||||
}
|
||||
|
||||
if (sqliteId !== null && !ids.includes(sqliteId)) {
|
||||
if (sqliteId !== null && !seen.has(sqliteId)) {
|
||||
seen.add(sqliteId);
|
||||
ids.push(sqliteId);
|
||||
metadatas.push(rawMetadatas[i] ?? null);
|
||||
distances.push(rawDistances[i] ?? 0);
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
ids,
|
||||
distances: results.distances?.[0] || [],
|
||||
metadatas: results.metadatas?.[0] || []
|
||||
};
|
||||
return { ids, distances, metadatas };
|
||||
} catch (error) {
|
||||
const errorMessage = error instanceof Error ? error.message : String(error);
|
||||
|
||||
@@ -832,12 +750,13 @@ export class ChromaSync {
|
||||
const isConnectionError =
|
||||
errorMessage.includes('ECONNREFUSED') ||
|
||||
errorMessage.includes('ENOTFOUND') ||
|
||||
errorMessage.includes('fetch failed');
|
||||
errorMessage.includes('fetch failed') ||
|
||||
errorMessage.includes('subprocess closed') ||
|
||||
errorMessage.includes('timed out');
|
||||
|
||||
if (isConnectionError) {
|
||||
// Reset connection state so next call attempts reconnect
|
||||
this.chromaClient = null;
|
||||
this.collection = null;
|
||||
// Reset collection state so next call attempts reconnect
|
||||
this.collectionCreated = false;
|
||||
logger.error('CHROMA_SYNC', 'Connection lost during query',
|
||||
{ project: this.project, query }, error as Error);
|
||||
throw new Error(`Chroma query failed - connection lost: ${errorMessage}`);
|
||||
@@ -849,13 +768,45 @@ export class ChromaSync {
|
||||
}
|
||||
|
||||
/**
|
||||
* Close the Chroma client connection
|
||||
* Server lifecycle is managed by ChromaServerManager, not here
|
||||
* Backfill all projects that have observations in SQLite but may be missing from Chroma.
|
||||
* Uses a single shared ChromaSync('claude-mem') instance and Chroma connection.
|
||||
* Per-project scoping is passed as a parameter to ensureBackfilled(), avoiding
|
||||
* instance state mutation. All documents land in the cm__claude-mem collection
|
||||
* with project scoped via metadata, matching how DatabaseManager and SearchManager operate.
|
||||
* Designed to be called fire-and-forget on worker startup.
|
||||
*/
|
||||
static async backfillAllProjects(): Promise<void> {
|
||||
const db = new SessionStore();
|
||||
const sync = new ChromaSync('claude-mem');
|
||||
try {
|
||||
const projects = db.db.prepare(
|
||||
'SELECT DISTINCT project FROM observations WHERE project IS NOT NULL AND project != ?'
|
||||
).all('') as { project: string }[];
|
||||
|
||||
logger.info('CHROMA_SYNC', `Backfill check for ${projects.length} projects`);
|
||||
|
||||
for (const { project } of projects) {
|
||||
try {
|
||||
await sync.ensureBackfilled(project);
|
||||
} catch (error) {
|
||||
logger.error('CHROMA_SYNC', `Backfill failed for project: ${project}`, {}, error as Error);
|
||||
// Continue to next project — don't let one failure stop others
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
await sync.close();
|
||||
db.close();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Close the ChromaSync instance
|
||||
* ChromaMcpManager is a singleton and manages its own lifecycle
|
||||
* We don't close it here - it's closed during graceful shutdown
|
||||
*/
|
||||
async close(): Promise<void> {
|
||||
// Just clear references - server lifecycle managed by ChromaServerManager
|
||||
this.chromaClient = null;
|
||||
this.collection = null;
|
||||
logger.info('CHROMA_SYNC', 'Chroma client closed', { project: this.project });
|
||||
// ChromaMcpManager is a singleton and manages its own lifecycle
|
||||
// We don't close it here - it's closed during graceful shutdown
|
||||
logger.info('CHROMA_SYNC', 'ChromaSync closed', { project: this.project });
|
||||
}
|
||||
}
|
||||
|
||||
+147
-37
@@ -18,7 +18,8 @@ import { HOOK_TIMEOUTS } from '../shared/hook-constants.js';
|
||||
import { SettingsDefaultsManager } from '../shared/SettingsDefaultsManager.js';
|
||||
import { getAuthMethodDescription } from '../shared/EnvManager.js';
|
||||
import { logger } from '../utils/logger.js';
|
||||
import { ChromaServerManager } from './sync/ChromaServerManager.js';
|
||||
import { ChromaMcpManager } from './sync/ChromaMcpManager.js';
|
||||
import { ChromaSync } from './sync/ChromaSync.js';
|
||||
|
||||
// Windows: avoid repeated spawn popups when startup fails (issue #921)
|
||||
const WINDOWS_SPAWN_COOLDOWN_MS = 2 * 60 * 1000;
|
||||
@@ -58,6 +59,10 @@ function clearWorkerSpawnAttempted(): void {
|
||||
}
|
||||
}
|
||||
|
||||
// Re-export for backward compatibility — canonical implementation in shared/plugin-state.ts
|
||||
export { isPluginDisabledInClaudeSettings } from '../shared/plugin-state.js';
|
||||
import { isPluginDisabledInClaudeSettings } from '../shared/plugin-state.js';
|
||||
|
||||
// Version injected at build time by esbuild define
|
||||
declare const __DEFAULT_PACKAGE_VERSION__: string;
|
||||
const packageVersion = typeof __DEFAULT_PACKAGE_VERSION__ !== 'undefined' ? __DEFAULT_PACKAGE_VERSION__ : '0.0.0-dev';
|
||||
@@ -68,14 +73,19 @@ import {
|
||||
readPidFile,
|
||||
removePidFile,
|
||||
getPlatformTimeout,
|
||||
cleanupOrphanedProcesses,
|
||||
aggressiveStartupCleanup,
|
||||
runOneTimeChromaMigration,
|
||||
cleanStalePidFile,
|
||||
isProcessAlive,
|
||||
spawnDaemon,
|
||||
createSignalHandler
|
||||
createSignalHandler,
|
||||
isPidFileRecent,
|
||||
touchPidFile
|
||||
} from './infrastructure/ProcessManager.js';
|
||||
import {
|
||||
isPortInUse,
|
||||
waitForHealth,
|
||||
waitForReadiness,
|
||||
waitForPortFree,
|
||||
httpShutdown,
|
||||
checkVersionMatch
|
||||
@@ -115,7 +125,7 @@ import { LogsRoutes } from './worker/http/routes/LogsRoutes.js';
|
||||
import { MemoryRoutes } from './worker/http/routes/MemoryRoutes.js';
|
||||
|
||||
// Process management for zombie cleanup (Issue #737)
|
||||
import { startOrphanReaper, reapOrphanedProcesses } from './worker/ProcessRegistry.js';
|
||||
import { startOrphanReaper, reapOrphanedProcesses, getProcessBySession, ensureProcessExit } from './worker/ProcessRegistry.js';
|
||||
|
||||
/**
|
||||
* Build JSON status output for hook framework communication.
|
||||
@@ -165,8 +175,8 @@ export class WorkerService {
|
||||
// Route handlers
|
||||
private searchRoutes: SearchRoutes | null = null;
|
||||
|
||||
// Chroma server (local mode)
|
||||
private chromaServer: ChromaServerManager | null = null;
|
||||
// Chroma MCP manager (lazy - connects on first use)
|
||||
private chromaMcpManager: ChromaMcpManager | null = null;
|
||||
|
||||
// Initialization tracking
|
||||
private initializationComplete: Promise<void>;
|
||||
@@ -175,6 +185,9 @@ export class WorkerService {
|
||||
// Orphan reaper cleanup function (Issue #737)
|
||||
private stopOrphanReaper: (() => void) | null = null;
|
||||
|
||||
// Stale session reaper interval (Issue #1168)
|
||||
private staleSessionReaperInterval: ReturnType<typeof setInterval> | null = null;
|
||||
|
||||
// AI interaction tracking for health endpoint
|
||||
private lastAiInteraction: {
|
||||
timestamp: number;
|
||||
@@ -363,36 +376,28 @@ export class WorkerService {
|
||||
*/
|
||||
private async initializeBackground(): Promise<void> {
|
||||
try {
|
||||
await cleanupOrphanedProcesses();
|
||||
await aggressiveStartupCleanup();
|
||||
|
||||
// Load mode configuration
|
||||
const { ModeManager } = await import('./domain/ModeManager.js');
|
||||
const { SettingsDefaultsManager } = await import('../shared/SettingsDefaultsManager.js');
|
||||
const { USER_SETTINGS_PATH } = await import('../shared/paths.js');
|
||||
const os = await import('os');
|
||||
|
||||
const settings = SettingsDefaultsManager.loadFromFile(USER_SETTINGS_PATH);
|
||||
|
||||
// Start Chroma server if in local mode
|
||||
const chromaMode = settings.CLAUDE_MEM_CHROMA_MODE || 'local';
|
||||
if (chromaMode === 'local') {
|
||||
logger.info('SYSTEM', 'Starting local Chroma server...');
|
||||
this.chromaServer = ChromaServerManager.getInstance({
|
||||
dataDir: path.join(os.homedir(), '.claude-mem', 'vector-db'),
|
||||
host: settings.CLAUDE_MEM_CHROMA_HOST || '127.0.0.1',
|
||||
port: parseInt(settings.CLAUDE_MEM_CHROMA_PORT || '8000', 10)
|
||||
});
|
||||
// One-time chroma wipe for users upgrading from versions with duplicate worker bugs.
|
||||
// Only runs in local mode (chroma is local-only). Backfill at line ~414 rebuilds from SQLite.
|
||||
if (settings.CLAUDE_MEM_MODE === 'local' || !settings.CLAUDE_MEM_MODE) {
|
||||
runOneTimeChromaMigration();
|
||||
}
|
||||
|
||||
const ready = await this.chromaServer.start(60000);
|
||||
|
||||
if (ready) {
|
||||
logger.success('SYSTEM', 'Chroma server ready');
|
||||
} else {
|
||||
logger.warn('SYSTEM', 'Chroma server failed to start - vector search disabled');
|
||||
this.chromaServer = null;
|
||||
}
|
||||
// Initialize ChromaMcpManager only if Chroma is enabled
|
||||
const chromaEnabled = settings.CLAUDE_MEM_CHROMA_ENABLED !== 'false';
|
||||
if (chromaEnabled) {
|
||||
this.chromaMcpManager = ChromaMcpManager.getInstance();
|
||||
logger.info('SYSTEM', 'ChromaMcpManager initialized (lazy - connects on first use)');
|
||||
} else {
|
||||
logger.info('SYSTEM', 'Chroma remote mode - skipping local server');
|
||||
logger.info('SYSTEM', 'Chroma disabled via CLAUDE_MEM_CHROMA_ENABLED=false, skipping ChromaMcpManager');
|
||||
}
|
||||
|
||||
const modeId = settings.CLAUDE_MEM_MODE;
|
||||
@@ -423,6 +428,22 @@ export class WorkerService {
|
||||
this.server.registerRoutes(this.searchRoutes);
|
||||
logger.info('WORKER', 'SearchManager initialized and search routes registered');
|
||||
|
||||
// DB and search are ready — mark initialization complete so hooks can proceed.
|
||||
// MCP connection is tracked separately via mcpReady and is NOT required for
|
||||
// the worker to serve context/search requests.
|
||||
this.initializationCompleteFlag = true;
|
||||
this.resolveInitialization();
|
||||
logger.info('SYSTEM', 'Core initialization complete (DB + search ready)');
|
||||
|
||||
// Auto-backfill Chroma for all projects if out of sync with SQLite (fire-and-forget)
|
||||
if (this.chromaMcpManager) {
|
||||
ChromaSync.backfillAllProjects().then(() => {
|
||||
logger.info('CHROMA_SYNC', 'Backfill check complete for all projects');
|
||||
}).catch(error => {
|
||||
logger.error('CHROMA_SYNC', 'Backfill failed (non-blocking)', {}, error as Error);
|
||||
});
|
||||
}
|
||||
|
||||
// Connect to MCP server
|
||||
const mcpServerPath = path.join(__dirname, 'mcp-server.cjs');
|
||||
const transport = new StdioClientTransport({
|
||||
@@ -439,11 +460,7 @@ export class WorkerService {
|
||||
|
||||
await Promise.race([mcpConnectionPromise, timeoutPromise]);
|
||||
this.mcpReady = true;
|
||||
logger.success('WORKER', 'Connected to MCP server');
|
||||
|
||||
this.initializationCompleteFlag = true;
|
||||
this.resolveInitialization();
|
||||
logger.info('SYSTEM', 'Background initialization complete');
|
||||
logger.success('WORKER', 'MCP server connected');
|
||||
|
||||
// Start orphan reaper to clean up zombie processes (Issue #737)
|
||||
this.stopOrphanReaper = startOrphanReaper(() => {
|
||||
@@ -455,6 +472,18 @@ export class WorkerService {
|
||||
});
|
||||
logger.info('SYSTEM', 'Started orphan reaper (runs every 5 minutes)');
|
||||
|
||||
// Reap stale sessions to unblock orphan process cleanup (Issue #1168)
|
||||
this.staleSessionReaperInterval = setInterval(async () => {
|
||||
try {
|
||||
const reaped = await this.sessionManager.reapStaleSessions();
|
||||
if (reaped > 0) {
|
||||
logger.info('SYSTEM', `Reaped ${reaped} stale sessions`);
|
||||
}
|
||||
} catch (e) {
|
||||
logger.error('SYSTEM', 'Stale session reaper error', { error: e instanceof Error ? e.message : String(e) });
|
||||
}
|
||||
}, 2 * 60 * 1000);
|
||||
|
||||
// Auto-recover orphaned queues (fire-and-forget with error logging)
|
||||
this.processPendingQueues(50).then(result => {
|
||||
if (result.sessionsStarted > 0) {
|
||||
@@ -517,6 +546,9 @@ export class WorkerService {
|
||||
|
||||
logger.info('SYSTEM', `Starting generator (${source}) using ${providerName}`, { sessionId: sid });
|
||||
|
||||
// Track generator activity for stale detection (Issue #1099)
|
||||
session.lastGeneratorActivity = Date.now();
|
||||
|
||||
session.generatorPromise = agent.startSession(session, this)
|
||||
.catch(async (error: unknown) => {
|
||||
const errorMessage = (error as Error)?.message || '';
|
||||
@@ -583,7 +615,13 @@ export class WorkerService {
|
||||
};
|
||||
throw error;
|
||||
})
|
||||
.finally(() => {
|
||||
.finally(async () => {
|
||||
// CRITICAL: Verify subprocess exit to prevent zombie accumulation (Issue #1168)
|
||||
const trackedProcess = getProcessBySession(session.sessionDbId);
|
||||
if (trackedProcess && !trackedProcess.process.killed && trackedProcess.process.exitCode === null) {
|
||||
await ensureProcessExit(trackedProcess, 5000);
|
||||
}
|
||||
|
||||
session.generatorPromise = null;
|
||||
|
||||
// Record successful AI interaction if no error occurred
|
||||
@@ -604,17 +642,16 @@ export class WorkerService {
|
||||
return;
|
||||
}
|
||||
|
||||
// Shared store for idle-reset and pending-count checks below
|
||||
// Store for pending-count check below
|
||||
const { PendingMessageStore } = require('./sqlite/PendingMessageStore.js');
|
||||
const pendingStore = new PendingMessageStore(this.dbManager.getSessionStore().db, 3);
|
||||
|
||||
// Idle timeout means no new work arrived for 3 minutes - don't restart
|
||||
// No need to reset stale processing messages here — claimNextMessage() self-heals
|
||||
if (session.idleTimedOut) {
|
||||
logger.info('SYSTEM', 'Generator exited due to idle timeout, not restarting', {
|
||||
sessionId: session.sessionDbId
|
||||
});
|
||||
// Reset stale processing messages so they can be picked up later
|
||||
pendingStore.resetStaleProcessingMessages(0, session.sessionDbId); // Reset this session's messages only
|
||||
session.idleTimedOut = false; // Reset flag
|
||||
this.broadcastProcessingStatus();
|
||||
return;
|
||||
@@ -814,12 +851,18 @@ export class WorkerService {
|
||||
this.stopOrphanReaper = null;
|
||||
}
|
||||
|
||||
// Stop stale session reaper (Issue #1168)
|
||||
if (this.staleSessionReaperInterval) {
|
||||
clearInterval(this.staleSessionReaperInterval);
|
||||
this.staleSessionReaperInterval = null;
|
||||
}
|
||||
|
||||
await performGracefulShutdown({
|
||||
server: this.server.getHttpServer(),
|
||||
sessionManager: this.sessionManager,
|
||||
mcpClient: this.mcpClient,
|
||||
dbManager: this.dbManager,
|
||||
chromaServer: this.chromaServer || undefined
|
||||
chromaMcpManager: this.chromaMcpManager || undefined
|
||||
});
|
||||
}
|
||||
|
||||
@@ -864,6 +907,23 @@ async function ensureWorkerStarted(port: number): Promise<boolean> {
|
||||
if (await waitForHealth(port, 1000)) {
|
||||
const versionCheck = await checkVersionMatch(port);
|
||||
if (!versionCheck.matches) {
|
||||
// Guard: If PID file was written recently, another session is likely already
|
||||
// restarting the worker. Poll health instead of starting a concurrent restart.
|
||||
// This prevents the "100 sessions all restart simultaneously" storm (#1145).
|
||||
const RESTART_COORDINATION_THRESHOLD_MS = 15000;
|
||||
if (isPidFileRecent(RESTART_COORDINATION_THRESHOLD_MS)) {
|
||||
logger.info('SYSTEM', 'Version mismatch detected but PID file is recent — another restart likely in progress, polling health', {
|
||||
pluginVersion: versionCheck.pluginVersion,
|
||||
workerVersion: versionCheck.workerVersion
|
||||
});
|
||||
const healthy = await waitForHealth(port, RESTART_COORDINATION_THRESHOLD_MS);
|
||||
if (healthy) {
|
||||
logger.info('SYSTEM', 'Worker became healthy after waiting for concurrent restart');
|
||||
return true;
|
||||
}
|
||||
logger.warn('SYSTEM', 'Worker did not become healthy after waiting — proceeding with own restart');
|
||||
}
|
||||
|
||||
logger.info('SYSTEM', 'Worker version mismatch detected - auto-restarting', {
|
||||
pluginVersion: versionCheck.pluginVersion,
|
||||
workerVersion: versionCheck.workerVersion
|
||||
@@ -920,7 +980,17 @@ async function ensureWorkerStarted(port: number): Promise<boolean> {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Health passed (HTTP listening). Now wait for DB + search initialization
|
||||
// so hooks that run immediately after can actually use the worker.
|
||||
const ready = await waitForReadiness(port, getPlatformTimeout(HOOK_TIMEOUTS.READINESS_WAIT));
|
||||
if (!ready) {
|
||||
logger.warn('SYSTEM', 'Worker is alive but readiness timed out — proceeding anyway');
|
||||
}
|
||||
|
||||
clearWorkerSpawnAttempted();
|
||||
// Touch PID file to signal other sessions that a restart just completed.
|
||||
// Other sessions checking isPidFileRecent() will see this and skip their own restart.
|
||||
touchPidFile();
|
||||
logger.info('SYSTEM', 'Worker started successfully');
|
||||
return true;
|
||||
}
|
||||
@@ -931,6 +1001,14 @@ async function ensureWorkerStarted(port: number): Promise<boolean> {
|
||||
|
||||
async function main() {
|
||||
const command = process.argv[2];
|
||||
|
||||
// Early exit if plugin is disabled in Claude Code settings (#781).
|
||||
// Only gate hook-initiated commands; CLI management (stop/status) still works.
|
||||
const hookInitiatedCommands = ['start', 'hook', 'restart', '--daemon'];
|
||||
if ((hookInitiatedCommands.includes(command) || command === undefined) && isPluginDisabledInClaudeSettings()) {
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
const port = getWorkerPort();
|
||||
|
||||
// Helper for JSON status output in 'start' command
|
||||
@@ -949,6 +1027,7 @@ async function main() {
|
||||
} else {
|
||||
exitWithStatus('error', 'Failed to start worker');
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
case 'stop': {
|
||||
@@ -960,6 +1039,7 @@ async function main() {
|
||||
removePidFile();
|
||||
logger.info('SYSTEM', 'Worker stopped successfully');
|
||||
process.exit(0);
|
||||
break;
|
||||
}
|
||||
|
||||
case 'restart': {
|
||||
@@ -996,6 +1076,7 @@ async function main() {
|
||||
|
||||
logger.info('SYSTEM', 'Worker restarted successfully');
|
||||
process.exit(0);
|
||||
break;
|
||||
}
|
||||
|
||||
case 'status': {
|
||||
@@ -1010,12 +1091,14 @@ async function main() {
|
||||
console.log('Worker is not running');
|
||||
}
|
||||
process.exit(0);
|
||||
break;
|
||||
}
|
||||
|
||||
case 'cursor': {
|
||||
const subcommand = process.argv[3];
|
||||
const cursorResult = await handleCursorCommand(subcommand, process.argv.slice(4));
|
||||
process.exit(cursorResult);
|
||||
break;
|
||||
}
|
||||
|
||||
case 'hook': {
|
||||
@@ -1069,6 +1152,7 @@ async function main() {
|
||||
const { generateClaudeMd } = await import('../cli/claude-md-commands.js');
|
||||
const result = await generateClaudeMd(dryRun);
|
||||
process.exit(result);
|
||||
break;
|
||||
}
|
||||
|
||||
case 'clean': {
|
||||
@@ -1076,10 +1160,33 @@ async function main() {
|
||||
const { cleanClaudeMd } = await import('../cli/claude-md-commands.js');
|
||||
const result = await cleanClaudeMd(dryRun);
|
||||
process.exit(result);
|
||||
break;
|
||||
}
|
||||
|
||||
case '--daemon':
|
||||
default: {
|
||||
// GUARD 1: Refuse to start if another worker is already alive (PID check).
|
||||
// Instant check (kill -0) — no HTTP dependency.
|
||||
const existingPidInfo = readPidFile();
|
||||
if (existingPidInfo && isProcessAlive(existingPidInfo.pid)) {
|
||||
logger.info('SYSTEM', 'Worker already running (PID alive), refusing to start duplicate', {
|
||||
existingPid: existingPidInfo.pid,
|
||||
existingPort: existingPidInfo.port,
|
||||
startedAt: existingPidInfo.startedAt
|
||||
});
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// GUARD 2: Refuse to start if the port is already bound.
|
||||
// Catches the race where two daemons start simultaneously before
|
||||
// either writes a PID file. Must run BEFORE constructing WorkerService
|
||||
// because the constructor registers signal handlers and timers that
|
||||
// prevent the process from exiting even if listen() fails later.
|
||||
if (await isPortInUse(port)) {
|
||||
logger.info('SYSTEM', 'Port already in use, refusing to start duplicate', { port });
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Prevent daemon from dying silently on unhandled errors.
|
||||
// The HTTP server can continue serving even if a background task throws.
|
||||
process.on('unhandledRejection', (reason) => {
|
||||
@@ -1110,5 +1217,8 @@ const isMainModule = typeof require !== 'undefined' && typeof module !== 'undefi
|
||||
: import.meta.url === `file://${process.argv[1]}` || process.argv[1]?.endsWith('worker-service');
|
||||
|
||||
if (isMainModule) {
|
||||
main();
|
||||
main().catch((error) => {
|
||||
logger.error('SYSTEM', 'Fatal error in main', {}, error instanceof Error ? error : undefined);
|
||||
process.exit(0); // Exit 0: don't block Claude Code, don't leave Windows Terminal tabs open
|
||||
});
|
||||
}
|
||||
|
||||
@@ -36,6 +36,7 @@ export interface ActiveSession {
|
||||
consecutiveRestarts: number; // Track consecutive restart attempts to prevent infinite loops
|
||||
forceInit?: boolean; // Force fresh SDK session (skip resume)
|
||||
idleTimedOut?: boolean; // Set when session exits due to idle timeout (prevents restart loop)
|
||||
lastGeneratorActivity: number; // Timestamp of last generator progress (for stale detection, Issue #1099)
|
||||
// CLAIM-CONFIRM FIX: Track IDs of messages currently being processed
|
||||
// These IDs will be confirmed (deleted) after successful storage
|
||||
processingMessageIds: number[];
|
||||
|
||||
@@ -11,6 +11,8 @@
|
||||
import { SessionStore } from '../sqlite/SessionStore.js';
|
||||
import { SessionSearch } from '../sqlite/SessionSearch.js';
|
||||
import { ChromaSync } from '../sync/ChromaSync.js';
|
||||
import { SettingsDefaultsManager } from '../../shared/SettingsDefaultsManager.js';
|
||||
import { USER_SETTINGS_PATH } from '../../shared/paths.js';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
import type { DBSession } from '../worker-types.js';
|
||||
|
||||
@@ -27,8 +29,14 @@ export class DatabaseManager {
|
||||
this.sessionStore = new SessionStore();
|
||||
this.sessionSearch = new SessionSearch();
|
||||
|
||||
// Initialize ChromaSync (lazy - connects on first search, not at startup)
|
||||
this.chromaSync = new ChromaSync('claude-mem');
|
||||
// Initialize ChromaSync only if Chroma is enabled (SQLite-only fallback when disabled)
|
||||
const settings = SettingsDefaultsManager.loadFromFile(USER_SETTINGS_PATH);
|
||||
const chromaEnabled = settings.CLAUDE_MEM_CHROMA_ENABLED !== 'false';
|
||||
if (chromaEnabled) {
|
||||
this.chromaSync = new ChromaSync('claude-mem');
|
||||
} else {
|
||||
logger.info('DB', 'Chroma disabled via CLAUDE_MEM_CHROMA_ENABLED=false, using SQLite-only search');
|
||||
}
|
||||
|
||||
logger.info('DB', 'Database initialized');
|
||||
}
|
||||
@@ -37,7 +45,7 @@ export class DatabaseManager {
|
||||
* Close database connection and cleanup all resources
|
||||
*/
|
||||
async close(): Promise<void> {
|
||||
// Close ChromaSync first (terminates uvx/python processes)
|
||||
// Close ChromaSync first (MCP connection lifecycle managed by ChromaMcpManager)
|
||||
if (this.chromaSync) {
|
||||
await this.chromaSync.close();
|
||||
this.chromaSync = null;
|
||||
@@ -75,12 +83,9 @@ export class DatabaseManager {
|
||||
}
|
||||
|
||||
/**
|
||||
* Get ChromaSync instance (throws if not initialized)
|
||||
* Get ChromaSync instance (returns null if Chroma is disabled)
|
||||
*/
|
||||
getChromaSync(): ChromaSync {
|
||||
if (!this.chromaSync) {
|
||||
throw new Error('ChromaSync not initialized');
|
||||
}
|
||||
getChromaSync(): ChromaSync | null {
|
||||
return this.chromaSync;
|
||||
}
|
||||
|
||||
|
||||
@@ -39,7 +39,7 @@ export class SearchManager {
|
||||
constructor(
|
||||
private sessionSearch: SessionSearch,
|
||||
private sessionStore: SessionStore,
|
||||
private chromaSync: ChromaSync,
|
||||
private chromaSync: ChromaSync | null,
|
||||
private formatter: FormattingService,
|
||||
private timelineService: TimelineService
|
||||
) {
|
||||
|
||||
@@ -155,7 +155,8 @@ export class SessionManager {
|
||||
conversationHistory: [], // Initialize empty - will be populated by agents
|
||||
currentProvider: null, // Will be set when generator starts
|
||||
consecutiveRestarts: 0, // Track consecutive restart attempts to prevent infinite loops
|
||||
processingMessageIds: [] // CLAIM-CONFIRM: Track message IDs for confirmProcessed()
|
||||
processingMessageIds: [], // CLAIM-CONFIRM: Track message IDs for confirmProcessed()
|
||||
lastGeneratorActivity: Date.now() // Initialize for stale detection (Issue #1099)
|
||||
};
|
||||
|
||||
logger.debug('SESSION', 'Creating new session object (memorySessionId cleared to prevent stale resume)', {
|
||||
@@ -286,11 +287,17 @@ export class SessionManager {
|
||||
// 1. Abort the SDK agent
|
||||
session.abortController.abort();
|
||||
|
||||
// 2. Wait for generator to finish
|
||||
// 2. Wait for generator to finish (with 30s timeout to prevent stale stall, Issue #1099)
|
||||
if (session.generatorPromise) {
|
||||
await session.generatorPromise.catch(() => {
|
||||
const generatorDone = session.generatorPromise.catch(() => {
|
||||
logger.debug('SYSTEM', 'Generator already failed, cleaning up', { sessionId: session.sessionDbId });
|
||||
});
|
||||
const timeoutDone = new Promise<void>(resolve => {
|
||||
AbortSignal.timeout(30_000).addEventListener('abort', () => resolve(), { once: true });
|
||||
});
|
||||
await Promise.race([generatorDone, timeoutDone]).then(() => {}, () => {
|
||||
logger.warn('SESSION', 'Generator did not exit within 30s after abort, forcing cleanup (#1099)', { sessionDbId });
|
||||
});
|
||||
}
|
||||
|
||||
// 3. Verify subprocess exit with 5s timeout (Issue #737 fix)
|
||||
@@ -341,6 +348,39 @@ export class SessionManager {
|
||||
}
|
||||
}
|
||||
|
||||
private static readonly MAX_SESSION_IDLE_MS = 15 * 60 * 1000; // 15 minutes
|
||||
|
||||
/**
|
||||
* Reap sessions with no active generator and no pending work that have been idle too long.
|
||||
* This unblocks the orphan reaper which skips processes for "active" sessions. (Issue #1168)
|
||||
*/
|
||||
async reapStaleSessions(): Promise<number> {
|
||||
const now = Date.now();
|
||||
const staleSessionIds: number[] = [];
|
||||
|
||||
for (const [sessionDbId, session] of this.sessions) {
|
||||
// Skip sessions with active generators
|
||||
if (session.generatorPromise) continue;
|
||||
|
||||
// Skip sessions with pending work
|
||||
const pendingCount = this.getPendingStore().getPendingCount(sessionDbId);
|
||||
if (pendingCount > 0) continue;
|
||||
|
||||
// No generator + no pending work + old enough = stale
|
||||
const sessionAge = now - session.startTime;
|
||||
if (sessionAge > SessionManager.MAX_SESSION_IDLE_MS) {
|
||||
staleSessionIds.push(sessionDbId);
|
||||
}
|
||||
}
|
||||
|
||||
for (const sessionDbId of staleSessionIds) {
|
||||
logger.warn('SESSION', `Reaping stale session ${sessionDbId} (no activity for >${Math.round(SessionManager.MAX_SESSION_IDLE_MS / 60000)}m)`, { sessionDbId });
|
||||
await this.deleteSession(sessionDbId);
|
||||
}
|
||||
|
||||
return staleSessionIds.length;
|
||||
}
|
||||
|
||||
/**
|
||||
* Shutdown all active sessions
|
||||
*/
|
||||
@@ -435,6 +475,9 @@ export class SessionManager {
|
||||
session.earliestPendingTimestamp = Math.min(session.earliestPendingTimestamp, message._originalTimestamp);
|
||||
}
|
||||
|
||||
// Update generator activity for stale detection (Issue #1099)
|
||||
session.lastGeneratorActivity = Date.now();
|
||||
|
||||
yield message;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -56,6 +56,9 @@ export async function processAgentResponse(
|
||||
agentName: string,
|
||||
projectRoot?: string
|
||||
): Promise<void> {
|
||||
// Track generator activity for stale detection (Issue #1099)
|
||||
session.lastGeneratorActivity = Date.now();
|
||||
|
||||
// Add assistant response to shared conversation history for provider interop
|
||||
if (text) {
|
||||
session.conversationHistory.push({ role: 'assistant', content: text });
|
||||
@@ -189,8 +192,8 @@ async function syncAndBroadcastObservations(
|
||||
const obs = observations[i];
|
||||
const chromaStart = Date.now();
|
||||
|
||||
// Sync to Chroma (fire-and-forget)
|
||||
dbManager.getChromaSync().syncObservation(
|
||||
// Sync to Chroma (fire-and-forget, skipped if Chroma is disabled)
|
||||
dbManager.getChromaSync()?.syncObservation(
|
||||
obsId,
|
||||
session.contentSessionId,
|
||||
session.project,
|
||||
@@ -282,8 +285,8 @@ async function syncAndBroadcastSummary(
|
||||
|
||||
const chromaStart = Date.now();
|
||||
|
||||
// Sync to Chroma (fire-and-forget)
|
||||
dbManager.getChromaSync().syncSummary(
|
||||
// Sync to Chroma (fire-and-forget, skipped if Chroma is disabled)
|
||||
dbManager.getChromaSync()?.syncSummary(
|
||||
result.summaryId,
|
||||
session.contentSessionId,
|
||||
session.project,
|
||||
|
||||
@@ -37,6 +37,8 @@ export function createMiddleware(
|
||||
callback(new Error('CORS not allowed'));
|
||||
}
|
||||
},
|
||||
methods: ['GET', 'HEAD', 'POST', 'PUT', 'PATCH', 'DELETE'],
|
||||
allowedHeaders: ['Content-Type', 'Authorization', 'X-Requested-With'],
|
||||
credentials: false
|
||||
}));
|
||||
|
||||
|
||||
@@ -114,7 +114,12 @@ export class DataRoutes extends BaseRouteHandler {
|
||||
* Body: { ids: number[], orderBy?: 'date_desc' | 'date_asc', limit?: number, project?: string }
|
||||
*/
|
||||
private handleGetObservationsByIds = this.wrapHandler((req: Request, res: Response): void => {
|
||||
const { ids, orderBy, limit, project } = req.body;
|
||||
let { ids, orderBy, limit, project } = req.body;
|
||||
|
||||
// Coerce string-encoded arrays from MCP clients (e.g. "[1,2,3]" or "1,2,3")
|
||||
if (typeof ids === 'string') {
|
||||
try { ids = JSON.parse(ids); } catch { ids = ids.split(',').map(Number); }
|
||||
}
|
||||
|
||||
if (!ids || !Array.isArray(ids)) {
|
||||
this.badRequest(res, 'ids must be an array of numbers');
|
||||
@@ -163,7 +168,12 @@ export class DataRoutes extends BaseRouteHandler {
|
||||
* Body: { memorySessionIds: string[] }
|
||||
*/
|
||||
private handleGetSdkSessionsByIds = this.wrapHandler((req: Request, res: Response): void => {
|
||||
const { memorySessionIds } = req.body;
|
||||
let { memorySessionIds } = req.body;
|
||||
|
||||
// Coerce string-encoded arrays from MCP clients (e.g. '["a","b"]' or "a,b")
|
||||
if (typeof memorySessionIds === 'string') {
|
||||
try { memorySessionIds = JSON.parse(memorySessionIds); } catch { memorySessionIds = memorySessionIds.split(',').map((s: string) => s.trim()); }
|
||||
}
|
||||
|
||||
if (!Array.isArray(memorySessionIds)) {
|
||||
this.badRequest(res, 'memorySessionIds must be an array');
|
||||
|
||||
@@ -5,12 +5,85 @@
|
||||
*/
|
||||
|
||||
import express, { Request, Response } from 'express';
|
||||
import { readFileSync, existsSync, writeFileSync, readdirSync } from 'fs';
|
||||
import { openSync, fstatSync, readSync, closeSync, existsSync, writeFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { logger } from '../../../../utils/logger.js';
|
||||
import { SettingsDefaultsManager } from '../../../../shared/SettingsDefaultsManager.js';
|
||||
import { BaseRouteHandler } from '../BaseRouteHandler.js';
|
||||
|
||||
/**
|
||||
* Read the last N lines from a file without loading the entire file into memory.
|
||||
* Reads backwards from the end of the file in chunks until enough lines are found.
|
||||
*/
|
||||
export function readLastLines(filePath: string, lineCount: number): { lines: string; totalEstimate: number } {
|
||||
const fd = openSync(filePath, 'r');
|
||||
try {
|
||||
const stat = fstatSync(fd);
|
||||
const fileSize = stat.size;
|
||||
|
||||
if (fileSize === 0) {
|
||||
return { lines: '', totalEstimate: 0 };
|
||||
}
|
||||
|
||||
// Start with a reasonable chunk size, expand if needed
|
||||
const INITIAL_CHUNK_SIZE = 64 * 1024; // 64KB
|
||||
const MAX_READ_SIZE = 10 * 1024 * 1024; // 10MB cap to prevent OOM on huge single-line files
|
||||
|
||||
let readSize = Math.min(INITIAL_CHUNK_SIZE, fileSize);
|
||||
let content = '';
|
||||
let newlineCount = 0;
|
||||
|
||||
while (readSize <= fileSize && readSize <= MAX_READ_SIZE) {
|
||||
const startPosition = Math.max(0, fileSize - readSize);
|
||||
const bytesToRead = fileSize - startPosition;
|
||||
const buffer = Buffer.alloc(bytesToRead);
|
||||
readSync(fd, buffer, 0, bytesToRead, startPosition);
|
||||
content = buffer.toString('utf-8');
|
||||
|
||||
// Count newlines to see if we have enough
|
||||
newlineCount = 0;
|
||||
for (let i = 0; i < content.length; i++) {
|
||||
if (content[i] === '\n') newlineCount++;
|
||||
}
|
||||
|
||||
// We need lineCount newlines to get lineCount full lines (trailing newline)
|
||||
if (newlineCount >= lineCount || startPosition === 0) {
|
||||
break;
|
||||
}
|
||||
|
||||
// Double the read size for next attempt
|
||||
readSize = Math.min(readSize * 2, fileSize, MAX_READ_SIZE);
|
||||
}
|
||||
|
||||
// Split and take the last N lines
|
||||
const allLines = content.split('\n');
|
||||
// Remove trailing empty element from final newline
|
||||
if (allLines.length > 0 && allLines[allLines.length - 1] === '') {
|
||||
allLines.pop();
|
||||
}
|
||||
|
||||
const startIndex = Math.max(0, allLines.length - lineCount);
|
||||
const resultLines = allLines.slice(startIndex);
|
||||
|
||||
// Estimate total lines: if we read the whole file, we know exactly; otherwise estimate
|
||||
let totalEstimate: number;
|
||||
if (fileSize <= readSize) {
|
||||
totalEstimate = allLines.length;
|
||||
} else {
|
||||
// Rough estimate based on average line length in the chunk we read
|
||||
const avgLineLength = content.length / Math.max(newlineCount, 1);
|
||||
totalEstimate = Math.round(fileSize / avgLineLength);
|
||||
}
|
||||
|
||||
return {
|
||||
lines: resultLines.join('\n'),
|
||||
totalEstimate,
|
||||
};
|
||||
} finally {
|
||||
closeSync(fd);
|
||||
}
|
||||
}
|
||||
|
||||
export class LogsRoutes extends BaseRouteHandler {
|
||||
private getLogFilePath(): string {
|
||||
const dataDir = SettingsDefaultsManager.get('CLAUDE_MEM_DATA_DIR');
|
||||
@@ -50,19 +123,15 @@ export class LogsRoutes extends BaseRouteHandler {
|
||||
const requestedLines = parseInt(req.query.lines as string || '1000', 10);
|
||||
const maxLines = Math.min(requestedLines, 10000); // Cap at 10k lines
|
||||
|
||||
const content = readFileSync(logFilePath, 'utf-8');
|
||||
const lines = content.split('\n');
|
||||
|
||||
// Return the last N lines
|
||||
const startIndex = Math.max(0, lines.length - maxLines);
|
||||
const recentLines = lines.slice(startIndex).join('\n');
|
||||
const { lines: recentLines, totalEstimate } = readLastLines(logFilePath, maxLines);
|
||||
const returnedLines = recentLines === '' ? 0 : recentLines.split('\n').length;
|
||||
|
||||
res.json({
|
||||
logs: recentLines,
|
||||
path: logFilePath,
|
||||
exists: true,
|
||||
totalLines: lines.length,
|
||||
returnedLines: lines.length - startIndex
|
||||
totalLines: totalEstimate,
|
||||
returnedLines,
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -21,6 +21,7 @@ import { SessionCompletionHandler } from '../../session/SessionCompletionHandler
|
||||
import { PrivacyCheckValidator } from '../../validation/PrivacyCheckValidator.js';
|
||||
import { SettingsDefaultsManager } from '../../../../shared/SettingsDefaultsManager.js';
|
||||
import { USER_SETTINGS_PATH } from '../../../../shared/paths.js';
|
||||
import { getProcessBySession, ensureProcessExit } from '../../ProcessRegistry.js';
|
||||
|
||||
export class SessionRoutes extends BaseRouteHandler {
|
||||
private completionHandler: SessionCompletionHandler;
|
||||
@@ -89,6 +90,8 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
* we let the current generator finish naturally (max 5s linger timeout).
|
||||
* The next generator will use the new provider with shared conversationHistory.
|
||||
*/
|
||||
private static readonly STALE_GENERATOR_THRESHOLD_MS = 30_000; // 30 seconds (#1099)
|
||||
|
||||
private ensureGeneratorRunning(sessionDbId: number, source: string): void {
|
||||
const session = this.sessionManager.getSession(sessionDbId);
|
||||
if (!session) return;
|
||||
@@ -108,6 +111,26 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
return;
|
||||
}
|
||||
|
||||
// Generator is running - check if stale (no activity for 30s) to prevent queue stall (#1099)
|
||||
const timeSinceActivity = Date.now() - session.lastGeneratorActivity;
|
||||
if (timeSinceActivity > SessionRoutes.STALE_GENERATOR_THRESHOLD_MS) {
|
||||
logger.warn('SESSION', 'Stale generator detected, aborting to prevent queue stall (#1099)', {
|
||||
sessionId: sessionDbId,
|
||||
timeSinceActivityMs: timeSinceActivity,
|
||||
thresholdMs: SessionRoutes.STALE_GENERATOR_THRESHOLD_MS,
|
||||
source
|
||||
});
|
||||
// Abort the stale generator and reset state
|
||||
session.abortController.abort();
|
||||
session.generatorPromise = null;
|
||||
session.abortController = new AbortController();
|
||||
session.lastGeneratorActivity = Date.now();
|
||||
// Start a fresh generator
|
||||
this.spawnInProgress.set(sessionDbId, true);
|
||||
this.startGeneratorWithProvider(session, selectedProvider, 'stale-recovery');
|
||||
return;
|
||||
}
|
||||
|
||||
// Generator is running - check if provider changed
|
||||
if (session.currentProvider && session.currentProvider !== selectedProvider) {
|
||||
logger.info('SESSION', `Provider changed, will switch after current generator finishes`, {
|
||||
@@ -154,8 +177,9 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
historyLength: session.conversationHistory.length
|
||||
});
|
||||
|
||||
// Track which provider is running
|
||||
// Track which provider is running and mark activity for stale detection (#1099)
|
||||
session.currentProvider = provider;
|
||||
session.lastGeneratorActivity = Date.now();
|
||||
|
||||
session.generatorPromise = agent.startSession(session, this.workerService)
|
||||
.catch(error => {
|
||||
@@ -184,7 +208,13 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
}, dbError as Error);
|
||||
}
|
||||
})
|
||||
.finally(() => {
|
||||
.finally(async () => {
|
||||
// CRITICAL: Verify subprocess exit to prevent zombie accumulation (Issue #1168)
|
||||
const tracked = getProcessBySession(session.sessionDbId);
|
||||
if (tracked && !tracked.process.killed && tracked.process.exitCode === null) {
|
||||
await ensureProcessExit(tracked, 5000);
|
||||
}
|
||||
|
||||
const sessionDbId = session.sessionDbId;
|
||||
this.spawnInProgress.delete(sessionDbId);
|
||||
const wasAborted = session.abortController.signal.aborted;
|
||||
@@ -497,57 +527,63 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
}
|
||||
}
|
||||
|
||||
const store = this.dbManager.getSessionStore();
|
||||
try {
|
||||
const store = this.dbManager.getSessionStore();
|
||||
|
||||
// Get or create session
|
||||
const sessionDbId = store.createSDKSession(contentSessionId, '', '');
|
||||
const promptNumber = store.getPromptNumberFromUserPrompts(contentSessionId);
|
||||
// Get or create session
|
||||
const sessionDbId = store.createSDKSession(contentSessionId, '', '');
|
||||
const promptNumber = store.getPromptNumberFromUserPrompts(contentSessionId);
|
||||
|
||||
// Privacy check: skip if user prompt was entirely private
|
||||
const userPrompt = PrivacyCheckValidator.checkUserPromptPrivacy(
|
||||
store,
|
||||
contentSessionId,
|
||||
promptNumber,
|
||||
'observation',
|
||||
sessionDbId,
|
||||
{ tool_name }
|
||||
);
|
||||
if (!userPrompt) {
|
||||
res.json({ status: 'skipped', reason: 'private' });
|
||||
return;
|
||||
// Privacy check: skip if user prompt was entirely private
|
||||
const userPrompt = PrivacyCheckValidator.checkUserPromptPrivacy(
|
||||
store,
|
||||
contentSessionId,
|
||||
promptNumber,
|
||||
'observation',
|
||||
sessionDbId,
|
||||
{ tool_name }
|
||||
);
|
||||
if (!userPrompt) {
|
||||
res.json({ status: 'skipped', reason: 'private' });
|
||||
return;
|
||||
}
|
||||
|
||||
// Strip memory tags from tool_input and tool_response
|
||||
const cleanedToolInput = tool_input !== undefined
|
||||
? stripMemoryTagsFromJson(JSON.stringify(tool_input))
|
||||
: '{}';
|
||||
|
||||
const cleanedToolResponse = tool_response !== undefined
|
||||
? stripMemoryTagsFromJson(JSON.stringify(tool_response))
|
||||
: '{}';
|
||||
|
||||
// Queue observation
|
||||
this.sessionManager.queueObservation(sessionDbId, {
|
||||
tool_name,
|
||||
tool_input: cleanedToolInput,
|
||||
tool_response: cleanedToolResponse,
|
||||
prompt_number: promptNumber,
|
||||
cwd: cwd || (() => {
|
||||
logger.error('SESSION', 'Missing cwd when queueing observation in SessionRoutes', {
|
||||
sessionId: sessionDbId,
|
||||
tool_name
|
||||
});
|
||||
return '';
|
||||
})()
|
||||
});
|
||||
|
||||
// Ensure SDK agent is running
|
||||
this.ensureGeneratorRunning(sessionDbId, 'observation');
|
||||
|
||||
// Broadcast observation queued event
|
||||
this.eventBroadcaster.broadcastObservationQueued(sessionDbId);
|
||||
|
||||
res.json({ status: 'queued' });
|
||||
} catch (error) {
|
||||
// Return 200 on recoverable errors so the hook doesn't break
|
||||
logger.error('SESSION', 'Observation storage failed', { contentSessionId, tool_name }, error as Error);
|
||||
res.json({ stored: false, reason: (error as Error).message });
|
||||
}
|
||||
|
||||
// Strip memory tags from tool_input and tool_response
|
||||
const cleanedToolInput = tool_input !== undefined
|
||||
? stripMemoryTagsFromJson(JSON.stringify(tool_input))
|
||||
: '{}';
|
||||
|
||||
const cleanedToolResponse = tool_response !== undefined
|
||||
? stripMemoryTagsFromJson(JSON.stringify(tool_response))
|
||||
: '{}';
|
||||
|
||||
// Queue observation
|
||||
this.sessionManager.queueObservation(sessionDbId, {
|
||||
tool_name,
|
||||
tool_input: cleanedToolInput,
|
||||
tool_response: cleanedToolResponse,
|
||||
prompt_number: promptNumber,
|
||||
cwd: cwd || (() => {
|
||||
logger.error('SESSION', 'Missing cwd when queueing observation in SessionRoutes', {
|
||||
sessionId: sessionDbId,
|
||||
tool_name
|
||||
});
|
||||
return '';
|
||||
})()
|
||||
});
|
||||
|
||||
// Ensure SDK agent is running
|
||||
this.ensureGeneratorRunning(sessionDbId, 'observation');
|
||||
|
||||
// Broadcast observation queued event
|
||||
this.eventBroadcaster.broadcastObservationQueued(sessionDbId);
|
||||
|
||||
res.json({ status: 'queued' });
|
||||
});
|
||||
|
||||
/**
|
||||
@@ -656,23 +692,30 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
* Returns: { sessionDbId, promptNumber, skipped: boolean, reason?: string }
|
||||
*/
|
||||
private handleSessionInitByClaudeId = this.wrapHandler((req: Request, res: Response): void => {
|
||||
const { contentSessionId, project, prompt } = req.body;
|
||||
const { contentSessionId } = req.body;
|
||||
|
||||
// Only contentSessionId is truly required — Cursor and other platforms
|
||||
// may omit prompt/project in their payload (#838, #1049)
|
||||
const project = req.body.project || 'unknown';
|
||||
const prompt = req.body.prompt || '[media prompt]';
|
||||
const customTitle = req.body.customTitle || undefined;
|
||||
|
||||
logger.info('HTTP', 'SessionRoutes: handleSessionInitByClaudeId called', {
|
||||
contentSessionId,
|
||||
project,
|
||||
prompt_length: prompt?.length
|
||||
prompt_length: prompt?.length,
|
||||
customTitle
|
||||
});
|
||||
|
||||
// Validate required parameters
|
||||
if (!this.validateRequired(req, res, ['contentSessionId', 'project', 'prompt'])) {
|
||||
if (!this.validateRequired(req, res, ['contentSessionId'])) {
|
||||
return;
|
||||
}
|
||||
|
||||
const store = this.dbManager.getSessionStore();
|
||||
|
||||
// Step 1: Create/get SDK session (idempotent INSERT OR IGNORE)
|
||||
const sessionDbId = store.createSDKSession(contentSessionId, project, prompt);
|
||||
const sessionDbId = store.createSDKSession(contentSessionId, project, prompt, customTitle);
|
||||
|
||||
// Verify session creation with DB lookup
|
||||
const dbSession = store.getSessionById(sessionDbId);
|
||||
@@ -716,16 +759,22 @@ export class SessionRoutes extends BaseRouteHandler {
|
||||
// Step 5: Save cleaned user prompt
|
||||
store.saveUserPrompt(contentSessionId, promptNumber, cleanedPrompt);
|
||||
|
||||
// Step 6: Check if SDK agent is already running for this session (#1079)
|
||||
// If contextInjected is true, the hook should skip re-initializing the SDK agent
|
||||
const contextInjected = this.sessionManager.getSession(sessionDbId) !== undefined;
|
||||
|
||||
// Debug-level log since CREATED already logged the key info
|
||||
logger.debug('SESSION', 'User prompt saved', {
|
||||
sessionId: sessionDbId,
|
||||
promptNumber
|
||||
promptNumber,
|
||||
contextInjected
|
||||
});
|
||||
|
||||
res.json({
|
||||
sessionDbId,
|
||||
promptNumber,
|
||||
skipped: false
|
||||
skipped: false,
|
||||
contextInjected
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
@@ -51,6 +51,7 @@ export interface SettingsDefaults {
|
||||
// Feature Toggles
|
||||
CLAUDE_MEM_CONTEXT_SHOW_LAST_SUMMARY: string;
|
||||
CLAUDE_MEM_CONTEXT_SHOW_LAST_MESSAGE: string;
|
||||
CLAUDE_MEM_CONTEXT_SHOW_TERMINAL_OUTPUT: string;
|
||||
CLAUDE_MEM_FOLDER_CLAUDEMD_ENABLED: string;
|
||||
// Process Management
|
||||
CLAUDE_MEM_MAX_CONCURRENT_AGENTS: string; // Max concurrent Claude SDK agent subprocesses (default: 2)
|
||||
@@ -58,6 +59,7 @@ export interface SettingsDefaults {
|
||||
CLAUDE_MEM_EXCLUDED_PROJECTS: string; // Comma-separated glob patterns for excluded project paths
|
||||
CLAUDE_MEM_FOLDER_MD_EXCLUDE: string; // JSON array of folder paths to exclude from CLAUDE.md generation
|
||||
// Chroma Vector Database Configuration
|
||||
CLAUDE_MEM_CHROMA_ENABLED: string; // 'true' | 'false' - set to 'false' for SQLite-only mode
|
||||
CLAUDE_MEM_CHROMA_MODE: string; // 'local' | 'remote'
|
||||
CLAUDE_MEM_CHROMA_HOST: string;
|
||||
CLAUDE_MEM_CHROMA_PORT: string;
|
||||
@@ -111,6 +113,7 @@ export class SettingsDefaultsManager {
|
||||
// Feature Toggles
|
||||
CLAUDE_MEM_CONTEXT_SHOW_LAST_SUMMARY: 'true',
|
||||
CLAUDE_MEM_CONTEXT_SHOW_LAST_MESSAGE: 'false',
|
||||
CLAUDE_MEM_CONTEXT_SHOW_TERMINAL_OUTPUT: 'true',
|
||||
CLAUDE_MEM_FOLDER_CLAUDEMD_ENABLED: 'false',
|
||||
// Process Management
|
||||
CLAUDE_MEM_MAX_CONCURRENT_AGENTS: '2', // Max concurrent Claude SDK agent subprocesses
|
||||
@@ -118,7 +121,8 @@ export class SettingsDefaultsManager {
|
||||
CLAUDE_MEM_EXCLUDED_PROJECTS: '', // Comma-separated glob patterns for excluded project paths
|
||||
CLAUDE_MEM_FOLDER_MD_EXCLUDE: '[]', // JSON array of folder paths to exclude from CLAUDE.md generation
|
||||
// Chroma Vector Database Configuration
|
||||
CLAUDE_MEM_CHROMA_MODE: 'local', // 'local' starts npx chroma run, 'remote' connects to existing server
|
||||
CLAUDE_MEM_CHROMA_ENABLED: 'true', // Set to 'false' to disable Chroma and use SQLite-only search
|
||||
CLAUDE_MEM_CHROMA_MODE: 'local', // 'local' uses persistent chroma-mcp via uvx, 'remote' connects to existing server
|
||||
CLAUDE_MEM_CHROMA_HOST: '127.0.0.1',
|
||||
CLAUDE_MEM_CHROMA_PORT: '8000',
|
||||
CLAUDE_MEM_CHROMA_SSL: 'false',
|
||||
|
||||
@@ -2,6 +2,7 @@ export const HOOK_TIMEOUTS = {
|
||||
DEFAULT: 300000, // Standard HTTP timeout (5 min for slow systems)
|
||||
HEALTH_CHECK: 3000, // Worker health check (3s — healthy worker responds in <100ms)
|
||||
POST_SPAWN_WAIT: 5000, // Wait for daemon to start after spawn (starts in <1s on Linux)
|
||||
READINESS_WAIT: 30000, // Wait for DB + search init after spawn (typically <5s)
|
||||
PORT_IN_USE_WAIT: 3000, // Wait when port occupied but health failing
|
||||
WORKER_STARTUP_WAIT: 1000,
|
||||
PRE_RESTART_SETTLE_DELAY: 2000, // Give files time to sync before restart
|
||||
|
||||
+6
-3
@@ -99,7 +99,9 @@ export function ensureAllClaudeDirs(): void {
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current project name from git root or cwd
|
||||
* Get current project name from git root or cwd.
|
||||
* Includes parent directory to avoid collisions when repos share a folder name
|
||||
* (e.g., ~/work/monorepo → "work/monorepo" vs ~/personal/monorepo → "personal/monorepo").
|
||||
*/
|
||||
export function getCurrentProjectName(): string {
|
||||
try {
|
||||
@@ -109,12 +111,13 @@ export function getCurrentProjectName(): string {
|
||||
stdio: ['pipe', 'pipe', 'ignore'],
|
||||
windowsHide: true
|
||||
}).trim();
|
||||
return basename(gitRoot);
|
||||
return basename(dirname(gitRoot)) + '/' + basename(gitRoot);
|
||||
} catch (error) {
|
||||
logger.debug('SYSTEM', 'Git root detection failed, using cwd basename', {
|
||||
cwd: process.cwd()
|
||||
}, error as Error);
|
||||
return basename(process.cwd());
|
||||
const cwd = process.cwd();
|
||||
return basename(dirname(cwd)) + '/' + basename(cwd);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
/**
|
||||
* Plugin state utilities for checking Claude Code's plugin settings.
|
||||
* Kept minimal — no heavy dependencies — so hooks can check quickly.
|
||||
*/
|
||||
|
||||
import { existsSync, readFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { homedir } from 'os';
|
||||
|
||||
const PLUGIN_SETTINGS_KEY = 'claude-mem@thedotmack';
|
||||
|
||||
/**
|
||||
* Check if claude-mem is disabled in Claude Code's settings (#781).
|
||||
* Sync read + JSON parse for speed — called before any async work.
|
||||
* Returns true only if the plugin is explicitly disabled (enabledPlugins[key] === false).
|
||||
*/
|
||||
export function isPluginDisabledInClaudeSettings(): boolean {
|
||||
try {
|
||||
const claudeConfigDir = process.env.CLAUDE_CONFIG_DIR || join(homedir(), '.claude');
|
||||
const settingsPath = join(claudeConfigDir, 'settings.json');
|
||||
if (!existsSync(settingsPath)) return false;
|
||||
const raw = readFileSync(settingsPath, 'utf-8');
|
||||
const settings = JSON.parse(raw);
|
||||
return settings?.enabledPlugins?.[PLUGIN_SETTINGS_KEY] === false;
|
||||
} catch {
|
||||
// If settings can't be read/parsed, assume not disabled
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
import { existsSync, readFileSync, writeFileSync, renameSync, mkdirSync } from 'fs';
|
||||
import { dirname } from 'path';
|
||||
import { dirname, resolve } from 'path';
|
||||
import { replaceTaggedContent } from './claude-md-utils.js';
|
||||
import { logger } from './logger.js';
|
||||
|
||||
@@ -10,6 +10,10 @@ import { logger } from './logger.js';
|
||||
export function writeAgentsMd(agentsPath: string, context: string): void {
|
||||
if (!agentsPath) return;
|
||||
|
||||
// Never write inside .git directories — corrupts refs (#1165)
|
||||
const resolvedPath = resolve(agentsPath);
|
||||
if (resolvedPath.includes('/.git/') || resolvedPath.includes('\\.git\\') || resolvedPath.endsWith('/.git') || resolvedPath.endsWith('\\.git')) return;
|
||||
|
||||
const dir = dirname(agentsPath);
|
||||
if (!existsSync(dir)) {
|
||||
mkdirSync(dir, { recursive: true });
|
||||
|
||||
@@ -114,6 +114,11 @@ export function replaceTaggedContent(existingContent: string, newContent: string
|
||||
* @param newContent - Content to write inside tags
|
||||
*/
|
||||
export function writeClaudeMdToFolder(folderPath: string, newContent: string): void {
|
||||
const resolvedPath = path.resolve(folderPath);
|
||||
|
||||
// Never write inside .git directories — corrupts refs (#1165)
|
||||
if (resolvedPath.includes('/.git/') || resolvedPath.includes('\\.git\\') || resolvedPath.endsWith('/.git') || resolvedPath.endsWith('\\.git')) return;
|
||||
|
||||
const claudeMdPath = path.join(folderPath, 'CLAUDE.md');
|
||||
const tempFile = `${claudeMdPath}.tmp`;
|
||||
|
||||
|
||||
+1
-1
@@ -15,7 +15,7 @@ export enum LogLevel {
|
||||
SILENT = 4
|
||||
}
|
||||
|
||||
export type Component = 'HOOK' | 'WORKER' | 'SDK' | 'PARSER' | 'DB' | 'SYSTEM' | 'HTTP' | 'SESSION' | 'CHROMA' | 'FOLDER_INDEX' | 'CLAUDE_MD';
|
||||
export type Component = 'HOOK' | 'WORKER' | 'SDK' | 'PARSER' | 'DB' | 'SYSTEM' | 'HTTP' | 'SESSION' | 'CHROMA' | 'CHROMA_MCP' | 'CHROMA_SYNC' | 'FOLDER_INDEX' | 'CLAUDE_MD' | 'QUEUE';
|
||||
|
||||
interface LogContext {
|
||||
sessionId?: number;
|
||||
|
||||
@@ -169,11 +169,11 @@ describe('MarkdownFormatter', () => {
|
||||
expect(result[0]).toContain('**Context Index:**');
|
||||
});
|
||||
|
||||
it('should mention MCP tools', () => {
|
||||
it('should mention mem-search skill', () => {
|
||||
const result = renderMarkdownContextIndex();
|
||||
const joined = result.join('\n');
|
||||
|
||||
expect(joined).toContain('MCP tools');
|
||||
expect(joined).toContain('mem-search');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -488,11 +488,11 @@ describe('MarkdownFormatter', () => {
|
||||
expect(joined).toContain('500');
|
||||
});
|
||||
|
||||
it('should mention MCP', () => {
|
||||
it('should mention claude-mem skill', () => {
|
||||
const result = renderMarkdownFooter(5000, 100);
|
||||
const joined = result.join('\n');
|
||||
|
||||
expect(joined).toContain('MCP');
|
||||
expect(joined).toContain('claude-mem');
|
||||
});
|
||||
|
||||
it('should round work tokens to nearest thousand', () => {
|
||||
|
||||
@@ -0,0 +1,374 @@
|
||||
/**
|
||||
* Tests for Hook Lifecycle Fixes (TRIAGE-04)
|
||||
*
|
||||
* Validates:
|
||||
* - Stop hook returns suppressOutput: true (prevents infinite loop #987)
|
||||
* - All handlers return suppressOutput: true (prevents conversation pollution #598, #784)
|
||||
* - Unknown event types handled gracefully (fixes #984)
|
||||
* - stderr suppressed in hook context (fixes #1181)
|
||||
* - Claude Code adapter defaults suppressOutput to true
|
||||
*/
|
||||
import { describe, it, expect, beforeEach, afterEach, mock } from 'bun:test';
|
||||
|
||||
// --- Event Handler Tests ---
|
||||
|
||||
describe('Hook Lifecycle - Event Handlers', () => {
|
||||
describe('getEventHandler', () => {
|
||||
it('should return handler for all recognized event types', async () => {
|
||||
const { getEventHandler } = await import('../src/cli/handlers/index.js');
|
||||
const recognizedTypes = [
|
||||
'context', 'session-init', 'observation',
|
||||
'summarize', 'session-complete', 'user-message', 'file-edit'
|
||||
];
|
||||
for (const type of recognizedTypes) {
|
||||
const handler = getEventHandler(type);
|
||||
expect(handler).toBeDefined();
|
||||
expect(handler.execute).toBeDefined();
|
||||
}
|
||||
});
|
||||
|
||||
it('should return no-op handler for unknown event types (#984)', async () => {
|
||||
const { getEventHandler } = await import('../src/cli/handlers/index.js');
|
||||
const handler = getEventHandler('nonexistent-event');
|
||||
expect(handler).toBeDefined();
|
||||
expect(handler.execute).toBeDefined();
|
||||
|
||||
const result = await handler.execute({
|
||||
sessionId: 'test-session',
|
||||
cwd: '/tmp'
|
||||
});
|
||||
expect(result.continue).toBe(true);
|
||||
expect(result.suppressOutput).toBe(true);
|
||||
expect(result.exitCode).toBe(0);
|
||||
});
|
||||
|
||||
it('should include session-complete as a recognized event type (#984)', async () => {
|
||||
const { getEventHandler } = await import('../src/cli/handlers/index.js');
|
||||
const handler = getEventHandler('session-complete');
|
||||
// session-complete should NOT be the no-op handler
|
||||
// We can verify this by checking it's not the same as an unknown type handler
|
||||
expect(handler).toBeDefined();
|
||||
// The real handler has different behavior than the no-op
|
||||
// (it tries to call the worker, while no-op just returns immediately)
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// --- Codex CLI Compatibility Tests (#744) ---
|
||||
|
||||
describe('Codex CLI Compatibility (#744)', () => {
|
||||
describe('getPlatformAdapter', () => {
|
||||
it('should return rawAdapter for unknown platforms like codex', async () => {
|
||||
const { getPlatformAdapter, rawAdapter } = await import('../src/cli/adapters/index.js');
|
||||
// Should not throw for unknown platforms — falls back to rawAdapter
|
||||
const adapter = getPlatformAdapter('codex');
|
||||
expect(adapter).toBe(rawAdapter);
|
||||
});
|
||||
|
||||
it('should return rawAdapter for any unrecognized platform string', async () => {
|
||||
const { getPlatformAdapter, rawAdapter } = await import('../src/cli/adapters/index.js');
|
||||
const adapter = getPlatformAdapter('some-future-cli');
|
||||
expect(adapter).toBe(rawAdapter);
|
||||
});
|
||||
});
|
||||
|
||||
describe('claudeCodeAdapter session_id fallbacks', () => {
|
||||
it('should use session_id when present', async () => {
|
||||
const { claudeCodeAdapter } = await import('../src/cli/adapters/claude-code.js');
|
||||
const input = claudeCodeAdapter.normalizeInput({ session_id: 'claude-123', cwd: '/tmp' });
|
||||
expect(input.sessionId).toBe('claude-123');
|
||||
});
|
||||
|
||||
it('should fall back to id field (Codex CLI format)', async () => {
|
||||
const { claudeCodeAdapter } = await import('../src/cli/adapters/claude-code.js');
|
||||
const input = claudeCodeAdapter.normalizeInput({ id: 'codex-456', cwd: '/tmp' });
|
||||
expect(input.sessionId).toBe('codex-456');
|
||||
});
|
||||
|
||||
it('should fall back to sessionId field (camelCase format)', async () => {
|
||||
const { claudeCodeAdapter } = await import('../src/cli/adapters/claude-code.js');
|
||||
const input = claudeCodeAdapter.normalizeInput({ sessionId: 'camel-789', cwd: '/tmp' });
|
||||
expect(input.sessionId).toBe('camel-789');
|
||||
});
|
||||
|
||||
it('should return undefined when no session ID field is present', async () => {
|
||||
const { claudeCodeAdapter } = await import('../src/cli/adapters/claude-code.js');
|
||||
const input = claudeCodeAdapter.normalizeInput({ cwd: '/tmp' });
|
||||
expect(input.sessionId).toBeUndefined();
|
||||
});
|
||||
|
||||
it('should handle undefined input gracefully', async () => {
|
||||
const { claudeCodeAdapter } = await import('../src/cli/adapters/claude-code.js');
|
||||
const input = claudeCodeAdapter.normalizeInput(undefined);
|
||||
expect(input.sessionId).toBeUndefined();
|
||||
expect(input.cwd).toBe(process.cwd());
|
||||
});
|
||||
});
|
||||
|
||||
describe('session-init handler undefined prompt', () => {
|
||||
it('should not throw when prompt is undefined', () => {
|
||||
// Verify the short-circuit logic works for undefined
|
||||
const rawPrompt: string | undefined = undefined;
|
||||
const prompt = (!rawPrompt || !rawPrompt.trim()) ? '[media prompt]' : rawPrompt;
|
||||
expect(prompt).toBe('[media prompt]');
|
||||
});
|
||||
|
||||
it('should not throw when prompt is empty string', () => {
|
||||
const rawPrompt = '';
|
||||
const prompt = (!rawPrompt || !rawPrompt.trim()) ? '[media prompt]' : rawPrompt;
|
||||
expect(prompt).toBe('[media prompt]');
|
||||
});
|
||||
|
||||
it('should not throw when prompt is whitespace-only', () => {
|
||||
const rawPrompt = ' ';
|
||||
const prompt = (!rawPrompt || !rawPrompt.trim()) ? '[media prompt]' : rawPrompt;
|
||||
expect(prompt).toBe('[media prompt]');
|
||||
});
|
||||
|
||||
it('should preserve valid prompts', () => {
|
||||
const rawPrompt = 'fix the bug';
|
||||
const prompt = (!rawPrompt || !rawPrompt.trim()) ? '[media prompt]' : rawPrompt;
|
||||
expect(prompt).toBe('fix the bug');
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// --- Cursor IDE Compatibility Tests (#838, #1049) ---
|
||||
|
||||
describe('Cursor IDE Compatibility (#838, #1049)', () => {
|
||||
describe('cursorAdapter session ID fallbacks', () => {
|
||||
it('should use conversation_id when present', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const input = cursorAdapter.normalizeInput({ conversation_id: 'conv-123', workspace_roots: ['/project'] });
|
||||
expect(input.sessionId).toBe('conv-123');
|
||||
});
|
||||
|
||||
it('should fall back to generation_id', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const input = cursorAdapter.normalizeInput({ generation_id: 'gen-456', workspace_roots: ['/project'] });
|
||||
expect(input.sessionId).toBe('gen-456');
|
||||
});
|
||||
|
||||
it('should fall back to id field', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const input = cursorAdapter.normalizeInput({ id: 'id-789', workspace_roots: ['/project'] });
|
||||
expect(input.sessionId).toBe('id-789');
|
||||
});
|
||||
|
||||
it('should return undefined when no session ID field is present', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const input = cursorAdapter.normalizeInput({ workspace_roots: ['/project'] });
|
||||
expect(input.sessionId).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('cursorAdapter prompt field fallbacks', () => {
|
||||
it('should use prompt when present', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const input = cursorAdapter.normalizeInput({ conversation_id: 'c1', prompt: 'fix the bug' });
|
||||
expect(input.prompt).toBe('fix the bug');
|
||||
});
|
||||
|
||||
it('should fall back to query field', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const input = cursorAdapter.normalizeInput({ conversation_id: 'c1', query: 'search for files' });
|
||||
expect(input.prompt).toBe('search for files');
|
||||
});
|
||||
|
||||
it('should fall back to input field', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const input = cursorAdapter.normalizeInput({ conversation_id: 'c1', input: 'user typed this' });
|
||||
expect(input.prompt).toBe('user typed this');
|
||||
});
|
||||
|
||||
it('should fall back to message field', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const input = cursorAdapter.normalizeInput({ conversation_id: 'c1', message: 'hello cursor' });
|
||||
expect(input.prompt).toBe('hello cursor');
|
||||
});
|
||||
|
||||
it('should return undefined when no prompt field is present', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const input = cursorAdapter.normalizeInput({ conversation_id: 'c1' });
|
||||
expect(input.prompt).toBeUndefined();
|
||||
});
|
||||
|
||||
it('should prefer prompt over query', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const input = cursorAdapter.normalizeInput({ conversation_id: 'c1', prompt: 'primary', query: 'secondary' });
|
||||
expect(input.prompt).toBe('primary');
|
||||
});
|
||||
});
|
||||
|
||||
describe('cursorAdapter cwd fallbacks', () => {
|
||||
it('should use workspace_roots[0] when present', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const input = cursorAdapter.normalizeInput({ conversation_id: 'c1', workspace_roots: ['/my/project'] });
|
||||
expect(input.cwd).toBe('/my/project');
|
||||
});
|
||||
|
||||
it('should fall back to cwd field', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const input = cursorAdapter.normalizeInput({ conversation_id: 'c1', cwd: '/fallback/dir' });
|
||||
expect(input.cwd).toBe('/fallback/dir');
|
||||
});
|
||||
|
||||
it('should fall back to process.cwd() when nothing provided', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const input = cursorAdapter.normalizeInput({ conversation_id: 'c1' });
|
||||
expect(input.cwd).toBe(process.cwd());
|
||||
});
|
||||
});
|
||||
|
||||
describe('cursorAdapter undefined input handling', () => {
|
||||
it('should handle undefined input gracefully', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const input = cursorAdapter.normalizeInput(undefined);
|
||||
expect(input.sessionId).toBeUndefined();
|
||||
expect(input.prompt).toBeUndefined();
|
||||
expect(input.cwd).toBe(process.cwd());
|
||||
});
|
||||
|
||||
it('should handle null input gracefully', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const input = cursorAdapter.normalizeInput(null);
|
||||
expect(input.sessionId).toBeUndefined();
|
||||
expect(input.prompt).toBeUndefined();
|
||||
expect(input.cwd).toBe(process.cwd());
|
||||
});
|
||||
});
|
||||
|
||||
describe('cursorAdapter formatOutput', () => {
|
||||
it('should return simple continue flag', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const output = cursorAdapter.formatOutput({ continue: true, suppressOutput: true });
|
||||
expect(output).toEqual({ continue: true });
|
||||
});
|
||||
|
||||
it('should default continue to true', async () => {
|
||||
const { cursorAdapter } = await import('../src/cli/adapters/cursor.js');
|
||||
const output = cursorAdapter.formatOutput({});
|
||||
expect(output).toEqual({ continue: true });
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// --- Platform Adapter Tests ---
|
||||
|
||||
describe('Hook Lifecycle - Claude Code Adapter', () => {
|
||||
it('should default suppressOutput to true when not explicitly set', async () => {
|
||||
const { claudeCodeAdapter } = await import('../src/cli/adapters/claude-code.js');
|
||||
|
||||
// Result with no suppressOutput field
|
||||
const output = claudeCodeAdapter.formatOutput({ continue: true });
|
||||
expect(output).toEqual({ continue: true, suppressOutput: true });
|
||||
});
|
||||
|
||||
it('should default both continue and suppressOutput to true for empty result', async () => {
|
||||
const { claudeCodeAdapter } = await import('../src/cli/adapters/claude-code.js');
|
||||
|
||||
const output = claudeCodeAdapter.formatOutput({});
|
||||
expect(output).toEqual({ continue: true, suppressOutput: true });
|
||||
});
|
||||
|
||||
it('should respect explicit suppressOutput: false', async () => {
|
||||
const { claudeCodeAdapter } = await import('../src/cli/adapters/claude-code.js');
|
||||
|
||||
const output = claudeCodeAdapter.formatOutput({ continue: true, suppressOutput: false });
|
||||
expect(output).toEqual({ continue: true, suppressOutput: false });
|
||||
});
|
||||
|
||||
it('should use hookSpecificOutput format for context injection', async () => {
|
||||
const { claudeCodeAdapter } = await import('../src/cli/adapters/claude-code.js');
|
||||
|
||||
const result = {
|
||||
hookSpecificOutput: { hookEventName: 'SessionStart', additionalContext: 'test context' },
|
||||
systemMessage: 'test message'
|
||||
};
|
||||
const output = claudeCodeAdapter.formatOutput(result) as Record<string, unknown>;
|
||||
expect(output.hookSpecificOutput).toEqual({ hookEventName: 'SessionStart', additionalContext: 'test context' });
|
||||
expect(output.systemMessage).toBe('test message');
|
||||
// Should NOT have continue/suppressOutput when using hookSpecificOutput
|
||||
expect(output.continue).toBeUndefined();
|
||||
expect(output.suppressOutput).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
// --- stderr Suppression Tests ---
|
||||
|
||||
describe('Hook Lifecycle - stderr Suppression (#1181)', () => {
|
||||
let originalStderrWrite: typeof process.stderr.write;
|
||||
let stderrOutput: string[];
|
||||
|
||||
beforeEach(() => {
|
||||
originalStderrWrite = process.stderr.write.bind(process.stderr);
|
||||
stderrOutput = [];
|
||||
// Capture stderr writes
|
||||
process.stderr.write = ((chunk: any) => {
|
||||
stderrOutput.push(String(chunk));
|
||||
return true;
|
||||
}) as typeof process.stderr.write;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
process.stderr.write = originalStderrWrite;
|
||||
});
|
||||
|
||||
it('should not use console.error in handlers/index.ts for unknown events', async () => {
|
||||
// Re-import to get fresh module
|
||||
const { getEventHandler } = await import('../src/cli/handlers/index.js');
|
||||
|
||||
// Clear any stderr from import
|
||||
stderrOutput.length = 0;
|
||||
|
||||
// Call with unknown event — should use logger (writes to file), not console.error (writes to stderr)
|
||||
const handler = getEventHandler('unknown-event-type');
|
||||
await handler.execute({ sessionId: 'test', cwd: '/tmp' });
|
||||
|
||||
// No stderr output should have leaked from the handler dispatcher itself
|
||||
// (logger may write to stderr as fallback if log file unavailable, but that's
|
||||
// the logger's responsibility, not the dispatcher's)
|
||||
const dispatcherStderr = stderrOutput.filter(s => s.includes('[claude-mem] Unknown event'));
|
||||
expect(dispatcherStderr).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
// --- Hook Response Constants ---
|
||||
|
||||
describe('Hook Lifecycle - Standard Response', () => {
|
||||
it('should define standard hook response with suppressOutput: true', async () => {
|
||||
const { STANDARD_HOOK_RESPONSE } = await import('../src/hooks/hook-response.js');
|
||||
const parsed = JSON.parse(STANDARD_HOOK_RESPONSE);
|
||||
expect(parsed.continue).toBe(true);
|
||||
expect(parsed.suppressOutput).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
// --- hookCommand stderr suppression ---
|
||||
|
||||
describe('hookCommand - stderr suppression', () => {
|
||||
it('should not use console.error for worker unavailable errors', async () => {
|
||||
// The hookCommand function should use logger.warn instead of console.error
|
||||
// for worker unavailable errors, so stderr stays clean (#1181)
|
||||
const { hookCommand } = await import('../src/cli/hook-command.js');
|
||||
|
||||
// Verify the import includes logger
|
||||
const hookCommandSource = await Bun.file(
|
||||
new URL('../src/cli/hook-command.ts', import.meta.url).pathname
|
||||
).text();
|
||||
|
||||
// Should import logger
|
||||
expect(hookCommandSource).toContain("import { logger }");
|
||||
// Should use logger.warn for worker unavailable
|
||||
expect(hookCommandSource).toContain("logger.warn('HOOK'");
|
||||
// Should use logger.error for hook errors
|
||||
expect(hookCommandSource).toContain("logger.error('HOOK'");
|
||||
// Should suppress stderr
|
||||
expect(hookCommandSource).toContain("process.stderr.write = (() => true)");
|
||||
// Should restore stderr in finally block
|
||||
expect(hookCommandSource).toContain("process.stderr.write = originalStderrWrite");
|
||||
// Should NOT have console.error for error reporting
|
||||
expect(hookCommandSource).not.toContain("console.error(`[claude-mem]");
|
||||
expect(hookCommandSource).not.toContain("console.error(`Hook error:");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,310 @@
|
||||
/**
|
||||
* Tests for Context Re-Injection Guard (#1079)
|
||||
*
|
||||
* Validates:
|
||||
* - session-init handler skips SDK agent init when contextInjected=true
|
||||
* - session-init handler proceeds with SDK agent init when contextInjected=false
|
||||
* - SessionManager.getSession returns undefined for uninitialized sessions
|
||||
* - SessionManager.getSession returns session after initialization
|
||||
*/
|
||||
import { describe, it, expect, beforeEach, afterEach, spyOn, mock } from 'bun:test';
|
||||
import { homedir } from 'os';
|
||||
import { join } from 'path';
|
||||
|
||||
// Mock modules that cause import chain issues - MUST be before handler imports
|
||||
// paths.ts calls SettingsDefaultsManager.get() at module load time
|
||||
mock.module('../../src/shared/SettingsDefaultsManager.js', () => ({
|
||||
SettingsDefaultsManager: {
|
||||
get: (key: string) => {
|
||||
if (key === 'CLAUDE_MEM_DATA_DIR') return join(homedir(), '.claude-mem');
|
||||
return '';
|
||||
},
|
||||
getInt: () => 0,
|
||||
loadFromFile: () => ({ CLAUDE_MEM_EXCLUDED_PROJECTS: [] }),
|
||||
},
|
||||
}));
|
||||
|
||||
mock.module('../../src/shared/worker-utils.js', () => ({
|
||||
ensureWorkerRunning: () => Promise.resolve(true),
|
||||
getWorkerPort: () => 37777,
|
||||
}));
|
||||
|
||||
mock.module('../../src/utils/project-name.js', () => ({
|
||||
getProjectName: () => 'test-project',
|
||||
}));
|
||||
|
||||
mock.module('../../src/utils/project-filter.js', () => ({
|
||||
isProjectExcluded: () => false,
|
||||
}));
|
||||
|
||||
// Now import after mocks
|
||||
import { logger } from '../../src/utils/logger.js';
|
||||
|
||||
// Suppress logger output during tests
|
||||
let loggerSpies: ReturnType<typeof spyOn>[] = [];
|
||||
|
||||
beforeEach(() => {
|
||||
loggerSpies = [
|
||||
spyOn(logger, 'info').mockImplementation(() => {}),
|
||||
spyOn(logger, 'debug').mockImplementation(() => {}),
|
||||
spyOn(logger, 'warn').mockImplementation(() => {}),
|
||||
spyOn(logger, 'error').mockImplementation(() => {}),
|
||||
spyOn(logger, 'failure').mockImplementation(() => {}),
|
||||
];
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
loggerSpies.forEach(spy => spy.mockRestore());
|
||||
});
|
||||
|
||||
describe('Context Re-Injection Guard (#1079)', () => {
|
||||
describe('session-init handler - contextInjected flag behavior', () => {
|
||||
it('should skip SDK agent init when contextInjected is true', async () => {
|
||||
const fetchedUrls: string[] = [];
|
||||
|
||||
const mockFetch = mock((url: string | URL | Request) => {
|
||||
const urlStr = typeof url === 'string' ? url : url.toString();
|
||||
fetchedUrls.push(urlStr);
|
||||
|
||||
if (urlStr.includes('/api/sessions/init')) {
|
||||
return Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({
|
||||
sessionDbId: 42,
|
||||
promptNumber: 2,
|
||||
skipped: false,
|
||||
contextInjected: true // SDK agent already running
|
||||
})
|
||||
});
|
||||
}
|
||||
|
||||
// The /sessions/42/init call — should NOT be reached
|
||||
return Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ status: 'initialized' })
|
||||
});
|
||||
});
|
||||
|
||||
const originalFetch = globalThis.fetch;
|
||||
globalThis.fetch = mockFetch as any;
|
||||
|
||||
try {
|
||||
const { sessionInitHandler } = await import('../../src/cli/handlers/session-init.js');
|
||||
|
||||
const result = await sessionInitHandler.execute({
|
||||
sessionId: 'test-session-123',
|
||||
cwd: '/test/project',
|
||||
prompt: 'second prompt in this session',
|
||||
platform: 'claude-code',
|
||||
});
|
||||
|
||||
// Should return success without making the second /sessions/42/init call
|
||||
expect(result.continue).toBe(true);
|
||||
expect(result.suppressOutput).toBe(true);
|
||||
|
||||
// Only the /api/sessions/init call should have been made
|
||||
const apiInitCalls = fetchedUrls.filter(u => u.includes('/api/sessions/init'));
|
||||
const sdkInitCalls = fetchedUrls.filter(u => u.includes('/sessions/42/init'));
|
||||
|
||||
expect(apiInitCalls.length).toBe(1);
|
||||
expect(sdkInitCalls.length).toBe(0);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
it('should proceed with SDK agent init when contextInjected is false', async () => {
|
||||
const fetchedUrls: string[] = [];
|
||||
|
||||
const mockFetch = mock((url: string | URL | Request) => {
|
||||
const urlStr = typeof url === 'string' ? url : url.toString();
|
||||
fetchedUrls.push(urlStr);
|
||||
|
||||
if (urlStr.includes('/api/sessions/init')) {
|
||||
return Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({
|
||||
sessionDbId: 42,
|
||||
promptNumber: 1,
|
||||
skipped: false,
|
||||
contextInjected: false // First prompt — SDK agent not yet started
|
||||
})
|
||||
});
|
||||
}
|
||||
|
||||
// The /sessions/42/init call — SHOULD be reached
|
||||
return Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ status: 'initialized' })
|
||||
});
|
||||
});
|
||||
|
||||
const originalFetch = globalThis.fetch;
|
||||
globalThis.fetch = mockFetch as any;
|
||||
|
||||
try {
|
||||
const { sessionInitHandler } = await import('../../src/cli/handlers/session-init.js');
|
||||
|
||||
const result = await sessionInitHandler.execute({
|
||||
sessionId: 'test-session-456',
|
||||
cwd: '/test/project',
|
||||
prompt: 'first prompt in session',
|
||||
platform: 'claude-code',
|
||||
});
|
||||
|
||||
expect(result.continue).toBe(true);
|
||||
expect(result.suppressOutput).toBe(true);
|
||||
|
||||
// Both calls should have been made
|
||||
const apiInitCalls = fetchedUrls.filter(u => u.includes('/api/sessions/init'));
|
||||
const sdkInitCalls = fetchedUrls.filter(u => u.includes('/sessions/42/init'));
|
||||
|
||||
expect(apiInitCalls.length).toBe(1);
|
||||
expect(sdkInitCalls.length).toBe(1);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
it('should proceed with SDK agent init when contextInjected is undefined (backward compat)', async () => {
|
||||
const fetchedUrls: string[] = [];
|
||||
|
||||
const mockFetch = mock((url: string | URL | Request) => {
|
||||
const urlStr = typeof url === 'string' ? url : url.toString();
|
||||
fetchedUrls.push(urlStr);
|
||||
|
||||
if (urlStr.includes('/api/sessions/init')) {
|
||||
return Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({
|
||||
sessionDbId: 42,
|
||||
promptNumber: 1,
|
||||
skipped: false
|
||||
// contextInjected not present (older worker version)
|
||||
})
|
||||
});
|
||||
}
|
||||
|
||||
return Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ status: 'initialized' })
|
||||
});
|
||||
});
|
||||
|
||||
const originalFetch = globalThis.fetch;
|
||||
globalThis.fetch = mockFetch as any;
|
||||
|
||||
try {
|
||||
const { sessionInitHandler } = await import('../../src/cli/handlers/session-init.js');
|
||||
|
||||
const result = await sessionInitHandler.execute({
|
||||
sessionId: 'test-session-789',
|
||||
cwd: '/test/project',
|
||||
prompt: 'test prompt',
|
||||
platform: 'claude-code',
|
||||
});
|
||||
|
||||
expect(result.continue).toBe(true);
|
||||
|
||||
// When contextInjected is undefined/missing, should still make the SDK init call
|
||||
const sdkInitCalls = fetchedUrls.filter(u => u.includes('/sessions/42/init'));
|
||||
expect(sdkInitCalls.length).toBe(1);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('SessionManager contextInjected logic', () => {
|
||||
it('should return undefined for getSession when no active session exists', async () => {
|
||||
const { SessionManager } = await import('../../src/services/worker/SessionManager.js');
|
||||
|
||||
const mockDbManager = {
|
||||
getSessionById: () => ({
|
||||
id: 1,
|
||||
content_session_id: 'test-session',
|
||||
project: 'test',
|
||||
user_prompt: 'test prompt',
|
||||
memory_session_id: null,
|
||||
status: 'active',
|
||||
started_at: new Date().toISOString(),
|
||||
completed_at: null,
|
||||
}),
|
||||
getSessionStore: () => ({ db: {} }),
|
||||
} as any;
|
||||
|
||||
const sessionManager = new SessionManager(mockDbManager);
|
||||
|
||||
// Session 42 has not been initialized in memory
|
||||
const session = sessionManager.getSession(42);
|
||||
expect(session).toBeUndefined();
|
||||
});
|
||||
|
||||
it('should return active session after initializeSession is called', async () => {
|
||||
const { SessionManager } = await import('../../src/services/worker/SessionManager.js');
|
||||
|
||||
const mockDbManager = {
|
||||
getSessionById: () => ({
|
||||
id: 42,
|
||||
content_session_id: 'test-session',
|
||||
project: 'test',
|
||||
user_prompt: 'test prompt',
|
||||
memory_session_id: null,
|
||||
status: 'active',
|
||||
started_at: new Date().toISOString(),
|
||||
completed_at: null,
|
||||
}),
|
||||
getSessionStore: () => ({
|
||||
db: {},
|
||||
clearMemorySessionId: () => {},
|
||||
}),
|
||||
} as any;
|
||||
|
||||
const sessionManager = new SessionManager(mockDbManager);
|
||||
|
||||
// Initialize session (simulates first SDK agent init)
|
||||
sessionManager.initializeSession(42, 'first prompt', 1);
|
||||
|
||||
// Now getSession should return the active session
|
||||
const session = sessionManager.getSession(42);
|
||||
expect(session).toBeDefined();
|
||||
expect(session!.contentSessionId).toBe('test-session');
|
||||
});
|
||||
|
||||
it('should return contextInjected=true pattern for subsequent prompts', async () => {
|
||||
const { SessionManager } = await import('../../src/services/worker/SessionManager.js');
|
||||
|
||||
const mockDbManager = {
|
||||
getSessionById: () => ({
|
||||
id: 42,
|
||||
content_session_id: 'test-session',
|
||||
project: 'test',
|
||||
user_prompt: 'test prompt',
|
||||
memory_session_id: 'sdk-session-abc',
|
||||
status: 'active',
|
||||
started_at: new Date().toISOString(),
|
||||
completed_at: null,
|
||||
}),
|
||||
getSessionStore: () => ({
|
||||
db: {},
|
||||
clearMemorySessionId: () => {},
|
||||
}),
|
||||
} as any;
|
||||
|
||||
const sessionManager = new SessionManager(mockDbManager);
|
||||
|
||||
// Before initialization: contextInjected would be false
|
||||
expect(sessionManager.getSession(42)).toBeUndefined();
|
||||
|
||||
// After initialization: contextInjected would be true
|
||||
sessionManager.initializeSession(42, 'first prompt', 1);
|
||||
expect(sessionManager.getSession(42)).toBeDefined();
|
||||
|
||||
// Second call to initializeSession returns existing session (idempotent)
|
||||
const session2 = sessionManager.initializeSession(42, 'second prompt', 2);
|
||||
expect(session2.contentSessionId).toBe('test-session');
|
||||
expect(session2.userPrompt).toBe('second prompt');
|
||||
expect(session2.lastPromptNumber).toBe(2);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -87,9 +87,9 @@ describe('GracefulShutdown', () => {
|
||||
})
|
||||
};
|
||||
|
||||
const mockChromaServer = {
|
||||
const mockChromaMcpManager = {
|
||||
stop: mock(async () => {
|
||||
callOrder.push('chromaServer.stop');
|
||||
callOrder.push('chromaMcpManager.stop');
|
||||
})
|
||||
};
|
||||
|
||||
@@ -102,7 +102,7 @@ describe('GracefulShutdown', () => {
|
||||
sessionManager: mockSessionManager,
|
||||
mcpClient: mockMcpClient,
|
||||
dbManager: mockDbManager,
|
||||
chromaServer: mockChromaServer
|
||||
chromaMcpManager: mockChromaMcpManager
|
||||
};
|
||||
|
||||
await performGracefulShutdown(config);
|
||||
@@ -112,7 +112,7 @@ describe('GracefulShutdown', () => {
|
||||
expect(callOrder).toContain('serverClose');
|
||||
expect(callOrder).toContain('sessionManager.shutdownAll');
|
||||
expect(callOrder).toContain('mcpClient.close');
|
||||
expect(callOrder).toContain('chromaServer.stop');
|
||||
expect(callOrder).toContain('chromaMcpManager.stop');
|
||||
expect(callOrder).toContain('dbManager.close');
|
||||
|
||||
// Verify server closes before session manager
|
||||
@@ -125,7 +125,7 @@ describe('GracefulShutdown', () => {
|
||||
expect(callOrder.indexOf('mcpClient.close')).toBeLessThan(callOrder.indexOf('dbManager.close'));
|
||||
|
||||
// Verify Chroma stops before DB closes
|
||||
expect(callOrder.indexOf('chromaServer.stop')).toBeLessThan(callOrder.indexOf('dbManager.close'));
|
||||
expect(callOrder.indexOf('chromaMcpManager.stop')).toBeLessThan(callOrder.indexOf('dbManager.close'));
|
||||
});
|
||||
|
||||
it('should remove PID file during shutdown', async () => {
|
||||
@@ -216,9 +216,9 @@ describe('GracefulShutdown', () => {
|
||||
})
|
||||
};
|
||||
|
||||
const mockChromaServer = {
|
||||
const mockChromaMcpManager = {
|
||||
stop: mock(async () => {
|
||||
callOrder.push('chromaServer');
|
||||
callOrder.push('chromaMcpManager');
|
||||
})
|
||||
};
|
||||
|
||||
@@ -227,12 +227,12 @@ describe('GracefulShutdown', () => {
|
||||
sessionManager: mockSessionManager,
|
||||
mcpClient: mockMcpClient,
|
||||
dbManager: mockDbManager,
|
||||
chromaServer: mockChromaServer
|
||||
chromaMcpManager: mockChromaMcpManager
|
||||
};
|
||||
|
||||
await performGracefulShutdown(config);
|
||||
|
||||
expect(callOrder).toEqual(['sessionManager', 'mcpClient', 'chromaServer', 'dbManager']);
|
||||
expect(callOrder).toEqual(['sessionManager', 'mcpClient', 'chromaMcpManager', 'dbManager']);
|
||||
});
|
||||
|
||||
it('should handle shutdown when PID file does not exist', async () => {
|
||||
|
||||
@@ -2,7 +2,9 @@ import { describe, it, expect, beforeEach, afterEach, mock } from 'bun:test';
|
||||
import {
|
||||
isPortInUse,
|
||||
waitForHealth,
|
||||
waitForPortFree
|
||||
waitForPortFree,
|
||||
getInstalledPluginVersion,
|
||||
checkVersionMatch
|
||||
} from '../../src/services/infrastructure/index.js';
|
||||
|
||||
describe('HealthMonitor', () => {
|
||||
@@ -122,6 +124,65 @@ describe('HealthMonitor', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('getInstalledPluginVersion', () => {
|
||||
it('should return a valid semver string', () => {
|
||||
const version = getInstalledPluginVersion();
|
||||
|
||||
// Should be a string matching semver pattern or 'unknown'
|
||||
if (version !== 'unknown') {
|
||||
expect(version).toMatch(/^\d+\.\d+\.\d+/);
|
||||
}
|
||||
});
|
||||
|
||||
it('should not throw on ENOENT (graceful degradation)', () => {
|
||||
// The function handles ENOENT internally — should not throw
|
||||
// If package.json exists, it returns the version; if not, 'unknown'
|
||||
expect(() => getInstalledPluginVersion()).not.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('checkVersionMatch', () => {
|
||||
it('should assume match when worker version is unavailable', async () => {
|
||||
global.fetch = mock(() => Promise.reject(new Error('ECONNREFUSED')));
|
||||
|
||||
const result = await checkVersionMatch(39999);
|
||||
|
||||
expect(result.matches).toBe(true);
|
||||
expect(result.workerVersion).toBeNull();
|
||||
});
|
||||
|
||||
it('should detect version mismatch', async () => {
|
||||
global.fetch = mock(() => Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ version: '0.0.0-definitely-wrong' })
|
||||
} as Response));
|
||||
|
||||
const result = await checkVersionMatch(37777);
|
||||
|
||||
// Unless the plugin version is also '0.0.0-definitely-wrong', this should be a mismatch
|
||||
const pluginVersion = getInstalledPluginVersion();
|
||||
if (pluginVersion !== 'unknown' && pluginVersion !== '0.0.0-definitely-wrong') {
|
||||
expect(result.matches).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it('should detect version match', async () => {
|
||||
const pluginVersion = getInstalledPluginVersion();
|
||||
if (pluginVersion === 'unknown') return; // Skip if can't read plugin version
|
||||
|
||||
global.fetch = mock(() => Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ version: pluginVersion })
|
||||
} as Response));
|
||||
|
||||
const result = await checkVersionMatch(37777);
|
||||
|
||||
expect(result.matches).toBe(true);
|
||||
expect(result.pluginVersion).toBe(pluginVersion);
|
||||
expect(result.workerVersion).toBe(pluginVersion);
|
||||
});
|
||||
});
|
||||
|
||||
describe('waitForPortFree', () => {
|
||||
it('should return true immediately when port is already free', async () => {
|
||||
global.fetch = mock(() => Promise.reject(new Error('ECONNREFUSED')));
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
import { describe, it, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { mkdirSync, writeFileSync, rmSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
import { isPluginDisabledInClaudeSettings } from '../../src/shared/plugin-state.js';
|
||||
|
||||
/**
|
||||
* Tests for isPluginDisabledInClaudeSettings() (#781).
|
||||
*
|
||||
* The function reads CLAUDE_CONFIG_DIR/settings.json and checks if
|
||||
* enabledPlugins["claude-mem@thedotmack"] === false.
|
||||
*
|
||||
* We test by setting CLAUDE_CONFIG_DIR to a temp directory with mock settings.
|
||||
*/
|
||||
|
||||
let tempDir: string;
|
||||
let originalClaudeConfigDir: string | undefined;
|
||||
|
||||
beforeEach(() => {
|
||||
tempDir = join(tmpdir(), `plugin-disabled-test-${Date.now()}-${Math.random().toString(36).slice(2)}`);
|
||||
mkdirSync(tempDir, { recursive: true });
|
||||
originalClaudeConfigDir = process.env.CLAUDE_CONFIG_DIR;
|
||||
process.env.CLAUDE_CONFIG_DIR = tempDir;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (originalClaudeConfigDir !== undefined) {
|
||||
process.env.CLAUDE_CONFIG_DIR = originalClaudeConfigDir;
|
||||
} else {
|
||||
delete process.env.CLAUDE_CONFIG_DIR;
|
||||
}
|
||||
try {
|
||||
rmSync(tempDir, { recursive: true, force: true });
|
||||
} catch {
|
||||
// Ignore cleanup errors
|
||||
}
|
||||
});
|
||||
|
||||
describe('isPluginDisabledInClaudeSettings (#781)', () => {
|
||||
it('should return false when settings.json does not exist', () => {
|
||||
expect(isPluginDisabledInClaudeSettings()).toBe(false);
|
||||
});
|
||||
|
||||
it('should return false when plugin is explicitly enabled', () => {
|
||||
const settings = {
|
||||
enabledPlugins: {
|
||||
'claude-mem@thedotmack': true
|
||||
}
|
||||
};
|
||||
writeFileSync(join(tempDir, 'settings.json'), JSON.stringify(settings));
|
||||
expect(isPluginDisabledInClaudeSettings()).toBe(false);
|
||||
});
|
||||
|
||||
it('should return true when plugin is explicitly disabled', () => {
|
||||
const settings = {
|
||||
enabledPlugins: {
|
||||
'claude-mem@thedotmack': false
|
||||
}
|
||||
};
|
||||
writeFileSync(join(tempDir, 'settings.json'), JSON.stringify(settings));
|
||||
expect(isPluginDisabledInClaudeSettings()).toBe(true);
|
||||
});
|
||||
|
||||
it('should return false when enabledPlugins key is missing', () => {
|
||||
const settings = {
|
||||
permissions: { allow: [] }
|
||||
};
|
||||
writeFileSync(join(tempDir, 'settings.json'), JSON.stringify(settings));
|
||||
expect(isPluginDisabledInClaudeSettings()).toBe(false);
|
||||
});
|
||||
|
||||
it('should return false when plugin key is absent from enabledPlugins', () => {
|
||||
const settings = {
|
||||
enabledPlugins: {
|
||||
'other-plugin@marketplace': true
|
||||
}
|
||||
};
|
||||
writeFileSync(join(tempDir, 'settings.json'), JSON.stringify(settings));
|
||||
expect(isPluginDisabledInClaudeSettings()).toBe(false);
|
||||
});
|
||||
|
||||
it('should return false when settings.json contains invalid JSON', () => {
|
||||
writeFileSync(join(tempDir, 'settings.json'), '{ invalid json }}}');
|
||||
expect(isPluginDisabledInClaudeSettings()).toBe(false);
|
||||
});
|
||||
|
||||
it('should return false when settings.json is empty', () => {
|
||||
writeFileSync(join(tempDir, 'settings.json'), '');
|
||||
expect(isPluginDisabledInClaudeSettings()).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,121 @@
|
||||
import { describe, it, expect } from 'bun:test';
|
||||
import { readFileSync, existsSync } from 'fs';
|
||||
import path from 'path';
|
||||
import { fileURLToPath } from 'url';
|
||||
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const projectRoot = path.resolve(__dirname, '../..');
|
||||
|
||||
/**
|
||||
* Regression tests for plugin distribution completeness.
|
||||
* Ensures all required files (skills, hooks, manifests) are present
|
||||
* and correctly structured for end-user installs.
|
||||
*
|
||||
* Prevents issue #1187 (missing skills/ directory after install).
|
||||
*/
|
||||
describe('Plugin Distribution - Skills', () => {
|
||||
const skillPath = path.join(projectRoot, 'plugin/skills/mem-search/SKILL.md');
|
||||
|
||||
it('should include plugin/skills/mem-search/SKILL.md', () => {
|
||||
expect(existsSync(skillPath)).toBe(true);
|
||||
});
|
||||
|
||||
it('should have valid YAML frontmatter with name and description', () => {
|
||||
const content = readFileSync(skillPath, 'utf-8');
|
||||
|
||||
// Must start with YAML frontmatter
|
||||
expect(content.startsWith('---\n')).toBe(true);
|
||||
|
||||
// Extract frontmatter
|
||||
const frontmatterEnd = content.indexOf('\n---\n', 4);
|
||||
expect(frontmatterEnd).toBeGreaterThan(0);
|
||||
|
||||
const frontmatter = content.slice(4, frontmatterEnd);
|
||||
expect(frontmatter).toContain('name:');
|
||||
expect(frontmatter).toContain('description:');
|
||||
});
|
||||
|
||||
it('should reference the 3-layer search workflow', () => {
|
||||
const content = readFileSync(skillPath, 'utf-8');
|
||||
// The skill must document the search → timeline → get_observations workflow
|
||||
expect(content).toContain('search');
|
||||
expect(content).toContain('timeline');
|
||||
expect(content).toContain('get_observations');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Plugin Distribution - Required Files', () => {
|
||||
const requiredFiles = [
|
||||
'plugin/hooks/hooks.json',
|
||||
'plugin/.claude-plugin/plugin.json',
|
||||
'plugin/skills/mem-search/SKILL.md',
|
||||
];
|
||||
|
||||
for (const filePath of requiredFiles) {
|
||||
it(`should include ${filePath}`, () => {
|
||||
const fullPath = path.join(projectRoot, filePath);
|
||||
expect(existsSync(fullPath)).toBe(true);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('Plugin Distribution - hooks.json Integrity', () => {
|
||||
it('should have valid JSON in hooks.json', () => {
|
||||
const hooksPath = path.join(projectRoot, 'plugin/hooks/hooks.json');
|
||||
const content = readFileSync(hooksPath, 'utf-8');
|
||||
const parsed = JSON.parse(content);
|
||||
expect(parsed.hooks).toBeDefined();
|
||||
});
|
||||
|
||||
it('should reference CLAUDE_PLUGIN_ROOT in all hook commands', () => {
|
||||
const hooksPath = path.join(projectRoot, 'plugin/hooks/hooks.json');
|
||||
const parsed = JSON.parse(readFileSync(hooksPath, 'utf-8'));
|
||||
|
||||
for (const [eventName, matchers] of Object.entries(parsed.hooks)) {
|
||||
for (const matcher of matchers as any[]) {
|
||||
for (const hook of matcher.hooks) {
|
||||
if (hook.type === 'command') {
|
||||
expect(hook.command).toContain('${CLAUDE_PLUGIN_ROOT}');
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it('should include CLAUDE_PLUGIN_ROOT fallback in all hook commands (#1215)', () => {
|
||||
const hooksPath = path.join(projectRoot, 'plugin/hooks/hooks.json');
|
||||
const parsed = JSON.parse(readFileSync(hooksPath, 'utf-8'));
|
||||
const expectedFallbackPath = '$HOME/.claude/plugins/marketplaces/thedotmack/plugin';
|
||||
|
||||
for (const [eventName, matchers] of Object.entries(parsed.hooks)) {
|
||||
for (const matcher of matchers as any[]) {
|
||||
for (const hook of matcher.hooks) {
|
||||
if (hook.type === 'command') {
|
||||
expect(hook.command).toContain(expectedFallbackPath);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('Plugin Distribution - package.json Files Field', () => {
|
||||
it('should include "plugin" in root package.json files field', () => {
|
||||
const packageJsonPath = path.join(projectRoot, 'package.json');
|
||||
const packageJson = JSON.parse(readFileSync(packageJsonPath, 'utf-8'));
|
||||
expect(packageJson.files).toBeDefined();
|
||||
expect(packageJson.files).toContain('plugin');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Plugin Distribution - Build Script Verification', () => {
|
||||
it('should verify distribution files in build-hooks.js', () => {
|
||||
const buildScriptPath = path.join(projectRoot, 'scripts/build-hooks.js');
|
||||
const content = readFileSync(buildScriptPath, 'utf-8');
|
||||
|
||||
// Build script must check for critical distribution files
|
||||
expect(content).toContain('plugin/skills/mem-search/SKILL.md');
|
||||
expect(content).toContain('plugin/hooks/hooks.json');
|
||||
expect(content).toContain('plugin/.claude-plugin/plugin.json');
|
||||
});
|
||||
});
|
||||
@@ -1,6 +1,7 @@
|
||||
import { describe, it, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { existsSync, readFileSync } from 'fs';
|
||||
import { existsSync, readFileSync, mkdirSync, writeFileSync, rmSync } from 'fs';
|
||||
import { homedir } from 'os';
|
||||
import { tmpdir } from 'os';
|
||||
import path from 'path';
|
||||
import {
|
||||
writePidFile,
|
||||
@@ -10,8 +11,11 @@ import {
|
||||
parseElapsedTime,
|
||||
isProcessAlive,
|
||||
cleanStalePidFile,
|
||||
isPidFileRecent,
|
||||
touchPidFile,
|
||||
spawnDaemon,
|
||||
resolveWorkerRuntimePath,
|
||||
runOneTimeChromaMigration,
|
||||
type PidInfo
|
||||
} from '../../src/services/infrastructure/index.js';
|
||||
|
||||
@@ -32,7 +36,6 @@ describe('ProcessManager', () => {
|
||||
afterEach(() => {
|
||||
// Restore original PID file or remove test one
|
||||
if (originalPidContent !== null) {
|
||||
const { writeFileSync } = require('fs');
|
||||
writeFileSync(PID_FILE, originalPidContent);
|
||||
originalPidContent = null;
|
||||
} else {
|
||||
@@ -105,7 +108,6 @@ describe('ProcessManager', () => {
|
||||
});
|
||||
|
||||
it('should return null for corrupted JSON', () => {
|
||||
const { writeFileSync } = require('fs');
|
||||
writeFileSync(PID_FILE, 'not valid json {{{');
|
||||
|
||||
const result = readPidFile();
|
||||
@@ -347,6 +349,58 @@ describe('ProcessManager', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('isPidFileRecent', () => {
|
||||
it('should return true for a recently written PID file', () => {
|
||||
writePidFile({ pid: process.pid, port: 37777, startedAt: new Date().toISOString() });
|
||||
|
||||
// File was just written, should be very recent
|
||||
expect(isPidFileRecent(15000)).toBe(true);
|
||||
});
|
||||
|
||||
it('should return false when PID file does not exist', () => {
|
||||
removePidFile();
|
||||
|
||||
expect(isPidFileRecent(15000)).toBe(false);
|
||||
});
|
||||
|
||||
it('should return false for a very short threshold on a real file', () => {
|
||||
writePidFile({ pid: process.pid, port: 37777, startedAt: new Date().toISOString() });
|
||||
|
||||
// With a 0ms threshold, even a just-written file should be "too old"
|
||||
// (mtime is at least 1ms in the past by the time we check)
|
||||
// Use a negative threshold to guarantee false
|
||||
expect(isPidFileRecent(-1)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('touchPidFile', () => {
|
||||
it('should update mtime of existing PID file', async () => {
|
||||
writePidFile({ pid: process.pid, port: 37777, startedAt: new Date().toISOString() });
|
||||
|
||||
// Wait a bit to ensure measurable mtime difference
|
||||
await new Promise(r => setTimeout(r, 50));
|
||||
|
||||
const statsBefore = require('fs').statSync(PID_FILE);
|
||||
const mtimeBefore = statsBefore.mtimeMs;
|
||||
|
||||
// Wait again to ensure mtime advances
|
||||
await new Promise(r => setTimeout(r, 50));
|
||||
|
||||
touchPidFile();
|
||||
|
||||
const statsAfter = require('fs').statSync(PID_FILE);
|
||||
const mtimeAfter = statsAfter.mtimeMs;
|
||||
|
||||
expect(mtimeAfter).toBeGreaterThanOrEqual(mtimeBefore);
|
||||
});
|
||||
|
||||
it('should not throw when PID file does not exist', () => {
|
||||
removePidFile();
|
||||
|
||||
expect(() => touchPidFile()).not.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('spawnDaemon', () => {
|
||||
it('should use setsid on Linux when available', () => {
|
||||
// setsid should exist at /usr/bin/setsid on Linux
|
||||
@@ -415,4 +469,53 @@ describe('ProcessManager', () => {
|
||||
// This is a logic verification test — actual signal delivery is tested manually
|
||||
});
|
||||
});
|
||||
|
||||
describe('runOneTimeChromaMigration', () => {
|
||||
let testDataDir: string;
|
||||
|
||||
beforeEach(() => {
|
||||
testDataDir = path.join(tmpdir(), `claude-mem-test-${Date.now()}-${Math.random().toString(36).slice(2)}`);
|
||||
mkdirSync(testDataDir, { recursive: true });
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
rmSync(testDataDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('should wipe chroma directory and write marker file', () => {
|
||||
// Create a fake chroma directory with data
|
||||
const chromaDir = path.join(testDataDir, 'chroma');
|
||||
mkdirSync(chromaDir, { recursive: true });
|
||||
writeFileSync(path.join(chromaDir, 'test-data.bin'), 'fake chroma data');
|
||||
|
||||
runOneTimeChromaMigration(testDataDir);
|
||||
|
||||
// Chroma dir should be gone
|
||||
expect(existsSync(chromaDir)).toBe(false);
|
||||
// Marker file should exist
|
||||
expect(existsSync(path.join(testDataDir, '.chroma-cleaned-v10.3'))).toBe(true);
|
||||
});
|
||||
|
||||
it('should skip when marker file already exists (idempotent)', () => {
|
||||
// Write marker file first
|
||||
writeFileSync(path.join(testDataDir, '.chroma-cleaned-v10.3'), 'already done');
|
||||
|
||||
// Create a chroma directory that should NOT be wiped
|
||||
const chromaDir = path.join(testDataDir, 'chroma');
|
||||
mkdirSync(chromaDir, { recursive: true });
|
||||
writeFileSync(path.join(chromaDir, 'important.bin'), 'should survive');
|
||||
|
||||
runOneTimeChromaMigration(testDataDir);
|
||||
|
||||
// Chroma dir should still exist (migration was skipped)
|
||||
expect(existsSync(chromaDir)).toBe(true);
|
||||
expect(existsSync(path.join(chromaDir, 'important.bin'))).toBe(true);
|
||||
});
|
||||
|
||||
it('should handle missing chroma directory gracefully', () => {
|
||||
// No chroma dir exists — should just write marker without error
|
||||
expect(() => runOneTimeChromaMigration(testDataDir)).not.toThrow();
|
||||
expect(existsSync(path.join(testDataDir, '.chroma-cleaned-v10.3'))).toBe(true);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -316,80 +316,52 @@ describe('ChromaSync Vector Sync Integration', () => {
|
||||
/**
|
||||
* Regression test for GitHub Issue #761:
|
||||
* "Feature Request: Option to disable Chroma (RAM usage / zombie processes)"
|
||||
*
|
||||
*
|
||||
* Root cause: When connection errors occur (MCP error -32000, Connection closed),
|
||||
* the code was resetting `connected` and `client` but NOT closing the transport,
|
||||
* leaving the chroma-mcp subprocess alive. Each reconnection attempt spawned
|
||||
* a NEW process while old ones accumulated as zombies.
|
||||
*
|
||||
* Fix: Close transport before resetting state in error handlers at:
|
||||
* - ensureCollection() error handling (~line 180)
|
||||
* - queryChroma() error handling (~line 840)
|
||||
*
|
||||
* Fix: Transport lifecycle is now managed by ChromaMcpManager (singleton),
|
||||
* which handles connect/disconnect/cleanup. ChromaSync delegates to it.
|
||||
*/
|
||||
it('should have transport cleanup in connection error handlers', async () => {
|
||||
// This test verifies the fix exists by checking the source code pattern
|
||||
// The actual runtime behavior depends on uvx/chroma availability
|
||||
const { ChromaSync } = await import('../../src/services/sync/ChromaSync.js');
|
||||
const sync = new ChromaSync(testProject);
|
||||
|
||||
// Verify the class has the expected structure
|
||||
const syncAny = sync as any;
|
||||
|
||||
// Initial state should be null/false
|
||||
expect(syncAny.client).toBeNull();
|
||||
expect(syncAny.transport).toBeNull();
|
||||
expect(syncAny.connected).toBe(false);
|
||||
|
||||
// The close() method should properly clean up all state
|
||||
// This is the reference implementation that error handlers should mirror
|
||||
await sync.close();
|
||||
|
||||
expect(syncAny.client).toBeNull();
|
||||
expect(syncAny.transport).toBeNull();
|
||||
expect(syncAny.connected).toBe(false);
|
||||
});
|
||||
|
||||
it('should reset state after close regardless of connection status', async () => {
|
||||
if (!chromaAvailable) {
|
||||
console.log(`Skipping: ${skipReason}`);
|
||||
return;
|
||||
}
|
||||
|
||||
const { ChromaSync } = await import('../../src/services/sync/ChromaSync.js');
|
||||
const sync = new ChromaSync(testProject);
|
||||
const syncAny = sync as any;
|
||||
|
||||
// Try to establish connection (may succeed or fail depending on environment)
|
||||
try {
|
||||
await sync.queryChroma('test', 5);
|
||||
} catch {
|
||||
// Connection or query may fail - that's OK
|
||||
}
|
||||
|
||||
// Regardless of whether connection succeeded, close() must clean up everything
|
||||
await sync.close();
|
||||
|
||||
// After close(), ALL state must be null/false - this prevents zombie processes
|
||||
expect(syncAny.connected).toBe(false);
|
||||
expect(syncAny.client).toBeNull();
|
||||
expect(syncAny.transport).toBeNull();
|
||||
});
|
||||
|
||||
it('should clean up transport in close() method', async () => {
|
||||
const { ChromaSync } = await import('../../src/services/sync/ChromaSync.js');
|
||||
|
||||
// Read the source to verify transport.close() is called
|
||||
// This is a static analysis test - verifies the fix exists
|
||||
it('should have transport cleanup in ChromaMcpManager error handlers', async () => {
|
||||
// ChromaSync now delegates connection management to ChromaMcpManager.
|
||||
// Verify that ChromaMcpManager source includes transport cleanup.
|
||||
const sourceFile = await Bun.file(
|
||||
new URL('../../src/services/sync/ChromaSync.ts', import.meta.url)
|
||||
new URL('../../src/services/sync/ChromaMcpManager.ts', import.meta.url)
|
||||
).text();
|
||||
|
||||
// Verify that error handlers include transport cleanup
|
||||
// The fix adds: if (this.transport) { await this.transport.close(); }
|
||||
expect(sourceFile).toContain('this.transport.close()');
|
||||
|
||||
|
||||
// Verify transport is set to null after close
|
||||
expect(sourceFile).toContain('this.transport = null');
|
||||
|
||||
// Verify connected is set to false after close
|
||||
expect(sourceFile).toContain('this.connected = false');
|
||||
});
|
||||
|
||||
it('should reset state after close regardless of connection status', async () => {
|
||||
// ChromaSync.close() is now a lightweight method that logs and returns.
|
||||
// Connection state is managed by ChromaMcpManager singleton.
|
||||
const { ChromaSync } = await import('../../src/services/sync/ChromaSync.js');
|
||||
const sync = new ChromaSync(testProject);
|
||||
|
||||
// close() should complete without error regardless of state
|
||||
await expect(sync.close()).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
it('should clean up transport in ChromaMcpManager close() method', async () => {
|
||||
// Read the ChromaMcpManager source to verify transport.close() is in the close path
|
||||
const sourceFile = await Bun.file(
|
||||
new URL('../../src/services/sync/ChromaMcpManager.ts', import.meta.url)
|
||||
).text();
|
||||
|
||||
// Verify the close/disconnect method properly cleans up transport
|
||||
expect(sourceFile).toContain('await this.transport.close()');
|
||||
expect(sourceFile).toContain('this.transport = null');
|
||||
expect(sourceFile).toContain('this.connected = false');
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -45,6 +45,12 @@ describe('Hook Execution E2E', () => {
|
||||
getMcpReady: () => true,
|
||||
onShutdown: mock(() => Promise.resolve()),
|
||||
onRestart: mock(() => Promise.resolve()),
|
||||
workerPath: '/test/worker-service.cjs',
|
||||
getAiStatus: () => ({
|
||||
provider: 'claude',
|
||||
authMethod: 'cli',
|
||||
lastInteraction: null,
|
||||
}),
|
||||
};
|
||||
|
||||
testPort = 40000 + Math.floor(Math.random() * 10000);
|
||||
@@ -96,6 +102,8 @@ describe('Hook Execution E2E', () => {
|
||||
getMcpReady: () => false,
|
||||
onShutdown: mock(() => Promise.resolve()),
|
||||
onRestart: mock(() => Promise.resolve()),
|
||||
workerPath: '/test/worker-service.cjs',
|
||||
getAiStatus: () => ({ provider: 'claude', authMethod: 'cli', lastInteraction: null }),
|
||||
};
|
||||
|
||||
server = new Server(uninitializedOptions);
|
||||
@@ -157,6 +165,8 @@ describe('Hook Execution E2E', () => {
|
||||
getMcpReady: () => true,
|
||||
onShutdown: mock(() => Promise.resolve()),
|
||||
onRestart: mock(() => Promise.resolve()),
|
||||
workerPath: '/test/worker-service.cjs',
|
||||
getAiStatus: () => ({ provider: 'claude', authMethod: 'cli', lastInteraction: null }),
|
||||
};
|
||||
|
||||
server = new Server(dynamicOptions);
|
||||
|
||||
@@ -45,6 +45,12 @@ describe('Worker API Endpoints Integration', () => {
|
||||
getMcpReady: () => true,
|
||||
onShutdown: mock(() => Promise.resolve()),
|
||||
onRestart: mock(() => Promise.resolve()),
|
||||
workerPath: '/test/worker-service.cjs',
|
||||
getAiStatus: () => ({
|
||||
provider: 'claude',
|
||||
authMethod: 'cli',
|
||||
lastInteraction: null,
|
||||
}),
|
||||
};
|
||||
|
||||
testPort = 40000 + Math.floor(Math.random() * 10000);
|
||||
@@ -88,6 +94,8 @@ describe('Worker API Endpoints Integration', () => {
|
||||
getMcpReady: () => false,
|
||||
onShutdown: mock(() => Promise.resolve()),
|
||||
onRestart: mock(() => Promise.resolve()),
|
||||
workerPath: '/test/worker-service.cjs',
|
||||
getAiStatus: () => ({ provider: 'claude', authMethod: 'cli', lastInteraction: null }),
|
||||
};
|
||||
|
||||
server = new Server(uninitOptions);
|
||||
@@ -121,6 +129,8 @@ describe('Worker API Endpoints Integration', () => {
|
||||
getMcpReady: () => false,
|
||||
onShutdown: mock(() => Promise.resolve()),
|
||||
onRestart: mock(() => Promise.resolve()),
|
||||
workerPath: '/test/worker-service.cjs',
|
||||
getAiStatus: () => ({ provider: 'claude', authMethod: 'cli', lastInteraction: null }),
|
||||
};
|
||||
|
||||
server = new Server(uninitOptions);
|
||||
@@ -236,6 +246,8 @@ describe('Worker API Endpoints Integration', () => {
|
||||
getMcpReady: () => true,
|
||||
onShutdown: mock(() => Promise.resolve()),
|
||||
onRestart: mock(() => Promise.resolve()),
|
||||
workerPath: '/test/worker-service.cjs',
|
||||
getAiStatus: () => ({ provider: 'claude', authMethod: 'cli', lastInteraction: null }),
|
||||
};
|
||||
|
||||
server = new Server(dynamicOptions);
|
||||
@@ -260,6 +272,8 @@ describe('Worker API Endpoints Integration', () => {
|
||||
getMcpReady: () => mcpReady,
|
||||
onShutdown: mock(() => Promise.resolve()),
|
||||
onRestart: mock(() => Promise.resolve()),
|
||||
workerPath: '/test/worker-service.cjs',
|
||||
getAiStatus: () => ({ provider: 'claude', authMethod: 'cli', lastInteraction: null }),
|
||||
};
|
||||
|
||||
server = new Server(dynamicOptions);
|
||||
|
||||
@@ -37,6 +37,7 @@ const EXCLUDED_PATTERNS = [
|
||||
/user-message-hook\.ts$/, // Deprecated - kept for reference only, not registered in hooks.json
|
||||
/cli\/hook-command\.ts$/, // CLI hook command uses console.log/error for hook protocol output
|
||||
/cli\/handlers\/user-message\.ts$/, // User message handler uses console.error for user-visible context
|
||||
/services\/transcripts\/cli\.ts$/, // CLI transcript subcommands use console.log for user-visible interactive output
|
||||
];
|
||||
|
||||
// Files that should always use logger (core business logic)
|
||||
|
||||
@@ -32,6 +32,12 @@ describe('Server', () => {
|
||||
getMcpReady: () => true,
|
||||
onShutdown: mock(() => Promise.resolve()),
|
||||
onRestart: mock(() => Promise.resolve()),
|
||||
workerPath: '/test/worker-service.cjs',
|
||||
getAiStatus: () => ({
|
||||
provider: 'claude',
|
||||
authMethod: 'cli',
|
||||
lastInteraction: null,
|
||||
}),
|
||||
};
|
||||
});
|
||||
|
||||
@@ -269,6 +275,8 @@ describe('Server', () => {
|
||||
getMcpReady: () => true,
|
||||
onShutdown: mock(() => Promise.resolve()),
|
||||
onRestart: mock(() => Promise.resolve()),
|
||||
workerPath: '/test/worker-service.cjs',
|
||||
getAiStatus: () => ({ provider: 'claude', authMethod: 'cli', lastInteraction: null }),
|
||||
};
|
||||
|
||||
server = new Server(dynamicOptions);
|
||||
@@ -326,6 +334,8 @@ describe('Server', () => {
|
||||
getMcpReady: () => false,
|
||||
onShutdown: mock(() => Promise.resolve()),
|
||||
onRestart: mock(() => Promise.resolve()),
|
||||
workerPath: '/test/worker-service.cjs',
|
||||
getAiStatus: () => ({ provider: 'claude', authMethod: 'cli', lastInteraction: null }),
|
||||
};
|
||||
|
||||
server = new Server(uninitializedOptions);
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
/**
|
||||
* Tests for readLastLines() — tail-read function for /api/logs endpoint (#1203)
|
||||
*
|
||||
* Verifies that log files are read from the end without loading the entire
|
||||
* file into memory, preventing OOM on large log files.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { writeFileSync, mkdirSync, rmSync, existsSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
import { readLastLines } from '../../src/services/worker/http/routes/LogsRoutes.js';
|
||||
|
||||
describe('readLastLines (#1203 OOM fix)', () => {
|
||||
const testDir = join(tmpdir(), `claude-mem-logs-test-${Date.now()}`);
|
||||
const testFile = join(testDir, 'test.log');
|
||||
|
||||
beforeEach(() => {
|
||||
mkdirSync(testDir, { recursive: true });
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (existsSync(testDir)) {
|
||||
rmSync(testDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('should return empty string for empty file', () => {
|
||||
writeFileSync(testFile, '', 'utf-8');
|
||||
const result = readLastLines(testFile, 10);
|
||||
expect(result.lines).toBe('');
|
||||
expect(result.totalEstimate).toBe(0);
|
||||
});
|
||||
|
||||
it('should return all lines when file has fewer lines than requested', () => {
|
||||
writeFileSync(testFile, 'line1\nline2\nline3\n', 'utf-8');
|
||||
const result = readLastLines(testFile, 10);
|
||||
expect(result.lines).toBe('line1\nline2\nline3');
|
||||
expect(result.totalEstimate).toBe(3);
|
||||
});
|
||||
|
||||
it('should return exactly the last N lines', () => {
|
||||
const lines = Array.from({ length: 20 }, (_, i) => `line${i + 1}`);
|
||||
writeFileSync(testFile, lines.join('\n') + '\n', 'utf-8');
|
||||
|
||||
const result = readLastLines(testFile, 5);
|
||||
expect(result.lines).toBe('line16\nline17\nline18\nline19\nline20');
|
||||
});
|
||||
|
||||
it('should return single line when requested', () => {
|
||||
writeFileSync(testFile, 'first\nsecond\nthird\n', 'utf-8');
|
||||
const result = readLastLines(testFile, 1);
|
||||
expect(result.lines).toBe('third');
|
||||
});
|
||||
|
||||
it('should handle file without trailing newline', () => {
|
||||
writeFileSync(testFile, 'line1\nline2\nline3', 'utf-8');
|
||||
const result = readLastLines(testFile, 2);
|
||||
expect(result.lines).toBe('line2\nline3');
|
||||
});
|
||||
|
||||
it('should handle single line file', () => {
|
||||
writeFileSync(testFile, 'only line\n', 'utf-8');
|
||||
const result = readLastLines(testFile, 5);
|
||||
expect(result.lines).toBe('only line');
|
||||
expect(result.totalEstimate).toBe(1);
|
||||
});
|
||||
|
||||
it('should handle file with exactly requested number of lines', () => {
|
||||
writeFileSync(testFile, 'a\nb\nc\n', 'utf-8');
|
||||
const result = readLastLines(testFile, 3);
|
||||
expect(result.lines).toBe('a\nb\nc');
|
||||
});
|
||||
|
||||
it('should work with lines larger than initial chunk size', () => {
|
||||
// Create a file where lines are long enough to exceed the 64KB initial chunk
|
||||
const longLine = 'X'.repeat(10000);
|
||||
const lines = Array.from({ length: 20 }, (_, i) => `${i}:${longLine}`);
|
||||
writeFileSync(testFile, lines.join('\n') + '\n', 'utf-8');
|
||||
|
||||
const result = readLastLines(testFile, 3);
|
||||
const resultLines = result.lines.split('\n');
|
||||
expect(resultLines.length).toBe(3);
|
||||
expect(resultLines[0]).toStartWith('17:');
|
||||
expect(resultLines[1]).toStartWith('18:');
|
||||
expect(resultLines[2]).toStartWith('19:');
|
||||
});
|
||||
|
||||
it('should provide accurate totalEstimate when entire file is read', () => {
|
||||
const lines = Array.from({ length: 5 }, (_, i) => `line${i}`);
|
||||
writeFileSync(testFile, lines.join('\n') + '\n', 'utf-8');
|
||||
|
||||
const result = readLastLines(testFile, 100);
|
||||
// When file fits in one chunk, totalEstimate should be exact
|
||||
expect(result.totalEstimate).toBe(5);
|
||||
});
|
||||
|
||||
it('should handle requesting zero lines', () => {
|
||||
writeFileSync(testFile, 'line1\nline2\n', 'utf-8');
|
||||
const result = readLastLines(testFile, 0);
|
||||
expect(result.lines).toBe('');
|
||||
});
|
||||
|
||||
it('should handle file with only newlines', () => {
|
||||
writeFileSync(testFile, '\n\n\n', 'utf-8');
|
||||
const result = readLastLines(testFile, 2);
|
||||
const resultLines = result.lines.split('\n');
|
||||
// The last two "lines" before trailing newline are empty strings
|
||||
expect(resultLines.length).toBe(2);
|
||||
});
|
||||
|
||||
it('should not load entire large file for small tail request', () => {
|
||||
// This test verifies the core fix: a file with many lines should
|
||||
// not be fully loaded when only a few lines are requested.
|
||||
// We create a file larger than the initial 64KB chunk.
|
||||
const line = 'A'.repeat(100) + '\n'; // ~101 bytes per line
|
||||
const lineCount = 1000; // ~101KB total
|
||||
writeFileSync(testFile, line.repeat(lineCount), 'utf-8');
|
||||
|
||||
const result = readLastLines(testFile, 5);
|
||||
const resultLines = result.lines.split('\n');
|
||||
expect(resultLines.length).toBe(5);
|
||||
// Each returned line should be our repeated 'A' pattern
|
||||
for (const l of resultLines) {
|
||||
expect(l).toBe('A'.repeat(100));
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -5,11 +5,11 @@ import type { PendingMessageStore, PersistentPendingMessage } from '../../../src
|
||||
|
||||
/**
|
||||
* Mock PendingMessageStore that returns null (empty queue) by default.
|
||||
* Individual tests can override claimAndDelete behavior.
|
||||
* Individual tests can override claimNextMessage behavior.
|
||||
*/
|
||||
function createMockStore(): PendingMessageStore {
|
||||
return {
|
||||
claimAndDelete: mock(() => null),
|
||||
claimNextMessage: mock(() => null),
|
||||
toPendingMessage: mock((msg: PersistentPendingMessage) => ({
|
||||
type: msg.message_type,
|
||||
tool_name: msg.tool_name || undefined,
|
||||
@@ -140,7 +140,7 @@ describe('SessionQueueProcessor', () => {
|
||||
let callCount = 0;
|
||||
|
||||
// Return a message on first call, then null
|
||||
(store.claimAndDelete as any) = mock(() => {
|
||||
(store.claimNextMessage as any) = mock(() => {
|
||||
callCount++;
|
||||
if (callCount === 1) {
|
||||
return createMockMessage({ id: 1 });
|
||||
@@ -170,7 +170,7 @@ describe('SessionQueueProcessor', () => {
|
||||
expect(results).toHaveLength(1);
|
||||
expect(results[0]._persistentId).toBe(1);
|
||||
|
||||
// Store's claimAndDelete should have been called at least twice
|
||||
// Store's claimNextMessage should have been called at least twice
|
||||
// (once returning message, once returning null)
|
||||
expect(callCount).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
@@ -206,7 +206,7 @@ describe('SessionQueueProcessor', () => {
|
||||
const onIdleTimeout = mock(() => {});
|
||||
|
||||
// Return null to trigger wait
|
||||
(store.claimAndDelete as any) = mock(() => null);
|
||||
(store.claimNextMessage as any) = mock(() => null);
|
||||
|
||||
const options: CreateIteratorOptions = {
|
||||
sessionDbId: 123,
|
||||
@@ -242,7 +242,7 @@ describe('SessionQueueProcessor', () => {
|
||||
// First call: return null (queue empty)
|
||||
// After message event: return message
|
||||
// Then return null again
|
||||
(store.claimAndDelete as any) = mock(() => {
|
||||
(store.claimNextMessage as any) = mock(() => {
|
||||
callCount++;
|
||||
if (callCount === 1) {
|
||||
// First check - queue empty, will wait
|
||||
@@ -312,7 +312,7 @@ describe('SessionQueueProcessor', () => {
|
||||
|
||||
it('should clean up event listeners when message received', async () => {
|
||||
// Return a message immediately
|
||||
(store.claimAndDelete as any) = mock(() => createMockMessage({ id: 1 }));
|
||||
(store.claimNextMessage as any) = mock(() => createMockMessage({ id: 1 }));
|
||||
|
||||
const options: CreateIteratorOptions = {
|
||||
sessionDbId: 123,
|
||||
@@ -344,7 +344,7 @@ describe('SessionQueueProcessor', () => {
|
||||
it('should continue after store error with backoff', async () => {
|
||||
let callCount = 0;
|
||||
|
||||
(store.claimAndDelete as any) = mock(() => {
|
||||
(store.claimNextMessage as any) = mock(() => {
|
||||
callCount++;
|
||||
if (callCount === 1) {
|
||||
throw new Error('Database error');
|
||||
@@ -377,7 +377,7 @@ describe('SessionQueueProcessor', () => {
|
||||
});
|
||||
|
||||
it('should exit cleanly if aborted during error backoff', async () => {
|
||||
(store.claimAndDelete as any) = mock(() => {
|
||||
(store.claimNextMessage as any) = mock(() => {
|
||||
throw new Error('Database error');
|
||||
});
|
||||
|
||||
@@ -413,7 +413,7 @@ describe('SessionQueueProcessor', () => {
|
||||
created_at_epoch: 1704067200000
|
||||
});
|
||||
|
||||
(store.claimAndDelete as any) = mock(() => mockPersistentMessage);
|
||||
(store.claimNextMessage as any) = mock(() => mockPersistentMessage);
|
||||
|
||||
const options: CreateIteratorOptions = {
|
||||
sessionDbId: 123,
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { ClaudeMemDatabase } from '../../../src/services/sqlite/Database.js';
|
||||
import { PendingMessageStore } from '../../../src/services/sqlite/PendingMessageStore.js';
|
||||
import { createSDKSession } from '../../../src/services/sqlite/Sessions.js';
|
||||
import type { PendingMessage } from '../../../src/services/worker-types.js';
|
||||
import type { Database } from 'bun:sqlite';
|
||||
|
||||
describe('PendingMessageStore - Self-Healing claimNextMessage', () => {
|
||||
let db: Database;
|
||||
let store: PendingMessageStore;
|
||||
let sessionDbId: number;
|
||||
const CONTENT_SESSION_ID = 'test-self-heal';
|
||||
|
||||
beforeEach(() => {
|
||||
db = new ClaudeMemDatabase(':memory:').db;
|
||||
store = new PendingMessageStore(db, 3);
|
||||
sessionDbId = createSDKSession(db, CONTENT_SESSION_ID, 'test-project', 'Test prompt');
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
db.close();
|
||||
});
|
||||
|
||||
function enqueueMessage(overrides: Partial<PendingMessage> = {}): number {
|
||||
const message: PendingMessage = {
|
||||
type: 'observation',
|
||||
tool_name: 'TestTool',
|
||||
tool_input: { test: 'input' },
|
||||
tool_response: { test: 'response' },
|
||||
prompt_number: 1,
|
||||
...overrides,
|
||||
};
|
||||
return store.enqueue(sessionDbId, CONTENT_SESSION_ID, message);
|
||||
}
|
||||
|
||||
/**
|
||||
* Helper to simulate a stuck processing message by directly updating the DB
|
||||
* to set started_processing_at_epoch to a time in the past (>60s ago)
|
||||
*/
|
||||
function makeMessageStaleProcessing(messageId: number): void {
|
||||
const staleTimestamp = Date.now() - 120_000; // 2 minutes ago (well past 60s threshold)
|
||||
db.run(
|
||||
`UPDATE pending_messages SET status = 'processing', started_processing_at_epoch = ? WHERE id = ?`,
|
||||
[staleTimestamp, messageId]
|
||||
);
|
||||
}
|
||||
|
||||
test('stuck processing messages are recovered on next claim', () => {
|
||||
// Enqueue a message and make it stuck in processing
|
||||
const msgId = enqueueMessage();
|
||||
makeMessageStaleProcessing(msgId);
|
||||
|
||||
// Verify it's stuck (status = processing)
|
||||
const beforeClaim = db.query('SELECT status FROM pending_messages WHERE id = ?').get(msgId) as { status: string };
|
||||
expect(beforeClaim.status).toBe('processing');
|
||||
|
||||
// claimNextMessage should self-heal: reset the stuck message, then claim it
|
||||
const claimed = store.claimNextMessage(sessionDbId);
|
||||
|
||||
expect(claimed).not.toBeNull();
|
||||
expect(claimed!.id).toBe(msgId);
|
||||
// It should now be in 'processing' status again (freshly claimed)
|
||||
const afterClaim = db.query('SELECT status FROM pending_messages WHERE id = ?').get(msgId) as { status: string };
|
||||
expect(afterClaim.status).toBe('processing');
|
||||
});
|
||||
|
||||
test('actively processing messages are NOT recovered', () => {
|
||||
// Enqueue two messages
|
||||
const activeId = enqueueMessage();
|
||||
const pendingId = enqueueMessage();
|
||||
|
||||
// Make the first one actively processing (recent timestamp, NOT stale)
|
||||
const recentTimestamp = Date.now() - 5_000; // 5 seconds ago (well within 60s threshold)
|
||||
db.run(
|
||||
`UPDATE pending_messages SET status = 'processing', started_processing_at_epoch = ? WHERE id = ?`,
|
||||
[recentTimestamp, activeId]
|
||||
);
|
||||
|
||||
// claimNextMessage should NOT reset the active one — should claim the pending one instead
|
||||
const claimed = store.claimNextMessage(sessionDbId);
|
||||
|
||||
expect(claimed).not.toBeNull();
|
||||
expect(claimed!.id).toBe(pendingId);
|
||||
|
||||
// The active message should still be processing
|
||||
const activeMsg = db.query('SELECT status FROM pending_messages WHERE id = ?').get(activeId) as { status: string };
|
||||
expect(activeMsg.status).toBe('processing');
|
||||
});
|
||||
|
||||
test('recovery and claim is atomic within single call', () => {
|
||||
// Enqueue three messages
|
||||
const stuckId = enqueueMessage();
|
||||
const pendingId1 = enqueueMessage();
|
||||
const pendingId2 = enqueueMessage();
|
||||
|
||||
// Make the first one stuck
|
||||
makeMessageStaleProcessing(stuckId);
|
||||
|
||||
// Single claimNextMessage should reset stuck AND claim oldest pending (which is the reset stuck one)
|
||||
const claimed = store.claimNextMessage(sessionDbId);
|
||||
|
||||
expect(claimed).not.toBeNull();
|
||||
// The stuck message was reset to pending, and being oldest, it gets claimed
|
||||
expect(claimed!.id).toBe(stuckId);
|
||||
|
||||
// The other two should still be pending
|
||||
const msg1 = db.query('SELECT status FROM pending_messages WHERE id = ?').get(pendingId1) as { status: string };
|
||||
const msg2 = db.query('SELECT status FROM pending_messages WHERE id = ?').get(pendingId2) as { status: string };
|
||||
expect(msg1.status).toBe('pending');
|
||||
expect(msg2.status).toBe('pending');
|
||||
});
|
||||
|
||||
test('no messages returns null without error', () => {
|
||||
const claimed = store.claimNextMessage(sessionDbId);
|
||||
expect(claimed).toBeNull();
|
||||
});
|
||||
|
||||
test('self-healing only affects the specified session', () => {
|
||||
// Create a second session
|
||||
const session2Id = createSDKSession(db, 'other-session', 'test-project', 'Test');
|
||||
|
||||
// Enqueue and make stuck in session 1
|
||||
const stuckInSession1 = enqueueMessage();
|
||||
makeMessageStaleProcessing(stuckInSession1);
|
||||
|
||||
// Enqueue in session 2
|
||||
const msg: PendingMessage = {
|
||||
type: 'observation',
|
||||
tool_name: 'TestTool',
|
||||
tool_input: { test: 'input' },
|
||||
tool_response: { test: 'response' },
|
||||
prompt_number: 1,
|
||||
};
|
||||
const session2MsgId = store.enqueue(session2Id, 'other-session', msg);
|
||||
makeMessageStaleProcessing(session2MsgId);
|
||||
|
||||
// Claim for session 2 — should only heal session 2's stuck message
|
||||
const claimed = store.claimNextMessage(session2Id);
|
||||
expect(claimed).not.toBeNull();
|
||||
expect(claimed!.id).toBe(session2MsgId);
|
||||
|
||||
// Session 1's stuck message should still be stuck (not healed by session 2's claim)
|
||||
const session1Msg = db.query('SELECT status FROM pending_messages WHERE id = ?').get(stuckInSession1) as { status: string };
|
||||
expect(session1Msg.status).toBe('processing');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,315 @@
|
||||
/**
|
||||
* Tests for MigrationRunner idempotency and schema initialization (#979)
|
||||
*
|
||||
* Mock Justification: NONE (0% mock code)
|
||||
* - Uses real SQLite with ':memory:' — tests actual migration SQL
|
||||
* - Validates idempotency by running migrations multiple times
|
||||
* - Covers the version-conflict scenario from issue #979
|
||||
*
|
||||
* Value: Prevents regression where old DatabaseManager migrations mask core table creation
|
||||
*/
|
||||
import { describe, it, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { Database } from 'bun:sqlite';
|
||||
import { MigrationRunner } from '../../../src/services/sqlite/migrations/runner.js';
|
||||
|
||||
interface TableNameRow {
|
||||
name: string;
|
||||
}
|
||||
|
||||
interface TableColumnInfo {
|
||||
name: string;
|
||||
type: string;
|
||||
notnull: number;
|
||||
}
|
||||
|
||||
interface SchemaVersion {
|
||||
version: number;
|
||||
}
|
||||
|
||||
interface ForeignKeyInfo {
|
||||
table: string;
|
||||
on_update: string;
|
||||
on_delete: string;
|
||||
}
|
||||
|
||||
function getTableNames(db: Database): string[] {
|
||||
const rows = db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%' ORDER BY name").all() as TableNameRow[];
|
||||
return rows.map(r => r.name);
|
||||
}
|
||||
|
||||
function getColumns(db: Database, table: string): TableColumnInfo[] {
|
||||
return db.prepare(`PRAGMA table_info(${table})`).all() as TableColumnInfo[];
|
||||
}
|
||||
|
||||
function getSchemaVersions(db: Database): number[] {
|
||||
const rows = db.prepare('SELECT version FROM schema_versions ORDER BY version').all() as SchemaVersion[];
|
||||
return rows.map(r => r.version);
|
||||
}
|
||||
|
||||
describe('MigrationRunner', () => {
|
||||
let db: Database;
|
||||
|
||||
beforeEach(() => {
|
||||
db = new Database(':memory:');
|
||||
db.run('PRAGMA journal_mode = WAL');
|
||||
db.run('PRAGMA foreign_keys = ON');
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
db.close();
|
||||
});
|
||||
|
||||
describe('fresh database initialization', () => {
|
||||
it('should create all core tables on a fresh database', () => {
|
||||
const runner = new MigrationRunner(db);
|
||||
runner.runAllMigrations();
|
||||
|
||||
const tables = getTableNames(db);
|
||||
expect(tables).toContain('schema_versions');
|
||||
expect(tables).toContain('sdk_sessions');
|
||||
expect(tables).toContain('observations');
|
||||
expect(tables).toContain('session_summaries');
|
||||
expect(tables).toContain('user_prompts');
|
||||
expect(tables).toContain('pending_messages');
|
||||
});
|
||||
|
||||
it('should create sdk_sessions with all expected columns', () => {
|
||||
const runner = new MigrationRunner(db);
|
||||
runner.runAllMigrations();
|
||||
|
||||
const columns = getColumns(db, 'sdk_sessions');
|
||||
const columnNames = columns.map(c => c.name);
|
||||
|
||||
expect(columnNames).toContain('id');
|
||||
expect(columnNames).toContain('content_session_id');
|
||||
expect(columnNames).toContain('memory_session_id');
|
||||
expect(columnNames).toContain('project');
|
||||
expect(columnNames).toContain('status');
|
||||
expect(columnNames).toContain('worker_port');
|
||||
expect(columnNames).toContain('prompt_counter');
|
||||
});
|
||||
|
||||
it('should create observations with all expected columns including content_hash', () => {
|
||||
const runner = new MigrationRunner(db);
|
||||
runner.runAllMigrations();
|
||||
|
||||
const columns = getColumns(db, 'observations');
|
||||
const columnNames = columns.map(c => c.name);
|
||||
|
||||
expect(columnNames).toContain('id');
|
||||
expect(columnNames).toContain('memory_session_id');
|
||||
expect(columnNames).toContain('project');
|
||||
expect(columnNames).toContain('type');
|
||||
expect(columnNames).toContain('title');
|
||||
expect(columnNames).toContain('narrative');
|
||||
expect(columnNames).toContain('prompt_number');
|
||||
expect(columnNames).toContain('discovery_tokens');
|
||||
expect(columnNames).toContain('content_hash');
|
||||
});
|
||||
|
||||
it('should record all migration versions', () => {
|
||||
const runner = new MigrationRunner(db);
|
||||
runner.runAllMigrations();
|
||||
|
||||
const versions = getSchemaVersions(db);
|
||||
// Core set of expected versions
|
||||
expect(versions).toContain(4); // initializeSchema
|
||||
expect(versions).toContain(5); // worker_port
|
||||
expect(versions).toContain(6); // prompt tracking
|
||||
expect(versions).toContain(7); // remove unique constraint
|
||||
expect(versions).toContain(8); // hierarchical fields
|
||||
expect(versions).toContain(9); // text nullable
|
||||
expect(versions).toContain(10); // user_prompts
|
||||
expect(versions).toContain(11); // discovery_tokens
|
||||
expect(versions).toContain(16); // pending_messages
|
||||
expect(versions).toContain(17); // rename columns
|
||||
expect(versions).toContain(19); // repair (noop)
|
||||
expect(versions).toContain(20); // failed_at_epoch
|
||||
expect(versions).toContain(21); // ON UPDATE CASCADE
|
||||
expect(versions).toContain(22); // content_hash
|
||||
});
|
||||
});
|
||||
|
||||
describe('idempotency — running migrations twice', () => {
|
||||
it('should succeed when run twice on the same database', () => {
|
||||
const runner = new MigrationRunner(db);
|
||||
|
||||
// First run
|
||||
runner.runAllMigrations();
|
||||
|
||||
// Second run — must not throw
|
||||
expect(() => runner.runAllMigrations()).not.toThrow();
|
||||
});
|
||||
|
||||
it('should produce identical schema when run twice', () => {
|
||||
const runner = new MigrationRunner(db);
|
||||
runner.runAllMigrations();
|
||||
|
||||
const tablesAfterFirst = getTableNames(db);
|
||||
const versionsAfterFirst = getSchemaVersions(db);
|
||||
|
||||
runner.runAllMigrations();
|
||||
|
||||
const tablesAfterSecond = getTableNames(db);
|
||||
const versionsAfterSecond = getSchemaVersions(db);
|
||||
|
||||
expect(tablesAfterSecond).toEqual(tablesAfterFirst);
|
||||
expect(versionsAfterSecond).toEqual(versionsAfterFirst);
|
||||
});
|
||||
});
|
||||
|
||||
describe('issue #979 — old DatabaseManager version conflict', () => {
|
||||
it('should create core tables even when old migration versions 1-7 are in schema_versions', () => {
|
||||
// Simulate the old DatabaseManager having applied its migrations 1-7
|
||||
// (which are completely different operations with the same version numbers)
|
||||
db.run(`
|
||||
CREATE TABLE IF NOT EXISTS schema_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
version INTEGER UNIQUE NOT NULL,
|
||||
applied_at TEXT NOT NULL
|
||||
)
|
||||
`);
|
||||
|
||||
const now = new Date().toISOString();
|
||||
for (let v = 1; v <= 7; v++) {
|
||||
db.prepare('INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)').run(v, now);
|
||||
}
|
||||
|
||||
// Now run MigrationRunner — core tables MUST still be created
|
||||
const runner = new MigrationRunner(db);
|
||||
runner.runAllMigrations();
|
||||
|
||||
const tables = getTableNames(db);
|
||||
expect(tables).toContain('sdk_sessions');
|
||||
expect(tables).toContain('observations');
|
||||
expect(tables).toContain('session_summaries');
|
||||
expect(tables).toContain('user_prompts');
|
||||
expect(tables).toContain('pending_messages');
|
||||
});
|
||||
|
||||
it('should handle version 5 conflict (old=drop tables, new=add column) correctly', () => {
|
||||
// Old migration 5 drops streaming_sessions/observation_queue
|
||||
// New migration 5 adds worker_port column to sdk_sessions
|
||||
// With old version 5 already recorded, MigrationRunner must still add the column
|
||||
db.run(`
|
||||
CREATE TABLE IF NOT EXISTS schema_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
version INTEGER UNIQUE NOT NULL,
|
||||
applied_at TEXT NOT NULL
|
||||
)
|
||||
`);
|
||||
db.prepare('INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)').run(5, new Date().toISOString());
|
||||
|
||||
const runner = new MigrationRunner(db);
|
||||
runner.runAllMigrations();
|
||||
|
||||
// sdk_sessions should exist and have worker_port (added by later migrations even if v5 is skipped)
|
||||
const columns = getColumns(db, 'sdk_sessions');
|
||||
const columnNames = columns.map(c => c.name);
|
||||
expect(columnNames).toContain('content_session_id');
|
||||
});
|
||||
});
|
||||
|
||||
describe('crash recovery — leftover temp tables', () => {
|
||||
it('should handle leftover session_summaries_new table from crashed migration 7', () => {
|
||||
const runner = new MigrationRunner(db);
|
||||
runner.runAllMigrations();
|
||||
|
||||
// Simulate a leftover temp table from a crash
|
||||
db.run(`
|
||||
CREATE TABLE session_summaries_new (
|
||||
id INTEGER PRIMARY KEY,
|
||||
test TEXT
|
||||
)
|
||||
`);
|
||||
|
||||
// Remove version 7 so migration tries to re-run
|
||||
db.prepare('DELETE FROM schema_versions WHERE version = 7').run();
|
||||
|
||||
// Re-run should handle the leftover table gracefully
|
||||
expect(() => runner.runAllMigrations()).not.toThrow();
|
||||
});
|
||||
|
||||
it('should handle leftover observations_new table from crashed migration 9', () => {
|
||||
const runner = new MigrationRunner(db);
|
||||
runner.runAllMigrations();
|
||||
|
||||
// Simulate a leftover temp table from a crash
|
||||
db.run(`
|
||||
CREATE TABLE observations_new (
|
||||
id INTEGER PRIMARY KEY,
|
||||
test TEXT
|
||||
)
|
||||
`);
|
||||
|
||||
// Remove version 9 so migration tries to re-run
|
||||
db.prepare('DELETE FROM schema_versions WHERE version = 9').run();
|
||||
|
||||
// Re-run should handle the leftover table gracefully
|
||||
expect(() => runner.runAllMigrations()).not.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('ON UPDATE CASCADE FK constraints', () => {
|
||||
it('should have ON UPDATE CASCADE on observations FK after migration 21', () => {
|
||||
const runner = new MigrationRunner(db);
|
||||
runner.runAllMigrations();
|
||||
|
||||
const fks = db.prepare('PRAGMA foreign_key_list(observations)').all() as ForeignKeyInfo[];
|
||||
const memorySessionFk = fks.find(fk => fk.table === 'sdk_sessions');
|
||||
|
||||
expect(memorySessionFk).toBeDefined();
|
||||
expect(memorySessionFk!.on_update).toBe('CASCADE');
|
||||
expect(memorySessionFk!.on_delete).toBe('CASCADE');
|
||||
});
|
||||
|
||||
it('should have ON UPDATE CASCADE on session_summaries FK after migration 21', () => {
|
||||
const runner = new MigrationRunner(db);
|
||||
runner.runAllMigrations();
|
||||
|
||||
const fks = db.prepare('PRAGMA foreign_key_list(session_summaries)').all() as ForeignKeyInfo[];
|
||||
const memorySessionFk = fks.find(fk => fk.table === 'sdk_sessions');
|
||||
|
||||
expect(memorySessionFk).toBeDefined();
|
||||
expect(memorySessionFk!.on_update).toBe('CASCADE');
|
||||
expect(memorySessionFk!.on_delete).toBe('CASCADE');
|
||||
});
|
||||
});
|
||||
|
||||
describe('data integrity during migration', () => {
|
||||
it('should preserve existing data through all migrations', () => {
|
||||
const runner = new MigrationRunner(db);
|
||||
runner.runAllMigrations();
|
||||
|
||||
// Insert test data
|
||||
const now = new Date().toISOString();
|
||||
const epoch = Date.now();
|
||||
|
||||
db.prepare(`
|
||||
INSERT INTO sdk_sessions (content_session_id, memory_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
`).run('test-content-1', 'test-memory-1', 'test-project', now, epoch, 'active');
|
||||
|
||||
db.prepare(`
|
||||
INSERT INTO observations (memory_session_id, project, text, type, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
`).run('test-memory-1', 'test-project', 'test observation', 'discovery', now, epoch);
|
||||
|
||||
db.prepare(`
|
||||
INSERT INTO session_summaries (memory_session_id, project, request, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
`).run('test-memory-1', 'test-project', 'test request', now, epoch);
|
||||
|
||||
// Run migrations again — data should survive
|
||||
runner.runAllMigrations();
|
||||
|
||||
const sessions = db.prepare('SELECT COUNT(*) as count FROM sdk_sessions').get() as { count: number };
|
||||
const observations = db.prepare('SELECT COUNT(*) as count FROM observations').get() as { count: number };
|
||||
const summaries = db.prepare('SELECT COUNT(*) as count FROM session_summaries').get() as { count: number };
|
||||
|
||||
expect(sessions.count).toBe(1);
|
||||
expect(observations.count).toBe(1);
|
||||
expect(summaries.count).toBe(1);
|
||||
});
|
||||
});
|
||||
});
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user