Compare commits
18 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Alex Newman
|
a32151a166 | ||
|
Alex Newman
|
97ea9e45fc | ||
|
Alex Newman
|
ecb09df420 | ||
|
Alex Newman
|
6c7acfbc1c | ||
|
Alex Newman
|
44a7b2fcb9 | ||
|
Alex Newman
|
7015301d8f | ||
|
Alex Newman
|
a5e86ad4ab | ||
|
Alex Newman
|
d93bde059e | ||
|
Alex Newman
|
d60ae14a9b | ||
|
Alex Newman
|
272391ec9d | ||
|
Alex Newman
|
0e502dbd21 | ||
|
Alex Newman
|
9ab119932a | ||
|
Alex Newman
|
50d1dfb7ee | ||
|
Alex Newman
|
0b034af98b | ||
|
Alex Newman
|
b43ad00f8b | ||
|
Alex Newman
|
dd1b812443 | ||
|
Alex Newman
|
ad3d236cec | ||
|
Alex Newman
|
494f681cbf |
@@ -10,7 +10,7 @@
|
||||
"plugins": [
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "10.4.1",
|
||||
"version": "10.5.3",
|
||||
"source": "./plugin",
|
||||
"description": "Persistent memory system for Claude Code - context compression across sessions"
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@ datasets/
|
||||
node_modules/
|
||||
dist/
|
||||
!installer/dist/
|
||||
**/_tree-sitter/
|
||||
*.log
|
||||
.DS_Store
|
||||
.env
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"MD013": false
|
||||
}
|
||||
@@ -0,0 +1,736 @@
|
||||
# Plan: NPX Distribution + Universal IDE/CLI Coverage for claude-mem
|
||||
|
||||
## Problem
|
||||
|
||||
1. **Installation is slow and fragile**: Current install clones the full git repo, runs `npm install`, and builds from source. The npm package already ships pre-built artifacts.
|
||||
|
||||
2. **IDE coverage is limited**: claude-mem only supports Claude Code (plugin) and Cursor (hooks installer). The AI coding tools landscape has exploded — Gemini CLI (95k stars), OpenCode (110k stars), Windsurf (~1M users), Codex CLI, Antigravity, Goose, Crush, Copilot CLI, and more all support extensibility.
|
||||
|
||||
## Key Insights
|
||||
|
||||
- **npm package already has everything**: `plugin/` directory ships pre-built. No git clone or build needed.
|
||||
- **Transcript watcher already exists**: `src/services/transcripts/` has a fully built schema-based JSONL tailer. It just needs schemas for more tools.
|
||||
- **3 integration tiers exist**: (1) Hook/plugin-based (Claude Code, Gemini CLI, OpenCode, Windsurf, Codex CLI, OpenClaw), (2) MCP-based (Cursor, Copilot CLI, Antigravity, Goose, Crush, Roo Code), (3) Transcript-based (anything with structured log files).
|
||||
- **OpenClaw plugin already built**: Full plugin at `openclaw/src/index.ts` (1000+ lines). Needs to be wired into the npx installer.
|
||||
- **Gemini CLI is architecturally near-identical to Claude Code**: 11 lifecycle hooks, JSON via stdin/stdout, exit code 0/2 convention, `GEMINI.md` context files, `~/.gemini/settings.json`. This is the easiest high-value integration.
|
||||
- **OpenCode has the richest plugin system**: 20+ hook events across 12 categories, JS/TS plugin modules, custom tool creation, MCP support. 110k stars — largest open-source AI CLI.
|
||||
- **`npx skills` by Vercel supports 41 agents** — proving the multi-IDE installer UX works. Their agent detection pattern (check if config dir exists) is the right model.
|
||||
- **All IDEs share a single worker on port 37777**: One worker serves all integrations. Session source (which IDE) is tracked via the `source` field in hook payloads. No per-IDE worker instances.
|
||||
- **This npx CLI fully replaces the old `claude-mem-installer`**: Not a supplement — the complete replacement.
|
||||
|
||||
## Solution
|
||||
|
||||
`npx claude-mem` becomes a unified CLI: install, configure any IDE, manage the worker, search memory.
|
||||
|
||||
```
|
||||
npx claude-mem # Interactive install + IDE selection
|
||||
npx claude-mem install # Same as above
|
||||
npx claude-mem install --ide windsurf # Direct IDE setup
|
||||
npx claude-mem start / stop / status # Worker management
|
||||
npx claude-mem search <query> # Search memory from terminal
|
||||
npx claude-mem transcript watch # Start transcript watcher
|
||||
```
|
||||
|
||||
## Platform Support
|
||||
|
||||
**Windows, macOS, and Linux are all first-class targets.** Platform-specific considerations:
|
||||
|
||||
- **Config paths**: Use `os.homedir()` and `path.join()` everywhere — never hardcode `/` or `~`
|
||||
- **Shebangs**: `#!/usr/bin/env node` for the CLI entry point (cross-platform via Node)
|
||||
- **Bun detection**: Check `PATH`, common install locations per platform (`%USERPROFILE%\.bun\bin\bun.exe` on Windows, `~/.bun/bin/bun` on Unix)
|
||||
- **File permissions**: `fs.chmod` is a no-op on Windows; don't gate on it
|
||||
- **Process management**: Worker start/stop uses signals on Unix, taskkill on Windows — match existing `worker-service.ts` patterns
|
||||
- **VS Code paths**: `~/Library/Application Support/Code/` (macOS), `~/.config/Code/` (Linux), `%APPDATA%/Code/` (Windows)
|
||||
- **Shell config**: `.bashrc`/`.zshrc` on Unix, PowerShell profile on Windows (for PATH modifications if needed)
|
||||
|
||||
---
|
||||
|
||||
## Phase 0: Research Findings
|
||||
|
||||
### IDE Integration Tiers
|
||||
|
||||
**Tier 1 — Native Hook/Plugin Systems** (highest fidelity, real-time capture):
|
||||
|
||||
| Tool | Hooks | Config Location | Context Injection | Stars/Users |
|
||||
|------|-------|----------------|-------------------|-------------|
|
||||
| Claude Code | 5 lifecycle hooks | `~/.claude/settings.json` | CLAUDE.md, plugins | ~25% market |
|
||||
| Gemini CLI | 11 lifecycle hooks | `~/.gemini/settings.json` | GEMINI.md | ~95k stars |
|
||||
| OpenCode | 20+ event hooks + plugin SDK | `~/.config/opencode/opencode.json` | AGENTS.md + rules dirs | ~110k stars |
|
||||
| Windsurf | 11 Cascade hooks | `.windsurf/hooks.json` | `.windsurf/rules/*.md` | ~1M users |
|
||||
| Codex CLI | `notify` hook | `~/.codex/config.toml` | `.codex/AGENTS.md`, MCP | Growing (OpenAI) |
|
||||
| OpenClaw | 8 event hooks + plugin SDK | `~/.openclaw/openclaw.json` | MEMORY.md sync | ~196k stars |
|
||||
|
||||
**Tier 2 — MCP Integration** (tool-based, search + context injection):
|
||||
|
||||
| Tool | MCP Support | Config Location | Context Injection |
|
||||
|------|------------|----------------|-------------------|
|
||||
| Cursor | First-class | `.cursor/mcp.json` | `.cursor/rules/*.mdc` |
|
||||
| Copilot CLI | First-class (default MCP) | `~/.copilot/config` | `.github/copilot-instructions.md` |
|
||||
| Antigravity | First-class + MCP Store | `~/.gemini/antigravity/mcp_config.json` | `.agent/rules/`, GEMINI.md |
|
||||
| Goose | Native MCP (co-developed protocol) | `~/.config/goose/config.yaml` | MCP context |
|
||||
| Crush | MCP + Skills | JSON config (charm.land schema) | Skills system |
|
||||
| Roo Code | First-class | `.roo/` | `.roo/rules/*.md`, `AGENTS.md` |
|
||||
| Warp | MCP + Warp Drive | `WARP.md` + Warp Drive UI | `WARP.md` |
|
||||
|
||||
**Tier 3 — Transcript File Watching** (passive, file-based):
|
||||
|
||||
| Tool | Transcript Location | Format |
|
||||
|------|-------------------|--------|
|
||||
| Claude Code | `~/.claude/projects/<proj>/<session>.jsonl` | JSONL |
|
||||
| Codex CLI | `~/.codex/sessions/**/*.jsonl` | JSONL |
|
||||
| Gemini CLI | `~/.gemini/tmp/<hash>/chats/` | JSON |
|
||||
| OpenCode | `.opencode/` (SQLite) | SQLite — needs export |
|
||||
|
||||
### What claude-mem Already Has
|
||||
|
||||
| Component | Status | Location |
|
||||
|-----------|--------|----------|
|
||||
| Claude Code plugin | Complete | `plugin/hooks/hooks.json` |
|
||||
| Cursor hooks installer | Complete | `src/services/integrations/CursorHooksInstaller.ts` |
|
||||
| Platform adapters | Claude Code + Cursor + raw | `src/cli/adapters/` |
|
||||
| Transcript watcher | Complete (schema-based JSONL) | `src/services/transcripts/` |
|
||||
| Codex transcript schema | Sample exists | `src/services/transcripts/config.ts` |
|
||||
| OpenClaw plugin | Complete (1000+ lines) | `openclaw/src/index.ts` |
|
||||
| MCP server | Complete | `plugin/scripts/mcp-server.cjs` |
|
||||
| Gemini CLI support | Not started | — |
|
||||
| OpenCode support | Not started | — |
|
||||
| Windsurf support | Not started | — |
|
||||
|
||||
### Patterns to Copy
|
||||
|
||||
- **Agent detection from `npx skills`** (`vercel-labs/skills/src/agents.ts`): Check if config directory exists
|
||||
- **Existing installer logic** (`installer/src/steps/install.ts:29-83`): registerMarketplace, registerPlugin, enablePluginInClaudeSettings — **extract shared logic** from existing installer into reusable modules (DRY with the new CLI)
|
||||
- **Bun resolution** (`plugin/scripts/bun-runner.js`): PATH lookup + common locations per platform
|
||||
- **CursorHooksInstaller** (`src/services/integrations/CursorHooksInstaller.ts`): Reference implementation for IDE hooks installation
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: NPX CLI Entry Point
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Add `bin` field to `package.json`**:
|
||||
```json
|
||||
"bin": {
|
||||
"claude-mem": "./dist/cli/index.js"
|
||||
}
|
||||
```
|
||||
|
||||
2. **Create `src/npx-cli/index.ts`** — a Node.js CLI router (NOT Bun) with command categories:
|
||||
|
||||
**Install commands** (pure Node.js, no Bun required):
|
||||
- `npx claude-mem` or `npx claude-mem install` → interactive install (IDE multi-select)
|
||||
- `npx claude-mem install --ide <name>` → direct IDE setup (only for implemented IDEs; unimplemented ones error with "Support for <name> coming soon")
|
||||
- `npx claude-mem update` → update to latest version
|
||||
- `npx claude-mem uninstall` → remove plugin and IDE configs
|
||||
- `npx claude-mem version` → print version
|
||||
|
||||
**Runtime commands** (delegate to Bun via installed plugin):
|
||||
- `npx claude-mem start` → spawns `bun worker-service.cjs start`
|
||||
- `npx claude-mem stop` → spawns `bun worker-service.cjs stop`
|
||||
- `npx claude-mem restart` → spawns `bun worker-service.cjs restart`
|
||||
- `npx claude-mem status` → spawns `bun worker-service.cjs status`
|
||||
- `npx claude-mem search <query>` → hits `GET http://localhost:37777/api/search?q=<query>`
|
||||
- `npx claude-mem transcript watch` → starts transcript watcher
|
||||
|
||||
**Runtime commands must check for installation first**: If plugin directory doesn't exist at `~/.claude/plugins/marketplaces/thedotmack/`, print "claude-mem is not installed. Run: npx claude-mem install" and exit.
|
||||
|
||||
3. **The install flow** (fully replaces git clone + build):
|
||||
- Detect the npm package's own location (`import.meta.url` or `__dirname`)
|
||||
- Copy `plugin/` from the npm package to `~/.claude/plugins/marketplaces/thedotmack/`
|
||||
- Copy `plugin/` to `~/.claude/plugins/cache/thedotmack/claude-mem/<version>/`
|
||||
- Register marketplace in `~/.claude/plugins/known_marketplaces.json`
|
||||
- Register plugin in `~/.claude/plugins/installed_plugins.json`
|
||||
- Enable in `~/.claude/settings.json`
|
||||
- Run `npm install` in the marketplace dir (for `@chroma-core/default-embed` — native ONNX binaries, can't be bundled)
|
||||
- Trigger smart-install.js for Bun/uv setup
|
||||
- Run IDE-specific setup for each selected IDE
|
||||
|
||||
4. **Interactive IDE selection** (auto-detect + prompt):
|
||||
- Auto-detect installed IDEs by checking config directories
|
||||
- Present multi-select with detected IDEs pre-selected
|
||||
- Detection map:
|
||||
- Claude Code: `~/.claude/` exists
|
||||
- Gemini CLI: `~/.gemini/` exists
|
||||
- OpenCode: `~/.config/opencode/` exists OR `opencode` in PATH
|
||||
- OpenClaw: `~/.openclaw/` exists
|
||||
- Windsurf: `~/.codeium/windsurf/` exists
|
||||
- Codex CLI: `~/.codex/` exists
|
||||
- Cursor: `~/.cursor/` exists
|
||||
- Copilot CLI: `copilot` in PATH (it's a CLI tool, not a config dir)
|
||||
- Antigravity: `~/.gemini/antigravity/` exists
|
||||
- Goose: `~/.config/goose/` exists OR `goose` in PATH
|
||||
- Crush: `crush` in PATH
|
||||
- Roo Code: check for VS Code extension directory containing `roo-code`
|
||||
- Warp: `~/.warp/` exists OR `warp` in PATH
|
||||
|
||||
5. **The runtime command routing**:
|
||||
- Locate the installed plugin directory
|
||||
- Find Bun binary (same logic as `bun-runner.js`, platform-aware)
|
||||
- Spawn `bun worker-service.cjs <command>` and pipe stdio through
|
||||
- For `search`: HTTP request to running worker
|
||||
|
||||
### Patterns to follow
|
||||
|
||||
- `installer/src/steps/install.ts:29-83` for marketplace registration — **extract to shared module**
|
||||
- `plugin/scripts/bun-runner.js` for Bun resolution
|
||||
- `vercel-labs/skills/src/agents.ts` for IDE auto-detection pattern
|
||||
|
||||
### Verification
|
||||
|
||||
- `npx claude-mem install` copies plugin to correct directories on macOS, Linux, and Windows
|
||||
- Auto-detection finds installed IDEs
|
||||
- `npx claude-mem start/stop/status` work after install
|
||||
- `npx claude-mem search "test"` returns results
|
||||
- `npx claude-mem start` before install prints helpful error message
|
||||
- `npx claude-mem update` and `npx claude-mem uninstall` work correctly
|
||||
- `npx claude-mem version` prints version
|
||||
|
||||
### Anti-patterns
|
||||
|
||||
- Do NOT require Bun for install commands — pure Node.js
|
||||
- Do NOT clone the git repo
|
||||
- Do NOT build from source at install time
|
||||
- Do NOT depend on `bun:sqlite` in the CLI entry point
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Build Pipeline Integration
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Add CLI build step to `scripts/build-hooks.js`**:
|
||||
- Compile `src/npx-cli/index.ts` → `dist/cli/index.js`
|
||||
- Bundle `@clack/prompts` and `picocolors` into the output (self-contained)
|
||||
- Shebang: `#!/usr/bin/env node`
|
||||
- Set executable permissions (no-op on Windows, that's fine)
|
||||
|
||||
2. **Move `@clack/prompts` and `picocolors`** to main package.json as dev dependencies (bundled by esbuild into dist/cli/index.js)
|
||||
|
||||
3. **Verify `package.json` `files` field**: Currently `["dist", "plugin"]`. `dist/cli/index.js` is already included since it's under `dist/`. No change needed.
|
||||
|
||||
4. **Update `prepublishOnly`** to ensure CLI is built before npm publish (already covered — `npm run build` calls `build-hooks.js`)
|
||||
|
||||
5. **Pre-build OpenClaw plugin**: Add an esbuild step that compiles `openclaw/src/index.ts` → `openclaw/dist/index.js` so it ships ready-to-use. No `tsc` at install time.
|
||||
|
||||
6. **Add `openclaw/dist/` to `package.json` `files` field** (or add `openclaw` if the whole directory should ship)
|
||||
|
||||
### Verification
|
||||
|
||||
- `npm run build` produces `dist/cli/index.js` with correct shebang
|
||||
- `npm run build` produces `openclaw/dist/index.js` pre-built
|
||||
- `npm pack` includes both `dist/cli/index.js` and `openclaw/dist/`
|
||||
- `node dist/cli/index.js --help` works without Bun
|
||||
- Package size is reasonable (check with `npm pack --dry-run`)
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Gemini CLI Integration (Tier 1 — Hook-Based)
|
||||
|
||||
**Why first among new IDEs**: Near-identical architecture to Claude Code. 11 lifecycle hooks with JSON stdin/stdout, same exit code conventions (0=success, 2=block), `GEMINI.md` context files. 95k GitHub stars. Lowest effort, highest confidence.
|
||||
|
||||
### Gemini CLI Hook Events
|
||||
|
||||
| Event | Map to claude-mem | Use |
|
||||
|-------|-------------------|-----|
|
||||
| `SessionStart` | `session-init` | Start tracking session |
|
||||
| `BeforeAgent` | `user-prompt` | Capture user prompt |
|
||||
| `AfterAgent` | `observation` | Capture full agent response |
|
||||
| `BeforeTool` | — | Skip (pre-execution, no result yet) |
|
||||
| `AfterTool` | `observation` | Capture tool name + input + response |
|
||||
| `BeforeModel` | — | Skip (too low-level, LLM request details) |
|
||||
| `AfterModel` | — | Skip (raw LLM response, redundant with AfterAgent) |
|
||||
| `BeforeToolSelection` | — | Skip (internal planning step) |
|
||||
| `PreCompress` | `summary` | Trigger summary before context compression |
|
||||
| `Notification` | — | Skip (system alerts, not session data) |
|
||||
| `SessionEnd` | `session-end` | Finalize session |
|
||||
|
||||
**Mapped**: 5 of 11 events. **Skipped**: 6 events that are either too low-level (BeforeModel/AfterModel), pre-execution (BeforeTool, BeforeToolSelection), or system-level (Notification).
|
||||
|
||||
### Verified Stdin Payload Schemas (from `packages/core/src/hooks/types.ts`)
|
||||
|
||||
**Base input (all hooks receive):**
|
||||
```typescript
|
||||
{ session_id: string, transcript_path: string, cwd: string, hook_event_name: string, timestamp: string }
|
||||
```
|
||||
|
||||
**Event-specific fields:**
|
||||
| Event | Additional Fields |
|
||||
|-------|-------------------|
|
||||
| `SessionStart` | `source: "startup" \| "resume" \| "clear"` |
|
||||
| `SessionEnd` | `reason: "exit" \| "clear" \| "logout" \| "prompt_input_exit" \| "other"` |
|
||||
| `BeforeAgent` | `prompt: string` |
|
||||
| `AfterAgent` | `prompt: string, prompt_response: string, stop_hook_active: boolean` |
|
||||
| `BeforeTool` | `tool_name: string, tool_input: Record<string, unknown>, mcp_context?: McpToolContext, original_request_name?: string` |
|
||||
| `AfterTool` | `tool_name: string, tool_input: Record<string, unknown>, tool_response: Record<string, unknown>, mcp_context?: McpToolContext` |
|
||||
| `PreCompress` | `trigger: "auto" \| "manual"` |
|
||||
| `Notification` | `notification_type: "ToolPermission", message: string, details: Record<string, unknown>` |
|
||||
|
||||
**Output (all hooks can return):**
|
||||
```typescript
|
||||
{ continue?: boolean, stopReason?: string, suppressOutput?: boolean, systemMessage?: string, decision?: "allow" | "deny" | "block" | "approve" | "ask", reason?: string, hookSpecificOutput?: Record<string, unknown> }
|
||||
```
|
||||
|
||||
**Advisory (non-blocking) hooks:** SessionStart, SessionEnd, PreCompress, Notification — `continue` and `decision` fields are ignored.
|
||||
|
||||
**Environment variables provided:** `GEMINI_PROJECT_DIR`, `GEMINI_SESSION_ID`, `GEMINI_CWD`, `CLAUDE_PROJECT_DIR` (compat alias)
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Create Gemini CLI platform adapter** at `src/cli/adapters/gemini-cli.ts`:
|
||||
- Normalize Gemini CLI's hook JSON to `NormalizedHookInput`
|
||||
- Base fields always present: `session_id`, `transcript_path`, `cwd`, `hook_event_name`, `timestamp`
|
||||
- Map per event:
|
||||
- `SessionStart`: `source` → session init metadata
|
||||
- `BeforeAgent`: `prompt` → user prompt text
|
||||
- `AfterAgent`: `prompt` + `prompt_response` → full conversation turn
|
||||
- `AfterTool`: `tool_name` + `tool_input` + `tool_response` → observation
|
||||
- `PreCompress`: `trigger` → summary trigger
|
||||
- `SessionEnd`: `reason` → session finalization
|
||||
|
||||
2. **Create Gemini CLI hooks installer** at `src/services/integrations/GeminiCliHooksInstaller.ts`:
|
||||
- Write hooks to `~/.gemini/settings.json` under the `hooks` key
|
||||
- Must **merge** with existing settings (read → parse → deep merge → write)
|
||||
- Hook config format (verified against official docs):
|
||||
```json
|
||||
{
|
||||
"hooks": {
|
||||
"AfterTool": [{
|
||||
"matcher": "*",
|
||||
"hooks": [{ "name": "claude-mem", "type": "command", "command": "<path-to-hook-script>", "timeout": 5000 }]
|
||||
}]
|
||||
}
|
||||
}
|
||||
```
|
||||
- Note: `matcher` uses regex for tool events, exact string for lifecycle events. `"*"` or `""` matches all.
|
||||
- Hook groups support `sequential: boolean` (default false = parallel execution)
|
||||
- Security: Project-level hooks are fingerprinted — if name/command changes, user is warned
|
||||
- Context injection via `~/.gemini/GEMINI.md` (append claude-mem section with `<claude-mem-context>` tags, same pattern as CLAUDE.md)
|
||||
- Settings hierarchy: project `.gemini/settings.json` > user `~/.gemini/settings.json` > system `/etc/gemini-cli/settings.json`
|
||||
|
||||
3. **Register `gemini-cli` in `getPlatformAdapter()`** at `src/cli/adapters/index.ts`
|
||||
|
||||
4. **Add Gemini CLI to installer IDE selection**
|
||||
|
||||
### Verification
|
||||
|
||||
- `npx claude-mem install --ide gemini-cli` merges hooks into `~/.gemini/settings.json`
|
||||
- Gemini CLI sessions are captured by the worker
|
||||
- `AfterTool` events produce observations with correct `tool_name`, `tool_input`, `tool_response`
|
||||
- `GEMINI.md` gets claude-mem context section
|
||||
- Existing Gemini CLI settings are preserved (merge, not overwrite)
|
||||
- Verify `session_id` from base input is used for session tracking
|
||||
|
||||
### Anti-patterns
|
||||
|
||||
- Do NOT overwrite `~/.gemini/settings.json` — must deep merge
|
||||
- Do NOT map all 11 events — the 6 skipped events would produce noise, not signal
|
||||
- Do NOT use `type: "runtime"` — that's for internal extensions only; use `type: "command"`
|
||||
- Advisory hooks (SessionStart, SessionEnd, PreCompress, Notification) cannot block — don't set `decision` or `continue` fields on them
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: OpenCode Integration (Tier 1 — Plugin-Based)
|
||||
|
||||
**Why next**: 110k stars, richest plugin ecosystem. OpenCode plugins are JS/TS modules auto-loaded from plugin directories. OpenCode also has a Claude Code compatibility fallback (reads `~/.claude/CLAUDE.md` if no global `AGENTS.md` exists, controllable via `OPENCODE_DISABLE_CLAUDE_CODE_PROMPT=1`).
|
||||
|
||||
### Verified Plugin API (from `packages/plugin/src/index.ts`)
|
||||
|
||||
**Plugin signature:**
|
||||
```typescript
|
||||
import { type Plugin, tool } from "@opencode-ai/plugin"
|
||||
|
||||
export const ClaudeMemPlugin: Plugin = async (ctx) => {
|
||||
// ctx: { client, project, directory, worktree, serverUrl, $ }
|
||||
return { /* hooks object */ }
|
||||
}
|
||||
```
|
||||
|
||||
**PluginInput type (6 properties, not 4):**
|
||||
```typescript
|
||||
type PluginInput = {
|
||||
client: ReturnType<typeof createOpencodeClient> // OpenCode SDK client
|
||||
project: Project // Current project info
|
||||
directory: string // Current working directory
|
||||
worktree: string // Git worktree path
|
||||
serverUrl: URL // Server URL
|
||||
$: BunShell // Bun shell API
|
||||
}
|
||||
```
|
||||
|
||||
**Two hook mechanisms (important distinction):**
|
||||
|
||||
1. **Direct interceptor hooks** — keys on the returned `Hooks` object, receive `(input, output)` allowing mutation:
|
||||
- `tool.execute.before`: `(input: { tool, sessionID, callID }, output: { args })`
|
||||
- `tool.execute.after`: `(input: { tool, sessionID, callID, args }, output: { title, output, metadata })`
|
||||
- `shell.env`, `chat.message`, `chat.params`, `chat.headers`, `permission.ask`, `command.execute.before`
|
||||
- Experimental: `experimental.session.compacting`, `experimental.chat.messages.transform`, `experimental.chat.system.transform`
|
||||
|
||||
2. **Bus event catch-all** — generic `event` hook, receives `{ event }` where `event.type` is the event name:
|
||||
- `session.created`, `session.compacted`, `session.deleted`, `session.idle`, `session.error`, `session.status`, `session.updated`, `session.diff`
|
||||
- `message.updated`, `message.part.updated`, `message.part.removed`, `message.removed`
|
||||
- `file.edited`, `file.watcher.updated`
|
||||
- `command.executed`, `todo.updated`, `installation.updated`, `server.connected`
|
||||
- `permission.asked`, `permission.replied`
|
||||
- `lsp.client.diagnostics`, `lsp.updated`
|
||||
- `tui.prompt.append`, `tui.command.execute`, `tui.toast.show`
|
||||
- Total: **27 bus events** across **12 categories**
|
||||
|
||||
**Custom tool registration (CORRECTED — name is the key, not positional arg):**
|
||||
```typescript
|
||||
return {
|
||||
tool: {
|
||||
claude_mem_search: tool({
|
||||
description: "Search claude-mem memory database",
|
||||
args: { query: tool.schema.string() },
|
||||
async execute(args, context) {
|
||||
// context: { sessionID, messageID, agent, directory, worktree, abort, metadata, ask }
|
||||
const response = await fetch(`http://localhost:37777/api/search?q=${encodeURIComponent(args.query)}`)
|
||||
return await response.text()
|
||||
},
|
||||
}),
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Create OpenCode plugin** at `src/integrations/opencode-plugin/index.ts`:
|
||||
- Export a `Plugin` function receiving full `PluginInput` context
|
||||
- Use **direct interceptor** `tool.execute.after` for tool observation capture (gives `tool`, `args`, `output`)
|
||||
- Use **bus event catch-all** `event` for session lifecycle:
|
||||
|
||||
| Mechanism | Event | Map to claude-mem |
|
||||
|-----------|-------|-------------------|
|
||||
| interceptor | `tool.execute.after` | `observation` (tool name + args + output) |
|
||||
| bus event | `session.created` | `session-init` |
|
||||
| bus event | `message.updated` | `observation` (assistant messages) |
|
||||
| bus event | `session.compacted` | `summary` |
|
||||
| bus event | `file.edited` | `observation` (file changes) |
|
||||
| bus event | `session.deleted` | `session-end` |
|
||||
|
||||
- Register `claude_mem_search` custom tool using correct `tool({ description, args, execute })` API
|
||||
- Hit `localhost:37777` API endpoints from the plugin
|
||||
|
||||
2. **Build the plugin** in the esbuild pipeline → `dist/opencode-plugin/index.js`
|
||||
|
||||
3. **Create OpenCode setup in installer** (two options, prefer file-based):
|
||||
- **Option A (file-based):** Copy plugin to `~/.config/opencode/plugins/claude-mem.ts` (auto-loaded at startup)
|
||||
- **Option B (npm-based):** Add to `~/.config/opencode/opencode.json` under `"plugin"` array: `["claude-mem"]`
|
||||
- Config also supports JSONC (`opencode.jsonc`) and legacy `config.json`
|
||||
- Context injection: Append to `~/.config/opencode/AGENTS.md` (or create it) with `<claude-mem-context>` tags
|
||||
- Additional context via `"instructions"` config key (supports file paths, globs, remote URLs)
|
||||
|
||||
4. **Add OpenCode to installer IDE selection**
|
||||
|
||||
### OpenCode Verification
|
||||
|
||||
- `npx claude-mem install --ide opencode` registers the plugin (file or npm)
|
||||
- OpenCode loads the plugin on next session
|
||||
- `tool.execute.after` interceptor produces observations with `tool`, `args`, `output`
|
||||
- Bus events (`session.created`, `session.deleted`) handle session lifecycle
|
||||
- `claude_mem_search` custom tool works in OpenCode sessions
|
||||
- Context is injected via AGENTS.md
|
||||
|
||||
### OpenCode Anti-patterns
|
||||
|
||||
- Do NOT try to use OpenCode's `session.diff` for full capture — it's a summary diff, not raw data
|
||||
- Do NOT use `tool('name', schema, handler)` — wrong signature. Name is the key in the `tool:{}` map
|
||||
- Do NOT assume bus events have the same `(input, output)` mutation pattern — they only receive `{ event }`
|
||||
- OpenCode plugins run in Bun — the plugin CAN use Bun APIs (unlike the npx CLI itself)
|
||||
- Do NOT hardcode `~/.config/opencode/` — respect `OPENCODE_CONFIG_DIR` env var if set
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Windsurf Integration (Tier 1 — Hook-Based)
|
||||
|
||||
**Why next**: 11 Cascade hooks, ~1M users. Hook architecture uses JSON stdin with a consistent envelope format.
|
||||
|
||||
### Verified Windsurf Hook Events (from docs.windsurf.com/windsurf/cascade/hooks)
|
||||
|
||||
**Naming pattern**: `pre_`/`post_` prefix + 5 action categories, plus 2 standalone post-only events.
|
||||
|
||||
| Event | Can Block? | Map to claude-mem | Use |
|
||||
|-------|-----------|-------------------|-----|
|
||||
| `pre_user_prompt` | Yes | `session-init` + `context` | Start session, inject context |
|
||||
| `pre_read_code` | Yes | — | Skip (pre-execution, can block file reads) |
|
||||
| `post_read_code` | No | — | Skip (too noisy, file reads are frequent) |
|
||||
| `pre_write_code` | Yes | — | Skip (pre-execution, can block writes) |
|
||||
| `post_write_code` | No | `observation` | Code generation |
|
||||
| `pre_run_command` | Yes | — | Skip (pre-execution, can block commands) |
|
||||
| `post_run_command` | No | `observation` | Shell command execution |
|
||||
| `pre_mcp_tool_use` | Yes | — | Skip (pre-execution, can block MCP calls) |
|
||||
| `post_mcp_tool_use` | No | `observation` | MCP tool results |
|
||||
| `post_cascade_response` | No | `observation` | Full AI response |
|
||||
| `post_setup_worktree` | No | — | Skip (informational) |
|
||||
|
||||
**Mapped**: 5 of 11 events (all post-action). **Skipped**: 4 pre-hooks (blocking-capable, pre-execution) + 2 low-value post-hooks.
|
||||
|
||||
### Verified Stdin Payload Schema
|
||||
|
||||
**Common envelope (all hooks):**
|
||||
```json
|
||||
{
|
||||
"agent_action_name": "string",
|
||||
"trajectory_id": "string",
|
||||
"execution_id": "string",
|
||||
"timestamp": "ISO 8601 string",
|
||||
"tool_info": { /* event-specific payload */ }
|
||||
}
|
||||
```
|
||||
|
||||
**Event-specific `tool_info` payloads:**
|
||||
|
||||
| Event | `tool_info` fields |
|
||||
|-------|-------------------|
|
||||
| `pre_user_prompt` | `{ user_prompt: string }` |
|
||||
| `pre_read_code` / `post_read_code` | `{ file_path: string }` |
|
||||
| `pre_write_code` / `post_write_code` | `{ file_path: string, edits: [{ old_string: string, new_string: string }] }` |
|
||||
| `pre_run_command` / `post_run_command` | `{ command_line: string, cwd: string }` |
|
||||
| `pre_mcp_tool_use` | `{ mcp_server_name: string, mcp_tool_name: string, mcp_tool_arguments: {} }` |
|
||||
| `post_mcp_tool_use` | `{ mcp_server_name: string, mcp_tool_name: string, mcp_tool_arguments: {}, mcp_result: string }` |
|
||||
| `post_cascade_response` | `{ response: string }` (markdown) |
|
||||
| `post_setup_worktree` | `{ worktree_path: string, root_workspace_path: string }` |
|
||||
|
||||
**Exit codes:** `0` = success, `2` = block (pre-hooks only; stderr shown to agent), any other = non-blocking warning.
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Create Windsurf platform adapter** at `src/cli/adapters/windsurf.ts`:
|
||||
- Normalize Windsurf's hook input format to `NormalizedHookInput`
|
||||
- Common envelope: `agent_action_name`, `trajectory_id`, `execution_id`, `timestamp`, `tool_info`
|
||||
- Map: `trajectory_id` → `sessionId`, `tool_info` fields per event type
|
||||
- For `post_write_code`: `tool_info.file_path` + `tool_info.edits` → file change observation
|
||||
- For `post_run_command`: `tool_info.command_line` + `tool_info.cwd` → command observation
|
||||
- For `post_mcp_tool_use`: `tool_info.mcp_tool_name` + `tool_info.mcp_tool_arguments` + `tool_info.mcp_result` → tool observation
|
||||
- For `post_cascade_response`: `tool_info.response` → full AI response observation
|
||||
|
||||
2. **Create Windsurf hooks installer** at `src/services/integrations/WindsurfHooksInstaller.ts`:
|
||||
- Write hooks to `~/.codeium/windsurf/hooks.json` (user-level, for global coverage)
|
||||
- Per-workspace override at `.windsurf/hooks.json` if user chooses workspace-level install
|
||||
- Config format (verified):
|
||||
```json
|
||||
{
|
||||
"hooks": {
|
||||
"post_write_code": [{
|
||||
"command": "<path-to-hook-script>",
|
||||
"show_output": false,
|
||||
"working_directory": "<optional>"
|
||||
}]
|
||||
}
|
||||
}
|
||||
```
|
||||
- Note: Tilde expansion (`~`) is NOT supported in `working_directory` — use absolute paths
|
||||
- Merge order: cloud → system → user → workspace (all hooks at all levels execute)
|
||||
- Context injection via `.windsurf/rules/claude-mem-context.md` (workspace-level; Windsurf rules are workspace-scoped)
|
||||
- Rule limits: 6,000 chars per file, 12,000 chars total across all rules
|
||||
|
||||
3. **Register `windsurf` in `getPlatformAdapter()`** at `src/cli/adapters/index.ts`
|
||||
|
||||
4. **Add Windsurf to installer IDE selection**
|
||||
|
||||
### Windsurf Verification
|
||||
|
||||
- `npx claude-mem install --ide windsurf` creates hooks config at `~/.codeium/windsurf/hooks.json`
|
||||
- Windsurf sessions are captured by the worker via post-action hooks
|
||||
- `trajectory_id` is used as session identifier
|
||||
- Context is injected via `.windsurf/rules/claude-mem-context.md` (under 6K char limit)
|
||||
- Existing hooks.json is preserved (merge, not overwrite)
|
||||
|
||||
### Windsurf Anti-patterns
|
||||
|
||||
- Do NOT use fabricated event names (`post_search_code`, `post_lint_code`, `on_error`, `pre_tool_execution`) — they don't exist
|
||||
- Do NOT assume Windsurf's stdin JSON matches Claude Code's — it uses `tool_info` envelope, not flat fields
|
||||
- Do NOT use tilde (`~`) in `working_directory` — not supported, use absolute paths
|
||||
- Do NOT exceed 6K chars in the context rule file — Windsurf truncates beyond that
|
||||
- Pre-hooks can block actions (exit 2) — only use post-hooks for observation capture
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Codex CLI Integration (Tier 1 — Hook + Transcript)
|
||||
|
||||
### Dedup strategy
|
||||
|
||||
Codex has both a `notify` hook (real-time) and transcript files (complete history). Use **transcript watching only** — it's more complete and avoids the complexity of dual capture paths. The `notify` hook is a simpler mechanism that doesn't provide enough granularity to justify maintaining two integration paths. If transcript watching proves insufficient, add the notify hook later.
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Create Codex transcript schema** — the sample in `src/services/transcripts/config.ts` is already production-quality. Verify against current Codex CLI JSONL format and update if needed.
|
||||
|
||||
2. **Create Codex setup in installer**:
|
||||
- Write transcript-watch config to `~/.claude-mem/transcript-watch.json`
|
||||
- Set up watch for `~/.codex/sessions/**/*.jsonl` using existing CODEX_SAMPLE_SCHEMA
|
||||
- Context injection via `.codex/AGENTS.md` (Codex reads this natively)
|
||||
- Must merge with existing `config.toml` if it exists (read → parse → merge → write)
|
||||
|
||||
3. **Add Codex CLI to installer IDE selection**
|
||||
|
||||
### Verification
|
||||
|
||||
- `npx claude-mem install --ide codex` creates transcript watch config
|
||||
- Codex sessions appear in claude-mem database
|
||||
- `AGENTS.md` updated with context after sessions
|
||||
- Existing `config.toml` is preserved
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: OpenClaw Integration (Tier 1 — Plugin-Based)
|
||||
|
||||
**Plugin is already fully built** at `openclaw/src/index.ts` (~1000 lines). Has event hooks, SSE observation feed, MEMORY.md sync, slash commands. Only wiring into the installer is needed.
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Wire OpenClaw into the npx installer**:
|
||||
- Detect `~/.openclaw/` directory
|
||||
- Copy pre-built plugin from `openclaw/dist/` (built in Phase 2) to OpenClaw plugins location
|
||||
- Register in `~/.openclaw/openclaw.json` under `plugins.claude-mem`
|
||||
- Configure worker port, project name, syncMemoryFile
|
||||
- Optionally prompt for observation feed setup (channel type + target ID)
|
||||
|
||||
2. **Add OpenClaw to IDE selection TUI** with hint about messaging channel support
|
||||
|
||||
### Verification
|
||||
|
||||
- `npx claude-mem install --ide openclaw` registers the plugin
|
||||
- OpenClaw gateway loads the plugin on restart
|
||||
- Observations are recorded from OpenClaw sessions
|
||||
- MEMORY.md syncs to agent workspaces
|
||||
|
||||
### Anti-patterns
|
||||
|
||||
- Do NOT rebuild the OpenClaw plugin from source at install time — it ships pre-built from Phase 2
|
||||
- Do NOT modify the plugin's event handling — it's battle-tested
|
||||
|
||||
---
|
||||
|
||||
## Phase 8: MCP-Based Integrations (Tier 2)
|
||||
|
||||
**These get the MCP server for free** — it already exists at `plugin/scripts/mcp-server.cjs`. The installer just needs to write the right config files per IDE.
|
||||
|
||||
MCP-only integrations provide: search tools + context injection. They do NOT capture transcripts or tool usage in real-time.
|
||||
|
||||
### What to implement
|
||||
|
||||
1. **Copilot CLI MCP setup**:
|
||||
- Write MCP config to `~/.copilot/config` (merge, not overwrite)
|
||||
- Context injection: `.github/copilot-instructions.md`
|
||||
- Detection: `copilot` command in PATH
|
||||
|
||||
2. **Antigravity MCP setup**:
|
||||
- Write MCP config to `~/.gemini/antigravity/mcp_config.json` (merge, not overwrite)
|
||||
- Context injection: `~/.gemini/GEMINI.md` (shared with Gemini CLI) and/or `.agent/rules/claude-mem-context.md`
|
||||
- Detection: `~/.gemini/antigravity/` exists
|
||||
- Note: Antigravity has NO hook system — MCP is the only integration path
|
||||
|
||||
3. **Goose MCP setup**:
|
||||
- Write MCP config to `~/.config/goose/config.yaml` (YAML merge — use a lightweight YAML parser or write the block manually if config doesn't exist)
|
||||
- Detection: `~/.config/goose/` exists OR `goose` in PATH
|
||||
- Note: Goose co-developed MCP with Anthropic, so MCP support is excellent
|
||||
|
||||
4. **Crush MCP setup**:
|
||||
- Write MCP config to Crush's JSON config
|
||||
- Detection: `crush` in PATH
|
||||
|
||||
5. **Roo Code MCP setup**:
|
||||
- Write MCP config to `.roo/` or workspace settings
|
||||
- Context injection: `.roo/rules/claude-mem-context.md`
|
||||
- Detection: Check for VS Code extension directory containing `roo-code`
|
||||
|
||||
6. **Warp MCP setup**:
|
||||
- Warp uses `WARP.md` in project root for context injection (similar to CLAUDE.md)
|
||||
- MCP servers configured via Warp Drive UI, but also via config files
|
||||
- Detection: `~/.warp/` exists OR `warp` in PATH
|
||||
- Note: Warp is a terminal replacement (~26k stars), not just a CLI tool — multi-agent orchestration with management UI
|
||||
|
||||
7. **For each**: Add to installer IDE detection and selection
|
||||
|
||||
### Config merging strategy
|
||||
|
||||
JSON configs: Read → parse → deep merge → write back. YAML configs (Goose): If file exists, read and append the MCP block. If not, create from template. Avoid pulling in a full YAML parser library — write the MCP block as a string append with proper indentation if the format is predictable.
|
||||
|
||||
### Verification
|
||||
|
||||
- Each IDE can search claude-mem via MCP tools
|
||||
- Context files are written to IDE-specific locations
|
||||
- Existing configs are preserved
|
||||
|
||||
### Anti-patterns
|
||||
|
||||
- MCP-only integrations do NOT capture transcripts — don't claim "full integration"
|
||||
- Do NOT overwrite existing config files — always merge
|
||||
- Do NOT add a heavy YAML parser dependency for one integration
|
||||
|
||||
---
|
||||
|
||||
## Phase 9: Remove Old Installer
|
||||
|
||||
This is a **full replacement**, not a deprecation.
|
||||
|
||||
### What to implement
|
||||
|
||||
1. Remove `claude-mem-installer` npm package (unpublish or mark deprecated with message pointing to `npx claude-mem`)
|
||||
2. Update `install/public/install.sh` → redirect to `npx claude-mem`
|
||||
3. Remove `installer/` directory from the repository (it's replaced by `src/npx-cli/`)
|
||||
4. Update docs site to reflect the new install command
|
||||
5. Update README.md install instructions
|
||||
|
||||
---
|
||||
|
||||
## Phase 10: Final Verification
|
||||
|
||||
### All platforms (macOS, Linux, Windows)
|
||||
|
||||
1. `npm run build` succeeds, produces `dist/cli/index.js` and `openclaw/dist/index.js`
|
||||
2. `node dist/cli/index.js install` works clean (no prior install)
|
||||
3. Auto-detects installed IDEs correctly per platform
|
||||
4. `npx claude-mem start/stop/status/search` all work
|
||||
5. `npx claude-mem update` updates correctly
|
||||
6. `npx claude-mem uninstall` cleans up all IDE configs
|
||||
7. `npx claude-mem version` prints version
|
||||
8. `npx claude-mem start` before install shows helpful error
|
||||
9. No Bun dependency at install time
|
||||
|
||||
### Per-integration verification
|
||||
|
||||
| Integration | Type | Captures Sessions | Search via MCP | Context Injection |
|
||||
|-------------|------|-------------------|----------------|-------------------|
|
||||
| Claude Code | Plugin | Yes (hooks) | Yes | CLAUDE.md |
|
||||
| Gemini CLI | Hooks | Yes (AfterTool, AfterAgent) | Yes (via hook) | GEMINI.md |
|
||||
| OpenCode | Plugin | Yes (tool.execute.after, message.updated) | Yes (custom tool) | AGENTS.md / rules |
|
||||
| Windsurf | Hooks | Yes (post_cascade_response, etc.) | Yes (via hook) | .windsurf/rules/ |
|
||||
| Codex CLI | Transcript | Yes (JSONL watcher) | No (passive only) | .codex/AGENTS.md |
|
||||
| OpenClaw | Plugin | Yes (event hooks) | Yes (slash commands) | MEMORY.md |
|
||||
| Copilot CLI | MCP | No | Yes | copilot-instructions.md |
|
||||
| Antigravity | MCP | No | Yes | .agent/rules/ |
|
||||
| Goose | MCP | No | Yes | MCP context |
|
||||
| Crush | MCP | No | Yes | Skills |
|
||||
| Roo Code | MCP | No | Yes | .roo/rules/ |
|
||||
| Warp | MCP | No | Yes | WARP.md |
|
||||
|
||||
---
|
||||
|
||||
## Priority Order & Impact
|
||||
|
||||
| Phase | IDE/Tool | Integration Type | Stars/Users | Effort |
|
||||
|-------|----------|-----------------|-------------|--------|
|
||||
| 1-2 | (infrastructure) | npx CLI + build pipeline | All users | Medium |
|
||||
| 3 | Gemini CLI | Hooks (Tier 1) | ~95k stars | Medium (near-identical to Claude Code) |
|
||||
| 4 | OpenCode | Plugin (Tier 1) | ~110k stars | Medium (rich plugin SDK) |
|
||||
| 5 | Windsurf | Hooks (Tier 1) | ~1M users | Medium |
|
||||
| 6 | Codex CLI | Transcript (Tier 3) | Growing (OpenAI) | Low (schema already exists) |
|
||||
| 7 | OpenClaw | Plugin (Tier 1) — pre-built | ~196k stars | Low (wire into installer) |
|
||||
| 8 | Copilot CLI, Antigravity, Goose, Crush, Warp, Roo Code | MCP (Tier 2) | 20M+ combined | Low per IDE |
|
||||
| 9 | (remove old installer) | — | — | Low |
|
||||
| 10 | (final verification) | — | — | Low |
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- **Removing Bun as runtime dependency**: Worker still requires Bun for `bun:sqlite`. Runtime commands delegate to Bun; install commands don't need it.
|
||||
- **JetBrains plugin**: Requires Kotlin/Java development — different ecosystem entirely.
|
||||
- **Zed extension**: WASM sandbox limits feasibility.
|
||||
- **Neovim/Emacs plugins**: Niche audiences, complex plugin ecosystems (Lua/Elisp). Could be added later via MCP (gptel supports it).
|
||||
- **Amazon Q / Kiro**: Amazon Q Developer CLI has been sunsetted in favor of Kiro (proprietary, no public extensibility API yet). Revisit when Kiro opens up.
|
||||
- **Aider**: Niche audience, writes Markdown transcripts (not JSONL), would require a markdown parser mode in the watcher. Add if demand materializes.
|
||||
- **Continue.dev**: Small user base relative to other MCP tools. Can be added as a Tier 2 MCP integration later if requested.
|
||||
- **Toad / Qwen Code / Oh-my-pi**: Too early-stage or too niche. Monitor for growth.
|
||||
- **OpenClaw plugin development**: The plugin is already complete. Only installer wiring is in scope.
|
||||
+92
-315
@@ -2,6 +2,98 @@
|
||||
|
||||
All notable changes to claude-mem.
|
||||
|
||||
## [v10.5.2] - 2026-02-26
|
||||
|
||||
## Smart Explore Benchmark Docs & Skill Update
|
||||
|
||||
### Documentation
|
||||
- Published smart-explore benchmark report to public docs — full A/B comparison with methodology, raw data tables, quality assessment, and decision framework
|
||||
- Added benchmark report to docs.json navigation under Best Practices
|
||||
|
||||
### Smart Explore Skill
|
||||
- Updated token economics with benchmark-accurate data (11-18x savings on exploration, 4-8x on file understanding)
|
||||
- Added "map first" core principle as decision heuristic for tool selection
|
||||
- Added AST completeness guarantee to smart_unfold documentation (never truncates, unlike Explore agents)
|
||||
- Added Explore agent escalation guidance for multi-file synthesis tasks
|
||||
- Updated smart_unfold token range from ~1-7k to ~400-2,100 based on measurements
|
||||
- Updated Explore agent token range from ~20-40k to ~39-59k based on measurements
|
||||
|
||||
## [v10.5.1] - 2026-02-26
|
||||
|
||||
### Bug Fix
|
||||
|
||||
- Restored hooks.json to pre-smart-explore configuration (re-adds Setup hook, separate worker start command, PostToolUse matcher)
|
||||
|
||||
## [v10.5.0] - 2026-02-26
|
||||
|
||||
## Smart Explore: AST-Powered Code Navigation
|
||||
|
||||
This release introduces **Smart Explore**, a token-optimized structural code search system built on tree-sitter AST parsing. It applies the same progressive disclosure pattern used in human-readable code outlines — but programmatically, for AI agents.
|
||||
|
||||
### Why This Matters
|
||||
|
||||
The standard exploration cycle (Glob → Grep → Read) forces agents to consume entire files to understand code structure. A typical 800-line file costs ~12,000 tokens to read. Smart Explore replaces this with a 3-layer progressive disclosure workflow that delivers the same understanding at **6-12x lower token cost**.
|
||||
|
||||
### 3 New MCP Tools
|
||||
|
||||
- **`smart_search`** — Walks directories, parses all code files via tree-sitter, and returns ranked symbols with signatures and line numbers. Replaces the Glob → Grep discovery cycle in a single call (~2-6k tokens).
|
||||
- **`smart_outline`** — Returns the complete structural skeleton of a file: all functions, classes, methods, properties, imports (~1-2k tokens vs ~12k for a full Read).
|
||||
- **`smart_unfold`** — Expands a single symbol to its full source code including JSDoc, decorators, and implementation (~1-7k tokens).
|
||||
|
||||
### Token Economics
|
||||
|
||||
| Approach | Tokens | Savings |
|
||||
|----------|--------|---------|
|
||||
| smart_outline + smart_unfold | ~3,100 | 8x vs Read |
|
||||
| smart_search (cross-file) | ~2,000-6,000 | 6-12x vs Explore agent |
|
||||
| Read (full file) | ~12,000+ | baseline |
|
||||
| Explore agent | ~20,000-40,000 | baseline |
|
||||
|
||||
### Language Support
|
||||
|
||||
10 languages via tree-sitter grammars: TypeScript, JavaScript, Python, Rust, Go, Java, C, C++, Ruby, PHP.
|
||||
|
||||
### Other Changes
|
||||
|
||||
- Simplified hooks configuration
|
||||
- Removed legacy setup.sh script
|
||||
- Security fix: replaced `execSync` with `execFileSync` to prevent command injection in file path handling
|
||||
|
||||
## [v10.4.4] - 2026-02-26
|
||||
|
||||
## Fix
|
||||
|
||||
- **Remove `save_observation` from MCP tool surface** — This tool was exposed as an MCP tool available to Claude, but it's an internal API-only feature. Removing it from the MCP server prevents unintended tool invocation and keeps the tool surface clean.
|
||||
|
||||
## [v10.4.3] - 2026-02-25
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- **Fix PostToolUse hook crashes and 5-second latency (#1220)**: Added missing `break` statements to all 7 switch cases in `worker-service.ts` preventing fall-through execution, added `.catch()` on `main()` to handle unhandled promise rejections, and removed redundant `start` commands from hook groups that triggered the 5-second `collectStdin()` timeout
|
||||
- **Fix CLAUDE_PLUGIN_ROOT fallback for Stop hooks (#1215)**: Added POSIX shell-level `CLAUDE_PLUGIN_ROOT` fallback in `hooks.json` for environments where the variable isn't injected, added script-level self-resolution via `import.meta.url` in `bun-runner.js`, and regression test added in `plugin-distribution.test.ts`
|
||||
|
||||
## Maintenance
|
||||
|
||||
- Synced all version files (plugin.json was stuck at 10.4.0)
|
||||
|
||||
## [v10.4.2] - 2026-02-25
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- **Fix PostToolUse hook crashes and 5-second latency (#1220)**: Added missing `break` statements to all 7 switch cases in `worker-service.ts` preventing fall-through execution, added `.catch()` on `main()` to handle unhandled promise rejections, and removed redundant `start` commands from hook groups that triggered the 5-second `collectStdin()` timeout
|
||||
- **Fix CLAUDE_PLUGIN_ROOT fallback for Stop hooks (#1215)**: Added POSIX shell-level `CLAUDE_PLUGIN_ROOT` fallback in `hooks.json` for environments where the variable isn't injected, added script-level self-resolution via `import.meta.url` in `bun-runner.js`, and regression test added in `plugin-distribution.test.ts`
|
||||
- **Sync plugin.json version**: Fixed `plugin.json` being stuck at 10.4.0 while other version files were at 10.4.1
|
||||
|
||||
## [v10.4.1] - 2026-02-24
|
||||
|
||||
### Refactor
|
||||
- **Skills Conversion**: Converted `/make-plan` and `/do` commands into first-class skills in `plugin/skills/`.
|
||||
- **Organization**: Centralized planning and execution instructions alongside `mem-search`.
|
||||
- **Compatibility**: Added symlinks for `openclaw/skills/` to ensure seamless integration with OpenClaw.
|
||||
|
||||
### Chore
|
||||
- **Version Bump**: Aligned all package and plugin manifests to v10.4.1.
|
||||
|
||||
## [v10.4.0] - 2026-02-24
|
||||
|
||||
## v10.4.0 — Stability & Platform Hardening
|
||||
@@ -1097,318 +1189,3 @@ This appears in both terminal (colored) output and markdown format, including em
|
||||
|
||||
- Added detailed analysis reports for GitHub issues #511, #514, #517, #520, #527, #531, #532
|
||||
|
||||
## [v8.5.7] - 2026-01-04
|
||||
|
||||
## Modular Architecture Refactor
|
||||
|
||||
This release refactors the monolithic service architecture into focused, single-responsibility modules with comprehensive test coverage.
|
||||
|
||||
### Architecture Improvements
|
||||
|
||||
- **SQLite Repositories** (`src/services/sqlite/`) - Modular repositories for sessions, observations, prompts, summaries, and timeline
|
||||
- **Worker Agents** (`src/services/worker/agents/`) - Extracted response processing, error handling, and session cleanup
|
||||
- **Search Strategies** (`src/services/worker/search/`) - Modular search with Chroma, SQLite, and Hybrid strategies plus orchestrator
|
||||
- **Context Generation** (`src/services/context/`) - Separated context building, token calculation, formatters, and renderers
|
||||
- **Infrastructure** (`src/services/infrastructure/`) - Graceful shutdown, health monitoring, and process management
|
||||
- **Server** (`src/services/server/`) - Express server setup, middleware, and error handling
|
||||
|
||||
### Test Coverage
|
||||
|
||||
- **595 tests** across 36 test files
|
||||
- **1,120 expect() assertions**
|
||||
- Coverage for SQLite repos, worker agents, search, context, infrastructure, and server modules
|
||||
|
||||
### Session ID Refactor
|
||||
|
||||
- Aligned tests with NULL-based memory session initialization pattern
|
||||
- Updated `SESSION_ID_ARCHITECTURE.md` documentation
|
||||
|
||||
### Other Improvements
|
||||
|
||||
- Added missing logger imports to 34 files for better observability
|
||||
- Updated esbuild and MCP SDK to latest versions
|
||||
- Removed `bun.lock` from version control
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.5.6...v8.5.7
|
||||
|
||||
## [v8.5.6] - 2026-01-04
|
||||
|
||||
## Major Architectural Refactoring
|
||||
|
||||
Decomposes monolithic services into modular, maintainable components:
|
||||
|
||||
### Worker Service
|
||||
Extracted infrastructure (GracefulShutdown, HealthMonitor, ProcessManager), server layer (ErrorHandler, Middleware, Server), and integrations (CursorHooksInstaller)
|
||||
|
||||
### Context Generator
|
||||
Split into ContextBuilder, ContextConfigLoader, ObservationCompiler, TokenCalculator, formatters (Color/Markdown), and section renderers (Header/Footer/Summary/Timeline)
|
||||
|
||||
### Search System
|
||||
Extracted SearchOrchestrator, ResultFormatter, TimelineBuilder, and strategy pattern (Chroma/SQLite/Hybrid search strategies) with dedicated filters (Date/Project/Type)
|
||||
|
||||
### Agent System
|
||||
Extracted shared logic into ResponseProcessor, ObservationBroadcaster, FallbackErrorHandler, and SessionCleanupHelper
|
||||
|
||||
### SQLite Layer
|
||||
Decomposed SessionStore into domain modules (observations, prompts, sessions, summaries, timeline) with proper type exports
|
||||
|
||||
## Bug Fixes
|
||||
- Fixed duplicate observation storage bug (observations stored multiple times when messages were batched)
|
||||
- Added duplicate observation cleanup script for production database remediation
|
||||
- Fixed FOREIGN KEY constraint and missing `failed_at_epoch` column issues
|
||||
|
||||
## Coming Next
|
||||
Comprehensive test suite in a new PR, targeting **v8.6.0**
|
||||
|
||||
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||||
|
||||
## [v8.5.5] - 2026-01-03
|
||||
|
||||
## Improved Error Handling and Logging
|
||||
|
||||
This patch release enhances error handling and logging across all worker services for better debugging and reliability.
|
||||
|
||||
### Changes
|
||||
- **Enhanced Error Logging**: Improved error context across SessionStore, SearchManager, SDKAgent, GeminiAgent, and OpenRouterAgent
|
||||
- **SearchManager**: Restored error handling for Chroma calls with improved logging
|
||||
- **SessionStore**: Enhanced error logging throughout database operations
|
||||
- **Bug Fix**: Fixed critical bug where `memory_session_id` could incorrectly equal `content_session_id`
|
||||
- **Hooks**: Streamlined error handling and loading states for better maintainability
|
||||
|
||||
### Investigation Reports
|
||||
- Added detailed analysis documents for generator failures and observation duplication regressions
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.5.4...v8.5.5
|
||||
|
||||
## [v8.5.4] - 2026-01-02
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
### Chroma Connection Error Handling
|
||||
Fixed a critical bug in ChromaSync where connection-related errors were misinterpreted as missing collections. The `ensureCollection()` method previously caught ALL errors and assumed they meant the collection doesn't exist, which caused connection errors to trigger unnecessary collection creation attempts. Now connection-related errors like "Not connected" are properly distinguished and re-thrown immediately, preventing false error handling paths and inappropriate fallback behavior.
|
||||
|
||||
### Removed Dead last_user_message Code
|
||||
Cleaned up dead code related to `last_user_message` handling in the summary flow. This field was being extracted from transcripts but never used anywhere - in Claude Code transcripts, "user" type messages are mostly tool_results rather than actual user input, and the user's original request is already stored in the user_prompts table. Removing this unused field eliminates confusing warnings like "Missing last_user_message when queueing summary". Changes span summary-hook, SessionRoutes, SessionManager, interface definitions, and all agent implementations.
|
||||
|
||||
## Improvements
|
||||
|
||||
### Enhanced Error Handling Across Services
|
||||
Comprehensive improvement to error handling across 8 core services:
|
||||
- **BranchManager** - Now logs recovery checkout failures
|
||||
- **PaginationHelper** - Logs when file paths are plain strings instead of valid JSON
|
||||
- **SDKAgent** - Enhanced logging for Claude executable detection failures
|
||||
- **SearchManager** - Logs plain string handling for files read and edited
|
||||
- **paths.ts** - Improved logging for git root detection failures
|
||||
- **timeline-formatting** - Enhanced JSON parsing errors with input previews
|
||||
- **transcript-parser** - Logs summary of parse errors after processing
|
||||
- **ChromaSync** - Logs full error context before attempting collection creation
|
||||
|
||||
### Error Handling Documentation & Tooling
|
||||
- Created `error-handling-baseline.txt` establishing baseline error handling practices
|
||||
- Documented error handling anti-pattern rules in CLAUDE.md
|
||||
- Added `detect-error-handling-antipatterns.ts` script to identify empty catch blocks, improper logging practices, and oversized try-catch blocks
|
||||
|
||||
## New Features
|
||||
|
||||
### Console Filter Bar with Log Parsing
|
||||
Implemented interactive log filtering in the viewer UI:
|
||||
- **Structured Log Parsing** - Extracts timestamp, level, component, correlation ID, and message content using regex pattern matching
|
||||
- **Level Filtering** - Toggle visibility for DEBUG, INFO, WARN, ERROR log levels
|
||||
- **Component Filtering** - Filter by 9 component types: HOOK, WORKER, SDK, PARSER, DB, SYSTEM, HTTP, SESSION, CHROMA
|
||||
- **Color-Coded Rendering** - Visual distinction with component-specific icons and log level colors
|
||||
- **Special Message Detection** - Recognizes markers like → (dataIn), ← (dataOut), ✓ (success), ✗ (failure), ⏱ (timing), [HAPPY-PATH]
|
||||
- **Smart Auto-Scroll** - Maintains scroll position when reviewing older logs
|
||||
- **Responsive Design** - Filter bar adapts to smaller screens
|
||||
|
||||
## [v8.5.3] - 2026-01-02
|
||||
|
||||
# 🛡️ Error Handling Hardening & Developer Tools
|
||||
|
||||
Version 8.5.3 introduces comprehensive error handling improvements that prevent silent failures and reduce debugging time from hours to minutes. This release also adds new developer tools for queue management and log monitoring.
|
||||
|
||||
---
|
||||
|
||||
## 🔴 Critical Error Handling Improvements
|
||||
|
||||
### The Problem
|
||||
A single overly-broad try-catch block caused a **10-hour debugging session** by silently swallowing errors. This pattern was pervasive throughout the codebase, creating invisible failure modes.
|
||||
|
||||
### The Solution
|
||||
|
||||
**Automated Anti-Pattern Detection** (`scripts/detect-error-handling-antipatterns.ts`)
|
||||
- Detects 7 categories of error handling anti-patterns
|
||||
- Enforces zero-tolerance policy for empty catch blocks
|
||||
- Identifies large try-catch blocks (>10 lines) that mask specific errors
|
||||
- Flags missing error logging that causes silent failures
|
||||
- Supports approved overrides with justification comments
|
||||
- Exit code 1 if critical issues detected (enforceable in CI)
|
||||
|
||||
**New Error Handling Standards** (Added to `CLAUDE.md`)
|
||||
- **5-Question Pre-Flight Checklist**: Required before writing any try-catch
|
||||
1. What SPECIFIC error am I catching?
|
||||
2. Show documentation proving this error can occur
|
||||
3. Why can't this error be prevented?
|
||||
4. What will the catch block DO?
|
||||
5. Why shouldn't this error propagate?
|
||||
- **Forbidden Patterns**: Empty catch, catch without logging, large try blocks, promise catch without handlers
|
||||
- **Allowed Patterns**: Specific errors, logged failures, minimal scope, explicit recovery
|
||||
- **Meta-Rule**: Uncertainty triggers research, NOT try-catch
|
||||
|
||||
### Fixes Applied
|
||||
|
||||
**Wave 1: Empty Catch Blocks** (5 files)
|
||||
- `import-xml-observations.ts` - Log skipped invalid JSON
|
||||
- `bun-path.ts` - Log when bun not in PATH
|
||||
- `cursor-utils.ts` - Log failed registry reads & corrupt MCP config
|
||||
- `worker-utils.ts` - Log failed health checks
|
||||
|
||||
**Wave 2: Promise Catches on Critical Paths** (8 locations)
|
||||
- `worker-service.ts` - Background initialization failures
|
||||
- `SDKAgent.ts` - Session processor errors (2 locations)
|
||||
- `GeminiAgent.ts` - Finalization failures (2 locations)
|
||||
- `OpenRouterAgent.ts` - Finalization failures (2 locations)
|
||||
- `SessionManager.ts` - Generator promise failures
|
||||
|
||||
**Wave 3: Comprehensive Audit** (29 catch blocks)
|
||||
- Added logging to 16 catch blocks (UI, servers, worker, routes, services)
|
||||
- Documented 13 intentional exceptions with justification comments
|
||||
- All patterns now follow error handling guidelines with appropriate log levels
|
||||
|
||||
### Approved Override System
|
||||
|
||||
For justified exceptions (performance-critical paths, expected failures), use:
|
||||
```typescript
|
||||
// [APPROVED OVERRIDE]: Brief technical justification
|
||||
try {
|
||||
// code
|
||||
} catch {
|
||||
// allowed exception
|
||||
}
|
||||
```
|
||||
|
||||
**Progress**: 163 anti-patterns → 26 approved overrides (84% reduction in silent failures)
|
||||
|
||||
---
|
||||
|
||||
## 🗂️ Queue Management Features
|
||||
|
||||
**New Commands**
|
||||
- `npm run queue:clear` - Interactive removal of failed messages
|
||||
- `npm run queue:clear -- --all` - Clear all messages (pending, processing, failed)
|
||||
- `npm run queue:clear -- --force` - Non-interactive mode
|
||||
|
||||
**HTTP API Endpoints**
|
||||
- `DELETE /api/pending-queue/failed` - Remove failed messages
|
||||
- `DELETE /api/pending-queue/all` - Complete queue reset
|
||||
|
||||
Failed messages exceed max retry count and remain for debugging. These commands provide clean queue maintenance.
|
||||
|
||||
---
|
||||
|
||||
## 🪵 Developer Console (Chrome DevTools Style)
|
||||
|
||||
**UI Improvements**
|
||||
- Bottom drawer console (slides up from bottom-left corner)
|
||||
- Draggable resize handle for height adjustment
|
||||
- Auto-refresh toggle (2s interval)
|
||||
- Clear logs button with confirmation
|
||||
- Monospace font (SF Mono/Monaco/Consolas)
|
||||
- Minimum height: 150px, adjustable to window height - 100px
|
||||
|
||||
**API Endpoints**
|
||||
- `GET /api/logs` - Fetch last 1000 lines of current day's log
|
||||
- `DELETE /api/logs` - Clear current log file
|
||||
|
||||
Logs viewer accessible via floating console button in UI.
|
||||
|
||||
---
|
||||
|
||||
## 📚 Architecture Documentation
|
||||
|
||||
**Session ID Architecture** (`docs/SESSION_ID_ARCHITECTURE.md`)
|
||||
- Comprehensive documentation of 1:1 session mapping guarantees
|
||||
- 19 validation tests proving UNIQUE constraints and resume consistency
|
||||
- Documents single-transition vulnerability (application-level enforcement)
|
||||
- Complete reference for session lifecycle management
|
||||
|
||||
---
|
||||
|
||||
## 📊 Impact Summary
|
||||
|
||||
- **Debugging Time**: 10 hours → minutes (proper error visibility)
|
||||
- **Test Coverage**: +19 critical architecture validation tests
|
||||
- **Silent Failures**: 84% reduction (163 → 26 approved exceptions)
|
||||
- **Protection**: Automated detection prevents regression
|
||||
- **Developer UX**: Console logs, queue management, comprehensive docs
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Technical Details
|
||||
|
||||
**Files Changed**: 25+ files across error handling, queue management, UI, and documentation
|
||||
|
||||
**Critical Path Protection**
|
||||
These files now have strict error propagation (no catch-and-continue):
|
||||
- `SDKAgent.ts`
|
||||
- `GeminiAgent.ts`
|
||||
- `OpenRouterAgent.ts`
|
||||
- `SessionStore.ts`
|
||||
- `worker-service.ts`
|
||||
|
||||
**Build Verification**: All changes tested, build successful
|
||||
|
||||
---
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v8.5.2...v8.5.3
|
||||
|
||||
## [v8.5.2] - 2025-12-31
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
### Fixed SDK Agent Memory Leak (#499)
|
||||
|
||||
Fixed a critical memory leak where Claude SDK child processes were never terminated after sessions completed. Over extended usage, this caused hundreds of orphaned processes consuming 40GB+ of RAM.
|
||||
|
||||
**Root Cause:**
|
||||
- When the SDK agent generator completed naturally (no more messages to process), the `AbortController` was never aborted
|
||||
- Child processes spawned by the Agent SDK remained running indefinitely
|
||||
- Sessions stayed in memory (by design for future events) but underlying processes were never cleaned up
|
||||
|
||||
**Fix:**
|
||||
- Added proper cleanup to SessionRoutes finally block
|
||||
- Now calls `abortController.abort()` when generator completes with no pending work
|
||||
- Creates new `AbortController` when crash recovery restarts generators
|
||||
- Ensures cleanup happens even if recovery logic fails
|
||||
|
||||
**Impact:**
|
||||
- Prevents orphaned `claude` processes from accumulating
|
||||
- Eliminates multi-gigabyte memory leaks during normal usage
|
||||
- Maintains crash recovery functionality with proper resource cleanup
|
||||
|
||||
Thanks to @yonnock for the detailed bug report and investigation in #499!
|
||||
|
||||
## [v8.5.1] - 2025-12-30
|
||||
|
||||
## Bug Fix
|
||||
|
||||
**Fixed**: Migration 17 column rename failing for databases in intermediate states (#481)
|
||||
|
||||
### Problem
|
||||
Migration 17 renamed session ID columns but used a single check to determine if ALL tables were migrated. This caused errors for databases in partial migration states:
|
||||
- `no such column: sdk_session_id` (when columns already renamed)
|
||||
- `table observations has no column named memory_session_id` (when not renamed)
|
||||
|
||||
### Solution
|
||||
- Rewrote migration 17 to check **each table individually** before renaming
|
||||
- Added `safeRenameColumn()` helper that handles all edge cases gracefully
|
||||
- Handles all database states: fresh, old, and partially migrated
|
||||
|
||||
### Who was affected
|
||||
- Users upgrading from pre-v8.2.6 versions
|
||||
- Users whose migration was interrupted (crash, restart, etc.)
|
||||
- Users who restored database from backup
|
||||
|
||||
---
|
||||
|
||||
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||||
|
||||
|
||||
@@ -62,7 +62,8 @@
|
||||
"icon": "lightbulb",
|
||||
"pages": [
|
||||
"context-engineering",
|
||||
"progressive-disclosure"
|
||||
"progressive-disclosure",
|
||||
"smart-explore-benchmark"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
---
|
||||
title: "Smart Explore Benchmark"
|
||||
description: "Token efficiency comparison between AST-based and traditional code exploration"
|
||||
---
|
||||
|
||||
# Smart Explore Benchmark
|
||||
|
||||
Smart Explore uses tree-sitter AST parsing to provide structural code navigation through three MCP tools: `smart_search`, `smart_outline`, and `smart_unfold`. This report documents a rigorous A/B comparison against the standard Explore agent (which uses Glob, Grep, and Read tools) to quantify the token savings and quality trade-offs.
|
||||
|
||||
## Executive Summary
|
||||
|
||||
| Metric | Smart Explore | Explore Agent | Advantage |
|
||||
|--------|:---:|:---:|---|
|
||||
| Discovery (cross-file search) | ~14,200 tokens | ~252,500 tokens | **17.8x cheaper** |
|
||||
| Targeted reads (specific symbols) | ~5,650 tokens | ~109,400 tokens | **19.4x cheaper** |
|
||||
| End-to-end (search + read) | ~4,200 tokens | ~45,000 tokens | **10-12x cheaper** |
|
||||
| Completeness | 5/5 full source returned | 4/5 (truncated longest method) | Smart Explore more reliable |
|
||||
| Speed | Under 2s per call | 5-66s per call | **10-30x faster** |
|
||||
|
||||
## Methodology
|
||||
|
||||
### Test Environment
|
||||
|
||||
- **Codebase**: claude-mem (`src/` directory, 194 TypeScript files, 1,206 parsed symbols)
|
||||
- **Model**: Claude Opus 4.6 for both approaches
|
||||
- **Measurement**: Token counts from tool response metadata (`total_tokens` for Explore agents, self-reported `~N tokens for folded view` for Smart Explore)
|
||||
|
||||
### Controls
|
||||
|
||||
The Explore agents were explicitly instructed: *"Do NOT use smart_search, smart_outline, or smart_unfold tools. Only use Glob, Grep, and Read tools."* This was verified necessary after an initial round where agents opportunistically used the Smart Explore tools, invalidating the comparison.
|
||||
|
||||
### Queries
|
||||
|
||||
Five queries were selected to represent common exploration tasks:
|
||||
|
||||
1. **"session processing"** -- Cross-cutting feature spanning multiple services
|
||||
2. **"shutdown"** -- Infrastructure concern touching 6+ files
|
||||
3. **"hook registration"** -- Architecture question about plugin system
|
||||
4. **"sqlite database"** -- Technology-specific search across the data layer
|
||||
5. **"worker-service.ts outline"** -- Single large file (1,225 lines) structural understanding
|
||||
|
||||
## Round 1: Discovery
|
||||
|
||||
*"What exists and where is it?"* -- Finding relevant files and symbols across the codebase.
|
||||
|
||||
### Results
|
||||
|
||||
| Query | Smart Explore | Explore Agent | Ratio | Explore Tool Calls |
|
||||
|-------|:---:|:---:|:---:|:---:|
|
||||
| session processing | ~4,391 t | 51,659 t | **11.8x** | 15 |
|
||||
| shutdown | ~3,852 t | 51,523 t | **13.4x** | 18 |
|
||||
| hook registration | ~1,930 t | 51,688 t | **26.8x** | 37 |
|
||||
| sqlite database | ~2,543 t | 58,633 t | **23.1x** | 16 |
|
||||
| worker-service outline | ~1,500 t | 38,973 t | **26.0x** | 15 |
|
||||
| **Total** | **~14,216 t** | **252,476 t** | **17.8x** | **101** |
|
||||
|
||||
### What Each Returned
|
||||
|
||||
**Smart Explore** (1 tool call each): 10 ranked symbols with signatures, line numbers, and JSDoc summaries, plus folded structural views of all matching files showing every function/class/interface with bodies collapsed.
|
||||
|
||||
**Explore Agent** (15-37 tool calls each): Synthesized narrative reports with architecture diagrams, design pattern analysis, data flow explanations, complete interface dumps, and file structure maps. Significantly more explanatory prose.
|
||||
|
||||
### Analysis
|
||||
|
||||
The token gap is widest for narrowly-scoped queries ("hook registration" at 26.8x) because the Explore agent reads multiple full files to find relatively few relevant symbols. For broad queries ("session processing" at 11.8x), more of the file content is relevant, narrowing the ratio.
|
||||
|
||||
Smart Explore's consistent 1-tool-call pattern means its cost is predictable. The Explore agent's cost varies with how many files it reads and how much it synthesizes -- ranging from 15 to 37 tool calls for comparable scope.
|
||||
|
||||
## Round 2: Targeted Reads
|
||||
|
||||
*"Show me this specific function."* -- Reading the implementation of a known symbol after discovery.
|
||||
|
||||
Based on the Round 1 results, five specific symbols were selected as natural drill-down targets:
|
||||
|
||||
| Target Symbol | File | Lines |
|
||||
|---------------|------|:---:|
|
||||
| `SessionManager.initializeSession` | services/worker/SessionManager.ts | 135 |
|
||||
| `performGracefulShutdown` | services/infrastructure/GracefulShutdown.ts | 48 |
|
||||
| `hookCommand` | cli/hook-command.ts | 45 |
|
||||
| `DatabaseManager.initialize` | services/sqlite/Database.ts | 27 |
|
||||
| `WorkerService.startSessionProcessor` | services/worker-service.ts | 158 |
|
||||
|
||||
### Results
|
||||
|
||||
| Symbol | Smart Unfold | Explore Agent | Ratio | Completeness |
|
||||
|--------|:---:|:---:|:---:|---|
|
||||
| initializeSession (135 lines) | ~1,800 t | 27,816 t | **15.5x** | Both returned full source |
|
||||
| performGracefulShutdown (48 lines) | ~700 t | 19,621 t | **28.0x** | Both returned full source |
|
||||
| hookCommand (45 lines) | ~650 t | 18,680 t | **28.7x** | Both returned full source |
|
||||
| DatabaseManager.initialize (27 lines) | ~400 t | 22,334 t | **55.8x** | Both returned full source |
|
||||
| startSessionProcessor (158 lines) | ~2,100 t | 20,906 t | **10.0x** | Smart Unfold: complete. Explore: **truncated** |
|
||||
| **Total** | **~5,650 t** | **109,357 t** | **19.4x** | |
|
||||
|
||||
### Analysis
|
||||
|
||||
**The ratio scales inversely with symbol size.** The smallest function (`initialize`, 27 lines) shows the biggest gap at 55.8x because the Explore agent still reads the entire 235-line file to extract 27 lines. The largest method (`startSessionProcessor`, 158 lines) narrows to 10x since more of the file is "useful."
|
||||
|
||||
**Smart Unfold returned more complete code.** For the longest method (158 lines), the Explore agent truncated the error handling section with "... error handling continues ...", while `smart_unfold` returned the complete implementation. This is because smart_unfold extracts by AST node boundaries, guaranteeing completeness regardless of symbol size.
|
||||
|
||||
**Explore agents add zero unique information for targeted reads.** When you already know the file path and symbol name, the agent's overhead is pure waste -- it reads the file, locates the function, and echoes it back. The only addition is a brief explanatory paragraph.
|
||||
|
||||
## Combined Workflow
|
||||
|
||||
The realistic workflow is discovery followed by targeted reading. Here is the end-to-end cost comparison for understanding a single function:
|
||||
|
||||
### Smart Explore: search + unfold
|
||||
|
||||
```
|
||||
smart_search("shutdown", path="./src") ~3,852 tokens
|
||||
smart_unfold("GracefulShutdown.ts", "performGracefulShutdown") ~700 tokens
|
||||
────────────────────────────────────────────────────────────────
|
||||
Total: ~4,552 tokens (2 tool calls, under 3 seconds)
|
||||
```
|
||||
|
||||
### Explore Agent: single query
|
||||
|
||||
```
|
||||
"Find and explain the shutdown logic" ~51,523 tokens
|
||||
────────────────────────────────────────────────────────────────
|
||||
Total: ~51,523 tokens (18 tool calls, ~43 seconds)
|
||||
```
|
||||
|
||||
**End-to-end ratio: 11.3x** -- and the Smart Explore workflow gives you the actual source code, while the Explore agent gives you a prose summary that may paraphrase or truncate.
|
||||
|
||||
## Quality Assessment
|
||||
|
||||
Neither approach is universally better. They optimize for different outcomes.
|
||||
|
||||
### Smart Explore Strengths
|
||||
|
||||
- **Predictable cost**: 1 tool call per operation, consistent token ranges
|
||||
- **Complete source code**: AST-based extraction guarantees full symbol bodies
|
||||
- **Structural context**: Folded views show every symbol in matching files
|
||||
- **Speed**: Sub-second responses enable rapid iteration
|
||||
- **Composability**: Search, outline, and unfold chain naturally
|
||||
|
||||
### Explore Agent Strengths
|
||||
|
||||
- **Synthesized understanding**: Produces architecture narratives, data flow diagrams, and design pattern analysis
|
||||
- **Cross-cutting explanation**: Connects concepts across files that individual symbol reads cannot
|
||||
- **Onboarding quality**: Output reads like documentation, not raw code
|
||||
- **Error handling insight**: Identifies edge cases and design decisions that require reading multiple related functions
|
||||
- **No prior knowledge needed**: Can answer open-ended questions without knowing file paths or symbol names
|
||||
|
||||
### Quality by Task Type
|
||||
|
||||
| Task | Better Tool | Why |
|
||||
|------|-------------|-----|
|
||||
| "Where is X defined?" | Smart Explore | One call, exact answer |
|
||||
| "What functions are in this file?" | Smart Explore | Outline returns complete structural map |
|
||||
| "Show me this function" | Smart Explore | Unfold returns exact source, never truncates |
|
||||
| "How does feature X work end-to-end?" | Explore Agent | Reads multiple files and synthesizes narrative |
|
||||
| "What design patterns are used here?" | Explore Agent | Requires reading and interpreting, not just extracting |
|
||||
| "Help me understand this codebase" | Explore Agent | Produces onboarding-quality documentation |
|
||||
|
||||
## When to Use Which
|
||||
|
||||
**Use Smart Explore when:**
|
||||
- You know what you are looking for (function name, concept, file)
|
||||
- You need source code, not explanation
|
||||
- You are iterating quickly (read, modify, read again)
|
||||
- Token budget matters (large codebases, long sessions)
|
||||
- You need file structure at a glance
|
||||
|
||||
**Use the Explore Agent when:**
|
||||
- You need synthesized cross-cutting understanding
|
||||
- The question is open-ended ("how does this system work?")
|
||||
- You are writing documentation or architecture reviews
|
||||
- You need to understand *why*, not just *what*
|
||||
- You are onboarding to an unfamiliar codebase
|
||||
|
||||
**Use both when:**
|
||||
- Start with Smart Explore for discovery and navigation
|
||||
- Escalate to Explore Agent only for deep analysis that requires multi-file synthesis
|
||||
- This hybrid approach captures most of the token savings while preserving access to deep understanding when needed
|
||||
|
||||
## Token Economics Reference
|
||||
|
||||
| Operation | Tokens | Use Case |
|
||||
|-----------|:---:|----------|
|
||||
| `smart_search` | 2,000-6,000 | Cross-file symbol discovery |
|
||||
| `smart_outline` | 1,000-2,000 | Single file structural map |
|
||||
| `smart_unfold` | 400-2,100 | Single symbol full source |
|
||||
| `smart_search` + `smart_unfold` | 3,000-8,000 | End-to-end: find and read |
|
||||
| Explore Agent (targeted) | 18,000-28,000 | Single function with explanation |
|
||||
| Explore Agent (cross-cutting) | 39,000-59,000 | Architecture-level understanding |
|
||||
| Read (full file) | 8,000-15,000+ | Complete file contents |
|
||||
|
||||
### Savings by Workflow
|
||||
|
||||
| Workflow | Smart Explore | Traditional | Savings |
|
||||
|----------|:---:|:---:|:---:|
|
||||
| Understand one file | outline + unfold (~3,100 t) | Read full file (~12,000 t) | **4x** |
|
||||
| Find a function across codebase | search (~3,500 t) | Explore agent (~50,000 t) | **14x** |
|
||||
| Find and read a specific function | search + unfold (~4,500 t) | Explore agent (~50,000 t) | **11x** |
|
||||
| Navigate a 1,200-line file | outline (~1,500 t) | Read full file (~12,000 t) | **8x** |
|
||||
+11
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "10.4.1",
|
||||
"version": "10.5.3",
|
||||
"description": "Memory compression system for Claude Code - persist context across sessions",
|
||||
"keywords": [
|
||||
"claude",
|
||||
@@ -117,6 +117,16 @@
|
||||
"@types/react-dom": "^18.3.0",
|
||||
"esbuild": "^0.27.2",
|
||||
"np": "^11.0.2",
|
||||
"tree-sitter-c": "^0.24.1",
|
||||
"tree-sitter-cli": "^0.26.5",
|
||||
"tree-sitter-cpp": "^0.23.4",
|
||||
"tree-sitter-go": "^0.25.0",
|
||||
"tree-sitter-java": "^0.23.5",
|
||||
"tree-sitter-javascript": "^0.25.0",
|
||||
"tree-sitter-python": "^0.25.0",
|
||||
"tree-sitter-ruby": "^0.23.1",
|
||||
"tree-sitter-rust": "^0.24.0",
|
||||
"tree-sitter-typescript": "^0.23.2",
|
||||
"tsx": "^4.20.6",
|
||||
"typescript": "^5.3.0"
|
||||
}
|
||||
|
||||
@@ -1,52 +0,0 @@
|
||||
# Fix: SessionStart Hook "startup hook error" — Worker Not Waiting
|
||||
|
||||
## Root Cause
|
||||
|
||||
The **installed plugin** (`~/.claude/plugins/marketplaces/thedotmack/`) is version **10.2.5** and has **none** of the recent fixes:
|
||||
|
||||
| Fix | Repo Status | Installed Status |
|
||||
|-----|-------------|-----------------|
|
||||
| Hook group split (smart-install isolated from worker start) | In `plugin/hooks/hooks.json` | **Missing** — all 3 hooks in one group, smart-install failure blocks worker |
|
||||
| `waitForReadiness()` after spawn | In `src/services/infrastructure/HealthMonitor.ts` | **Missing** — 0 occurrences in installed `worker-service.cjs` |
|
||||
| Early `initializationCompleteFlag` (after DB+search, not MCP) | In `src/services/worker-service.ts` | **Missing** — flag set after MCP connection (5+ minute wait) |
|
||||
|
||||
The changes exist in source code but were **never built and synced** to the installed location.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Build and Sync
|
||||
|
||||
```bash
|
||||
npm run build-and-sync
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
# 1. Confirm waitForReadiness exists in installed build
|
||||
grep -c "waitForReadiness" ~/.claude/plugins/marketplaces/thedotmack/plugin/scripts/worker-service.cjs
|
||||
# Expected: > 0
|
||||
|
||||
# 2. Confirm hooks.json has two SessionStart groups (the split)
|
||||
python3 -c "import json; d=json.load(open('$(echo $HOME)/.claude/plugins/marketplaces/thedotmack/plugin/hooks/hooks.json')); print('SessionStart groups:', len(d['hooks']['SessionStart']))"
|
||||
# Expected: 2
|
||||
|
||||
# 3. Confirm initializationCompleteFlag is set before MCP connection
|
||||
grep -n "Core initialization complete" ~/.claude/plugins/marketplaces/thedotmack/plugin/scripts/worker-service.cjs | head -1
|
||||
# Expected: appears BEFORE "MCP server connected"
|
||||
```
|
||||
|
||||
## Phase 2: Restart Worker and Test
|
||||
|
||||
```bash
|
||||
# Stop existing worker
|
||||
bun plugin/scripts/worker-service.cjs stop
|
||||
|
||||
# Verify stopped
|
||||
curl -s http://127.0.0.1:37777/api/health && echo "STILL RUNNING" || echo "STOPPED"
|
||||
```
|
||||
|
||||
Then start a new Claude Code session and verify:
|
||||
- No "SessionStart:startup hook error" messages
|
||||
- Worker is running: `curl http://127.0.0.1:37777/api/health`
|
||||
- Readiness endpoint works: `curl http://127.0.0.1:37777/api/readiness`
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "10.4.0",
|
||||
"version": "10.5.3",
|
||||
"description": "Persistent memory system for Claude Code - seamlessly preserve context across sessions",
|
||||
"author": {
|
||||
"name": "Alex Newman"
|
||||
|
||||
+8
-23
@@ -7,7 +7,7 @@
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "${CLAUDE_PLUGIN_ROOT}/scripts/setup.sh",
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; \"$_R/scripts/setup.sh\"",
|
||||
"timeout": 300
|
||||
}
|
||||
]
|
||||
@@ -19,7 +19,7 @@
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/smart-install.js\"",
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/smart-install.js\"",
|
||||
"timeout": 300
|
||||
}
|
||||
]
|
||||
@@ -29,12 +29,12 @@
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" start",
|
||||
"timeout": 60
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" hook claude-code context",
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code context",
|
||||
"timeout": 60
|
||||
}
|
||||
]
|
||||
@@ -45,12 +45,7 @@
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
|
||||
"timeout": 60
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" hook claude-code session-init",
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code session-init",
|
||||
"timeout": 60
|
||||
}
|
||||
]
|
||||
@@ -62,12 +57,7 @@
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
|
||||
"timeout": 60
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" hook claude-code observation",
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code observation",
|
||||
"timeout": 120
|
||||
}
|
||||
]
|
||||
@@ -78,17 +68,12 @@
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" start",
|
||||
"timeout": 60
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" hook claude-code summarize",
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code summarize",
|
||||
"timeout": 120
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/scripts/bun-runner.js\" \"${CLAUDE_PLUGIN_ROOT}/scripts/worker-service.cjs\" hook claude-code session-complete",
|
||||
"command": "_R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code session-complete",
|
||||
"timeout": 30
|
||||
}
|
||||
]
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"name": "Law Study (Chill)",
|
||||
"prompts": {
|
||||
"recording_focus": "WHAT TO RECORD (HIGH SIGNAL ONLY)\n----------------------------------\nOnly record what would be painful to reconstruct later:\n- Issue-spotting triggers: specific fact patterns that signal a testable issue\n- Professor's explicit emphasis, frameworks, or exam tips\n- Counterintuitive holdings or gotchas that contradict intuition\n- Cross-case connections that reframe how a doctrine works\n- A synthesized rule only if it distills something non-obvious from multiple sources\n\nSkip anything that could be looked up in a casebook in under 60 seconds.\n\nUse verbs like: held, established, revealed, distinguished, flagged",
|
||||
"skip_guidance": "WHEN TO SKIP (LIBERAL — WHEN IN DOUBT, SKIP)\n---------------------------------------------\nSkip freely:\n- All case briefs, even condensed ones, unless the holding is counterintuitive\n- Any rule or doctrine stated plainly in the casebook without nuance\n- Definitions of standard legal terms\n- Procedural history\n- Any fact pattern or case that wasn't specifically emphasized by the professor\n- Anything you could find again in under 60 seconds\n- **No output necessary if skipping.**"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,85 @@
|
||||
# Legal Study Assistant
|
||||
|
||||
You are a rigorous legal study partner for a law student. Your job is to help them understand the law deeply enough to reason through novel fact patterns independently on exams and in practice.
|
||||
|
||||
---
|
||||
|
||||
## Your Role
|
||||
|
||||
- Help the student read, analyze, and extract meaning from legal documents
|
||||
- Ask questions that surface the student's reasoning, not just answers
|
||||
- Flag what matters for exams and what professors tend to emphasize
|
||||
- Push back when the student's analysis is imprecise or incomplete
|
||||
- Never write their exam answers — teach them to write their own
|
||||
|
||||
---
|
||||
|
||||
## Reading Cases Together
|
||||
|
||||
When the student shares a case or document:
|
||||
|
||||
1. Read it fully before saying anything. No skimming.
|
||||
2. Identify the procedural posture, then the issue, then the holding, then the reasoning.
|
||||
3. Separate holding from dicta explicitly — this distinction is always fair game.
|
||||
4. Surface ambiguity when the court was evasive. That ambiguity is often the exam question.
|
||||
5. Ask: "Which facts were outcome-determinative? What if those facts changed?"
|
||||
|
||||
**Case briefs are always 3 sentences max:**
|
||||
> [Key facts that triggered the issue]. The court held [holding + extracted rule]. [Why this rule exists or how it fits the doctrine — only if non-obvious.]
|
||||
|
||||
---
|
||||
|
||||
## Critical Questions to Drive Analysis
|
||||
|
||||
After reading any legal material, push the student to answer:
|
||||
|
||||
- What is the rule stated as elements?
|
||||
- What did the dissent argue and why does it matter?
|
||||
- How does this fit — or conflict with — earlier cases?
|
||||
- What fact pattern on an exam triggers this rule?
|
||||
- What does the professor emphasize about this? Their framing is the exam framing.
|
||||
- Is the law settled or contested here?
|
||||
|
||||
---
|
||||
|
||||
## Issue Spotting
|
||||
|
||||
When working through a fact pattern:
|
||||
|
||||
1. Read the entire hypo before naming any issues.
|
||||
2. List every potential claim and defense — err toward inclusion.
|
||||
3. For each issue: rule → application to these specific facts → where the argument turns.
|
||||
4. Treat "irrelevant" facts as planted triggers. Nothing in an exam hypo is accidental.
|
||||
5. Calibrate to the professor's emphasis — they wrote the exam.
|
||||
|
||||
---
|
||||
|
||||
## Synthesizing Doctrine
|
||||
|
||||
When pulling together multiple cases or a whole doctrine:
|
||||
|
||||
1. Find the common principle across all the cases.
|
||||
2. Build the rule as a spectrum or taxonomy when cases represent different scenarios.
|
||||
3. State the limiting principle — where does this rule stop and why.
|
||||
4. Majority rule first, then minority positions with their rationale.
|
||||
5. Identify the live tension — what the courts haven't resolved yet.
|
||||
|
||||
---
|
||||
|
||||
## Tone and Pace
|
||||
|
||||
- Be direct. Law school trains precision — model it.
|
||||
- When the student is vague, say so and ask them to be specific.
|
||||
- Celebrate when they spot something sharp. Legal reasoning is hard.
|
||||
- Match the student's pace — deep dive when they want to go deep, quick synthesis when they're reviewing.
|
||||
|
||||
---
|
||||
|
||||
## Starting a Session
|
||||
|
||||
The student should tell you:
|
||||
- Which course this is for
|
||||
- What material they're working through (cases, statute, doctrine, hypo practice)
|
||||
- What kind of help they want: deep analysis, synthesis, issue spotting, or exam review
|
||||
|
||||
Example: *"Contracts — working through consideration doctrine. Here are four cases. Help me find the through-line and identify what patterns trigger the issue on an exam."*
|
||||
@@ -0,0 +1,120 @@
|
||||
{
|
||||
"name": "Law Study",
|
||||
"description": "Legal study and exam preparation for law students",
|
||||
"version": "1.0.0",
|
||||
"observation_types": [
|
||||
{
|
||||
"id": "case-holding",
|
||||
"label": "Case Holding",
|
||||
"description": "Case brief (2-3 sentences: key facts + holding) with extracted legal rule",
|
||||
"emoji": "⚖️",
|
||||
"work_emoji": "📖"
|
||||
},
|
||||
{
|
||||
"id": "issue-pattern",
|
||||
"label": "Issue Pattern",
|
||||
"description": "Exam trigger or fact pattern that signals a legal issue to spot",
|
||||
"emoji": "🎯",
|
||||
"work_emoji": "🔍"
|
||||
},
|
||||
{
|
||||
"id": "prof-framework",
|
||||
"label": "Prof Framework",
|
||||
"description": "Professor's analytical lens, emphasis, or approach to a topic or doctrine",
|
||||
"emoji": "🧑🏫",
|
||||
"work_emoji": "📝"
|
||||
},
|
||||
{
|
||||
"id": "doctrine-rule",
|
||||
"label": "Doctrine / Rule",
|
||||
"description": "Legal test, standard, or doctrine synthesized from cases, statutes, or restatements",
|
||||
"emoji": "📜",
|
||||
"work_emoji": "🔍"
|
||||
},
|
||||
{
|
||||
"id": "argument-structure",
|
||||
"label": "Argument Structure",
|
||||
"description": "Legal argument or counter-argument worked through with analytical steps",
|
||||
"emoji": "🗣️",
|
||||
"work_emoji": "⚖️"
|
||||
},
|
||||
{
|
||||
"id": "cross-case-connection",
|
||||
"label": "Cross-Case Connection",
|
||||
"description": "Insight linking multiple cases, doctrines, or topics that reveals a deeper principle",
|
||||
"emoji": "🔗",
|
||||
"work_emoji": "🔍"
|
||||
}
|
||||
],
|
||||
"observation_concepts": [
|
||||
{
|
||||
"id": "exam-relevant",
|
||||
"label": "Exam Relevant",
|
||||
"description": "Flagged by professor or likely to appear on exams based on emphasis"
|
||||
},
|
||||
{
|
||||
"id": "minority-position",
|
||||
"label": "Minority Position",
|
||||
"description": "Dissent, minority rule, or alternative jurisdictional approach worth knowing"
|
||||
},
|
||||
{
|
||||
"id": "gotcha",
|
||||
"label": "Gotcha",
|
||||
"description": "Subtle nuance, counterintuitive result, or common mistake students get wrong"
|
||||
},
|
||||
{
|
||||
"id": "unsettled-law",
|
||||
"label": "Unsettled Law",
|
||||
"description": "Circuit split, open question, or evolving area of law"
|
||||
},
|
||||
{
|
||||
"id": "policy-rationale",
|
||||
"label": "Policy Rationale",
|
||||
"description": "Normative or policy argument underlying a rule or holding"
|
||||
},
|
||||
{
|
||||
"id": "course-theme",
|
||||
"label": "Course Theme",
|
||||
"description": "How this case or rule connects to the overarching narrative or theory of the course"
|
||||
}
|
||||
],
|
||||
"prompts": {
|
||||
"system_identity": "You are Claude-Mem, a specialized observer tool for creating searchable memory FOR FUTURE SESSIONS.\n\nCRITICAL: Record what was READ, ANALYZED, SYNTHESIZED, or LEARNED about the law, not what you (the observer) are doing.\n\nYou do not have access to tools. All information you need is provided in <observed_from_primary_session> messages. Create observations from what you observe - no investigation needed.",
|
||||
"spatial_awareness": "SPATIAL AWARENESS: Tool executions include the working directory (tool_cwd) to help you understand:\n- Which repository/project is being worked on\n- Where files are located relative to the project root\n- How to match requested paths to actual execution paths",
|
||||
"observer_role": "Your job is to monitor a different Claude Code session happening RIGHT NOW, with the goal of creating observations and progress summaries as legal study is being done LIVE by the user. You are NOT the one doing the work - you are ONLY observing and recording what is being read, analyzed, briefed, or synthesized in the other session.",
|
||||
"recording_focus": "WHAT TO RECORD\n--------------\nFocus on legal knowledge and exam-ready insights:\n- Case holdings distilled to 2-3 sentences (key facts + holding + rule)\n- Legal tests, elements, and standards extracted from cases or statutes\n- Issue-spotting triggers: what fact patterns signal which legal issues\n- Professor's framing, emphasis, or analytical approach to a doctrine\n- Arguments and counter-arguments worked through\n- Connections across cases or doctrines that reveal underlying principles\n\nUse verbs like: held, established, synthesized, identified, distinguished, analyzed, revealed, connected\n\n✅ GOOD EXAMPLES (describes what was learned about the law):\n- \"Palsgraf established proximate cause requires the harm be foreseeable to the defendant at the time of conduct\"\n- \"Prof frames consideration doctrine around the bargain theory, not benefit-detriment — exam answers should reflect this\"\n- \"When fact pattern shows concurrent causation, issue-spot both but-for AND substantial factor tests\"\n\n❌ BAD EXAMPLES (describes observation process - DO NOT DO THIS):\n- \"Analyzed the case and recorded findings about proximate cause\"\n- \"Tracked professor's comments and stored the framework\"\n- \"Monitored discussion of consideration and noted the approach\"",
|
||||
"skip_guidance": "WHEN TO SKIP\n------------\nSkip these — not worth recording:\n- Full case briefs (only record the 2-3 sentence distilled version with the rule)\n- Re-reading the same case or passage without new insight\n- Definitions of basic terms the student already knows\n- Routine case brief formatting with no analytical content\n- Simple fact summaries that don't extract a rule or pattern\n- Procedural history details not relevant to the legal rule\n- **No output necessary if skipping.**",
|
||||
"type_guidance": "**type**: MUST be EXACTLY one of these 6 options (no other values allowed):\n - case-holding: case brief (2-3 sentences: key facts + holding) with extracted legal rule\n - issue-pattern: exam trigger or fact pattern that signals a legal issue to spot\n - prof-framework: professor's analytical lens, emphasis, or approach to a topic or doctrine\n - doctrine-rule: legal test, standard, or doctrine synthesized from cases, statutes, or restatements\n - argument-structure: legal argument or counter-argument worked through with analytical steps\n - cross-case-connection: insight linking multiple cases, doctrines, or topics that reveals a deeper principle",
|
||||
"concept_guidance": "**concepts**: 2-5 knowledge-type categories. MUST use ONLY these exact keywords:\n - exam-relevant: flagged by professor or likely to appear on exams\n - minority-position: dissent, minority rule, or alternative jurisdictional approach\n - gotcha: subtle nuance, counterintuitive result, or common mistake\n - unsettled-law: circuit split, open question, or evolving area\n - policy-rationale: normative or policy argument underlying a rule\n - course-theme: connects to the overarching narrative or theory of the course\n\n IMPORTANT: Do NOT include the observation type (case-holding/issue-pattern/etc.) as a concept.\n Types and concepts are separate dimensions.",
|
||||
"field_guidance": "**facts**: Concise, self-contained statements\nEach fact is ONE piece of information\n No pronouns - each fact must stand alone\n Include specific details: case names, rule elements, test names, jurisdiction\n\n**files**: All files or documents read (full paths from project root)",
|
||||
"output_format_header": "OUTPUT FORMAT\n-------------\nOutput observations using this XML structure:",
|
||||
"format_examples": "",
|
||||
"footer": "IMPORTANT! DO NOT do any work right now other than generating this OBSERVATIONS from tool use messages - and remember that you are a memory agent designed to summarize a DIFFERENT claude code session, not this one.\n\nNever reference yourself or your own actions. Do not output anything other than the observation content formatted in the XML structure above. All other output is ignored by the system, and the system has been designed to be smart about token usage. Please spend your tokens wisely on useful observations.\n\nRemember that we record these observations as a way of helping us stay on track with our progress, and to help us keep important decisions and changes at the forefront of our minds! :) Thank you so much for your help!",
|
||||
|
||||
"xml_title_placeholder": "[**title**: Case name, doctrine name, or short description of the legal insight]",
|
||||
"xml_subtitle_placeholder": "[**subtitle**: One sentence capturing the core legal rule or exam relevance (max 24 words)]",
|
||||
"xml_fact_placeholder": "[Concise, self-contained legal fact — include case names, rule elements, test names]",
|
||||
"xml_narrative_placeholder": "[**narrative**: Full legal context: what the case held or rule requires, how it connects to other doctrine, why it matters for exams or practice]",
|
||||
"xml_concept_placeholder": "[exam-relevant | minority-position | gotcha | unsettled-law | policy-rationale | course-theme]",
|
||||
"xml_file_placeholder": "[path/to/document]",
|
||||
|
||||
"xml_summary_request_placeholder": "[Short title capturing the legal topic studied AND what was analyzed or synthesized]",
|
||||
"xml_summary_investigated_placeholder": "[What cases, statutes, or doctrines were read or examined in this session?]",
|
||||
"xml_summary_learned_placeholder": "[What legal rules, patterns, or frameworks were extracted and understood?]",
|
||||
"xml_summary_completed_placeholder": "[What study work was completed? Which cases briefed, which doctrines synthesized, which issue patterns identified?]",
|
||||
"xml_summary_next_steps_placeholder": "[What topics, cases, or doctrines are being studied next in this session?]",
|
||||
"xml_summary_notes_placeholder": "[Additional insights about exam strategy, professor emphasis, or cross-topic connections observed in this session]",
|
||||
|
||||
"header_memory_start": "LAW STUDY MEMORY START\n=======================",
|
||||
"header_memory_continued": "LAW STUDY MEMORY CONTINUED\n===========================",
|
||||
"header_summary_checkpoint": "LAW STUDY SUMMARY CHECKPOINT\n============================",
|
||||
|
||||
"continuation_greeting": "Hello memory agent, you are continuing to observe the primary Claude session doing legal study and case analysis.",
|
||||
"continuation_instruction": "IMPORTANT: Continue generating observations from tool use messages using the XML structure below.",
|
||||
|
||||
"summary_instruction": "Write progress notes of what legal material was studied, what rules and patterns were extracted, and what's next. This is a checkpoint to capture study progress so far. The session is ongoing - more cases or doctrines may be analyzed after this summary. Write \"next_steps\" as the current study trajectory (what topics or cases are actively being worked through), not as post-session plans. Always write at least a minimal summary explaining current progress, even if study is still early, so that users see a summary output tied to each study block.",
|
||||
"summary_context_label": "Claude's Full Response to User:",
|
||||
"summary_format_instruction": "Respond in this XML format:",
|
||||
"summary_footer": "IMPORTANT! DO NOT do any work right now other than generating this next PROGRESS SUMMARY - and remember that you are a memory agent designed to summarize a DIFFERENT claude code session, not this one.\n\nNever reference yourself or your own actions. Do not output anything other than the summary content formatted in the XML structure above. All other output is ignored by the system, and the system has been designed to be smart about token usage. Please spend your tokens wisely on useful summary content.\n\nThank you, this summary will be very useful for keeping track of legal study progress!"
|
||||
}
|
||||
}
|
||||
+11
-2
@@ -1,11 +1,20 @@
|
||||
{
|
||||
"name": "claude-mem-plugin",
|
||||
"version": "10.4.1",
|
||||
"version": "10.5.3",
|
||||
"private": true,
|
||||
"description": "Runtime dependencies for claude-mem bundled hooks",
|
||||
"type": "module",
|
||||
"dependencies": {
|
||||
"@chroma-core/default-embed": "^0.1.9"
|
||||
"tree-sitter-cli": "^0.26.5",
|
||||
"tree-sitter-c": "^0.24.1",
|
||||
"tree-sitter-cpp": "^0.23.4",
|
||||
"tree-sitter-go": "^0.25.0",
|
||||
"tree-sitter-java": "^0.23.5",
|
||||
"tree-sitter-javascript": "^0.25.0",
|
||||
"tree-sitter-python": "^0.25.0",
|
||||
"tree-sitter-ruby": "^0.23.1",
|
||||
"tree-sitter-rust": "^0.24.0",
|
||||
"tree-sitter-typescript": "^0.23.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18.0.0",
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
Never read built source files in this directory. These are compiled outputs — read the source files in `src/` instead.
|
||||
|
||||
<claude-mem-context>
|
||||
# Recent Activity
|
||||
|
||||
|
||||
@@ -13,11 +13,36 @@
|
||||
*/
|
||||
import { spawnSync, spawn } from 'child_process';
|
||||
import { existsSync, readFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { join, dirname, resolve } from 'path';
|
||||
import { homedir } from 'os';
|
||||
import { fileURLToPath } from 'url';
|
||||
|
||||
const IS_WINDOWS = process.platform === 'win32';
|
||||
|
||||
// Self-resolve plugin root when CLAUDE_PLUGIN_ROOT is not set by Claude Code.
|
||||
// Upstream bug: anthropics/claude-code#24529 — Stop hooks (and on Linux, all hooks)
|
||||
// don't receive CLAUDE_PLUGIN_ROOT, causing script paths to resolve to /scripts/...
|
||||
// which doesn't exist. This fallback derives the plugin root from bun-runner.js's
|
||||
// own filesystem location (this file lives in <plugin-root>/scripts/).
|
||||
const __bun_runner_dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const RESOLVED_PLUGIN_ROOT = process.env.CLAUDE_PLUGIN_ROOT || resolve(__bun_runner_dirname, '..');
|
||||
|
||||
/**
|
||||
* Fix script path arguments that were broken by empty CLAUDE_PLUGIN_ROOT.
|
||||
* When CLAUDE_PLUGIN_ROOT is empty, "${CLAUDE_PLUGIN_ROOT}/scripts/foo.cjs"
|
||||
* expands to "/scripts/foo.cjs" which doesn't exist. Detect this and rewrite
|
||||
* the path using our self-resolved plugin root.
|
||||
*/
|
||||
function fixBrokenScriptPath(argPath) {
|
||||
if (argPath.startsWith('/scripts/') && !existsSync(argPath)) {
|
||||
const fixedPath = join(RESOLVED_PLUGIN_ROOT, argPath);
|
||||
if (existsSync(fixedPath)) {
|
||||
return fixedPath;
|
||||
}
|
||||
}
|
||||
return argPath;
|
||||
}
|
||||
|
||||
/**
|
||||
* Find Bun executable - checks PATH first, then common install locations
|
||||
*/
|
||||
@@ -80,6 +105,9 @@ if (args.length === 0) {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Fix broken script paths caused by empty CLAUDE_PLUGIN_ROOT (#1215)
|
||||
args[0] = fixBrokenScriptPath(args[0]);
|
||||
|
||||
const bunPath = findBun();
|
||||
|
||||
if (!bunPath) {
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -1,228 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# claude-mem Setup Hook
|
||||
# Ensures dependencies are installed before plugin runs
|
||||
#
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Use CLAUDE_PLUGIN_ROOT if available, otherwise detect from script location
|
||||
if [[ -z "${CLAUDE_PLUGIN_ROOT:-}" ]]; then
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
else
|
||||
ROOT="$CLAUDE_PLUGIN_ROOT"
|
||||
fi
|
||||
|
||||
MARKER="$ROOT/.install-version"
|
||||
PKG_JSON="$ROOT/package.json"
|
||||
|
||||
# Colors (when terminal supports it)
|
||||
if [[ -t 2 ]]; then
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[0;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
else
|
||||
RED='' GREEN='' YELLOW='' BLUE='' NC=''
|
||||
fi
|
||||
|
||||
log_info() { echo -e "${BLUE}ℹ${NC} $*" >&2; }
|
||||
log_ok() { echo -e "${GREEN}✓${NC} $*" >&2; }
|
||||
log_warn() { echo -e "${YELLOW}⚠${NC} $*" >&2; }
|
||||
log_error() { echo -e "${RED}✗${NC} $*" >&2; }
|
||||
|
||||
#
|
||||
# Detect Bun - check PATH and common locations
|
||||
#
|
||||
find_bun() {
|
||||
# Try PATH first
|
||||
if command -v bun &>/dev/null; then
|
||||
echo "bun"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Check common install locations
|
||||
local paths=(
|
||||
"$HOME/.bun/bin/bun"
|
||||
"/usr/local/bin/bun"
|
||||
"/opt/homebrew/bin/bun"
|
||||
)
|
||||
|
||||
for p in "${paths[@]}"; do
|
||||
if [[ -x "$p" ]]; then
|
||||
echo "$p"
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
#
|
||||
# Detect uv - check PATH and common locations
|
||||
#
|
||||
find_uv() {
|
||||
# Try PATH first
|
||||
if command -v uv &>/dev/null; then
|
||||
echo "uv"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Check common install locations
|
||||
local paths=(
|
||||
"$HOME/.local/bin/uv"
|
||||
"$HOME/.cargo/bin/uv"
|
||||
"/usr/local/bin/uv"
|
||||
"/opt/homebrew/bin/uv"
|
||||
)
|
||||
|
||||
for p in "${paths[@]}"; do
|
||||
if [[ -x "$p" ]]; then
|
||||
echo "$p"
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
#
|
||||
# Get package.json version
|
||||
#
|
||||
get_pkg_version() {
|
||||
if [[ -f "$PKG_JSON" ]]; then
|
||||
# Simple grep-based extraction (no jq dependency)
|
||||
grep -o '"version"[[:space:]]*:[[:space:]]*"[^"]*"' "$PKG_JSON" | head -1 | sed 's/.*"\([^"]*\)"$/\1/'
|
||||
fi
|
||||
}
|
||||
|
||||
#
|
||||
# Get marker version (if exists)
|
||||
#
|
||||
get_marker_version() {
|
||||
if [[ -f "$MARKER" ]]; then
|
||||
grep -o '"version"[[:space:]]*:[[:space:]]*"[^"]*"' "$MARKER" | head -1 | sed 's/.*"\([^"]*\)"$/\1/'
|
||||
fi
|
||||
}
|
||||
|
||||
#
|
||||
# Get marker's recorded bun version
|
||||
#
|
||||
get_marker_bun() {
|
||||
if [[ -f "$MARKER" ]]; then
|
||||
grep -o '"bun"[[:space:]]*:[[:space:]]*"[^"]*"' "$MARKER" | head -1 | sed 's/.*"\([^"]*\)"$/\1/'
|
||||
fi
|
||||
}
|
||||
|
||||
#
|
||||
# Check if install is needed
|
||||
#
|
||||
needs_install() {
|
||||
# No node_modules? Definitely need install
|
||||
if [[ ! -d "$ROOT/node_modules" ]]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
# No marker? Need install
|
||||
if [[ ! -f "$MARKER" ]]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
local pkg_ver marker_ver bun_ver marker_bun
|
||||
pkg_ver=$(get_pkg_version)
|
||||
marker_ver=$(get_marker_version)
|
||||
|
||||
# Version mismatch? Need install
|
||||
if [[ "$pkg_ver" != "$marker_ver" ]]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Bun version changed? Need install
|
||||
if BUN_PATH=$(find_bun); then
|
||||
bun_ver=$("$BUN_PATH" --version 2>/dev/null || echo "")
|
||||
marker_bun=$(get_marker_bun)
|
||||
if [[ -n "$bun_ver" && "$bun_ver" != "$marker_bun" ]]; then
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# All good, no install needed
|
||||
return 1
|
||||
}
|
||||
|
||||
#
|
||||
# Write version marker after successful install
|
||||
#
|
||||
write_marker() {
|
||||
local bun_ver uv_ver pkg_ver
|
||||
pkg_ver=$(get_pkg_version)
|
||||
bun_ver=$("$BUN_PATH" --version 2>/dev/null || echo "unknown")
|
||||
|
||||
if UV_PATH=$(find_uv); then
|
||||
uv_ver=$("$UV_PATH" --version 2>/dev/null | head -1 || echo "unknown")
|
||||
else
|
||||
uv_ver="not-installed"
|
||||
fi
|
||||
|
||||
cat > "$MARKER" <<EOF
|
||||
{
|
||||
"version": "$pkg_ver",
|
||||
"bun": "$bun_ver",
|
||||
"uv": "$uv_ver",
|
||||
"installedAt": "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
}
|
||||
EOF
|
||||
}
|
||||
|
||||
#
|
||||
# Main
|
||||
#
|
||||
|
||||
# 1. Check for Bun
|
||||
BUN_PATH=$(find_bun) || true
|
||||
if [[ -z "$BUN_PATH" ]]; then
|
||||
log_error "Bun runtime not found!"
|
||||
echo "" >&2
|
||||
echo "claude-mem requires Bun to run. Please install it:" >&2
|
||||
echo "" >&2
|
||||
echo " curl -fsSL https://bun.sh/install | bash" >&2
|
||||
echo "" >&2
|
||||
echo "Or on macOS with Homebrew:" >&2
|
||||
echo "" >&2
|
||||
echo " brew install oven-sh/bun/bun" >&2
|
||||
echo "" >&2
|
||||
echo "Then restart your terminal and try again." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
BUN_VERSION=$("$BUN_PATH" --version 2>/dev/null || echo "unknown")
|
||||
log_ok "Bun $BUN_VERSION found at $BUN_PATH"
|
||||
|
||||
# 2. Check for uv (optional - for Python/Chroma support)
|
||||
UV_PATH=$(find_uv) || true
|
||||
if [[ -z "$UV_PATH" ]]; then
|
||||
log_warn "uv not found (optional - needed for Python/Chroma vector search)"
|
||||
echo " To install: curl -LsSf https://astral.sh/uv/install.sh | sh" >&2
|
||||
else
|
||||
UV_VERSION=$("$UV_PATH" --version 2>/dev/null | head -1 || echo "unknown")
|
||||
log_ok "uv $UV_VERSION found"
|
||||
fi
|
||||
|
||||
# 3. Install dependencies if needed
|
||||
if needs_install; then
|
||||
log_info "Installing dependencies with Bun..."
|
||||
|
||||
if ! "$BUN_PATH" install --cwd "$ROOT"; then
|
||||
log_error "Failed to install dependencies"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
write_marker
|
||||
log_ok "Dependencies installed ($(get_pkg_version))"
|
||||
else
|
||||
log_ok "Dependencies up to date ($(get_marker_version))"
|
||||
fi
|
||||
|
||||
exit 0
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,145 @@
|
||||
---
|
||||
name: smart-explore
|
||||
description: Token-optimized structural code search using tree-sitter AST parsing. Use instead of reading full files when you need to understand code structure, find functions, or explore a codebase efficiently.
|
||||
---
|
||||
|
||||
# Smart Explore
|
||||
|
||||
Structural code exploration using AST parsing. **This skill overrides your default exploration behavior.** While this skill is active, use smart_search/smart_outline/smart_unfold as your primary tools instead of Read, Grep, and Glob.
|
||||
|
||||
**Core principle:** Index first, fetch on demand. Give yourself a map of the code before loading implementation details. The question before every file read should be: "do I need to see all of this, or can I get a structural overview first?" The answer is almost always: get the map.
|
||||
|
||||
## Your Next Tool Call
|
||||
|
||||
This skill only loads instructions. You must call the MCP tools yourself. Your next action should be one of:
|
||||
|
||||
```
|
||||
smart_search(query="<topic>", path="./src") -- discover files + symbols across a directory
|
||||
smart_outline(file_path="<file>") -- structural skeleton of one file
|
||||
smart_unfold(file_path="<file>", symbol_name="<name>") -- full source of one symbol
|
||||
```
|
||||
|
||||
Do NOT run Grep, Glob, Read, or find to discover files first. `smart_search` walks directories, parses all code files, and returns ranked symbols in one call. It replaces the Glob → Grep → Read discovery cycle.
|
||||
|
||||
## 3-Layer Workflow
|
||||
|
||||
### Step 1: Search -- Discover Files and Symbols
|
||||
|
||||
```
|
||||
smart_search(query="shutdown", path="./src", max_results=15)
|
||||
```
|
||||
|
||||
**Returns:** Ranked symbols with signatures, line numbers, match reasons, plus folded file views (~2-6k tokens)
|
||||
|
||||
```
|
||||
-- Matching Symbols --
|
||||
function performGracefulShutdown (services/infrastructure/GracefulShutdown.ts:56)
|
||||
function httpShutdown (services/infrastructure/HealthMonitor.ts:92)
|
||||
method WorkerService.shutdown (services/worker-service.ts:846)
|
||||
|
||||
-- Folded File Views --
|
||||
services/infrastructure/GracefulShutdown.ts (7 symbols)
|
||||
services/worker-service.ts (12 symbols)
|
||||
```
|
||||
|
||||
This is your discovery tool. It finds relevant files AND shows their structure. No Glob/find pre-scan needed.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
- `query` (string, required) -- What to search for (function name, concept, class name)
|
||||
- `path` (string) -- Root directory to search (defaults to cwd)
|
||||
- `max_results` (number) -- Max matching symbols, default 20, max 50
|
||||
- `file_pattern` (string, optional) -- Filter to specific files/paths
|
||||
|
||||
### Step 2: Outline -- Get File Structure
|
||||
|
||||
```
|
||||
smart_outline(file_path="services/worker-service.ts")
|
||||
```
|
||||
|
||||
**Returns:** Complete structural skeleton -- all functions, classes, methods, properties, imports (~1-2k tokens per file)
|
||||
|
||||
**Skip this step** when Step 1's folded file views already provide enough structure. Most useful for files not covered by the search results.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
- `file_path` (string, required) -- Path to the file
|
||||
|
||||
### Step 3: Unfold -- See Implementation
|
||||
|
||||
Review symbols from Steps 1-2. Pick the ones you need. Unfold only those:
|
||||
|
||||
```
|
||||
smart_unfold(file_path="services/worker-service.ts", symbol_name="shutdown")
|
||||
```
|
||||
|
||||
**Returns:** Full source code of the specified symbol including JSDoc, decorators, and complete implementation (~400-2,100 tokens depending on symbol size). AST node boundaries guarantee completeness regardless of symbol size — unlike Read + agent summarization, which may truncate long methods.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
- `file_path` (string, required) -- Path to the file (as returned by search/outline)
|
||||
- `symbol_name` (string, required) -- Name of the function/class/method to expand
|
||||
|
||||
## When to Use Standard Tools Instead
|
||||
|
||||
Use these only when smart_* tools are the wrong fit:
|
||||
|
||||
- **Grep:** Exact string/regex search ("find all TODO comments", "where is `ensureWorkerStarted` defined?")
|
||||
- **Read:** Small files under ~100 lines, non-code files (JSON, markdown, config)
|
||||
- **Glob:** File path patterns ("find all test files")
|
||||
- **Explore agent:** When you need synthesized understanding across 6+ files, architecture narratives, or answers to open-ended questions like "how does this entire system work end-to-end?" Smart-explore is a scalpel — it answers "where is this?" and "show me that." It doesn't synthesize cross-file data flows, design decisions, or edge cases across an entire feature.
|
||||
|
||||
For code files over ~100 lines, prefer smart_outline + smart_unfold over Read.
|
||||
|
||||
## Workflow Examples
|
||||
|
||||
**Discover how a feature works (cross-cutting):**
|
||||
|
||||
```
|
||||
1. smart_search(query="shutdown", path="./src")
|
||||
-> 14 symbols across 7 files, full picture in one call
|
||||
2. smart_unfold(file_path="services/infrastructure/GracefulShutdown.ts", symbol_name="performGracefulShutdown")
|
||||
-> See the core implementation
|
||||
```
|
||||
|
||||
**Navigate a large file:**
|
||||
|
||||
```
|
||||
1. smart_outline(file_path="services/worker-service.ts")
|
||||
-> 1,466 tokens: 12 functions, WorkerService class with 24 members
|
||||
2. smart_unfold(file_path="services/worker-service.ts", symbol_name="startSessionProcessor")
|
||||
-> 1,610 tokens: the specific method you need
|
||||
Total: ~3,076 tokens vs ~12,000 to Read the full file
|
||||
```
|
||||
|
||||
**Write documentation about code (hybrid workflow):**
|
||||
|
||||
```
|
||||
1. smart_search(query="feature name", path="./src") -- discover all relevant files and symbols
|
||||
2. smart_outline on key files -- understand structure
|
||||
3. smart_unfold on important functions -- get implementation details
|
||||
4. Read on small config/markdown/plan files -- get non-code context
|
||||
```
|
||||
|
||||
Use smart_* tools for code exploration, Read for non-code files. Mix freely.
|
||||
|
||||
**Exploration then precision:**
|
||||
|
||||
```
|
||||
1. smart_search(query="session", path="./src", max_results=10)
|
||||
-> 10 ranked symbols: SessionMetadata, SessionQueueProcessor, SessionSummary...
|
||||
2. Pick the relevant one, unfold it
|
||||
```
|
||||
|
||||
## Token Economics
|
||||
|
||||
| Approach | Tokens | Use Case |
|
||||
|----------|--------|----------|
|
||||
| smart_outline | ~1,000-2,000 | "What's in this file?" |
|
||||
| smart_unfold | ~400-2,100 | "Show me this function" |
|
||||
| smart_search | ~2,000-6,000 | "Find all X across the codebase" |
|
||||
| search + unfold | ~3,000-8,000 | End-to-end: find and read (the primary workflow) |
|
||||
| Read (full file) | ~12,000+ | When you truly need everything |
|
||||
| Explore agent | ~39,000-59,000 | Cross-file synthesis with narrative |
|
||||
|
||||
**4-8x savings** on file understanding (outline + unfold vs Read). **11-18x savings** on codebase exploration vs Explore agent. The narrower the query, the wider the gap — a 27-line function costs 55x less to read via unfold than via an Explore agent, because the agent still reads the entire file.
|
||||
@@ -0,0 +1 @@
|
||||
Never read built source files in this directory. These are compiled outputs — read the source files in `src/` instead.
|
||||
+24
-3
@@ -59,8 +59,16 @@ async function buildHooks() {
|
||||
description: 'Runtime dependencies for claude-mem bundled hooks',
|
||||
type: 'module',
|
||||
dependencies: {
|
||||
// Chroma embedding function with native ONNX binaries (can't be bundled)
|
||||
'@chroma-core/default-embed': '^0.1.9'
|
||||
'tree-sitter-cli': '^0.26.5',
|
||||
'tree-sitter-c': '^0.24.1',
|
||||
'tree-sitter-cpp': '^0.23.4',
|
||||
'tree-sitter-go': '^0.25.0',
|
||||
'tree-sitter-java': '^0.23.5',
|
||||
'tree-sitter-javascript': '^0.25.0',
|
||||
'tree-sitter-python': '^0.25.0',
|
||||
'tree-sitter-ruby': '^0.23.1',
|
||||
'tree-sitter-rust': '^0.24.0',
|
||||
'tree-sitter-typescript': '^0.23.2',
|
||||
},
|
||||
engines: {
|
||||
node: '>=18.0.0',
|
||||
@@ -128,7 +136,19 @@ async function buildHooks() {
|
||||
outfile: `${hooksDir}/${MCP_SERVER.name}.cjs`,
|
||||
minify: true,
|
||||
logLevel: 'error',
|
||||
external: ['bun:sqlite'],
|
||||
external: [
|
||||
'bun:sqlite',
|
||||
'tree-sitter-cli',
|
||||
'tree-sitter-javascript',
|
||||
'tree-sitter-typescript',
|
||||
'tree-sitter-python',
|
||||
'tree-sitter-go',
|
||||
'tree-sitter-rust',
|
||||
'tree-sitter-ruby',
|
||||
'tree-sitter-java',
|
||||
'tree-sitter-c',
|
||||
'tree-sitter-cpp',
|
||||
],
|
||||
define: {
|
||||
'__DEFAULT_PACKAGE_VERSION__': `"${version}"`
|
||||
},
|
||||
@@ -166,6 +186,7 @@ async function buildHooks() {
|
||||
console.log('\n📋 Verifying distribution files...');
|
||||
const requiredDistributionFiles = [
|
||||
'plugin/skills/mem-search/SKILL.md',
|
||||
'plugin/skills/smart-explore/SKILL.md',
|
||||
'plugin/hooks/hooks.json',
|
||||
'plugin/.claude-plugin/plugin.json',
|
||||
];
|
||||
|
||||
@@ -76,7 +76,7 @@ try {
|
||||
const gitignoreExcludes = getGitignoreExcludes(rootDir);
|
||||
|
||||
execSync(
|
||||
`rsync -av --delete --exclude=.git --exclude=/.mcp.json --exclude=bun.lock --exclude=package-lock.json ${gitignoreExcludes} ./ ~/.claude/plugins/marketplaces/thedotmack/`,
|
||||
`rsync -av --delete --exclude=.git --exclude=bun.lock --exclude=package-lock.json ${gitignoreExcludes} ./ ~/.claude/plugins/marketplaces/thedotmack/`,
|
||||
{ stdio: 'inherit' }
|
||||
);
|
||||
|
||||
|
||||
+101
-10
@@ -28,6 +28,10 @@ import {
|
||||
ListToolsRequestSchema,
|
||||
} from '@modelcontextprotocol/sdk/types.js';
|
||||
import { getWorkerPort, getWorkerHost } from '../shared/worker-utils.js';
|
||||
import { searchCodebase, formatSearchResults } from '../services/smart-file-read/search.js';
|
||||
import { parseFile, formatFoldedView, unfoldSymbol } from '../services/smart-file-read/parser.js';
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import { resolve } from 'node:path';
|
||||
|
||||
/**
|
||||
* Worker HTTP API configuration
|
||||
@@ -235,28 +239,115 @@ NEVER fetch full details without filtering first. 10x token savings.`,
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'save_observation',
|
||||
description: 'Save an observation to the database. Params: text (required), title, project',
|
||||
name: 'smart_search',
|
||||
description: 'Search codebase for symbols, functions, classes using tree-sitter AST parsing. Returns folded structural views with token counts. Use path parameter to scope the search.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
text: {
|
||||
query: {
|
||||
type: 'string',
|
||||
description: 'Content to remember (required)'
|
||||
description: 'Search term — matches against symbol names, file names, and file content'
|
||||
},
|
||||
title: {
|
||||
path: {
|
||||
type: 'string',
|
||||
description: 'Short title (auto-generated from text if omitted)'
|
||||
description: 'Root directory to search (default: current working directory)'
|
||||
},
|
||||
project: {
|
||||
max_results: {
|
||||
type: 'number',
|
||||
description: 'Maximum results to return (default: 20)'
|
||||
},
|
||||
file_pattern: {
|
||||
type: 'string',
|
||||
description: 'Project name (uses "claude-mem" if omitted)'
|
||||
description: 'Substring filter for file paths (e.g. ".ts", "src/services")'
|
||||
}
|
||||
},
|
||||
required: ['text']
|
||||
required: ['query']
|
||||
},
|
||||
handler: async (args: any) => {
|
||||
return await callWorkerAPIPost('/api/memory/save', args);
|
||||
const rootDir = resolve(args.path || process.cwd());
|
||||
const result = await searchCodebase(rootDir, args.query, {
|
||||
maxResults: args.max_results || 20,
|
||||
filePattern: args.file_pattern
|
||||
});
|
||||
const formatted = formatSearchResults(result, args.query);
|
||||
return {
|
||||
content: [{ type: 'text' as const, text: formatted }]
|
||||
};
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'smart_unfold',
|
||||
description: 'Expand a specific symbol (function, class, method) from a file. Returns the full source code of just that symbol. Use after smart_search or smart_outline to read specific code.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
file_path: {
|
||||
type: 'string',
|
||||
description: 'Path to the source file'
|
||||
},
|
||||
symbol_name: {
|
||||
type: 'string',
|
||||
description: 'Name of the symbol to unfold (function, class, method, etc.)'
|
||||
}
|
||||
},
|
||||
required: ['file_path', 'symbol_name']
|
||||
},
|
||||
handler: async (args: any) => {
|
||||
const filePath = resolve(args.file_path);
|
||||
const content = await readFile(filePath, 'utf-8');
|
||||
const unfolded = unfoldSymbol(content, filePath, args.symbol_name);
|
||||
if (unfolded) {
|
||||
return {
|
||||
content: [{ type: 'text' as const, text: unfolded }]
|
||||
};
|
||||
}
|
||||
// Symbol not found — show available symbols
|
||||
const parsed = parseFile(content, filePath);
|
||||
if (parsed.symbols.length > 0) {
|
||||
const available = parsed.symbols.map(s => ` - ${s.name} (${s.kind})`).join('\n');
|
||||
return {
|
||||
content: [{
|
||||
type: 'text' as const,
|
||||
text: `Symbol "${args.symbol_name}" not found in ${args.file_path}.\n\nAvailable symbols:\n${available}`
|
||||
}]
|
||||
};
|
||||
}
|
||||
return {
|
||||
content: [{
|
||||
type: 'text' as const,
|
||||
text: `Could not parse ${args.file_path}. File may be unsupported or empty.`
|
||||
}]
|
||||
};
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'smart_outline',
|
||||
description: 'Get structural outline of a file — shows all symbols (functions, classes, methods, types) with signatures but bodies folded. Much cheaper than reading the full file.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
file_path: {
|
||||
type: 'string',
|
||||
description: 'Path to the source file'
|
||||
}
|
||||
},
|
||||
required: ['file_path']
|
||||
},
|
||||
handler: async (args: any) => {
|
||||
const filePath = resolve(args.file_path);
|
||||
const content = await readFile(filePath, 'utf-8');
|
||||
const parsed = parseFile(content, filePath);
|
||||
if (parsed.symbols.length > 0) {
|
||||
return {
|
||||
content: [{ type: 'text' as const, text: formatFoldedView(parsed) }]
|
||||
};
|
||||
}
|
||||
return {
|
||||
content: [{
|
||||
type: 'text' as const,
|
||||
text: `Could not parse ${args.file_path}. File may use an unsupported language or be empty.`
|
||||
}]
|
||||
};
|
||||
}
|
||||
}
|
||||
];
|
||||
|
||||
@@ -0,0 +1,666 @@
|
||||
/**
|
||||
* Code structure parser — shells out to tree-sitter CLI for AST-based extraction.
|
||||
*
|
||||
* No native bindings. No WASM. Just the CLI binary + query patterns.
|
||||
*
|
||||
* Supported: JS, TS, Python, Go, Rust, Ruby, Java, C, C++
|
||||
*
|
||||
* by Copter Labs
|
||||
*/
|
||||
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { writeFileSync, mkdtempSync, rmSync, existsSync } from "node:fs";
|
||||
import { join, dirname } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { createRequire } from "node:module";
|
||||
|
||||
// CJS-safe require for resolving external packages at runtime.
|
||||
// In ESM: import.meta.url works. In CJS bundle (esbuild): __filename works.
|
||||
// typeof check avoids ReferenceError in ESM where __filename doesn't exist.
|
||||
const _require = typeof __filename !== 'undefined'
|
||||
? createRequire(__filename)
|
||||
: createRequire(import.meta.url);
|
||||
|
||||
// --- Types ---
|
||||
|
||||
export interface CodeSymbol {
|
||||
name: string;
|
||||
kind: "function" | "class" | "method" | "interface" | "type" | "const" | "variable" | "export" | "struct" | "enum" | "trait" | "impl" | "property" | "getter" | "setter";
|
||||
signature: string;
|
||||
jsdoc?: string;
|
||||
lineStart: number;
|
||||
lineEnd: number;
|
||||
parent?: string;
|
||||
exported: boolean;
|
||||
children?: CodeSymbol[];
|
||||
}
|
||||
|
||||
export interface FoldedFile {
|
||||
filePath: string;
|
||||
language: string;
|
||||
symbols: CodeSymbol[];
|
||||
imports: string[];
|
||||
totalLines: number;
|
||||
foldedTokenEstimate: number;
|
||||
}
|
||||
|
||||
// --- Language detection ---
|
||||
|
||||
const LANG_MAP: Record<string, string> = {
|
||||
".js": "javascript",
|
||||
".mjs": "javascript",
|
||||
".cjs": "javascript",
|
||||
".jsx": "tsx",
|
||||
".ts": "typescript",
|
||||
".tsx": "tsx",
|
||||
".py": "python",
|
||||
".pyw": "python",
|
||||
".go": "go",
|
||||
".rs": "rust",
|
||||
".rb": "ruby",
|
||||
".java": "java",
|
||||
".c": "c",
|
||||
".h": "c",
|
||||
".cpp": "cpp",
|
||||
".cc": "cpp",
|
||||
".cxx": "cpp",
|
||||
".hpp": "cpp",
|
||||
".hh": "cpp",
|
||||
};
|
||||
|
||||
export function detectLanguage(filePath: string): string {
|
||||
const ext = filePath.slice(filePath.lastIndexOf("."));
|
||||
return LANG_MAP[ext] || "unknown";
|
||||
}
|
||||
|
||||
// --- Grammar path resolution ---
|
||||
|
||||
const GRAMMAR_PACKAGES: Record<string, string> = {
|
||||
javascript: "tree-sitter-javascript",
|
||||
typescript: "tree-sitter-typescript/typescript",
|
||||
tsx: "tree-sitter-typescript/tsx",
|
||||
python: "tree-sitter-python",
|
||||
go: "tree-sitter-go",
|
||||
rust: "tree-sitter-rust",
|
||||
ruby: "tree-sitter-ruby",
|
||||
java: "tree-sitter-java",
|
||||
c: "tree-sitter-c",
|
||||
cpp: "tree-sitter-cpp",
|
||||
};
|
||||
|
||||
function resolveGrammarPath(language: string): string | null {
|
||||
const pkg = GRAMMAR_PACKAGES[language];
|
||||
if (!pkg) return null;
|
||||
try {
|
||||
const packageJsonPath = _require.resolve(pkg + "/package.json");
|
||||
return dirname(packageJsonPath);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// --- Query patterns (declarative symbol extraction) ---
|
||||
|
||||
const QUERIES: Record<string, string> = {
|
||||
jsts: `
|
||||
(function_declaration name: (identifier) @name) @func
|
||||
(lexical_declaration (variable_declarator name: (identifier) @name value: [(arrow_function) (function_expression)])) @const_func
|
||||
(class_declaration name: (type_identifier) @name) @cls
|
||||
(method_definition name: (property_identifier) @name) @method
|
||||
(interface_declaration name: (type_identifier) @name) @iface
|
||||
(type_alias_declaration name: (type_identifier) @name) @tdef
|
||||
(enum_declaration name: (identifier) @name) @enm
|
||||
(import_statement) @imp
|
||||
(export_statement) @exp
|
||||
`,
|
||||
|
||||
python: `
|
||||
(function_definition name: (identifier) @name) @func
|
||||
(class_definition name: (identifier) @name) @cls
|
||||
(import_statement) @imp
|
||||
(import_from_statement) @imp
|
||||
`,
|
||||
|
||||
go: `
|
||||
(function_declaration name: (identifier) @name) @func
|
||||
(method_declaration name: (field_identifier) @name) @method
|
||||
(type_declaration (type_spec name: (type_identifier) @name)) @tdef
|
||||
(import_declaration) @imp
|
||||
`,
|
||||
|
||||
rust: `
|
||||
(function_item name: (identifier) @name) @func
|
||||
(struct_item name: (type_identifier) @name) @struct_def
|
||||
(enum_item name: (type_identifier) @name) @enm
|
||||
(trait_item name: (type_identifier) @name) @trait_def
|
||||
(impl_item type: (type_identifier) @name) @impl_def
|
||||
(use_declaration) @imp
|
||||
`,
|
||||
|
||||
ruby: `
|
||||
(method name: (identifier) @name) @func
|
||||
(class name: (constant) @name) @cls
|
||||
(module name: (constant) @name) @cls
|
||||
(call method: (identifier) @name) @imp
|
||||
`,
|
||||
|
||||
java: `
|
||||
(method_declaration name: (identifier) @name) @method
|
||||
(class_declaration name: (identifier) @name) @cls
|
||||
(interface_declaration name: (identifier) @name) @iface
|
||||
(enum_declaration name: (identifier) @name) @enm
|
||||
(import_declaration) @imp
|
||||
`,
|
||||
|
||||
generic: `
|
||||
(function_declaration name: (identifier) @name) @func
|
||||
(function_definition name: (identifier) @name) @func
|
||||
(class_declaration name: (identifier) @name) @cls
|
||||
(class_definition name: (identifier) @name) @cls
|
||||
(import_statement) @imp
|
||||
(import_declaration) @imp
|
||||
`,
|
||||
};
|
||||
|
||||
function getQueryKey(language: string): string {
|
||||
switch (language) {
|
||||
case "javascript":
|
||||
case "typescript":
|
||||
case "tsx":
|
||||
return "jsts";
|
||||
case "python": return "python";
|
||||
case "go": return "go";
|
||||
case "rust": return "rust";
|
||||
case "ruby": return "ruby";
|
||||
case "java": return "java";
|
||||
default: return "generic";
|
||||
}
|
||||
}
|
||||
|
||||
// --- Temp file management ---
|
||||
|
||||
let queryTmpDir: string | null = null;
|
||||
const queryFileCache = new Map<string, string>();
|
||||
|
||||
function getQueryFile(queryKey: string): string {
|
||||
if (queryFileCache.has(queryKey)) return queryFileCache.get(queryKey)!;
|
||||
|
||||
if (!queryTmpDir) {
|
||||
queryTmpDir = mkdtempSync(join(tmpdir(), "smart-read-queries-"));
|
||||
}
|
||||
|
||||
const filePath = join(queryTmpDir, `${queryKey}.scm`);
|
||||
writeFileSync(filePath, QUERIES[queryKey]);
|
||||
queryFileCache.set(queryKey, filePath);
|
||||
return filePath;
|
||||
}
|
||||
|
||||
// --- CLI execution ---
|
||||
|
||||
let cachedBinPath: string | null = null;
|
||||
|
||||
function getTreeSitterBin(): string {
|
||||
if (cachedBinPath) return cachedBinPath;
|
||||
|
||||
// Try direct binary from tree-sitter-cli package
|
||||
try {
|
||||
const pkgPath = _require.resolve("tree-sitter-cli/package.json");
|
||||
const binPath = join(dirname(pkgPath), "tree-sitter");
|
||||
if (existsSync(binPath)) {
|
||||
cachedBinPath = binPath;
|
||||
return binPath;
|
||||
}
|
||||
} catch { /* fall through */ }
|
||||
|
||||
// Fallback: assume it's on PATH
|
||||
cachedBinPath = "tree-sitter";
|
||||
return cachedBinPath;
|
||||
}
|
||||
|
||||
interface RawCapture {
|
||||
tag: string;
|
||||
startRow: number;
|
||||
startCol: number;
|
||||
endRow: number;
|
||||
endCol: number;
|
||||
text?: string;
|
||||
}
|
||||
|
||||
interface RawMatch {
|
||||
pattern: number;
|
||||
captures: RawCapture[];
|
||||
}
|
||||
|
||||
function runQuery(queryFile: string, sourceFile: string, grammarPath: string): RawMatch[] {
|
||||
const result = runBatchQuery(queryFile, [sourceFile], grammarPath);
|
||||
return result.get(sourceFile) || [];
|
||||
}
|
||||
|
||||
function runBatchQuery(queryFile: string, sourceFiles: string[], grammarPath: string): Map<string, RawMatch[]> {
|
||||
if (sourceFiles.length === 0) return new Map();
|
||||
|
||||
const bin = getTreeSitterBin();
|
||||
const execArgs = ["query", "-p", grammarPath, queryFile, ...sourceFiles];
|
||||
|
||||
let output: string;
|
||||
try {
|
||||
output = execFileSync(bin, execArgs, { encoding: "utf-8", timeout: 30000, stdio: ["pipe", "pipe", "pipe"] });
|
||||
} catch {
|
||||
return new Map();
|
||||
}
|
||||
|
||||
return parseMultiFileQueryOutput(output);
|
||||
}
|
||||
|
||||
function parseMultiFileQueryOutput(output: string): Map<string, RawMatch[]> {
|
||||
const fileMatches = new Map<string, RawMatch[]>();
|
||||
let currentFile: string | null = null;
|
||||
let currentMatch: RawMatch | null = null;
|
||||
|
||||
for (const line of output.split("\n")) {
|
||||
// File header: a line that doesn't start with whitespace and isn't empty
|
||||
if (line.length > 0 && !line.startsWith(" ") && !line.startsWith("\t")) {
|
||||
currentFile = line.trim();
|
||||
if (!fileMatches.has(currentFile)) {
|
||||
fileMatches.set(currentFile, []);
|
||||
}
|
||||
currentMatch = null;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!currentFile) continue;
|
||||
|
||||
const patternMatch = line.match(/^\s+pattern:\s+(\d+)/);
|
||||
if (patternMatch) {
|
||||
currentMatch = { pattern: parseInt(patternMatch[1]), captures: [] };
|
||||
fileMatches.get(currentFile)!.push(currentMatch);
|
||||
continue;
|
||||
}
|
||||
|
||||
const captureMatch = line.match(
|
||||
/^\s+capture:\s+(?:\d+\s*-\s*)?(\w+),\s*start:\s*\((\d+),\s*(\d+)\),\s*end:\s*\((\d+),\s*(\d+)\)(?:,\s*text:\s*`([^`]*)`)?/
|
||||
);
|
||||
if (captureMatch && currentMatch) {
|
||||
currentMatch.captures.push({
|
||||
tag: captureMatch[1],
|
||||
startRow: parseInt(captureMatch[2]),
|
||||
startCol: parseInt(captureMatch[3]),
|
||||
endRow: parseInt(captureMatch[4]),
|
||||
endCol: parseInt(captureMatch[5]),
|
||||
text: captureMatch[6],
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return fileMatches;
|
||||
}
|
||||
|
||||
// --- Symbol building ---
|
||||
|
||||
const KIND_MAP: Record<string, CodeSymbol["kind"]> = {
|
||||
func: "function",
|
||||
const_func: "function",
|
||||
cls: "class",
|
||||
method: "method",
|
||||
iface: "interface",
|
||||
tdef: "type",
|
||||
enm: "enum",
|
||||
struct_def: "struct",
|
||||
trait_def: "trait",
|
||||
impl_def: "impl",
|
||||
};
|
||||
|
||||
const CONTAINER_KINDS = new Set(["class", "struct", "impl", "trait"]);
|
||||
|
||||
function extractSignatureFromLines(lines: string[], startRow: number, endRow: number, maxLen: number = 200): string {
|
||||
const firstLine = lines[startRow] || "";
|
||||
let sig = firstLine;
|
||||
|
||||
if (!sig.trimEnd().endsWith("{") && !sig.trimEnd().endsWith(":")) {
|
||||
const chunk = lines.slice(startRow, Math.min(startRow + 10, endRow + 1)).join("\n");
|
||||
const braceIdx = chunk.indexOf("{");
|
||||
if (braceIdx !== -1 && braceIdx < 500) {
|
||||
sig = chunk.slice(0, braceIdx).replace(/\n/g, " ").replace(/\s+/g, " ").trim();
|
||||
}
|
||||
}
|
||||
|
||||
sig = sig.replace(/\s*[{:]\s*$/, "").trim();
|
||||
if (sig.length > maxLen) sig = sig.slice(0, maxLen - 3) + "...";
|
||||
return sig;
|
||||
}
|
||||
|
||||
function findCommentAbove(lines: string[], startRow: number): string | undefined {
|
||||
const commentLines: string[] = [];
|
||||
let foundComment = false;
|
||||
|
||||
for (let i = startRow - 1; i >= 0; i--) {
|
||||
const trimmed = lines[i].trim();
|
||||
if (trimmed === "") {
|
||||
if (foundComment) break;
|
||||
continue;
|
||||
}
|
||||
if (trimmed.startsWith("/**") || trimmed.startsWith("*") || trimmed.startsWith("*/") ||
|
||||
trimmed.startsWith("//") || trimmed.startsWith("///") || trimmed.startsWith("//!") ||
|
||||
trimmed.startsWith("#") || trimmed.startsWith("@")) {
|
||||
commentLines.unshift(lines[i]);
|
||||
foundComment = true;
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return commentLines.length > 0 ? commentLines.join("\n").trim() : undefined;
|
||||
}
|
||||
|
||||
function findPythonDocstringFromLines(lines: string[], startRow: number, endRow: number): string | undefined {
|
||||
for (let i = startRow + 1; i <= Math.min(startRow + 3, endRow); i++) {
|
||||
const trimmed = lines[i]?.trim();
|
||||
if (!trimmed) continue;
|
||||
if (trimmed.startsWith('"""') || trimmed.startsWith("'''")) return trimmed;
|
||||
break;
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function isExported(
|
||||
name: string, startRow: number, endRow: number,
|
||||
exportRanges: Array<{ startRow: number; endRow: number }>,
|
||||
lines: string[], language: string
|
||||
): boolean {
|
||||
switch (language) {
|
||||
case "javascript":
|
||||
case "typescript":
|
||||
case "tsx":
|
||||
return exportRanges.some(r => startRow >= r.startRow && endRow <= r.endRow);
|
||||
case "python":
|
||||
return !name.startsWith("_");
|
||||
case "go":
|
||||
return name.length > 0 && name[0] === name[0].toUpperCase() && name[0] !== name[0].toLowerCase();
|
||||
case "rust":
|
||||
return lines[startRow]?.trimStart().startsWith("pub") ?? false;
|
||||
default:
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
function buildSymbols(matches: RawMatch[], lines: string[], language: string): { symbols: CodeSymbol[]; imports: string[] } {
|
||||
const symbols: CodeSymbol[] = [];
|
||||
const imports: string[] = [];
|
||||
const exportRanges: Array<{ startRow: number; endRow: number }> = [];
|
||||
const containers: Array<{ sym: CodeSymbol; startRow: number; endRow: number }> = [];
|
||||
|
||||
// Collect exports and imports
|
||||
for (const match of matches) {
|
||||
for (const cap of match.captures) {
|
||||
if (cap.tag === "exp") {
|
||||
exportRanges.push({ startRow: cap.startRow, endRow: cap.endRow });
|
||||
}
|
||||
if (cap.tag === "imp") {
|
||||
imports.push(cap.text || lines[cap.startRow]?.trim() || "");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Build symbols
|
||||
for (const match of matches) {
|
||||
const kindCapture = match.captures.find(c => KIND_MAP[c.tag]);
|
||||
const nameCapture = match.captures.find(c => c.tag === "name");
|
||||
if (!kindCapture) continue;
|
||||
|
||||
const name = nameCapture?.text || "anonymous";
|
||||
const startRow = kindCapture.startRow;
|
||||
const endRow = kindCapture.endRow;
|
||||
const kind = KIND_MAP[kindCapture.tag];
|
||||
|
||||
const comment = findCommentAbove(lines, startRow);
|
||||
const docstring = language === "python" ? findPythonDocstringFromLines(lines, startRow, endRow) : undefined;
|
||||
|
||||
const sym: CodeSymbol = {
|
||||
name,
|
||||
kind,
|
||||
signature: extractSignatureFromLines(lines, startRow, endRow),
|
||||
jsdoc: comment || docstring,
|
||||
lineStart: startRow,
|
||||
lineEnd: endRow,
|
||||
exported: isExported(name, startRow, endRow, exportRanges, lines, language),
|
||||
};
|
||||
|
||||
if (CONTAINER_KINDS.has(kind)) {
|
||||
sym.children = [];
|
||||
containers.push({ sym, startRow, endRow });
|
||||
}
|
||||
|
||||
symbols.push(sym);
|
||||
}
|
||||
|
||||
// Nest methods inside containers
|
||||
const nested = new Set<CodeSymbol>();
|
||||
for (const container of containers) {
|
||||
for (const sym of symbols) {
|
||||
if (sym === container.sym) continue;
|
||||
if (sym.lineStart > container.startRow && sym.lineEnd <= container.endRow) {
|
||||
if (sym.kind === "function") sym.kind = "method";
|
||||
container.sym.children!.push(sym);
|
||||
nested.add(sym);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return { symbols: symbols.filter(s => !nested.has(s)), imports };
|
||||
}
|
||||
|
||||
// --- Main parse functions ---
|
||||
|
||||
export function parseFile(content: string, filePath: string): FoldedFile {
|
||||
const language = detectLanguage(filePath);
|
||||
const lines = content.split("\n");
|
||||
|
||||
const grammarPath = resolveGrammarPath(language);
|
||||
if (!grammarPath) {
|
||||
return {
|
||||
filePath, language, symbols: [], imports: [],
|
||||
totalLines: lines.length, foldedTokenEstimate: 50,
|
||||
};
|
||||
}
|
||||
|
||||
const queryKey = getQueryKey(language);
|
||||
const queryFile = getQueryFile(queryKey);
|
||||
|
||||
// Write content to temp file with correct extension for language detection
|
||||
const ext = filePath.slice(filePath.lastIndexOf(".")) || ".txt";
|
||||
const tmpDir = mkdtempSync(join(tmpdir(), "smart-src-"));
|
||||
const tmpFile = join(tmpDir, `source${ext}`);
|
||||
writeFileSync(tmpFile, content);
|
||||
|
||||
try {
|
||||
const matches = runQuery(queryFile, tmpFile, grammarPath);
|
||||
const result = buildSymbols(matches, lines, language);
|
||||
|
||||
const folded = formatFoldedView({
|
||||
filePath, language,
|
||||
symbols: result.symbols, imports: result.imports,
|
||||
totalLines: lines.length, foldedTokenEstimate: 0,
|
||||
});
|
||||
|
||||
return {
|
||||
filePath, language,
|
||||
symbols: result.symbols, imports: result.imports,
|
||||
totalLines: lines.length,
|
||||
foldedTokenEstimate: Math.ceil(folded.length / 4),
|
||||
};
|
||||
} finally {
|
||||
rmSync(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Batch parse multiple on-disk files. Groups by language for one CLI call per language.
|
||||
* Much faster than calling parseFile() per file (one process spawn per language vs per file).
|
||||
*/
|
||||
export function parseFilesBatch(
|
||||
files: Array<{ absolutePath: string; relativePath: string; content: string }>
|
||||
): Map<string, FoldedFile> {
|
||||
const results = new Map<string, FoldedFile>();
|
||||
|
||||
// Group files by language (and thus by query + grammar)
|
||||
const languageGroups = new Map<string, typeof files>();
|
||||
for (const file of files) {
|
||||
const language = detectLanguage(file.relativePath);
|
||||
if (!languageGroups.has(language)) languageGroups.set(language, []);
|
||||
languageGroups.get(language)!.push(file);
|
||||
}
|
||||
|
||||
for (const [language, groupFiles] of languageGroups) {
|
||||
const grammarPath = resolveGrammarPath(language);
|
||||
if (!grammarPath) {
|
||||
// No grammar — return empty results for these files
|
||||
for (const file of groupFiles) {
|
||||
const lines = file.content.split("\n");
|
||||
results.set(file.relativePath, {
|
||||
filePath: file.relativePath, language, symbols: [], imports: [],
|
||||
totalLines: lines.length, foldedTokenEstimate: 50,
|
||||
});
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
const queryKey = getQueryKey(language);
|
||||
const queryFile = getQueryFile(queryKey);
|
||||
|
||||
// Run one batch query for all files of this language
|
||||
const absolutePaths = groupFiles.map(f => f.absolutePath);
|
||||
const batchResults = runBatchQuery(queryFile, absolutePaths, grammarPath);
|
||||
|
||||
// Build FoldedFile for each file using the batch results
|
||||
for (const file of groupFiles) {
|
||||
const lines = file.content.split("\n");
|
||||
const matches = batchResults.get(file.absolutePath) || [];
|
||||
const symbolResult = buildSymbols(matches, lines, language);
|
||||
|
||||
const folded = formatFoldedView({
|
||||
filePath: file.relativePath, language,
|
||||
symbols: symbolResult.symbols, imports: symbolResult.imports,
|
||||
totalLines: lines.length, foldedTokenEstimate: 0,
|
||||
});
|
||||
|
||||
results.set(file.relativePath, {
|
||||
filePath: file.relativePath, language,
|
||||
symbols: symbolResult.symbols, imports: symbolResult.imports,
|
||||
totalLines: lines.length,
|
||||
foldedTokenEstimate: Math.ceil(folded.length / 4),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
// --- Formatting ---
|
||||
|
||||
export function formatFoldedView(file: FoldedFile): string {
|
||||
const parts: string[] = [];
|
||||
|
||||
parts.push(`📁 ${file.filePath} (${file.language}, ${file.totalLines} lines)`);
|
||||
parts.push("");
|
||||
|
||||
if (file.imports.length > 0) {
|
||||
parts.push(` 📦 Imports: ${file.imports.length} statements`);
|
||||
for (const imp of file.imports.slice(0, 10)) {
|
||||
parts.push(` ${imp}`);
|
||||
}
|
||||
if (file.imports.length > 10) {
|
||||
parts.push(` ... +${file.imports.length - 10} more`);
|
||||
}
|
||||
parts.push("");
|
||||
}
|
||||
|
||||
for (const sym of file.symbols) {
|
||||
parts.push(formatSymbol(sym, " "));
|
||||
}
|
||||
|
||||
return parts.join("\n");
|
||||
}
|
||||
|
||||
function formatSymbol(sym: CodeSymbol, indent: string): string {
|
||||
const parts: string[] = [];
|
||||
|
||||
const icon = getSymbolIcon(sym.kind);
|
||||
const exportTag = sym.exported ? " [exported]" : "";
|
||||
const lineRange = sym.lineStart === sym.lineEnd
|
||||
? `L${sym.lineStart + 1}`
|
||||
: `L${sym.lineStart + 1}-${sym.lineEnd + 1}`;
|
||||
|
||||
parts.push(`${indent}${icon} ${sym.name}${exportTag} (${lineRange})`);
|
||||
parts.push(`${indent} ${sym.signature}`);
|
||||
|
||||
if (sym.jsdoc) {
|
||||
const jsdocLines = sym.jsdoc.split("\n");
|
||||
const firstLine = jsdocLines.find(l => {
|
||||
const t = l.replace(/^[\s*/]+/, "").replace(/^['"`]{3}/, "").trim();
|
||||
return t.length > 0 && !t.startsWith("/**");
|
||||
});
|
||||
if (firstLine) {
|
||||
const cleaned = firstLine.replace(/^[\s*/]+/, "").replace(/^['"`]{3}/, "").replace(/['"`]{3}$/, "").trim();
|
||||
if (cleaned) {
|
||||
parts.push(`${indent} 💬 ${cleaned}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (sym.children && sym.children.length > 0) {
|
||||
for (const child of sym.children) {
|
||||
parts.push(formatSymbol(child, indent + " "));
|
||||
}
|
||||
}
|
||||
|
||||
return parts.join("\n");
|
||||
}
|
||||
|
||||
function getSymbolIcon(kind: CodeSymbol["kind"]): string {
|
||||
const icons: Record<string, string> = {
|
||||
function: "ƒ", method: "ƒ", class: "◆", interface: "◇",
|
||||
type: "◇", const: "●", variable: "○", export: "→",
|
||||
struct: "◆", enum: "▣", trait: "◇", impl: "◈",
|
||||
property: "○", getter: "⇢", setter: "⇠",
|
||||
};
|
||||
return icons[kind] || "·";
|
||||
}
|
||||
|
||||
// --- Unfold ---
|
||||
|
||||
export function unfoldSymbol(content: string, filePath: string, symbolName: string): string | null {
|
||||
const file = parseFile(content, filePath);
|
||||
|
||||
const findSymbol = (symbols: CodeSymbol[]): CodeSymbol | null => {
|
||||
for (const sym of symbols) {
|
||||
if (sym.name === symbolName) return sym;
|
||||
if (sym.children) {
|
||||
const found = findSymbol(sym.children);
|
||||
if (found) return found;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
const symbol = findSymbol(file.symbols);
|
||||
if (!symbol) return null;
|
||||
|
||||
const lines = content.split("\n");
|
||||
|
||||
// Include preceding comments/decorators
|
||||
let start = symbol.lineStart;
|
||||
for (let i = symbol.lineStart - 1; i >= 0; i--) {
|
||||
const trimmed = lines[i].trim();
|
||||
if (trimmed === "" || trimmed.startsWith("*") || trimmed.startsWith("/**") ||
|
||||
trimmed.startsWith("///") || trimmed.startsWith("//") ||
|
||||
trimmed.startsWith("#") || trimmed.startsWith("@") ||
|
||||
trimmed === "*/") {
|
||||
start = i;
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
const extracted = lines.slice(start, symbol.lineEnd + 1).join("\n");
|
||||
return `// 📍 ${filePath} L${start + 1}-${symbol.lineEnd + 1}\n${extracted}`;
|
||||
}
|
||||
@@ -0,0 +1,316 @@
|
||||
/**
|
||||
* Search module — finds code files and symbols matching a query.
|
||||
*
|
||||
* Two search modes:
|
||||
* 1. Grep-style: find files/lines containing the query string
|
||||
* 2. Structural: parse files and match against symbol names/signatures
|
||||
*
|
||||
* Both return folded views, not raw content.
|
||||
*
|
||||
* Uses batch parsing (one CLI call per language) for fast multi-file search.
|
||||
*/
|
||||
|
||||
import { readFile, readdir, stat } from "node:fs/promises";
|
||||
import { join, relative } from "node:path";
|
||||
import { parseFilesBatch, formatFoldedView, type FoldedFile } from "./parser.js";
|
||||
|
||||
const CODE_EXTENSIONS = new Set([
|
||||
".js", ".jsx", ".ts", ".tsx", ".mjs", ".cjs",
|
||||
".py", ".pyw",
|
||||
".go",
|
||||
".rs",
|
||||
".rb",
|
||||
".java",
|
||||
".cs",
|
||||
".cpp", ".c", ".h", ".hpp",
|
||||
".swift",
|
||||
".kt",
|
||||
".php",
|
||||
".vue", ".svelte",
|
||||
]);
|
||||
|
||||
const IGNORE_DIRS = new Set([
|
||||
"node_modules", ".git", "dist", "build", ".next", "__pycache__",
|
||||
".venv", "venv", "env", ".env", "target", "vendor",
|
||||
".cache", ".turbo", "coverage", ".nyc_output",
|
||||
".claude", ".smart-file-read",
|
||||
]);
|
||||
|
||||
const MAX_FILE_SIZE = 512 * 1024; // 512KB — skip huge files
|
||||
|
||||
export interface SearchResult {
|
||||
foldedFiles: FoldedFile[];
|
||||
matchingSymbols: SymbolMatch[];
|
||||
totalFilesScanned: number;
|
||||
totalSymbolsFound: number;
|
||||
tokenEstimate: number;
|
||||
}
|
||||
|
||||
export interface SymbolMatch {
|
||||
filePath: string;
|
||||
symbolName: string;
|
||||
kind: string;
|
||||
signature: string;
|
||||
jsdoc?: string;
|
||||
lineStart: number;
|
||||
lineEnd: number;
|
||||
matchReason: string; // why this matched
|
||||
}
|
||||
|
||||
/**
|
||||
* Walk a directory recursively, yielding file paths.
|
||||
*/
|
||||
async function* walkDir(dir: string, rootDir: string, maxDepth: number = 20): AsyncGenerator<string> {
|
||||
if (maxDepth <= 0) return;
|
||||
|
||||
let entries;
|
||||
try {
|
||||
entries = await readdir(dir, { withFileTypes: true });
|
||||
} catch {
|
||||
return; // permission denied, etc.
|
||||
}
|
||||
|
||||
for (const entry of entries) {
|
||||
if (entry.name.startsWith(".") && entry.name !== ".") continue;
|
||||
if (IGNORE_DIRS.has(entry.name)) continue;
|
||||
|
||||
const fullPath = join(dir, entry.name);
|
||||
|
||||
if (entry.isDirectory()) {
|
||||
yield* walkDir(fullPath, rootDir, maxDepth - 1);
|
||||
} else if (entry.isFile()) {
|
||||
const ext = entry.name.slice(entry.name.lastIndexOf("."));
|
||||
if (CODE_EXTENSIONS.has(ext)) {
|
||||
yield fullPath;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a file safely, skipping if too large or binary.
|
||||
*/
|
||||
async function safeReadFile(filePath: string): Promise<string | null> {
|
||||
try {
|
||||
const stats = await stat(filePath);
|
||||
if (stats.size > MAX_FILE_SIZE) return null;
|
||||
if (stats.size === 0) return null;
|
||||
|
||||
const content = await readFile(filePath, "utf-8");
|
||||
|
||||
// Quick binary check — if first 1000 chars have null bytes, skip
|
||||
if (content.slice(0, 1000).includes("\0")) return null;
|
||||
|
||||
return content;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Search a codebase for symbols matching a query.
|
||||
*
|
||||
* Phase 1: Collect files and read content
|
||||
* Phase 2: Batch parse all files (one CLI call per language)
|
||||
* Phase 3: Match query against parsed symbols
|
||||
*/
|
||||
export async function searchCodebase(
|
||||
rootDir: string,
|
||||
query: string,
|
||||
options: {
|
||||
maxResults?: number;
|
||||
includeImports?: boolean;
|
||||
filePattern?: string;
|
||||
} = {}
|
||||
): Promise<SearchResult> {
|
||||
const maxResults = options.maxResults || 20;
|
||||
const queryLower = query.toLowerCase();
|
||||
const queryParts = queryLower.split(/[\s_\-./]+/).filter(p => p.length > 0);
|
||||
|
||||
// Phase 1: Collect files
|
||||
const filesToParse: Array<{ absolutePath: string; relativePath: string; content: string }> = [];
|
||||
|
||||
for await (const filePath of walkDir(rootDir, rootDir)) {
|
||||
if (options.filePattern) {
|
||||
const relPath = relative(rootDir, filePath);
|
||||
if (!relPath.toLowerCase().includes(options.filePattern.toLowerCase())) continue;
|
||||
}
|
||||
|
||||
const content = await safeReadFile(filePath);
|
||||
if (!content) continue;
|
||||
|
||||
filesToParse.push({
|
||||
absolutePath: filePath,
|
||||
relativePath: relative(rootDir, filePath),
|
||||
content,
|
||||
});
|
||||
}
|
||||
|
||||
// Phase 2: Batch parse (one CLI call per language)
|
||||
const parsedFiles = parseFilesBatch(filesToParse);
|
||||
|
||||
// Phase 3: Match query against symbols
|
||||
const foldedFiles: FoldedFile[] = [];
|
||||
const matchingSymbols: SymbolMatch[] = [];
|
||||
let totalSymbolsFound = 0;
|
||||
|
||||
for (const [relPath, parsed] of parsedFiles) {
|
||||
totalSymbolsFound += countSymbols(parsed);
|
||||
|
||||
const pathMatch = matchScore(relPath.toLowerCase(), queryParts);
|
||||
let fileHasMatch = pathMatch > 0;
|
||||
const fileSymbolMatches: SymbolMatch[] = [];
|
||||
|
||||
const checkSymbols = (symbols: typeof parsed.symbols, parent?: string) => {
|
||||
for (const sym of symbols) {
|
||||
let score = 0;
|
||||
let reason = "";
|
||||
|
||||
const nameScore = matchScore(sym.name.toLowerCase(), queryParts);
|
||||
if (nameScore > 0) {
|
||||
score += nameScore * 3;
|
||||
reason = "name match";
|
||||
}
|
||||
|
||||
if (sym.signature.toLowerCase().includes(queryLower)) {
|
||||
score += 2;
|
||||
reason = reason ? `${reason} + signature` : "signature match";
|
||||
}
|
||||
|
||||
if (sym.jsdoc && sym.jsdoc.toLowerCase().includes(queryLower)) {
|
||||
score += 1;
|
||||
reason = reason ? `${reason} + jsdoc` : "jsdoc match";
|
||||
}
|
||||
|
||||
if (score > 0) {
|
||||
fileHasMatch = true;
|
||||
fileSymbolMatches.push({
|
||||
filePath: relPath,
|
||||
symbolName: parent ? `${parent}.${sym.name}` : sym.name,
|
||||
kind: sym.kind,
|
||||
signature: sym.signature,
|
||||
jsdoc: sym.jsdoc,
|
||||
lineStart: sym.lineStart,
|
||||
lineEnd: sym.lineEnd,
|
||||
matchReason: reason,
|
||||
});
|
||||
}
|
||||
|
||||
if (sym.children) {
|
||||
checkSymbols(sym.children, sym.name);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
checkSymbols(parsed.symbols);
|
||||
|
||||
if (fileHasMatch) {
|
||||
foldedFiles.push(parsed);
|
||||
matchingSymbols.push(...fileSymbolMatches);
|
||||
}
|
||||
}
|
||||
|
||||
// Sort by relevance and trim
|
||||
matchingSymbols.sort((a, b) => {
|
||||
const aScore = matchScore(a.symbolName.toLowerCase(), queryParts);
|
||||
const bScore = matchScore(b.symbolName.toLowerCase(), queryParts);
|
||||
return bScore - aScore;
|
||||
});
|
||||
|
||||
const trimmedSymbols = matchingSymbols.slice(0, maxResults);
|
||||
const relevantFiles = new Set(trimmedSymbols.map(s => s.filePath));
|
||||
const trimmedFiles = foldedFiles.filter(f => relevantFiles.has(f.filePath)).slice(0, maxResults);
|
||||
|
||||
const tokenEstimate = trimmedFiles.reduce((sum, f) => sum + f.foldedTokenEstimate, 0);
|
||||
|
||||
return {
|
||||
foldedFiles: trimmedFiles,
|
||||
matchingSymbols: trimmedSymbols,
|
||||
totalFilesScanned: filesToParse.length,
|
||||
totalSymbolsFound,
|
||||
tokenEstimate,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Score how well query parts match a string.
|
||||
* Returns 0 for no match, higher for better matches.
|
||||
*/
|
||||
function matchScore(text: string, queryParts: string[]): number {
|
||||
let score = 0;
|
||||
for (const part of queryParts) {
|
||||
if (text === part) {
|
||||
score += 10; // exact match
|
||||
} else if (text.includes(part)) {
|
||||
score += 5; // substring match
|
||||
} else {
|
||||
// Fuzzy: check if all chars appear in order
|
||||
let ti = 0;
|
||||
let matched = 0;
|
||||
for (const ch of part) {
|
||||
const idx = text.indexOf(ch, ti);
|
||||
if (idx !== -1) {
|
||||
matched++;
|
||||
ti = idx + 1;
|
||||
}
|
||||
}
|
||||
if (matched === part.length) {
|
||||
score += 1; // loose fuzzy match
|
||||
}
|
||||
}
|
||||
}
|
||||
return score;
|
||||
}
|
||||
|
||||
function countSymbols(file: FoldedFile): number {
|
||||
let count = file.symbols.length;
|
||||
for (const sym of file.symbols) {
|
||||
if (sym.children) count += sym.children.length;
|
||||
}
|
||||
return count;
|
||||
}
|
||||
|
||||
/**
|
||||
* Format search results for LLM consumption.
|
||||
*/
|
||||
export function formatSearchResults(result: SearchResult, query: string): string {
|
||||
const parts: string[] = [];
|
||||
|
||||
parts.push(`🔍 Smart Search: "${query}"`);
|
||||
parts.push(` Scanned ${result.totalFilesScanned} files, found ${result.totalSymbolsFound} symbols`);
|
||||
parts.push(` ${result.matchingSymbols.length} matches across ${result.foldedFiles.length} files (~${result.tokenEstimate} tokens for folded view)`);
|
||||
parts.push("");
|
||||
|
||||
if (result.matchingSymbols.length === 0) {
|
||||
parts.push(" No matching symbols found.");
|
||||
return parts.join("\n");
|
||||
}
|
||||
|
||||
// Show matching symbols first (compact)
|
||||
parts.push("── Matching Symbols ──");
|
||||
parts.push("");
|
||||
for (const match of result.matchingSymbols) {
|
||||
parts.push(` ${match.kind} ${match.symbolName} (${match.filePath}:${match.lineStart + 1})`);
|
||||
parts.push(` ${match.signature}`);
|
||||
if (match.jsdoc) {
|
||||
const firstLine = match.jsdoc.split("\n").find(l => l.replace(/^[\s*/]+/, "").trim().length > 0);
|
||||
if (firstLine) {
|
||||
parts.push(` 💬 ${firstLine.replace(/^[\s*/]+/, "").trim()}`);
|
||||
}
|
||||
}
|
||||
parts.push("");
|
||||
}
|
||||
|
||||
// Show folded file views
|
||||
parts.push("── Folded File Views ──");
|
||||
parts.push("");
|
||||
for (const file of result.foldedFiles) {
|
||||
parts.push(formatFoldedView(file));
|
||||
parts.push("");
|
||||
}
|
||||
|
||||
parts.push("── Actions ──");
|
||||
parts.push(' To see full implementation: use smart_unfold with file path and symbol name');
|
||||
|
||||
return parts.join("\n");
|
||||
}
|
||||
@@ -1027,6 +1027,7 @@ async function main() {
|
||||
} else {
|
||||
exitWithStatus('error', 'Failed to start worker');
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
case 'stop': {
|
||||
@@ -1038,6 +1039,7 @@ async function main() {
|
||||
removePidFile();
|
||||
logger.info('SYSTEM', 'Worker stopped successfully');
|
||||
process.exit(0);
|
||||
break;
|
||||
}
|
||||
|
||||
case 'restart': {
|
||||
@@ -1074,6 +1076,7 @@ async function main() {
|
||||
|
||||
logger.info('SYSTEM', 'Worker restarted successfully');
|
||||
process.exit(0);
|
||||
break;
|
||||
}
|
||||
|
||||
case 'status': {
|
||||
@@ -1088,12 +1091,14 @@ async function main() {
|
||||
console.log('Worker is not running');
|
||||
}
|
||||
process.exit(0);
|
||||
break;
|
||||
}
|
||||
|
||||
case 'cursor': {
|
||||
const subcommand = process.argv[3];
|
||||
const cursorResult = await handleCursorCommand(subcommand, process.argv.slice(4));
|
||||
process.exit(cursorResult);
|
||||
break;
|
||||
}
|
||||
|
||||
case 'hook': {
|
||||
@@ -1147,6 +1152,7 @@ async function main() {
|
||||
const { generateClaudeMd } = await import('../cli/claude-md-commands.js');
|
||||
const result = await generateClaudeMd(dryRun);
|
||||
process.exit(result);
|
||||
break;
|
||||
}
|
||||
|
||||
case 'clean': {
|
||||
@@ -1154,6 +1160,7 @@ async function main() {
|
||||
const { cleanClaudeMd } = await import('../cli/claude-md-commands.js');
|
||||
const result = await cleanClaudeMd(dryRun);
|
||||
process.exit(result);
|
||||
break;
|
||||
}
|
||||
|
||||
case '--daemon':
|
||||
@@ -1210,5 +1217,8 @@ const isMainModule = typeof require !== 'undefined' && typeof module !== 'undefi
|
||||
: import.meta.url === `file://${process.argv[1]}` || process.argv[1]?.endsWith('worker-service');
|
||||
|
||||
if (isMainModule) {
|
||||
main();
|
||||
main().catch((error) => {
|
||||
logger.error('SYSTEM', 'Fatal error in main', {}, error instanceof Error ? error : undefined);
|
||||
process.exit(0); // Exit 0: don't block Claude Code, don't leave Windows Terminal tabs open
|
||||
});
|
||||
}
|
||||
|
||||
@@ -81,6 +81,22 @@ describe('Plugin Distribution - hooks.json Integrity', () => {
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it('should include CLAUDE_PLUGIN_ROOT fallback in all hook commands (#1215)', () => {
|
||||
const hooksPath = path.join(projectRoot, 'plugin/hooks/hooks.json');
|
||||
const parsed = JSON.parse(readFileSync(hooksPath, 'utf-8'));
|
||||
const expectedFallbackPath = '$HOME/.claude/plugins/marketplaces/thedotmack/plugin';
|
||||
|
||||
for (const [eventName, matchers] of Object.entries(parsed.hooks)) {
|
||||
for (const matcher of matchers as any[]) {
|
||||
for (const hook of matcher.hooks) {
|
||||
if (hook.type === 'command') {
|
||||
expect(hook.command).toContain(expectedFallbackPath);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('Plugin Distribution - package.json Files Field', () => {
|
||||
|
||||
Reference in New Issue
Block a user