Replace search skill with mem-search (#91)
* feat: add mem-search skill with progressive disclosure architecture Add comprehensive mem-search skill for accessing claude-mem's persistent cross-session memory database. Implements progressive disclosure workflow and token-efficient search patterns. Features: - 12 search operations (observations, sessions, prompts, by-type, by-concept, by-file, timelines, etc.) - Progressive disclosure principles to minimize token usage - Anti-patterns documentation to guide LLM behavior - HTTP API integration for all search functionality - Common workflows with composition examples Structure: - SKILL.md: Entry point with temporal trigger patterns - principles/: Progressive disclosure + anti-patterns - operations/: 12 search operation files 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * docs: add CHANGELOG entry for mem-search skill Document mem-search skill addition in Unreleased section with: - 100% effectiveness compliance metrics - Comparison to previous search skill implementation - Progressive disclosure architecture details - Reference to audit report documentation 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * docs: add mem-search skill audit report Add comprehensive audit report validating mem-search skill against Anthropic's official skill-creator documentation. Report includes: - Effectiveness metrics comparison (search vs mem-search) - Critical issues analysis for production readiness - Compliance validation across 6 key dimensions - Reference implementation guidance Result: mem-search achieves 100% compliance vs search's 67% 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * feat: Add comprehensive search architecture analysis document - Document current state of dual search architectures (HTTP API and MCP) - Analyze HTTP endpoints and MCP search server architectures - Identify DRY violations across search implementations - Evaluate the use of curl as the optimal approach for search - Provide architectural recommendations for immediate and long-term improvements - Outline action plan for cleanup, feature parity, DRY refactoring * refactor: Remove deprecated search skill documentation and operations * refactor: Reorganize documentation into public and context directories Changes: - Created docs/public/ for Mintlify documentation (.mdx files) - Created docs/context/ for internal planning and implementation docs - Moved all .mdx files and assets to docs/public/ - Moved all internal .md files to docs/context/ - Added CLAUDE.md to both directories explaining their purpose - Updated docs.json paths to work with new structure Benefits: - Clear separation between user-facing and internal documentation - Easier to maintain Mintlify docs in dedicated directory - Internal context files organized separately 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Enhance session management and continuity in hooks - Updated new-hook.ts to clarify session_id threading and idempotent session creation. - Modified prompts.ts to require claudeSessionId for continuation prompts, ensuring session context is maintained. - Improved SessionStore.ts documentation on createSDKSession to emphasize idempotent behavior and session connection. - Refined SDKAgent.ts to detail continuation prompt logic and its reliance on session.claudeSessionId for unified session handling. --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Alex Newman <thedotmack@gmail.com>
This commit is contained in:
basher83
@@ -0,0 +1,88 @@
|
||||
# Claude-Mem Public Documentation
|
||||
|
||||
## What This Folder Is
|
||||
|
||||
This `docs/public/` folder contains the **Mintlify documentation site** - the official user-facing documentation for claude-mem. It's a structured documentation platform with a specific file format and organization.
|
||||
|
||||
## Folder Structure
|
||||
|
||||
```
|
||||
docs/
|
||||
├── public/ ← You are here (Mintlify MDX files)
|
||||
│ ├── *.mdx - User-facing documentation pages
|
||||
│ ├── docs.json - Mintlify configuration and navigation
|
||||
│ ├── architecture/ - Technical architecture docs
|
||||
│ ├── usage/ - User guides and workflows
|
||||
│ └── *.webp, *.gif - Assets (logos, screenshots)
|
||||
└── context/ ← Internal documentation (DO NOT put here)
|
||||
└── *.md - Planning docs, audits, references
|
||||
```
|
||||
|
||||
## File Requirements
|
||||
|
||||
### Mintlify Documentation Files (.mdx)
|
||||
All official documentation files must be:
|
||||
- Written in `.mdx` format (Markdown with JSX support)
|
||||
- Listed in `docs.json` navigation structure
|
||||
- Follow Mintlify's schema and conventions
|
||||
|
||||
The documentation is organized into these sections:
|
||||
- **Get Started**: Introduction, installation, usage guides
|
||||
- **Best Practices**: Context engineering, progressive disclosure
|
||||
- **Configuration & Development**: Settings, dev workflow, troubleshooting
|
||||
- **Architecture**: System design, components, technical details
|
||||
|
||||
### Configuration File
|
||||
`docs.json` defines:
|
||||
- Site metadata (name, description, theme)
|
||||
- Navigation structure
|
||||
- Branding (logos, colors)
|
||||
- Footer links and social media
|
||||
|
||||
## What Does NOT Belong Here
|
||||
|
||||
**Planning documents, design docs, and reference materials go in `/docs/context/` instead:**
|
||||
|
||||
Files that belong in `/docs/context/` (NOT here):
|
||||
- Planning documents (`*-plan.md`, `*-outline.md`)
|
||||
- Implementation analysis (`*-audit.md`, `*-code-reference.md`)
|
||||
- Error tracking (`typescript-errors.md`)
|
||||
- Internal design documents
|
||||
- PR review responses
|
||||
- Reference materials (like `agent-sdk-ref.md`)
|
||||
- Work-in-progress documentation
|
||||
|
||||
## How to Add Official Documentation
|
||||
|
||||
1. Create a new `.mdx` file in the appropriate subdirectory
|
||||
2. Add the file path to `docs.json` navigation
|
||||
3. Use Mintlify's frontmatter and components
|
||||
4. Follow the existing documentation style
|
||||
5. Test locally: `npx mintlify dev`
|
||||
|
||||
## Development Workflow
|
||||
|
||||
**For contributors working on claude-mem:**
|
||||
- Read `/CLAUDE.md` in the project root for development instructions
|
||||
- Place planning/design docs in `/docs/context/`
|
||||
- Only add user-facing documentation to `/docs/public/`
|
||||
- Test documentation locally with Mintlify CLI before committing
|
||||
|
||||
## Testing Documentation
|
||||
|
||||
```bash
|
||||
# Validate docs structure
|
||||
npx mintlify validate
|
||||
|
||||
# Check for broken links
|
||||
npx mintlify broken-links
|
||||
|
||||
# Run local dev server
|
||||
npx mintlify dev
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
**Simple Rule**:
|
||||
- `/docs/public/` = Official user documentation (Mintlify .mdx files) ← YOU ARE HERE
|
||||
- `/docs/context/` = Internal docs, plans, references, audits
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,309 @@
|
||||
---
|
||||
title: "Database Architecture"
|
||||
description: "SQLite schema, FTS5 search, and data storage"
|
||||
---
|
||||
|
||||
# Database Architecture
|
||||
|
||||
Claude-Mem uses SQLite 3 with the better-sqlite3 native module for persistent storage and FTS5 for full-text search.
|
||||
|
||||
## Database Location
|
||||
|
||||
- **Current**: `~/.claude-mem/claude-mem.db`
|
||||
|
||||
**Note**: Despite the README claiming v4.0.0+ moved the database to `${CLAUDE_PLUGIN_ROOT}/data/`, the actual implementation still uses `~/.claude-mem/`.
|
||||
|
||||
## Database Implementation
|
||||
|
||||
**Primary Implementation**: better-sqlite3 (native SQLite module)
|
||||
- Used by: SessionStore and SessionSearch
|
||||
- Format: Synchronous API with better performance
|
||||
- **Note**: Database.ts (using bun:sqlite) is legacy code
|
||||
|
||||
## Core Tables
|
||||
|
||||
### 1. sdk_sessions
|
||||
|
||||
Tracks active and completed sessions.
|
||||
|
||||
```sql
|
||||
CREATE TABLE sdk_sessions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
sdk_session_id TEXT UNIQUE NOT NULL,
|
||||
claude_session_id TEXT,
|
||||
project TEXT NOT NULL,
|
||||
prompt_counter INTEGER DEFAULT 0,
|
||||
status TEXT NOT NULL DEFAULT 'active',
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
completed_at TEXT,
|
||||
completed_at_epoch INTEGER,
|
||||
last_activity_at TEXT,
|
||||
last_activity_epoch INTEGER
|
||||
);
|
||||
```
|
||||
|
||||
**Indexes**:
|
||||
- `idx_sdk_sessions_claude_session` on `claude_session_id`
|
||||
- `idx_sdk_sessions_project` on `project`
|
||||
- `idx_sdk_sessions_status` on `status`
|
||||
- `idx_sdk_sessions_created_at` on `created_at_epoch DESC`
|
||||
|
||||
### 2. observations
|
||||
|
||||
Individual tool executions with hierarchical structure.
|
||||
|
||||
```sql
|
||||
CREATE TABLE observations (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_id TEXT NOT NULL,
|
||||
sdk_session_id TEXT NOT NULL,
|
||||
claude_session_id TEXT,
|
||||
project TEXT NOT NULL,
|
||||
prompt_number INTEGER,
|
||||
tool_name TEXT NOT NULL,
|
||||
correlation_id TEXT,
|
||||
|
||||
-- Hierarchical fields
|
||||
title TEXT,
|
||||
subtitle TEXT,
|
||||
narrative TEXT,
|
||||
text TEXT,
|
||||
facts TEXT,
|
||||
concepts TEXT,
|
||||
type TEXT,
|
||||
files_read TEXT,
|
||||
files_modified TEXT,
|
||||
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
|
||||
FOREIGN KEY (sdk_session_id) REFERENCES sdk_sessions(sdk_session_id)
|
||||
);
|
||||
```
|
||||
|
||||
**Observation Types**:
|
||||
- `decision` - Architectural or design decisions
|
||||
- `bugfix` - Bug fixes and corrections
|
||||
- `feature` - New features or capabilities
|
||||
- `refactor` - Code refactoring and cleanup
|
||||
- `discovery` - Learnings about the codebase
|
||||
- `change` - General changes and modifications
|
||||
|
||||
**Indexes**:
|
||||
- `idx_observations_session` on `session_id`
|
||||
- `idx_observations_sdk_session` on `sdk_session_id`
|
||||
- `idx_observations_project` on `project`
|
||||
- `idx_observations_tool_name` on `tool_name`
|
||||
- `idx_observations_created_at` on `created_at_epoch DESC`
|
||||
- `idx_observations_type` on `type`
|
||||
|
||||
### 3. session_summaries
|
||||
|
||||
AI-generated session summaries (multiple per session).
|
||||
|
||||
```sql
|
||||
CREATE TABLE session_summaries (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
sdk_session_id TEXT NOT NULL,
|
||||
claude_session_id TEXT,
|
||||
project TEXT NOT NULL,
|
||||
prompt_number INTEGER,
|
||||
|
||||
-- Summary fields
|
||||
request TEXT,
|
||||
investigated TEXT,
|
||||
learned TEXT,
|
||||
completed TEXT,
|
||||
next_steps TEXT,
|
||||
notes TEXT,
|
||||
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
|
||||
FOREIGN KEY (sdk_session_id) REFERENCES sdk_sessions(sdk_session_id)
|
||||
);
|
||||
```
|
||||
|
||||
**Indexes**:
|
||||
- `idx_session_summaries_sdk_session` on `sdk_session_id`
|
||||
- `idx_session_summaries_project` on `project`
|
||||
- `idx_session_summaries_created_at` on `created_at_epoch DESC`
|
||||
|
||||
### 4. user_prompts
|
||||
|
||||
Raw user prompts with FTS5 search (as of v4.2.0).
|
||||
|
||||
```sql
|
||||
CREATE TABLE user_prompts (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
sdk_session_id TEXT NOT NULL,
|
||||
claude_session_id TEXT,
|
||||
project TEXT NOT NULL,
|
||||
prompt_number INTEGER,
|
||||
prompt_text TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL,
|
||||
created_at_epoch INTEGER NOT NULL,
|
||||
|
||||
FOREIGN KEY (sdk_session_id) REFERENCES sdk_sessions(sdk_session_id)
|
||||
);
|
||||
```
|
||||
|
||||
**Indexes**:
|
||||
- `idx_user_prompts_sdk_session` on `sdk_session_id`
|
||||
- `idx_user_prompts_project` on `project`
|
||||
- `idx_user_prompts_created_at` on `created_at_epoch DESC`
|
||||
|
||||
### Legacy Tables
|
||||
|
||||
- **sessions**: Legacy session tracking (v3.x)
|
||||
- **memories**: Legacy compressed memory chunks (v3.x)
|
||||
- **overviews**: Legacy session summaries (v3.x)
|
||||
|
||||
## FTS5 Full-Text Search
|
||||
|
||||
SQLite FTS5 (Full-Text Search) virtual tables enable fast full-text search across observations, summaries, and user prompts.
|
||||
|
||||
### FTS5 Virtual Tables
|
||||
|
||||
#### observations_fts
|
||||
|
||||
```sql
|
||||
CREATE VIRTUAL TABLE observations_fts USING fts5(
|
||||
title,
|
||||
subtitle,
|
||||
narrative,
|
||||
text,
|
||||
facts,
|
||||
concepts,
|
||||
content='observations',
|
||||
content_rowid='id'
|
||||
);
|
||||
```
|
||||
|
||||
#### session_summaries_fts
|
||||
|
||||
```sql
|
||||
CREATE VIRTUAL TABLE session_summaries_fts USING fts5(
|
||||
request,
|
||||
investigated,
|
||||
learned,
|
||||
completed,
|
||||
next_steps,
|
||||
notes,
|
||||
content='session_summaries',
|
||||
content_rowid='id'
|
||||
);
|
||||
```
|
||||
|
||||
#### user_prompts_fts
|
||||
|
||||
```sql
|
||||
CREATE VIRTUAL TABLE user_prompts_fts USING fts5(
|
||||
prompt_text,
|
||||
content='user_prompts',
|
||||
content_rowid='id'
|
||||
);
|
||||
```
|
||||
|
||||
### Automatic Synchronization
|
||||
|
||||
FTS5 tables stay in sync via triggers:
|
||||
|
||||
```sql
|
||||
-- Insert trigger example
|
||||
CREATE TRIGGER observations_ai AFTER INSERT ON observations BEGIN
|
||||
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
|
||||
END;
|
||||
|
||||
-- Update trigger example
|
||||
CREATE TRIGGER observations_au AFTER UPDATE ON observations BEGIN
|
||||
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
|
||||
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
|
||||
END;
|
||||
|
||||
-- Delete trigger example
|
||||
CREATE TRIGGER observations_ad AFTER DELETE ON observations BEGIN
|
||||
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
|
||||
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
|
||||
END;
|
||||
```
|
||||
|
||||
### FTS5 Query Syntax
|
||||
|
||||
FTS5 supports rich query syntax:
|
||||
|
||||
- **Simple**: `"error handling"`
|
||||
- **AND**: `"error" AND "handling"`
|
||||
- **OR**: `"bug" OR "fix"`
|
||||
- **NOT**: `"bug" NOT "feature"`
|
||||
- **Phrase**: `"'exact phrase'"`
|
||||
- **Column**: `title:"authentication"`
|
||||
|
||||
### Security
|
||||
|
||||
As of v4.2.3, all FTS5 queries are properly escaped to prevent SQL injection:
|
||||
- Double quotes are escaped: `query.replace(/"/g, '""')`
|
||||
- Comprehensive test suite with 332 injection attack tests
|
||||
|
||||
## Database Classes
|
||||
|
||||
### SessionStore
|
||||
|
||||
CRUD operations for sessions, observations, summaries, and user prompts.
|
||||
|
||||
**Location**: `src/services/sqlite/SessionStore.ts`
|
||||
|
||||
**Methods**:
|
||||
- `createSession()`
|
||||
- `getSession()`
|
||||
- `updateSession()`
|
||||
- `createObservation()`
|
||||
- `getObservations()`
|
||||
- `createSummary()`
|
||||
- `getSummaries()`
|
||||
- `createUserPrompt()`
|
||||
|
||||
### SessionSearch
|
||||
|
||||
FTS5 full-text search with 8 specialized search methods.
|
||||
|
||||
**Location**: `src/services/sqlite/SessionSearch.ts`
|
||||
|
||||
**Methods**:
|
||||
- `searchObservations()` - Full-text search across observations
|
||||
- `searchSessions()` - Full-text search across summaries
|
||||
- `searchUserPrompts()` - Full-text search across user prompts
|
||||
- `findByConcept()` - Find by concept tags
|
||||
- `findByFile()` - Find by file references
|
||||
- `findByType()` - Find by observation type
|
||||
- `getRecentContext()` - Get recent session context
|
||||
- `advancedSearch()` - Combined filters
|
||||
|
||||
## Migrations
|
||||
|
||||
Database schema is managed via migrations in `src/services/sqlite/migrations.ts`.
|
||||
|
||||
**Migration History**:
|
||||
- Migration 001: Initial schema (sessions, memories, overviews, diagnostics, transcript_events)
|
||||
- Migration 002: Hierarchical memory fields (title, subtitle, facts, concepts, files_touched)
|
||||
- Migration 003: SDK sessions and observations
|
||||
- Migration 004: Session summaries
|
||||
- Migration 005: Multi-prompt sessions (prompt_counter, prompt_number)
|
||||
- Migration 006: FTS5 virtual tables and triggers
|
||||
- Migration 007-010: Various improvements and user prompts table
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
- **Indexes**: All foreign keys and frequently queried columns are indexed
|
||||
- **FTS5**: Full-text search is significantly faster than LIKE queries
|
||||
- **Triggers**: Automatic synchronization has minimal overhead
|
||||
- **Connection Pooling**: better-sqlite3 reuses connections efficiently
|
||||
- **Synchronous API**: better-sqlite3 uses synchronous API for better performance
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
See [Troubleshooting - Database Issues](../troubleshooting.md#database-issues) for common problems and solutions.
|
||||
@@ -0,0 +1,269 @@
|
||||
---
|
||||
title: "Plugin Hooks"
|
||||
description: "7 hook scripts that power Claude-Mem"
|
||||
---
|
||||
|
||||
# Plugin Hooks
|
||||
|
||||
Claude-Mem integrates with Claude Code through 7 hook scripts across 5 lifecycle events that capture events and inject context.
|
||||
|
||||
## Hook Overview
|
||||
|
||||
| Hook Name | Purpose | Timeout | Script |
|
||||
|---------------------|--------------------------------------|---------|-------------------------|
|
||||
| SessionStart | Smart dependency installation | 300s | smart-install.js |
|
||||
| SessionStart | Inject context from previous sessions| 300s | context-hook.js |
|
||||
| SessionStart | Display first-time setup message | 10s | user-message-hook.js |
|
||||
| UserPromptSubmit | Create/track new sessions | 120s | new-hook.js |
|
||||
| PostToolUse | Capture tool execution observations | 120s | save-hook.js |
|
||||
| Stop | Generate session summaries | 120s | summary-hook.js |
|
||||
| SessionEnd | Mark sessions complete | 120s | cleanup-hook.js |
|
||||
|
||||
## Hook Configuration
|
||||
|
||||
Hooks are configured in `plugin/hooks/hooks.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"description": "Claude-mem memory system hooks",
|
||||
"hooks": {
|
||||
"SessionStart": [{
|
||||
"matcher": "startup|clear|compact",
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/../scripts/smart-install.js\" && node ${CLAUDE_PLUGIN_ROOT}/scripts/context-hook.js",
|
||||
"timeout": 300
|
||||
}, {
|
||||
"type": "command",
|
||||
"command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/user-message-hook.js",
|
||||
"timeout": 10
|
||||
}]
|
||||
}],
|
||||
"UserPromptSubmit": [{
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/new-hook.js",
|
||||
"timeout": 120
|
||||
}]
|
||||
}],
|
||||
"PostToolUse": [{
|
||||
"matcher": "*",
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/save-hook.js",
|
||||
"timeout": 120
|
||||
}]
|
||||
}],
|
||||
"Stop": [{
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/summary-hook.js",
|
||||
"timeout": 120
|
||||
}]
|
||||
}],
|
||||
"SessionEnd": [{
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/cleanup-hook.js",
|
||||
"timeout": 120
|
||||
}]
|
||||
}]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 1. SessionStart Hook - Smart Install (`smart-install.js`)
|
||||
|
||||
**Purpose**: Intelligently manage dependencies and ensure worker service is running.
|
||||
|
||||
**Behavior**:
|
||||
- Checks if dependencies need installation using version marker (`.install-version`)
|
||||
- Only runs npm install when:
|
||||
- First-time installation (no node_modules)
|
||||
- Version changed in package.json
|
||||
- Critical dependency missing (e.g., better-sqlite3)
|
||||
- Provides Windows-specific error messages for build tool issues
|
||||
- Auto-starts PM2 worker service after installation
|
||||
- Fast when already installed (~10ms vs 2-5 seconds)
|
||||
|
||||
**Input** (via stdin):
|
||||
```json
|
||||
{
|
||||
"session_id": "claude-session-123",
|
||||
"cwd": "/path/to/project",
|
||||
"source": "startup"
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation**: `scripts/smart-install.js`
|
||||
|
||||
**Key Features**:
|
||||
- Version caching prevents redundant installs
|
||||
- Cross-platform compatible (Windows, macOS, Linux)
|
||||
- Helpful error messages with troubleshooting steps
|
||||
- Non-blocking worker startup
|
||||
|
||||
**v5.0.3 Enhancement**: Smart caching eliminates 2-5 second npm install on every SessionStart, reducing to ~10ms for already-installed dependencies.
|
||||
|
||||
## 2. SessionStart Hook - Context Injection (`context-hook.js`)
|
||||
|
||||
**Purpose**: Inject context from previous sessions into Claude's initial context.
|
||||
|
||||
**Behavior**:
|
||||
- Retrieves last 10 session summaries with three-tier verbosity (v4.2.0)
|
||||
- Retrieves last 50 observations (configurable via `CLAUDE_MEM_CONTEXT_OBSERVATIONS`)
|
||||
- Returns context via `hookSpecificOutput` in JSON format (fixed in v4.1.0)
|
||||
- Formats results as progressive disclosure index
|
||||
|
||||
**Input** (via stdin):
|
||||
```json
|
||||
{
|
||||
"session_id": "claude-session-123",
|
||||
"cwd": "/path/to/project",
|
||||
"source": "startup"
|
||||
}
|
||||
```
|
||||
|
||||
**Output** (via stdout):
|
||||
```json
|
||||
{
|
||||
"hookSpecificOutput": "# Recent Sessions\n\n## Session 1...\n"
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation**: `src/hooks/context-hook.ts`
|
||||
|
||||
## 3. SessionStart Hook - User Message (`user-message-hook.js`)
|
||||
|
||||
**Purpose**: Display helpful user messages during first-time setup or when viewing context.
|
||||
|
||||
**Behavior**:
|
||||
- Shows first-time setup message when node_modules is missing
|
||||
- Displays formatted context information with colors
|
||||
- Provides tips for using claude-mem effectively
|
||||
- Shows link to web viewer UI (http://localhost:37777)
|
||||
- Exits with code 3 (informational, not error)
|
||||
|
||||
**Output Example**:
|
||||
```
|
||||
📝 Claude-Mem Context Loaded
|
||||
ℹ️ Note: This appears as stderr but is informational only
|
||||
|
||||
[Context details...]
|
||||
|
||||
📺 Watch live in browser http://localhost:37777/ (New! v5.1)
|
||||
```
|
||||
|
||||
**Implementation**: `plugin/scripts/user-message-hook.js` (minified)
|
||||
|
||||
**Key Features**:
|
||||
- User-friendly first-time setup experience
|
||||
- Visual context display with colors
|
||||
- Links to new features (viewer UI)
|
||||
- Non-intrusive messaging
|
||||
|
||||
## 4. UserPromptSubmit Hook (`new-hook.js`)
|
||||
|
||||
**Purpose**: Create new session records and initialize session tracking.
|
||||
|
||||
**Behavior**:
|
||||
- Creates new session in database
|
||||
- Initializes session tracking
|
||||
- Saves raw user prompts for full-text search (as of v4.2.0)
|
||||
- Sends init signal to worker service
|
||||
|
||||
**Input** (via stdin):
|
||||
```json
|
||||
{
|
||||
"session_id": "claude-session-123",
|
||||
"cwd": "/path/to/project",
|
||||
"prompt": "User's actual prompt text"
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation**: `src/hooks/new-hook.ts`
|
||||
|
||||
## 5. PostToolUse Hook (`save-hook.js`)
|
||||
|
||||
**Purpose**: Capture tool execution observations.
|
||||
|
||||
**Behavior**:
|
||||
- Fires after EVERY tool execution (Read, Write, Edit, Bash, etc.)
|
||||
- Sends observations to worker service for processing
|
||||
- Includes correlation IDs for tracing
|
||||
- Filters low-value observations
|
||||
|
||||
**Input** (via stdin):
|
||||
```json
|
||||
{
|
||||
"session_id": "claude-session-123",
|
||||
"cwd": "/path/to/project",
|
||||
"tool_name": "Read",
|
||||
"tool_input": {...},
|
||||
"tool_result": "...",
|
||||
"correlation_id": "abc-123"
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation**: `src/hooks/save-hook.ts`
|
||||
|
||||
## 6. Stop Hook (`summary-hook.js`)
|
||||
|
||||
**Purpose**: Generate session summaries when Claude stops.
|
||||
|
||||
**Behavior**:
|
||||
- Triggers final summary generation
|
||||
- Sends summarize request to worker service
|
||||
- Summary includes: request, completed, learned, next_steps
|
||||
|
||||
**Input** (via stdin):
|
||||
```json
|
||||
{
|
||||
"session_id": "claude-session-123",
|
||||
"cwd": "/path/to/project",
|
||||
"source": "user_stop"
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation**: `src/hooks/summary-hook.ts`
|
||||
|
||||
## 7. SessionEnd Hook (`cleanup-hook.js`)
|
||||
|
||||
**Purpose**: Mark sessions as completed (graceful cleanup as of v4.1.0).
|
||||
|
||||
**Behavior**:
|
||||
- Marks sessions as completed
|
||||
- Skips cleanup on `/clear` commands to preserve ongoing sessions
|
||||
- Allows workers to finish pending operations naturally
|
||||
- Previously sent DELETE requests; now uses graceful completion
|
||||
|
||||
**Input** (via stdin):
|
||||
```json
|
||||
{
|
||||
"session_id": "claude-session-123",
|
||||
"cwd": "/path/to/project",
|
||||
"source": "normal"
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation**: `src/hooks/cleanup-hook.ts`
|
||||
|
||||
## Hook Development
|
||||
|
||||
### Adding a New Hook
|
||||
|
||||
1. Create hook implementation in `src/hooks/your-hook.ts`
|
||||
2. Add to `plugin/hooks/hooks.json`
|
||||
3. Rebuild with `npm run build`
|
||||
|
||||
### Hook Best Practices
|
||||
|
||||
- **Fast execution**: Hooks should complete quickly (< 1s ideal)
|
||||
- **Graceful degradation**: Don't block Claude if worker is down
|
||||
- **Structured logging**: Use logger for debugging
|
||||
- **Error handling**: Catch and log errors, don't crash
|
||||
- **JSON output**: Use `hookSpecificOutput` for context injection
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
See [Troubleshooting - Hook Issues](../troubleshooting.md#hook-issues) for common problems and solutions.
|
||||
@@ -0,0 +1,230 @@
|
||||
---
|
||||
title: "Architecture Overview"
|
||||
description: "System components and data flow in Claude-Mem"
|
||||
---
|
||||
|
||||
# Architecture Overview
|
||||
|
||||
## System Components
|
||||
|
||||
Claude-Mem operates as a Claude Code plugin with five core components:
|
||||
|
||||
1. **Plugin Hooks** - Capture lifecycle events (7 hook files)
|
||||
2. **Worker Service** - Process observations via Claude Agent SDK + HTTP API (10 search endpoints)
|
||||
3. **Database Layer** - Store sessions and observations (SQLite + FTS5 + ChromaDB)
|
||||
4. **Search Skill** - Skill-based search with progressive disclosure (v5.4.0+)
|
||||
5. **Viewer UI** - Web-based real-time memory stream visualization
|
||||
|
||||
## Technology Stack
|
||||
|
||||
| Layer | Technology |
|
||||
|------------------------|-------------------------------------------|
|
||||
| **Language** | TypeScript (ES2022, ESNext modules) |
|
||||
| **Runtime** | Node.js 18+ |
|
||||
| **Database** | SQLite 3 with better-sqlite3 driver |
|
||||
| **Vector Store** | ChromaDB (optional, for semantic search) |
|
||||
| **HTTP Server** | Express.js 4.18 |
|
||||
| **Real-time** | Server-Sent Events (SSE) |
|
||||
| **UI Framework** | React + TypeScript |
|
||||
| **AI SDK** | @anthropic-ai/claude-agent-sdk |
|
||||
| **Build Tool** | esbuild (bundles TypeScript) |
|
||||
| **Process Manager** | PM2 |
|
||||
| **Testing** | Node.js built-in test runner |
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Memory Pipeline
|
||||
```
|
||||
Hook (stdin) → Database → Worker Service → SDK Processor → Database → Next Session Hook
|
||||
```
|
||||
|
||||
1. **Input**: Claude Code sends tool execution data via stdin to hooks
|
||||
2. **Storage**: Hooks write observations to SQLite database
|
||||
3. **Processing**: Worker service reads observations, processes via SDK
|
||||
4. **Output**: Processed summaries written back to database
|
||||
5. **Retrieval**: Next session's context hook reads summaries from database
|
||||
|
||||
### Search Pipeline (v5.4.0+)
|
||||
```
|
||||
User Query → Skill Invoked → HTTP API → SessionSearch Service → FTS5 Database → Search Results → Claude
|
||||
```
|
||||
|
||||
1. **User Query**: User asks naturally: "What bugs did we fix?"
|
||||
2. **Skill Invoked**: Claude recognizes intent and invokes search skill
|
||||
3. **HTTP API**: Skill uses curl to call HTTP endpoint (e.g., `/api/search/observations`)
|
||||
4. **SessionSearch**: Worker service queries FTS5 virtual tables
|
||||
5. **Format**: Results formatted and returned to skill
|
||||
6. **Return**: Claude presents formatted results to user
|
||||
|
||||
**Token Savings**: ~2,250 tokens per session vs MCP approach through progressive disclosure
|
||||
|
||||
## Session Lifecycle
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 0. Smart Install Hook Fires │
|
||||
│ Checks dependencies (cached), only runs on version changes │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 1. Session Starts → Context Hook Fires │
|
||||
│ Starts PM2 worker if needed, injects context from previous │
|
||||
│ sessions (configurable observation count) │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 2. User Types Prompt → UserPromptSubmit Hook Fires │
|
||||
│ Creates session in database, saves raw user prompt for FTS5 │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 3. Claude Uses Tools → PostToolUse Hook Fires (100+ times) │
|
||||
│ Captures tool executions, sends to worker for AI compression │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 4. Worker Processes → Claude Agent SDK Analyzes │
|
||||
│ Extracts structured learnings via iterative AI processing │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 5. Claude Stops → Summary Hook Fires │
|
||||
│ Generates final summary with request, completions, learnings │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 6. Session Ends → Cleanup Hook Fires │
|
||||
│ Marks session complete (graceful, not DELETE), ready for │
|
||||
│ next session context. Skips on /clear to preserve ongoing │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
claude-mem/
|
||||
├── src/
|
||||
│ ├── hooks/ # Hook implementations (7 hooks)
|
||||
│ │ ├── smart-install.ts # Dependency check (cached)
|
||||
│ │ ├── context-hook.ts # SessionStart
|
||||
│ │ ├── user-message-hook.ts # UserMessage (for debugging)
|
||||
│ │ ├── new-hook.ts # UserPromptSubmit
|
||||
│ │ ├── save-hook.ts # PostToolUse
|
||||
│ │ ├── summary-hook.ts # Stop
|
||||
│ │ └── cleanup-hook.ts # SessionEnd
|
||||
│ │
|
||||
│ ├── sdk/ # Claude Agent SDK integration
|
||||
│ │ ├── prompts.ts # XML prompt builders
|
||||
│ │ ├── parser.ts # XML response parser
|
||||
│ │ └── worker.ts # Main SDK agent loop
|
||||
│ │
|
||||
│ ├── services/
|
||||
│ │ ├── worker-service.ts # Express HTTP + SSE service
|
||||
│ │ └── sqlite/ # Database layer
|
||||
│ │ ├── SessionStore.ts # CRUD operations
|
||||
│ │ ├── SessionSearch.ts # FTS5 search service
|
||||
│ │ ├── migrations.ts
|
||||
│ │ └── types.ts
|
||||
│ │
|
||||
│ ├── ui/ # Viewer UI
|
||||
│ │ └── viewer/ # React + TypeScript web interface
|
||||
│ │ ├── components/ # UI components
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── utils/ # Utilities
|
||||
│ │ └── assets/ # Fonts, logos
|
||||
│ │
|
||||
│ ├── shared/ # Shared utilities
|
||||
│ │ ├── config.ts
|
||||
│ │ ├── paths.ts
|
||||
│ │ └── storage.ts
|
||||
│ │
|
||||
│ └── utils/
|
||||
│ ├── logger.ts
|
||||
│ ├── platform.ts
|
||||
│ └── port-allocator.ts
|
||||
│
|
||||
├── plugin/ # Plugin distribution
|
||||
│ ├── .claude-plugin/
|
||||
│ │ └── plugin.json
|
||||
│ ├── hooks/
|
||||
│ │ └── hooks.json
|
||||
│ ├── scripts/ # Built executables
|
||||
│ │ ├── smart-install.js
|
||||
│ │ ├── context-hook.js
|
||||
│ │ ├── user-message-hook.js
|
||||
│ │ ├── new-hook.js
|
||||
│ │ ├── save-hook.js
|
||||
│ │ ├── summary-hook.js
|
||||
│ │ ├── cleanup-hook.js
|
||||
│ │ └── worker-service.cjs # Background worker + HTTP API
|
||||
│ │
|
||||
│ ├── skills/ # Agent skills (v5.4.0+)
|
||||
│ │ ├── search/ # Search skill with progressive disclosure
|
||||
│ │ │ ├── SKILL.md # Skill frontmatter (~250 tokens)
|
||||
│ │ │ └── operations/ # Detailed operation docs
|
||||
│ │ ├── troubleshoot/ # Troubleshooting skill
|
||||
│ │ └── version-bump/ # Version management skill
|
||||
│ │
|
||||
│ └── ui/ # Built viewer UI
|
||||
│ └── viewer.html # Self-contained bundle
|
||||
│
|
||||
├── tests/ # Test suite
|
||||
├── docs/ # Documentation
|
||||
└── ecosystem.config.cjs # PM2 configuration
|
||||
```
|
||||
|
||||
## Component Details
|
||||
|
||||
### 1. Plugin Hooks (7 Hooks)
|
||||
- **smart-install.js** - Cached dependency checker (only runs on version changes)
|
||||
- **context-hook.js** - SessionStart: Starts PM2 worker, injects context
|
||||
- **user-message-hook.js** - UserMessage: Debugging hook
|
||||
- **new-hook.js** - UserPromptSubmit: Creates session, saves prompt
|
||||
- **save-hook.js** - PostToolUse: Captures tool executions
|
||||
- **summary-hook.js** - Stop: Generates session summary
|
||||
- **cleanup-hook.js** - SessionEnd: Marks session complete
|
||||
|
||||
See [Plugin Hooks](/architecture/hooks) for detailed hook documentation.
|
||||
|
||||
### 2. Worker Service
|
||||
Express.js HTTP server on port 37777 (configurable) with:
|
||||
- 10 search HTTP API endpoints (v5.4.0+)
|
||||
- 8 viewer UI HTTP/SSE endpoints
|
||||
- Async observation processing via Claude Agent SDK
|
||||
- Real-time updates via Server-Sent Events
|
||||
- Auto-managed by PM2 process manager
|
||||
|
||||
See [Worker Service](/architecture/worker-service) for HTTP API and endpoints.
|
||||
|
||||
### 3. Database Layer
|
||||
SQLite3 with better-sqlite3 driver featuring:
|
||||
- FTS5 virtual tables for full-text search
|
||||
- SessionStore for CRUD operations
|
||||
- SessionSearch for FTS5 queries
|
||||
- Location: `~/.claude-mem/claude-mem.db`
|
||||
|
||||
See [Database Architecture](/architecture/database) for schema and FTS5 search.
|
||||
|
||||
### 4. Search Skill (v5.4.0+)
|
||||
Skill-based search with progressive disclosure providing 10 search operations:
|
||||
- Search observations, sessions, prompts (full-text FTS5)
|
||||
- Filter by type, concept, file
|
||||
- Get recent context, timeline, timeline by query
|
||||
- API help documentation
|
||||
|
||||
**Token Savings**: ~2,250 tokens per session vs MCP approach
|
||||
- Skill frontmatter: ~250 tokens (loaded at session start)
|
||||
- Full instructions: ~2,500 tokens (loaded on-demand when invoked)
|
||||
- HTTP API endpoints instead of MCP tools
|
||||
|
||||
See [Search Architecture](/architecture/search-architecture) for technical details and examples.
|
||||
|
||||
### 5. Viewer UI
|
||||
React + TypeScript web interface at http://localhost:37777 featuring:
|
||||
- Real-time memory stream via Server-Sent Events
|
||||
- Infinite scroll pagination with automatic deduplication
|
||||
- Project filtering and settings persistence
|
||||
- GPU-accelerated animations
|
||||
- Self-contained HTML bundle (viewer.html)
|
||||
|
||||
Built with esbuild into a single file deployment.
|
||||
@@ -0,0 +1,440 @@
|
||||
---
|
||||
title: "Search Architecture"
|
||||
description: "Skill-based search with HTTP API and progressive disclosure"
|
||||
---
|
||||
|
||||
# Search Architecture
|
||||
|
||||
Claude-Mem uses a skill-based search architecture that provides intelligent memory retrieval through natural language queries. This replaced the MCP-based approach in v5.4.0, saving ~2,250 tokens per session start.
|
||||
|
||||
## Overview
|
||||
|
||||
**Architecture**: Skill-Based Search + HTTP API + Progressive Disclosure
|
||||
|
||||
**Key Components**:
|
||||
1. **Search Skill** (`plugin/skills/search/SKILL.md`) - Auto-invoked when users ask about past work
|
||||
2. **HTTP API Endpoints** (10 routes) - Fast, efficient search operations on port 37777
|
||||
3. **Worker Service** - Express.js server with FTS5 full-text search
|
||||
4. **SQLite Database** - Persistent storage with FTS5 virtual tables
|
||||
5. **Chroma Vector DB** - Semantic search with hybrid retrieval
|
||||
|
||||
## How It Works
|
||||
|
||||
### 1. User Query (Natural Language)
|
||||
|
||||
```
|
||||
User: "What bugs did we fix last session?"
|
||||
```
|
||||
|
||||
### 2. Skill Invocation
|
||||
|
||||
Claude recognizes the intent and invokes the search skill:
|
||||
- Skill frontmatter (~250 tokens) loaded at session start
|
||||
- Full skill instructions loaded on-demand when skill is invoked
|
||||
- Progressive disclosure pattern minimizes context overhead
|
||||
|
||||
### 3. HTTP API Call
|
||||
|
||||
The skill uses `curl` to call the HTTP API:
|
||||
|
||||
```bash
|
||||
curl "http://localhost:37777/api/search/observations?query=bugs&type=bugfix&limit=5"
|
||||
```
|
||||
|
||||
### 4. FTS5 Search
|
||||
|
||||
Worker service queries SQLite FTS5 virtual tables:
|
||||
|
||||
```sql
|
||||
SELECT * FROM observations_fts
|
||||
WHERE observations_fts MATCH ?
|
||||
AND type = 'bugfix'
|
||||
ORDER BY rank
|
||||
LIMIT 5
|
||||
```
|
||||
|
||||
### 5. Results Formatted
|
||||
|
||||
Skill formats results and returns to Claude:
|
||||
|
||||
```
|
||||
## Recent Bugfixes
|
||||
|
||||
1. [bugfix] Fixed authentication token expiry
|
||||
Date: 2025-11-08 14:23:45
|
||||
Files: src/auth/jwt.ts
|
||||
|
||||
2. [bugfix] Resolved database connection leak
|
||||
Date: 2025-11-08 13:15:22
|
||||
Files: src/services/database.ts
|
||||
```
|
||||
|
||||
### 6. User Sees Answer
|
||||
|
||||
Claude presents the formatted results naturally in conversation.
|
||||
|
||||
## Architecture Change (v5.4.0)
|
||||
|
||||
### Before: MCP-Based Search
|
||||
|
||||
**Approach**: 9 MCP tools registered at session start
|
||||
|
||||
**Token Cost**: ~2,500 tokens in tool definitions per session
|
||||
- Each tool's schema, parameters, descriptions loaded
|
||||
- All 9 tools available whether needed or not
|
||||
- No progressive disclosure
|
||||
|
||||
**Example MCP Tool**:
|
||||
```json
|
||||
{
|
||||
"name": "search_observations",
|
||||
"description": "Full-text search across observations...",
|
||||
"inputSchema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"query": { "type": "string", "description": "..." },
|
||||
"type": { "type": "array", "items": { "enum": [...] } },
|
||||
"format": { "enum": ["index", "full"] },
|
||||
// ... many more parameters
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### After: Skill-Based Search
|
||||
|
||||
**Approach**: 1 search skill with progressive disclosure
|
||||
|
||||
**Token Cost**: ~250 tokens in skill frontmatter per session
|
||||
- Only skill description loaded at session start
|
||||
- Full instructions loaded on-demand when skill is invoked
|
||||
- HTTP API endpoints instead of MCP protocol
|
||||
|
||||
**Example Skill Frontmatter**:
|
||||
```markdown
|
||||
# Claude-Mem Search Skill
|
||||
|
||||
Access claude-mem's persistent memory through a comprehensive HTTP API.
|
||||
Search for past work, understand context, and learn from previous decisions.
|
||||
|
||||
## When to Use This Skill
|
||||
|
||||
Invoke this skill when users ask about:
|
||||
- Past work: "What did we do last session?"
|
||||
- Bug fixes: "Did we fix this before?"
|
||||
- Features: "How did we implement authentication?"
|
||||
...
|
||||
```
|
||||
|
||||
**Token Savings**: ~2,250 tokens per session start (90% reduction)
|
||||
|
||||
## HTTP API Endpoints
|
||||
|
||||
The worker service exposes 10 search endpoints:
|
||||
|
||||
### Full-Text Search
|
||||
|
||||
```
|
||||
GET /api/search/observations
|
||||
GET /api/search/sessions
|
||||
GET /api/search/prompts
|
||||
```
|
||||
|
||||
**Parameters**:
|
||||
- `query` - FTS5 search query (required)
|
||||
- `type` - Filter by type (bugfix, feature, refactor, etc.)
|
||||
- `project` - Filter by project name
|
||||
- `limit` - Maximum results (default: 20)
|
||||
- `offset` - Pagination offset
|
||||
- `format` - Response format (index or full)
|
||||
|
||||
**Example**:
|
||||
```bash
|
||||
curl "http://localhost:37777/api/search/observations?query=authentication&type=decision&limit=5"
|
||||
```
|
||||
|
||||
### Filtered Search
|
||||
|
||||
```
|
||||
GET /api/search/by-type
|
||||
GET /api/search/by-concept
|
||||
GET /api/search/by-file
|
||||
```
|
||||
|
||||
**Parameters**:
|
||||
- `type` / `concept` / `filePath` - Filter criteria (required)
|
||||
- `project` - Filter by project
|
||||
- `limit` - Maximum results
|
||||
- `format` - Response format
|
||||
|
||||
**Example**:
|
||||
```bash
|
||||
curl "http://localhost:37777/api/search/by-file?filePath=worker-service.ts&limit=10"
|
||||
```
|
||||
|
||||
### Context Retrieval
|
||||
|
||||
```
|
||||
GET /api/context/recent
|
||||
GET /api/context/timeline
|
||||
GET /api/timeline/by-query
|
||||
```
|
||||
|
||||
**Parameters**:
|
||||
- `project` - Filter by project
|
||||
- `limit` - Number of sessions/records
|
||||
- `anchor` - Timeline anchor point (ID or timestamp)
|
||||
- `depth_before` - Records before anchor
|
||||
- `depth_after` - Records after anchor
|
||||
|
||||
**Example**:
|
||||
```bash
|
||||
curl "http://localhost:37777/api/context/recent?project=claude-mem&limit=5"
|
||||
```
|
||||
|
||||
### Documentation
|
||||
|
||||
```
|
||||
GET /api/search/help
|
||||
```
|
||||
|
||||
Returns API documentation in JSON format.
|
||||
|
||||
## Progressive Disclosure Pattern
|
||||
|
||||
The search skill uses progressive disclosure to minimize token usage:
|
||||
|
||||
### Layer 1: Skill Frontmatter (Session Start)
|
||||
|
||||
**What's Loaded**: Skill description and when to use it (~250 tokens)
|
||||
|
||||
**Purpose**: Claude can recognize when to invoke the skill
|
||||
|
||||
**Example**:
|
||||
```markdown
|
||||
# Claude-Mem Search Skill
|
||||
|
||||
Access claude-mem's persistent memory through a comprehensive HTTP API.
|
||||
|
||||
## When to Use This Skill
|
||||
Invoke this skill when users ask about:
|
||||
- Past work: "What did we do last session?"
|
||||
- Bug fixes: "Did we fix this before?"
|
||||
...
|
||||
```
|
||||
|
||||
### Layer 2: Full Skill Instructions (On-Demand)
|
||||
|
||||
**What's Loaded**: Complete operation documentation (~2,500 tokens)
|
||||
|
||||
**Purpose**: Detailed instructions for each search operation
|
||||
|
||||
**When Loaded**: Only when Claude invokes the skill
|
||||
|
||||
**Example Structure**:
|
||||
```
|
||||
/skills/search/
|
||||
├── SKILL.md (main frontmatter)
|
||||
├── operations/
|
||||
│ ├── observations.md (detailed instructions)
|
||||
│ ├── sessions.md
|
||||
│ ├── prompts.md
|
||||
│ ├── by-type.md
|
||||
│ ├── by-concept.md
|
||||
│ ├── by-file.md
|
||||
│ ├── recent-context.md
|
||||
│ ├── timeline.md
|
||||
│ ├── timeline-by-query.md
|
||||
│ ├── help.md
|
||||
│ ├── formatting.md
|
||||
│ └── common-workflows.md
|
||||
```
|
||||
|
||||
### Layer 3: API Response
|
||||
|
||||
**What's Returned**: Search results in requested format
|
||||
|
||||
**Format Options**:
|
||||
- `index` - Titles, dates, IDs only (~50-100 tokens per result)
|
||||
- `full` - Complete details (~500-1000 tokens per result)
|
||||
|
||||
**Progressive Usage**: Start with `index`, drill down with `full` as needed
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Search Skill Structure
|
||||
|
||||
```
|
||||
plugin/skills/search/
|
||||
├── SKILL.md # Main frontmatter (~250 tokens)
|
||||
├── operations/
|
||||
│ ├── observations.md # Search observations
|
||||
│ ├── sessions.md # Search sessions
|
||||
│ ├── prompts.md # Search prompts
|
||||
│ ├── by-type.md # Filter by type
|
||||
│ ├── by-concept.md # Filter by concept
|
||||
│ ├── by-file.md # Filter by file
|
||||
│ ├── recent-context.md # Get recent context
|
||||
│ ├── timeline.md # Timeline around point
|
||||
│ ├── timeline-by-query.md # Search + timeline
|
||||
│ ├── help.md # API documentation
|
||||
│ ├── formatting.md # Result formatting guide
|
||||
│ └── common-workflows.md # Usage patterns
|
||||
```
|
||||
|
||||
### Worker Service Integration
|
||||
|
||||
**File**: `src/services/worker-service.ts`
|
||||
|
||||
**Search Routes**:
|
||||
```typescript
|
||||
// Full-text search
|
||||
app.get('/api/search/observations', handleSearchObservations);
|
||||
app.get('/api/search/sessions', handleSearchSessions);
|
||||
app.get('/api/search/prompts', handleSearchPrompts);
|
||||
|
||||
// Filtered search
|
||||
app.get('/api/search/by-type', handleSearchByType);
|
||||
app.get('/api/search/by-concept', handleSearchByConcept);
|
||||
app.get('/api/search/by-file', handleSearchByFile);
|
||||
|
||||
// Context retrieval
|
||||
app.get('/api/context/recent', handleRecentContext);
|
||||
app.get('/api/context/timeline', handleTimeline);
|
||||
app.get('/api/timeline/by-query', handleTimelineByQuery);
|
||||
|
||||
// Documentation
|
||||
app.get('/api/search/help', handleHelp);
|
||||
```
|
||||
|
||||
**Database Access**:
|
||||
- Uses `SessionSearch` service for FTS5 queries
|
||||
- Uses `SessionStore` for structured queries
|
||||
- Hybrid search with ChromaDB for semantic similarity
|
||||
|
||||
### Security
|
||||
|
||||
**FTS5 Injection Prevention** (v4.2.3):
|
||||
```typescript
|
||||
function escapeFTS5Query(query: string): string {
|
||||
return query.replace(/"/g, '""');
|
||||
}
|
||||
```
|
||||
|
||||
All user-provided search queries are properly escaped to prevent SQL injection.
|
||||
|
||||
**Comprehensive Testing**: 332 injection attack tests covering:
|
||||
- Special characters
|
||||
- SQL keywords
|
||||
- Quote escaping
|
||||
- Boolean operators
|
||||
|
||||
## Benefits
|
||||
|
||||
### 1. Token Efficiency
|
||||
|
||||
**Before (MCP)**:
|
||||
- Session start: ~2,500 tokens for tool definitions
|
||||
- Every session pays this cost
|
||||
- No progressive disclosure
|
||||
|
||||
**After (Skill)**:
|
||||
- Session start: ~250 tokens for skill frontmatter
|
||||
- Full instructions: ~2,500 tokens (only when invoked)
|
||||
- Net savings: ~2,250 tokens per session (~90% reduction)
|
||||
|
||||
### 2. Natural Language Interface
|
||||
|
||||
**Before**: Users needed to learn MCP tool syntax
|
||||
```
|
||||
search_observations with query="authentication" and type="decision"
|
||||
```
|
||||
|
||||
**After**: Users ask naturally
|
||||
```
|
||||
"What decisions did we make about authentication?"
|
||||
```
|
||||
|
||||
Claude translates to appropriate API call.
|
||||
|
||||
### 3. Flexibility
|
||||
|
||||
**HTTP API Benefits**:
|
||||
- Can be called from skills, MCP tools, or other clients
|
||||
- Easy to test with curl
|
||||
- Standard REST conventions
|
||||
- JSON responses
|
||||
|
||||
**Progressive Disclosure**:
|
||||
- Loads only what's needed
|
||||
- Can add more operations without increasing base cost
|
||||
- Documentation co-located with operations
|
||||
|
||||
### 4. Performance
|
||||
|
||||
**Fast Queries**: FTS5 full-text search <10ms for typical queries
|
||||
|
||||
**Caching**: HTTP layer allows response caching
|
||||
|
||||
**Pagination**: Efficient result pagination with offset/limit
|
||||
|
||||
## Migration Notes
|
||||
|
||||
### For Users
|
||||
|
||||
**No Action Required**: The migration from MCP to skill-based search is transparent.
|
||||
|
||||
**Same Questions Work**: Natural language queries work exactly the same way.
|
||||
|
||||
**Invisible Change**: Users won't notice any difference except better performance.
|
||||
|
||||
### For Developers
|
||||
|
||||
**Deprecated**: MCP search server (`src/servers/search-server.ts`)
|
||||
- Source file kept for reference
|
||||
- No longer built or registered
|
||||
- MCP configuration removed from `plugin/.mcp.json`
|
||||
|
||||
**New Implementation**: Skill-based search
|
||||
- Skill files: `plugin/skills/search/`
|
||||
- HTTP endpoints: `src/services/worker-service.ts` (lines 200-400)
|
||||
- Build script: `npm run build` includes skill files
|
||||
- Sync script: `npm run sync-marketplace` copies to plugin directory
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Worker Service Not Running
|
||||
|
||||
If searches fail, check worker service:
|
||||
|
||||
```bash
|
||||
pm2 list # Check status
|
||||
npm run worker:restart # Restart worker
|
||||
npm run worker:logs # View logs
|
||||
```
|
||||
|
||||
### HTTP Endpoints Not Responding
|
||||
|
||||
Test endpoints directly:
|
||||
|
||||
```bash
|
||||
# Health check
|
||||
curl http://localhost:37777/health
|
||||
|
||||
# Search test
|
||||
curl "http://localhost:37777/api/search/observations?query=test&limit=1"
|
||||
```
|
||||
|
||||
### Skill Not Invoking
|
||||
|
||||
If Claude doesn't invoke the skill:
|
||||
|
||||
1. Check skill files exist: `ls ~/.claude/plugins/marketplaces/thedotmack/plugin/skills/search/`
|
||||
2. Restart Claude Code session
|
||||
3. Try explicit skill invocation: `/skill search`
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Search Tools Usage](/usage/search-tools) - User guide with examples
|
||||
- [Worker Service Architecture](/architecture/worker-service) - HTTP API details
|
||||
- [Database Schema](/architecture/database) - FTS5 tables and indexes
|
||||
@@ -0,0 +1,443 @@
|
||||
---
|
||||
title: "Worker Service"
|
||||
description: "HTTP API and PM2 process management"
|
||||
---
|
||||
|
||||
# Worker Service
|
||||
|
||||
The worker service is a long-running HTTP API built with Express.js and managed by PM2. It processes observations through the Claude Agent SDK separately from hook execution to prevent timeout issues.
|
||||
|
||||
## Overview
|
||||
|
||||
- **Technology**: Express.js HTTP server
|
||||
- **Process Manager**: PM2
|
||||
- **Port**: Fixed port 37777 (configurable via `CLAUDE_MEM_WORKER_PORT`)
|
||||
- **Location**: `src/services/worker-service.ts`
|
||||
- **Built Output**: `plugin/scripts/worker-service.cjs`
|
||||
- **Model**: Configurable via `CLAUDE_MEM_MODEL` environment variable (default: claude-sonnet-4-5)
|
||||
|
||||
## REST API Endpoints
|
||||
|
||||
The worker service exposes 14 HTTP endpoints organized into four categories:
|
||||
|
||||
### Viewer & Health Endpoints
|
||||
|
||||
#### 1. Viewer UI
|
||||
```
|
||||
GET /
|
||||
```
|
||||
|
||||
**Purpose**: Serves the web-based viewer UI (v5.1.0+)
|
||||
|
||||
**Response**: HTML page with embedded React application
|
||||
|
||||
**Features**:
|
||||
- Real-time memory stream visualization
|
||||
- Infinite scroll pagination
|
||||
- Project filtering
|
||||
- SSE-based live updates
|
||||
- Theme toggle (light/dark mode) as of v5.1.2
|
||||
|
||||
#### 2. Health Check
|
||||
```
|
||||
GET /health
|
||||
```
|
||||
|
||||
**Purpose**: Worker health status check
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"uptime": 12345,
|
||||
"port": 37777
|
||||
}
|
||||
```
|
||||
|
||||
#### 3. Server-Sent Events Stream
|
||||
```
|
||||
GET /stream
|
||||
```
|
||||
|
||||
**Purpose**: Real-time updates for viewer UI
|
||||
|
||||
**Response**: SSE stream with events:
|
||||
- `observation-created`: New observation added
|
||||
- `session-summary-created`: New summary generated
|
||||
- `user-prompt-created`: New prompt recorded
|
||||
|
||||
**Event Format**:
|
||||
```
|
||||
event: observation-created
|
||||
data: {"id": 123, "title": "...", ...}
|
||||
```
|
||||
|
||||
### Data Retrieval Endpoints
|
||||
|
||||
#### 4. Get Prompts
|
||||
```
|
||||
GET /api/prompts?project=my-project&limit=20&offset=0
|
||||
```
|
||||
|
||||
**Purpose**: Retrieve paginated user prompts
|
||||
|
||||
**Query Parameters**:
|
||||
- `project` (optional): Filter by project name
|
||||
- `limit` (default: 20): Number of results
|
||||
- `offset` (default: 0): Pagination offset
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"prompts": [{
|
||||
"id": 1,
|
||||
"session_id": "abc123",
|
||||
"prompt": "User's prompt text",
|
||||
"prompt_number": 1,
|
||||
"created_at": "2025-11-06T10:30:00Z"
|
||||
}],
|
||||
"total": 150,
|
||||
"hasMore": true
|
||||
}
|
||||
```
|
||||
|
||||
#### 5. Get Observations
|
||||
```
|
||||
GET /api/observations?project=my-project&limit=20&offset=0
|
||||
```
|
||||
|
||||
**Purpose**: Retrieve paginated observations
|
||||
|
||||
**Query Parameters**:
|
||||
- `project` (optional): Filter by project name
|
||||
- `limit` (default: 20): Number of results
|
||||
- `offset` (default: 0): Pagination offset
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"observations": [{
|
||||
"id": 123,
|
||||
"title": "Fix authentication bug",
|
||||
"type": "bugfix",
|
||||
"narrative": "...",
|
||||
"created_at": "2025-11-06T10:30:00Z"
|
||||
}],
|
||||
"total": 500,
|
||||
"hasMore": true
|
||||
}
|
||||
```
|
||||
|
||||
#### 6. Get Summaries
|
||||
```
|
||||
GET /api/summaries?project=my-project&limit=20&offset=0
|
||||
```
|
||||
|
||||
**Purpose**: Retrieve paginated session summaries
|
||||
|
||||
**Query Parameters**:
|
||||
- `project` (optional): Filter by project name
|
||||
- `limit` (default: 20): Number of results
|
||||
- `offset` (default: 0): Pagination offset
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"summaries": [{
|
||||
"id": 456,
|
||||
"session_id": "abc123",
|
||||
"request": "User's original request",
|
||||
"completed": "Work finished",
|
||||
"created_at": "2025-11-06T10:30:00Z"
|
||||
}],
|
||||
"total": 100,
|
||||
"hasMore": true
|
||||
}
|
||||
```
|
||||
|
||||
#### 7. Get Stats
|
||||
```
|
||||
GET /api/stats
|
||||
```
|
||||
|
||||
**Purpose**: Get database statistics by project
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"byProject": {
|
||||
"my-project": {
|
||||
"observations": 245,
|
||||
"summaries": 12,
|
||||
"prompts": 48
|
||||
},
|
||||
"other-project": {
|
||||
"observations": 156,
|
||||
"summaries": 8,
|
||||
"prompts": 32
|
||||
}
|
||||
},
|
||||
"total": {
|
||||
"observations": 401,
|
||||
"summaries": 20,
|
||||
"prompts": 80,
|
||||
"sessions": 20
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Settings Endpoints
|
||||
|
||||
#### 8. Get Settings
|
||||
```
|
||||
GET /api/settings
|
||||
```
|
||||
|
||||
**Purpose**: Retrieve user settings
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"sidebarOpen": true,
|
||||
"selectedProject": "my-project",
|
||||
"theme": "dark"
|
||||
}
|
||||
```
|
||||
|
||||
#### 9. Save Settings
|
||||
```
|
||||
POST /api/settings
|
||||
```
|
||||
|
||||
**Purpose**: Persist user settings
|
||||
|
||||
**Request Body**:
|
||||
```json
|
||||
{
|
||||
"sidebarOpen": false,
|
||||
"selectedProject": "other-project",
|
||||
"theme": "light"
|
||||
}
|
||||
```
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"success": true
|
||||
}
|
||||
```
|
||||
|
||||
### Session Management Endpoints
|
||||
|
||||
#### 10. Initialize Session
|
||||
```
|
||||
POST /sessions/:sessionDbId/init
|
||||
```
|
||||
|
||||
**Request Body**:
|
||||
```json
|
||||
{
|
||||
"sdk_session_id": "abc-123",
|
||||
"project": "my-project"
|
||||
}
|
||||
```
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"session_id": "abc-123"
|
||||
}
|
||||
```
|
||||
|
||||
#### 11. Add Observation
|
||||
```
|
||||
POST /sessions/:sessionDbId/observations
|
||||
```
|
||||
|
||||
**Request Body**:
|
||||
```json
|
||||
{
|
||||
"tool_name": "Read",
|
||||
"tool_input": {...},
|
||||
"tool_result": "...",
|
||||
"correlation_id": "xyz-789"
|
||||
}
|
||||
```
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"observation_id": 123
|
||||
}
|
||||
```
|
||||
|
||||
#### 12. Generate Summary
|
||||
```
|
||||
POST /sessions/:sessionDbId/summarize
|
||||
```
|
||||
|
||||
**Request Body**:
|
||||
```json
|
||||
{
|
||||
"trigger": "stop"
|
||||
}
|
||||
```
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"summary_id": 456
|
||||
}
|
||||
```
|
||||
|
||||
#### 13. Session Status
|
||||
```
|
||||
GET /sessions/:sessionDbId/status
|
||||
```
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"session_id": "abc-123",
|
||||
"status": "active",
|
||||
"observation_count": 42,
|
||||
"summary_count": 1
|
||||
}
|
||||
```
|
||||
|
||||
#### 14. Delete Session
|
||||
```
|
||||
DELETE /sessions/:sessionDbId
|
||||
```
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"success": true
|
||||
}
|
||||
```
|
||||
|
||||
**Note**: As of v4.1.0, the cleanup hook no longer calls this endpoint. Sessions are marked complete instead of deleted to allow graceful worker shutdown.
|
||||
|
||||
## PM2 Management
|
||||
|
||||
### Configuration
|
||||
|
||||
The worker is configured via `ecosystem.config.cjs`:
|
||||
|
||||
```javascript
|
||||
module.exports = {
|
||||
apps: [{
|
||||
name: 'claude-mem-worker',
|
||||
script: './plugin/scripts/worker-service.cjs',
|
||||
instances: 1,
|
||||
autorestart: true,
|
||||
watch: false,
|
||||
max_memory_restart: '1G',
|
||||
env: {
|
||||
NODE_ENV: 'production',
|
||||
FORCE_COLOR: '1'
|
||||
}
|
||||
}]
|
||||
};
|
||||
```
|
||||
|
||||
### Commands
|
||||
|
||||
```bash
|
||||
# Start worker (auto-starts on first session)
|
||||
npm run worker:start
|
||||
|
||||
# Stop worker
|
||||
npm run worker:stop
|
||||
|
||||
# Restart worker
|
||||
npm run worker:restart
|
||||
|
||||
# View logs
|
||||
npm run worker:logs
|
||||
|
||||
# Check status
|
||||
npm run worker:status
|
||||
```
|
||||
|
||||
### Auto-Start Behavior
|
||||
|
||||
As of v4.0.0, the worker service auto-starts when the SessionStart hook fires. Manual start is optional.
|
||||
|
||||
## Claude Agent SDK Integration
|
||||
|
||||
The worker service routes observations to the Claude Agent SDK for AI-powered processing:
|
||||
|
||||
### Processing Flow
|
||||
|
||||
1. **Observation Queue**: Observations accumulate in memory
|
||||
2. **SDK Processing**: Observations sent to Claude via Agent SDK
|
||||
3. **XML Parsing**: Responses parsed for structured data
|
||||
4. **Database Storage**: Processed observations stored in SQLite
|
||||
|
||||
### SDK Components
|
||||
|
||||
- **Prompts** (`src/sdk/prompts.ts`): Builds XML-structured prompts
|
||||
- **Parser** (`src/sdk/parser.ts`): Parses Claude's XML responses
|
||||
- **Worker** (`src/sdk/worker.ts`): Main SDK agent loop
|
||||
|
||||
### Model Configuration
|
||||
|
||||
Set the AI model used for processing via environment variable:
|
||||
|
||||
```bash
|
||||
export CLAUDE_MEM_MODEL=claude-sonnet-4-5
|
||||
```
|
||||
|
||||
Available models:
|
||||
- `claude-haiku-4-5` - Fast, cost-efficient
|
||||
- `claude-sonnet-4-5` - Balanced (default)
|
||||
- `claude-opus-4` - Most capable
|
||||
- `claude-3-7-sonnet` - Alternative version
|
||||
|
||||
## Port Allocation
|
||||
|
||||
The worker uses a fixed port (37777 by default) for consistent communication:
|
||||
|
||||
- **Default**: Port 37777
|
||||
- **Override**: Set `CLAUDE_MEM_WORKER_PORT` environment variable
|
||||
- **Port File**: `${CLAUDE_PLUGIN_ROOT}/data/worker.port` tracks current port
|
||||
|
||||
If port 37777 is in use, the worker will fail to start. Set a custom port via environment variable.
|
||||
|
||||
## Data Storage
|
||||
|
||||
The worker service stores data in the plugin data directory:
|
||||
|
||||
```
|
||||
${CLAUDE_PLUGIN_ROOT}/data/
|
||||
├── claude-mem.db # SQLite database
|
||||
├── worker.port # Current worker port file
|
||||
└── logs/
|
||||
├── worker-out.log # Worker stdout logs
|
||||
└── worker-error.log # Worker stderr logs
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
The worker implements graceful degradation:
|
||||
|
||||
- **Database Errors**: Logged but don't crash the service
|
||||
- **SDK Errors**: Retried with exponential backoff
|
||||
- **Network Errors**: Logged and skipped
|
||||
- **Invalid Input**: Validated and rejected with error response
|
||||
|
||||
## Performance
|
||||
|
||||
- **Async Processing**: Observations processed asynchronously
|
||||
- **In-Memory Queue**: Fast observation accumulation
|
||||
- **Batch Processing**: Multiple observations processed together
|
||||
- **Connection Pooling**: SQLite connections reused
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
See [Troubleshooting - Worker Issues](../troubleshooting.md#worker-service-issues) for common problems and solutions.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 78 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 78 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 42 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.1 MiB |
@@ -0,0 +1,342 @@
|
||||
---
|
||||
title: "Configuration"
|
||||
description: "Environment variables and settings for Claude-Mem"
|
||||
---
|
||||
|
||||
# Configuration
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|-------------------------------|---------------------------------|---------------------------------------|
|
||||
| `CLAUDE_PLUGIN_ROOT` | Set by Claude Code | Plugin installation directory |
|
||||
| `CLAUDE_MEM_DATA_DIR` | `~/.claude-mem/` | Data directory (production default) |
|
||||
| `CLAUDE_CODE_PATH` | Auto-detected | Path to Claude Code CLI (for Windows) |
|
||||
| `CLAUDE_MEM_WORKER_PORT` | `37777` | Worker service port |
|
||||
| `CLAUDE_MEM_MODEL` | `claude-sonnet-4-5` | AI model for processing observations |
|
||||
| `CLAUDE_MEM_CONTEXT_OBSERVATIONS` | `50` | Number of observations to inject |
|
||||
| `NODE_ENV` | `production` | Environment mode |
|
||||
| `FORCE_COLOR` | `1` | Enable colored logs |
|
||||
|
||||
## Model Configuration
|
||||
|
||||
Configure which AI model processes your observations.
|
||||
|
||||
### Available Models
|
||||
|
||||
- `claude-haiku-4-5` - Fast, cost-efficient
|
||||
- `claude-sonnet-4-5` - Balanced (default)
|
||||
- `claude-opus-4` - Most capable
|
||||
- `claude-3-7-sonnet` - Alternative version
|
||||
|
||||
### Using the Interactive Script
|
||||
|
||||
```bash
|
||||
./claude-mem-settings.sh
|
||||
```
|
||||
|
||||
This script manages `CLAUDE_MEM_MODEL` in `~/.claude/settings.json`.
|
||||
|
||||
### Manual Configuration
|
||||
|
||||
Edit `~/.claude/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"CLAUDE_MEM_MODEL": "claude-sonnet-4-5"
|
||||
}
|
||||
```
|
||||
|
||||
## Files and Directories
|
||||
|
||||
### Data Directory Structure
|
||||
|
||||
The data directory location depends on the environment:
|
||||
- **Production (installed plugin)**: `~/.claude-mem/` (always, regardless of CLAUDE_PLUGIN_ROOT)
|
||||
- **Development**: Can be overridden with `CLAUDE_MEM_DATA_DIR`
|
||||
|
||||
```
|
||||
~/.claude-mem/
|
||||
├── claude-mem.db # SQLite database
|
||||
├── .install-version # Cached version for smart installer
|
||||
├── worker.port # Current worker port file
|
||||
└── logs/
|
||||
├── worker-out.log # Worker stdout logs
|
||||
└── worker-error.log # Worker stderr logs
|
||||
```
|
||||
|
||||
### Plugin Directory Structure
|
||||
|
||||
```
|
||||
${CLAUDE_PLUGIN_ROOT}/
|
||||
├── .claude-plugin/
|
||||
│ └── plugin.json # Plugin metadata
|
||||
├── .mcp.json # MCP server configuration
|
||||
├── hooks/
|
||||
│ └── hooks.json # Hook configuration
|
||||
├── scripts/ # Built executables
|
||||
│ ├── smart-install.js # Smart installer script
|
||||
│ ├── context-hook.js # Context injection hook
|
||||
│ ├── new-hook.js # Session creation hook
|
||||
│ ├── save-hook.js # Observation capture hook
|
||||
│ ├── summary-hook.js # Summary generation hook
|
||||
│ ├── cleanup-hook.js # Session cleanup hook
|
||||
│ ├── worker-service.cjs # Worker service (CJS)
|
||||
│ └── search-server.mjs # MCP search server (ESM)
|
||||
└── ui/
|
||||
└── viewer.html # Web viewer UI bundle
|
||||
```
|
||||
|
||||
## Plugin Configuration
|
||||
|
||||
### Hooks Configuration
|
||||
|
||||
Hooks are configured in `plugin/hooks/hooks.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"description": "Claude-mem memory system hooks",
|
||||
"hooks": {
|
||||
"SessionStart": [{
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/smart-install.js && node ${CLAUDE_PLUGIN_ROOT}/scripts/context-hook.js",
|
||||
"timeout": 120
|
||||
}]
|
||||
}],
|
||||
"UserPromptSubmit": [{
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/new-hook.js",
|
||||
"timeout": 120
|
||||
}]
|
||||
}],
|
||||
"PostToolUse": [{
|
||||
"matcher": "*",
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/save-hook.js",
|
||||
"timeout": 120
|
||||
}]
|
||||
}],
|
||||
"Stop": [{
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/summary-hook.js",
|
||||
"timeout": 120
|
||||
}]
|
||||
}],
|
||||
"SessionEnd": [{
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/cleanup-hook.js",
|
||||
"timeout": 120
|
||||
}]
|
||||
}]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Search Configuration (v5.4.0+)
|
||||
|
||||
**Migration Note**: As of v5.4.0, Claude-Mem uses skill-based search instead of MCP tools.
|
||||
|
||||
**Previous (v5.3.x and earlier)**: MCP search server with 9 tools (~2,500 tokens per session)
|
||||
**Current (v5.4.0+)**: Search skill with HTTP API (~250 tokens per session)
|
||||
|
||||
**No configuration required** - the search skill is automatically available in Claude Code sessions.
|
||||
|
||||
Search operations are now provided via:
|
||||
- **Skill**: `plugin/skills/search/SKILL.md` (auto-invoked when users ask about past work)
|
||||
- **HTTP API**: 10 endpoints on worker service port 37777
|
||||
- **Progressive Disclosure**: Full instructions loaded on-demand only when needed
|
||||
|
||||
## PM2 Configuration
|
||||
|
||||
Worker service is managed by PM2 via `ecosystem.config.cjs`:
|
||||
|
||||
```javascript
|
||||
module.exports = {
|
||||
apps: [{
|
||||
name: 'claude-mem-worker',
|
||||
script: './plugin/scripts/worker-service.cjs',
|
||||
instances: 1,
|
||||
autorestart: true,
|
||||
watch: false,
|
||||
max_memory_restart: '1G',
|
||||
env: {
|
||||
NODE_ENV: 'production',
|
||||
FORCE_COLOR: '1'
|
||||
}
|
||||
}]
|
||||
};
|
||||
```
|
||||
|
||||
### PM2 Settings
|
||||
|
||||
- **instances**: 1 (single instance)
|
||||
- **autorestart**: true (auto-restart on crash)
|
||||
- **watch**: false (no file watching)
|
||||
- **max_memory_restart**: 1G (restart if memory exceeds 1GB)
|
||||
|
||||
## Context Injection Configuration
|
||||
|
||||
### CLAUDE_MEM_CONTEXT_OBSERVATIONS
|
||||
|
||||
Controls how many observations are injected into each new session for context continuity.
|
||||
|
||||
**Default**: 50 observations
|
||||
|
||||
**What it does**:
|
||||
- Fetches the most recent N observations from the database
|
||||
- Injects them as context at SessionStart
|
||||
- Allows Claude to maintain awareness of recent work across sessions
|
||||
|
||||
**Configuration** in `~/.claude/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"env": {
|
||||
"CLAUDE_MEM_CONTEXT_OBSERVATIONS": "100"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Considerations**:
|
||||
- **Higher values** = More context but slower SessionStart and more tokens used
|
||||
- **Lower values** = Faster SessionStart but less historical awareness
|
||||
- Default of 50 balances context richness with performance
|
||||
|
||||
**Note**: This injects individual observations, not entire sessions. Each observation represents a single tool execution (Read, Write, Edit, etc.) that was compressed into a semantic learning.
|
||||
|
||||
## Customization
|
||||
|
||||
### Custom Data Directory
|
||||
|
||||
For development or testing, override the data directory:
|
||||
|
||||
```bash
|
||||
export CLAUDE_MEM_DATA_DIR=/custom/path
|
||||
```
|
||||
|
||||
### Custom Worker Port
|
||||
|
||||
If port 37777 is in use:
|
||||
|
||||
```bash
|
||||
export CLAUDE_MEM_WORKER_PORT=38000
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
### Custom Model
|
||||
|
||||
Use a different AI model:
|
||||
|
||||
```bash
|
||||
export CLAUDE_MEM_MODEL=claude-opus-4
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
### Hook Timeouts
|
||||
|
||||
Modify timeouts in `plugin/hooks/hooks.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"timeout": 120 // Default: 120 seconds
|
||||
}
|
||||
```
|
||||
|
||||
Recommended values:
|
||||
- SessionStart: 120s (needs time for smart install check and context retrieval)
|
||||
- UserPromptSubmit: 60s
|
||||
- PostToolUse: 120s (can process many observations)
|
||||
- Stop: 60s
|
||||
- SessionEnd: 60s
|
||||
|
||||
**Note**: With smart install caching (v5.0.3+), SessionStart is typically very fast (10ms) unless dependencies need installation.
|
||||
|
||||
### Worker Memory Limit
|
||||
|
||||
Modify PM2 memory limit in `ecosystem.config.cjs`:
|
||||
|
||||
```javascript
|
||||
{
|
||||
max_memory_restart: '2G' // Increase if needed
|
||||
}
|
||||
```
|
||||
|
||||
### Logging Verbosity
|
||||
|
||||
Enable debug logging:
|
||||
|
||||
```bash
|
||||
export DEBUG=claude-mem:*
|
||||
npm run worker:restart
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
## Configuration Best Practices
|
||||
|
||||
1. **Use defaults**: Default configuration works for most use cases
|
||||
2. **Override selectively**: Only change what you need
|
||||
3. **Document changes**: Keep track of custom configurations
|
||||
4. **Test after changes**: Verify worker restarts successfully
|
||||
5. **Monitor logs**: Check worker logs after configuration changes
|
||||
|
||||
## Troubleshooting Configuration
|
||||
|
||||
### Configuration Not Applied
|
||||
|
||||
1. Restart worker after changes:
|
||||
```bash
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
2. Verify environment variables:
|
||||
```bash
|
||||
echo $CLAUDE_MEM_MODEL
|
||||
echo $CLAUDE_MEM_WORKER_PORT
|
||||
```
|
||||
|
||||
3. Check worker logs:
|
||||
```bash
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
### Invalid Model Name
|
||||
|
||||
If you specify an invalid model name, the worker will fall back to `claude-sonnet-4-5` and log a warning.
|
||||
|
||||
Valid models:
|
||||
- claude-haiku-4-5
|
||||
- claude-sonnet-4-5
|
||||
- claude-opus-4
|
||||
- claude-3-7-sonnet
|
||||
|
||||
### Port Already in Use
|
||||
|
||||
If port 37777 is already in use:
|
||||
|
||||
1. Set custom port:
|
||||
```bash
|
||||
export CLAUDE_MEM_WORKER_PORT=38000
|
||||
```
|
||||
|
||||
2. Restart worker:
|
||||
```bash
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
3. Verify new port:
|
||||
```bash
|
||||
cat ~/.claude-mem/worker.port
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Architecture Overview](architecture/overview) - Understand the system
|
||||
- [Troubleshooting](troubleshooting) - Common issues
|
||||
- [Development](development) - Building from source
|
||||
@@ -0,0 +1,222 @@
|
||||
# Context Engineering for AI Agents: Best Practices Cheat Sheet
|
||||
|
||||
## Core Principle
|
||||
**Find the smallest possible set of high-signal tokens that maximize the likelihood of your desired outcome.**
|
||||
|
||||
---
|
||||
|
||||
## Context Engineering vs Prompt Engineering
|
||||
|
||||
**Prompt Engineering**: Writing and organizing LLM instructions for optimal outcomes (one-time task)
|
||||
|
||||
**Context Engineering**: Curating and maintaining the optimal set of tokens during inference across multiple turns (iterative process)
|
||||
|
||||
Context engineering manages:
|
||||
- System instructions
|
||||
- Tools
|
||||
- Model Context Protocol (MCP)
|
||||
- External data
|
||||
- Message history
|
||||
- Runtime data retrieval
|
||||
|
||||
---
|
||||
|
||||
## The Problem: Context Rot
|
||||
|
||||
**Key Insight**: LLMs have an "attention budget" that gets depleted as context grows
|
||||
|
||||
- Every token attends to every other token (n² relationships)
|
||||
- As context length increases, model accuracy decreases
|
||||
- Models have less training experience with longer sequences
|
||||
- Context must be treated as a finite resource with diminishing marginal returns
|
||||
|
||||
---
|
||||
|
||||
## System Prompts: Find the "Right Altitude"
|
||||
|
||||
### The Goldilocks Zone
|
||||
|
||||
**Too Prescriptive** ❌
|
||||
- Hardcoded if-else logic
|
||||
- Brittle and fragile
|
||||
- High maintenance complexity
|
||||
|
||||
**Too Vague** ❌
|
||||
- High-level guidance without concrete signals
|
||||
- Falsely assumes shared context
|
||||
- Lacks actionable direction
|
||||
|
||||
**Just Right** ✅
|
||||
- Specific enough to guide behavior effectively
|
||||
- Flexible enough to provide strong heuristics
|
||||
- Minimal set of information that fully outlines expected behavior
|
||||
|
||||
### Best Practices
|
||||
- Use simple, direct language
|
||||
- Organize into distinct sections (`<background_information>`, `<instructions>`, `## Tool guidance`, etc.)
|
||||
- Use XML tags or Markdown headers for structure
|
||||
- Start with minimal prompt, add based on failure modes
|
||||
- Note: Minimal ≠ short (provide sufficient information upfront)
|
||||
|
||||
---
|
||||
|
||||
## Tools: Minimal and Clear
|
||||
|
||||
### Design Principles
|
||||
- **Self-contained**: Each tool has a single, clear purpose
|
||||
- **Robust to error**: Handle edge cases gracefully
|
||||
- **Extremely clear**: Intended use is unambiguous
|
||||
- **Token-efficient**: Returns relevant information without bloat
|
||||
- **Descriptive parameters**: Unambiguous input names (e.g., `user_id` not `user`)
|
||||
|
||||
### Critical Rule
|
||||
**If a human engineer can't definitively say which tool to use in a given situation, an AI agent can't be expected to do better.**
|
||||
|
||||
### Common Failure Modes to Avoid
|
||||
- Bloated tool sets covering too much functionality
|
||||
- Tools with overlapping purposes
|
||||
- Ambiguous decision points about which tool to use
|
||||
|
||||
---
|
||||
|
||||
## Examples: Diverse, Not Exhaustive
|
||||
|
||||
**Do** ✅
|
||||
- Curate a set of diverse, canonical examples
|
||||
- Show expected behavior effectively
|
||||
- Think "pictures worth a thousand words"
|
||||
|
||||
**Don't** ❌
|
||||
- Stuff in a laundry list of edge cases
|
||||
- Try to articulate every possible rule
|
||||
- Overwhelm with exhaustive scenarios
|
||||
|
||||
---
|
||||
|
||||
## Context Retrieval Strategies
|
||||
|
||||
### Just-In-Time Context (Recommended for Agents)
|
||||
**Approach**: Maintain lightweight identifiers (file paths, queries, links) and dynamically load data at runtime
|
||||
|
||||
**Benefits**:
|
||||
- Avoids context pollution
|
||||
- Enables progressive disclosure
|
||||
- Mirrors human cognition (we don't memorize everything)
|
||||
- Leverages metadata (file names, folder structure, timestamps)
|
||||
- Agents discover context incrementally
|
||||
|
||||
**Trade-offs**:
|
||||
- Slower than pre-computed retrieval
|
||||
- Requires proper tool guidance to avoid dead-ends
|
||||
|
||||
### Pre-Inference Retrieval (Traditional RAG)
|
||||
**Approach**: Use embedding-based retrieval to surface context before inference
|
||||
|
||||
**When to Use**: Static content that won't change during interaction
|
||||
|
||||
### Hybrid Strategy (Best of Both)
|
||||
**Approach**: Retrieve some data upfront, enable autonomous exploration as needed
|
||||
|
||||
**Example**: Claude Code loads CLAUDE.md files upfront, uses glob/grep for just-in-time retrieval
|
||||
|
||||
**Rule of Thumb**: "Do the simplest thing that works"
|
||||
|
||||
---
|
||||
|
||||
## Long-Horizon Tasks: Three Techniques
|
||||
|
||||
### 1. Compaction
|
||||
**What**: Summarize conversation nearing context limit, reinitiate with summary
|
||||
|
||||
**Implementation**:
|
||||
- Pass message history to model for compression
|
||||
- Preserve critical details (architectural decisions, bugs, implementation)
|
||||
- Discard redundant outputs
|
||||
- Continue with compressed context + recently accessed files
|
||||
|
||||
**Tuning Process**:
|
||||
1. **First**: Maximize recall (capture all relevant information)
|
||||
2. **Then**: Improve precision (eliminate superfluous content)
|
||||
|
||||
**Low-Hanging Fruit**: Clear old tool calls and results
|
||||
|
||||
**Best For**: Tasks requiring extensive back-and-forth
|
||||
|
||||
### 2. Structured Note-Taking (Agentic Memory)
|
||||
**What**: Agent writes notes persisted outside context window, retrieved later
|
||||
|
||||
**Examples**:
|
||||
- To-do lists
|
||||
- NOTES.md files
|
||||
- Game state tracking (Pokémon example: tracking 1,234 steps of training)
|
||||
- Project progress logs
|
||||
|
||||
**Benefits**:
|
||||
- Persistent memory with minimal overhead
|
||||
- Maintains critical context across tool calls
|
||||
- Enables multi-hour coherent strategies
|
||||
|
||||
**Best For**: Iterative development with clear milestones
|
||||
|
||||
### 3. Sub-Agent Architectures
|
||||
**What**: Specialized sub-agents handle focused tasks with clean context windows
|
||||
|
||||
**How It Works**:
|
||||
- Main agent coordinates high-level plan
|
||||
- Sub-agents perform deep technical work
|
||||
- Sub-agents explore extensively (tens of thousands of tokens)
|
||||
- Return condensed summaries (1,000-2,000 tokens)
|
||||
|
||||
**Benefits**:
|
||||
- Clear separation of concerns
|
||||
- Parallel exploration
|
||||
- Detailed context remains isolated
|
||||
|
||||
**Best For**: Complex research and analysis tasks
|
||||
|
||||
---
|
||||
|
||||
## Quick Decision Framework
|
||||
|
||||
| Scenario | Recommended Approach |
|
||||
|----------|---------------------|
|
||||
| Static content | Pre-inference retrieval or hybrid |
|
||||
| Dynamic exploration needed | Just-in-time context |
|
||||
| Extended back-and-forth | Compaction |
|
||||
| Iterative development | Structured note-taking |
|
||||
| Complex research | Sub-agent architectures |
|
||||
| Rapid model improvement | "Do the simplest thing that works" |
|
||||
|
||||
---
|
||||
|
||||
## Key Takeaways
|
||||
|
||||
1. **Context is finite**: Treat it as a precious resource with an attention budget
|
||||
2. **Think holistically**: Consider the entire state available to the LLM
|
||||
3. **Stay minimal**: More context isn't always better
|
||||
4. **Be iterative**: Context curation happens each time you pass to the model
|
||||
5. **Design for autonomy**: As models improve, let them act intelligently
|
||||
6. **Start simple**: Test with minimal setup, add based on failure modes
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns to Avoid
|
||||
|
||||
- ❌ Cramming everything into prompts
|
||||
- ❌ Creating brittle if-else logic
|
||||
- ❌ Building bloated tool sets
|
||||
- ❌ Stuffing exhaustive edge cases as examples
|
||||
- ❌ Assuming larger context windows solve everything
|
||||
- ❌ Ignoring context pollution over long interactions
|
||||
|
||||
---
|
||||
|
||||
## Remember
|
||||
|
||||
> "Even as models continue to improve, the challenge of maintaining coherence across extended interactions will remain central to building more effective agents."
|
||||
|
||||
Context engineering will evolve, but the core principle stays the same: **optimize signal-to-noise ratio in your token budget**.
|
||||
|
||||
---
|
||||
|
||||
*Based on Anthropic's "Effective context engineering for AI agents" (September 2025)*
|
||||
@@ -0,0 +1,668 @@
|
||||
---
|
||||
title: "Development"
|
||||
description: "Build from source, run tests, and contribute to Claude-Mem"
|
||||
---
|
||||
|
||||
# Development Guide
|
||||
|
||||
## Building from Source
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Node.js 18.0.0 or higher
|
||||
- npm (comes with Node.js)
|
||||
- Git
|
||||
|
||||
### Clone and Build
|
||||
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://github.com/thedotmack/claude-mem.git
|
||||
cd claude-mem
|
||||
|
||||
# Install dependencies
|
||||
npm install
|
||||
|
||||
# Build all components
|
||||
npm run build
|
||||
```
|
||||
|
||||
### Build Process
|
||||
|
||||
The build process uses esbuild to compile TypeScript:
|
||||
|
||||
1. Compiles TypeScript to JavaScript
|
||||
2. Creates standalone executables for each hook in `plugin/scripts/`
|
||||
3. Bundles MCP search server to `plugin/scripts/search-server.mjs`
|
||||
4. Bundles worker service to `plugin/scripts/worker-service.cjs`
|
||||
5. Bundles web viewer UI to `plugin/ui/viewer.html`
|
||||
|
||||
**Build Output**:
|
||||
- Hook executables: `*-hook.js` (ESM format)
|
||||
- Smart installer: `smart-install.js` (ESM format)
|
||||
- Worker service: `worker-service.cjs` (CJS format)
|
||||
- Search server: `search-server.mjs` (ESM format)
|
||||
- Viewer UI: `viewer.html` (self-contained HTML bundle)
|
||||
|
||||
### Build Scripts
|
||||
|
||||
```bash
|
||||
# Build everything
|
||||
npm run build
|
||||
|
||||
# Build only hooks
|
||||
npm run build:hooks
|
||||
|
||||
# The build script is defined in scripts/build-hooks.js
|
||||
```
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### 1. Make Changes
|
||||
|
||||
Edit TypeScript source files in `src/`:
|
||||
|
||||
```
|
||||
src/
|
||||
├── hooks/ # Hook implementations (entry points + logic)
|
||||
├── services/ # Worker service and database
|
||||
├── servers/ # MCP search server
|
||||
├── sdk/ # Claude Agent SDK integration
|
||||
├── shared/ # Shared utilities
|
||||
├── ui/
|
||||
│ └── viewer/ # React web viewer UI components
|
||||
└── utils/ # General utilities
|
||||
```
|
||||
|
||||
### 2. Build
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
### 3. Test
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
npm test
|
||||
|
||||
# Test specific file
|
||||
node --test tests/session-lifecycle.test.ts
|
||||
|
||||
# Test context injection
|
||||
npm run test:context
|
||||
|
||||
# Verbose context test
|
||||
npm run test:context:verbose
|
||||
```
|
||||
|
||||
### 4. Manual Testing
|
||||
|
||||
```bash
|
||||
# Start worker manually
|
||||
npm run worker:start
|
||||
|
||||
# Check worker status
|
||||
npm run worker:status
|
||||
|
||||
# View logs
|
||||
npm run worker:logs
|
||||
|
||||
# Test hooks manually
|
||||
echo '{"session_id":"test-123","cwd":"'$(pwd)'","source":"startup"}' | node plugin/scripts/context-hook.js
|
||||
```
|
||||
|
||||
### 5. Iterate
|
||||
|
||||
Repeat steps 1-4 until your changes work as expected.
|
||||
|
||||
## Viewer UI Development
|
||||
|
||||
### Working with the React Viewer
|
||||
|
||||
The web viewer UI is a React application built into a self-contained HTML bundle.
|
||||
|
||||
**Location**: `src/ui/viewer/`
|
||||
|
||||
**Structure**:
|
||||
```
|
||||
src/ui/viewer/
|
||||
├── index.tsx # Entry point
|
||||
├── App.tsx # Main application component
|
||||
├── components/ # React components
|
||||
│ ├── Header.tsx # Header with logo and actions
|
||||
│ ├── Sidebar.tsx # Project filter sidebar
|
||||
│ ├── Feed.tsx # Main feed with infinite scroll
|
||||
│ ├── cards/ # Card components
|
||||
│ │ ├── ObservationCard.tsx
|
||||
│ │ ├── PromptCard.tsx
|
||||
│ │ ├── SummaryCard.tsx
|
||||
│ │ └── SkeletonCard.tsx
|
||||
├── hooks/ # Custom React hooks
|
||||
│ ├── useSSE.ts # Server-Sent Events connection
|
||||
│ ├── usePagination.ts # Infinite scroll pagination
|
||||
│ ├── useSettings.ts # Settings persistence
|
||||
│ └── useStats.ts # Database statistics
|
||||
├── utils/ # Utilities
|
||||
│ ├── constants.ts # Constants (API URLs, etc.)
|
||||
│ ├── formatters.ts # Date/time formatting
|
||||
│ └── merge.ts # Data merging and deduplication
|
||||
└── assets/ # Static assets (fonts, logos)
|
||||
```
|
||||
|
||||
### Building Viewer UI
|
||||
|
||||
```bash
|
||||
# Build everything including viewer
|
||||
npm run build
|
||||
|
||||
# The viewer is built to plugin/ui/viewer.html
|
||||
# It's a self-contained HTML file with inlined JS and CSS
|
||||
```
|
||||
|
||||
### Testing Viewer Changes
|
||||
|
||||
1. Make changes to React components in `src/ui/viewer/`
|
||||
2. Build: `npm run build`
|
||||
3. Sync to installed plugin: `npm run sync-marketplace`
|
||||
4. Restart worker: `npm run worker:restart`
|
||||
5. Refresh browser at http://localhost:37777
|
||||
|
||||
**Hot Reload**: Not currently supported. Full rebuild + restart required for changes.
|
||||
|
||||
### Adding New Viewer Features
|
||||
|
||||
**Example: Adding a new card type**
|
||||
|
||||
1. Create component in `src/ui/viewer/components/cards/YourCard.tsx`:
|
||||
|
||||
```tsx
|
||||
import React from 'react';
|
||||
|
||||
export interface YourCardProps {
|
||||
// Your data structure
|
||||
}
|
||||
|
||||
export const YourCard: React.FC<YourCardProps> = ({ ... }) => {
|
||||
return (
|
||||
<div className="card">
|
||||
{/* Your UI */}
|
||||
</div>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
2. Import and use in `Feed.tsx`:
|
||||
|
||||
```tsx
|
||||
import { YourCard } from './cards/YourCard';
|
||||
|
||||
// In render logic:
|
||||
{item.type === 'your_type' && <YourCard {...item} />}
|
||||
```
|
||||
|
||||
3. Update types if needed in `src/ui/viewer/types.ts`
|
||||
|
||||
4. Rebuild and test
|
||||
|
||||
### Viewer UI Architecture
|
||||
|
||||
**Data Flow**:
|
||||
1. Worker service exposes HTTP + SSE endpoints
|
||||
2. React app fetches initial data via HTTP (paginated)
|
||||
3. SSE connection provides real-time updates
|
||||
4. Custom hooks handle state management and data merging
|
||||
5. Components render cards based on item type
|
||||
|
||||
**Key Patterns**:
|
||||
- **Infinite Scroll**: `usePagination` hook with Intersection Observer
|
||||
- **Real-Time Updates**: `useSSE` hook with auto-reconnection
|
||||
- **Deduplication**: `merge.ts` utilities prevent duplicate items
|
||||
- **Settings Persistence**: `useSettings` hook with localStorage
|
||||
- **Theme Support**: CSS variables with light/dark/system themes
|
||||
|
||||
## Adding New Features
|
||||
|
||||
### Adding a New Hook
|
||||
|
||||
1. Create hook implementation in `src/hooks/your-hook.ts`:
|
||||
|
||||
```typescript
|
||||
#!/usr/bin/env node
|
||||
import { readStdin } from '../shared/stdin';
|
||||
|
||||
async function main() {
|
||||
const input = await readStdin();
|
||||
|
||||
// Hook implementation
|
||||
const result = {
|
||||
hookSpecificOutput: 'Optional output'
|
||||
};
|
||||
|
||||
console.log(JSON.stringify(result));
|
||||
}
|
||||
|
||||
main().catch(console.error);
|
||||
```
|
||||
|
||||
**Note**: As of v4.3.1, hooks are self-contained files. The shebang will be added automatically by esbuild during the build process.
|
||||
|
||||
2. Add to `plugin/hooks/hooks.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"YourHook": [{
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/your-hook.js",
|
||||
"timeout": 120
|
||||
}]
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
4. Rebuild:
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
### Modifying Database Schema
|
||||
|
||||
1. Add migration to `src/services/sqlite/migrations.ts`:
|
||||
|
||||
```typescript
|
||||
export const migration011: Migration = {
|
||||
version: 11,
|
||||
up: (db: Database) => {
|
||||
db.run(`
|
||||
ALTER TABLE observations ADD COLUMN new_field TEXT;
|
||||
`);
|
||||
},
|
||||
down: (db: Database) => {
|
||||
// Optional: define rollback
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
2. Update types in `src/services/sqlite/types.ts`:
|
||||
|
||||
```typescript
|
||||
export interface Observation {
|
||||
// ... existing fields
|
||||
new_field?: string;
|
||||
}
|
||||
```
|
||||
|
||||
3. Update database methods in `src/services/sqlite/SessionStore.ts`:
|
||||
|
||||
```typescript
|
||||
createObservation(obs: Observation) {
|
||||
// Include new_field in INSERT
|
||||
}
|
||||
```
|
||||
|
||||
4. Test migration:
|
||||
|
||||
```bash
|
||||
# Backup database first!
|
||||
cp ~/.claude-mem/claude-mem.db ~/.claude-mem/claude-mem.db.backup
|
||||
|
||||
# Run tests
|
||||
npm test
|
||||
```
|
||||
|
||||
### Extending SDK Prompts
|
||||
|
||||
1. Modify prompts in `src/sdk/prompts.ts`:
|
||||
|
||||
```typescript
|
||||
export function buildObservationPrompt(observation: Observation): string {
|
||||
return `
|
||||
<observation>
|
||||
<!-- Add new XML structure -->
|
||||
</observation>
|
||||
`;
|
||||
}
|
||||
```
|
||||
|
||||
2. Update parser in `src/sdk/parser.ts`:
|
||||
|
||||
```typescript
|
||||
export function parseObservation(xml: string): ParsedObservation {
|
||||
// Parse new XML fields
|
||||
}
|
||||
```
|
||||
|
||||
3. Test:
|
||||
|
||||
```bash
|
||||
npm test
|
||||
```
|
||||
|
||||
### Adding MCP Search Tools
|
||||
|
||||
1. Add tool definition in `src/servers/search-server.ts`:
|
||||
|
||||
```typescript
|
||||
server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
||||
if (request.params.name === 'your_new_tool') {
|
||||
// Implement tool logic
|
||||
const results = await search.yourNewSearch(params);
|
||||
return formatResults(results);
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
2. Add search method in `src/services/sqlite/SessionSearch.ts`:
|
||||
|
||||
```typescript
|
||||
yourNewSearch(params: YourParams): SearchResult[] {
|
||||
// Implement FTS5 search
|
||||
}
|
||||
```
|
||||
|
||||
3. Rebuild and test:
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
npm test
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# All tests
|
||||
npm test
|
||||
|
||||
# Specific test file
|
||||
node --test tests/your-test.test.ts
|
||||
|
||||
# With coverage (if configured)
|
||||
npm test -- --coverage
|
||||
```
|
||||
|
||||
### Writing Tests
|
||||
|
||||
Create test files in `tests/`:
|
||||
|
||||
```typescript
|
||||
import { describe, it } from 'node:test';
|
||||
import assert from 'node:assert';
|
||||
|
||||
describe('YourFeature', () => {
|
||||
it('should do something', () => {
|
||||
// Test implementation
|
||||
assert.strictEqual(result, expected);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Test Database
|
||||
|
||||
Use a separate test database:
|
||||
|
||||
```typescript
|
||||
import { SessionStore } from '../src/services/sqlite/SessionStore';
|
||||
|
||||
const store = new SessionStore(':memory:'); // In-memory database
|
||||
```
|
||||
|
||||
## Code Style
|
||||
|
||||
### TypeScript Guidelines
|
||||
|
||||
- Use TypeScript strict mode
|
||||
- Define interfaces for all data structures
|
||||
- Use async/await for asynchronous code
|
||||
- Handle errors explicitly
|
||||
- Add JSDoc comments for public APIs
|
||||
|
||||
### Formatting
|
||||
|
||||
- Follow existing code formatting
|
||||
- Use 2-space indentation
|
||||
- Use single quotes for strings
|
||||
- Add trailing commas in objects/arrays
|
||||
|
||||
### Example
|
||||
|
||||
```typescript
|
||||
/**
|
||||
* Create a new observation in the database
|
||||
*/
|
||||
export async function createObservation(
|
||||
obs: Observation
|
||||
): Promise<number> {
|
||||
try {
|
||||
const result = await db.insert('observations', {
|
||||
session_id: obs.session_id,
|
||||
tool_name: obs.tool_name,
|
||||
// ...
|
||||
});
|
||||
return result.id;
|
||||
} catch (error) {
|
||||
logger.error('Failed to create observation', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Debugging
|
||||
|
||||
### Enable Debug Logging
|
||||
|
||||
```bash
|
||||
export DEBUG=claude-mem:*
|
||||
npm run worker:restart
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
### Inspect Database
|
||||
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db
|
||||
|
||||
# View schema
|
||||
.schema observations
|
||||
|
||||
# Query data
|
||||
SELECT * FROM observations LIMIT 10;
|
||||
```
|
||||
|
||||
### Trace Observations
|
||||
|
||||
Use correlation IDs to trace observations through the pipeline:
|
||||
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db
|
||||
SELECT correlation_id, tool_name, created_at
|
||||
FROM observations
|
||||
WHERE session_id = 'YOUR_SESSION_ID'
|
||||
ORDER BY created_at;
|
||||
```
|
||||
|
||||
### Debug Hooks
|
||||
|
||||
Run hooks manually with test input:
|
||||
|
||||
```bash
|
||||
# Test context hook
|
||||
echo '{"session_id":"test-123","cwd":"'$(pwd)'","source":"startup"}' | node plugin/scripts/context-hook.js
|
||||
|
||||
# Test new hook
|
||||
echo '{"session_id":"test-123","cwd":"'$(pwd)'","prompt":"test"}' | node plugin/scripts/new-hook.js
|
||||
```
|
||||
|
||||
## Publishing
|
||||
|
||||
### NPM Publishing
|
||||
|
||||
```bash
|
||||
# Update version in package.json
|
||||
npm version patch # or minor, or major
|
||||
|
||||
# Build
|
||||
npm run build
|
||||
|
||||
# Publish to NPM
|
||||
npm run release
|
||||
```
|
||||
|
||||
The `release` script:
|
||||
1. Runs tests
|
||||
2. Builds all components
|
||||
3. Publishes to NPM registry
|
||||
|
||||
### Creating a Release
|
||||
|
||||
1. Update version in `package.json`
|
||||
2. Update `CHANGELOG.md`
|
||||
3. Commit changes
|
||||
4. Create git tag
|
||||
5. Push to GitHub
|
||||
6. Publish to NPM
|
||||
|
||||
```bash
|
||||
# Use the version bump skill (recommended as of v4.3.0)
|
||||
# In Claude Code, run: /skill version-bump
|
||||
# This updates package.json, marketplace.json, and CLAUDE.md
|
||||
|
||||
# Or manually:
|
||||
npm version 4.3.2
|
||||
|
||||
# Update changelog
|
||||
# Edit CHANGELOG.md manually
|
||||
|
||||
# Commit
|
||||
git add .
|
||||
git commit -m "chore: Release v4.3.2"
|
||||
|
||||
# Tag
|
||||
git tag v4.3.2
|
||||
|
||||
# Push
|
||||
git push origin main --tags
|
||||
|
||||
# Publish to NPM
|
||||
npm run release
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
### Contribution Workflow
|
||||
|
||||
1. Fork the repository
|
||||
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
|
||||
3. Make your changes
|
||||
4. Write tests
|
||||
5. Update documentation
|
||||
6. Commit your changes (`git commit -m 'Add amazing feature'`)
|
||||
7. Push to the branch (`git push origin feature/amazing-feature`)
|
||||
8. Open a Pull Request
|
||||
|
||||
### Pull Request Guidelines
|
||||
|
||||
- **Clear title**: Describe what the PR does
|
||||
- **Description**: Explain why the change is needed
|
||||
- **Tests**: Include tests for new features
|
||||
- **Documentation**: Update docs as needed
|
||||
- **Changelog**: Add entry to CHANGELOG.md
|
||||
- **Commits**: Use clear, descriptive commit messages
|
||||
|
||||
### Code Review Process
|
||||
|
||||
1. Automated tests must pass
|
||||
2. Code review by maintainer
|
||||
3. Address feedback
|
||||
4. Final approval
|
||||
5. Merge to main
|
||||
|
||||
## Development Tools
|
||||
|
||||
### Recommended VSCode Extensions
|
||||
|
||||
- TypeScript
|
||||
- ESLint
|
||||
- Prettier
|
||||
- SQLite Viewer
|
||||
|
||||
### Useful Commands
|
||||
|
||||
```bash
|
||||
# Check TypeScript types
|
||||
npx tsc --noEmit
|
||||
|
||||
# Lint code (if configured)
|
||||
npm run lint
|
||||
|
||||
# Format code (if configured)
|
||||
npm run format
|
||||
|
||||
# Clean build artifacts
|
||||
rm -rf plugin/scripts/*.js plugin/scripts/*.cjs
|
||||
```
|
||||
|
||||
## Troubleshooting Development
|
||||
|
||||
### Build Fails
|
||||
|
||||
1. Clean node_modules:
|
||||
```bash
|
||||
rm -rf node_modules
|
||||
npm install
|
||||
```
|
||||
|
||||
2. Check Node.js version:
|
||||
```bash
|
||||
node --version # Should be >= 18.0.0
|
||||
```
|
||||
|
||||
3. Check for syntax errors:
|
||||
```bash
|
||||
npx tsc --noEmit
|
||||
```
|
||||
|
||||
### Tests Fail
|
||||
|
||||
1. Check database:
|
||||
```bash
|
||||
rm ~/.claude-mem/claude-mem.db
|
||||
npm test
|
||||
```
|
||||
|
||||
2. Check worker status:
|
||||
```bash
|
||||
npm run worker:status
|
||||
```
|
||||
|
||||
3. View logs:
|
||||
```bash
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
### Worker Won't Start
|
||||
|
||||
1. Kill existing process:
|
||||
```bash
|
||||
pm2 delete claude-mem-worker
|
||||
```
|
||||
|
||||
2. Check port:
|
||||
```bash
|
||||
lsof -i :37777
|
||||
```
|
||||
|
||||
3. Try custom port:
|
||||
```bash
|
||||
export CLAUDE_MEM_WORKER_PORT=38000
|
||||
npm run worker:start
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Architecture Overview](architecture/overview) - Understand the system
|
||||
- [Configuration](configuration) - Customize Claude-Mem
|
||||
- [Troubleshooting](troubleshooting) - Common issues
|
||||
@@ -0,0 +1,125 @@
|
||||
{
|
||||
"$schema": "https://mintlify.com/schema.json",
|
||||
"name": "Claude-Mem",
|
||||
"description": "Persistent memory compression system that preserves context across Claude Code sessions",
|
||||
"theme": "mint",
|
||||
"favicon": "/claude-mem-logomark.webp",
|
||||
"logo": {
|
||||
"light": "/claude-mem-logo-for-light-mode.webp",
|
||||
"dark": "/claude-mem-logo-for-dark-mode.webp",
|
||||
"href": "https://github.com/thedotmack/claude-mem"
|
||||
},
|
||||
"colors": {
|
||||
"primary": "#3B82F6",
|
||||
"light": "#EFF6FF",
|
||||
"dark": "#1E40AF"
|
||||
},
|
||||
"navbar": {
|
||||
"links": [
|
||||
{
|
||||
"label": "GitHub",
|
||||
"href": "https://github.com/thedotmack/claude-mem"
|
||||
}
|
||||
],
|
||||
"primary": {
|
||||
"type": "button",
|
||||
"label": "Install",
|
||||
"href": "https://github.com/thedotmack/claude-mem#quick-start"
|
||||
}
|
||||
},
|
||||
"navigation": {
|
||||
"groups": [
|
||||
{
|
||||
"group": "Get Started",
|
||||
"icon": "rocket",
|
||||
"pages": [
|
||||
"introduction",
|
||||
"installation",
|
||||
"usage/getting-started",
|
||||
"usage/search-tools"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Best Practices",
|
||||
"icon": "lightbulb",
|
||||
"pages": [
|
||||
"context-engineering",
|
||||
"progressive-disclosure"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Configuration & Development",
|
||||
"icon": "gear",
|
||||
"pages": [
|
||||
"configuration",
|
||||
"development",
|
||||
"troubleshooting"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Architecture",
|
||||
"icon": "diagram-project",
|
||||
"pages": [
|
||||
"architecture/overview",
|
||||
"architecture-evolution",
|
||||
"hooks-architecture",
|
||||
"architecture/hooks",
|
||||
"architecture/worker-service",
|
||||
"architecture/database",
|
||||
"architecture/search-architecture"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"footer": {
|
||||
"socials": {
|
||||
"github": "https://github.com/thedotmack/claude-mem"
|
||||
},
|
||||
"links": [
|
||||
{
|
||||
"header": "Resources",
|
||||
"items": [
|
||||
{
|
||||
"label": "Documentation",
|
||||
"href": "https://github.com/thedotmack/claude-mem"
|
||||
},
|
||||
{
|
||||
"label": "Issues",
|
||||
"href": "https://github.com/thedotmack/claude-mem/issues"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"header": "Legal",
|
||||
"items": [
|
||||
{
|
||||
"label": "License (AGPL-3.0)",
|
||||
"href": "https://github.com/thedotmack/claude-mem/blob/main/LICENSE"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"seo": {
|
||||
"indexing": "all",
|
||||
"metatags": {
|
||||
"og:type": "website",
|
||||
"og:site_name": "Claude-Mem Documentation",
|
||||
"og:description": "Persistent memory compression system that preserves context across Claude Code sessions"
|
||||
}
|
||||
},
|
||||
"contextual": {
|
||||
"options": [
|
||||
"copy",
|
||||
"view",
|
||||
"chatgpt",
|
||||
"claude",
|
||||
"cursor"
|
||||
]
|
||||
},
|
||||
"integrations": {
|
||||
"telemetry": {
|
||||
"enabled": false
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,867 @@
|
||||
# How Claude-Mem Uses Hooks: A Lifecycle-Driven Architecture
|
||||
|
||||
## Core Principle
|
||||
**Observe the main Claude Code session from the outside, process observations in the background, inject context at the right time.**
|
||||
|
||||
---
|
||||
|
||||
## The Big Picture
|
||||
|
||||
Claude-Mem is fundamentally a **hook-driven system**. Every piece of functionality happens in response to lifecycle events:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ CLAUDE CODE SESSION │
|
||||
│ (Main session - user interacting with Claude) │
|
||||
│ │
|
||||
│ SessionStart → UserPromptSubmit → Tool Use → Stop │
|
||||
│ ↓ ↓ ↓ ↓ ↓ ↓ │
|
||||
│ [3 Hooks] [Hook] [Hook] [Hook] │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓ ↓ ↓ ↓ ↓ ↓
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ CLAUDE-MEM SYSTEM │
|
||||
│ │
|
||||
│ Smart Context User New Obs │
|
||||
│ Install Inject Message Session Capture │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Key insight:** Claude-Mem doesn't interrupt or modify Claude Code's behavior. It observes from the outside and provides value through lifecycle hooks.
|
||||
|
||||
---
|
||||
|
||||
## Why Hooks?
|
||||
|
||||
### The Non-Invasive Requirement
|
||||
|
||||
Claude-Mem had several architectural constraints:
|
||||
|
||||
1. **Can't modify Claude Code**: It's a closed-source binary
|
||||
2. **Must be fast**: Can't slow down the main session
|
||||
3. **Must be reliable**: Can't break Claude Code if it fails
|
||||
4. **Must be portable**: Works on any project without configuration
|
||||
|
||||
**Solution:** External command hooks configured via settings.json
|
||||
|
||||
### The Hook System Advantage
|
||||
|
||||
Claude Code's hook system provides exactly what we need:
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Lifecycle Events" icon="clock">
|
||||
SessionStart, UserPromptSubmit, PostToolUse, Stop
|
||||
</Card>
|
||||
|
||||
<Card title="Non-Blocking" icon="forward">
|
||||
Hooks run in parallel, don't wait for completion
|
||||
</Card>
|
||||
|
||||
<Card title="Context Injection" icon="upload">
|
||||
SessionStart and UserPromptSubmit can add context
|
||||
</Card>
|
||||
|
||||
<Card title="Tool Observation" icon="eye">
|
||||
PostToolUse sees all tool inputs and outputs
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
---
|
||||
|
||||
## The Seven Hook Scripts
|
||||
|
||||
Claude-Mem uses 7 hook scripts across 5 lifecycle events. SessionStart runs 3 hooks in sequence.
|
||||
|
||||
### Hook 1: SessionStart - Smart Install
|
||||
|
||||
**Purpose:** Intelligently manage dependencies and start worker service
|
||||
|
||||
**When:** Claude Code starts (startup, clear, or compact)
|
||||
|
||||
**What it does:**
|
||||
1. Checks if dependencies need installation (version marker)
|
||||
2. Only runs `npm install` when necessary:
|
||||
- First-time installation
|
||||
- Version changed in package.json
|
||||
- Critical dependency missing (better-sqlite3)
|
||||
3. Provides Windows-specific error messages
|
||||
4. Starts PM2 worker service
|
||||
|
||||
**Configuration:**
|
||||
```json
|
||||
{
|
||||
"hooks": {
|
||||
"SessionStart": [{
|
||||
"matcher": "startup|clear|compact",
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "node \"${CLAUDE_PLUGIN_ROOT}/../scripts/smart-install.js\" && node ${CLAUDE_PLUGIN_ROOT}/scripts/context-hook.js",
|
||||
"timeout": 300
|
||||
}]
|
||||
}]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key Features:**
|
||||
- ✅ Version caching (`.install-version` file)
|
||||
- ✅ Fast when already installed (~10ms vs 2-5 seconds)
|
||||
- ✅ Cross-platform compatible
|
||||
- ✅ Helpful Windows error messages for build tools
|
||||
|
||||
**v5.0.3 Enhancement:** Smart caching eliminates redundant installs
|
||||
|
||||
**Source:** `scripts/smart-install.js`
|
||||
|
||||
---
|
||||
|
||||
### Hook 2: SessionStart - Context Injection
|
||||
|
||||
**Purpose:** Inject relevant context from previous sessions
|
||||
|
||||
**When:** Claude Code starts (runs after smart-install)
|
||||
|
||||
**What it does:**
|
||||
1. Extracts project name from current working directory
|
||||
2. Queries SQLite for recent session summaries (last 10)
|
||||
3. Queries SQLite for recent observations (configurable, default 50)
|
||||
4. Formats as progressive disclosure index
|
||||
5. Outputs to stdout (automatically injected into context)
|
||||
|
||||
**Key decisions:**
|
||||
- ✅ Runs on startup, clear, and compact
|
||||
- ✅ 300-second timeout (allows for npm install if needed)
|
||||
- ✅ Progressive disclosure format (index, not full details)
|
||||
- ✅ Configurable observation count via `CLAUDE_MEM_CONTEXT_OBSERVATIONS`
|
||||
|
||||
**Output format:**
|
||||
```markdown
|
||||
# [claude-mem] recent context
|
||||
|
||||
**Legend:** 🎯 session-request | 🔴 gotcha | 🟡 problem-solution ...
|
||||
|
||||
### Oct 26, 2025
|
||||
|
||||
**General**
|
||||
| ID | Time | T | Title | Tokens |
|
||||
|----|------|---|-------|--------|
|
||||
| #2586 | 12:58 AM | 🔵 | Context hook file empty | ~51 |
|
||||
|
||||
*Use claude-mem MCP search to access full details*
|
||||
```
|
||||
|
||||
**Source:** `src/hooks/context-hook.ts` → `plugin/scripts/context-hook.js`
|
||||
|
||||
---
|
||||
|
||||
### Hook 3: SessionStart - User Message
|
||||
|
||||
**Purpose:** Display helpful user messages during first-time setup
|
||||
|
||||
**When:** Claude Code starts (runs after context-hook)
|
||||
|
||||
**What it does:**
|
||||
1. Checks if dependencies are installed
|
||||
2. Shows first-time setup message if needed
|
||||
3. Displays formatted context information with colors
|
||||
4. Shows link to viewer UI (http://localhost:37777)
|
||||
5. Exits with code 3 (informational, not error)
|
||||
|
||||
**Configuration:**
|
||||
```json
|
||||
{
|
||||
"hooks": {
|
||||
"SessionStart": [{
|
||||
"matcher": "startup|clear|compact",
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "node ${CLAUDE_PLUGIN_ROOT}/scripts/user-message-hook.js",
|
||||
"timeout": 10
|
||||
}]
|
||||
}]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Output Example:**
|
||||
```
|
||||
📝 Claude-Mem Context Loaded
|
||||
ℹ️ Note: This appears as stderr but is informational only
|
||||
|
||||
[Context details with colors...]
|
||||
|
||||
📺 Watch live in browser http://localhost:37777/ (New! v5.1)
|
||||
```
|
||||
|
||||
**Key Features:**
|
||||
- ✅ User-friendly first-time experience
|
||||
- ✅ Visual context display
|
||||
- ✅ Links to viewer UI
|
||||
- ✅ Non-intrusive (exit code 3)
|
||||
|
||||
**Source:** `plugin/scripts/user-message-hook.js` (minified)
|
||||
|
||||
---
|
||||
|
||||
### Hook 4: UserPromptSubmit (New Session Hook)
|
||||
|
||||
**Purpose:** Initialize session tracking when user submits a prompt
|
||||
|
||||
**When:** Before Claude processes the user's message
|
||||
|
||||
**What it does:**
|
||||
1. Reads user prompt and session ID from stdin
|
||||
2. Creates new session record in SQLite
|
||||
3. Saves raw user prompt for full-text search (v4.2.0+)
|
||||
4. Starts PM2 worker service if not running
|
||||
5. Returns immediately (non-blocking)
|
||||
|
||||
**Configuration:**
|
||||
```json
|
||||
{
|
||||
"hooks": {
|
||||
"UserPromptSubmit": [{
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "${CLAUDE_PLUGIN_ROOT}/scripts/new-hook.js"
|
||||
}]
|
||||
}]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key decisions:**
|
||||
- ✅ No matcher (runs for all prompts)
|
||||
- ✅ Creates session record immediately
|
||||
- ✅ Stores raw prompts for search (privacy note: local SQLite only)
|
||||
- ✅ Auto-starts worker service
|
||||
- ✅ Suppresses output (`suppressOutput: true`)
|
||||
|
||||
**Database operations:**
|
||||
```sql
|
||||
INSERT INTO sdk_sessions (claude_session_id, project, user_prompt, ...)
|
||||
VALUES (?, ?, ?, ...)
|
||||
|
||||
INSERT INTO user_prompts (session_id, prompt, prompt_number, ...)
|
||||
VALUES (?, ?, ?, ...)
|
||||
```
|
||||
|
||||
**Source:** `src/hooks/new-hook.ts` → `plugin/scripts/new-hook.js`
|
||||
|
||||
---
|
||||
|
||||
### Hook 5: PostToolUse (Save Observation Hook)
|
||||
|
||||
**Purpose:** Capture tool execution observations for later processing
|
||||
|
||||
**When:** Immediately after any tool completes successfully
|
||||
|
||||
**What it does:**
|
||||
1. Receives tool name, input, output from stdin
|
||||
2. Finds active session for current project
|
||||
3. Enqueues observation in observation_queue table
|
||||
4. Returns immediately (processing happens in worker)
|
||||
|
||||
**Configuration:**
|
||||
```json
|
||||
{
|
||||
"hooks": {
|
||||
"PostToolUse": [{
|
||||
"matcher": "*",
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "${CLAUDE_PLUGIN_ROOT}/scripts/save-hook.js"
|
||||
}]
|
||||
}]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key decisions:**
|
||||
- ✅ Matcher: `*` (captures all tools)
|
||||
- ✅ Non-blocking (just enqueues, doesn't process)
|
||||
- ✅ Worker processes observations asynchronously
|
||||
- ✅ Parallel execution safe (each hook gets own stdin)
|
||||
|
||||
**Database operations:**
|
||||
```sql
|
||||
INSERT INTO observation_queue (session_id, tool_name, tool_input, tool_output, ...)
|
||||
VALUES (?, ?, ?, ?, ...)
|
||||
```
|
||||
|
||||
**What gets queued:**
|
||||
```json
|
||||
{
|
||||
"session_id": "abc123",
|
||||
"tool_name": "Edit",
|
||||
"tool_input": {
|
||||
"file_path": "/path/to/file.ts",
|
||||
"old_string": "...",
|
||||
"new_string": "..."
|
||||
},
|
||||
"tool_output": {
|
||||
"success": true,
|
||||
"linesChanged": 5
|
||||
},
|
||||
"created_at_epoch": 1698765432
|
||||
}
|
||||
```
|
||||
|
||||
**Source:** `src/hooks/save-hook.ts` → `plugin/scripts/save-hook.js`
|
||||
|
||||
---
|
||||
|
||||
### Hook 6: Summary Hook (Mid-Session Checkpoint)
|
||||
|
||||
**Purpose:** Generate AI-powered session summaries during the session
|
||||
|
||||
**When:** Triggered programmatically by the worker service
|
||||
|
||||
**What it does:**
|
||||
1. Gathers session observations from database
|
||||
2. Sends to Claude Agent SDK for summarization
|
||||
3. Processes response and extracts structured summary
|
||||
4. Stores in session_summaries table
|
||||
|
||||
**Configuration:**
|
||||
```json
|
||||
{
|
||||
"hooks": {
|
||||
"Summary": [{
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "${CLAUDE_PLUGIN_ROOT}/scripts/summary-hook.js"
|
||||
}]
|
||||
}]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key decisions:**
|
||||
- ✅ Triggered by worker, not by Claude Code lifecycle
|
||||
- ✅ Multiple summaries per session (v4.2.0+)
|
||||
- ✅ Summaries are checkpoints, not endings
|
||||
- ✅ Uses Claude Agent SDK for AI compression
|
||||
|
||||
**Summary structure:**
|
||||
```xml
|
||||
<summary>
|
||||
<request>User's original request</request>
|
||||
<investigated>What was examined</investigated>
|
||||
<learned>Key discoveries</learned>
|
||||
<completed>Work finished</completed>
|
||||
<next_steps>Remaining tasks</next_steps>
|
||||
<files_read>
|
||||
<file>path/to/file1.ts</file>
|
||||
<file>path/to/file2.ts</file>
|
||||
</files_read>
|
||||
<files_modified>
|
||||
<file>path/to/file3.ts</file>
|
||||
</files_modified>
|
||||
<notes>Additional context</notes>
|
||||
</summary>
|
||||
```
|
||||
|
||||
**Source:** `src/hooks/summary-hook.ts` → `plugin/scripts/summary-hook.js`
|
||||
|
||||
---
|
||||
|
||||
### Hook 7: SessionEnd (Cleanup Hook)
|
||||
|
||||
**Purpose:** Mark sessions as completed when they end
|
||||
|
||||
**When:** Claude Code session ends (not on `/clear`)
|
||||
|
||||
**What it does:**
|
||||
1. Marks session as completed in database
|
||||
2. Allows worker to finish processing
|
||||
3. Performs graceful cleanup
|
||||
|
||||
**Configuration:**
|
||||
```json
|
||||
{
|
||||
"hooks": {
|
||||
"SessionEnd": [{
|
||||
"hooks": [{
|
||||
"type": "command",
|
||||
"command": "${CLAUDE_PLUGIN_ROOT}/scripts/cleanup-hook.js"
|
||||
}]
|
||||
}]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key decisions:**
|
||||
- ✅ Graceful completion (v4.1.0+)
|
||||
- ✅ No longer sends DELETE to workers
|
||||
- ✅ Skips cleanup on `/clear` commands
|
||||
- ✅ Preserves ongoing sessions
|
||||
|
||||
**Why graceful cleanup?**
|
||||
|
||||
**Old approach (v3):**
|
||||
```typescript
|
||||
// ❌ Aggressive cleanup
|
||||
SessionEnd → DELETE /worker/session → Worker stops immediately
|
||||
```
|
||||
|
||||
**Problems:**
|
||||
- Interrupted summary generation
|
||||
- Lost pending observations
|
||||
- Race conditions
|
||||
|
||||
**New approach (v4.1.0+):**
|
||||
```typescript
|
||||
// ✅ Graceful completion
|
||||
SessionEnd → UPDATE sessions SET completed_at = NOW()
|
||||
Worker sees completion → Finishes processing → Exits naturally
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Worker finishes important operations
|
||||
- Summaries complete successfully
|
||||
- Clean state transitions
|
||||
|
||||
**Source:** `src/hooks/cleanup-hook.ts` → `plugin/scripts/cleanup-hook.js`
|
||||
|
||||
---
|
||||
|
||||
## Hook Execution Flow
|
||||
|
||||
### Session Lifecycle
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User
|
||||
participant Claude
|
||||
participant Hooks
|
||||
participant Worker
|
||||
participant DB
|
||||
|
||||
User->>Claude: Start Claude Code
|
||||
Claude->>Hooks: SessionStart hook
|
||||
Hooks->>DB: Query recent context
|
||||
DB-->>Hooks: Session summaries + observations
|
||||
Hooks-->>Claude: Inject context
|
||||
Note over Claude: Context available for session
|
||||
|
||||
User->>Claude: Submit prompt
|
||||
Claude->>Hooks: UserPromptSubmit hook
|
||||
Hooks->>DB: Create session record
|
||||
Hooks->>Worker: Start worker (if not running)
|
||||
Worker-->>DB: Ready to process
|
||||
|
||||
Claude->>Claude: Execute tools
|
||||
Claude->>Hooks: PostToolUse (multiple times)
|
||||
Hooks->>DB: Queue observations
|
||||
Note over Worker: Polls queue, processes observations
|
||||
|
||||
Worker->>Worker: AI compression
|
||||
Worker->>DB: Store compressed observations
|
||||
Worker->>Hooks: Trigger summary hook
|
||||
Hooks->>DB: Store session summary
|
||||
|
||||
User->>Claude: Finish
|
||||
Claude->>Hooks: SessionEnd hook
|
||||
Hooks->>DB: Mark session complete
|
||||
Worker->>DB: Check completion
|
||||
Worker->>Worker: Finish processing
|
||||
Worker->>Worker: Exit gracefully
|
||||
```
|
||||
|
||||
### Hook Timing
|
||||
|
||||
| Event | Timing | Blocking | Timeout | Output Handling |
|
||||
|-------|--------|----------|---------|-----------------|
|
||||
| **SessionStart (smart-install)** | Before session | No | 300s | stderr (info) |
|
||||
| **SessionStart (context)** | Before session | No | 300s | stdout → context |
|
||||
| **SessionStart (user-message)** | Before session | No | 10s | stderr (info) |
|
||||
| **UserPromptSubmit** | Before processing | No | 120s | stdout → context |
|
||||
| **PostToolUse** | After tool | No | 120s | Transcript only |
|
||||
| **Summary** | Worker triggered | No | 120s | Database |
|
||||
| **SessionEnd** | On exit | No | 120s | Log only |
|
||||
|
||||
---
|
||||
|
||||
## The Worker Service Architecture
|
||||
|
||||
### Why a Background Worker?
|
||||
|
||||
**Problem:** Hooks must be fast (< 1 second)
|
||||
|
||||
**Reality:** AI compression takes 5-30 seconds per observation
|
||||
|
||||
**Solution:** Hooks enqueue observations, worker processes async
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ HOOK (Fast) │
|
||||
│ 1. Read stdin (< 1ms) │
|
||||
│ 2. Insert into queue (< 10ms) │
|
||||
│ 3. Return success (< 20ms total) │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓ (queue)
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ WORKER (Slow) │
|
||||
│ 1. Poll queue every 1s │
|
||||
│ 2. Process observation via Claude SDK (5-30s) │
|
||||
│ 3. Parse and store results │
|
||||
│ 4. Mark observation processed │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### PM2 Process Management
|
||||
|
||||
**Technology:** PM2 (process manager for Node.js)
|
||||
|
||||
**Why PM2:**
|
||||
- Auto-restart on failure
|
||||
- Log management
|
||||
- Process monitoring
|
||||
- Cross-platform (works on macOS, Linux, Windows)
|
||||
- No systemd/launchd needed
|
||||
|
||||
**Configuration:**
|
||||
```javascript
|
||||
// ecosystem.config.cjs
|
||||
module.exports = {
|
||||
apps: [{
|
||||
name: 'claude-mem-worker',
|
||||
script: './plugin/scripts/worker-service.cjs',
|
||||
instances: 1,
|
||||
autorestart: true,
|
||||
watch: false,
|
||||
max_memory_restart: '500M',
|
||||
env: {
|
||||
NODE_ENV: 'production',
|
||||
CLAUDE_MEM_WORKER_PORT: 37777
|
||||
}
|
||||
}]
|
||||
};
|
||||
```
|
||||
|
||||
**Worker lifecycle:**
|
||||
```bash
|
||||
# Started by new-hook (if not running)
|
||||
pm2 start ecosystem.config.cjs
|
||||
|
||||
# Status check
|
||||
pm2 status claude-mem-worker
|
||||
|
||||
# View logs
|
||||
pm2 logs claude-mem-worker
|
||||
|
||||
# Restart
|
||||
pm2 restart claude-mem-worker
|
||||
```
|
||||
|
||||
### Worker HTTP API
|
||||
|
||||
**Technology:** Express.js REST API on port 37777
|
||||
|
||||
**Endpoints:**
|
||||
|
||||
| Endpoint | Method | Purpose |
|
||||
|----------|--------|---------|
|
||||
| `/health` | GET | Health check |
|
||||
| `/sessions` | POST | Create session |
|
||||
| `/sessions/:id` | GET | Get session status |
|
||||
| `/sessions/:id` | PATCH | Update session |
|
||||
| `/observations` | POST | Enqueue observation |
|
||||
| `/observations/:id` | GET | Get observation |
|
||||
|
||||
**Why HTTP API?**
|
||||
- Language-agnostic (hooks can be any language)
|
||||
- Easy debugging (curl commands)
|
||||
- Standard error handling
|
||||
- Proper async handling
|
||||
|
||||
---
|
||||
|
||||
## Design Patterns
|
||||
|
||||
### Pattern 1: Fire-and-Forget Hooks
|
||||
|
||||
**Principle:** Hooks should return immediately, not wait for completion
|
||||
|
||||
```typescript
|
||||
// ❌ Bad: Hook waits for processing
|
||||
export async function saveHook(stdin: HookInput) {
|
||||
const observation = parseInput(stdin);
|
||||
await processObservation(observation); // BLOCKS!
|
||||
return success();
|
||||
}
|
||||
|
||||
// ✅ Good: Hook enqueues and returns
|
||||
export async function saveHook(stdin: HookInput) {
|
||||
const observation = parseInput(stdin);
|
||||
await enqueueObservation(observation); // Fast
|
||||
return success(); // Immediate
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern 2: Queue-Based Processing
|
||||
|
||||
**Principle:** Decouple capture from processing
|
||||
|
||||
```
|
||||
Hook (capture) → Queue (buffer) → Worker (process)
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Parallel hook execution safe
|
||||
- Worker failure doesn't affect hooks
|
||||
- Retry logic centralized
|
||||
- Backpressure handling
|
||||
|
||||
### Pattern 3: Graceful Degradation
|
||||
|
||||
**Principle:** Memory system failure shouldn't break Claude Code
|
||||
|
||||
```typescript
|
||||
try {
|
||||
await captureObservation();
|
||||
} catch (error) {
|
||||
// Log error, but don't throw
|
||||
console.error('Memory capture failed:', error);
|
||||
return { continue: true, suppressOutput: true };
|
||||
}
|
||||
```
|
||||
|
||||
**Failure modes:**
|
||||
- Database locked → Skip observation, log error
|
||||
- Worker crashed → Auto-restart via PM2
|
||||
- Network issue → Retry with exponential backoff
|
||||
- Disk full → Warn user, disable memory
|
||||
|
||||
### Pattern 4: Progressive Enhancement
|
||||
|
||||
**Principle:** Core functionality works without memory, memory enhances it
|
||||
|
||||
```
|
||||
Without memory: Claude Code works normally
|
||||
With memory: Claude Code + context from past sessions
|
||||
Memory broken: Falls back to working normally
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Hook Debugging
|
||||
|
||||
### Debug Mode
|
||||
|
||||
Enable detailed hook execution logs:
|
||||
|
||||
```bash
|
||||
claude --debug
|
||||
```
|
||||
|
||||
**Output:**
|
||||
```
|
||||
[DEBUG] Executing hooks for PostToolUse:Write
|
||||
[DEBUG] Getting matching hook commands for PostToolUse with query: Write
|
||||
[DEBUG] Found 1 hook matchers in settings
|
||||
[DEBUG] Matched 1 hooks for query "Write"
|
||||
[DEBUG] Found 1 hook commands to execute
|
||||
[DEBUG] Executing hook command: ${CLAUDE_PLUGIN_ROOT}/scripts/save-hook.js with timeout 60000ms
|
||||
[DEBUG] Hook command completed with status 0: {"continue":true,"suppressOutput":true}
|
||||
```
|
||||
|
||||
### Common Issues
|
||||
|
||||
<AccordionGroup>
|
||||
<Accordion title="Hook not executing">
|
||||
**Symptoms:** Hook command never runs
|
||||
|
||||
**Debugging:**
|
||||
1. Check `/hooks` menu - is hook registered?
|
||||
2. Verify matcher pattern (case-sensitive!)
|
||||
3. Test command manually: `echo '{}' | node save-hook.js`
|
||||
4. Check file permissions (executable?)
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Hook times out">
|
||||
**Symptoms:** Hook execution exceeds timeout
|
||||
|
||||
**Debugging:**
|
||||
1. Check timeout setting (default 60s)
|
||||
2. Identify slow operation (database? network?)
|
||||
3. Move slow operation to worker
|
||||
4. Increase timeout if necessary
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Context not injecting">
|
||||
**Symptoms:** SessionStart hook runs but context missing
|
||||
|
||||
**Debugging:**
|
||||
1. Check stdout (must be valid JSON or plain text)
|
||||
2. Verify no stderr output (pollutes JSON)
|
||||
3. Check exit code (must be 0)
|
||||
4. Look for npm install output (v4.3.1 fix)
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Observations not captured">
|
||||
**Symptoms:** PostToolUse hook runs but observations missing
|
||||
|
||||
**Debugging:**
|
||||
1. Check database: `sqlite3 ~/.claude-mem/claude-mem.db "SELECT * FROM observation_queue"`
|
||||
2. Verify session exists: `SELECT * FROM sdk_sessions`
|
||||
3. Check worker status: `pm2 status`
|
||||
4. View worker logs: `pm2 logs claude-mem-worker`
|
||||
</Accordion>
|
||||
</AccordionGroup>
|
||||
|
||||
### Testing Hooks Manually
|
||||
|
||||
```bash
|
||||
# Test context hook
|
||||
echo '{
|
||||
"session_id": "test123",
|
||||
"cwd": "/Users/alex/projects/my-app",
|
||||
"hook_event_name": "SessionStart",
|
||||
"source": "startup"
|
||||
}' | node plugin/scripts/context-hook.js
|
||||
|
||||
# Test save hook
|
||||
echo '{
|
||||
"session_id": "test123",
|
||||
"tool_name": "Edit",
|
||||
"tool_input": {"file_path": "test.ts"},
|
||||
"tool_output": {"success": true}
|
||||
}' | node plugin/scripts/save-hook.js
|
||||
|
||||
# Test with actual Claude Code
|
||||
claude --debug
|
||||
/hooks # View registered hooks
|
||||
# Submit prompt and watch debug output
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Hook Execution Time
|
||||
|
||||
**Target:** < 100ms per hook
|
||||
|
||||
**Actual measurements:**
|
||||
|
||||
| Hook | Average | p95 | p99 |
|
||||
|------|---------|-----|-----|
|
||||
| SessionStart (smart-install, cached) | 10ms | 20ms | 40ms |
|
||||
| SessionStart (smart-install, first run) | 2500ms | 5000ms | 8000ms |
|
||||
| SessionStart (context) | 45ms | 120ms | 250ms |
|
||||
| SessionStart (user-message) | 5ms | 10ms | 15ms |
|
||||
| UserPromptSubmit | 12ms | 25ms | 50ms |
|
||||
| PostToolUse | 8ms | 15ms | 30ms |
|
||||
| SessionEnd | 5ms | 10ms | 20ms |
|
||||
|
||||
**Why smart-install is sometimes slow:**
|
||||
- First-time: Full npm install (2-5 seconds)
|
||||
- Cached: Version check only (~10ms)
|
||||
- Version change: Full npm install + PM2 restart
|
||||
|
||||
**Optimization (v5.0.3):**
|
||||
- Version caching with `.install-version` marker
|
||||
- Only install on version change or missing deps
|
||||
- Windows-specific error messages with build tool help
|
||||
|
||||
### Database Performance
|
||||
|
||||
**Schema optimizations:**
|
||||
- Indexes on `project`, `created_at_epoch`, `claude_session_id`
|
||||
- FTS5 virtual tables for full-text search
|
||||
- WAL mode for concurrent reads/writes
|
||||
|
||||
**Query patterns:**
|
||||
```sql
|
||||
-- Fast: Uses index on (project, created_at_epoch)
|
||||
SELECT * FROM session_summaries
|
||||
WHERE project = ?
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT 10
|
||||
|
||||
-- Fast: Uses index on claude_session_id
|
||||
SELECT * FROM sdk_sessions
|
||||
WHERE claude_session_id = ?
|
||||
LIMIT 1
|
||||
|
||||
-- Fast: FTS5 full-text search
|
||||
SELECT * FROM observations_fts
|
||||
WHERE observations_fts MATCH ?
|
||||
ORDER BY rank
|
||||
LIMIT 20
|
||||
```
|
||||
|
||||
### Worker Throughput
|
||||
|
||||
**Bottleneck:** Claude API latency (5-30s per observation)
|
||||
|
||||
**Mitigation:**
|
||||
- Process observations sequentially (simpler, more predictable)
|
||||
- Skip low-value observations (TodoWrite, ListMcpResourcesTool)
|
||||
- Batch summaries (generate every N observations, not every observation)
|
||||
|
||||
**Future optimization:**
|
||||
- Parallel processing (multiple workers)
|
||||
- Smart batching (combine related observations)
|
||||
- Lazy summarization (summarize only when needed)
|
||||
|
||||
---
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Hook Command Safety
|
||||
|
||||
**Risk:** Hooks execute arbitrary commands with user permissions
|
||||
|
||||
**Mitigations:**
|
||||
1. **Frozen at startup:** Hook configuration captured at start, changes require review
|
||||
2. **User review required:** `/hooks` menu shows changes, requires approval
|
||||
3. **Plugin isolation:** `${CLAUDE_PLUGIN_ROOT}` prevents path traversal
|
||||
4. **Input validation:** Hooks validate stdin schema before processing
|
||||
|
||||
### Data Privacy
|
||||
|
||||
**What gets stored:**
|
||||
- User prompts (raw text) - v4.2.0+
|
||||
- Tool inputs and outputs
|
||||
- File paths read/modified
|
||||
- Session summaries
|
||||
|
||||
**Privacy guarantees:**
|
||||
- All data stored locally in `~/.claude-mem/claude-mem.db`
|
||||
- No cloud uploads (API calls only for AI compression)
|
||||
- SQLite file permissions: user-only read/write
|
||||
- No analytics or telemetry
|
||||
|
||||
### API Key Protection
|
||||
|
||||
**Configuration:**
|
||||
- Anthropic API key in `~/.anthropic/api_key` or `ANTHROPIC_API_KEY` env var
|
||||
- Worker inherits environment from Claude Code
|
||||
- Never logged or stored in database
|
||||
|
||||
---
|
||||
|
||||
## Key Takeaways
|
||||
|
||||
1. **Hooks are interfaces**: They define clean boundaries between systems
|
||||
2. **Non-blocking is critical**: Hooks must return fast, workers do the heavy lifting
|
||||
3. **Graceful degradation**: Memory system can fail without breaking Claude Code
|
||||
4. **Queue-based decoupling**: Capture and processing happen independently
|
||||
5. **Progressive disclosure**: Context injection uses index-first approach
|
||||
6. **Lifecycle alignment**: Each hook has a clear, single purpose
|
||||
|
||||
---
|
||||
|
||||
## Further Reading
|
||||
|
||||
- [Claude Code Hooks Reference](https://docs.claude.com/claude-code/hooks) - Official documentation
|
||||
- [Progressive Disclosure](progressive-disclosure) - Context priming philosophy
|
||||
- [Architecture Evolution](architecture-evolution) - v3 to v4 journey
|
||||
- [Worker Service Design](architecture/worker-service) - Background processing details
|
||||
|
||||
---
|
||||
|
||||
*The hook-driven architecture enables Claude-Mem to be both powerful and invisible. Users never notice the memory system working - it just makes Claude smarter over time.*
|
||||
@@ -0,0 +1,113 @@
|
||||
---
|
||||
title: "Installation"
|
||||
description: "Install Claude-Mem plugin for persistent memory across sessions"
|
||||
---
|
||||
|
||||
# Installation Guide
|
||||
|
||||
## Quick Start
|
||||
|
||||
Install Claude-Mem directly from the plugin marketplace:
|
||||
|
||||
```bash
|
||||
/plugin marketplace add thedotmack/claude-mem
|
||||
/plugin install claude-mem
|
||||
```
|
||||
|
||||
That's it! The plugin will automatically:
|
||||
- Download prebuilt binaries (no compilation needed)
|
||||
- Install all dependencies (including PM2 and SQLite binaries)
|
||||
- Configure hooks for session lifecycle management
|
||||
- Set up the MCP search server
|
||||
- Auto-start the worker service on first session
|
||||
|
||||
Start a new Claude Code session and you'll see context from previous sessions automatically loaded.
|
||||
|
||||
## System Requirements
|
||||
|
||||
- **Node.js**: 18.0.0 or higher
|
||||
- **Claude Code**: Latest version with plugin support
|
||||
- **PM2**: Process manager (bundled with plugin - no global install required)
|
||||
- **SQLite 3**: For persistent storage (bundled)
|
||||
|
||||
## Advanced Installation
|
||||
|
||||
For development or testing, you can clone and build from source:
|
||||
|
||||
### Clone and Build
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/thedotmack/claude-mem.git
|
||||
cd claude-mem
|
||||
|
||||
# Install dependencies
|
||||
npm install
|
||||
|
||||
# Build hooks and worker service
|
||||
npm run build
|
||||
|
||||
# Worker service will auto-start on first Claude Code session
|
||||
# Or manually start with:
|
||||
npm run worker:start
|
||||
|
||||
# Verify worker is running
|
||||
npm run worker:status
|
||||
```
|
||||
|
||||
### Post-Installation Verification
|
||||
|
||||
#### 1. Automatic Dependency Installation
|
||||
|
||||
Dependencies are installed automatically during plugin installation. The SessionStart hook also ensures dependencies are up-to-date on each session start (this is fast and idempotent). Works cross-platform on Windows, macOS, and Linux.
|
||||
|
||||
#### 2. Verify Plugin Installation
|
||||
|
||||
Check that hooks are configured in Claude Code:
|
||||
```bash
|
||||
cat plugin/hooks/hooks.json
|
||||
```
|
||||
|
||||
#### 3. Data Directory Location
|
||||
|
||||
v4.0.0+ stores data in `${CLAUDE_PLUGIN_ROOT}/data/`:
|
||||
- Database: `${CLAUDE_PLUGIN_ROOT}/data/claude-mem.db`
|
||||
- Worker port file: `${CLAUDE_PLUGIN_ROOT}/data/worker.port`
|
||||
- Logs: `${CLAUDE_PLUGIN_ROOT}/data/logs/`
|
||||
|
||||
For development/testing, you can override:
|
||||
```bash
|
||||
export CLAUDE_MEM_DATA_DIR=/custom/path
|
||||
```
|
||||
|
||||
#### 4. Check Worker Logs
|
||||
|
||||
```bash
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
#### 5. Test Context Retrieval
|
||||
|
||||
```bash
|
||||
npm run test:context
|
||||
```
|
||||
|
||||
## Upgrading from v3.x
|
||||
|
||||
**BREAKING CHANGES - Please Read:**
|
||||
|
||||
v4.0.0 introduces breaking changes:
|
||||
|
||||
- **Data Location Changed**: Database moved from `~/.claude-mem/` to `${CLAUDE_PLUGIN_ROOT}/data/` (inside plugin directory)
|
||||
- **Fresh Start Required**: No automatic migration from v3.x. You must start with a clean database
|
||||
- **Worker Auto-Starts**: Worker service now starts automatically - no manual `npm run worker:start` needed
|
||||
- **MCP Search Server**: 7 new search tools with full-text search and citations
|
||||
- **Enhanced Architecture**: Improved plugin integration and data organization
|
||||
|
||||
See [CHANGELOG](https://github.com/thedotmack/claude-mem/blob/main/CHANGELOG.md) for complete details.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Getting Started Guide](usage/getting-started) - Learn how Claude-Mem works automatically
|
||||
- [MCP Search Tools](usage/search-tools) - Query your project history
|
||||
- [Configuration](configuration) - Customize Claude-Mem behavior
|
||||
@@ -0,0 +1,105 @@
|
||||
---
|
||||
title: "Introduction"
|
||||
description: "Persistent memory compression system for Claude Code"
|
||||
---
|
||||
|
||||
# Claude-Mem
|
||||
|
||||
**Persistent memory compression system for Claude Code**
|
||||
|
||||
Claude-Mem seamlessly preserves context across sessions by automatically capturing tool usage observations, generating semantic summaries, and making them available to future sessions. This enables Claude to maintain continuity of knowledge about projects even after sessions end or reconnect.
|
||||
|
||||
## Quick Start
|
||||
|
||||
Start a new Claude Code session in the terminal and enter the following commands:
|
||||
|
||||
```bash
|
||||
/plugin marketplace add thedotmack/claude-mem
|
||||
/plugin install claude-mem
|
||||
```
|
||||
|
||||
Restart Claude Code. Context from previous sessions will automatically appear in new sessions.
|
||||
|
||||
## Key Features
|
||||
|
||||
- 🧠 **Persistent Memory** - Context survives across sessions
|
||||
- 🔍 **Skill-Based Search** - Query your project history with natural language (~2,250 token savings)
|
||||
- 🌐 **Web Viewer UI** - Real-time memory stream visualization at http://localhost:37777
|
||||
- 🎨 **Theme Toggle** - Light, dark, and system preference themes
|
||||
- 🤖 **Automatic Operation** - No manual intervention required
|
||||
- 📊 **FTS5 Search** - Fast full-text search across observations
|
||||
- 🔗 **Citations** - Reference past decisions with `claude-mem://` URIs
|
||||
|
||||
## How It Works
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Session Start → Inject context from last 10 sessions │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ User Prompts → Create session, save user prompts │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Tool Executions → Capture observations (Read, Write, etc.) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Worker Processes → Extract learnings via Claude Agent SDK │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Session Ends → Generate summary, ready for next session │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Core Components:**
|
||||
1. **5 Lifecycle Hooks** - SessionStart, UserPromptSubmit, PostToolUse, Stop, SessionEnd
|
||||
2. **Worker Service** - HTTP API on port 37777 managed by PM2
|
||||
3. **SQLite Database** - Stores sessions, observations, summaries with FTS5 search
|
||||
4. **9 MCP Search Tools** - Query historical context with citations
|
||||
5. **Web Viewer UI** - Real-time visualization with SSE and infinite scroll
|
||||
|
||||
See [Architecture Overview](architecture/overview) for details.
|
||||
|
||||
## System Requirements
|
||||
|
||||
- **Node.js**: 18.0.0 or higher
|
||||
- **Claude Code**: Latest version with plugin support
|
||||
- **PM2**: Process manager (bundled - no global install required)
|
||||
- **SQLite 3**: For persistent storage (bundled)
|
||||
|
||||
## What's New in v5.1.2
|
||||
|
||||
**Latest Updates (v5.1.2):**
|
||||
- Theme toggle for light, dark, and system preferences in viewer UI
|
||||
- Improved visual design with theme-aware components
|
||||
|
||||
**Recent Updates (v5.1.0):**
|
||||
- Web-based viewer UI for real-time memory stream visualization
|
||||
- Server-Sent Events (SSE) for instant updates
|
||||
- Infinite scroll pagination with project filtering
|
||||
- 8 new HTTP/SSE endpoints in worker service
|
||||
|
||||
**Previous Updates (v4.3.1):**
|
||||
- Fixed SessionStart hook context injection
|
||||
- Smart install caching for Windows compatibility
|
||||
- Progressive disclosure context with observation timelines
|
||||
|
||||
## Next Steps
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Installation" icon="download" href="/installation">
|
||||
Quick start & advanced installation
|
||||
</Card>
|
||||
<Card title="Getting Started" icon="rocket" href="/usage/getting-started">
|
||||
Learn how Claude-Mem works automatically
|
||||
</Card>
|
||||
<Card title="Architecture" icon="sitemap" href="/architecture/overview">
|
||||
System components & data flow
|
||||
</Card>
|
||||
<Card title="Search Tools" icon="magnifying-glass" href="/usage/search-tools">
|
||||
Query your project history
|
||||
</Card>
|
||||
</CardGroup>
|
||||
@@ -0,0 +1,655 @@
|
||||
# Progressive Disclosure: Claude-Mem's Context Priming Philosophy
|
||||
|
||||
## Core Principle
|
||||
**Show what exists and its retrieval cost first. Let the agent decide what to fetch based on relevance and need.**
|
||||
|
||||
---
|
||||
|
||||
## What is Progressive Disclosure?
|
||||
|
||||
Progressive disclosure is an information architecture pattern where you reveal complexity gradually rather than all at once. In the context of AI agents, it means:
|
||||
|
||||
1. **Layer 1 (Index)**: Show lightweight metadata (titles, dates, types, token counts)
|
||||
2. **Layer 2 (Details)**: Fetch full content only when needed
|
||||
3. **Layer 3 (Deep Dive)**: Read original source files if required
|
||||
|
||||
This mirrors how humans work: We scan headlines before reading articles, review table of contents before diving into chapters, and check file names before opening files.
|
||||
|
||||
---
|
||||
|
||||
## The Problem: Context Pollution
|
||||
|
||||
Traditional RAG (Retrieval-Augmented Generation) systems fetch everything upfront:
|
||||
|
||||
```
|
||||
❌ Traditional Approach:
|
||||
┌─────────────────────────────────────┐
|
||||
│ Session Start │
|
||||
│ │
|
||||
│ [15,000 tokens of past sessions] │
|
||||
│ [8,000 tokens of observations] │
|
||||
│ [12,000 tokens of file summaries] │
|
||||
│ │
|
||||
│ Total: 35,000 tokens │
|
||||
│ Relevant: ~2,000 tokens (6%) │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Problems:**
|
||||
- Wastes 94% of attention budget on irrelevant context
|
||||
- User prompt gets buried under mountain of history
|
||||
- Agent must process everything before understanding task
|
||||
- No way to know what's actually useful until after reading
|
||||
|
||||
---
|
||||
|
||||
## Claude-Mem's Solution: Progressive Disclosure
|
||||
|
||||
```
|
||||
✅ Progressive Disclosure Approach:
|
||||
┌─────────────────────────────────────┐
|
||||
│ Session Start │
|
||||
│ │
|
||||
│ Index of 50 observations: ~800 tokens│
|
||||
│ ↓ │
|
||||
│ Agent sees: "🔴 Hook timeout issue" │
|
||||
│ Agent decides: "Relevant!" │
|
||||
│ ↓ │
|
||||
│ Fetch observation #2543: ~120 tokens│
|
||||
│ │
|
||||
│ Total: 920 tokens │
|
||||
│ Relevant: 920 tokens (100%) │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Agent controls its own context consumption
|
||||
- Directly relevant to current task
|
||||
- Can fetch more if needed
|
||||
- Can skip everything if not relevant
|
||||
- Clear cost/benefit for each retrieval decision
|
||||
|
||||
---
|
||||
|
||||
## How It Works in Claude-Mem
|
||||
|
||||
### The Index Format
|
||||
|
||||
Every SessionStart hook provides a compact index:
|
||||
|
||||
```markdown
|
||||
### Oct 26, 2025
|
||||
|
||||
**General**
|
||||
| ID | Time | T | Title | Tokens |
|
||||
|----|------|---|-------|--------|
|
||||
| #2586 | 12:58 AM | 🔵 | Context hook file exists but is empty | ~51 |
|
||||
| #2587 | ″ | 🔵 | Context hook script file is empty | ~46 |
|
||||
| #2589 | ″ | 🟡 | Investigated hook debug output docs | ~105 |
|
||||
|
||||
**src/hooks/context-hook.ts**
|
||||
| ID | Time | T | Title | Tokens |
|
||||
|----|------|---|-------|--------|
|
||||
| #2591 | 1:15 AM | ⚖️ | Stderr messaging abandoned | ~155 |
|
||||
| #2592 | 1:16 AM | ⚖️ | Web UI strategy redesigned | ~193 |
|
||||
```
|
||||
|
||||
**What the agent sees:**
|
||||
- **What exists**: Observation titles give semantic meaning
|
||||
- **When it happened**: Timestamps for temporal context
|
||||
- **What type**: Icons indicate observation category
|
||||
- **Retrieval cost**: Token counts for informed decisions
|
||||
- **Where to get it**: MCP search tools referenced at bottom
|
||||
|
||||
### The Legend System
|
||||
|
||||
```
|
||||
🎯 session-request - User's original goal
|
||||
🔴 gotcha - Critical edge case or pitfall
|
||||
🟡 problem-solution - Bug fix or workaround
|
||||
🔵 how-it-works - Technical explanation
|
||||
🟢 what-changed - Code/architecture change
|
||||
🟣 discovery - Learning or insight
|
||||
🟠 why-it-exists - Design rationale
|
||||
🟤 decision - Architecture decision
|
||||
⚖️ trade-off - Deliberate compromise
|
||||
```
|
||||
|
||||
**Purpose:**
|
||||
- Visual scanning (humans and AI both benefit)
|
||||
- Semantic categorization
|
||||
- Priority signaling (🔴 gotchas are more critical)
|
||||
- Pattern recognition across sessions
|
||||
|
||||
### Progressive Disclosure Instructions
|
||||
|
||||
The index includes usage guidance:
|
||||
|
||||
```markdown
|
||||
💡 **Progressive Disclosure:** This index shows WHAT exists and retrieval COST.
|
||||
- Use MCP search tools to fetch full observation details on-demand
|
||||
- Prefer searching observations over re-reading code for past decisions
|
||||
- Critical types (🔴 gotcha, 🟤 decision, ⚖️ trade-off) often worth fetching immediately
|
||||
```
|
||||
|
||||
**What this does:**
|
||||
- Teaches the agent the pattern
|
||||
- Suggests when to fetch (critical types)
|
||||
- Recommends search over code re-reading (efficiency)
|
||||
- Makes the system self-documenting
|
||||
|
||||
---
|
||||
|
||||
## The Philosophy: Context as Currency
|
||||
|
||||
### Mental Model: Token Budget as Money
|
||||
|
||||
Think of context window as a bank account:
|
||||
|
||||
| Approach | Metaphor | Outcome |
|
||||
|----------|----------|---------|
|
||||
| **Dump everything** | Spending your entire paycheck on groceries you might need someday | Waste, clutter, can't afford what you actually need |
|
||||
| **Fetch nothing** | Refusing to spend any money | Starvation, can't accomplish tasks |
|
||||
| **Progressive disclosure** | Check your pantry, make a shopping list, buy only what you need | Efficiency, room for unexpected needs |
|
||||
|
||||
### The Attention Budget
|
||||
|
||||
LLMs have finite attention:
|
||||
- Every token attends to every other token (n² relationships)
|
||||
- 100,000 token window ≠ 100,000 tokens of useful attention
|
||||
- Context "rot" happens as window fills
|
||||
- Later tokens get less attention than earlier ones
|
||||
|
||||
**Claude-Mem's approach:**
|
||||
- Start with ~1,000 tokens of index
|
||||
- Agent has 99,000 tokens free for task
|
||||
- Agent fetches ~200 tokens when needed
|
||||
- Final budget: ~98,000 tokens for actual work
|
||||
|
||||
### Design for Autonomy
|
||||
|
||||
> "As models improve, let them act intelligently"
|
||||
|
||||
Progressive disclosure treats the agent as an **intelligent information forager**, not a passive recipient of pre-selected context.
|
||||
|
||||
**Traditional RAG:**
|
||||
```
|
||||
System → [Decides relevance] → Agent
|
||||
↑
|
||||
Hope this helps!
|
||||
```
|
||||
|
||||
**Progressive Disclosure:**
|
||||
```
|
||||
System → [Shows index] → Agent → [Decides relevance] → [Fetches details]
|
||||
↑
|
||||
You know best!
|
||||
```
|
||||
|
||||
The agent knows:
|
||||
- The current task context
|
||||
- What information would help
|
||||
- How much budget to spend
|
||||
- When to stop searching
|
||||
|
||||
We don't.
|
||||
|
||||
---
|
||||
|
||||
## Implementation Principles
|
||||
|
||||
### 1. Make Costs Visible
|
||||
|
||||
Every item in the index shows token count:
|
||||
|
||||
```
|
||||
| #2591 | 1:15 AM | ⚖️ | Stderr messaging abandoned | ~155 |
|
||||
^^^^
|
||||
Retrieval cost
|
||||
```
|
||||
|
||||
**Why:**
|
||||
- Agent can make informed ROI decisions
|
||||
- Small observations (~50 tokens) are "cheap" to fetch
|
||||
- Large observations (~500 tokens) require stronger justification
|
||||
- Matches how humans think about effort
|
||||
|
||||
### 2. Use Semantic Compression
|
||||
|
||||
Titles compress full observations into ~10 words:
|
||||
|
||||
**Bad title:**
|
||||
```
|
||||
Observation about a thing
|
||||
```
|
||||
|
||||
**Good title:**
|
||||
```
|
||||
🔴 Hook timeout issue: 60s default too short for npm install
|
||||
```
|
||||
|
||||
**What makes a good title:**
|
||||
- Specific: Identifies exact issue
|
||||
- Actionable: Clear what to do
|
||||
- Self-contained: Doesn't require reading observation
|
||||
- Searchable: Contains key terms (hook, timeout, npm)
|
||||
- Categorized: Icon indicates type
|
||||
|
||||
### 3. Group by Context
|
||||
|
||||
Observations are grouped by:
|
||||
- **Date**: Temporal context
|
||||
- **File path**: Spatial context (work on specific files)
|
||||
- **Project**: Logical context
|
||||
|
||||
```markdown
|
||||
**src/hooks/context-hook.ts**
|
||||
| ID | Time | T | Title | Tokens |
|
||||
|----|------|---|-------|--------|
|
||||
| #2591 | 1:15 AM | ⚖️ | Stderr messaging abandoned | ~155 |
|
||||
| #2594 | 1:17 AM | 🟠 | Removed stderr section from docs | ~93 |
|
||||
```
|
||||
|
||||
**Benefit:** If agent is working on `src/hooks/context-hook.ts`, related observations are already grouped together.
|
||||
|
||||
### 4. Provide Retrieval Tools
|
||||
|
||||
The index is useless without retrieval mechanisms:
|
||||
|
||||
```markdown
|
||||
*Use claude-mem MCP search to access records with the given ID*
|
||||
```
|
||||
|
||||
**Available tools:**
|
||||
- `search_observations` - Full-text search
|
||||
- `find_by_concept` - Concept-based retrieval
|
||||
- `find_by_file` - File-based retrieval
|
||||
- `find_by_type` - Type-based retrieval
|
||||
- `get_recent_context` - Recent session summaries
|
||||
|
||||
Each tool supports `format: "index"` (default) and `format: "full"`.
|
||||
|
||||
---
|
||||
|
||||
## Real-World Example
|
||||
|
||||
### Scenario: Agent asked to fix a bug in hooks
|
||||
|
||||
**Without progressive disclosure:**
|
||||
```
|
||||
SessionStart injects 25,000 tokens of past context
|
||||
Agent reads everything
|
||||
Agent finds 1 relevant observation (buried in middle)
|
||||
Total tokens consumed: 25,000
|
||||
Relevant tokens: ~200
|
||||
Efficiency: 0.8%
|
||||
```
|
||||
|
||||
**With progressive disclosure:**
|
||||
```
|
||||
SessionStart shows index: ~800 tokens
|
||||
Agent sees title: "🔴 Hook timeout issue: 60s too short"
|
||||
Agent thinks: "This looks relevant to my bug!"
|
||||
Agent fetches observation #2543: ~155 tokens
|
||||
Total tokens consumed: 955
|
||||
Relevant tokens: 955
|
||||
Efficiency: 100%
|
||||
```
|
||||
|
||||
### The Index Entry
|
||||
|
||||
```markdown
|
||||
| #2543 | 2:14 PM | 🔴 | Hook timeout: 60s too short for npm install | ~155 |
|
||||
```
|
||||
|
||||
**What the agent learns WITHOUT fetching:**
|
||||
- There's a known gotcha (🔴) about hook timeouts
|
||||
- It's related to npm install taking too long
|
||||
- Full details are ~155 tokens (cheap)
|
||||
- Happened at 2:14 PM (recent)
|
||||
|
||||
**Decision tree:**
|
||||
```
|
||||
Is my task related to hooks? → YES
|
||||
Is my task related to timeouts? → YES
|
||||
Is my task related to npm? → YES
|
||||
155 tokens is cheap → FETCH IT
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## The Two-Tier Search Strategy
|
||||
|
||||
Claude-Mem implements progressive disclosure in search results too:
|
||||
|
||||
### Tier 1: Index Format (Default)
|
||||
|
||||
```typescript
|
||||
search_observations({
|
||||
query: "hook timeout",
|
||||
format: "index" // Default
|
||||
})
|
||||
```
|
||||
|
||||
**Returns:**
|
||||
```
|
||||
Found 3 observations matching "hook timeout":
|
||||
|
||||
| ID | Date | Type | Title | Tokens |
|
||||
|----|------|------|-------|--------|
|
||||
| #2543 | Oct 26 | gotcha | Hook timeout: 60s too short | ~155 |
|
||||
| #2891 | Oct 25 | how-it-works | Hook timeout configuration | ~203 |
|
||||
| #2102 | Oct 20 | problem-solution | Fixed timeout in CI | ~89 |
|
||||
```
|
||||
|
||||
**Cost:** ~100 tokens for 3 results
|
||||
**Value:** Agent can scan and decide which to fetch
|
||||
|
||||
### Tier 2: Full Format (On-Demand)
|
||||
|
||||
```typescript
|
||||
search_observations({
|
||||
query: "hook timeout",
|
||||
format: "full",
|
||||
limit: 1 // Fetch just the most relevant
|
||||
})
|
||||
```
|
||||
|
||||
**Returns:**
|
||||
```
|
||||
#2543 🔴 Hook timeout: 60s too short for npm install
|
||||
─────────────────────────────────────────────────
|
||||
Date: Oct 26, 2025 2:14 PM
|
||||
Type: gotcha
|
||||
Project: claude-mem
|
||||
|
||||
Narrative:
|
||||
Discovered that the default 60-second hook timeout is insufficient
|
||||
for npm install operations, especially with large dependency trees
|
||||
or slow network conditions. This causes SessionStart hook to fail
|
||||
silently, preventing context injection.
|
||||
|
||||
Facts:
|
||||
- Default timeout: 60 seconds
|
||||
- npm install with cold cache: ~90 seconds
|
||||
- Configured timeout: 120 seconds in plugin/hooks/hooks.json:25
|
||||
|
||||
Files Modified:
|
||||
- plugin/hooks/hooks.json
|
||||
|
||||
Concepts: hooks, timeout, npm, configuration
|
||||
```
|
||||
|
||||
**Cost:** ~155 tokens for full details
|
||||
**Value:** Complete understanding of the issue
|
||||
|
||||
---
|
||||
|
||||
## Cognitive Load Theory
|
||||
|
||||
Progressive disclosure is grounded in **Cognitive Load Theory**:
|
||||
|
||||
### Intrinsic Load
|
||||
The inherent difficulty of the task itself.
|
||||
|
||||
**Example:** "Fix authentication bug"
|
||||
- Must understand auth system
|
||||
- Must understand the bug
|
||||
- Must write the fix
|
||||
|
||||
This load is unavoidable.
|
||||
|
||||
### Extraneous Load
|
||||
The cognitive burden of poorly presented information.
|
||||
|
||||
**Traditional RAG adds extraneous load:**
|
||||
- Scanning irrelevant observations
|
||||
- Filtering out noise
|
||||
- Remembering what to ignore
|
||||
- Re-contextualizing after each section
|
||||
|
||||
**Progressive disclosure minimizes extraneous load:**
|
||||
- Scan titles (low effort)
|
||||
- Fetch only relevant (targeted effort)
|
||||
- Full attention on current task
|
||||
|
||||
### Germane Load
|
||||
The effort of building mental models and schemas.
|
||||
|
||||
**Progressive disclosure supports germane load:**
|
||||
- Consistent structure (legend, grouping)
|
||||
- Clear categorization (types, icons)
|
||||
- Semantic compression (good titles)
|
||||
- Explicit costs (token counts)
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns to Avoid
|
||||
|
||||
### ❌ Verbose Titles
|
||||
|
||||
**Bad:**
|
||||
```
|
||||
| #2543 | 2:14 PM | 🔴 | Investigation into the issue where hooks time out | ~155 |
|
||||
```
|
||||
|
||||
**Good:**
|
||||
```
|
||||
| #2543 | 2:14 PM | 🔴 | Hook timeout: 60s too short for npm install | ~155 |
|
||||
```
|
||||
|
||||
### ❌ Hiding Costs
|
||||
|
||||
**Bad:**
|
||||
```
|
||||
| #2543 | 2:14 PM | 🔴 | Hook timeout issue |
|
||||
```
|
||||
|
||||
**Good:**
|
||||
```
|
||||
| #2543 | 2:14 PM | 🔴 | Hook timeout issue | ~155 |
|
||||
```
|
||||
|
||||
### ❌ No Retrieval Path
|
||||
|
||||
**Bad:**
|
||||
```
|
||||
Here are 10 observations. [No instructions on how to get full details]
|
||||
```
|
||||
|
||||
**Good:**
|
||||
```
|
||||
Here are 10 observations.
|
||||
*Use MCP search tools to fetch full observation details on-demand*
|
||||
```
|
||||
|
||||
### ❌ Defaulting to Full Format
|
||||
|
||||
**Bad:**
|
||||
```typescript
|
||||
search_observations({
|
||||
query: "hooks",
|
||||
format: "full" // Fetches everything
|
||||
})
|
||||
```
|
||||
|
||||
**Good:**
|
||||
```typescript
|
||||
search_observations({
|
||||
query: "hooks",
|
||||
format: "index", // Scan first
|
||||
limit: 20
|
||||
})
|
||||
|
||||
// Then, if needed:
|
||||
search_observations({
|
||||
query: "hooks",
|
||||
format: "full",
|
||||
limit: 1 // Just the most relevant
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
### Why Token Counts?
|
||||
|
||||
**Decision:** Show approximate token counts (~155, ~203) rather than exact counts.
|
||||
|
||||
**Rationale:**
|
||||
- Communicates scale (50 vs 500) without false precision
|
||||
- Maps to human intuition (small/medium/large)
|
||||
- Allows agent to budget attention
|
||||
- Encourages cost-conscious retrieval
|
||||
|
||||
### Why Icons Instead of Text Labels?
|
||||
|
||||
**Decision:** Use emoji icons (🔴, 🟡, 🔵) rather than text (GOTCHA, PROBLEM, HOWTO).
|
||||
|
||||
**Rationale:**
|
||||
- Visual scanning (pattern recognition)
|
||||
- Token efficient (1 char vs 10 chars)
|
||||
- Language-agnostic
|
||||
- Aesthetically distinct
|
||||
- Works for both humans and AI
|
||||
|
||||
### Why Index-First, Not Smart Pre-Fetch?
|
||||
|
||||
**Decision:** Always show index first, even if we "know" what's relevant.
|
||||
|
||||
**Rationale:**
|
||||
- We can't know what's relevant better than the agent
|
||||
- Pre-fetching assumes we understand the task
|
||||
- Agent knows current context, we don't
|
||||
- Respects agent autonomy
|
||||
- Fails gracefully (can always fetch more)
|
||||
|
||||
### Why Group by File Path?
|
||||
|
||||
**Decision:** Group observations by file path in addition to date.
|
||||
|
||||
**Rationale:**
|
||||
- Spatial locality: Work on file X likely needs context about file X
|
||||
- Reduces scanning effort
|
||||
- Matches how developers think
|
||||
- Clear semantic boundaries
|
||||
|
||||
---
|
||||
|
||||
## Measuring Success
|
||||
|
||||
Progressive disclosure is working when:
|
||||
|
||||
### ✅ Low Waste Ratio
|
||||
```
|
||||
Relevant Tokens / Total Context Tokens > 80%
|
||||
```
|
||||
|
||||
Most of the context consumed is actually useful.
|
||||
|
||||
### ✅ Selective Fetching
|
||||
```
|
||||
Index Shown: 50 observations
|
||||
Details Fetched: 2-3 observations
|
||||
```
|
||||
|
||||
Agent is being selective, not fetching everything.
|
||||
|
||||
### ✅ Fast Task Completion
|
||||
```
|
||||
Session with index: 30 seconds to find relevant context
|
||||
Session without: 90 seconds scanning all context
|
||||
```
|
||||
|
||||
Time-to-relevant-information is faster.
|
||||
|
||||
### ✅ Appropriate Depth
|
||||
```
|
||||
Simple task: Only index needed
|
||||
Medium task: 1-2 observations fetched
|
||||
Complex task: 5-10 observations + code reads
|
||||
```
|
||||
|
||||
Depth scales with task complexity.
|
||||
|
||||
---
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Adaptive Index Size
|
||||
|
||||
```typescript
|
||||
// Vary index size based on session type
|
||||
SessionStart({ source: "startup" }):
|
||||
→ Show last 10 sessions (small index)
|
||||
|
||||
SessionStart({ source: "resume" }):
|
||||
→ Show only current session (micro index)
|
||||
|
||||
SessionStart({ source: "compact" }):
|
||||
→ Show last 20 sessions (larger index)
|
||||
```
|
||||
|
||||
### Relevance Scoring
|
||||
|
||||
```typescript
|
||||
// Use embeddings to pre-sort index by relevance
|
||||
search_observations({
|
||||
query: "authentication bug",
|
||||
format: "index",
|
||||
sort: "relevance" // Based on semantic similarity
|
||||
})
|
||||
```
|
||||
|
||||
### Cost Forecasting
|
||||
|
||||
```markdown
|
||||
💡 **Budget Estimate:**
|
||||
- Fetching all 🔴 gotchas: ~450 tokens
|
||||
- Fetching all file-related: ~1,200 tokens
|
||||
- Fetching everything: ~8,500 tokens
|
||||
```
|
||||
|
||||
### Progressive Detail Levels
|
||||
|
||||
```
|
||||
Layer 1: Index (titles only)
|
||||
Layer 2: Summaries (2-3 sentences)
|
||||
Layer 3: Full details (complete observation)
|
||||
Layer 4: Source files (referenced code)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Takeaways
|
||||
|
||||
1. **Show, don't tell**: Index reveals what exists without forcing consumption
|
||||
2. **Cost-conscious**: Make retrieval costs visible for informed decisions
|
||||
3. **Agent autonomy**: Let the agent decide what's relevant
|
||||
4. **Semantic compression**: Good titles make or break the system
|
||||
5. **Consistent structure**: Patterns reduce cognitive load
|
||||
6. **Two-tier everything**: Index first, details on-demand
|
||||
7. **Context as currency**: Spend wisely on high-value information
|
||||
|
||||
---
|
||||
|
||||
## Remember
|
||||
|
||||
> "The best interface is one that disappears when not needed, and appears exactly when it is."
|
||||
|
||||
Progressive disclosure respects the agent's intelligence and autonomy. We provide the map; the agent chooses the path.
|
||||
|
||||
---
|
||||
|
||||
## Further Reading
|
||||
|
||||
- [Context Engineering for AI Agents](context-engineering) - Foundational principles
|
||||
- [Claude-Mem Architecture](architecture/overview) - How it all fits together
|
||||
- Cognitive Load Theory (Sweller, 1988)
|
||||
- Information Foraging Theory (Pirolli & Card, 1999)
|
||||
- Progressive Disclosure (Nielsen Norman Group)
|
||||
|
||||
---
|
||||
|
||||
*This philosophy emerged from real-world usage of Claude-Mem across hundreds of coding sessions. The pattern works because it aligns with both human cognition and LLM attention mechanics.*
|
||||
@@ -0,0 +1,865 @@
|
||||
---
|
||||
title: "Troubleshooting"
|
||||
description: "Common issues and solutions for Claude-Mem"
|
||||
---
|
||||
|
||||
# Troubleshooting Guide
|
||||
|
||||
## Quick Diagnostic Tool
|
||||
|
||||
**NEW:** Use the automated troubleshooting skill for instant diagnosis:
|
||||
|
||||
```
|
||||
/skill troubleshoot
|
||||
```
|
||||
|
||||
This skill will:
|
||||
- ✅ Check PM2 worker status and health
|
||||
- ✅ Verify database existence and integrity
|
||||
- ✅ Test worker service connectivity
|
||||
- ✅ Validate dependencies installation
|
||||
- ✅ Check port configuration and availability
|
||||
- ✅ Provide automated fixes for common issues
|
||||
|
||||
The skill includes comprehensive diagnostics, automated repair sequences, and detailed troubleshooting workflows for all common issues. Use it before manually troubleshooting below.
|
||||
|
||||
---
|
||||
|
||||
## v5.x Specific Issues
|
||||
|
||||
### Viewer UI Not Loading
|
||||
|
||||
**Symptoms**: Cannot access http://localhost:37777, page doesn't load, or shows connection error.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Check if worker is running on port 37777:
|
||||
```bash
|
||||
lsof -i :37777
|
||||
# or
|
||||
npm run worker:status
|
||||
```
|
||||
|
||||
2. Verify worker is healthy:
|
||||
```bash
|
||||
curl http://localhost:37777/health
|
||||
```
|
||||
|
||||
3. Check worker logs for errors:
|
||||
```bash
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
4. Restart worker service:
|
||||
```bash
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
5. Check for port conflicts:
|
||||
```bash
|
||||
# If port 37777 is in use by another service
|
||||
export CLAUDE_MEM_WORKER_PORT=38000
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
### Theme Toggle Not Persisting
|
||||
|
||||
**Symptoms**: Theme preference (light/dark mode) resets after browser refresh.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Check browser localStorage is enabled:
|
||||
```javascript
|
||||
// In browser console
|
||||
localStorage.getItem('claude-mem-settings')
|
||||
```
|
||||
|
||||
2. Verify settings endpoint is working:
|
||||
```bash
|
||||
curl http://localhost:37777/api/settings
|
||||
```
|
||||
|
||||
3. Clear localStorage and try again:
|
||||
```javascript
|
||||
// In browser console
|
||||
localStorage.removeItem('claude-mem-settings')
|
||||
```
|
||||
|
||||
4. Check for browser privacy mode (blocks localStorage)
|
||||
|
||||
### SSE Connection Issues
|
||||
|
||||
**Symptoms**: Viewer shows "Disconnected" status, updates not appearing in real-time.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Check SSE endpoint is accessible:
|
||||
```bash
|
||||
curl -N http://localhost:37777/stream
|
||||
```
|
||||
|
||||
2. Check browser console for errors:
|
||||
- Open DevTools (F12)
|
||||
- Look for EventSource errors
|
||||
- Check Network tab for failed /stream requests
|
||||
|
||||
3. Verify worker is running:
|
||||
```bash
|
||||
npm run worker:status
|
||||
```
|
||||
|
||||
4. Check for network/proxy issues blocking SSE
|
||||
- Corporate firewalls may block SSE
|
||||
- Try disabling VPN temporarily
|
||||
|
||||
5. Restart worker and refresh browser:
|
||||
```bash
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
### Chroma/Python Dependency Issues (v5.0.0+)
|
||||
|
||||
**Symptoms**: Installation fails with chromadb or Python-related errors.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Verify Python 3.8+ is installed:
|
||||
```bash
|
||||
python --version
|
||||
# or
|
||||
python3 --version
|
||||
```
|
||||
|
||||
2. Install chromadb manually:
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack
|
||||
npm install chromadb
|
||||
```
|
||||
|
||||
3. Check chromadb health:
|
||||
```bash
|
||||
npm run chroma:health
|
||||
```
|
||||
|
||||
4. Windows-specific: Ensure Python is in PATH:
|
||||
```bash
|
||||
where python
|
||||
# Should show Python installation path
|
||||
```
|
||||
|
||||
5. If Chroma continues to fail, hybrid search will gracefully degrade to SQLite FTS5 only
|
||||
|
||||
### Smart Install Caching Issues (v5.0.3+)
|
||||
|
||||
**Symptoms**: Dependencies not updating after plugin update, stale version marker.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Clear install cache:
|
||||
```bash
|
||||
rm ~/.claude/plugins/marketplaces/thedotmack/.install-version
|
||||
```
|
||||
|
||||
2. Force reinstall:
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack
|
||||
npm install --force
|
||||
```
|
||||
|
||||
3. Check version marker:
|
||||
```bash
|
||||
cat ~/.claude/plugins/marketplaces/thedotmack/.install-version
|
||||
cat ~/.claude/plugins/marketplaces/thedotmack/package.json | grep version
|
||||
```
|
||||
|
||||
4. Restart Claude Code after manual install
|
||||
|
||||
### PM2 ENOENT Error on Windows (v5.1.1 Fix)
|
||||
|
||||
**Symptoms**: Worker fails to start with "ENOENT" error on Windows.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. This was fixed in v5.1.1 - update to latest version:
|
||||
```bash
|
||||
/plugin update claude-mem
|
||||
```
|
||||
|
||||
2. If still experiencing issues, verify PM2 path:
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack
|
||||
dir node_modules\.bin\pm2.cmd
|
||||
```
|
||||
|
||||
3. Manual PM2 install if needed:
|
||||
```bash
|
||||
npm install pm2@latest
|
||||
```
|
||||
|
||||
## Worker Service Issues
|
||||
|
||||
### Worker Service Not Starting
|
||||
|
||||
**Symptoms**: Worker doesn't start, or `pm2 status` shows no processes.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Check if PM2 is running:
|
||||
```bash
|
||||
pm2 status
|
||||
```
|
||||
|
||||
2. Try starting manually:
|
||||
```bash
|
||||
npm run worker:start
|
||||
```
|
||||
|
||||
3. Check worker logs for errors:
|
||||
```bash
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
4. Full reset:
|
||||
```bash
|
||||
pm2 delete claude-mem-worker
|
||||
npm run worker:start
|
||||
```
|
||||
|
||||
5. Verify PM2 is installed:
|
||||
```bash
|
||||
which pm2
|
||||
npm list pm2
|
||||
```
|
||||
|
||||
### Port Allocation Failed
|
||||
|
||||
**Symptoms**: Worker fails to start with "port already in use" error.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Check if port 37777 is in use:
|
||||
```bash
|
||||
lsof -i :37777
|
||||
```
|
||||
|
||||
2. Kill process using the port:
|
||||
```bash
|
||||
kill -9 $(lsof -t -i:37777)
|
||||
```
|
||||
|
||||
3. Or use a different port:
|
||||
```bash
|
||||
export CLAUDE_MEM_WORKER_PORT=38000
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
4. Verify new port:
|
||||
```bash
|
||||
cat ~/.claude-mem/worker.port
|
||||
```
|
||||
|
||||
### Worker Keeps Crashing
|
||||
|
||||
**Symptoms**: Worker restarts repeatedly, PM2 shows high restart count.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Check error logs:
|
||||
```bash
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
2. Check memory usage:
|
||||
```bash
|
||||
pm2 status
|
||||
```
|
||||
|
||||
3. Increase memory limit in `ecosystem.config.cjs`:
|
||||
```javascript
|
||||
{
|
||||
max_memory_restart: '2G' // Increase if needed
|
||||
}
|
||||
```
|
||||
|
||||
4. Check database for corruption:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;"
|
||||
```
|
||||
|
||||
### Worker Not Processing Observations
|
||||
|
||||
**Symptoms**: Observations saved but not processed, no summaries generated.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Check worker is running:
|
||||
```bash
|
||||
npm run worker:status
|
||||
```
|
||||
|
||||
2. Check worker logs:
|
||||
```bash
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
3. Verify database has observations:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations;"
|
||||
```
|
||||
|
||||
4. Restart worker:
|
||||
```bash
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
## Hook Issues
|
||||
|
||||
### Hooks Not Firing
|
||||
|
||||
**Symptoms**: No context appears, observations not saved.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Verify hooks are configured:
|
||||
```bash
|
||||
cat plugin/hooks/hooks.json
|
||||
```
|
||||
|
||||
2. Test hooks manually:
|
||||
```bash
|
||||
# Test context hook
|
||||
echo '{"session_id":"test-123","cwd":"'$(pwd)'","source":"startup"}' | node plugin/scripts/context-hook.js
|
||||
```
|
||||
|
||||
3. Check hook permissions:
|
||||
```bash
|
||||
ls -la plugin/scripts/*.js
|
||||
```
|
||||
|
||||
4. Verify hooks.json is valid JSON:
|
||||
```bash
|
||||
cat plugin/hooks/hooks.json | jq .
|
||||
```
|
||||
|
||||
### Context Not Appearing
|
||||
|
||||
**Symptoms**: No session context when Claude starts.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Check if summaries exist:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM session_summaries;"
|
||||
```
|
||||
|
||||
2. View recent sessions:
|
||||
```bash
|
||||
npm run test:context:verbose
|
||||
```
|
||||
|
||||
3. Check database integrity:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;"
|
||||
```
|
||||
|
||||
4. Manually test context hook:
|
||||
```bash
|
||||
npm run test:context
|
||||
```
|
||||
|
||||
### Hooks Timeout
|
||||
|
||||
**Symptoms**: Hook execution times out, errors in Claude Code.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Increase timeout in `plugin/hooks/hooks.json`:
|
||||
```json
|
||||
{
|
||||
"timeout": 180 // Increase from 120
|
||||
}
|
||||
```
|
||||
|
||||
2. Check worker is running (prevents timeout waiting for worker):
|
||||
```bash
|
||||
npm run worker:status
|
||||
```
|
||||
|
||||
3. Check database size (large database = slow queries):
|
||||
```bash
|
||||
ls -lh ~/.claude-mem/claude-mem.db
|
||||
```
|
||||
|
||||
4. Optimize database:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "VACUUM;"
|
||||
```
|
||||
|
||||
### Dependencies Not Installing
|
||||
|
||||
**Symptoms**: SessionStart hook fails with "module not found" errors.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Manually install dependencies:
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack
|
||||
npm install
|
||||
```
|
||||
|
||||
2. Check npm is available:
|
||||
```bash
|
||||
which npm
|
||||
npm --version
|
||||
```
|
||||
|
||||
3. Check package.json exists:
|
||||
```bash
|
||||
ls -la ~/.claude/plugins/marketplaces/thedotmack/package.json
|
||||
```
|
||||
|
||||
## Database Issues
|
||||
|
||||
### Database Locked
|
||||
|
||||
**Symptoms**: "database is locked" errors in logs.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Close all connections:
|
||||
```bash
|
||||
pm2 stop claude-mem-worker
|
||||
```
|
||||
|
||||
2. Check for stale locks:
|
||||
```bash
|
||||
lsof ~/.claude-mem/claude-mem.db
|
||||
```
|
||||
|
||||
3. Kill processes holding locks:
|
||||
```bash
|
||||
kill -9 <PID>
|
||||
```
|
||||
|
||||
4. Restart worker:
|
||||
```bash
|
||||
npm run worker:start
|
||||
```
|
||||
|
||||
### Database Corruption
|
||||
|
||||
**Symptoms**: Integrity check fails, weird errors.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Check database integrity:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;"
|
||||
```
|
||||
|
||||
2. Backup database:
|
||||
```bash
|
||||
cp ~/.claude-mem/claude-mem.db ~/.claude-mem/claude-mem.db.backup
|
||||
```
|
||||
|
||||
3. Try to repair:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "VACUUM;"
|
||||
```
|
||||
|
||||
4. Nuclear option - recreate database:
|
||||
```bash
|
||||
rm ~/.claude-mem/claude-mem.db
|
||||
npm run worker:start # Will recreate schema
|
||||
```
|
||||
|
||||
### FTS5 Search Not Working
|
||||
|
||||
**Symptoms**: Search returns no results, FTS5 errors.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Check FTS5 tables exist:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT name FROM sqlite_master WHERE type='table' AND name LIKE '%_fts';"
|
||||
```
|
||||
|
||||
2. Rebuild FTS5 tables:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
INSERT INTO observations_fts(observations_fts) VALUES('rebuild');
|
||||
INSERT INTO session_summaries_fts(session_summaries_fts) VALUES('rebuild');
|
||||
INSERT INTO user_prompts_fts(user_prompts_fts) VALUES('rebuild');
|
||||
"
|
||||
```
|
||||
|
||||
3. Check triggers exist:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT name FROM sqlite_master WHERE type='trigger';"
|
||||
```
|
||||
|
||||
### Database Too Large
|
||||
|
||||
**Symptoms**: Slow performance, large database file.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Check database size:
|
||||
```bash
|
||||
ls -lh ~/.claude-mem/claude-mem.db
|
||||
```
|
||||
|
||||
2. Vacuum database:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "VACUUM;"
|
||||
```
|
||||
|
||||
3. Delete old sessions:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
DELETE FROM observations WHERE created_at_epoch < $(date -v-30d +%s);
|
||||
DELETE FROM session_summaries WHERE created_at_epoch < $(date -v-30d +%s);
|
||||
DELETE FROM sdk_sessions WHERE created_at_epoch < $(date -v-30d +%s);
|
||||
"
|
||||
```
|
||||
|
||||
4. Rebuild FTS5 after deletion:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
INSERT INTO observations_fts(observations_fts) VALUES('rebuild');
|
||||
INSERT INTO session_summaries_fts(session_summaries_fts) VALUES('rebuild');
|
||||
"
|
||||
```
|
||||
|
||||
## MCP Search Issues
|
||||
|
||||
### Search Tools Not Available
|
||||
|
||||
**Symptoms**: MCP search tools not visible in Claude Code.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Check MCP configuration:
|
||||
```bash
|
||||
cat plugin/.mcp.json
|
||||
```
|
||||
|
||||
2. Verify search server is built:
|
||||
```bash
|
||||
ls -l plugin/scripts/search-server.js
|
||||
```
|
||||
|
||||
3. Rebuild if needed:
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
4. Restart Claude Code
|
||||
|
||||
### Search Returns No Results
|
||||
|
||||
**Symptoms**: Valid queries return empty results.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Check database has data:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations;"
|
||||
```
|
||||
|
||||
2. Verify FTS5 tables populated:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations_fts;"
|
||||
```
|
||||
|
||||
3. Test simple query:
|
||||
```bash
|
||||
# In Claude Code
|
||||
search_observations with query="test"
|
||||
```
|
||||
|
||||
4. Check query syntax:
|
||||
```bash
|
||||
# Bad: Special characters
|
||||
search_observations with query="[test]"
|
||||
|
||||
# Good: Simple words
|
||||
search_observations with query="test"
|
||||
```
|
||||
|
||||
### Token Limit Errors
|
||||
|
||||
**Symptoms**: "exceeded token limit" errors from MCP.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Use index format:
|
||||
```bash
|
||||
search_observations with query="..." and format="index"
|
||||
```
|
||||
|
||||
2. Reduce limit:
|
||||
```bash
|
||||
search_observations with query="..." and limit=3
|
||||
```
|
||||
|
||||
3. Use filters to narrow results:
|
||||
```bash
|
||||
search_observations with query="..." and type="decision" and limit=5
|
||||
```
|
||||
|
||||
4. Paginate results:
|
||||
```bash
|
||||
# First page
|
||||
search_observations with query="..." and limit=5 and offset=0
|
||||
|
||||
# Second page
|
||||
search_observations with query="..." and limit=5 and offset=5
|
||||
```
|
||||
|
||||
## Performance Issues
|
||||
|
||||
### Slow Context Injection
|
||||
|
||||
**Symptoms**: SessionStart hook takes too long.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Reduce context sessions:
|
||||
```typescript
|
||||
// In src/hooks/context.ts
|
||||
const CONTEXT_SESSIONS = 5; // Reduce from 10
|
||||
```
|
||||
|
||||
2. Optimize database:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
ANALYZE;
|
||||
VACUUM;
|
||||
"
|
||||
```
|
||||
|
||||
3. Add indexes (if missing):
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
CREATE INDEX IF NOT EXISTS idx_sessions_project_created ON sdk_sessions(project, created_at_epoch DESC);
|
||||
"
|
||||
```
|
||||
|
||||
### Slow Search Queries
|
||||
|
||||
**Symptoms**: MCP search tools take too long.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Use more specific queries
|
||||
2. Add date range filters
|
||||
3. Add type/concept filters
|
||||
4. Reduce result limit
|
||||
5. Use index format instead of full format
|
||||
|
||||
### High Memory Usage
|
||||
|
||||
**Symptoms**: Worker uses too much memory, frequent restarts.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Check current usage:
|
||||
```bash
|
||||
pm2 status
|
||||
```
|
||||
|
||||
2. Increase memory limit:
|
||||
```javascript
|
||||
// In ecosystem.config.cjs
|
||||
{
|
||||
max_memory_restart: '2G'
|
||||
}
|
||||
```
|
||||
|
||||
3. Restart worker:
|
||||
```bash
|
||||
npm run worker:restart
|
||||
```
|
||||
|
||||
4. Clean up old data (see "Database Too Large" above)
|
||||
|
||||
## Installation Issues
|
||||
|
||||
### Plugin Not Found
|
||||
|
||||
**Symptoms**: `/plugin install claude-mem` fails.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Add marketplace first:
|
||||
```bash
|
||||
/plugin marketplace add thedotmack/claude-mem
|
||||
```
|
||||
|
||||
2. Then install:
|
||||
```bash
|
||||
/plugin install claude-mem
|
||||
```
|
||||
|
||||
3. Verify installation:
|
||||
```bash
|
||||
ls -la ~/.claude/plugins/marketplaces/thedotmack/
|
||||
```
|
||||
|
||||
### Build Failures
|
||||
|
||||
**Symptoms**: `npm run build` fails.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Clean and reinstall:
|
||||
```bash
|
||||
rm -rf node_modules package-lock.json
|
||||
npm install
|
||||
```
|
||||
|
||||
2. Check Node.js version:
|
||||
```bash
|
||||
node --version # Should be >= 18.0.0
|
||||
```
|
||||
|
||||
3. Check for TypeScript errors:
|
||||
```bash
|
||||
npx tsc --noEmit
|
||||
```
|
||||
|
||||
### Missing Dependencies
|
||||
|
||||
**Symptoms**: "Cannot find module" errors.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Install dependencies:
|
||||
```bash
|
||||
npm install
|
||||
```
|
||||
|
||||
2. Check package.json:
|
||||
```bash
|
||||
cat package.json
|
||||
```
|
||||
|
||||
3. Verify node_modules exists:
|
||||
```bash
|
||||
ls -la node_modules/
|
||||
```
|
||||
|
||||
## Debugging
|
||||
|
||||
### Enable Verbose Logging
|
||||
|
||||
```bash
|
||||
export DEBUG=claude-mem:*
|
||||
npm run worker:restart
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
### Check Correlation IDs
|
||||
|
||||
Trace observations through the pipeline:
|
||||
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT correlation_id, tool_name, created_at
|
||||
FROM observations
|
||||
WHERE session_id = 'YOUR_SESSION_ID'
|
||||
ORDER BY created_at;
|
||||
"
|
||||
```
|
||||
|
||||
### Inspect Worker State
|
||||
|
||||
```bash
|
||||
# Check if worker is running
|
||||
pm2 status
|
||||
|
||||
# View logs
|
||||
pm2 logs claude-mem-worker
|
||||
|
||||
# Check port file
|
||||
cat ~/.claude-mem/worker.port
|
||||
|
||||
# Test worker health
|
||||
curl http://localhost:37777/health
|
||||
```
|
||||
|
||||
### Database Inspection
|
||||
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db
|
||||
|
||||
# View schema
|
||||
.schema
|
||||
|
||||
# Check table counts
|
||||
SELECT 'sessions', COUNT(*) FROM sdk_sessions
|
||||
UNION ALL
|
||||
SELECT 'observations', COUNT(*) FROM observations
|
||||
UNION ALL
|
||||
SELECT 'summaries', COUNT(*) FROM session_summaries
|
||||
UNION ALL
|
||||
SELECT 'prompts', COUNT(*) FROM user_prompts;
|
||||
|
||||
# View recent activity
|
||||
SELECT created_at, tool_name FROM observations ORDER BY created_at DESC LIMIT 10;
|
||||
```
|
||||
|
||||
## Common Error Messages
|
||||
|
||||
### "Worker service not responding"
|
||||
|
||||
**Cause**: Worker not running or port mismatch.
|
||||
|
||||
**Solution**: Restart worker with `npm run worker:restart`.
|
||||
|
||||
### "Database is locked"
|
||||
|
||||
**Cause**: Multiple processes accessing database.
|
||||
|
||||
**Solution**: Stop worker, kill stale processes, restart.
|
||||
|
||||
### "FTS5: syntax error"
|
||||
|
||||
**Cause**: Invalid search query syntax.
|
||||
|
||||
**Solution**: Use simpler query, avoid special characters.
|
||||
|
||||
### "SQLITE_CANTOPEN"
|
||||
|
||||
**Cause**: Database file permissions or missing directory.
|
||||
|
||||
**Solution**: Check `~/.claude-mem/` exists and is writable.
|
||||
|
||||
### "Module not found"
|
||||
|
||||
**Cause**: Missing dependencies.
|
||||
|
||||
**Solution**: Run `npm install`.
|
||||
|
||||
## Getting Help
|
||||
|
||||
If none of these solutions work:
|
||||
|
||||
1. **Check logs**:
|
||||
```bash
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
2. **Create issue**: [GitHub Issues](https://github.com/thedotmack/claude-mem/issues)
|
||||
- Include error messages
|
||||
- Include relevant logs
|
||||
- Include steps to reproduce
|
||||
|
||||
3. **Check existing issues**: Someone may have already solved your problem
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Configuration](configuration) - Customize Claude-Mem
|
||||
- [Development](development) - Build from source
|
||||
- [Architecture](architecture/overview) - Understand the system
|
||||
@@ -0,0 +1,215 @@
|
||||
---
|
||||
title: "Getting Started"
|
||||
description: "Learn how Claude-Mem works automatically in the background"
|
||||
---
|
||||
|
||||
# Getting Started with Claude-Mem
|
||||
|
||||
## Automatic Operation
|
||||
|
||||
Claude-Mem works automatically once installed. No manual intervention required!
|
||||
|
||||
### The Full Cycle
|
||||
|
||||
1. **Start Claude Code** - Context from last 10 sessions appears automatically
|
||||
2. **Work normally** - Every tool execution is captured
|
||||
3. **Claude finishes responding** - Stop hook automatically generates and saves a summary
|
||||
4. **Next session** - Previous work appears in context
|
||||
|
||||
### What Gets Captured
|
||||
|
||||
Every time Claude uses a tool, claude-mem captures it:
|
||||
|
||||
- **Read** - File reads and content access
|
||||
- **Write** - New file creation
|
||||
- **Edit** - File modifications
|
||||
- **Bash** - Command executions
|
||||
- **Glob** - File pattern searches
|
||||
- **Grep** - Content searches
|
||||
- And all other Claude Code tools
|
||||
|
||||
### What Gets Processed
|
||||
|
||||
The worker service processes tool observations and extracts:
|
||||
|
||||
- **Title** - Brief description of what happened
|
||||
- **Subtitle** - Additional context
|
||||
- **Narrative** - Detailed explanation
|
||||
- **Facts** - Key learnings as bullet points
|
||||
- **Concepts** - Relevant tags and categories
|
||||
- **Type** - Classification (decision, bugfix, feature, etc.)
|
||||
- **Files** - Which files were read or modified
|
||||
|
||||
### Session Summaries
|
||||
|
||||
When Claude finishes responding (triggering the Stop hook), a summary is automatically generated with:
|
||||
|
||||
- **Request** - What you asked for
|
||||
- **Investigated** - What Claude explored
|
||||
- **Learned** - Key discoveries and insights
|
||||
- **Completed** - What was accomplished
|
||||
- **Next Steps** - What to do next
|
||||
|
||||
### Context Injection
|
||||
|
||||
When you start a new Claude Code session, the SessionStart hook:
|
||||
|
||||
1. Queries the database for recent observations in your project (default: 50)
|
||||
2. Retrieves recent session summaries for context
|
||||
3. Displays observations in a chronological timeline with session markers
|
||||
4. Shows full summary details (Investigated, Learned, Completed, Next Steps) **only if the summary was generated after the last observation**
|
||||
5. Injects formatted context into Claude's initial context
|
||||
|
||||
**Summary Display Logic:**
|
||||
|
||||
The most recent summary's full details appear at the end of the context display **only when** the summary was generated after the most recent observation. This ensures you see summary details when they represent the latest state of your project, but not when new observations have been captured since the last summary.
|
||||
|
||||
For example:
|
||||
- ✅ **Shows summary**: Last observation at 2:00 PM, summary generated at 2:05 PM → Summary details appear
|
||||
- ❌ **Hides summary**: Summary generated at 2:00 PM, new observation at 2:05 PM → Summary details hidden (outdated)
|
||||
|
||||
This prevents showing stale summaries when new work has been captured but not yet summarized.
|
||||
|
||||
This means Claude "remembers" what happened in previous sessions!
|
||||
|
||||
## Manual Commands (Optional)
|
||||
|
||||
### Worker Management
|
||||
|
||||
v4.0+ auto-starts the worker on first session. Manual commands below are optional.
|
||||
|
||||
```bash
|
||||
# Start worker service (optional - auto-starts automatically)
|
||||
npm run worker:start
|
||||
|
||||
# Stop worker service
|
||||
npm run worker:stop
|
||||
|
||||
# Restart worker service
|
||||
npm run worker:restart
|
||||
|
||||
# View worker logs
|
||||
npm run worker:logs
|
||||
|
||||
# Check worker status
|
||||
npm run worker:status
|
||||
```
|
||||
|
||||
### Testing
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
npm test
|
||||
|
||||
# Test context injection
|
||||
npm run test:context
|
||||
|
||||
# Verbose context test
|
||||
npm run test:context:verbose
|
||||
```
|
||||
|
||||
### Development
|
||||
|
||||
```bash
|
||||
# Build hooks and worker
|
||||
npm run build
|
||||
|
||||
# Build only hooks
|
||||
npm run build:hooks
|
||||
|
||||
# Publish to NPM (maintainers only)
|
||||
npm run publish:npm
|
||||
```
|
||||
|
||||
## Viewing Stored Context
|
||||
|
||||
Context is stored in SQLite database at `~/.claude-mem/claude-mem.db`.
|
||||
|
||||
Query the database directly:
|
||||
|
||||
```bash
|
||||
# Open database
|
||||
sqlite3 ~/.claude-mem/claude-mem.db
|
||||
|
||||
# View recent sessions
|
||||
SELECT session_id, project, created_at, status
|
||||
FROM sdk_sessions
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 10;
|
||||
|
||||
# View session summaries
|
||||
SELECT session_id, request, completed, learned
|
||||
FROM session_summaries
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 5;
|
||||
|
||||
# View observations for a session
|
||||
SELECT tool_name, created_at
|
||||
FROM observations
|
||||
WHERE session_id = 'YOUR_SESSION_ID';
|
||||
```
|
||||
|
||||
## Understanding Progressive Disclosure
|
||||
|
||||
Context injection uses progressive disclosure for efficient token usage:
|
||||
|
||||
### Layer 1: Index Display (Session Start)
|
||||
- Shows observation titles with token cost estimates
|
||||
- Displays session markers in chronological timeline
|
||||
- Groups observations by file for visual clarity
|
||||
- Shows full summary details **only if** generated after last observation
|
||||
- Token cost: ~50-200 tokens for index view
|
||||
|
||||
### Layer 2: On-Demand Details (Skill-Based Search)
|
||||
- Ask naturally: "What bugs did we fix?" or "How did we implement X?"
|
||||
- Claude auto-invokes search skill to fetch full details
|
||||
- Search by concept, file, type, or keyword
|
||||
- Timeline context around specific observations
|
||||
- Token cost: ~100-500 tokens per observation fetched
|
||||
- Skill uses HTTP API (v5.4.0+) for efficient retrieval
|
||||
|
||||
### Layer 3: Perfect Recall (Code Access)
|
||||
- Read source files directly when needed
|
||||
- Access original transcripts and raw data
|
||||
- Full context available on-demand
|
||||
|
||||
This ensures efficient token usage while maintaining access to complete history when needed.
|
||||
|
||||
## Multi-Prompt Sessions & `/clear` Behavior
|
||||
|
||||
Claude-Mem supports sessions that span multiple user prompts:
|
||||
|
||||
- **prompt_counter**: Tracks total prompts in a session
|
||||
- **prompt_number**: Identifies specific prompt within session
|
||||
- **Session continuity**: Observations and summaries link across prompts
|
||||
|
||||
### Important Note About `/clear`
|
||||
|
||||
When you use `/clear`, the session doesn't end - it continues with a new prompt number. This means:
|
||||
|
||||
- ✅ **Context is re-injected** from recent sessions (SessionStart hook fires with `source: "clear"`)
|
||||
- ✅ **Observations are still being captured** and added to the current session
|
||||
- ✅ **A summary will be generated** when Claude finishes responding (Stop hook fires)
|
||||
|
||||
The `/clear` command clears the conversation context visible to Claude AND re-injects fresh context from recent sessions, while the underlying session continues tracking observations.
|
||||
|
||||
## Searching Your History (v5.4.0+)
|
||||
|
||||
Claude-Mem now uses skill-based search for querying your project history. Simply ask naturally:
|
||||
|
||||
```
|
||||
"What bugs did we fix last session?"
|
||||
"How did we implement authentication?"
|
||||
"What changes were made to worker-service.ts?"
|
||||
"Show me recent work on this project"
|
||||
```
|
||||
|
||||
Claude automatically recognizes your intent and invokes the search skill, which uses HTTP API endpoints to query your memory efficiently.
|
||||
|
||||
**Token Savings**: ~2,250 tokens per session start vs previous MCP approach
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Skill-Based Search](/usage/search-tools) - Learn how to search your project history
|
||||
- [Architecture Overview](/architecture/overview) - Understand how it works
|
||||
- [Troubleshooting](/troubleshooting) - Common issues and solutions
|
||||
@@ -0,0 +1,402 @@
|
||||
---
|
||||
title: "Skill-Based Search"
|
||||
description: "Query your project history with natural language"
|
||||
---
|
||||
|
||||
# Skill-Based Search Usage
|
||||
|
||||
Once claude-mem is installed as a plugin, you can search your project history using natural language. Claude automatically invokes the search skill when you ask about past work.
|
||||
|
||||
## How It Works
|
||||
|
||||
**v5.4.0 Migration**: Claude-Mem now uses a skill-based search architecture instead of MCP tools, saving ~2,250 tokens per session start through progressive disclosure.
|
||||
|
||||
**Simple Usage:**
|
||||
- Just ask naturally: *"What did we do last session?"*
|
||||
- Claude recognizes the intent and invokes the search skill
|
||||
- The skill uses HTTP API endpoints to query your memory
|
||||
- Results are formatted and presented to you
|
||||
|
||||
**Benefits:**
|
||||
- **Token Efficient**: ~250 tokens (skill frontmatter) vs ~2,500 tokens (MCP tool definitions)
|
||||
- **Natural Language**: No need to learn specific tool syntax
|
||||
- **Progressive Disclosure**: Only loads detailed instructions when needed
|
||||
- **Auto-Invoked**: Claude knows when to search based on your questions
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Operation | Purpose |
|
||||
|-------------------------|----------------------------------------------|
|
||||
| Search Observations | Full-text search across observations |
|
||||
| Search Sessions | Full-text search across session summaries |
|
||||
| Search Prompts | Full-text search across raw user prompts |
|
||||
| By Concept | Find observations tagged with concepts |
|
||||
| By File | Find observations referencing files |
|
||||
| By Type | Find observations by type |
|
||||
| Recent Context | Get recent session context |
|
||||
| Timeline | Get unified timeline around a specific point |
|
||||
| Timeline by Query | Search and get timeline context in one step |
|
||||
| API Help | Get search API documentation |
|
||||
|
||||
## Example Queries
|
||||
|
||||
### Natural Language Queries
|
||||
|
||||
**Search Observations:**
|
||||
```
|
||||
"What bugs did we fix related to authentication?"
|
||||
"Show me all decisions about the build system"
|
||||
"Find refactoring work on the database"
|
||||
```
|
||||
|
||||
**Search Sessions:**
|
||||
```
|
||||
"What did we learn about hooks?"
|
||||
"What was accomplished in the API implementation?"
|
||||
"Show me recent work on this project"
|
||||
```
|
||||
|
||||
**Search Prompts:**
|
||||
```
|
||||
"When did I ask about authentication features?"
|
||||
"Find all my requests about dark mode"
|
||||
```
|
||||
|
||||
**Note**: Claude automatically translates your natural language queries into the appropriate search operations.
|
||||
|
||||
### Search by File
|
||||
|
||||
```
|
||||
"Show me everything related to worker-service.ts"
|
||||
"What changes were made to migrations.ts?"
|
||||
"Find all work on the database file"
|
||||
```
|
||||
|
||||
### Search by Concept
|
||||
|
||||
```
|
||||
"Show observations tagged with architecture"
|
||||
"Find all security-related observations"
|
||||
"What patterns have we used?"
|
||||
```
|
||||
|
||||
### Search by Type
|
||||
|
||||
```
|
||||
"Find all feature implementations"
|
||||
"Show me all decisions and discoveries"
|
||||
"What bugs have we fixed?"
|
||||
```
|
||||
|
||||
### Recent Context
|
||||
|
||||
```
|
||||
"Show me what we've been working on"
|
||||
"Get context from the last 5 sessions"
|
||||
"What happened recently on this project?"
|
||||
```
|
||||
|
||||
### Timeline Queries
|
||||
|
||||
**Get timeline around a specific point:**
|
||||
```
|
||||
"What was happening when we implemented authentication?"
|
||||
"Show me the context around that bug fix"
|
||||
"What led to the decision to refactor the database?"
|
||||
```
|
||||
|
||||
**Timeline by query:**
|
||||
```
|
||||
"Find when we added the viewer UI and show what happened around that time"
|
||||
"Search for authentication work and show the timeline"
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- See the complete narrative arc around key events
|
||||
- All record types (observations, sessions, prompts) in chronological view
|
||||
- Understand what was happening before and after important changes
|
||||
|
||||
## Search Strategy
|
||||
|
||||
The search skill uses a progressive disclosure pattern to efficiently retrieve information:
|
||||
|
||||
### 1. Ask Naturally
|
||||
|
||||
Start with a natural language question:
|
||||
```
|
||||
"What bugs did we fix related to authentication?"
|
||||
```
|
||||
|
||||
### 2. Claude Invokes Search Skill
|
||||
|
||||
Claude recognizes your intent and loads the search skill (~250 tokens for skill frontmatter).
|
||||
|
||||
### 3. Skill Uses HTTP API
|
||||
|
||||
The skill calls the appropriate HTTP endpoint (e.g., `/api/search/observations`) with the query.
|
||||
|
||||
### 4. Results Formatted
|
||||
|
||||
Results are formatted and presented to you, usually starting with an index/summary format.
|
||||
|
||||
### 5. Deep Dive if Needed
|
||||
|
||||
If you need more details, ask follow-up questions:
|
||||
```
|
||||
"Tell me more about observation #123"
|
||||
"Show me the full details of that decision"
|
||||
```
|
||||
|
||||
**Benefits of This Approach:**
|
||||
- **Token Efficient**: Only loads what you need, when you need it
|
||||
- **Natural**: No syntax to learn
|
||||
- **Progressive**: Start with overview, drill down as needed
|
||||
- **Automatic**: Claude handles the search invocation
|
||||
|
||||
## Advanced Filtering
|
||||
|
||||
You can refine searches using natural language filters:
|
||||
|
||||
### Date Ranges
|
||||
|
||||
```
|
||||
"What bugs did we fix in October?"
|
||||
"Show me work from last week"
|
||||
"Find decisions made between October 1-31"
|
||||
```
|
||||
|
||||
### Multiple Types
|
||||
|
||||
```
|
||||
"Show me all decisions and features"
|
||||
"Find bugfixes and refactorings"
|
||||
```
|
||||
|
||||
### Concepts
|
||||
|
||||
```
|
||||
"Find database work related to architecture and performance"
|
||||
"Show security observations"
|
||||
```
|
||||
|
||||
### File-Specific
|
||||
|
||||
```
|
||||
"Show refactoring work that touched worker-service.ts"
|
||||
"Find changes to auth files"
|
||||
```
|
||||
|
||||
### Project Filtering
|
||||
|
||||
```
|
||||
"Show authentication work on my-app project"
|
||||
"What have we done on this codebase?"
|
||||
```
|
||||
|
||||
**Note**: Claude translates your natural language into the appropriate API filters automatically.
|
||||
|
||||
## Under the Hood: HTTP API
|
||||
|
||||
The search skill uses HTTP endpoints on the worker service (port 37777):
|
||||
|
||||
- `GET /api/search/observations` - Full-text search observations
|
||||
- `GET /api/search/sessions` - Full-text search session summaries
|
||||
- `GET /api/search/prompts` - Full-text search user prompts
|
||||
- `GET /api/search/by-concept` - Find observations by concept tag
|
||||
- `GET /api/search/by-file` - Find work related to specific files
|
||||
- `GET /api/search/by-type` - Find observations by type
|
||||
- `GET /api/context/recent` - Get recent session context
|
||||
- `GET /api/context/timeline` - Get timeline around specific point
|
||||
- `GET /api/timeline/by-query` - Search + timeline in one call
|
||||
- `GET /api/search/help` - API documentation
|
||||
|
||||
These endpoints use FTS5 full-text search with support for:
|
||||
- Boolean operators (AND, OR, NOT)
|
||||
- Phrase searches
|
||||
- Column-specific searches
|
||||
- Date range filtering
|
||||
- Project filtering
|
||||
|
||||
## Result Metadata
|
||||
|
||||
All results include rich metadata:
|
||||
|
||||
```
|
||||
## JWT authentication decision
|
||||
|
||||
**Type**: decision
|
||||
**Date**: 2025-10-21 14:23:45
|
||||
**Concepts**: authentication, security, architecture
|
||||
**Files Read**: src/auth/middleware.ts, src/utils/jwt.ts
|
||||
**Files Modified**: src/auth/jwt-strategy.ts
|
||||
|
||||
**Narrative**:
|
||||
Decided to implement JWT-based authentication instead of session-based
|
||||
authentication for better scalability and stateless design...
|
||||
|
||||
**Facts**:
|
||||
• JWT tokens expire after 1 hour
|
||||
• Refresh tokens stored in httpOnly cookies
|
||||
• Token signing uses RS256 algorithm
|
||||
• Public keys rotated every 30 days
|
||||
```
|
||||
|
||||
## Citations
|
||||
|
||||
All search results include citations using the `claude-mem://` URI scheme:
|
||||
|
||||
- `claude-mem://observation/123` - Specific observation
|
||||
- `claude-mem://session/abc-456` - Specific session
|
||||
- `claude-mem://user-prompt/789` - Specific user prompt
|
||||
|
||||
These citations enable referencing specific historical context in your work.
|
||||
|
||||
## Token Management
|
||||
|
||||
### Token Efficiency Tips
|
||||
|
||||
1. **Start with index format**: ~50-100 tokens per result
|
||||
2. **Use small limits**: Start with 3-5 results
|
||||
3. **Apply filters**: Narrow results before searching
|
||||
4. **Paginate**: Use offset to browse results in batches
|
||||
|
||||
### Token Estimates
|
||||
|
||||
| Format | Tokens per Result |
|
||||
|--------|-------------------|
|
||||
| Index | 50-100 |
|
||||
| Full | 500-1000 |
|
||||
|
||||
**Example**:
|
||||
- 20 results in index format: ~1,000-2,000 tokens
|
||||
- 20 results in full format: ~10,000-20,000 tokens
|
||||
|
||||
## Common Use Cases
|
||||
|
||||
### 1. Debugging Issues
|
||||
|
||||
Find what went wrong:
|
||||
```
|
||||
search_observations with query="error database connection" and type="bugfix"
|
||||
```
|
||||
|
||||
### 2. Understanding Decisions
|
||||
|
||||
Review architectural choices:
|
||||
```
|
||||
find_by_type with type="decision" and format="index"
|
||||
```
|
||||
|
||||
Then deep dive on specific decisions:
|
||||
```
|
||||
search_observations with query="[DECISION TITLE]" and format="full"
|
||||
```
|
||||
|
||||
### 3. Code Archaeology
|
||||
|
||||
Find when a file was modified:
|
||||
```
|
||||
find_by_file with filePath="worker-service.ts"
|
||||
```
|
||||
|
||||
### 4. Feature History
|
||||
|
||||
Track feature development:
|
||||
```
|
||||
search_sessions with query="authentication feature"
|
||||
search_user_prompts with query="add authentication"
|
||||
```
|
||||
|
||||
### 5. Learning from Past Work
|
||||
|
||||
Review refactoring patterns:
|
||||
```
|
||||
find_by_type with type="refactor" and limit=10
|
||||
```
|
||||
|
||||
### 6. Context Recovery
|
||||
|
||||
Restore context after time away:
|
||||
```
|
||||
get_recent_context with limit=5
|
||||
search_sessions with query="[YOUR PROJECT NAME]" and orderBy="date_desc"
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Index first, full later**: Always start with index format
|
||||
2. **Small limits**: Start with 3-5 results to avoid token limits
|
||||
3. **Use filters**: Narrow results before searching
|
||||
4. **Specific queries**: More specific = better results
|
||||
5. **Review citations**: Use citations to reference past decisions
|
||||
6. **Date filtering**: Use date ranges for time-based searches
|
||||
7. **Type filtering**: Use types to categorize searches
|
||||
8. **Concept tags**: Use concepts for thematic searches
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### No Results Found
|
||||
|
||||
1. Check database has data:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations;"
|
||||
```
|
||||
|
||||
2. Try broader natural language query:
|
||||
```
|
||||
"Show me anything about authentication" # Broader
|
||||
vs
|
||||
"Find exact JWT authentication implementation" # Too specific
|
||||
```
|
||||
|
||||
3. Ask without filters first:
|
||||
```
|
||||
"What do we have about auth?"
|
||||
# Then narrow down
|
||||
"Show me auth-related decisions"
|
||||
```
|
||||
|
||||
### Worker Service Not Running
|
||||
|
||||
If search isn't working, check the worker service:
|
||||
|
||||
```bash
|
||||
pm2 list # Check worker status
|
||||
npm run worker:restart # Restart if needed
|
||||
npm run worker:logs # View logs
|
||||
```
|
||||
|
||||
Or use the troubleshooting skill:
|
||||
```
|
||||
/skill troubleshoot
|
||||
```
|
||||
|
||||
### Performance Issues
|
||||
|
||||
If searches seem slow:
|
||||
1. Be more specific in your queries
|
||||
2. Ask for recent work (naturally filters by date)
|
||||
3. Specify the project you're interested in
|
||||
4. Ask for fewer results initially
|
||||
|
||||
## Technical Details
|
||||
|
||||
**Architecture Change (v5.4.0)**:
|
||||
- **Before**: 9 MCP tools (~2,500 tokens in tool definitions per session start)
|
||||
- **After**: 1 search skill (~250 tokens in frontmatter, full instructions loaded on-demand)
|
||||
- **Savings**: ~2,250 tokens per session start
|
||||
- **Migration**: Transparent - users don't need to change how they ask questions
|
||||
|
||||
**How the Skill Works:**
|
||||
1. User asks a question about past work
|
||||
2. Claude recognizes the intent matches the search skill description
|
||||
3. Skill loads full instructions from `plugin/skills/search/SKILL.md`
|
||||
4. Skill uses `curl` to call HTTP API endpoints
|
||||
5. Results formatted and returned to Claude
|
||||
6. Claude presents results to user
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Architecture Overview](/architecture/overview) - System components
|
||||
- [Database Schema](/architecture/database) - Understanding the data
|
||||
- [Getting Started](/usage/getting-started) - Automatic operation
|
||||
Reference in New Issue
Block a user