Alex Newman
5d64df2ba5
Documentation Updates: - README.md: Updated version badge, What's New, and search section - docs/usage/search-tools.mdx: Rewrote for skill-based natural language approach - docs/architecture/mcp-search.mdx → search-architecture.mdx: Complete rewrite - docs/architecture/overview.mdx: Updated components and search pipeline - docs/usage/getting-started.mdx: Added skill-based search section - docs/configuration.mdx: Updated search configuration for v5.4.0 - docs/introduction.mdx: Updated key features - docs/docs.json: Updated navigation to search-architecture Key Changes: - Emphasized ~2,250 token savings per session start - Converted all examples to natural language queries - Documented 10 HTTP API endpoints - Explained progressive disclosure pattern - Added migration notes (transparent, no user action required) - Removed outdated MCP references 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
231 lines
12 KiB
Plaintext
231 lines
12 KiB
Plaintext
---
|
|
title: "Architecture Overview"
|
|
description: "System components and data flow in Claude-Mem"
|
|
---
|
|
|
|
# Architecture Overview
|
|
|
|
## System Components
|
|
|
|
Claude-Mem operates as a Claude Code plugin with five core components:
|
|
|
|
1. **Plugin Hooks** - Capture lifecycle events (7 hook files)
|
|
2. **Worker Service** - Process observations via Claude Agent SDK + HTTP API (10 search endpoints)
|
|
3. **Database Layer** - Store sessions and observations (SQLite + FTS5 + ChromaDB)
|
|
4. **Search Skill** - Skill-based search with progressive disclosure (v5.4.0+)
|
|
5. **Viewer UI** - Web-based real-time memory stream visualization
|
|
|
|
## Technology Stack
|
|
|
|
| Layer | Technology |
|
|
|------------------------|-------------------------------------------|
|
|
| **Language** | TypeScript (ES2022, ESNext modules) |
|
|
| **Runtime** | Node.js 18+ |
|
|
| **Database** | SQLite 3 with better-sqlite3 driver |
|
|
| **Vector Store** | ChromaDB (optional, for semantic search) |
|
|
| **HTTP Server** | Express.js 4.18 |
|
|
| **Real-time** | Server-Sent Events (SSE) |
|
|
| **UI Framework** | React + TypeScript |
|
|
| **AI SDK** | @anthropic-ai/claude-agent-sdk |
|
|
| **Build Tool** | esbuild (bundles TypeScript) |
|
|
| **Process Manager** | PM2 |
|
|
| **Testing** | Node.js built-in test runner |
|
|
|
|
## Data Flow
|
|
|
|
### Memory Pipeline
|
|
```
|
|
Hook (stdin) → Database → Worker Service → SDK Processor → Database → Next Session Hook
|
|
```
|
|
|
|
1. **Input**: Claude Code sends tool execution data via stdin to hooks
|
|
2. **Storage**: Hooks write observations to SQLite database
|
|
3. **Processing**: Worker service reads observations, processes via SDK
|
|
4. **Output**: Processed summaries written back to database
|
|
5. **Retrieval**: Next session's context hook reads summaries from database
|
|
|
|
### Search Pipeline (v5.4.0+)
|
|
```
|
|
User Query → Skill Invoked → HTTP API → SessionSearch Service → FTS5 Database → Search Results → Claude
|
|
```
|
|
|
|
1. **User Query**: User asks naturally: "What bugs did we fix?"
|
|
2. **Skill Invoked**: Claude recognizes intent and invokes search skill
|
|
3. **HTTP API**: Skill uses curl to call HTTP endpoint (e.g., `/api/search/observations`)
|
|
4. **SessionSearch**: Worker service queries FTS5 virtual tables
|
|
5. **Format**: Results formatted and returned to skill
|
|
6. **Return**: Claude presents formatted results to user
|
|
|
|
**Token Savings**: ~2,250 tokens per session vs MCP approach through progressive disclosure
|
|
|
|
## Session Lifecycle
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ 0. Smart Install Hook Fires │
|
|
│ Checks dependencies (cached), only runs on version changes │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
↓
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ 1. Session Starts → Context Hook Fires │
|
|
│ Starts PM2 worker if needed, injects context from previous │
|
|
│ sessions (configurable observation count) │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
↓
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ 2. User Types Prompt → UserPromptSubmit Hook Fires │
|
|
│ Creates session in database, saves raw user prompt for FTS5 │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
↓
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ 3. Claude Uses Tools → PostToolUse Hook Fires (100+ times) │
|
|
│ Captures tool executions, sends to worker for AI compression │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
↓
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ 4. Worker Processes → Claude Agent SDK Analyzes │
|
|
│ Extracts structured learnings via iterative AI processing │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
↓
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ 5. Claude Stops → Summary Hook Fires │
|
|
│ Generates final summary with request, completions, learnings │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
↓
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ 6. Session Ends → Cleanup Hook Fires │
|
|
│ Marks session complete (graceful, not DELETE), ready for │
|
|
│ next session context. Skips on /clear to preserve ongoing │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Directory Structure
|
|
|
|
```
|
|
claude-mem/
|
|
├── src/
|
|
│ ├── hooks/ # Hook implementations (7 hooks)
|
|
│ │ ├── smart-install.ts # Dependency check (cached)
|
|
│ │ ├── context-hook.ts # SessionStart
|
|
│ │ ├── user-message-hook.ts # UserMessage (for debugging)
|
|
│ │ ├── new-hook.ts # UserPromptSubmit
|
|
│ │ ├── save-hook.ts # PostToolUse
|
|
│ │ ├── summary-hook.ts # Stop
|
|
│ │ └── cleanup-hook.ts # SessionEnd
|
|
│ │
|
|
│ ├── sdk/ # Claude Agent SDK integration
|
|
│ │ ├── prompts.ts # XML prompt builders
|
|
│ │ ├── parser.ts # XML response parser
|
|
│ │ └── worker.ts # Main SDK agent loop
|
|
│ │
|
|
│ ├── services/
|
|
│ │ ├── worker-service.ts # Express HTTP + SSE service
|
|
│ │ └── sqlite/ # Database layer
|
|
│ │ ├── SessionStore.ts # CRUD operations
|
|
│ │ ├── SessionSearch.ts # FTS5 search service
|
|
│ │ ├── migrations.ts
|
|
│ │ └── types.ts
|
|
│ │
|
|
│ ├── ui/ # Viewer UI
|
|
│ │ └── viewer/ # React + TypeScript web interface
|
|
│ │ ├── components/ # UI components
|
|
│ │ ├── hooks/ # React hooks
|
|
│ │ ├── utils/ # Utilities
|
|
│ │ └── assets/ # Fonts, logos
|
|
│ │
|
|
│ ├── shared/ # Shared utilities
|
|
│ │ ├── config.ts
|
|
│ │ ├── paths.ts
|
|
│ │ └── storage.ts
|
|
│ │
|
|
│ └── utils/
|
|
│ ├── logger.ts
|
|
│ ├── platform.ts
|
|
│ └── port-allocator.ts
|
|
│
|
|
├── plugin/ # Plugin distribution
|
|
│ ├── .claude-plugin/
|
|
│ │ └── plugin.json
|
|
│ ├── hooks/
|
|
│ │ └── hooks.json
|
|
│ ├── scripts/ # Built executables
|
|
│ │ ├── smart-install.js
|
|
│ │ ├── context-hook.js
|
|
│ │ ├── user-message-hook.js
|
|
│ │ ├── new-hook.js
|
|
│ │ ├── save-hook.js
|
|
│ │ ├── summary-hook.js
|
|
│ │ ├── cleanup-hook.js
|
|
│ │ └── worker-service.cjs # Background worker + HTTP API
|
|
│ │
|
|
│ ├── skills/ # Agent skills (v5.4.0+)
|
|
│ │ ├── search/ # Search skill with progressive disclosure
|
|
│ │ │ ├── SKILL.md # Skill frontmatter (~250 tokens)
|
|
│ │ │ └── operations/ # Detailed operation docs
|
|
│ │ ├── troubleshoot/ # Troubleshooting skill
|
|
│ │ └── version-bump/ # Version management skill
|
|
│ │
|
|
│ └── ui/ # Built viewer UI
|
|
│ └── viewer.html # Self-contained bundle
|
|
│
|
|
├── tests/ # Test suite
|
|
├── docs/ # Documentation
|
|
└── ecosystem.config.cjs # PM2 configuration
|
|
```
|
|
|
|
## Component Details
|
|
|
|
### 1. Plugin Hooks (7 Hooks)
|
|
- **smart-install.js** - Cached dependency checker (only runs on version changes)
|
|
- **context-hook.js** - SessionStart: Starts PM2 worker, injects context
|
|
- **user-message-hook.js** - UserMessage: Debugging hook
|
|
- **new-hook.js** - UserPromptSubmit: Creates session, saves prompt
|
|
- **save-hook.js** - PostToolUse: Captures tool executions
|
|
- **summary-hook.js** - Stop: Generates session summary
|
|
- **cleanup-hook.js** - SessionEnd: Marks session complete
|
|
|
|
See [Plugin Hooks](/architecture/hooks) for detailed hook documentation.
|
|
|
|
### 2. Worker Service
|
|
Express.js HTTP server on port 37777 (configurable) with:
|
|
- 10 search HTTP API endpoints (v5.4.0+)
|
|
- 8 viewer UI HTTP/SSE endpoints
|
|
- Async observation processing via Claude Agent SDK
|
|
- Real-time updates via Server-Sent Events
|
|
- Auto-managed by PM2 process manager
|
|
|
|
See [Worker Service](/architecture/worker-service) for HTTP API and endpoints.
|
|
|
|
### 3. Database Layer
|
|
SQLite3 with better-sqlite3 driver featuring:
|
|
- FTS5 virtual tables for full-text search
|
|
- SessionStore for CRUD operations
|
|
- SessionSearch for FTS5 queries
|
|
- Location: `~/.claude-mem/claude-mem.db`
|
|
|
|
See [Database Architecture](/architecture/database) for schema and FTS5 search.
|
|
|
|
### 4. Search Skill (v5.4.0+)
|
|
Skill-based search with progressive disclosure providing 10 search operations:
|
|
- Search observations, sessions, prompts (full-text FTS5)
|
|
- Filter by type, concept, file
|
|
- Get recent context, timeline, timeline by query
|
|
- API help documentation
|
|
|
|
**Token Savings**: ~2,250 tokens per session vs MCP approach
|
|
- Skill frontmatter: ~250 tokens (loaded at session start)
|
|
- Full instructions: ~2,500 tokens (loaded on-demand when invoked)
|
|
- HTTP API endpoints instead of MCP tools
|
|
|
|
See [Search Architecture](/architecture/search-architecture) for technical details and examples.
|
|
|
|
### 5. Viewer UI
|
|
React + TypeScript web interface at http://localhost:37777 featuring:
|
|
- Real-time memory stream via Server-Sent Events
|
|
- Infinite scroll pagination with automatic deduplication
|
|
- Project filtering and settings persistence
|
|
- GPU-accelerated animations
|
|
- Self-contained HTML bundle (viewer.html)
|
|
|
|
Built with esbuild into a single file deployment.
|