docs: Update all documentation to reflect v5.4.0 skill-based search

Documentation Updates:
- README.md: Updated version badge, What's New, and search section
- docs/usage/search-tools.mdx: Rewrote for skill-based natural language approach
- docs/architecture/mcp-search.mdx → search-architecture.mdx: Complete rewrite
- docs/architecture/overview.mdx: Updated components and search pipeline
- docs/usage/getting-started.mdx: Added skill-based search section
- docs/configuration.mdx: Updated search configuration for v5.4.0
- docs/introduction.mdx: Updated key features
- docs/docs.json: Updated navigation to search-architecture

Key Changes:
- Emphasized ~2,250 token savings per session start
- Converted all examples to natural language queries
- Documented 10 HTTP API endpoints
- Explained progressive disclosure pattern
- Added migration notes (transparent, no user action required)
- Removed outdated MCP references

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Alex Newman
2025-11-09 19:11:22 -05:00
parent 22a04ac461
commit 5d64df2ba5
9 changed files with 721 additions and 782 deletions
+35 -23
View File
@@ -10,9 +10,9 @@ description: "System components and data flow in Claude-Mem"
Claude-Mem operates as a Claude Code plugin with five core components:
1. **Plugin Hooks** - Capture lifecycle events (7 hook files)
2. **Worker Service** - Process observations via Claude Agent SDK
3. **Database Layer** - Store sessions and observations (SQLite + FTS5)
4. **MCP Search Server** - Query historical context (9 search tools)
2. **Worker Service** - Process observations via Claude Agent SDK + HTTP API (10 search endpoints)
3. **Database Layer** - Store sessions and observations (SQLite + FTS5 + ChromaDB)
4. **Search Skill** - Skill-based search with progressive disclosure (v5.4.0+)
5. **Viewer UI** - Web-based real-time memory stream visualization
## Technology Stack
@@ -44,16 +44,19 @@ Hook (stdin) → Database → Worker Service → SDK Processor → Database →
4. **Output**: Processed summaries written back to database
5. **Retrieval**: Next session's context hook reads summaries from database
### Search Pipeline
### Search Pipeline (v5.4.0+)
```
Claude Request → MCP Server → SessionSearch Service → FTS5 Database → Search Results → Claude
User Query → Skill Invoked → HTTP API → SessionSearch Service → FTS5 Database → Search Results → Claude
```
1. **Query**: Claude uses MCP search tools (e.g., `search_observations`)
2. **Search**: MCP server calls SessionSearch service with query parameters
3. **FTS5**: Full-text search executes against FTS5 virtual tables
4. **Format**: Results formatted as `search_result` blocks with citations
5. **Return**: Claude receives citable search results for analysis
1. **User Query**: User asks naturally: "What bugs did we fix?"
2. **Skill Invoked**: Claude recognizes intent and invokes search skill
3. **HTTP API**: Skill uses curl to call HTTP endpoint (e.g., `/api/search/observations`)
4. **SessionSearch**: Worker service queries FTS5 virtual tables
5. **Format**: Results formatted and returned to skill
6. **Return**: Claude presents formatted results to user
**Token Savings**: ~2,250 tokens per session vs MCP approach through progressive disclosure
## Session Lifecycle
@@ -110,9 +113,6 @@ claude-mem/
│ │ ├── summary-hook.ts # Stop
│ │ └── cleanup-hook.ts # SessionEnd
│ │
│ ├── servers/ # MCP servers
│ │ └── search-server.ts # MCP search tools server (9 tools)
│ │
│ ├── sdk/ # Claude Agent SDK integration
│ │ ├── prompts.ts # XML prompt builders
│ │ ├── parser.ts # XML response parser
@@ -146,7 +146,6 @@ claude-mem/
├── plugin/ # Plugin distribution
│ ├── .claude-plugin/
│ │ └── plugin.json
│ ├── .mcp.json # MCP server configuration
│ ├── hooks/
│ │ └── hooks.json
│ ├── scripts/ # Built executables
@@ -157,8 +156,14 @@ claude-mem/
│ │ ├── save-hook.js
│ │ ├── summary-hook.js
│ │ ├── cleanup-hook.js
│ │ ── worker-service.cjs # Background worker
│ │ └── search-server.mjs # MCP search server
│ │ ── worker-service.cjs # Background worker + HTTP API
│ │
│ ├── skills/ # Agent skills (v5.4.0+)
│ │ ├── search/ # Search skill with progressive disclosure
│ │ │ ├── SKILL.md # Skill frontmatter (~250 tokens)
│ │ │ └── operations/ # Detailed operation docs
│ │ ├── troubleshoot/ # Troubleshooting skill
│ │ └── version-bump/ # Version management skill
│ │
│ └── ui/ # Built viewer UI
│ └── viewer.html # Self-contained bundle
@@ -183,7 +188,8 @@ See [Plugin Hooks](/architecture/hooks) for detailed hook documentation.
### 2. Worker Service
Express.js HTTP server on port 37777 (configurable) with:
- 8 HTTP/SSE endpoints for viewer UI
- 10 search HTTP API endpoints (v5.4.0+)
- 8 viewer UI HTTP/SSE endpoints
- Async observation processing via Claude Agent SDK
- Real-time updates via Server-Sent Events
- Auto-managed by PM2 process manager
@@ -199,13 +205,19 @@ SQLite3 with better-sqlite3 driver featuring:
See [Database Architecture](/architecture/database) for schema and FTS5 search.
### 4. MCP Search Server (9 Tools)
Provides 9 specialized search tools:
- search_observations, search_sessions, search_user_prompts
- find_by_concept, find_by_file, find_by_type
- get_recent_context, get_context_timeline, get_timeline_by_query
### 4. Search Skill (v5.4.0+)
Skill-based search with progressive disclosure providing 10 search operations:
- Search observations, sessions, prompts (full-text FTS5)
- Filter by type, concept, file
- Get recent context, timeline, timeline by query
- API help documentation
See [MCP Search Server](/architecture/mcp-search) for search tools and examples.
**Token Savings**: ~2,250 tokens per session vs MCP approach
- Skill frontmatter: ~250 tokens (loaded at session start)
- Full instructions: ~2,500 tokens (loaded on-demand when invoked)
- HTTP API endpoints instead of MCP tools
See [Search Architecture](/architecture/search-architecture) for technical details and examples.
### 5. Viewer UI
React + TypeScript web interface at http://localhost:37777 featuring: