Compare commits

...

22 Commits

Author SHA1 Message Date
Alex Newman 58f4053a61 chore: Bump version to 6.2.0 2025-11-22 00:27:34 -05:00
claude[bot] a08e5068c8 fix: Update version to 6.1.1 across package.json and CLAUDE.md
Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
2025-11-22 04:44:20 +00:00
claude[bot] 9f1b653f2f fix: Update CLAUDE.md version to match package.json (6.0.9)
Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
2025-11-22 04:17:31 +00:00
copilot-swe-agent[bot] c5d89142b9 chore: Update package-lock.json after merge and build
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-22 00:18:14 +00:00
copilot-swe-agent[bot] b44d4702b1 Merge main branch, prefer this branch's version except simpler CLAUDE.md 2025-11-22 00:17:24 +00:00
copilot-swe-agent[bot] 4988b4b317 Initial plan 2025-11-22 00:12:58 +00:00
Alex Newman c5e68a17c8 refactor: Clean up search architecture, remove experimental contextualize endpoint (#133)
* Refactor code structure for improved readability and maintainability

* Add test results for search API and related functionalities

- Created test result files for various search-related functionalities, including:
  - test-11-search-server-changes.json
  - test-12-context-hook-changes.json
  - test-13-worker-service-changes.json
  - test-14-patterns.json
  - test-15-gotchas.json
  - test-16-discoveries.json
  - test-17-all-bugfixes.json
  - test-18-all-features.json
  - test-19-all-decisions.json
  - test-20-session-search.json
  - test-21-prompt-search.json
  - test-22-decisions-endpoint.json
  - test-23-changes-endpoint.json
  - test-24-how-it-works-endpoint.json
  - test-25-contextualize-endpoint.json
  - test-26-timeline-around-observation.json
  - test-27-multi-param-combo.json
  - test-28-file-type-combo.json

- Each test result file captures specific search failures or outcomes, including issues with undefined properties and successful execution of search queries.
- Enhanced documentation of search architecture and testing strategies, ensuring compliance with established guidelines and improving overall search functionality.

* feat: Enhance unified search API with catch-all parameters and backward compatibility

- Implemented a unified search API at /api/search that accepts catch-all parameters for filtering by type, observation type, concepts, and files.
- Maintained backward compatibility by keeping granular endpoints functional while routing through the same infrastructure.
- Completed comprehensive testing of search capabilities with real-world query scenarios.

fix: Address missing debug output in search API query tests

- Flushed PM2 logs and executed search queries to verify functionality.
- Diagnosed absence of "Raw Chroma" debug messages in worker logs, indicating potential issues with logging or query processing.

refactor: Improve build and deployment pipeline for claude-mem plugin

- Successfully built and synced all hooks and services to the marketplace directory.
- Ensured all dependencies are installed and up-to-date in the deployment location.

feat: Implement hybrid search filters with 90-day recency window

- Enhanced search server to apply a 90-day recency filter to Chroma results before categorizing by document type.

fix: Correct parameter handling in searchUserPrompts method

- Added support for filter-only queries and improved dual-path logic for clarity.

refactor: Rename FTS5 method to clarify fallback status

- Renamed escapeFTS5 to escapeFTS5_fallback_when_chroma_unavailable to indicate its temporary usage.

feat: Introduce contextualize tool for comprehensive project overview

- Added a new tool to fetch recent observations, sessions, and user prompts, providing a quick project overview.

feat: Add semantic shortcut tools for common search patterns

- Implemented 'decisions', 'changes', and 'how_it_works' tools for convenient access to frequently searched observation categories.

feat: Unified timeline tool supports anchor and query modes

- Combined get_context_timeline and get_timeline_by_query into a single interface for timeline exploration.

feat: Unified search tool added to MCP server

- New tool queries all memory types simultaneously, providing combined chronological results for improved search efficiency.

* Refactor search functionality to clarify FTS5 fallback usage

- Updated `worker-service.cjs` to replace FTS5 fallback function with a more descriptive name and improved error handling.
- Enhanced documentation in `SKILL.md` to specify the unified API endpoint and clarify the behavior of the search engine, including the conditions under which FTS5 is used.
- Modified `search-server.ts` to provide clearer logging and descriptions regarding the fallback to FTS5 when UVX/Python is unavailable.
- Renamed and updated the `SessionSearch.ts` methods to reflect the conditions for using FTS5, emphasizing the lack of semantic understanding in fallback scenarios.

* feat: Add ID-based fetch endpoints and simplify mem-search skill

**Problem:**
- Search returns IDs but no way to fetch by ID
- Skill documentation was bloated with too many options
- Claude wasn't using IDs because we didn't tell it how

**Solution:**
1. Added three new HTTP endpoints:
   - GET /api/observation/:id
   - GET /api/session/:id
   - GET /api/prompt/:id

2. Completely rewrote SKILL.md:
   - Stripped complexity down to essentials
   - Clear 3-step prescriptive workflow: Search → Review IDs → Fetch by ID
   - Emphasized ID usage: "The IDs are there for a reason - USE THEM"
   - Removed confusing multi-endpoint documentation
   - Kept only unified search with filters

**Impact:**
- Token efficiency: Claude can now fetch full details only for relevant IDs
- Clarity: One clear workflow instead of 10+ options to choose from
- Usability: IDs are no longer wasted context - they're actionable

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: Move internal docs to private directory

Moved POSTMORTEM and planning docs to ./private to exclude from PR reviews.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Remove experimental contextualize endpoint

- Removed contextualize MCP tool from search-server (saves ~4KB)
- Disabled FTS5 fallback paths in SessionSearch (now vector-first)
- Cleaned up CLAUDE.md documentation
- Removed contextualize-rewrite-plan.md doc

Rationale:
- Contextualize is better suited as a skill (LLM-powered) than an endpoint
- Search API already provides vector search with configurable limits
- Created issue #132 to track future contextualize skill implementation

Changes:
- src/servers/search-server.ts: Removed contextualize tool definition
- src/services/sqlite/SessionSearch.ts: Disabled FTS5 fallback, added deprecation warnings
- CLAUDE.md: Cleaned up outdated skill documentation
- docs/: Removed contextualize plan document

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Complete FTS5 cleanup - remove all deprecated search code

This completes the FTS5 cleanup work by removing all commented-out
FTS5 search code while preserving database tables for backward compatibility.

Changes:
- Removed 200+ lines of commented FTS5 search code from SessionSearch.ts
- Removed deprecated degraded_search_query__when_uvx_unavailable method
- Updated all method documentation to clarify vector-first architecture
- Updated class documentation to reflect filter-only query support
- Updated CLAUDE.md to remove FTS5 search references
- Clarified that FTS5 tables exist for backward compatibility only
- Updated "Why SQLite FTS5" section to "Why Vector-First Search"

Database impact: NONE - FTS5 tables remain intact for existing installations

Search architecture:
- ChromaDB: All text-based vector search queries
- SQLite: Filter-only queries (date ranges, metadata, no query text)
- FTS5 tables: Maintained but unused (backward compatibility)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Remove all FTS5 fallback execution code from search-server

Completes the FTS5 cleanup by removing all fallback execution paths
that attempted to use FTS5 when ChromaDB was unavailable.

Changes:
- Removed all FTS5 fallback code execution paths
- When ChromaDB fails or is unavailable, return empty results with helpful error messages
- Updated all deprecated tool descriptions (search_observations, search_sessions, search_user_prompts)
- Changed error messages to indicate FTS5 fallback has been removed
- Added installation instructions for UVX/Python when vector search is unavailable
- Updated comments from "hybrid search" to "vector-first search"
- Removed ~100 lines of dead FTS5 fallback code

Database impact: NONE - FTS5 tables remain intact (backward compatibility)

Search behavior when ChromaDB unavailable:
- Text queries: Return empty results with error explaining ChromaDB is required
- Filter-only queries (no text): Continue to work via direct SQLite

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Address PR 133 review feedback

Critical fixes:
- Remove contextualize endpoint from worker-service (route + handler)
- Fix build script logging to show correct .cjs extension (was .mjs)

Documentation improvements:
- Add comprehensive FTS5 retention rationale documentation
- Include v7.0.0 removal TODO for future cleanup

Testing:
- Build succeeds with correct output logging
- Worker restarts successfully (30th restart)
- Contextualize endpoint properly removed (404 response)
- Search endpoint verified working

This addresses all critical review feedback from PR 133.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-21 18:59:23 -05:00
Alex Newman f35764aa14 Update link for AI Memory Economics analysis (#143) 2025-11-21 00:59:54 -05:00
Alex Newman 601bded789 docs: Update CHANGELOG.md for v6.1.1 2025-11-20 20:40:40 -05:00
Alex Newman c231e8d076 chore: Bump version to 6.1.1 2025-11-20 20:39:40 -05:00
Alex Newman b62e93577c fix: Use dynamic project name detection for ChromaDB collections and observations (#142)
* fix: Use dynamic project name detection instead of hardcoded values

Fixes #140

Previously, the worker process used hardcoded "claude-mem" for:
- ChromaSync instantiation in DatabaseManager
- ChromaDB collection naming in search-server

This caused all observations to be tagged with "claude-mem" regardless
of the actual project being worked on.

Now both services use getCurrentProjectName() to dynamically detect the
project based on the worker's current working directory.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Simplify viewer UI overflow CSS to enable scrolling

- Remove overcomplicated nested overflow containers
- Use explicit 100vh for layout height
- Add overflow: hidden to main-col to constrain feed
- Keep simple overflow-y: auto on feed element
- Fix issue where feed content wouldn't scroll

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-20 20:37:57 -05:00
Alex Newman 0787e47b1a Add AI Memory Economics analysis to README
Added a new section for AI Memory Economics analysis with key insights and a link.
2025-11-20 00:47:44 -05:00
Alex Newman f9843fe593 docs: Compress CLAUDE.md to bare essentials
Removed verbose explanations and kept only critical reference info:
- What the project is
- Architecture overview (hooks, worker, database, skills, chroma, viewer)
- Build commands for different scenarios
- Environment variables
- File locations
- Quick reference commands

Moved general coding standards to ~/.claude/CLAUDE.md (global config).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-19 18:12:27 -05:00
Alex Newman 5b772ee768 docs: Fix v6.1.0 release notes accuracy 2025-11-19 17:24:21 -05:00
Alex Newman baabf14bfa docs: Update CHANGELOG.md for v6.1.0
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-19 17:14:45 -05:00
Alex Newman c126a7083f chore: Bump version to 6.1.0
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-19 17:14:09 -05:00
Alex Newman f1170512d6 feat: Add responsive mobile navigation with Discord-style sidebar layout (#138)
Implements a mobile-first navigation reorganization that moves controls into the sidebar at smaller breakpoints:

- Community button moves to sidebar at 600px
- Projects dropdown moves to sidebar at 480px
- Sidebar gains proper width constraints (100% width, 400px max-width)
- Full-height layout styling fixes for proper flex behavior
- Clean separation between header and sidebar responsibilities

This creates a Discord-like mobile experience where the sidebar becomes the primary navigation container on smaller screens.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-19 17:13:24 -05:00
Alex Newman d64939c379 docs: Add semantic naming principle and clean up migration docs
Added semantic naming to coding standards emphasizing verbose,
self-documenting names for comprehension. Cleaned up database
migration section removing TODO in favor of clear evergreen statement.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 15:45:33 -05:00
Alex Newman 97d8bd3e62 docs: Update CHANGELOG.md for v6.0.9
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 15:18:25 -05:00
Alex Newman 6cd904a81c chore: Bump version to 6.0.9
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 15:17:23 -05:00
Alex Newman 74c8afd0e0 feat: Add real-time queue depth indicator to viewer UI
Implements a visual badge that displays the count of active work items (queued + currently processing) in the worker service. The badge appears next to the claude-mem logo and updates in real-time via SSE.

Features:
- Shows count of pending messages + active SDK generators
- Only displays when queueDepth > 0
- Subtle pulse animation for visual feedback
- Theme-aware styling

Backend changes:
- Added getTotalActiveWork() method to SessionManager
- Updated worker-service to broadcast queueDepth via SSE
- Enhanced processing status API endpoint

Frontend changes:
- Updated Header component to display queue bubble
- Enhanced useSSE hook to track queueDepth state
- Added CSS styling with pulse animation

Closes #122
Closes #96
Closes #97

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 15:11:37 -05:00
Alex Newman 02444392da docs: Update CHANGELOG.md for v6.0.8
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 14:43:39 -05:00
53 changed files with 2701 additions and 749 deletions
+1 -1
View File
@@ -10,7 +10,7 @@
"plugins": [
{
"name": "claude-mem",
"version": "6.0.8",
"version": "6.2.0",
"source": "./plugin",
"description": "Persistent memory system for Claude Code - context compression across sessions"
}
+53
View File
@@ -4,6 +4,59 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## [6.0.9] - 2025-11-17
## Queue Depth Indicator Feature
Added a real-time queue depth indicator to the viewer UI that displays the count of active work items (queued + currently processing).
### Features
- Visual badge next to claude-mem logo
- Shows count of pending messages + active SDK generators
- Only displays when queueDepth > 0
- Subtle pulse animation for visual feedback
- Theme-aware styling
- Real-time updates via SSE
### Implementation
- Backend: Added `getTotalActiveWork()` method to SessionManager
- Backend: Updated worker-service to broadcast queueDepth via SSE
- Frontend: Enhanced Header component to display queue bubble
- Frontend: Updated useSSE hook to track queueDepth state
- Frontend: Added CSS styling with pulse animation
### Closes
- #122 - Implement queue depth indicator feature
- #96 - Add real-time queue depth indicator to viewer UI
- #97 - Fix inconsistent queue depth calculation
### Credit
Original implementation by @thedotmack in PR #96
Bug fix by @copilot-swe-agent in PR #97
## [6.0.8] - 2025-11-17
## Critical Fix
This patch release fixes a critical bug where the PM2 worker process would start from the wrong directory (development folder instead of marketplace folder), causing the plugin to malfunction when installed via the marketplace.
### What's Fixed
- **Worker Startup Path Resolution** (`src/shared/worker-utils.ts:61`)
Added `cwd: pluginRoot` option to `execSync` when starting PM2
This ensures the worker always starts from the correct marketplace directory (`~/.claude/plugins/marketplaces/thedotmack/`), regardless of where the hook is invoked from.
### Impact
Users will no longer experience issues with the worker starting from the wrong location. The plugin now works correctly when installed via marketplace without manual intervention.
### Verification
Run `pm2 info claude-mem-worker` to verify:
- **exec cwd** should be: `/Users/[username]/.claude/plugins/marketplaces/thedotmack`
- **script path** should be: `/Users/[username]/.claude/plugins/marketplaces/thedotmack/plugin/scripts/worker-service.cjs`
## [6.0.7] - 2025-11-17
## Critical Hotfix: Database Migration Issue (#121)
+32 -185
View File
@@ -1,178 +1,38 @@
/* To @claude: be vigilant about only leaving evergreen context in this file, claude-mem handles working context separately. */
# Claude-Mem: AI Development Instructions
## What This Project Is
Claude-mem is a Claude Code plugin providing persistent memory across sessions. It captures tool usage, compresses observations using the Claude Agent SDK, and injects relevant context into future sessions.
**Your Role**: You are working on the plugin itself. When users interact with Claude Code with this plugin installed, your observations get captured and become their persistent memory.
**Current Version**: 6.2.0
**Current Version**: 6.0.8
## Architecture
## IMPORTANT: Skills Are Auto-Invoked
**5 Lifecycle Hooks**: SessionStart → UserPromptSubmit → PostToolUse → Summary → SessionEnd
**There is no `/skill` command.** Skills auto-invoke based on description metadata matching user queries. Don't document manual invocation (e.g., "Run `/skill troubleshoot`"). Instead: "The troubleshoot skill auto-activates when issues are detected."
**Hooks** (`src/hooks/*.ts`) - TypeScript → ESM, built to `plugin/scripts/*-hook.js`
## Critical Architecture Knowledge
**Worker Service** (`src/services/worker-service.ts`) - Express API on port 37777, PM2-managed, handles AI processing asynchronously
### The Lifecycle Flow
**Database** (`src/services/sqlite/`) - SQLite3 at `~/.claude-mem/claude-mem.db` with FTS5 full-text search
1. **SessionStart** → smart-install.js runs first (pre-hook), then `context-hook.ts` runs
- Smart installer checks dependencies (cached, only runs on version changes)
- Starts PM2 worker if not healthy
- Injects context from previous sessions (configurable observation count)
**Search Skill** (`plugin/skills/mem-search/SKILL.md`) - HTTP API for searching past work, auto-invoked when users ask about history
2. **UserPromptSubmit**`new-hook.ts` runs
- Creates session record in SQLite
- Saves raw user prompt for FTS5 search
**Chroma** (`src/services/sync/ChromaSync.ts`) - Vector embeddings for semantic search
3. **PostToolUse**`save-hook.ts` runs
- Captures your tool executions
- Sends to worker service for AI compression
**Viewer UI** (`src/ui/viewer/`) - React interface at http://localhost:37777, built to `plugin/ui/viewer.html`
4. **Summary** → Summary hook generates session summaries
## Build Commands
5. **SessionEnd**`cleanup-hook.ts` runs
- Marks session complete (graceful, not DELETE)
- Skips on `/clear` to preserve ongoing sessions
**Hooks only**: `npm run build && npm run sync-marketplace`
**Note**: smart-install.js is a pre-hook script (not a lifecycle hook). It's called before context-hook via command chaining in hooks.json and only runs when dependencies need updating.
**Worker changes**: `npm run build && npm run sync-marketplace && npm run worker:restart`
### Key Components
**Skills only**: `npm run sync-marketplace`
**Hooks** (`src/hooks/*.ts`)
- Built to `plugin/scripts/*-hook.js` (ESM format)
- Must output valid JSON to `hookSpecificOutput` field
- Called by Claude Code lifecycle events
**Worker Service** (`src/services/worker-service.ts`)
- Express.js API on port 37777 (configurable via `CLAUDE_MEM_WORKER_PORT`)
- Managed by PM2 (auto-started by hooks)
- Built to `plugin/worker-service.cjs` (CJS format)
- Handles AI processing asynchronously to avoid hook timeouts
**Database** (`src/services/sqlite/`)
- SQLite3 with better-sqlite3 (NOT bun:sqlite - that's legacy)
- Location: `~/.claude-mem/claude-mem.db`
- FTS5 virtual tables for full-text search
- `SessionStore` = CRUD, `SessionSearch` = FTS5 queries
**Search Skill** (`plugin/skills/mem-search/SKILL.md`)
- Provides access to all search functionality via HTTP API + skill
- Auto-invoked when users ask about past work, decisions, or history
- Uses HTTP endpoints instead of MCP tools (~2,250 token savings per session)
- 10 search operations: observations, sessions, prompts, by-type, by-file, by-concept, timelines, etc.
**Chroma Vector Database** (`src/services/sync/ChromaSync.ts`)
- Hybrid semantic + keyword search architecture
- Automatic vector embedding synchronization
- 90-day recency filtering for relevant results
- Combined with SQLite FTS5 for optimal search performance
**Viewer UI** (`src/ui/viewer/`)
- React + TypeScript web interface accessible at http://localhost:37777
- Real-time memory stream visualization via Server-Sent Events (SSE)
- Infinite scroll pagination for observations, sessions, and user prompts
- Project filtering and settings persistence
- Built to `plugin/ui/viewer.html` (self-contained bundle via esbuild)
- Auto-reconnection and error recovery
## How to Make Changes
### When You Modify Hooks
```bash
npm run build
npm run sync-marketplace
```
Changes take effect on next Claude Code session. No worker restart needed.
### When You Modify Worker Service
```bash
npm run build
npm run sync-marketplace
npm run worker:restart
```
Must restart PM2 worker for changes to take effect.
### When You Modify Search Skill
```bash
npm run sync-marketplace
```
Skill changes take effect immediately on next Claude Code session. No build or restart needed (skills are markdown).
### When You Modify Viewer UI
```bash
npm run build
npm run sync-marketplace
npm run worker:restart
```
Changes to React components, styles, or viewer logic require rebuilding and restarting the worker. Refresh browser to see changes.
### Build Pipeline
1. `npm run build` → Compiles TypeScript, outputs to `plugin/`
2. `npm run sync-marketplace` → Syncs to `~/.claude/plugins/marketplaces/thedotmack/`
3. Changes are live for next session (hooks/skills) or after restart (worker)
## Coding Standards
**Philosophy**: Write the dumb, obvious thing first. Add complexity only when you hit the problem.
**Key Principles:**
1. **YAGNI**: Don't build it until you need it
2. **DRY**: Extract patterns after second duplication, not before
3. **Fail Fast**: Explicit errors beat silent failures
4. **Simple First**: Write the obvious solution, optimize only if needed
5. **Delete Aggressively**: Less code = fewer bugs
**Common anti-patterns to avoid:**
- Ceremonial wrapper functions for constants (just export the constant)
- Unused default parameters (remove if never used)
- Magic numbers without named constants
- Silent failures instead of explicit errors
- Fragile string parsing (use structured JSON output)
- Copy-pasted promise wrappers (extract helper functions)
- Overengineered "defensive" code for problems you don't have
## Common Tasks
### Adding a New Hook
1. Create `src/hooks/new-hook.ts`
2. Add to `scripts/build-hooks.js` build list
3. Add configuration to `plugin/hooks/hooks.json`
4. Build and sync: `npm run build && npm run sync-marketplace`
**Note**: smart-install.js is not a hook - it's a pre-hook dependency checker that runs before context-hook via command chaining.
### Modifying Database Schema
1. Update schema in `src/services/sqlite/schema.ts`
2. Update SessionStore/SessionSearch classes
3. Migration strategy: The plugin currently recreates on schema changes (acceptable for alpha)
4. TODO: Add proper migrations for production
### Debugging Worker Issues
```bash
pm2 list # Check worker status
npm run worker:logs # View logs
npm run worker:restart # Restart if needed
pm2 delete claude-mem-worker # Force clean start
```
### Testing Changes Locally
1. Make changes in `src/`
2. `npm run build && npm run sync-marketplace`
3. Start new Claude Code session (hooks) or restart worker (worker changes)
4. Check `~/.claude-mem/claude-mem.db` for database state
5. Use mem-search skill to verify behavior (auto-invoked when asking about past work)
### Version Bumps
Use the `version-bump` skill (auto-invokes when requesting version updates). It handles:
- Semantic version increments (patch/minor/major)
- Updates all version references (package.json, plugin.json, CLAUDE.md, marketplace.json)
- Creates git tags and GitHub releases
- Auto-generates CHANGELOG.md from releases
## Investigation Best Practices
When investigations fail persistently, use Task agents for comprehensive file analysis instead of repeated grep/search. Deploy agents to read full files and answer specific questions - more efficient than multiple rounds of searching.
**Viewer UI**: `npm run build && npm run sync-marketplace && npm run worker:restart`
## Environment Variables
@@ -180,37 +40,24 @@ When investigations fail persistently, use Task agents for comprehensive file an
- `CLAUDE_MEM_CONTEXT_OBSERVATIONS` - Observations injected at SessionStart (default: 50)
- `CLAUDE_MEM_WORKER_PORT` - Worker service port (default: 37777)
## Key Design Decisions
### Why PM2 Instead of Direct Process
Hooks have strict timeout limits. PM2 manages a persistent background worker, allowing AI processing to continue after hooks complete.
### Why SQLite FTS5
Enables instant full-text search across thousands of observations without external dependencies. Automatic sync triggers keep FTS5 tables synchronized.
### Why Graceful Cleanup
Changed from aggressive DELETE requests to marking sessions complete. Prevents interrupting summary generation and other async operations.
### Why Smart Install Caching
npm install is expensive (2-5s). Caching version state and only installing on changes makes SessionStart nearly instant (10ms).
### Why Web-Based Viewer UI
Real-time visibility into memory stream helps users understand what's being captured and how context is being built. SSE provides instant updates without polling. Self-contained HTML bundle (esbuild) eliminates deployment complexity - everything served from a single file.
## File Locations
**Source**: `<project-root>/src/` - TypeScript source files
**Built Plugin**: `<project-root>/plugin/` - Compiled JavaScript outputs
**Installed Plugin**: `~/.claude/plugins/marketplaces/thedotmack/` - User's installed plugin location
**Database**: `~/.claude-mem/claude-mem.db` - SQLite database with observations, sessions, summaries
**Chroma Database**: `~/.claude-mem/chroma/` - Vector embeddings for semantic search
**Usage Logs**: `~/.claude-mem/usage-logs/usage-YYYY-MM-DD.jsonl` - Daily API usage tracking
- **Source**: `<project-root>/src/`
- **Built Plugin**: `<project-root>/plugin/`
- **Installed Plugin**: `~/.claude/plugins/marketplaces/thedotmack/`
- **Database**: `~/.claude-mem/claude-mem.db`
- **Chroma**: `~/.claude-mem/chroma/`
- **Usage Logs**: `~/.claude-mem/usage-logs/usage-YYYY-MM-DD.jsonl`
## Quick Reference
**Build**: `npm run build`
**Sync**: `npm run sync-marketplace`
**Worker Restart**: `npm run worker:restart`
**Worker Logs**: `npm run worker:logs`
**Usage Analysis**: `npm run usage:today`
**Viewer UI**: http://localhost:37777 (auto-starts with worker)
```bash
npm run build # Compile TypeScript
npm run sync-marketplace # Copy to ~/.claude/plugins
npm run worker:restart # Restart PM2 worker
npm run worker:logs # View worker logs
pm2 list # Check worker status
pm2 delete claude-mem-worker # Force clean start
```
**Viewer UI**: http://localhost:37777
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "claude-mem",
"version": "6.0.3",
"version": "6.0.9",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "claude-mem",
"version": "6.0.3",
"version": "6.0.9",
"license": "AGPL-3.0",
"dependencies": {
"@anthropic-ai/claude-agent-sdk": "^0.1.27",
+3 -2
View File
@@ -1,6 +1,6 @@
{
"name": "claude-mem",
"version": "6.0.8",
"version": "6.2.0",
"description": "Memory compression system for Claude Code - persist context across sessions",
"keywords": [
"claude",
@@ -39,7 +39,8 @@
"worker:start": "pm2 start ecosystem.config.cjs",
"worker:stop": "pm2 stop claude-mem-worker",
"worker:restart": "pm2 restart claude-mem-worker",
"worker:logs": "pm2 logs claude-mem-worker",
"worker:logs": "pm2 flush claude-mem-worker && pm2 logs claude-mem-worker --lines 100 --nostream",
"worker:logs:no-flush": "pm2 logs claude-mem-worker --lines 100 --nostream",
"changelog:generate": "node scripts/generate-changelog.js",
"usage:analyze": "node scripts/analyze-usage.js",
"usage:today": "node scripts/analyze-usage.js $(date +%Y-%m-%d)"
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "claude-mem",
"version": "6.0.8",
"version": "6.2.0",
"description": "Persistent memory system for Claude Code - seamlessly preserve context across sessions",
"author": {
"name": "Alex Newman"
+648
View File
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+83 -156
View File
@@ -1,196 +1,123 @@
---
name: mem-search
description: Search claude-mem's persistent cross-session memory database to find work from previous conversations days, weeks, or months ago. Access past session summaries, bug fixes, feature implementations, and decisions that are NOT in the current conversation context. Use when user asks "did we already solve this?", "how did we do X last time?", "what happened in last week's session?", or needs information from previous sessions stored in the PM2-managed database. Searches observations, session summaries, and user prompts across entire project history.
description: Search claude-mem's persistent cross-session memory database. Use when user asks "did we already solve this?", "how did we do X last time?", or needs work from previous sessions.
---
# Memory Search
Access claude-mem's persistent cross-session memory through HTTP API. Find past work, understand context across sessions, and learn from previous decisions.
Search past work across all sessions. Simple workflow: search → get IDs → fetch details by ID.
## When to Use This Skill
## When to Use
**Use when users ask about work from PREVIOUS sessions** (not current conversation):
Use when users ask about PREVIOUS sessions (not current conversation):
- "Did we already fix this?"
- "How did we solve X last time?"
- "What happened last week?"
### Temporal Triggers (Key Indicators)
- "Did we **already** fix this bug?" or "Have we seen this error **before**?"
- "How did we solve X **last time**?" or "What approach did we take **previously**?"
- "What did we do in **yesterday's/last week's/last month's** session?"
- "**When** did we last work on this?" or "What's the **history** of this file?"
## The Workflow
### Cross-Session Queries
- "Show me all authentication-related changes **across all sessions**"
- "What features did we add **last month**?" (not "today" or "this session")
- "Why did we choose this approach **before**?" (decisions from past sessions)
- "What files did we modify **when we added X**?" (historical context)
**ALWAYS follow this exact flow:**
**Do NOT use** for current session work, future planning, or questions Claude can answer from current conversation context.
1. **Search** - Get an index of results with IDs
2. **Review** - Look at titles/dates, pick relevant IDs
3. **Fetch** - Get full details ONLY for those IDs
## Common Trigger Phrases
### Step 1: Search Everything
This skill activates when detecting phrases about **cross-session history**:
```bash
curl "http://localhost:37777/api/search?query=authentication&format=index&limit=5"
```
- "Did we already solve this?" / "Have we done this before?"
- "How did we implement X last time?"
- "What did we work on yesterday/last week/last month?"
- "Show me the history of [file/feature/decision]"
- "When did we fix/add/change X?"
- "What was happening around [date/time]?"
- "Catch me up on recent sessions" / "What have we been doing?"
- "What changes to [filename] across all sessions?"
**Required parameters:**
- `query` - Search term
- `format=index` - ALWAYS start with index (lightweight)
- `limit=5` - Start small (3-5 results)
**Unique identifiers:** claude-mem, persistent memory, cross-session database, session history, PM2-managed database
**Returns:**
```
1. [feature] Added JWT authentication
Date: 11/17/2025, 3:48:45 PM
ID: 11131
## Available Operations
2. [bugfix] Fixed auth token expiration
Date: 11/16/2025, 2:15:22 PM
ID: 10942
```
### Full-Text Search
- **observations** - Search all observations by keyword (bugs, features, decisions, discoveries, changes)
- Use when: "How did we implement X?" or "What bugs did we fix?"
- Example: Search for "authentication JWT" to find auth-related work
### Step 2: Pick IDs
- **sessions** - Search session summaries to find what was accomplished when
- Use when: "What did we accomplish last time?" or "What was the goal of that session?"
- Example: Find sessions where "added login feature"
Review the index results. Identify which IDs are actually relevant. Discard the rest.
- **prompts** - Find what users have asked about in previous sessions
- Use when: "Did I ask about this before?" or "What did I request last week?"
- Example: Search for "database migration" in past user prompts
### Step 3: Fetch by ID
### Filtered Search
- **by-type** - Filter observations by type (bugfix, feature, refactor, decision, discovery, change)
- Use when: "Show me all bug fixes" or "List features we added"
- Example: Get all observations with type=bugfix from last month
For each relevant ID, fetch full details:
- **by-concept** - Find observations tagged with specific concepts (problem-solution, how-it-works, gotcha)
- Use when: "What patterns did we discover?" or "Show me gotchas"
- Example: Find all observations tagged with concept "gotcha"
```bash
# Fetch observation
curl "http://localhost:37777/api/observation/11131"
- **by-file** - Find all work related to a specific file path across all sessions
- Use when: "What changes to auth.ts?" or "History of this file"
- Example: Get all work related to "src/auth/login.ts"
# Fetch session
curl "http://localhost:37777/api/session/2005"
### Timeline & Context
- **recent-context** - Get last N sessions with summaries and observations
- Use when: "What's been happening?" or "Catch me up on recent work"
- Example: Get last 3 sessions with limit=3
# Fetch prompt
curl "http://localhost:37777/api/prompt/5421"
```
- **timeline** - Get chronological context around a specific point in time (before/after window)
- Use when: "What was happening around [date]?" or "Show me context from that time"
- Example: Timeline around session 123 with depth 5 before and after
**ID formats:**
- Observations: Just the number (11131)
- Sessions: Just the number (2005) from "S2005"
- Prompts: Just the number (5421)
- **timeline-by-query** - Search first, then get timeline context around best match
- Use when: "When did we implement auth?" combined with "show me context around that time"
- Example: Search for "OAuth implementation" then get surrounding timeline
## Search Parameters
For detailed instructions on any operation, read the corresponding file in [operations/](operations/).
**Basic:**
- `query` - What to search for (required)
- `format` - "index" or "full" (always use "index" first)
- `limit` - How many results (default 5, max 100)
## Quick Decision Guide
**Filters (optional):**
- `type` - Filter to "observations", "sessions", or "prompts"
- `project` - Filter by project name
- `dateRange[start]` - Start date (YYYY-MM-DD)
- `dateRange[end]` - End date (YYYY-MM-DD)
- `obs_type` - Filter observations by: bugfix, feature, decision, discovery, change
**What is the user asking about?**
## Examples
1. **Recent work** (last 3-5 sessions) → Use **recent-context** with limit=3-5
2. **Specific topic/keyword** (bugs, features, decisions) → Use **observations** search
3. **Specific file history** (changes to a file) → Use **by-file** search
4. **Timeline/chronology** (what happened when) → Use **timeline** or **timeline-by-query**
5. **Type-specific** (all bug fixes, all features) → Use **by-type** filter
**Find recent bug fixes:**
```bash
curl "http://localhost:37777/api/search?query=bug&type=observations&obs_type=bugfix&format=index&limit=5"
```
**Most common:** Use **observations** search for general "how did we..." questions.
**Find what happened last week:**
```bash
curl "http://localhost:37777/api/search?query=&type=observations&dateRange[start]=2025-11-11&format=index&limit=10"
```
## Progressive Disclosure Workflow (Token Efficiency)
**Search everything:**
```bash
curl "http://localhost:37777/api/search?query=database+migration&format=index&limit=5"
```
**Core Principle**: Find high-signal items in index format FIRST (~50-100 tokens each), then request full details ONLY for relevant items (~500-1000 tokens each).
## Why This Workflow?
**4-Step Workflow:**
**Token efficiency:**
- Index format: ~50-100 tokens per result
- Full format: ~500-1000 tokens per result
- **10x difference** - only fetch full when you know it's relevant
1. **Start with Index Format**
- Always use `format=index` initially
- Set `limit=3-5` (not 10-20)
- Review titles and dates to assess relevance
- Token cost: ~50-100 per result
2. **Identify Relevant Items**
- Scan index results
- Discard irrelevant items from list
- Keep only 1-3 most relevant
3. **Request Full Details Selectively**
- Use `format=full` ONLY for specific relevant items
- Token cost: ~500-1000 per result
- **10x cost difference** - be selective
4. **Refine with Filters**
- Use type, dateRange, concepts, files filters
- Paginate with offset if needed
- Narrow scope before expanding limits
**DO:**
- ✅ Start with `format=index` and `limit=3-5`
- ✅ Use filters (type, dateRange, concepts, files) to narrow results
- ✅ Request `format=full` ONLY for specific relevant items
- ✅ Use offset for pagination instead of large limits
**DON'T:**
- ❌ Jump straight to `format=full`
- ❌ Request `limit=20` without assessing index results first
- ❌ Load full details for all results upfront
- ❌ Skip index format to "save a step" (costs 10x more tokens)
See [principles/progressive-disclosure.md](principles/progressive-disclosure.md) for complete workflow with examples and token calculations.
## Quick Reference Table
| Need | Operation | Key Parameters |
|------|-----------|----------------|
| Recent context | recent-context | limit=3-5 |
| Search observations | observations | query, format=index, limit=5 |
| Search sessions | sessions | query, format=index, limit=5 |
| Find by type | by-type | type=(bugfix\|feature\|decision), format=index |
| Find by file | by-file | filePath, format=index |
| Timeline around event | timeline | anchor=(sessionDbId), depth_before=5, depth_after=5 |
| Search + timeline | timeline-by-query | query, mode=auto |
## Common Workflows
For step-by-step guides on typical user requests, see [operations/common-workflows.md](operations/common-workflows.md):
- Understanding past work from previous sessions
- Finding specific bug fixes from history
- Understanding file history across sessions
- Timeline investigation workflows
- Search composition patterns
## Response Formatting
For guidelines on presenting search results to users, see [operations/formatting.md](operations/formatting.md):
- Index format responses (compact lists with titles/dates)
- Full format responses (complete observation details)
- Timeline responses (chronologically grouped)
- Error handling and user-friendly messages
## Technical Notes
- **Port:** Default 37777 (configurable via `CLAUDE_MEM_WORKER_PORT`)
- **Response format:** Always JSON
- **Search engine:** ChromaDB semantic search (primary ranking) + SQLite FTS5 (fallback) + 90-day recency filter + temporal ordering (hybrid architecture)
- **All operations:** HTTP GET with query parameters
- **Worker:** PM2-managed background process
**Clarity:**
- See everything first
- Pick what matters
- Get details only for what you need
## Error Handling
If HTTP request fails:
1. Inform user the claude-mem search service isn't available
2. Suggest checking if worker is running: `pm2 list` or `pm2 status claude-mem-worker`
3. Offer to help troubleshoot using the troubleshoot skill
## Resources
**Principles:**
- [principles/progressive-disclosure.md](principles/progressive-disclosure.md) - Complete 4-step workflow with token calculations
- [principles/anti-patterns.md](principles/anti-patterns.md) - 5 anti-patterns to avoid with LLM behavior insights
**Operations:**
- [operations/](operations/) - Detailed instructions for each operation (9 operations + help)
- [operations/common-workflows.md](operations/common-workflows.md) - Step-by-step workflow guides
- [operations/formatting.md](operations/formatting.md) - Response formatting templates
If search fails, tell the user the worker isn't available and suggest:
```bash
pm2 list # Check if worker is running
```
---
**Remember:** This skill searches **cross-session persistent memory**, NOT current conversation. Start with index format for token efficiency. Use temporal triggers to differentiate from native Claude memory.
**Remember:** ALWAYS search with format=index first. ALWAYS fetch by ID for details. The IDs are there for a reason - USE THEM.
@@ -17,10 +17,16 @@ curl -s "http://localhost:37777/api/search/observations?query=authentication&for
## Parameters
- **query** (required): Natural language search query - uses semantic search (ChromaDB) for ranking with SQLite FTS5 fallback (e.g., "authentication", "bug fix", "database migration")
- **query** (optional): Natural language search query - uses semantic search (ChromaDB) for ranking with SQLite FTS5 fallback (e.g., "authentication", "bug fix", "database migration"). Can be omitted for filter-only searches.
- **format**: "index" (summary) or "full" (complete details). Default: "full"
- **limit**: Number of results (default: 20, max: 100)
- **project**: Filter by project name (optional)
- **dateRange**: Filter by date range (optional) - `dateRange[start]` and/or `dateRange[end]`
- **obs_type**: Filter by observation type: bugfix, feature, refactor, decision, discovery, change (optional)
- **concepts**: Filter by concept tags (optional)
- **files**: Filter by file paths (optional)
**Important**: When omitting `query`, you MUST provide at least one filter (project, dateRange, obs_type, concepts, or files)
## When to Use Each Format
@@ -76,13 +82,28 @@ Found 5 results for "authentication":
For complete formatting guidelines, see formatting.md (documentation coming soon).
## Filter-Only Examples
Search without query text (direct SQLite filtering):
```bash
# Get all observations from November 2025
curl -s "http://localhost:37777/api/search?type=observations&dateRange[start]=2025-11-01&format=index"
# Get all bug fixes from a specific project
curl -s "http://localhost:37777/api/search?type=observations&obs_type=bugfix&project=api-server&format=index"
# Get all observations from last 7 days
curl -s "http://localhost:37777/api/search?type=observations&dateRange[start]=2025-11-11&format=index"
```
## Error Handling
**Missing query parameter:**
**Missing query and filters:**
```json
{"error": "Missing required parameter: query"}
{"error": "Either query or filters required for search"}
```
Fix: Add the query parameter
Fix: Provide either a query parameter OR at least one filter (project, dateRange, obs_type, concepts, files)
**No results found:**
```json
File diff suppressed because one or more lines are too long
+30
View File
@@ -376,6 +376,36 @@
animation: spin 1.5s linear infinite;
}
.queue-bubble {
position: absolute;
top: -8px;
right: -8px;
background: var(--color-accent-primary);
color: var(--color-text-button);
font-size: 10px;
font-weight: 600;
font-family: 'Monaspace Radon', monospace;
min-width: 18px;
height: 18px;
border-radius: 9px;
display: flex;
align-items: center;
justify-content: center;
padding: 0 5px;
box-shadow: 0 2px 4px rgba(0, 0, 0, 0.2);
animation: pulse 2s ease-in-out infinite;
z-index: 10;
}
@keyframes pulse {
0%, 100% {
transform: scale(1);
}
50% {
transform: scale(1.1);
}
}
.logo-text {
font-family: 'Monaspace Radon', monospace;
font-weight: 100;
+109
View File
@@ -0,0 +1,109 @@
# Postmortem: Worker Debug Failure - 2025-11-17
## Incident Summary
Attempted to fix broken worker service. Worker was in crash loop with 225 restarts, failing with "MCP error -32000: Connection closed". Debug attempt failed and changes were reverted.
## What Went Wrong
### 1. **Jumped to Symptoms, Not Root Cause**
- Saw "MCP connection failed" errors in logs
- Immediately focused on MCP/Chroma connection code
- Never asked: "Why is this suddenly broken when it worked before?"
- Classic symptom chasing instead of root cause analysis
### 2. **Ignored the Build Pipeline**
- Worker file wasn't in the expected location (`plugin/worker-service.cjs` vs `plugin/scripts/worker-service.cjs`)
- Build output existed but search server was producing corrupted/error output
- Never investigated: "Is the build system broken?"
- Should have compared built artifacts between main and current branch
### 3. **Tried to Fix by Disabling Instead of Understanding**
- Final approach: comment out Chroma, comment out search server
- This is the opposite of debugging - it's just making things "work" by removing functionality
- User called this out as "duct tape around 5 things unrelated to the problem"
- Violated YAGNI/KISS by adding defensive complexity instead of fixing the actual issue
### 4. **Didn't Compare Working vs Broken State**
- User specifically said "we fixed this before"
- Should have immediately: `git diff main src/services/worker-service.ts`
- Did this eventually but didn't follow through on the findings
- The diff showed only search-everything additions - the core worker code was UNCHANGED
- This should have been a huge red flag: "If the code is the same, why is it broken?"
### 5. **Overcomplicated the Investigation**
- Started reading through ChromaSync implementation
- Traced through MCP connection code
- Analyzed startup sequences
- All of this was unnecessary if the root cause was a build issue
## What Should Have Happened
### Correct Debug Sequence:
1. ✅ Check worker status (`pm2 list`) - DONE
2. ✅ Check error logs - DONE
3. ❌ **Compare current code to main branch** - SKIPPED INITIALLY
4. ❌ **Check if built files are correct** - SKIPPED
5. ❌ **Test the build pipeline** - NEVER DONE
6. ❌ **Verify dependencies are installed** - NEVER CHECKED
### The Real Questions:
- Is this a code change or a build issue?
- What changed between working state and broken state?
- Are the built artifacts corrupted?
- Is the search server build actually valid?
- Are there missing dependencies in plugin/scripts/node_modules?
## Likely Root Causes (Untested)
Based on evidence:
1. **Build artifacts are corrupted** - search-server.mjs threw syntax errors when run
2. **Node modules missing/outdated** - plugin/scripts/node_modules may be stale
3. **ESM/CJS bundling issue** - esbuild may have produced invalid output
4. **search-everything branch has broken build config** - scripts/build-hooks.js may have issues
## Key Lessons
### KISS/DRY/YAGNI Violations
- Added complexity (disabling features) instead of removing it
- Tried to work around symptoms instead of fixing root cause
- Ignored the principle: "If it worked before and code is same, it's environment/build"
### Debugging Anti-Patterns
1. **Symptom Chasing**: Following error messages down rabbit holes
2. **Defensive Coding**: Commenting out "broken" features instead of fixing them
3. **Ignoring History**: Not comparing working vs broken states
4. **Build Blindness**: Assuming built artifacts are correct without verification
### What Good Debugging Looks Like
1. Compare working state (main) vs broken state (current branch)
2. Identify what actually changed (code? deps? build?)
3. Test the simplest hypothesis first (build issue vs code issue)
4. Never disable features to "fix" things - that's not fixing
## Action Items for Next Attempt
### Before Writing Any Code:
- [ ] `git diff main` for all modified files
- [ ] Check if `plugin/scripts/` artifacts are valid JavaScript
- [ ] Compare build process: `npm run build` output on main vs current branch
- [ ] Verify `plugin/scripts/node_modules` exists and is current
- [ ] Test search-server.mjs in isolation: `node plugin/scripts/search-server.mjs`
### If Build is Broken:
- [ ] Check scripts/build-hooks.js for recent changes
- [ ] Verify esbuild configuration
- [ ] Test build on main branch, then on current branch
- [ ] Don't modify source code until build is proven working
### If Code is Broken:
- [ ] Create minimal repro (which specific change broke it?)
- [ ] Fix the actual bug, don't add workarounds
- [ ] Test the fix in isolation
## Conclusion
This failure exemplifies "debugging by making changes" instead of "debugging by understanding". The instinct to fix symptoms (MCP errors) instead of investigating root cause (why is it broken now?) led to wasted effort and ultimately no solution.
The user's frustration was justified - I was adding defensive duct tape instead of finding and fixing the real problem. This is exactly what KISS/DRY/YAGNI principles are meant to prevent.
**Next time: Compare, verify, understand, THEN fix. Never disable features to make errors go away.**
+119
View File
@@ -0,0 +1,119 @@
Unified Search API Consolidation Plan
Overview
Consolidate 10 search endpoints into 6 powerful, semantic endpoints with intelligent aliasing for backward compatibility.
New Endpoint Structure
1. /search - Unified Cross-Type Search
- Searches all record types (observations + sessions + prompts) via Chroma multi-collection search
- Optional type filter to narrow down
- Replaces: search_observations, search_sessions, search_user_prompts
- Params: query, response=[index|full], type, project, dateRange, limit, offset, orderBy
2. /timeline - Unified Timeline Tool
- Supports both anchor-based and query-based modes via params
- If anchor → direct timeline lookup (like get_context_timeline)
- If query → search-first then timeline (like get_timeline_by_query)
- Params: anchor OR query, depth_before, depth_after, response, mode, project
3. /decisions - Decision Observations
- Metadata-first search for type=decision observations
- Uses specialized search logic for precision
- Params: response, project, dateRange, limit, offset, orderBy
4. /changes - Change Observations
- Metadata-first search for type=change observations
- Same pattern as /decisions
5. /how-it-works - How-It-Works Concept
- Metadata-first search for concept=how-it-works observations
- Same pattern as concept endpoints
6. /contextualize - Intelligent Context Builder
- Complex hybrid endpoint:
a. Get 7 latest decisions + 7 latest changes + 3 latest how-it-works
b. Find newest date across all results
c. Get timeline (7 before + 7 after) around that date
d. Merge & re-sort into single timeline (newest → oldest)
e. Return timeline + narratives of each concept's latest result
- Params: query (for contextualization), project
Implementation Phases
Phase 1: Core Unified Search (search-server.ts)
- Create search tool with Chroma multi-collection query
- Add type filtering support
- Alias old tools: search_observations → search(type=['observations'])
Phase 2: Unified Timeline (search-server.ts)
- Merge get_context_timeline + get_timeline_by_query logic
- Support both anchor and query params (mutually exclusive)
- Alias old timeline tools to new unified implementation
Phase 3: Specialized Concept Endpoints (search-server.ts)
- Create decisions, changes, how_it_works tools
- Use metadata-first search strategy
- Update find_by_type and find_by_concept to call these internally
Phase 4: Contextualize Endpoint (search-server.ts)
- Implement parallel fetching (7 decisions, 7 changes, 3 how-it-works)
- Find newest date, get timeline around it
- Merge, re-sort, extract narratives
- Return structured response with timeline + narratives
Phase 5: HTTP API Routes (worker-service.ts)
- Add 6 new routes: /api/search, /api/timeline, /api/decisions, /api/changes, /api/how-it-works, /api/contextualize
- Update old routes to alias new implementations
- Maintain backward compatibility
Phase 6: Chroma Multi-Collection Search (ChromaSync.ts)
- Add searchAll() method to query all collections in parallel
- Include source collection metadata in results
- Merge and rank by similarity score
Phase 7: SQLite Fallback (SessionSearch.ts)
- Add searchAll() for FTS5 fallback when Chroma unavailable
- Merge results from all three FTS5 tables
Phase 8: Documentation & Skill Updates
- Update mem-search skill with new endpoints
- Update CLAUDE.md and README.md
- Add examples and migration guide
Phase 9: Testing & Deployment
- Unit tests for all new tools
- Integration tests for aliasing
- Manual testing via mem-search skill
- Build → sync → worker restart
Key Design Decisions
✅ Aliasing Strategy: Old endpoints call new implementations internally (zero breaking changes)
✅ Unified Search: Chroma multi-collection search for true cross-type queries
✅ Flexible Timeline: Single tool supports both direct and query-based modes
✅ Specialized Shortcuts: /decisions, /changes, /how-it-works for common queries
✅ Intelligent Context: /contextualize auto-builds rich context with narratives
Migration Impact
- Users: Zero breaking changes, old endpoints work via aliasing
- Codebase: Simplified from 10 conceptual endpoints to 6
- Performance: Improved via Chroma multi-collection search
- Developer UX: Cleaner, more semantic API
+31 -11
View File
@@ -26,12 +26,10 @@ const WORKER_SERVICE = {
source: 'src/services/worker-service.ts'
};
// DEPRECATED: MCP search server replaced by skill-based search
// Keeping source file for reference: src/servers/search-server.ts
// const SEARCH_SERVER = {
// name: 'search-server',
// source: 'src/servers/search-server.ts'
// };
const SEARCH_SERVER = {
name: 'search-server',
source: 'src/servers/search-server.ts'
};
async function buildHooks() {
console.log('🔨 Building claude-mem hooks and worker service...\n');
@@ -94,6 +92,31 @@ async function buildHooks() {
const workerStats = fs.statSync(`${hooksDir}/${WORKER_SERVICE.name}.cjs`);
console.log(`✓ worker-service built (${(workerStats.size / 1024).toFixed(2)} KB)`);
// Build search server
console.log(`\n🔧 Building search server...`);
await build({
entryPoints: [SEARCH_SERVER.source],
bundle: true,
platform: 'node',
target: 'node18',
format: 'cjs',
outfile: `${hooksDir}/${SEARCH_SERVER.name}.cjs`,
minify: true,
logLevel: 'error',
external: ['better-sqlite3'],
define: {
'__DEFAULT_PACKAGE_VERSION__': `"${version}"`
},
banner: {
js: '#!/usr/bin/env node'
}
});
// Make search server executable
fs.chmodSync(`${hooksDir}/${SEARCH_SERVER.name}.cjs`, 0o755);
const searchServerStats = fs.statSync(`${hooksDir}/${SEARCH_SERVER.name}.cjs`);
console.log(`✓ search-server built (${(searchServerStats.size / 1024).toFixed(2)} KB)`);
// Build each hook
for (const hook of HOOKS) {
console.log(`\n🔧 Building ${hook.name}...`);
@@ -126,14 +149,11 @@ async function buildHooks() {
console.log(`${hook.name} built (${sizeInKB} KB)`);
}
// DEPRECATED: MCP search server no longer built (replaced by skill-based search)
// Search functionality now provided via HTTP API + search skill
// Source file kept for reference: src/servers/search-server.ts
console.log('\n✅ All hooks and worker service built successfully!');
console.log('\n✅ All hooks, worker service, and search server built successfully!');
console.log(` Output: ${hooksDir}/`);
console.log(` - Hooks: *-hook.js`);
console.log(` - Worker: worker-service.cjs`);
console.log(` - Search Server: search-server.cjs`);
console.log(` - Skills: plugin/skills/`);
console.log('\n💡 Note: Dependencies will be auto-installed on first hook execution');
File diff suppressed because it is too large Load Diff
+91 -151
View File
@@ -13,7 +13,8 @@ import {
/**
* Search interface for session-based memory
* Provides FTS5 full-text search and structured queries for sessions, observations, and summaries
* Provides filter-only structured queries for sessions, observations, and user prompts
* Vector search is handled by ChromaDB - this class only supports filtering without query text
*/
export class SessionSearch {
private db: Database.Database;
@@ -31,7 +32,18 @@ export class SessionSearch {
}
/**
* Ensure FTS5 tables exist (inline migration)
* Ensure FTS5 tables exist (backward compatibility only - no longer used for search)
*
* FTS5 tables are maintained for backward compatibility but not used for search.
* Vector search (Chroma) is now the primary search mechanism.
*
* Retention Rationale:
* - Prevents breaking existing installations with FTS5 tables
* - Allows graceful migration path for users
* - Tables maintained but search paths removed
* - Triggers still fire to keep tables synchronized
*
* TODO: Remove FTS5 infrastructure in future major version (v7.0.0)
*/
private ensureFTSTables(): void {
try {
@@ -134,22 +146,6 @@ export class SessionSearch {
}
}
/**
* Escape FTS5 special characters in user input
*
* FTS5 uses double quotes for phrase searches and treats certain characters
* as operators (*, AND, OR, NOT, parentheses, etc.). To prevent injection,
* we wrap user input in double quotes and escape internal quotes by doubling them.
* This converts any user input into a safe phrase search.
*
* @param text - User input to escape for FTS5 MATCH queries
* @returns Safely escaped FTS5 query string
*/
private escapeFTS5(text: string): string {
// Escape internal double quotes by doubling them (FTS5 standard)
// Then wrap the entire string in double quotes for phrase search
return `"${text.replace(/"/g, '""')}"`;
}
/**
* Build WHERE clause for structured filters
@@ -243,118 +239,78 @@ export class SessionSearch {
}
/**
* Search observations using FTS5 full-text search
* Search observations using filter-only direct SQLite query.
* Vector search is handled by ChromaDB - this only supports filtering without query text.
*/
searchObservations(query: string, options: SearchOptions = {}): ObservationSearchResult[] {
searchObservations(query: string | undefined, options: SearchOptions = {}): ObservationSearchResult[] {
const params: any[] = [];
const { limit = 50, offset = 0, orderBy = 'relevance', ...filters } = options;
// Build FTS5 match query
const ftsQuery = this.escapeFTS5(query);
params.push(ftsQuery);
// FILTER-ONLY PATH: When no query text, query table directly
// This enables date filtering which Chroma cannot do (requires direct SQLite access)
if (!query) {
const filterClause = this.buildFilterClause(filters, params, 'o');
if (!filterClause) {
throw new Error('Either query or filters required for search');
}
// Build filter conditions
const filterClause = this.buildFilterClause(filters, params, 'o');
const whereClause = filterClause ? `AND ${filterClause}` : '';
const orderClause = this.buildOrderClause(orderBy, false);
// Build ORDER BY
const orderClause = this.buildOrderClause(orderBy, true);
const sql = `
SELECT o.*, o.discovery_tokens
FROM observations o
WHERE ${filterClause}
${orderClause}
LIMIT ? OFFSET ?
`;
// Main query with FTS5
const sql = `
SELECT
o.*,
o.discovery_tokens,
observations_fts.rank as rank
FROM observations o
JOIN observations_fts ON o.id = observations_fts.rowid
WHERE observations_fts MATCH ?
${whereClause}
${orderClause}
LIMIT ? OFFSET ?
`;
params.push(limit, offset);
const results = this.db.prepare(sql).all(...params) as ObservationSearchResult[];
// Normalize rank to score (0-1, higher is better)
if (results.length > 0) {
const minRank = Math.min(...results.map(r => r.rank || 0));
const maxRank = Math.max(...results.map(r => r.rank || 0));
const range = maxRank - minRank || 1;
results.forEach(r => {
if (r.rank !== undefined) {
// Invert rank (lower rank = better match) and normalize to 0-1
r.score = 1 - ((r.rank - minRank) / range);
}
});
params.push(limit, offset);
return this.db.prepare(sql).all(...params) as ObservationSearchResult[];
}
return results;
// Vector search with query text should be handled by ChromaDB
// This method only supports filter-only queries (query=undefined)
console.warn('[SessionSearch] Text search not supported - use ChromaDB for vector search');
return [];
}
/**
* Search session summaries using FTS5 full-text search
* Search session summaries using filter-only direct SQLite query.
* Vector search is handled by ChromaDB - this only supports filtering without query text.
*/
searchSessions(query: string, options: SearchOptions = {}): SessionSummarySearchResult[] {
searchSessions(query: string | undefined, options: SearchOptions = {}): SessionSummarySearchResult[] {
const params: any[] = [];
const { limit = 50, offset = 0, orderBy = 'relevance', ...filters } = options;
// Build FTS5 match query
const ftsQuery = this.escapeFTS5(query);
params.push(ftsQuery);
// FILTER-ONLY PATH: When no query text, query session_summaries table directly
if (!query) {
const filterOptions = { ...filters };
delete filterOptions.type;
const filterClause = this.buildFilterClause(filterOptions, params, 's');
if (!filterClause) {
throw new Error('Either query or filters required for search');
}
// Build filter conditions (without type filter - not applicable to summaries)
const filterOptions = { ...filters };
delete filterOptions.type;
const filterClause = this.buildFilterClause(filterOptions, params, 's');
const whereClause = filterClause ? `AND ${filterClause}` : '';
const orderClause = orderBy === 'date_asc'
? 'ORDER BY s.created_at_epoch ASC'
: 'ORDER BY s.created_at_epoch DESC';
// Note: session_summaries don't have files_read/files_modified in the same way
// We'll need to adjust the filter clause
const adjustedWhereClause = whereClause.replace(/files_read/g, 'files_read').replace(/files_modified/g, 'files_edited');
const sql = `
SELECT s.*, s.discovery_tokens
FROM session_summaries s
WHERE ${filterClause}
${orderClause}
LIMIT ? OFFSET ?
`;
// Build ORDER BY
const orderClause = orderBy === 'relevance'
? 'ORDER BY session_summaries_fts.rank ASC'
: orderBy === 'date_asc'
? 'ORDER BY s.created_at_epoch ASC'
: 'ORDER BY s.created_at_epoch DESC';
// Main query with FTS5
const sql = `
SELECT
s.*,
s.discovery_tokens,
session_summaries_fts.rank as rank
FROM session_summaries s
JOIN session_summaries_fts ON s.id = session_summaries_fts.rowid
WHERE session_summaries_fts MATCH ?
${adjustedWhereClause}
${orderClause}
LIMIT ? OFFSET ?
`;
params.push(limit, offset);
const results = this.db.prepare(sql).all(...params) as SessionSummarySearchResult[];
// Normalize rank to score
if (results.length > 0) {
const minRank = Math.min(...results.map(r => r.rank || 0));
const maxRank = Math.max(...results.map(r => r.rank || 0));
const range = maxRank - minRank || 1;
results.forEach(r => {
if (r.rank !== undefined) {
r.score = 1 - ((r.rank - minRank) / range);
}
});
params.push(limit, offset);
return this.db.prepare(sql).all(...params) as SessionSummarySearchResult[];
}
return results;
// Vector search with query text should be handled by ChromaDB
// This method only supports filter-only queries (query=undefined)
console.warn('[SessionSearch] Text search not supported - use ChromaDB for vector search');
return [];
}
/**
@@ -485,16 +441,13 @@ export class SessionSearch {
}
/**
* Search user prompts with full-text search
* Search user prompts using filter-only direct SQLite query.
* Vector search is handled by ChromaDB - this only supports filtering without query text.
*/
searchUserPrompts(query: string, options: SearchOptions = {}): UserPromptSearchResult[] {
searchUserPrompts(query: string | undefined, options: SearchOptions = {}): UserPromptSearchResult[] {
const params: any[] = [];
const { limit = 20, offset = 0, orderBy = 'relevance', ...filters } = options;
// Build FTS5 match query
const ftsQuery = this.escapeFTS5(query);
params.push(ftsQuery);
// Build filter conditions (join with sdk_sessions for project filtering)
const baseConditions: string[] = [];
if (filters.project) {
@@ -516,47 +469,34 @@ export class SessionSearch {
}
}
const whereClause = baseConditions.length > 0 ? `AND ${baseConditions.join(' AND ')}` : '';
// FILTER-ONLY PATH: When no query text, query user_prompts table directly
if (!query) {
if (baseConditions.length === 0) {
throw new Error('Either query or filters required for search');
}
// Build ORDER BY
const orderClause = orderBy === 'relevance'
? 'ORDER BY user_prompts_fts.rank ASC'
: orderBy === 'date_asc'
? 'ORDER BY up.created_at_epoch ASC'
: 'ORDER BY up.created_at_epoch DESC';
const whereClause = `WHERE ${baseConditions.join(' AND ')}`;
const orderClause = orderBy === 'date_asc'
? 'ORDER BY up.created_at_epoch ASC'
: 'ORDER BY up.created_at_epoch DESC';
// Main query with FTS5 (join sdk_sessions for project filtering)
const sql = `
SELECT
up.*,
user_prompts_fts.rank as rank
FROM user_prompts up
JOIN user_prompts_fts ON up.id = user_prompts_fts.rowid
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
WHERE user_prompts_fts MATCH ?
${whereClause}
${orderClause}
LIMIT ? OFFSET ?
`;
const sql = `
SELECT up.*
FROM user_prompts up
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
${whereClause}
${orderClause}
LIMIT ? OFFSET ?
`;
params.push(limit, offset);
const results = this.db.prepare(sql).all(...params) as UserPromptSearchResult[];
// Normalize rank to score
if (results.length > 0) {
const minRank = Math.min(...results.map(r => r.rank || 0));
const maxRank = Math.max(...results.map(r => r.rank || 0));
const range = maxRank - minRank || 1;
results.forEach(r => {
if (r.rank !== undefined) {
r.score = 1 - ((r.rank - minRank) / range);
}
});
params.push(limit, offset);
return this.db.prepare(sql).all(...params) as UserPromptSearchResult[];
}
return results;
// Vector search with query text should be handled by ChromaDB
// This method only supports filter-only queries (query=undefined)
console.warn('[SessionSearch] Text search not supported - use ChromaDB for vector search');
return [];
}
/**
+200 -7
View File
@@ -155,6 +155,12 @@ export class WorkerService {
this.app.get('/api/observations', this.handleGetObservations.bind(this));
this.app.get('/api/summaries', this.handleGetSummaries.bind(this));
this.app.get('/api/prompts', this.handleGetPrompts.bind(this));
// Fetch by ID
this.app.get('/api/observation/:id', this.handleGetObservationById.bind(this));
this.app.get('/api/session/:id', this.handleGetSessionById.bind(this));
this.app.get('/api/prompt/:id', this.handleGetPromptById.bind(this));
this.app.get('/api/stats', this.handleGetStats.bind(this));
this.app.get('/api/processing-status', this.handleGetProcessingStatus.bind(this));
this.app.post('/api/processing', this.handleSetProcessing.bind(this));
@@ -168,6 +174,14 @@ export class WorkerService {
this.app.post('/api/mcp/toggle', this.handleToggleMcp.bind(this));
// Search API endpoints (for skill-based search)
// Unified endpoints (new consolidated API)
this.app.get('/api/search', this.handleUnifiedSearch.bind(this));
this.app.get('/api/timeline', this.handleUnifiedTimeline.bind(this));
this.app.get('/api/decisions', this.handleDecisions.bind(this));
this.app.get('/api/changes', this.handleChanges.bind(this));
this.app.get('/api/how-it-works', this.handleHowItWorks.bind(this));
// Backward compatibility endpoints (use /api/search with type param instead)
this.app.get('/api/search/observations', this.handleSearchObservations.bind(this));
this.app.get('/api/search/sessions', this.handleSearchSessions.bind(this));
this.app.get('/api/search/prompts', this.handleSearchPrompts.bind(this));
@@ -223,7 +237,7 @@ export class WorkerService {
await this.dbManager.initialize();
// Connect to MCP search server
const searchServerPath = path.join(__dirname, '..', '..', 'plugin', 'scripts', 'search-server.mjs');
const searchServerPath = path.join(__dirname, '..', '..', 'plugin', 'scripts', 'search-server.cjs');
const transport = new StdioClientTransport({
command: 'node',
args: [searchServerPath],
@@ -322,9 +336,11 @@ export class WorkerService {
// Send initial processing status (based on queue depth + active generators)
const isProcessing = this.sessionManager.isAnySessionProcessing();
const queueDepth = this.sessionManager.getTotalActiveWork(); // Includes queued + actively processing
this.sseBroadcaster.broadcast({
type: 'processing_status',
isProcessing
isProcessing,
queueDepth
});
}
@@ -657,6 +673,87 @@ export class WorkerService {
}
}
/**
* Get observation by ID
* GET /api/observation/:id
*/
private handleGetObservationById(req: Request, res: Response): void {
try {
const id = parseInt(req.params.id, 10);
if (isNaN(id)) {
res.status(400).json({ error: 'Invalid observation ID' });
return;
}
const store = this.dbManager.getSessionStore();
const observation = store.getObservationById(id);
if (!observation) {
res.status(404).json({ error: `Observation #${id} not found` });
return;
}
res.json(observation);
} catch (error) {
logger.failure('WORKER', 'Get observation by ID failed', {}, error as Error);
res.status(500).json({ error: (error as Error).message });
}
}
/**
* Get session by ID
* GET /api/session/:id
*/
private handleGetSessionById(req: Request, res: Response): void {
try {
const id = parseInt(req.params.id, 10);
if (isNaN(id)) {
res.status(400).json({ error: 'Invalid session ID' });
return;
}
const store = this.dbManager.getSessionStore();
const sessions = store.getSessionSummariesByIds([id]);
if (sessions.length === 0) {
res.status(404).json({ error: `Session #${id} not found` });
return;
}
res.json(sessions[0]);
} catch (error) {
logger.failure('WORKER', 'Get session by ID failed', {}, error as Error);
res.status(500).json({ error: (error as Error).message });
}
}
/**
* Get user prompt by ID
* GET /api/prompt/:id
*/
private handleGetPromptById(req: Request, res: Response): void {
try {
const id = parseInt(req.params.id, 10);
if (isNaN(id)) {
res.status(400).json({ error: 'Invalid prompt ID' });
return;
}
const store = this.dbManager.getSessionStore();
const prompts = store.getUserPromptsByIds([id]);
if (prompts.length === 0) {
res.status(404).json({ error: `Prompt #${id} not found` });
return;
}
res.json(prompts[0]);
} catch (error) {
logger.failure('WORKER', 'Get prompt by ID failed', {}, error as Error);
res.status(500).json({ error: (error as Error).message });
}
}
/**
* Get database statistics (with worker metadata)
*/
@@ -806,11 +903,12 @@ export class WorkerService {
}
/**
* Get processing status (for viewer UI spinner)
* Get processing status (for viewer UI spinner and queue indicator)
*/
private handleGetProcessingStatus(req: Request, res: Response): void {
const isProcessing = this.sessionManager.isAnySessionProcessing();
res.json({ isProcessing });
const queueDepth = this.sessionManager.getTotalActiveWork(); // Includes queued + actively processing
res.json({ isProcessing, queueDepth });
}
// ============================================================================
@@ -823,7 +921,7 @@ export class WorkerService {
*/
broadcastProcessingStatus(): void {
const isProcessing = this.sessionManager.isAnySessionProcessing();
const queueDepth = this.sessionManager.getTotalQueueDepth();
const queueDepth = this.sessionManager.getTotalActiveWork(); // Includes queued + actively processing
const activeSessions = this.sessionManager.getActiveSessionCount();
logger.info('WORKER', 'Broadcasting processing status', {
@@ -834,7 +932,8 @@ export class WorkerService {
this.sseBroadcaster.broadcast({
type: 'processing_status',
isProcessing
isProcessing,
queueDepth
});
}
@@ -940,8 +1039,102 @@ export class WorkerService {
// Search API Handlers (for skill-based search)
// ============================================================================
// ============================================================================
// Unified Search API Handlers (New Consolidated API)
// ============================================================================
/**
* Search observations
* Unified search across all memory types (observations, sessions, prompts)
* GET /api/search?query=...&format=index&limit=20
*/
private async handleUnifiedSearch(req: Request, res: Response): Promise<void> {
try {
const result = await this.mcpClient.callTool({
name: 'search',
arguments: req.query
});
res.json(result.content);
} catch (error) {
logger.failure('WORKER', 'Unified search failed', {}, error as Error);
res.status(500).json({ error: (error as Error).message });
}
}
/**
* Unified timeline (anchor or query-based)
* GET /api/timeline?anchor=123 OR GET /api/timeline?query=...
*/
private async handleUnifiedTimeline(req: Request, res: Response): Promise<void> {
try {
const result = await this.mcpClient.callTool({
name: 'timeline',
arguments: req.query
});
res.json(result.content);
} catch (error) {
logger.failure('WORKER', 'Unified timeline failed', {}, error as Error);
res.status(500).json({ error: (error as Error).message });
}
}
/**
* Semantic shortcut for finding decision observations
* GET /api/decisions?format=index&limit=20
*/
private async handleDecisions(req: Request, res: Response): Promise<void> {
try {
const result = await this.mcpClient.callTool({
name: 'decisions',
arguments: req.query
});
res.json(result.content);
} catch (error) {
logger.failure('WORKER', 'Decisions search failed', {}, error as Error);
res.status(500).json({ error: (error as Error).message });
}
}
/**
* Semantic shortcut for finding change-related observations
* GET /api/changes?format=index&limit=20
*/
private async handleChanges(req: Request, res: Response): Promise<void> {
try {
const result = await this.mcpClient.callTool({
name: 'changes',
arguments: req.query
});
res.json(result.content);
} catch (error) {
logger.failure('WORKER', 'Changes search failed', {}, error as Error);
res.status(500).json({ error: (error as Error).message });
}
}
/**
* Semantic shortcut for finding "how it works" explanations
* GET /api/how-it-works?format=index&limit=20
*/
private async handleHowItWorks(req: Request, res: Response): Promise<void> {
try {
const result = await this.mcpClient.callTool({
name: 'how_it_works',
arguments: req.query
});
res.json(result.content);
} catch (error) {
logger.failure('WORKER', 'How it works search failed', {}, error as Error);
res.status(500).json({ error: (error as Error).message });
}
}
// ============================================================================
// Backward Compatibility API Handlers
// All functionality available via /api/search with type/obs_type/concepts/files params
// ============================================================================
/**
* Search observations (use /api/search?type=observations instead)
* GET /api/search/observations?query=...&format=index&limit=20&project=...
*/
private async handleSearchObservations(req: Request, res: Response): Promise<void> {
+17
View File
@@ -256,6 +256,23 @@ export class SessionManager {
return total;
}
/**
* Get total active work (queued + currently processing)
* Counts both pending messages and items actively being processed by SDK agents
*/
getTotalActiveWork(): number {
let total = 0;
for (const session of this.sessions.values()) {
// Count queued messages
total += session.pendingMessages.length;
// Count currently processing item (1 per active generator)
if (session.generatorPromise !== null) {
total += 1;
}
}
return total;
}
/**
* Check if any session is actively processing (has pending messages OR active generator)
* Used for activity indicator to prevent spinner from stopping while SDK is processing
+30
View File
@@ -376,6 +376,36 @@
animation: spin 1.5s linear infinite;
}
.queue-bubble {
position: absolute;
top: -8px;
right: -8px;
background: var(--color-accent-primary);
color: var(--color-text-button);
font-size: 10px;
font-weight: 600;
font-family: 'Monaspace Radon', monospace;
min-width: 18px;
height: 18px;
border-radius: 9px;
display: flex;
align-items: center;
justify-content: center;
padding: 0 5px;
box-shadow: 0 2px 4px rgba(0, 0, 0, 0.2);
animation: pulse 2s ease-in-out infinite;
z-index: 10;
}
@keyframes pulse {
0%, 100% {
transform: scale(1);
}
50% {
transform: scale(1.1);
}
}
.logo-text {
font-family: 'Monaspace Radon', monospace;
font-weight: 100;
+2 -1
View File
@@ -17,7 +17,7 @@ export function App() {
const [paginatedSummaries, setPaginatedSummaries] = useState<Summary[]>([]);
const [paginatedPrompts, setPaginatedPrompts] = useState<UserPrompt[]>([]);
const { observations, summaries, prompts, projects, isProcessing, isConnected } = useSSE();
const { observations, summaries, prompts, projects, isProcessing, queueDepth, isConnected } = useSSE();
const { settings, saveSettings, isSaving, saveStatus } = useSettings();
const { stats, refreshStats } = useStats();
const { preference, resolvedTheme, setThemePreference } = useTheme();
@@ -96,6 +96,7 @@ export function App() {
onSettingsToggle={toggleSidebar}
sidebarOpen={sidebarOpen}
isProcessing={isProcessing}
queueDepth={queueDepth}
themePreference={preference}
onThemeChange={setThemePreference}
/>
+10 -1
View File
@@ -10,6 +10,7 @@ interface HeaderProps {
onSettingsToggle: () => void;
sidebarOpen: boolean;
isProcessing: boolean;
queueDepth: number;
themePreference: ThemePreference;
onThemeChange: (theme: ThemePreference) => void;
}
@@ -22,13 +23,21 @@ export function Header({
onSettingsToggle,
sidebarOpen,
isProcessing,
queueDepth,
themePreference,
onThemeChange
}: HeaderProps) {
return (
<div className="header">
<h1>
<img src="claude-mem-logomark.webp" alt="" className={`logomark ${isProcessing ? 'spinning' : ''}`} />
<div style={{ position: 'relative', display: 'inline-block' }}>
<img src="claude-mem-logomark.webp" alt="" className={`logomark ${isProcessing ? 'spinning' : ''}`} />
{queueDepth > 0 && (
<div className="queue-bubble">
{queueDepth}
</div>
)}
</div>
<span className="logo-text">claude-mem</span>
</h1>
<div className="status">
+4 -2
View File
@@ -10,6 +10,7 @@ export function useSSE() {
const [projects, setProjects] = useState<string[]>([]);
const [isConnected, setIsConnected] = useState(false);
const [isProcessing, setIsProcessing] = useState(false);
const [queueDepth, setQueueDepth] = useState(0);
const eventSourceRef = useRef<EventSource | null>(null);
const reconnectTimeoutRef = useRef<NodeJS.Timeout>();
@@ -83,8 +84,9 @@ export function useSSE() {
case 'processing_status':
if (typeof data.isProcessing === 'boolean') {
console.log('[SSE] Processing status:', data.isProcessing);
console.log('[SSE] Processing status:', data.isProcessing, 'Queue depth:', data.queueDepth);
setIsProcessing(data.isProcessing);
setQueueDepth(data.queueDepth || 0);
}
break;
}
@@ -107,5 +109,5 @@ export function useSSE() {
};
}, []);
return { observations, summaries, prompts, projects, isProcessing, isConnected };
return { observations, summaries, prompts, projects, isProcessing, queueDepth, isConnected };
}
+86
View File
@@ -0,0 +1,86 @@
#!/bin/bash
# Comprehensive Search API Test Suite
# Tests all endpoints and parameter combinations
API_URL="http://localhost:37777"
RESULTS_DIR="test-results"
echo "🔍 Starting comprehensive search API tests..."
echo ""
# SEMANTIC QUERIES - Understanding how things work
echo "📚 Running semantic queries..."
curl -s "$API_URL/api/search?type=observations&query=worker%20service%20startup&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-01-worker-service-startup.json"
curl -s "$API_URL/api/search?type=observations&query=SQLite%20FTS5%20implementation&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-02-sqlite-fts5-implementation.json"
curl -s "$API_URL/api/search?type=observations&query=hook%20lifecycle%20flow&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-03-hook-lifecycle-flow.json"
curl -s "$API_URL/api/search?type=observations&query=build%20pipeline%20process&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-04-build-pipeline-process.json"
echo "✅ Semantic queries complete (4 tests)"
# DECISION QUERIES - Architectural choices
echo "⚖️ Running decision queries..."
curl -s "$API_URL/api/search?type=observations&obs_type=decision&query=PM2%20instead%20of%20direct%20process&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-05-pm2-decision.json"
curl -s "$API_URL/api/search?type=observations&obs_type=decision&query=search%20architecture%20guidelines&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-06-search-architecture-decision.json"
curl -s "$API_URL/api/search?type=observations&obs_type=decision&query=MCP%20as%20DRY%20source&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-07-mcp-dry-decision.json"
echo "✅ Decision queries complete (3 tests)"
# TROUBLESHOOTING QUERIES - Finding bugfixes
echo "🔴 Running troubleshooting queries..."
curl -s "$API_URL/api/search?type=observations&obs_type=bugfix&query=worker%20service%20debugging&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-08-worker-debugging.json"
curl -s "$API_URL/api/search?type=observations&query=hook%20timeout%20problems&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-09-hook-timeout.json"
curl -s "$API_URL/api/search?type=observations&query=database%20migration%20issues&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-10-database-migration.json"
echo "✅ Troubleshooting queries complete (3 tests)"
# FILE-SPECIFIC QUERIES - Tracking file changes
echo "📁 Running file-specific queries..."
curl -s "$API_URL/api/search?type=observations&files=search-server.ts&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-11-search-server-changes.json"
curl -s "$API_URL/api/search?type=observations&files=context-hook&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-12-context-hook-changes.json"
curl -s "$API_URL/api/search?type=observations&files=worker-service&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-13-worker-service-changes.json"
echo "✅ File-specific queries complete (3 tests)"
# CONCEPT-BASED QUERIES - Patterns, gotchas, discoveries
echo "🏷️ Running concept-based queries..."
curl -s "$API_URL/api/search?type=observations&concepts=pattern&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-14-patterns.json"
curl -s "$API_URL/api/search?type=observations&concepts=gotcha&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-15-gotchas.json"
curl -s "$API_URL/api/search?type=observations&concepts=discovery&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-16-discoveries.json"
echo "✅ Concept-based queries complete (3 tests)"
# TYPE-FILTERED QUERIES - Bugfixes, features, decisions
echo "🔖 Running type-filtered queries..."
curl -s "$API_URL/api/search?type=observations&obs_type=bugfix&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-17-all-bugfixes.json"
curl -s "$API_URL/api/search?type=observations&obs_type=feature&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-18-all-features.json"
curl -s "$API_URL/api/search?type=observations&obs_type=decision&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-19-all-decisions.json"
echo "✅ Type-filtered queries complete (3 tests)"
# SESSION QUERIES - Testing session search
echo "📝 Running session queries..."
curl -s "$API_URL/api/search?type=sessions&query=search%20architecture&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-20-session-search.json"
echo "✅ Session queries complete (1 test)"
# USER PROMPT QUERIES - Testing prompt search
echo "💬 Running user prompt queries..."
curl -s "$API_URL/api/search?type=prompts&query=build%20and%20deploy&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-21-prompt-search.json"
echo "✅ User prompt queries complete (1 test)"
# DEDICATED ENDPOINTS - Timeline and semantic shortcuts
echo "🎯 Running dedicated endpoint tests..."
curl -s "$API_URL/api/decisions?format=full&limit=5" > "$RESULTS_DIR/test-22-decisions-endpoint.json"
curl -s "$API_URL/api/changes?format=full&limit=5" > "$RESULTS_DIR/test-23-changes-endpoint.json"
curl -s "$API_URL/api/how-it-works?format=full&limit=5" > "$RESULTS_DIR/test-24-how-it-works-endpoint.json"
curl -s "$API_URL/api/contextualize?format=full" > "$RESULTS_DIR/test-25-contextualize-endpoint.json"
echo "✅ Dedicated endpoint tests complete (4 tests)"
# TIMELINE QUERY - Get context around a specific observation
echo "⏱️ Running timeline query..."
curl -s "$API_URL/api/timeline?anchor=10630&depth_before=3&depth_after=3&format=full" > "$RESULTS_DIR/test-26-timeline-around-observation.json"
echo "✅ Timeline query complete (1 test)"
# MULTI-PARAMETER COMBO - Test complex query combinations
echo "🎛️ Running multi-parameter combination tests..."
curl -s "$API_URL/api/search?type=observations&obs_type=decision&concepts=pattern&query=search&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-27-multi-param-combo.json"
curl -s "$API_URL/api/search?type=observations&files=search-server&obs_type=feature&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-28-file-type-combo.json"
echo "✅ Multi-parameter tests complete (2 tests)"
echo ""
echo "✨ All tests complete! 28 total queries executed."
echo "📊 Results saved to $RESULTS_DIR/"
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+1
View File
@@ -0,0 +1 @@
[{"type":"text","text":"## User Prompt #5\n*Source: claude-mem://user-prompt/2735*\n\nAre you ok? You seem a little excited and it's hard to do this project. Can you tell me what's going on?\n\n---\nDate: 11/17/2025, 9:38:06 PM\n\n---\n\n## User Prompt #2\n*Source: claude-mem://user-prompt/2719*\n\nedit the build script, we need it to always do this.\n\n---\nDate: 11/17/2025, 7:49:17 PM\n\n---\n\n## User Prompt #3\n*Source: claude-mem://user-prompt/2662*\n\nthose built files, the js, the cjs, if the source files don't have merge issues, then these are just way outdated build files.... is that the case?\n\n---\nDate: 11/17/2025, 3:03:13 PM\n\n---\n\n## User Prompt #12\n*Source: claude-mem://user-prompt/2651*\n\nI FUCKING DID BUILD AND SYNC AND DELETE. I would not have told you to fucking check pm2 info had I not done that\n\n---\nDate: 11/17/2025, 2:25:18 PM\n\n---\n\n## User Prompt #7\n*Source: claude-mem://user-prompt/2646*\n\nCan you fix it so that it runs from the marketplace folder so it doesn't break on EVERYONES SYSTEM WHO DOESNT FUCKING BUILD MANUALLY\n\n---\nDate: 11/17/2025, 2:19:29 PM"}]
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
[{"type":"text","text":"# Project Context: claude-mem\n\nNo activity found for this project."}]
@@ -0,0 +1 @@
[{"type":"text","text":"# Timeline around anchor: 10630\n**Window:** 3 records before → 3 records after | **Items:** 3 (3 obs, 0 sessions, 0 prompts)\n\n**Legend:** 🎯 session-request | 🔴 bugfix | 🟣 feature | 🔄 refactor | ✅ change | 🔵 discovery | 🧠 decision\n\n### Nov 17, 2025\n\n**General**\n| ID | Time | T | Title | Tokens |\n|----|------|---|-------|--------|\n| #10756 | 11:50 PM | 🔴 | Fixed Incorrect Parameter Array in searchUserPrompts FTS5 Path | ~213 |\n| #10757 | ″ | 🔵 | Unified search handler implements Chroma-first with FTS5 fallback on zero results | ~224 |\n| #10758 | 11:51 PM | ✅ | Build and sync claude-mem plugin to marketplace location | ~189 |\n"}]
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long