* Refactor code structure for improved readability and maintainability
* Add test results for search API and related functionalities
- Created test result files for various search-related functionalities, including:
- test-11-search-server-changes.json
- test-12-context-hook-changes.json
- test-13-worker-service-changes.json
- test-14-patterns.json
- test-15-gotchas.json
- test-16-discoveries.json
- test-17-all-bugfixes.json
- test-18-all-features.json
- test-19-all-decisions.json
- test-20-session-search.json
- test-21-prompt-search.json
- test-22-decisions-endpoint.json
- test-23-changes-endpoint.json
- test-24-how-it-works-endpoint.json
- test-25-contextualize-endpoint.json
- test-26-timeline-around-observation.json
- test-27-multi-param-combo.json
- test-28-file-type-combo.json
- Each test result file captures specific search failures or outcomes, including issues with undefined properties and successful execution of search queries.
- Enhanced documentation of search architecture and testing strategies, ensuring compliance with established guidelines and improving overall search functionality.
* feat: Enhance unified search API with catch-all parameters and backward compatibility
- Implemented a unified search API at /api/search that accepts catch-all parameters for filtering by type, observation type, concepts, and files.
- Maintained backward compatibility by keeping granular endpoints functional while routing through the same infrastructure.
- Completed comprehensive testing of search capabilities with real-world query scenarios.
fix: Address missing debug output in search API query tests
- Flushed PM2 logs and executed search queries to verify functionality.
- Diagnosed absence of "Raw Chroma" debug messages in worker logs, indicating potential issues with logging or query processing.
refactor: Improve build and deployment pipeline for claude-mem plugin
- Successfully built and synced all hooks and services to the marketplace directory.
- Ensured all dependencies are installed and up-to-date in the deployment location.
feat: Implement hybrid search filters with 90-day recency window
- Enhanced search server to apply a 90-day recency filter to Chroma results before categorizing by document type.
fix: Correct parameter handling in searchUserPrompts method
- Added support for filter-only queries and improved dual-path logic for clarity.
refactor: Rename FTS5 method to clarify fallback status
- Renamed escapeFTS5 to escapeFTS5_fallback_when_chroma_unavailable to indicate its temporary usage.
feat: Introduce contextualize tool for comprehensive project overview
- Added a new tool to fetch recent observations, sessions, and user prompts, providing a quick project overview.
feat: Add semantic shortcut tools for common search patterns
- Implemented 'decisions', 'changes', and 'how_it_works' tools for convenient access to frequently searched observation categories.
feat: Unified timeline tool supports anchor and query modes
- Combined get_context_timeline and get_timeline_by_query into a single interface for timeline exploration.
feat: Unified search tool added to MCP server
- New tool queries all memory types simultaneously, providing combined chronological results for improved search efficiency.
* Refactor search functionality to clarify FTS5 fallback usage
- Updated `worker-service.cjs` to replace FTS5 fallback function with a more descriptive name and improved error handling.
- Enhanced documentation in `SKILL.md` to specify the unified API endpoint and clarify the behavior of the search engine, including the conditions under which FTS5 is used.
- Modified `search-server.ts` to provide clearer logging and descriptions regarding the fallback to FTS5 when UVX/Python is unavailable.
- Renamed and updated the `SessionSearch.ts` methods to reflect the conditions for using FTS5, emphasizing the lack of semantic understanding in fallback scenarios.
* feat: Add ID-based fetch endpoints and simplify mem-search skill
**Problem:**
- Search returns IDs but no way to fetch by ID
- Skill documentation was bloated with too many options
- Claude wasn't using IDs because we didn't tell it how
**Solution:**
1. Added three new HTTP endpoints:
- GET /api/observation/:id
- GET /api/session/:id
- GET /api/prompt/:id
2. Completely rewrote SKILL.md:
- Stripped complexity down to essentials
- Clear 3-step prescriptive workflow: Search → Review IDs → Fetch by ID
- Emphasized ID usage: "The IDs are there for a reason - USE THEM"
- Removed confusing multi-endpoint documentation
- Kept only unified search with filters
**Impact:**
- Token efficiency: Claude can now fetch full details only for relevant IDs
- Clarity: One clear workflow instead of 10+ options to choose from
- Usability: IDs are no longer wasted context - they're actionable
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* chore: Move internal docs to private directory
Moved POSTMORTEM and planning docs to ./private to exclude from PR reviews.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor: Remove experimental contextualize endpoint
- Removed contextualize MCP tool from search-server (saves ~4KB)
- Disabled FTS5 fallback paths in SessionSearch (now vector-first)
- Cleaned up CLAUDE.md documentation
- Removed contextualize-rewrite-plan.md doc
Rationale:
- Contextualize is better suited as a skill (LLM-powered) than an endpoint
- Search API already provides vector search with configurable limits
- Created issue #132 to track future contextualize skill implementation
Changes:
- src/servers/search-server.ts: Removed contextualize tool definition
- src/services/sqlite/SessionSearch.ts: Disabled FTS5 fallback, added deprecation warnings
- CLAUDE.md: Cleaned up outdated skill documentation
- docs/: Removed contextualize plan document
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor: Complete FTS5 cleanup - remove all deprecated search code
This completes the FTS5 cleanup work by removing all commented-out
FTS5 search code while preserving database tables for backward compatibility.
Changes:
- Removed 200+ lines of commented FTS5 search code from SessionSearch.ts
- Removed deprecated degraded_search_query__when_uvx_unavailable method
- Updated all method documentation to clarify vector-first architecture
- Updated class documentation to reflect filter-only query support
- Updated CLAUDE.md to remove FTS5 search references
- Clarified that FTS5 tables exist for backward compatibility only
- Updated "Why SQLite FTS5" section to "Why Vector-First Search"
Database impact: NONE - FTS5 tables remain intact for existing installations
Search architecture:
- ChromaDB: All text-based vector search queries
- SQLite: Filter-only queries (date ranges, metadata, no query text)
- FTS5 tables: Maintained but unused (backward compatibility)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor: Remove all FTS5 fallback execution code from search-server
Completes the FTS5 cleanup by removing all fallback execution paths
that attempted to use FTS5 when ChromaDB was unavailable.
Changes:
- Removed all FTS5 fallback code execution paths
- When ChromaDB fails or is unavailable, return empty results with helpful error messages
- Updated all deprecated tool descriptions (search_observations, search_sessions, search_user_prompts)
- Changed error messages to indicate FTS5 fallback has been removed
- Added installation instructions for UVX/Python when vector search is unavailable
- Updated comments from "hybrid search" to "vector-first search"
- Removed ~100 lines of dead FTS5 fallback code
Database impact: NONE - FTS5 tables remain intact (backward compatibility)
Search behavior when ChromaDB unavailable:
- Text queries: Return empty results with error explaining ChromaDB is required
- Filter-only queries (no text): Continue to work via direct SQLite
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: Address PR 133 review feedback
Critical fixes:
- Remove contextualize endpoint from worker-service (route + handler)
- Fix build script logging to show correct .cjs extension (was .mjs)
Documentation improvements:
- Add comprehensive FTS5 retention rationale documentation
- Include v7.0.0 removal TODO for future cleanup
Testing:
- Build succeeds with correct output logging
- Worker restarts successfully (30th restart)
- Contextualize endpoint properly removed (404 response)
- Search endpoint verified working
This addresses all critical review feedback from PR 133.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix: Use dynamic project name detection instead of hardcoded values
Fixes#140
Previously, the worker process used hardcoded "claude-mem" for:
- ChromaSync instantiation in DatabaseManager
- ChromaDB collection naming in search-server
This caused all observations to be tagged with "claude-mem" regardless
of the actual project being worked on.
Now both services use getCurrentProjectName() to dynamically detect the
project based on the worker's current working directory.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: Simplify viewer UI overflow CSS to enable scrolling
- Remove overcomplicated nested overflow containers
- Use explicit 100vh for layout height
- Add overflow: hidden to main-col to constrain feed
- Keep simple overflow-y: auto on feed element
- Fix issue where feed content wouldn't scroll
---------
Co-authored-by: Claude <noreply@anthropic.com>
Removed verbose explanations and kept only critical reference info:
- What the project is
- Architecture overview (hooks, worker, database, skills, chroma, viewer)
- Build commands for different scenarios
- Environment variables
- File locations
- Quick reference commands
Moved general coding standards to ~/.claude/CLAUDE.md (global config).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implements a mobile-first navigation reorganization that moves controls into the sidebar at smaller breakpoints:
- Community button moves to sidebar at 600px
- Projects dropdown moves to sidebar at 480px
- Sidebar gains proper width constraints (100% width, 400px max-width)
- Full-height layout styling fixes for proper flex behavior
- Clean separation between header and sidebar responsibilities
This creates a Discord-like mobile experience where the sidebar becomes the primary navigation container on smaller screens.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude <noreply@anthropic.com>
Added semantic naming to coding standards emphasizing verbose,
self-documenting names for comprehension. Cleaned up database
migration section removing TODO in favor of clear evergreen statement.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implements a visual badge that displays the count of active work items (queued + currently processing) in the worker service. The badge appears next to the claude-mem logo and updates in real-time via SSE.
Features:
- Shows count of pending messages + active SDK generators
- Only displays when queueDepth > 0
- Subtle pulse animation for visual feedback
- Theme-aware styling
Backend changes:
- Added getTotalActiveWork() method to SessionManager
- Updated worker-service to broadcast queueDepth via SSE
- Enhanced processing status API endpoint
Frontend changes:
- Updated Header component to display queue bubble
- Enhanced useSSE hook to track queueDepth state
- Added CSS styling with pulse animation
Closes#122Closes#96Closes#97🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Critical hotfix for PM2 worker startup issue. The worker now correctly
starts from the marketplace directory automatically.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Updated `new-hook.js` and `save-hook.js` to enhance logging functionality, including better error messages and structured logging.
- Improved the handling of worker startup in `summary-hook.js` and `worker-utils.ts`, ensuring proper directory context for PM2.
- Added checks for worker health and streamlined session management across hooks.
- Added a new script to simulate token savings from Endless Mode using real observation data.
- Introduced functions to calculate token costs with and without Endless Mode, showcasing the differences in context handling.
- Enhanced output to display detailed token usage statistics and potential savings at scale.
- Integrated assumptions for user activity to estimate weekly and annual token savings.
- Updated worker-utils to automatically start the worker using PM2 if not running, improving service reliability.
Root cause: The ensureDiscoveryTokensColumn migration was using version 7,
which was already taken by removeSessionSummariesUniqueConstraint. This
duplicate version number caused migration tracking issues in some databases.
Changes:
- Updated migration version from 7 to 11 (next available)
- Added schema_versions check to prevent unnecessary re-runs
- Updated comments to clarify the version number conflict
- Added error propagation (already present, but now more reliable)
This resolves issue #121 where users were seeing "no such column: discovery_tokens"
errors after upgrading to v6.0.6.
Affected users can manually add the columns:
ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0;
ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0;
Or wait for v6.0.7 release which includes this fix.
Fixes#121🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Critical bugfix for database migration issue (Issue #121)
Changes:
- Fix migration logic to always verify column existence
- Remove early return that trusted schema_versions alone
- Ensures discovery_tokens columns exist before queries
- Prevents "no such column" errors for all users
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Removed `IMPLEMENTATION_PLAN_ROI_METRICS.md` which detailed the implementation plan for ROI metrics and discovery cost tracking.
- Deleted `MEMORY_LEAK_FIXES.md` and `MEMORY_LEAK_SUMMARY.md` that contained information on memory leak fixes and their summaries.
Automatic cleanup of orphaned MCP server processes on worker startup
Removed manual cleanup notice from session context
Self-healing maintenance on every worker restart
Generated with Claude Code
- Implemented a new method `cleanupOrphanedProcesses` to identify and terminate orphaned `uvx` processes from previous sessions.
- Integrated the cleanup method into the `start` process of the WorkerService to ensure a clean environment at startup.
- Added logging for process cleanup actions and handled potential errors gracefully without failing the service startup.
* feat: Add discovery_tokens for ROI tracking in observations and session summaries
- Introduced `discovery_tokens` column in `observations` and `session_summaries` tables to track token costs associated with discovering and creating each observation and summary.
- Updated relevant services and hooks to calculate and display ROI metrics based on discovery tokens.
- Enhanced context economics reporting to include savings from reusing previous observations.
- Implemented migration to ensure the new column is added to existing tables.
- Adjusted data models and sync processes to accommodate the new `discovery_tokens` field.
* refactor: streamline context hook by removing unused functions and updating terminology
- Removed the estimateTokens and getObservations helper functions as they were not utilized.
- Updated the legend and output messages to replace "discovery" with "work" for clarity.
- Changed the emoji representation for different observation types to better reflect their purpose.
- Enhanced output formatting for improved readability and understanding of token usage.
* Refactor user-message-hook and context-hook for improved clarity and functionality
- Updated user-message-hook.js to enhance error messaging and improve variable naming for clarity.
- Modified context-hook.ts to include a new column key section, improved context index instructions, and added emoji icons for observation types.
- Adjusted footer messages in context-hook.ts to emphasize token savings and access to past research.
- Changed user-message-hook.ts to update the feedback and support message for clarity.
* fix: Critical ROI tracking fixes from PR review
Addresses critical findings from PR #111 review:
1. **Fixed incorrect discovery token calculation** (src/services/worker/SDKAgent.ts)
- Changed from passing cumulative total to per-response delta
- Now correctly tracks token cost for each observation/summary
- Captures token state before/after response processing
- Prevents all observations getting inflated cumulative values
2. **Fixed schema version mismatch** (src/services/sqlite/SessionStore.ts)
- Changed ensureDiscoveryTokensColumn() from version 11 to version 7
- Now matches migration007 definition in migrations.ts
- Ensures consistent version tracking across migration system
These fixes ensure ROI metrics accurately reflect token costs.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: Update search documentation to reflect hybrid ChromaDB architecture
The backend correctly implements ChromaDB-first semantic search with SQLite
temporal ordering and FTS5 fallback, but documentation incorrectly described
it as "FTS5 full-text search". This fix aligns all skill guides and tool
descriptions with the actual implementation.
Changes:
- Update SKILL.md to describe hybrid architecture with ChromaDB primary
- Update observations.md title and query parameter descriptions
- Update all three search tool descriptions in search-server.ts:
* search_observations
* search_sessions
* search_user_prompts
All tools now correctly document:
- ChromaDB semantic search (primary ranking)
- 90-day recency filter
- SQLite temporal ordering
- FTS5 fallback (when ChromaDB unavailable)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: Add discovery_tokens column to observations and session_summaries tables
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat: Add discovery_tokens for ROI tracking in observations and session summaries
- Introduced `discovery_tokens` column in `observations` and `session_summaries` tables to track token costs associated with discovering and creating each observation and summary.
- Updated relevant services and hooks to calculate and display ROI metrics based on discovery tokens.
- Enhanced context economics reporting to include savings from reusing previous observations.
- Implemented migration to ensure the new column is added to existing tables.
- Adjusted data models and sync processes to accommodate the new `discovery_tokens` field.
* refactor: streamline context hook by removing unused functions and updating terminology
- Removed the estimateTokens and getObservations helper functions as they were not utilized.
- Updated the legend and output messages to replace "discovery" with "work" for clarity.
- Changed the emoji representation for different observation types to better reflect their purpose.
- Enhanced output formatting for improved readability and understanding of token usage.
* Refactor user-message-hook and context-hook for improved clarity and functionality
- Updated user-message-hook.js to enhance error messaging and improve variable naming for clarity.
- Modified context-hook.ts to include a new column key section, improved context index instructions, and added emoji icons for observation types.
- Adjusted footer messages in context-hook.ts to emphasize token savings and access to past research.
- Changed user-message-hook.ts to update the feedback and support message for clarity.
* fix: Critical ROI tracking fixes from PR review
Addresses critical findings from PR #111 review:
1. **Fixed incorrect discovery token calculation** (src/services/worker/SDKAgent.ts)
- Changed from passing cumulative total to per-response delta
- Now correctly tracks token cost for each observation/summary
- Captures token state before/after response processing
- Prevents all observations getting inflated cumulative values
2. **Fixed schema version mismatch** (src/services/sqlite/SessionStore.ts)
- Changed ensureDiscoveryTokensColumn() from version 11 to version 7
- Now matches migration007 definition in migrations.ts
- Ensures consistent version tracking across migration system
These fixes ensure ROI metrics accurately reflect token costs.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
- Introduced icon-thin-learned.svg with detailed path definitions and color styling.
- Added icon-thin-next-steps.svg featuring a unique design and color scheme.
Alex Newman