Alex Newman
c5e68a17c8
* Refactor code structure for improved readability and maintainability * Add test results for search API and related functionalities - Created test result files for various search-related functionalities, including: - test-11-search-server-changes.json - test-12-context-hook-changes.json - test-13-worker-service-changes.json - test-14-patterns.json - test-15-gotchas.json - test-16-discoveries.json - test-17-all-bugfixes.json - test-18-all-features.json - test-19-all-decisions.json - test-20-session-search.json - test-21-prompt-search.json - test-22-decisions-endpoint.json - test-23-changes-endpoint.json - test-24-how-it-works-endpoint.json - test-25-contextualize-endpoint.json - test-26-timeline-around-observation.json - test-27-multi-param-combo.json - test-28-file-type-combo.json - Each test result file captures specific search failures or outcomes, including issues with undefined properties and successful execution of search queries. - Enhanced documentation of search architecture and testing strategies, ensuring compliance with established guidelines and improving overall search functionality. * feat: Enhance unified search API with catch-all parameters and backward compatibility - Implemented a unified search API at /api/search that accepts catch-all parameters for filtering by type, observation type, concepts, and files. - Maintained backward compatibility by keeping granular endpoints functional while routing through the same infrastructure. - Completed comprehensive testing of search capabilities with real-world query scenarios. fix: Address missing debug output in search API query tests - Flushed PM2 logs and executed search queries to verify functionality. - Diagnosed absence of "Raw Chroma" debug messages in worker logs, indicating potential issues with logging or query processing. refactor: Improve build and deployment pipeline for claude-mem plugin - Successfully built and synced all hooks and services to the marketplace directory. - Ensured all dependencies are installed and up-to-date in the deployment location. feat: Implement hybrid search filters with 90-day recency window - Enhanced search server to apply a 90-day recency filter to Chroma results before categorizing by document type. fix: Correct parameter handling in searchUserPrompts method - Added support for filter-only queries and improved dual-path logic for clarity. refactor: Rename FTS5 method to clarify fallback status - Renamed escapeFTS5 to escapeFTS5_fallback_when_chroma_unavailable to indicate its temporary usage. feat: Introduce contextualize tool for comprehensive project overview - Added a new tool to fetch recent observations, sessions, and user prompts, providing a quick project overview. feat: Add semantic shortcut tools for common search patterns - Implemented 'decisions', 'changes', and 'how_it_works' tools for convenient access to frequently searched observation categories. feat: Unified timeline tool supports anchor and query modes - Combined get_context_timeline and get_timeline_by_query into a single interface for timeline exploration. feat: Unified search tool added to MCP server - New tool queries all memory types simultaneously, providing combined chronological results for improved search efficiency. * Refactor search functionality to clarify FTS5 fallback usage - Updated `worker-service.cjs` to replace FTS5 fallback function with a more descriptive name and improved error handling. - Enhanced documentation in `SKILL.md` to specify the unified API endpoint and clarify the behavior of the search engine, including the conditions under which FTS5 is used. - Modified `search-server.ts` to provide clearer logging and descriptions regarding the fallback to FTS5 when UVX/Python is unavailable. - Renamed and updated the `SessionSearch.ts` methods to reflect the conditions for using FTS5, emphasizing the lack of semantic understanding in fallback scenarios. * feat: Add ID-based fetch endpoints and simplify mem-search skill **Problem:** - Search returns IDs but no way to fetch by ID - Skill documentation was bloated with too many options - Claude wasn't using IDs because we didn't tell it how **Solution:** 1. Added three new HTTP endpoints: - GET /api/observation/:id - GET /api/session/:id - GET /api/prompt/:id 2. Completely rewrote SKILL.md: - Stripped complexity down to essentials - Clear 3-step prescriptive workflow: Search → Review IDs → Fetch by ID - Emphasized ID usage: "The IDs are there for a reason - USE THEM" - Removed confusing multi-endpoint documentation - Kept only unified search with filters **Impact:** - Token efficiency: Claude can now fetch full details only for relevant IDs - Clarity: One clear workflow instead of 10+ options to choose from - Usability: IDs are no longer wasted context - they're actionable 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: Move internal docs to private directory Moved POSTMORTEM and planning docs to ./private to exclude from PR reviews. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: Remove experimental contextualize endpoint - Removed contextualize MCP tool from search-server (saves ~4KB) - Disabled FTS5 fallback paths in SessionSearch (now vector-first) - Cleaned up CLAUDE.md documentation - Removed contextualize-rewrite-plan.md doc Rationale: - Contextualize is better suited as a skill (LLM-powered) than an endpoint - Search API already provides vector search with configurable limits - Created issue #132 to track future contextualize skill implementation Changes: - src/servers/search-server.ts: Removed contextualize tool definition - src/services/sqlite/SessionSearch.ts: Disabled FTS5 fallback, added deprecation warnings - CLAUDE.md: Cleaned up outdated skill documentation - docs/: Removed contextualize plan document 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: Complete FTS5 cleanup - remove all deprecated search code This completes the FTS5 cleanup work by removing all commented-out FTS5 search code while preserving database tables for backward compatibility. Changes: - Removed 200+ lines of commented FTS5 search code from SessionSearch.ts - Removed deprecated degraded_search_query__when_uvx_unavailable method - Updated all method documentation to clarify vector-first architecture - Updated class documentation to reflect filter-only query support - Updated CLAUDE.md to remove FTS5 search references - Clarified that FTS5 tables exist for backward compatibility only - Updated "Why SQLite FTS5" section to "Why Vector-First Search" Database impact: NONE - FTS5 tables remain intact for existing installations Search architecture: - ChromaDB: All text-based vector search queries - SQLite: Filter-only queries (date ranges, metadata, no query text) - FTS5 tables: Maintained but unused (backward compatibility) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: Remove all FTS5 fallback execution code from search-server Completes the FTS5 cleanup by removing all fallback execution paths that attempted to use FTS5 when ChromaDB was unavailable. Changes: - Removed all FTS5 fallback code execution paths - When ChromaDB fails or is unavailable, return empty results with helpful error messages - Updated all deprecated tool descriptions (search_observations, search_sessions, search_user_prompts) - Changed error messages to indicate FTS5 fallback has been removed - Added installation instructions for UVX/Python when vector search is unavailable - Updated comments from "hybrid search" to "vector-first search" - Removed ~100 lines of dead FTS5 fallback code Database impact: NONE - FTS5 tables remain intact (backward compatibility) Search behavior when ChromaDB unavailable: - Text queries: Return empty results with error explaining ChromaDB is required - Filter-only queries (no text): Continue to work via direct SQLite 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix: Address PR 133 review feedback Critical fixes: - Remove contextualize endpoint from worker-service (route + handler) - Fix build script logging to show correct .cjs extension (was .mjs) Documentation improvements: - Add comprehensive FTS5 retention rationale documentation - Include v7.0.0 removal TODO for future cleanup Testing: - Build succeeds with correct output logging - Worker restarts successfully (30th restart) - Contextualize endpoint properly removed (404 response) - Search endpoint verified working This addresses all critical review feedback from PR 133. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
1 line
9.0 KiB
JSON
1 line
9.0 KiB
JSON
[{"type":"text","text":"## Search Results Return Structured Observation Content with Rich Metadata\n*Source: claude-mem://observation/10695*\n\n**Worker service query returned six observations with complete narratives, facts, concepts, and source links**\n\nExamination of test-01-worker-service-startup.json reveals the structure and quality of search API responses. The worker service startup query successfully retrieved six relevant observations demonstrating that FTS5 keyword search provides meaningful results. Each observation includes comprehensive metadata: title, subtitle (one-line summary), full narrative text explaining the context, structured facts list with specific details, concept tags (what-changed, how-it-works, problem-solution, gotcha), associated file paths, creation dates, and observation type classification. Results are formatted with source links using the claude-mem://observation/ID format for easy reference. The retrieved observations span multiple aspects of worker service startup including port file handling migration to plugin data directory, getPackageRoot path resolution dependencies, comprehensive production failure analysis documenting 8 critical issues causing 100% failure rate, and ChromaSync initialization timing problems. The diversity of observation types and relevance to the search query demonstrates the search system's ability to provide rich, contextual results for SessionStart injection.\n\n---\nType: discovery | Facts: Search API returns observations with title, subtitle, full narrative, structured facts list, concept tags, file paths, creation dates, and observation type; Results tagged with source links using claude-mem://observation/ID format for reference; Worker service startup query retrieved six relevant observations spanning October 18 to November 17, 2025; Retrieved observations include diverse types (change, discovery, bugfix) covering port file handling, getPackageRoot issues, production failures, and ChromaSync initialization; Test result format stores array of observation objects with type and text fields containing formatted markdown content | Concepts: how-it-works, pattern | Files: test-results/test-05-pm2-decision.json, test-results/test-01-worker-service-startup.json\n\n---\nDate: 11/17/2025, 9:52:44 PM\n\n---\n\n## Test Results Return Full Observation Content with Metadata\n*Source: claude-mem://observation/10649*\n\n**Worker service startup query retrieved six relevant observations with complete details, context, and source links**\n\nThe first test result file revealed the structure and quality of search API responses. The worker service startup query successfully retrieved six relevant observations with comprehensive details. Each result includes the observation title, subtitle (one-line summary), full narrative text, structured facts list, concept tags (what-changed, how-it-works, problem-solution, gotcha), associated file paths, and creation dates. Results are tagged with source links using the claude-mem://observation/ID format. The retrieved observations span multiple aspects of worker service startup including port file handling migration, getPackageRoot path resolution issues, production failure analysis, and ChromaSync initialization timing. The diversity of observation types (change, discovery, bugfix) and the relevance to the search query demonstrate that FTS5 keyword search is providing meaningful results despite Chroma semantic search failures.\n\n---\nType: discovery | Facts: Test result contains array of observation objects with type and text fields; Each observation includes title, subtitle, narrative, facts list, concepts, files, and dates; First test query retrieved six observations about worker service startup and configuration; Results include observations about port file writing, getPackageRoot dependencies, production failures, and ChromaSync initialization; Each observation tagged with source link format claude-mem://observation/ID; Observations span date range from October 18, 2025 to November 17, 2025; Results include diverse observation types: change, discovery, and bugfix; Metadata includes observation type, facts list, concepts tags, and file paths | Concepts: how-it-works | Files: test-results/test-01-worker-service-startup.json\n\n---\nDate: 11/17/2025, 9:31:24 PM\n\n---\n\n## Build and Deployment Pipeline Executed\n*Source: claude-mem://observation/10634*\n\n**Built all hooks, worker service, search server, synced to marketplace, and restarted worker.**\n\nA complete build and deployment pipeline was executed for the claude-mem plugin version 6.0.9. The build process compiled all TypeScript hooks (context-hook, new-hook, save-hook, summary-hook, cleanup-hook, user-message-hook), the worker service, and the search server into their final JavaScript distributions. The React-based viewer UI was also bundled with its associated assets including fonts and icons. After successful compilation, the entire plugin directory was synchronized to the Claude marketplace location using rsync, transferring 11,751 files while excluding .git directories. The deployment concluded with an npm install to refresh dependencies and a worker service restart to activate the newly built code. This pipeline ensures that all code changes are properly compiled, deployed, and running in the Claude environment.\n\n---\nType: change | Facts: Package claude-mem version 6.0.9 successfully built all components; Built components include worker-service.cjs (1337.16 KB), search-server.mjs (332.02 KB), and seven hooks ranging from 2.24 KB to 38.29 KB; React viewer bundle built with viewer-bundle.js, viewer.html, fonts, and 4 SVG icon files; Files synced from /Users/alexnewman/Scripts/claude-mem to ~/.claude/plugins/marketplaces/thedotmack/ using rsync with 11,751 files transferred; npm install executed in marketplace directory to ensure dependencies are current; Worker service restarted after successful build and sync | Concepts: what-changed, how-it-works, pattern | Files: plugin/scripts/worker-service.cjs, plugin/scripts/search-server.mjs, plugin/scripts/context-hook.js, plugin/scripts/new-hook.js, plugin/scripts/save-hook.js, plugin/scripts/summary-hook.js, plugin/scripts/cleanup-hook.js, plugin/scripts/user-message-hook.js, plugin/ui/viewer-bundle.js, plugin/ui/viewer.html\n\n---\nDate: 11/17/2025, 9:22:48 PM\n\n---\n\n## Build, Sync, and Worker Restart Executed\n*Source: claude-mem://observation/10622*\n\n**Project built successfully with all hooks and services, synced to marketplace directory, and worker service restarted.**\n\nA complete build, sync, and restart operation was executed for the claude-mem project version 6.0.9. The build process compiled all hook scripts, the worker service, the search server, and the React viewer UI with associated assets. The build system used scripts/build-hooks.js to bundle all components, producing optimized output in the plugin/scripts/ directory. After successful compilation, rsync transferred the entire project to the Claude marketplace plugin directory (~/.claude/plugins/marketplaces/thedotmack/), removing any files not present in the source with the --delete flag. The operation concluded with npm install to ensure all dependencies were properly installed in the marketplace location, preparing the plugin for use.\n\n---\nType: change | Facts: Project version is 6.0.9; Build output includes context-hook (38.29 KB), new-hook (32.14 KB), save-hook (32.49 KB), summary-hook (34.61 KB), cleanup-hook (31.38 KB), and user-message-hook (2.24 KB); Worker service built at 1337.16 KB; Search server built at 331.02 KB; React viewer built successfully with viewer-bundle.js, viewer.html, font assets, and 4 icon SVG files; Project synced from /Users/alexnewman/Scripts/claude-mem to ~/.claude/plugins/marketplaces/thedotmack/ using rsync with delete flag; Transfer included 11751 files with npm install executed in target directory | Concepts: what-changed, how-it-works | Files: plugin/scripts/context-hook.js, plugin/scripts/new-hook.js, plugin/scripts/save-hook.js, plugin/scripts/summary-hook.js, plugin/scripts/cleanup-hook.js, plugin/scripts/user-message-hook.js, plugin/scripts/worker-service.cjs, plugin/scripts/search-server.mjs, plugin/ui/viewer-bundle.js, plugin/ui/viewer.html\n\n---\nDate: 11/17/2025, 9:16:40 PM\n\n---\n\n## Phase 3 Testing Initiated\n*Source: claude-mem://observation/10528*\n\n**Beginning phase 3 testing with the feature enabled and worker process running.**\n\nPhase 3 testing has begun with the necessary components in place. The phase 3 feature has been enabled in the system configuration, and the associated worker process is confirmed to be running. This represents a transition from development/configuration to active testing of phase 3 functionality. The testing phase will validate that the enabled feature and running worker interact correctly and perform as expected.\n\n---\nType: change | Facts: Phase 3 testing has been initiated; Phase 3 feature is currently enabled; Worker process is running and operational | Concepts: what-changed, how-it-works\n\n---\nDate: 11/17/2025, 7:02:59 PM"}] |