c5e68a17c8
* Refactor code structure for improved readability and maintainability * Add test results for search API and related functionalities - Created test result files for various search-related functionalities, including: - test-11-search-server-changes.json - test-12-context-hook-changes.json - test-13-worker-service-changes.json - test-14-patterns.json - test-15-gotchas.json - test-16-discoveries.json - test-17-all-bugfixes.json - test-18-all-features.json - test-19-all-decisions.json - test-20-session-search.json - test-21-prompt-search.json - test-22-decisions-endpoint.json - test-23-changes-endpoint.json - test-24-how-it-works-endpoint.json - test-25-contextualize-endpoint.json - test-26-timeline-around-observation.json - test-27-multi-param-combo.json - test-28-file-type-combo.json - Each test result file captures specific search failures or outcomes, including issues with undefined properties and successful execution of search queries. - Enhanced documentation of search architecture and testing strategies, ensuring compliance with established guidelines and improving overall search functionality. * feat: Enhance unified search API with catch-all parameters and backward compatibility - Implemented a unified search API at /api/search that accepts catch-all parameters for filtering by type, observation type, concepts, and files. - Maintained backward compatibility by keeping granular endpoints functional while routing through the same infrastructure. - Completed comprehensive testing of search capabilities with real-world query scenarios. fix: Address missing debug output in search API query tests - Flushed PM2 logs and executed search queries to verify functionality. - Diagnosed absence of "Raw Chroma" debug messages in worker logs, indicating potential issues with logging or query processing. refactor: Improve build and deployment pipeline for claude-mem plugin - Successfully built and synced all hooks and services to the marketplace directory. - Ensured all dependencies are installed and up-to-date in the deployment location. feat: Implement hybrid search filters with 90-day recency window - Enhanced search server to apply a 90-day recency filter to Chroma results before categorizing by document type. fix: Correct parameter handling in searchUserPrompts method - Added support for filter-only queries and improved dual-path logic for clarity. refactor: Rename FTS5 method to clarify fallback status - Renamed escapeFTS5 to escapeFTS5_fallback_when_chroma_unavailable to indicate its temporary usage. feat: Introduce contextualize tool for comprehensive project overview - Added a new tool to fetch recent observations, sessions, and user prompts, providing a quick project overview. feat: Add semantic shortcut tools for common search patterns - Implemented 'decisions', 'changes', and 'how_it_works' tools for convenient access to frequently searched observation categories. feat: Unified timeline tool supports anchor and query modes - Combined get_context_timeline and get_timeline_by_query into a single interface for timeline exploration. feat: Unified search tool added to MCP server - New tool queries all memory types simultaneously, providing combined chronological results for improved search efficiency. * Refactor search functionality to clarify FTS5 fallback usage - Updated `worker-service.cjs` to replace FTS5 fallback function with a more descriptive name and improved error handling. - Enhanced documentation in `SKILL.md` to specify the unified API endpoint and clarify the behavior of the search engine, including the conditions under which FTS5 is used. - Modified `search-server.ts` to provide clearer logging and descriptions regarding the fallback to FTS5 when UVX/Python is unavailable. - Renamed and updated the `SessionSearch.ts` methods to reflect the conditions for using FTS5, emphasizing the lack of semantic understanding in fallback scenarios. * feat: Add ID-based fetch endpoints and simplify mem-search skill **Problem:** - Search returns IDs but no way to fetch by ID - Skill documentation was bloated with too many options - Claude wasn't using IDs because we didn't tell it how **Solution:** 1. Added three new HTTP endpoints: - GET /api/observation/:id - GET /api/session/:id - GET /api/prompt/:id 2. Completely rewrote SKILL.md: - Stripped complexity down to essentials - Clear 3-step prescriptive workflow: Search → Review IDs → Fetch by ID - Emphasized ID usage: "The IDs are there for a reason - USE THEM" - Removed confusing multi-endpoint documentation - Kept only unified search with filters **Impact:** - Token efficiency: Claude can now fetch full details only for relevant IDs - Clarity: One clear workflow instead of 10+ options to choose from - Usability: IDs are no longer wasted context - they're actionable 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: Move internal docs to private directory Moved POSTMORTEM and planning docs to ./private to exclude from PR reviews. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: Remove experimental contextualize endpoint - Removed contextualize MCP tool from search-server (saves ~4KB) - Disabled FTS5 fallback paths in SessionSearch (now vector-first) - Cleaned up CLAUDE.md documentation - Removed contextualize-rewrite-plan.md doc Rationale: - Contextualize is better suited as a skill (LLM-powered) than an endpoint - Search API already provides vector search with configurable limits - Created issue #132 to track future contextualize skill implementation Changes: - src/servers/search-server.ts: Removed contextualize tool definition - src/services/sqlite/SessionSearch.ts: Disabled FTS5 fallback, added deprecation warnings - CLAUDE.md: Cleaned up outdated skill documentation - docs/: Removed contextualize plan document 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: Complete FTS5 cleanup - remove all deprecated search code This completes the FTS5 cleanup work by removing all commented-out FTS5 search code while preserving database tables for backward compatibility. Changes: - Removed 200+ lines of commented FTS5 search code from SessionSearch.ts - Removed deprecated degraded_search_query__when_uvx_unavailable method - Updated all method documentation to clarify vector-first architecture - Updated class documentation to reflect filter-only query support - Updated CLAUDE.md to remove FTS5 search references - Clarified that FTS5 tables exist for backward compatibility only - Updated "Why SQLite FTS5" section to "Why Vector-First Search" Database impact: NONE - FTS5 tables remain intact for existing installations Search architecture: - ChromaDB: All text-based vector search queries - SQLite: Filter-only queries (date ranges, metadata, no query text) - FTS5 tables: Maintained but unused (backward compatibility) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: Remove all FTS5 fallback execution code from search-server Completes the FTS5 cleanup by removing all fallback execution paths that attempted to use FTS5 when ChromaDB was unavailable. Changes: - Removed all FTS5 fallback code execution paths - When ChromaDB fails or is unavailable, return empty results with helpful error messages - Updated all deprecated tool descriptions (search_observations, search_sessions, search_user_prompts) - Changed error messages to indicate FTS5 fallback has been removed - Added installation instructions for UVX/Python when vector search is unavailable - Updated comments from "hybrid search" to "vector-first search" - Removed ~100 lines of dead FTS5 fallback code Database impact: NONE - FTS5 tables remain intact (backward compatibility) Search behavior when ChromaDB unavailable: - Text queries: Return empty results with error explaining ChromaDB is required - Filter-only queries (no text): Continue to work via direct SQLite 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix: Address PR 133 review feedback Critical fixes: - Remove contextualize endpoint from worker-service (route + handler) - Fix build script logging to show correct .cjs extension (was .mjs) Documentation improvements: - Add comprehensive FTS5 retention rationale documentation - Include v7.0.0 removal TODO for future cleanup Testing: - Build succeeds with correct output logging - Worker restarts successfully (30th restart) - Contextualize endpoint properly removed (404 response) - Search endpoint verified working This addresses all critical review feedback from PR 133. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
1 line
8.7 KiB
JSON
1 line
8.7 KiB
JSON
[{"type":"text","text":"## Added PreToolUse hook to lifecycle configuration\n*Source: claude-mem://observation/10541*\n\n**New PreToolUse hook executes pre-tool-use-hook.js before any tool is invoked by Claude.**\n\nA new PreToolUse lifecycle hook was added to the claude-mem plugin configuration to enable capturing state before tools are executed. This complements the existing PostToolUse hook that runs save-hook.js after tool execution. By adding pre-tool-use-hook.js with the same wildcard matcher and timeout settings as PostToolUse, the system can now monitor and potentially intercept or prepare for tool invocations before they occur. This creates a complete before/after hook pattern for tool usage tracking.\n\n---\nType: feature | Facts: Added PreToolUse hook section to plugin/hooks/hooks.json before the existing PostToolUse hook; PreToolUse hook runs pre-tool-use-hook.js script with 120s timeout; PreToolUse hook uses matcher \"*\" to trigger on all tool invocations; The hook system now captures both before and after states of tool usage | Concepts: what-changed, pattern, why-it-exists | Files: plugin/hooks/hooks.json\n\n---\nDate: 11/17/2025, 7:16:47 PM\n\n---\n\n## Claude-mem plugin hooks configuration structure\n*Source: claude-mem://observation/10539*\n\n**The hooks.json file defines five lifecycle hooks that trigger Node.js scripts at different session stages.**\n\nThe hooks.json configuration file reveals the complete lifecycle management system for the claude-mem plugin. It defines five distinct lifecycle events with corresponding Node.js scripts. At session start, the system performs installation checks and loads context. During the session, it captures user prompts and saves tool usage data. At session stop and end, it generates summaries and performs cleanup. All scripts are referenced using ${CLAUDE_PLUGIN_ROOT} environment variable for portability, and each has defined timeout limits to prevent hanging operations.\n\n---\nType: discovery | Facts: File plugin/hooks/hooks.json configures SessionStart, UserPromptSubmit, PostToolUse, Stop, and SessionEnd hooks; SessionStart hook runs smart-install.js, context-hook.js, and user-message-hook.js with timeouts of 300s and 10s; UserPromptSubmit hook executes new-hook.js with 120s timeout on every user prompt submission; PostToolUse hook runs save-hook.js with 120s timeout after any tool use (matcher: \"*\"); Stop hook triggers summary-hook.js and SessionEnd triggers cleanup-hook.js, both with 120s timeouts; SessionStart hooks use matcher \"startup|clear|compact\" to control when they execute | Concepts: how-it-works, why-it-exists, pattern | Files: plugin/hooks/hooks.json\n\n---\nDate: 11/17/2025, 7:13:54 PM\n\n---\n\n## Phase 3: Conditional Blocking Logic Implemented in Save Hook\n*Source: claude-mem://observation/10478*\n\n**Save hook now conditionally blocks based on Endless Mode configuration, waits for observations, and transforms transcripts.**\n\nPhase 3 of the Endless Mode implementation has been completed in the save-hook.ts file. The hook now implements conditional blocking logic based on the EndlessModeConfig.enabled flag. When Endless Mode is enabled (and tool_use_id and transcript_path are available), the hook switches to synchronous mode by appending wait_until_obs_is_saved=true to the endpoint URL and increasing the timeout from 2 to 90 seconds. After sending the observation request, the hook awaits the JSON response and handles three possible states. On 'completed' status, it receives the observation data and calls transformTranscript() to replace the tool output in the JSONL file with compressed observation markdown. On 'timeout' status, it logs a warning and falls back to using the full output while the observation continues processing in the background. On 'queued' status (async mode), it simply logs success. The implementation includes error handling for transcript transformation failures that allows execution to continue even if compression fails. This completes the integration of all three phases: synchronous observation endpoint (Phase 1), transcript transformation logic (Phase 2), and conditional blocking behavior (Phase 3).\n\n---\nType: feature | Facts: Save hook checks EndlessModeConfig.getConfig().enabled to determine blocking behavior; Blocking mode requires enabled flag, tool_use_id, and transcript_path to all be present; Endpoint URL conditionally includes query parameter wait_until_obs_is_saved=true for synchronous mode; Timeout increased from 2 seconds to 90 seconds when Endless Mode is enabled; Hook awaits response JSON and handles three states: completed, timeout, and queued; On status 'completed', hook calls transformTranscript() to replace tool output with compressed observation; On status 'timeout', hook logs warning and falls back to full output while observation completes asynchronously; Transcript transformation errors are caught and logged but don't fail the hook execution; Async mode behavior remains unchanged when Endless Mode is disabled | Concepts: how-it-works, what-changed, pattern | Files: src/hooks/save-hook.ts\n\n---\nDate: 11/17/2025, 6:36:54 PM\n\n---\n\n## Endless Mode Implementation Plan Documentation Created\n*Source: claude-mem://observation/10420*\n\n**Comprehensive four-phase plan documented for completing Endless Mode with blocking observation creation and transcript transformation.**\n\nA comprehensive implementation plan was documented to complete the Endless Mode feature. The plan acknowledges that infrastructure components (TransformLayer.ts, EndlessModeConfig.ts, tool_use_id tracking) are already in place on the experiment/endless-mode branch, but critical components are missing. The implementation is divided into four phases: Phase 1 creates a synchronous observation endpoint that blocks until observation creation completes (with timeout fallback), Phase 2 implements transcript file transformation to replace full tool outputs with compressed observations on disk, Phase 3 adds conditional blocking behavior to the save-hook based on configuration, and Phase 4 defines comprehensive testing procedures. The plan includes specific success criteria for each phase and overall success metrics targeting 80-95% token reduction while maintaining sub-120s hook execution times. This completes the architecture described in endless-mode-blocking-hooks-plan.md and supersedes the earlier UserPromptSubmit hook approach.\n\n---\nType: decision | Facts: Created endless-mode-implementation-plan.md in docs/context directory; Infrastructure is complete including TransformLayer.ts, EndlessModeConfig.ts, and tool_use_id tracking; Missing components identified: synchronous observation endpoint, transcript file transformation, and blocking save-hook behavior; Phase 1 focuses on creating synchronous observation endpoint in worker-service.ts with 60-90s wait time; Phase 2 implements transcript transformation in save-hook.ts to replace full tool results with compressed observation markdown; Phase 3 adds conditional blocking logic to save-hook based on EndlessModeConfig with timeout fallback; Phase 4 defines testing and validation procedures targeting 80-95% token reduction; TransformLayer.ts currently only handles in-memory SDK transformation, not on-disk transcript transformation | Concepts: pattern, trade-off, how-it-works, problem-solution | Files: docs/context/endless-mode-implementation-plan.md\n\n---\nDate: 11/17/2025, 4:00:05 PM\n\n---\n\n## Save-hook sends tool_use_id to worker observation endpoint\n*Source: claude-mem://observation/10406*\n\n**Modified POST request body to include extracted tool_use_id alongside tool data for observation storage**\n\nThe final portion of the save-hook changes completes the tool_use_id integration by including it in the HTTP request body sent to the worker service. After extracting the tool_use_id from the transcript via getLatestToolUseId, the hook adds it to the observation data payload alongside the tool name, inputs, outputs, prompt number, and working directory. This ensures the worker service receives the linking information needed to store observations with their corresponding tool_use_id in the database. The logging also captures this ID for debugging, displaying either the actual ID or '(none)' when extraction fails. This completes the data flow from transcript parsing through observation storage, establishing the foundation for EndlessMode's later retrieval and compression operations.\n\n---\nType: feature | Facts: POST request to worker observations endpoint now includes tool_use_id field in body; Tool_use_id added alongside existing fields: tool_name, tool_input, tool_response, prompt_number, cwd; Request maintains 2-second timeout via AbortSignal; Logger output includes toolUseId field showing extracted ID or '(none)' if not found | Concepts: what-changed, how-it-works | Files: src/hooks/save-hook.ts\n\n---\nDate: 11/17/2025, 3:49:43 PM"}] |