Fixes#353 - Observations not being saved due to incomplete pending messages implementation
Changes:
- PendingMessageStore.markProcessed() now clears tool_input and tool_response
- PendingMessageStore.cleanupProcessed() changed from time-based to count-based retention
- Keeps most recent 100 processed messages for UI display
- SDKAgent.processSDKResponse() calls cleanup after marking messages processed
This prevents the database from growing unbounded with duplicate transcript data.
The pending_messages table now only stores full transcripts for pending/processing
messages, while processed messages keep metadata only.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This is a backup of all work done by the 3 Phase 1 agents:
Agent A - Command Injection Fix (Issue #354):
- Fixed command injection in BranchManager.ts
- Fixed unnecessary shell usage in bun-path.ts
- Added comprehensive security test suite
- Created SECURITY.md and SECURITY_AUDIT_REPORT.md
Agent B - Observation Persistence Fix (Issue #353):
- Added PendingMessageStore from PR #335
- Integrated persistent queue into SessionManager
- Modified SDKAgent to mark messages complete
- Updated SessionStore with pending_messages migration
- Updated worker-types.ts with new interfaces
Agent C - Batch Endpoint Verification (Issue #348):
- Created batch-observations.test.ts
- Updated worker-service.mdx documentation
Also includes:
- Documentation context files (biomimetic, windows struggles)
- Build artifacts from agent testing
This work will be re-evaluated after v7.3.0 release.
* feat: Add batch fetching for observations and update documentation
- Implemented a new endpoint for fetching multiple observations by IDs in a single request.
- Updated the DataRoutes to include a POST /api/observations/batch endpoint.
- Enhanced SKILL.md documentation to reflect changes in the search process and batch fetching capabilities.
- Increased the default limit for search results from 5 to 40 for better usability.
* feat!: Fix timeline parameter passing with SearchManager alignment
BREAKING CHANGE: Timeline MCP tools now use standardized parameter names
- anchor_id → anchor
- before → depth_before
- after → depth_after
- obs_type → type (timeline tool only)
Fixes timeline endpoint failures caused by parameter name mismatch between
MCP layer and SearchManager. Adds new SessionStore methods for fetching
prompts and session summaries by ID.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* docs: reframe timeline parameter fix as bug fix, not breaking change
The timeline tools were completely broken due to parameter name mismatch.
There's nothing to migrate from since the old parameters never worked.
Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
* Refactor mem-search documentation and optimize API tool definitions
- Updated SKILL.md to emphasize batch fetching for observations, clarifying usage and efficiency.
- Removed deprecated tools from mcp-server.ts and streamlined tool definitions for clarity.
- Enhanced formatting in FormattingService.ts for better output readability.
- Adjusted SearchManager.ts to improve result headers and removed unnecessary search tips from combined text.
* Refactor FormattingService and SearchManager for table-based output
- Updated FormattingService to format search results as tables, including methods for formatting observations, sessions, and user prompts.
- Removed JSON format handling from SearchManager and streamlined result formatting to consistently use table format.
- Enhanced readability and consistency in search tips and formatting logic.
- Introduced token estimation for observations and improved time formatting.
* refactor: update documentation and API references for version bump and search functionalities
* Refactor code structure for improved readability and maintainability
* chore: change default model from haiku to sonnet
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: unify timeline formatting across search and context services
Extract shared timeline formatting utilities into reusable module to align
MCP search output format with context-generator's date/file-grouped format.
Changes:
- Create src/shared/timeline-formatting.ts with reusable utilities
(parseJsonArray, formatDateTime, formatTime, formatDate, toRelativePath,
extractFirstFile, groupByDate)
- Refactor context-generator.ts to use shared utilities
- Update SearchManager.search() to use date/file grouping
- Add search-specific row formatters to FormattingService
- Fix timeline methods to extract actual file paths from metadata
instead of hardcoding 'General'
- Remove Work column from search output (kept in context output)
Result: Consistent date/file-grouped markdown formatting across both
systems while maintaining their different column requirements.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* refactor: remove redundant legend from search output
Remove legend from search/timeline results since it's already shown
in SessionStart context. Saves ~30 tokens per search result.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Refactor session summary rendering to remove links
- Removed link generation for session summaries in context generation and search manager.
- Updated output formatting to exclude links while maintaining the session summary structure.
- Adjusted related components in TimelineService to ensure consistency across the application.
* fix: move skillPath declaration outside try block to fix scoping bug
The skillPath variable was declared inside the try block but referenced
in the catch block for error logging. Since const is block-scoped, this
would cause a ReferenceError when the error handler executes.
Moved skillPath declaration before the try block so it's accessible in
both try and catch scopes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: address PR #317 code review feedback
**Critical Fixes:**
- Replace happy_path_error__with_fallback debug calls with proper logger methods in mcp-server.ts
- All HTTP API calls now use logger.debug/error for consistent logging
**Code Quality Improvements:**
- Extract 90-day recency window magic numbers to named constants
- Added RECENCY_WINDOW_DAYS and RECENCY_WINDOW_MS constants in SearchManager
**Documentation:**
- Document model cost implications of Haiku → Sonnet upgrade in CHANGELOG
- Provide clear migration path for users who want to revert to Haiku
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* refactor: simplify CHANGELOG - remove cost documentation
Removed model cost comparison documentation per user feedback.
Kept only the technical code quality improvements.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
- Removed all instances of happy_path_error__with_fallback from various hooks, services, and utilities.
- Introduced logger.happyPathError for consistent logging of unexpected nulls and fallback values.
- Updated the logger utility to include a new happyPathError method with enhanced context and stack trace.
- Deprecated silent-debug utility as all logging functionality has been migrated to the logger.
* Enhance error logging in hooks
- Added detailed error logging in context-hook, new-hook, save-hook, and summary-hook to capture status, project, port, and relevant session information on failures.
- Improved error messages thrown in save-hook and summary-hook to include specific context about the failure.
* Refactor migration logging to use console.log instead of console.error
- Updated SessionSearch and SessionStore classes to replace console.error with console.log for migration-related messages.
- Added notes in the documentation to clarify the use of console.log for migration messages due to the unavailability of the structured logger during constructor execution.
* Refactor SDKAgent and silent-debug utility to simplify error handling
- Updated SDKAgent to use direct defaults instead of happy_path_error__with_fallback for non-critical fields such as last_user_message, last_assistant_message, title, filesRead, filesModified, concepts, and summary.request.
- Enhanced silent-debug documentation to clarify appropriate use cases for happy_path_error__with_fallback, emphasizing its role in handling unexpected null/undefined values while discouraging its use for nullable fields with valid defaults.
* fix: correct happy_path_error__with_fallback usage to prevent false errors
Fixes false "Missing cwd" and "Missing transcript_path" errors that were
flooding silent.log even when values were present.
Root cause: happy_path_error__with_fallback was being called unconditionally
instead of only when the value was actually missing.
Pattern changed from:
value: happy_path_error__with_fallback('Missing', {}, value || '')
To correct usage:
value: value || happy_path_error__with_fallback('Missing', {}, '')
Fixed in:
- src/hooks/save-hook.ts (PostToolUse hook)
- src/hooks/summary-hook.ts (Stop hook)
- src/services/worker/http/routes/SessionRoutes.ts (2 instances)
Impact: Eliminates false error noise, making actual errors visible.
Addresses issue #260 - users were seeing "Missing cwd" errors despite
Claude Code correctly passing all required fields.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Enhance error logging and handling across services
- Improved error messages in SessionStore to include project context when fetching boundary observations and timestamps.
- Updated ChromaSync error handling to provide more informative messages regarding client initialization failures, including the project context.
- Enhanced error logging in WorkerService to include the package path when reading version fails.
- Added detailed error logging in worker-utils to capture expected and running versions during health checks.
- Extended WorkerErrorMessageOptions to include actualError for more informative restart instructions.
* Refactor error handling in hooks to use standardized fetch error handler
- Introduced a new error handler `handleFetchError` in `shared/error-handler.ts` to standardize logging and user-facing error messages for fetch failures across hooks.
- Updated `context-hook.ts`, `new-hook.ts`, `save-hook.ts`, and `summary-hook.ts` to utilize the new error handler, improving consistency and maintainability.
- Removed redundant imports and error handling logic related to worker restart instructions from the hooks.
* feat: add comprehensive error handling tests for hooks and ChromaSync client
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
- Implemented a method to check if Bun is available in the system PATH.
- Updated the startWithBun method to return an error if Bun is not found.
- Enhanced PID file parsing to validate required fields and their types.
- Cleaned up stale PID files if the process is no longer alive.
fix(SettingsRoutes): clear port cache after updating settings
- Added a call to clearPortCache after writing updated settings to ensure the application uses the latest configuration.
Add feature/bun-executable to the allowed branches list in the Version Channel selector, enabling users to switch to this branch from the settings UI.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Alex Newman <thedotmack@users.noreply.github.com>
- Replaced hardcoded migration port with dynamic port retrieval using `getWorkerPort()` in worker-cli.ts.
- Updated context generator to clarify error handling comments.
- Introduced timeout constants in ProcessManager for better maintainability.
- Configured SQLite settings using constants for mmap size and cache size in DatabaseManager.
- Added timeout constants for Git and NPM commands in BranchManager.
- Enhanced error logging in FormattingService and SearchManager to provide more context on failures.
- Removed deprecated silentDebug function and replaced its usage with logger.debug.
- Updated tests to use dynamic worker port retrieval instead of hardcoded values.
Critical bug fix:
- Pass project filter to getSessionSummariesByIds() and getUserPromptsByIds() in SearchManager
- Previously only observations were filtered by project, sessions and prompts leaked from other projects
Documentation improvements:
- Update "FTS5 search" to "hybrid search" (accurate terminology)
- Add privacy warning about sensitive data in exports
- Document --project parameter for filtered exports
- Add "Export by Project" examples to advanced usage
Verified with test export using --project=claude-mem filter.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Two critical fixes for the memory export/import feature:
1. **SearchManager empty results bug**: The /api/search endpoint
with format=json now returns {observations: [], sessions: [],
prompts: []} when no results are found, instead of the MCP
protocol format. This fixes the export-memories script which
expects consistent JSON structure regardless of result count.
2. **Project filtering support**: Updated SessionStore methods
(getObservationsByIds, getSessionSummariesByIds, getUserPromptsByIds)
to accept and apply project filter parameter. This enables
proper project-based filtering during ChromaDB hybrid search
result hydration.
Testing:
- Export with results: ✅ 50 observations exported
- Export with empty results: ✅ Proper JSON structure
- Round-trip import: ✅ Duplicate prevention working
- Project filtering: ✅ claude-mem (51 obs) vs rad-mem (1 obs)
Fixes export/import feature blocking bugs.
All search route handlers were returning only result.content (array) instead
of the complete result object containing {content, isError}. This caused the
MCP server to receive an invalid response format, resulting in all search
tools hanging indefinitely.
Changes:
- Updated 14 route handlers to return full result object
- Ensures MCP protocol compatibility by providing expected response structure
Fixes search functionality for all claude-mem search tools including:
- Unified search, timeline, decisions, changes
- Observations, sessions, prompts search
- Concept, file, type filtering
- Recent context and timeline queries
* Fix critical session persistence bug in new-hook
Restores the second POST call to /sessions/:sessionDbId/init that was
incorrectly removed as "duplicate" in the hooks refactor.
The two POST calls serve different purposes:
- POST /api/sessions/init: Creates DB session, saves prompt (no SDK agent)
- POST /sessions/:sessionDbId/init: Starts the SDK agent session
Without the second call, the SDK agent never started, causing:
- First prompts saved to DB but never processed
- Subsequent prompts queued but no agent running to consume them
- Each new prompt creating orphaned sessions instead of continuing
This fix restores proper session continuation across multiple prompts.
Fixes conversation fragmentation reported in production.
* updated agent-sdk
* Skip meta-observations for session-memory file operations
Added a check to skip meta-observations when file operations (Edit, Write, Read, NotebookEdit) are performed on session-memory files. If the file path includes 'session-memory', a debug log is generated and the response indicates that the observation was skipped.
* refactor: Clean up hook-response and new-hook files
- Removed 'PreCompact' hook type and associated logic from hook-response.ts for improved type safety.
- Deleted extensive architecture comments in new-hook.ts to streamline code readability.
- Simplified debug logging in new-hook.ts to reduce verbosity.
- Enhanced ensureWorkerRunning function in worker-utils.ts with a final health check before throwing errors.
- Added a new documentation file outlining the hooks cleanup process and future improvements.
* Refactor cleanup and user message hooks
- Updated cleanup-hook.js to improve error handling and remove unnecessary input checks.
- Simplified user-message-hook.js by removing time-sensitive announcements and streamlining output.
- Enhanced logging functionality in both hooks for better debugging and clarity.
* Refactor error handling in hooks to use centralized error handler
- Introduced `handleWorkerError` function in `src/shared/hook-error-handler.ts` to manage worker-related errors.
- Updated `context-hook.ts`, `new-hook.ts`, `save-hook.ts`, and `summary-hook.ts` to utilize the new error handler, simplifying error management and improving code readability.
- Removed repetitive error handling logic from individual hooks, ensuring consistent user-friendly messages for connection issues.
* Refactor user-message and summary hooks to utilize shared transcript parser; introduce hook exit codes
- Moved user message extraction logic to a new shared module `transcript-parser.ts` for better code reuse.
- Updated `summary-hook.ts` to use the new `extractLastMessage` function for retrieving user and assistant messages.
- Replaced direct exit code usage in `user-message-hook.ts` with constants from `hook-constants.ts` for improved readability and maintainability.
- Added `HOOK_EXIT_CODES` to `hook-constants.ts` to standardize exit codes across hooks.
* Refactor hook input interfaces to enforce required fields
- Updated `SessionStartInput`, `UserPromptSubmitInput`, `PostToolUseInput`, and `StopInput` interfaces to require `session_id`, `transcript_path`, and `cwd` fields, ensuring better type safety and clarity in hook inputs.
- Removed optional index signatures from these interfaces to prevent unintended properties and improve code maintainability.
- Adjusted related hook implementations to align with the new interface definitions.
* Refactor save-hook to remove tool skipping logic; enhance summary-hook to handle spinner stopping with error logging; update SessionRoutes to load skip tools from settings; add CLAUDE_MEM_SKIP_TOOLS to SettingsDefaultsManager for configurable tool exclusion.
* Document CLAUDE_MEM_SKIP_TOOLS setting in public docs
Added documentation for the new CLAUDE_MEM_SKIP_TOOLS configuration
setting in response to PR review feedback. Users can now discover and
customize which tools are excluded from observations.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
**Problem:**
Worker service crashed on startup with:
TypeError: Cannot read properties of undefined (reading 'get')
at new Wd (.../worker-service.cjs:52:131469)
**Root Cause:**
Circular dependency between SettingsDefaultsManager and logger:
1. SettingsDefaultsManager imports logger
2. logger imports SettingsDefaultsManager
3. logger constructor calls SettingsDefaultsManager.get() at init time
4. When CommonJS resolves the cycle, SettingsDefaultsManager is undefined
**Solution:**
Break the circular dependency by making logger lazy-load its configuration:
- Change logger.level from initialized in constructor to lazy-loaded
- Add getLevel() method that loads on first access
- Update all level checks to use getLevel()
This allows SettingsDefaultsManager to import logger without triggering
the circular dependency, since logger no longer accesses SettingsDefaultsManager
during module initialization.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Moved SettingsDefaultsManager from worker/settings to shared directory.
- Updated all import paths across the codebase to reflect the new location.
- Removed early-settings.ts as its functionality is now handled by SettingsDefaultsManager.
- Adjusted logger and paths to utilize SettingsDefaultsManager for configuration values.
- Replaced instances of silentDebug with happy_path_error__with_fallback across multiple files to improve error logging and handling.
- Updated the utility function to provide clearer semantics for error handling when expected values are missing.
- Introduced a script to find potential silent failures in the codebase that may need to be addressed with the new error handling approach.
- Removed direct database operations in new-hook.ts and replaced them with an HTTP call to initialize sessions.
- Added error handling for HTTP requests and improved logging for session initialization.
- Updated SessionRoutes to handle new session initialization and privacy checks.
- Enhanced privacy tag stripping logic to prevent saving fully private prompts.
- Improved overall error handling and debugging messages throughout the session management process.
- Updated paths in troubleshooting documentation to reflect new settings file location.
- Modified diagnostics and reference files to read from ~/.claude-mem/settings.json.
- Introduced getWorkerPort utility for cleaner worker port retrieval.
- Enhanced ChromaSync and SDKAgent to load Python version and Claude path from settings.
- Updated SettingsRoutes to validate new settings: CLAUDE_MEM_LOG_LEVEL and CLAUDE_MEM_PYTHON_VERSION.
- Added early-settings module to load settings for logger and other early-stage modules.
- Adjusted logger to use early-loaded log level setting.
- Refactored paths to utilize early-loaded data directory setting.
- Updated all documentation references from search-server to mcp-server
- Removed legacy search-server.cjs file
- Updated debug log messages to use [mcp-server] prefix
- Updated build output references in docs
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Delete empty SearchRoutes.ts.new leftover file
- Standardize on 127.0.0.1 in context-hook.ts (avoids DNS lookup)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Introduced PrivacyCheckValidator class to encapsulate logic for checking if user prompts are private.
- Updated SessionRoutes to utilize PrivacyCheckValidator for determining prompt privacy during observation and summarization operations.
- Removed duplicate privacy check logic from SessionRoutes, improving code maintainability and readability.
- Added SessionEventBroadcaster to handle broadcasting of session lifecycle events, consolidating SSE broadcasting and processing status updates.
- Refactored SessionRoutes to utilize SessionEventBroadcaster for broadcasting events related to new prompts, session starts, and completions.
- Created SessionCompletionHandler to centralize session completion logic, reducing duplication across multiple endpoints.
- Updated WorkerService to initialize SessionEventBroadcaster and pass it to SessionRoutes.
- Introduced SettingsDefaultsManager to centralize default settings and loading logic.
- Updated context-generator, SDKAgent, SettingsRoutes, and worker-utils to utilize the new manager for loading settings.
- Removed redundant code for reading settings from files and environment variables.
- Ensured fallback to default values when settings file is missing or invalid.
- Introduced BaseRouteHandler class to centralize error handling and response management.
- Updated SettingsRoutes to use wrapHandler for automatic error logging and response.
- Refactored ViewerRoutes to extend BaseRouteHandler and utilize wrapHandler for health check and UI serving.
- Enhanced error handling in SettingsRoutes and ViewerRoutes for better maintainability and readability.
Created database.ts with proper database result types and replaced 38+ 'as any' casts throughout the codebase with proper type annotations.
Changes:
- Created src/types/database.ts with TableColumnInfo, IndexInfo, and database record types
- Fixed all type casts in SessionStore.ts (migrations, query results)
- Fixed type casts in SessionSearch.ts, SettingsManager.ts, SettingsRoutes.ts
- Improved MCP server JSON schema typing
All builds pass and worker service runs successfully.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Extracted SQL query from handleSessionInit route handler into SessionStore.getLatestUserPrompt()
method to fix abstraction leak and improve type safety.
Changes:
- Added getLatestUserPrompt() method to SessionStore with proper return type
- Replaced raw SQL query in SessionRoutes with type-safe method call
- Removed direct database access through dbManager.getSessionStore().db
- Improved separation of concerns (data layer vs HTTP layer)
Benefits:
- Type safety: Explicit return type instead of 'as any' cast
- Maintainability: SQL query logic belongs in data layer
- Testability: Can test query logic independently from HTTP layer
- Consistency: Follows existing pattern of query methods in SessionStore
Phase 3 Complete: SQL abstraction leak fixed in session init endpoint.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Extracted duplicated generator auto-start logic (44 lines × 4 occurrences) into
single private method ensureGeneratorRunning() that accepts source parameter.
Benefits:
- Reduced code duplication from 72 lines to 24 lines (net -48 lines)
- Single source of truth for generator lifecycle management
- Improved maintainability - changes only needed in one place
- Better logging with parameterized source tracking
Phase 2 Complete: All 4 handler methods now use shared method:
- handleObservations (legacy endpoint)
- handleSummarize (legacy endpoint)
- handleObservationsByClaudeId (new endpoint)
- handleSummarizeByClaudeId (new endpoint)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Added optional chaining and nullish coalescing to handle potential undefined values in Chroma results and timeline items.
- Updated logging statements to provide clearer information when no results are found.
- Refactored destructuring of parameters in findByConcept and findByFile methods for consistency.
- Introduced SearchManager to handle search operations directly instead of proxying to MCP server.
- Updated WorkerService to initialize SearchManager after database setup.
- Modified SearchRoutes to call SearchManager methods for search operations.
- Adjusted SearchManager to manage timeline and formatting services.
- Enhanced error handling and logging for search operations.
- Extracted timeline-related functionality from mcp-server.ts to a dedicated TimelineService class.
- Added methods to build, filter, and format timeline items based on observations, sessions, and prompts.
- Introduced interfaces for TimelineItem and TimelineData to standardize data structures.
- Implemented sorting and grouping of timeline items by date, with markdown formatting for output.
- Included utility methods for date and time formatting, as well as token estimation.
The /api/context/inject endpoint was using an incorrect relative import
path for context-generator.cjs. Since SearchRoutes is in
src/services/worker/http/routes/, the correct path to reach
src/services/context-generator.ts is ../../../context-generator.js
This fixes the session start context injection issue where the hook
would print correctly but no context would load in the session.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
Major architectural improvements to the worker service:
- Extracted monolithic WorkerService (~1900 lines) into organized route classes
- New HTTP layer with dedicated route handlers:
- SessionRoutes: Session lifecycle operations
- DataRoutes: Data retrieval endpoints
- SearchRoutes: Search/MCP proxy operations
- SettingsRoutes: Settings and configuration
- ViewerRoutes: Health, UI, and SSE streaming
- Added comprehensive README documenting worker architecture
- Improved build script to handle worker service compilation
- Added context-generator for hook context operations
This is Phase 1 of worker refactoring - pure code reorganization with zero
functional changes. All existing behavior preserved while improving
maintainability and code organization.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: Hide console window on Windows when spawning child processes
Add windowsHide: true to spawnSync and execSync calls to prevent
empty console windows from appearing on Windows when hooks execute.
Fixes two spawn points:
- worker-utils.ts: PM2 spawn when starting worker service
- user-message-hook.ts: Node spawn for context display
Reference: https://nodejs.org/api/child_process.html (windowsHide option)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: Add windowsHide to remaining execSync calls for complete Windows console window hiding
This completes the Windows console window fix by adding `windowsHide: true` to all remaining `execSync` calls:
- src/services/worker-service.ts:220 - pgrep command for orphaned process detection
- src/services/worker-service.ts:226 - pkill command for process cleanup
- src/services/worker/SDKAgent.ts:414 - where/which claude command for finding executable
These operations are less frequent than the user-prompt hook, but should still avoid spawning console windows on Windows for a complete fix.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
Adds Version Channel section to Settings sidebar allowing users to:
- See current branch (main or beta/7.0) and stability status
- Switch to beta branch to access Endless Mode features
- Switch back to stable for production use
- Pull updates for current branch
Implementation:
- BranchManager.ts: Git operations for branch detection/switching
- worker-service.ts: /api/branch/* endpoints (status, switch, update)
- Sidebar.tsx: Version Channel UI with branch state and handlers
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
**Problem:**
- Summaries created with empty project names even after database fix
- 13 summaries had empty project, couldn't appear in context-hook
- In-memory sessions cached stale empty project value
**Root Cause:**
SessionManager caches ActiveSession objects in memory. When reusing
an existing session, it never refreshed the project field from the
database. Even though createSDKSession() now updates the DB with the
correct project, the in-memory session.project remained empty, causing
summaries to be stored with empty project.
**Flow:**
1. Session created with empty project (cached in memory)
2. new-hook UPDATE fixes project in database
3. SessionManager reuses cached session (stale project="")
4. Summary stored with session.project (empty)
5. Context-hook can't find summary (WHERE project = 'claude-mem')
**Solution:**
When SessionManager.initializeSession() reuses an existing session,
refresh the project field from the database. This ensures summaries
and observations always use the latest project value.
**Impact:**
- Future summaries will have correct project name
- Backfilled 13 historical summaries with project='claude-mem'
- Context injection will now show recent summaries
- Both observations AND summaries fixed
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implements a visual badge that displays the count of active work items (queued + currently processing) in the worker service. The badge appears next to the claude-mem logo and updates in real-time via SSE.
Features:
- Shows count of pending messages + active SDK generators
- Only displays when queueDepth > 0
- Subtle pulse animation for visual feedback
- Theme-aware styling
Backend changes:
- Added getTotalActiveWork() method to SessionManager
- Updated worker-service to broadcast queueDepth via SSE
- Enhanced processing status API endpoint
Frontend changes:
- Updated Header component to display queue bubble
- Enhanced useSSE hook to track queueDepth state
- Added CSS styling with pulse animation
Closes#122Closes#96Closes#97🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* feat: Add discovery_tokens for ROI tracking in observations and session summaries
- Introduced `discovery_tokens` column in `observations` and `session_summaries` tables to track token costs associated with discovering and creating each observation and summary.
- Updated relevant services and hooks to calculate and display ROI metrics based on discovery tokens.
- Enhanced context economics reporting to include savings from reusing previous observations.
- Implemented migration to ensure the new column is added to existing tables.
- Adjusted data models and sync processes to accommodate the new `discovery_tokens` field.
* refactor: streamline context hook by removing unused functions and updating terminology
- Removed the estimateTokens and getObservations helper functions as they were not utilized.
- Updated the legend and output messages to replace "discovery" with "work" for clarity.
- Changed the emoji representation for different observation types to better reflect their purpose.
- Enhanced output formatting for improved readability and understanding of token usage.
* Refactor user-message-hook and context-hook for improved clarity and functionality
- Updated user-message-hook.js to enhance error messaging and improve variable naming for clarity.
- Modified context-hook.ts to include a new column key section, improved context index instructions, and added emoji icons for observation types.
- Adjusted footer messages in context-hook.ts to emphasize token savings and access to past research.
- Changed user-message-hook.ts to update the feedback and support message for clarity.
* fix: Critical ROI tracking fixes from PR review
Addresses critical findings from PR #111 review:
1. **Fixed incorrect discovery token calculation** (src/services/worker/SDKAgent.ts)
- Changed from passing cumulative total to per-response delta
- Now correctly tracks token cost for each observation/summary
- Captures token state before/after response processing
- Prevents all observations getting inflated cumulative values
2. **Fixed schema version mismatch** (src/services/sqlite/SessionStore.ts)
- Changed ensureDiscoveryTokensColumn() from version 11 to version 7
- Now matches migration007 definition in migrations.ts
- Ensures consistent version tracking across migration system
These fixes ensure ROI metrics accurately reflect token costs.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* refactor: Reduce continuation prompt token usage by 95 lines
Removed redundant instructions from continuation prompt that were originally
added to mitigate a session continuity issue. That issue has since been
resolved, making these detailed instructions unnecessary on every continuation.
Changes:
- Reduced continuation prompt from ~106 lines to ~11 lines (~95 line reduction)
- Changed "User's Goal:" to "Next Prompt in Session:" (more accurate framing)
- Removed redundant WHAT TO RECORD, WHEN TO SKIP, and OUTPUT FORMAT sections
- Kept concise reminder: "Continue generating observations and progress summaries..."
- Initial prompt still contains all detailed instructions
Impact:
- Significant token savings on every continuation prompt
- Faster context injection with no loss of functionality
- Instructions remain comprehensive in initial prompt
Files modified:
- src/sdk/prompts.ts (buildContinuationPrompt function)
- plugin/scripts/worker-service.cjs (compiled output)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor: Enhance observation and summary prompts for clarity and token efficiency
* Enhance prompt clarity and instructions in prompts.ts
- Added a reminder to think about instructions before starting work.
- Simplified the continuation prompt instruction by removing "for this ongoing session."
* feat: Enhance settings.json with permissions and deny access to sensitive files
refactor: Remove PLAN-full-observation-display.md and PR_SUMMARY.md as they are no longer needed
chore: Delete SECURITY_SUMMARY.md since it is redundant after recent changes
fix: Update worker-service.cjs to streamline observation generation instructions
cleanup: Remove src-analysis.md and src-tree.md for a cleaner codebase
refactor: Modify prompts.ts to clarify instructions for memory processing
* refactor: Remove legacy worker service implementation
* feat: Enhance summary hook to extract last assistant message and improve logging
- Added function to extract the last assistant message from the transcript.
- Updated summary hook to include last assistant message in the summary request.
- Modified SDKSession interface to store last assistant message.
- Adjusted buildSummaryPrompt to utilize last assistant message for generating summaries.
- Updated worker service and session manager to handle last assistant message in summarize requests.
- Introduced silentDebug utility for improved logging and diagnostics throughout the summary process.
* docs: Add comprehensive implementation plan for ROI metrics feature
Added detailed implementation plan covering:
- Token usage capture from Agent SDK
- Database schema changes (migration #8)
- Discovery cost tracking per observation
- Context hook display with ROI metrics
- Testing and rollout strategy
Timeline: ~20 hours over 4 days
Goal: Empirical data for YC application amendment
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* feat: Add transcript processing scripts for analysis and formatting
- Implemented `dump-transcript-readable.ts` to generate a readable markdown dump of transcripts, excluding certain entry types.
- Created `extract-rich-context-examples.ts` to extract and showcase rich context examples from transcripts, highlighting user requests and assistant reasoning.
- Developed `format-transcript-context.ts` to format transcript context into a structured markdown format for improved observation generation.
- Added `test-transcript-parser.ts` for validating data extraction from transcript JSONL files, including statistics and error reporting.
- Introduced `transcript-to-markdown.ts` for a complete representation of transcript data in markdown format, showing all context data.
- Enhanced type definitions in `transcript.ts` to support new features and ensure type safety.
- Built `transcript-parser.ts` to handle parsing of transcript JSONL files, including error handling and data extraction methods.
* Refactor hooks and SDKAgent for improved observation handling
- Updated `new-hook.ts` to clean user prompts by stripping leading slashes for better semantic clarity.
- Enhanced `save-hook.ts` to include additional tools in the SKIP_TOOLS set, preventing unnecessary observations from certain command invocations.
- Modified `prompts.ts` to change the structure of observation prompts, emphasizing the observational role and providing a detailed XML output format for observations.
- Adjusted `SDKAgent.ts` to enforce stricter tool usage restrictions, ensuring the memory agent operates solely as an observer without any tool access.
* feat: Enhance session initialization to accept user prompts and prompt numbers
- Updated `handleSessionInit` in `worker-service.ts` to extract `userPrompt` and `promptNumber` from the request body and pass them to `initializeSession`.
- Modified `initializeSession` in `SessionManager.ts` to handle optional `currentUserPrompt` and `promptNumber` parameters.
- Added logic to update the existing session's `userPrompt` and `lastPromptNumber` if a `currentUserPrompt` is provided.
- Implemented debug logging for session initialization and updates to track user prompts and prompt numbers.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat: Enhance summary hook to include last user message from transcript
- Added functionality to extract the last user message from a JSONL transcript file in the summary hook.
- Updated the summary hook to send the last user message along with the summary request.
- Modified the SDKSession interface to include an optional last_user_message field.
- Updated the summary prompt to incorporate the last user message in the output format.
- Refactored worker service to handle the last user message in the summarize queue.
- Enhanced session manager to track and broadcast processing status based on active sessions and queue depth.
- Improved error handling and logging for better traceability during transcript reading and processing.
* feat(worker): enhance processing status broadcasting and session management
- Added immediate broadcasting of processing status when a prompt is received.
- Implemented logging for generator completion in multiple locations.
- Updated `broadcastProcessingStatus` to include queue depth and active session count in logs.
- Modified session iterator to stop yielding messages after a summary is yielded, with appropriate logging.
* feat: add mem-search skill with progressive disclosure architecture
Add comprehensive mem-search skill for accessing claude-mem's persistent
cross-session memory database. Implements progressive disclosure workflow
and token-efficient search patterns.
Features:
- 12 search operations (observations, sessions, prompts, by-type, by-concept, by-file, timelines, etc.)
- Progressive disclosure principles to minimize token usage
- Anti-patterns documentation to guide LLM behavior
- HTTP API integration for all search functionality
- Common workflows with composition examples
Structure:
- SKILL.md: Entry point with temporal trigger patterns
- principles/: Progressive disclosure + anti-patterns
- operations/: 12 search operation files
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* docs: add CHANGELOG entry for mem-search skill
Document mem-search skill addition in Unreleased section with:
- 100% effectiveness compliance metrics
- Comparison to previous search skill implementation
- Progressive disclosure architecture details
- Reference to audit report documentation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* docs: add mem-search skill audit report
Add comprehensive audit report validating mem-search skill against
Anthropic's official skill-creator documentation.
Report includes:
- Effectiveness metrics comparison (search vs mem-search)
- Critical issues analysis for production readiness
- Compliance validation across 6 key dimensions
- Reference implementation guidance
Result: mem-search achieves 100% compliance vs search's 67%
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* feat: Add comprehensive search architecture analysis document
- Document current state of dual search architectures (HTTP API and MCP)
- Analyze HTTP endpoints and MCP search server architectures
- Identify DRY violations across search implementations
- Evaluate the use of curl as the optimal approach for search
- Provide architectural recommendations for immediate and long-term improvements
- Outline action plan for cleanup, feature parity, DRY refactoring
* refactor: Remove deprecated search skill documentation and operations
* refactor: Reorganize documentation into public and context directories
Changes:
- Created docs/public/ for Mintlify documentation (.mdx files)
- Created docs/context/ for internal planning and implementation docs
- Moved all .mdx files and assets to docs/public/
- Moved all internal .md files to docs/context/
- Added CLAUDE.md to both directories explaining their purpose
- Updated docs.json paths to work with new structure
Benefits:
- Clear separation between user-facing and internal documentation
- Easier to maintain Mintlify docs in dedicated directory
- Internal context files organized separately
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Enhance session management and continuity in hooks
- Updated new-hook.ts to clarify session_id threading and idempotent session creation.
- Modified prompts.ts to require claudeSessionId for continuation prompts, ensuring session context is maintained.
- Improved SessionStore.ts documentation on createSDKSession to emphasize idempotent behavior and session connection.
- Refined SDKAgent.ts to detail continuation prompt logic and its reliance on session.claudeSessionId for unified session handling.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Alex Newman <thedotmack@gmail.com>
* Initial plan
* Initial analysis: Found root cause of double entries bug
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
* Fix double entries by assigning generatorPromise in handleSessionInit
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
* feat(logging): Enhance HTTP request logging and session management
- Added middleware for logging HTTP requests and responses, excluding static assets and health checks.
- Introduced a method to summarize request bodies for specific endpoints.
- Improved logging for user prompt synchronization with Chroma, including duration tracking.
- Enhanced session initialization logging to include additional session details.
- Updated observation and summary logging to provide more context and error handling during Chroma synchronization.
- Refactored tool name formatting for logging in the SessionManager.
- Expanded logger component types to include 'HTTP', 'SESSION', and 'CHROMA'.
* Refactor SDK prompts and logging for improved clarity and functionality
- Updated buildInitPrompt to clarify the observer's role and what to record.
- Enhanced buildSummaryPrompt with clearer instructions for summarizing ongoing sessions.
- Improved buildContinuationPrompt to emphasize the focus on deliverables and capabilities.
- Refactored WorkerService to utilize a centralized tool formatting function for logging.
- Added truncation for logged responses and observations to improve readability.
- Updated SessionManager to log the queuing of summarize actions with session details.
- Enhanced App and Sidebar components to support refreshing stats on sidebar open.
- Refactored useStats hook to allow manual refreshing of stats while maintaining automatic loading on mount.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
Alex Newman