Commit Graph

87 Commits

Author SHA1 Message Date
Alex Newman 3ea0b60b9f feat: Mode system with inheritance and multilingual support (#412)
* feat: add domain management system with support for multiple domain profiles

- Introduced DomainManager class for loading and managing domain profiles.
- Added support for a default domain ('code') and fallback mechanisms.
- Implemented domain configuration validation and error handling.
- Created types for domain configuration, observation types, and concepts.
- Added new directory for domain profiles and ensured its existence.
- Updated SettingsDefaultsManager to include CLAUDE_MEM_DOMAIN setting.

* Refactor domain management to mode management

- Removed DomainManager class and replaced it with ModeManager for better clarity and functionality.
- Updated types from DomainConfig to ModeConfig and DomainPrompts to ModePrompts.
- Changed references from domains to modes in the settings and paths.
- Ensured backward compatibility by maintaining the fallback mechanism to the 'code' mode.

* feat: add migration 008 to support mode-agnostic observations and refactor service layer references in documentation

* feat: add new modes for code development and email investigation with detailed observation types and concepts

* Refactor observation parsing and prompt generation to incorporate mode-specific configurations

- Updated `parseObservations` function to use 'observation' as a universal fallback type instead of 'change', utilizing active mode's valid observation types.
- Modified `buildInitPrompt` and `buildContinuationPrompt` functions to accept a `ModeConfig` parameter, allowing for dynamic prompt content based on the active mode.
- Enhanced `ModePrompts` interface to include additional guidance for observers, such as recording focus and skip guidance.
- Adjusted the SDKAgent to load the active mode and pass it to prompt generation functions, ensuring prompts are tailored to the current mode's context.

* fix: correct mode prompt injection to preserve exact wording and type list visibility

- Add script to extract prompts from main branch prompts.ts into code.yaml
- Fix prompts.ts to show type list in XML template (e.g., "[ bugfix | feature | ... ]")
- Keep 'change' as fallback type in parser.ts (maintain backwards compatibility)
- Regenerate code.yaml with exact wording from original hardcoded prompts
- Build succeeds with no TypeScript errors

* fix: update ModeManager to load JSON mode files and improve validation

- Changed ModeManager to load mode configurations from JSON files instead of YAML.
- Removed the requirement for an "observation" type and updated validation to require at least one observation type.
- Updated fallback behavior in the parser to use the first type from the active mode's type list.
- Added comprehensive tests for mode loading, prompt injection, and parser integration, ensuring correct behavior across different modes.
- Introduced new mode JSON files for "Code Development" and "Email Investigation" with detailed observation types and prompts.

* Add mode configuration loading and update licensing information for Ragtime

- Implemented loading of mode configuration in WorkerService before database initialization.
- Added PolyForm Noncommercial License 1.0.0 to Ragtime directory.
- Created README.md for Ragtime with licensing details and usage guidelines.

* fix: add datasets directory to .gitignore to prevent accidental commits

* refactor: remove unused plugin package.json file

* chore: add package.json for claude-mem plugin with version 7.4.5

* refactor: remove outdated tests and improve error handling

- Deleted tests for ChromaSync error handling, smart install, strip memory tags, and user prompt tag stripping due to redundancy or outdated logic.
- Removed vitest configuration as it is no longer needed.
- Added a comprehensive implementation plan for fixing the modes system, addressing critical issues and improving functionality.
- Created a detailed test analysis report highlighting the quality and effectiveness of the current test suite, identifying areas for improvement.
- Introduced a new plugin package.json for runtime dependencies related to claude-mem hooks.

* refactor: remove parser regression tests to streamline codebase

* docs: update CLAUDE.md to clarify test management and changelog generation

* refactor: remove migration008 for mode-agnostic observations

* Refactor observation type handling to use ModeManager for icons and emojis

- Removed direct mappings of observation types to icons and work emojis in context-generator, FormattingService, SearchManager, and TimelineService.
- Integrated ModeManager to dynamically retrieve icons and emojis based on the active mode.
- Improved maintainability by centralizing the logic for observation type representation.

* Refactor observation metadata constants and update context generator

- Removed the explicit declaration of OBSERVATION_TYPES and OBSERVATION_CONCEPTS from observation-metadata.ts.
- Introduced fallback default strings for DEFAULT_OBSERVATION_TYPES_STRING and DEFAULT_OBSERVATION_CONCEPTS_STRING.
- Updated context-generator.ts to utilize observation types and concepts from ModeManager instead of constants.

* refactor: remove intermediate error handling from hooks (Phase 1)

Apply "fail fast" error handling strategy - errors propagate and crash loud
instead of being caught, wrapped, and re-thrown at intermediate layers.

Changes:
- Remove try/catch around fetch calls in all hooks - let errors throw
- Add try/catch ONLY around JSON.parse at entry points
- Delete error-handler.ts and hook-error-handler.ts (no longer needed)
- Update worker-utils.ts: functions now throw instead of returning null
- Update transcript-parser.ts: throw on missing path, empty file, malformed JSON
- Remove all handleWorkerError, handleFetchError imports

Philosophy: If something breaks, we KNOW it broke. No silent failures.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: remove intermediate error handling from worker service (Phase 2)

Apply "fail fast" error handling strategy to worker service layer.

Changes:
- worker-service.ts: Remove try/catch from version endpoint, cleanup,
  MCP close, process enumeration, force kill, and isAlive check
- SessionRoutes.ts: Remove try/catch from JSON.stringify calls, remove
  .catch() from Chroma sync and SDK agent calls
- SettingsRoutes.ts: Remove try/catch from toggleMcp()
- DatabaseManager.ts: Remove .catch() from backfill and close operations
- SDKAgent.ts: Keep outer try/catch (top-level), remove .catch() from
  Chroma sync operations
- SSEBroadcaster.ts: Remove try/catch from broadcast and sendToClient

Philosophy: Errors propagate and crash loud. BaseRouteHandler.wrapHandler
provides top-level catching for HTTP routes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: remove error swallowing from SQLite services (Phase 3)

Apply "fail fast" error handling strategy to database layer.

SessionStore.ts migrations:
- ensureWorkerPortColumn(): Remove outer try/catch, let it throw
- ensurePromptTrackingColumns(): Remove outer try/catch, let it throw
- removeSessionSummariesUniqueConstraint(): Keep inner transaction
  rollback, remove outer catch
- addObservationHierarchicalFields(): Remove outer try/catch
- makeObservationsTextNullable(): Keep inner transaction rollback,
  remove outer catch
- createUserPromptsTable(): Keep inner transaction rollback, remove
  outer catch
- getFilesForSession(): Remove try/catch around JSON.parse

SessionSearch.ts:
- ensureFTSTables(): Remove try/catch, let it throw

Philosophy: Migration errors that are swallowed mean we think the
database is fine when it's not. Keep only inner transaction rollback
try/catch blocks.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: remove error hiding from utilities (Phase 4)

Apply "fail fast" error handling strategy to utility layer.

logger.ts:
- formatTool(): Remove try/catch, let JSON.parse throw on malformed input

context-generator.ts:
- loadContextConfig(): Remove try/catch, let parseInt throw on invalid settings
- Transcript extraction: Remove try/catch, let file read errors propagate

ChromaSync.ts:
- close(): Remove nested try/catch blocks, let close errors propagate

Philosophy: No silent fallbacks or hidden defaults. If something breaks,
we know it broke immediately.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: serve static UI assets and update package root path

- Added middleware to serve static UI assets (JS, CSS, fonts, etc.) in ViewerRoutes.
- Updated getPackageRoot function to correctly return the package root directory as one level up from the current directory.

* feat: Enhance mode loading with inheritance support

- Introduced parseInheritance method to handle parent--override mode IDs.
- Added deepMerge method for recursively merging mode configurations.
- Updated loadMode method to support inheritance, loading parent modes and applying overrides.
- Improved error handling for missing mode files and logging for better traceability.

* fix(modes): correct inheritance file resolution and path handling

* Refactor code structure for improved readability and maintainability

* feat: Add mode configuration documentation and examples

* fix: Improve concurrency handling in translateReadme function

* Refactor SDK prompts to enhance clarity and structure

- Updated the `buildInitPrompt` and `buildContinuationPrompt` functions in `prompts.ts` to improve the organization of prompt components, including the addition of language instructions and footer messages.
- Removed redundant instructions and emphasized the importance of recording observations.
- Modified the `ModePrompts` interface in `types.ts` to include new properties for system identity, language instructions, and output format header, ensuring better flexibility and clarity in prompt generation.

* Enhance prompts with language instructions and XML formatting

- Updated `buildInitPrompt`, `buildSummaryPrompt`, and `buildContinuationPrompt` functions to include detailed language instructions in XML comments.
- Ensured that language instructions guide users to keep XML tags in English while writing content in the specified language.
- Modified the `buildSummaryPrompt` function to accept `mode` as a parameter for consistency.
- Adjusted the call to `buildSummaryPrompt` in `SDKAgent` to pass the `mode` argument.

* Refactor XML prompt generation in SDK

- Updated the buildInitPrompt, buildSummaryPrompt, and buildContinuationPrompt functions to use new placeholders for XML elements, improving maintainability and readability.
- Removed redundant language instructions in comments for clarity.
- Added new properties to ModePrompts interface for better structure and organization of XML placeholders and section headers.

* feat: Update observation prompts and structure across multiple languages

* chore: Remove planning docs and update Ragtime README

Remove ephemeral development artifacts:
- .claude/plans/modes-system-fixes.md
- .claude/test-analysis-report.md
- PROMPT_INJECTION_ANALYSIS.md

Update ragtime/README.md to explain:
- Feature is not yet implemented
- Dependency on modes system (now complete in PR #412)
- Ready to be scripted out in future release

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: Move summary prompts to mode files for multilingual support

Summary prompts were hardcoded in English in prompts.ts, breaking
multilingual support. Now properly mode-based:

- Added summary_instruction, summary_context_label,
  summary_format_instruction, summary_footer to code.json
- Updated buildSummaryPrompt() to use mode fields instead of hardcoded text
- Added summary_footer with language instructions to all 10 language modes
- Language modes keep English prompts + language requirement footer

This fixes the gaslighting where we claimed full multilingual support
but summaries were still generated in English.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* chore: Clean up README by removing local preview instructions and streamlining beta features section

* Add translated README files for Ukrainian, Vietnamese, and Chinese languages

* Add new language modes for code development in multiple languages

- Introduced JSON configurations for Code Development in Greek, Finnish, Hebrew, Hindi, Hungarian, Indonesian, Italian, Dutch, Norwegian, Polish, Brazilian Portuguese, Romanian, Swedish, Turkish, and Ukrainian.
- Each configuration includes prompts for observations, summaries, and instructions tailored to the respective language.
- Ensured that all prompts emphasize the importance of generating observations without referencing the agent's actions.

* Add multilingual support links to README files in various languages

- Updated README.id.md, README.it.md, README.ja.md, README.ko.md, README.nl.md, README.no.md, README.pl.md, README.pt-br.md, README.ro.md, README.ru.md, README.sv.md, README.th.md, README.tr.md, README.uk.md, README.vi.md, and README.zh.md to include links to other language versions.
- Each README now features a centered paragraph with flags and links for easy navigation between different language documents.

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-22 20:14:18 -05:00
Alex Newman a537433eae Code quality: comprehensive nonsense audit cleanup (20 issues) (#400)
* fix: prevent initialization promise from resolving on failure

Background initialization was resolving the promise even when it failed,
causing the readiness check to incorrectly indicate the worker was ready.
Now the promise stays pending on failure, ensuring /api/readiness
continues returning 503 until initialization succeeds.

Fixes critical issue #1 from nonsense audit.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: improve error handling in context inject and settings update routes

* Enhance error handling for ChromaDB failures in SearchManager

- Introduced a flag to track ChromaDB failure states.
- Updated logging messages to provide clearer feedback on ChromaDB initialization and failure.
- Modified the response structure to inform users when semantic search is unavailable due to ChromaDB issues, including installation instructions for UVX/Python.

* refactor: remove deprecated silent-debug utility functions

* Enhance error handling and validation in hooks

- Added validation for required fields in `summary-hook.ts` and `save-hook.ts` to ensure necessary inputs are provided before processing.
- Improved error messages for missing `cwd` in `save-hook.ts` and `transcript_path` in `summary-hook.ts`.
- Cleaned up code by removing unnecessary error handling logic and directly throwing errors when required fields are missing.
- Updated binary file `mem-search.zip` to reflect changes in the plugin.

* fix: improve error handling in summary hook to ensure errors are not masked

* fix: add error handling for unknown message content format in transcript parser

* fix: log error when failing to notify worker of session end

* Refactor date formatting functions: move to shared module

- Removed redundant date formatting functions from SearchManager.ts.
- Consolidated date formatting logic into shared timeline-formatting.ts.
- Updated functions to accept both ISO date strings and epoch milliseconds.

* Refactor tag stripping functions to extract shared logic

- Introduced a new internal function `stripTagsInternal` to handle the common logic for stripping memory tags from both JSON and prompt content.
- Updated `stripMemoryTagsFromJson` to utilize the new internal function, simplifying its implementation.
- Modified `stripMemoryTagsFromPrompt` to also call `stripTagsInternal`, reducing code duplication and improving maintainability.
- Removed redundant type checks and logging from both functions, as they now rely on the internal function for processing.

* Refactor settings validation in SettingsRoutes

- Consolidated multiple individual setting validations into a single validateSettings method.
- Updated handleUpdateSettings to use the new validation method for improved maintainability.
- Each setting now has its validation logic encapsulated within validateSettings, ensuring a single source of truth for validation rules.

* fix: add error logging to ProcessManager.getPidInfo()

Previously getPidInfo() returned null silently for three cases:
1. File not found (expected - no action needed)
2. JSON parse error (corrupted file - now logs warning)
3. Type validation failure (malformed data - now logs warning)

This fix adds warning logs for cases 2 and 3 to provide visibility
into PID file corruption issues. Logs include context like parsed
data structure or error message with file path.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: remove overly defensive try-catch in SessionRoutes

Remove unnecessary try-catch block that was masking potential errors when
checking file paths for session-memory meta-observations. Property access
on parsed JSON objects never throws - existing truthiness checks already
safely handle undefined/null values.

Issue #12 from nonsense audit: SessionRoutes catch-all exception masking

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: remove redundant try-catch from getWorkerPort()

Simplified getWorkerPort() by removing unnecessary try-catch wrapper.
SettingsDefaultsManager.loadFromFile() already handles missing files
by returning defaults, and .get() never throws - making the catch
block completely redundant.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* refactor: eliminate ceremonial wrapper in hook-response.ts

Replace buildHookResponse() function with direct constant export.
Most hook responses were calling a function just to return the same
constant object. Only SessionStart with context needs special handling.

Changes:
- Export STANDARD_HOOK_RESPONSE constant directly
- Simplify createHookResponse() to only handle SessionStart special case
- Update all hooks to use STANDARD_HOOK_RESPONSE instead of function call
- Eliminate buildHookResponse() function with redundant branching

Files modified:
- src/hooks/hook-response.ts: Export constant, simplify function
- src/hooks/new-hook.ts: Use STANDARD_HOOK_RESPONSE
- src/hooks/save-hook.ts: Use STANDARD_HOOK_RESPONSE
- src/hooks/summary-hook.ts: Use STANDARD_HOOK_RESPONSE

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: make getWorkerHost() consistent with getWorkerPort()

- Use SettingsDefaultsManager.get('CLAUDE_MEM_DATA_DIR') for path resolution
  instead of hardcoded ~/.claude-mem (supports custom data directories)
- Add caching to getWorkerHost() (same pattern as getWorkerPort())
- Update clearPortCache() to also clear host cache
- Both functions now have identical patterns: caching, consistent path
  resolution, and same error handling via SettingsDefaultsManager

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* refactor: inline single-use timeout constants in ProcessManager

Remove 6 timeout constants used only once each, inlining their values
directly at the point of use. Following YAGNI principle - constants
should only exist when used multiple times.

Removed constants:
- PROCESS_STOP_TIMEOUT_MS (5000ms)
- HEALTH_CHECK_TIMEOUT_MS (10000ms)
- HEALTH_CHECK_INTERVAL_MS (200ms)
- HEALTH_CHECK_FETCH_TIMEOUT_MS (1000ms)
- PROCESS_EXIT_CHECK_INTERVAL_MS (100ms)
- HTTP_SHUTDOWN_TIMEOUT_MS (2000ms)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: replace overly broad path filter in HTTP logging middleware

Replace `req.path.includes('.')` with explicit static file extension
checking to prevent incorrectly skipping API endpoint logging.

- Add `staticExtensions` array with legitimate asset types
- Use `.endsWith()` matching instead of `.includes()`
- API endpoints containing periods (if any) now logged correctly
- Static assets (.js, .css, .svg, etc.) still skip logging as intended

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* refactor: expand logger.formatTool() to handle all tool types

Replace hard-coded tool formatting for 4 tools with comprehensive coverage:

File operations (Read, Edit, Write, NotebookEdit):
- Consolidated file_path handling for all file operations
- Added notebook_path support for NotebookEdit
- Shows filename only (not full path)

Search tools (Glob, Grep):
- Glob: shows pattern
- Grep: shows pattern (truncated if > 30 chars)

Network tools (WebFetch, WebSearch):
- Shows URL or query (truncated if > 40 chars)

Meta tools (Task, Skill, LSP):
- Task: shows subagent_type or description
- Skill: shows skill name
- LSP: shows operation type

This eliminates the "hard-coded 4 tools" limitation and provides
meaningful log output for all tool types.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: remove all truncation from logger.formatTool()

Truncation hides critical debugging information. Show everything:

- Bash: full command (was truncated at 50 chars)
- File operations: full path (was showing filename only)
- Grep: full pattern (was truncated at 30 chars)
- WebFetch/WebSearch: full URL/query (was truncated at 40 chars)
- Task: full description (was truncated at 30 chars)

Logs exist to provide complete information. Never hide details.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* refactor: replace array indexing with regex capture for drive letter

Use explicit regex capture group to extract Windows drive letter instead
of assuming cwd[0] is always the first character. Safer and more explicit.

- Changed cwd.match(/^[A-Z]:\\/i) to cwd.match(/^([A-Z]):\\/i)
- Extract drive letter from driveMatch[1] instead of cwd[0]
- Restructured control flow to avoid nested conditionals

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: return computed values from DataRoutes processing endpoint

The handleSetProcessing endpoint was computing queueDepth and
activeSessions but not including them in the response. This commit
includes all computed values in the API response.

- Return queueDepth and activeSessions in /api/processing response
- Eliminates dead code pattern where values are computed but unused
- API callers can now access these metrics

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: move error handling into SettingsDefaultsManager.loadFromFile()

Wrap the entire loadFromFile() method in try-catch so it handles ALL
error cases (missing file, corrupted JSON, permission errors, I/O failures)
instead of forcing every caller to add redundant try-catch blocks.

This follows DRY principle: one function owns error handling, all callers
stay simple and clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Refactor hook response handling and optimize token estimation

- Removed the HookType and HookResponse types and the createHookResponse function from hook-response.ts to simplify the response handling for hooks.
- Introduced a standardized hook response for all hooks in hook-response.ts.
- Moved the estimateTokens function from SearchManager.ts to timeline-formatting.ts for better reusability and clarity.
- Cleaned up redundant estimateTokens function definitions in SearchManager.ts.

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-20 19:41:33 -05:00
Alex Newman c4af31f48d Optimize MCP tool token usage with schema reference pattern
Reduces MCP tool token consumption by ~90% through progressive disclosure. Tools now show minimal schemas with get_schema() for details on demand.
2025-12-17 22:47:30 -05:00
Alex Newman 0cb3256b2d fix(security): add localhost-only protection for admin endpoints
Adds middleware to restrict /api/admin/restart and /api/admin/shutdown
to localhost-only access. This prevents DoS attacks when the worker
service is bound to 0.0.0.0 for remote UI access.

Implementation:
- Created requireLocalhost middleware in middleware.ts
- Applied to both admin endpoints
- Checks client IP against localhost addresses (127.0.0.1, ::1, etc.)
- Returns 403 Forbidden for non-localhost requests

Addresses security concern raised in PR #368 with cleaner DRY approach.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-17 19:06:33 -05:00
Alex Newman bff10d49c9 fix(windows): Windows platform stabilization improvements (#378)
* chore: bump version to 7.3.6 in package.json

* Enhance worker readiness checks and MCP connection handling

- Updated health check endpoint to /api/readiness for better initialization tracking.
- Increased timeout for health checks and worker startup retries, especially for Windows.
- Added initialization flags to track MCP readiness and overall worker initialization status.
- Implemented a timeout guard for MCP connection to prevent hanging.
- Adjusted logging to reflect readiness state and errors more accurately.

* fix(windows): use Bun PATH detection in worker wrapper

Phase 2/8: Fix Bun PATH Detection in Worker Wrapper

- Import getBunPath() in worker-wrapper.ts for Bun detection
- Add Bun path resolution before spawning inner worker process
- Update spawn call to use detected Bun path instead of process.execPath
- Add logging to bun-path.ts when PATH detection succeeds
- Add logging when fallback paths are used
- Add Windows-specific validation for .exe extension
- Log warning with searched paths when Bun not found
- Fail fast with clear error message if Bun cannot be detected

This ensures worker-wrapper uses the correct Bun executable on Windows
even when Bun is not in PATH, fixing issue #371 where users reported
"Bun not in PATH" errors despite Bun being installed.

Addresses: #371

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(windows): standardize child process spawning with windowsHide

Phase 3/8: Standardize Child Process Spawning (Windows)

Changes:
- Added windowsHide flag to ChromaSync MCP subprocess spawn
- Added Windows-specific process tracking (childPid) in ChromaSync
- Force-kill subprocess on Windows before closing transport to prevent zombie processes
- Updated cleanupOrphanedProcesses() to support Windows using PowerShell Get-CimInstance
- Use taskkill /T /F for proper process tree cleanup on Windows
- Audited BranchManager - confirmed windowsHide already present on all spawn calls

This prevents PowerShell windows from appearing during ChromaSync operations
and ensures proper cleanup of subprocess trees on Windows.

Addresses: #363, #361, #367, #371, #373, #374

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(windows): enhance socket cleanup with recursive process tree management

Phase 4/8: Enhanced Socket Cleanup & Process Tree Management

Changes:
- Added recursive process tree enumeration in worker-wrapper.ts for Windows
- Enhanced killInner() to enumerate all descendants before killing
- Added fallback individual process kill if taskkill /T fails
- Added 10s timeout to ChromaSync.close() in DatabaseManager to prevent hangs
- Force nullify ChromaSync even on close failure to prevent resource leaks
- Improved logging to show full process tree during cleanup

This ensures complete cleanup of all child processes (ChromaSync MCP subprocess,
Python processes, etc.) preventing socket leaks and CLOSE_WAIT states.

Addresses: #363, #361

* fix(windows): consolidate project name extraction with drive root handling

Phase 5/8: Project Name Extraction Consolidation

- Created shared getProjectName() utility in src/utils/project-name.ts
- Handles edge case: drive roots (C:\, J:\) now return "drive-X" format
- Handles edge case: null/undefined/empty cwd now returns "unknown-project"
- Fixed missing null check bug in new-hook.ts
- Replaced duplicated path.basename(cwd) logic in:
  - src/hooks/context-hook.ts
  - src/hooks/new-hook.ts
  - src/services/context-generator.ts

Addresses: #374

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(windows): increase timeouts and improve error messages

Phase 6/8: Increase Timeouts & Improve Error Messages

- Enhanced logger.ts with platform prefix (WIN32/DARWIN) and PID in all logs
- Added comprehensive Windows troubleshooting to ProcessManager error messages
- Enhanced Bun detection error message with Windows-specific troubleshooting
- All error messages now include GitHub issue numbers and docs links
- Windows timeout already increased to 2.0x multiplier in previous phases

Changes:
- src/utils/logger.ts: Added platform prefix and PID to all log output
- src/services/process/ProcessManager.ts: Enhanced error messages with troubleshooting steps
- src/utils/bun-path.ts: Added Windows-specific Bun detection error guidance

Addresses: #363, #361, #367, #371, #373, #374

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(windows): add comprehensive Windows CI testing

Phase 7/8: Add Windows CI Testing

- Create automated Windows testing workflow
- Test worker startup/shutdown cycles
- Verify Bun PATH detection on Windows
- Test rapid restart scenarios
- Validate port cleanup after shutdown
- Check for zombie processes
- Run on all pushes and PRs to main/fix/feature branches

Addresses: #363, #361, #367, #371, #373, #374

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* ci(windows): remove build steps from Windows CI workflow

Build files are already included in the plugin folder, so npm install
and npm run build are unnecessary steps in the CI workflow.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* revert: remove Windows CI workflow

The CI workflow cannot be properly implemented in the current architecture
due to limitations in testing the worker service in CI environments.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* security: add PID validation and improve ChromaSync timeout handling

Address critical security and reliability issues identified in PR review:

**Security Fixes:**
- Add PID validation before all PowerShell/taskkill command execution
- Validate PIDs are positive integers to prevent command injection
- Apply validation in worker-wrapper.ts, worker-service.ts, and ChromaSync.ts

**Reliability Improvements:**
- Add timeout handling to ChromaSync client.close() (10s timeout)
- Add timeout handling to ChromaSync transport.close() (5s timeout)
- Implement force-kill fallback when ChromaSync close operations timeout
- Prevents hanging on shutdown and ensures subprocess cleanup

**Implementation Details:**
- PID validation checks: Number.isInteger(pid) && pid > 0
- Applied before all execSync taskkill calls on Windows
- Applied in process enumeration (Get-CimInstance) PowerShell commands
- ChromaSync.close() uses Promise.race for timeout enforcement
- Graceful degradation with force-kill fallback on timeout

Addresses PR #378 review feedback

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Refactor ChromaSync client and transport closure logic

- Removed timeout handling for closing the Chroma client and transport.
- Simplified error logging for client and transport closure.
- Ensured subprocess cleanup logic is more straightforward.

* fix(worker): streamline Windows process management and cleanup

* revert: remove speculative LLM-generated complexity

Reverts defensive code that was added speculatively without user-reported issues:

- ChromaSync: Remove PID extraction and explicit taskkill (wrapper handles this)
- worker-wrapper: Restore simple taskkill /T /F (validated in v7.3.5)
- DatabaseManager: Remove Promise.race timeout wrapper
- hook-constants: Restore original timeout values
- logger: Remove platform/PID additions to every log line
- bun-path: Remove speculative logging

Keeps only changes that map to actual GitHub issues:
- #374: Drive root project name fix (getProjectName utility)
- #363: Readiness endpoint and Windows orphan cleanup
- #367: windowsHide on ChromaSync transport

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-17 18:44:04 -05:00
ToxMox a5bf653a47 fix(windows): solve zombie port problem with wrapper architecture (#372)
On Windows, Bun doesn't properly release socket handles when the worker
process exits, causing "zombie ports" that remain bound even after all
processes are dead. This required a system reboot to clear.

Solution: Introduce a wrapper process (worker-wrapper.cjs) that:
- Spawns the actual worker as a child with IPC channel
- On restart/shutdown, uses `taskkill /T /F` to kill the entire process tree
- Exits itself, allowing hooks to start fresh

The wrapper has no sockets, so Bun's socket cleanup bug doesn't affect it.
When the wrapper kills the inner worker tree and exits, the port is properly
released and can be immediately reused.

Key changes:
- New worker-wrapper.ts for Windows process lifecycle management
- ProcessManager starts wrapper on Windows, worker directly on Unix
- Worker sends IPC messages to wrapper for restart/shutdown
- Health endpoint now includes debug info (build ID, managed status, hasIpc)

Tested: Restart API now properly releases port and new worker binds to same port.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-17 14:45:41 -05:00
Alex Newman 61488042d8 Mem-search enhancements: table output, simplified API, Sonnet default, and removed fake URIs (#317)
* feat: Add batch fetching for observations and update documentation

- Implemented a new endpoint for fetching multiple observations by IDs in a single request.
- Updated the DataRoutes to include a POST /api/observations/batch endpoint.
- Enhanced SKILL.md documentation to reflect changes in the search process and batch fetching capabilities.
- Increased the default limit for search results from 5 to 40 for better usability.

* feat!: Fix timeline parameter passing with SearchManager alignment

BREAKING CHANGE: Timeline MCP tools now use standardized parameter names
- anchor_id → anchor
- before → depth_before
- after → depth_after
- obs_type → type (timeline tool only)

Fixes timeline endpoint failures caused by parameter name mismatch between
MCP layer and SearchManager. Adds new SessionStore methods for fetching
prompts and session summaries by ID.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* docs: reframe timeline parameter fix as bug fix, not breaking change

The timeline tools were completely broken due to parameter name mismatch.
There's nothing to migrate from since the old parameters never worked.

Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>

* Refactor mem-search documentation and optimize API tool definitions

- Updated SKILL.md to emphasize batch fetching for observations, clarifying usage and efficiency.
- Removed deprecated tools from mcp-server.ts and streamlined tool definitions for clarity.
- Enhanced formatting in FormattingService.ts for better output readability.
- Adjusted SearchManager.ts to improve result headers and removed unnecessary search tips from combined text.

* Refactor FormattingService and SearchManager for table-based output

- Updated FormattingService to format search results as tables, including methods for formatting observations, sessions, and user prompts.
- Removed JSON format handling from SearchManager and streamlined result formatting to consistently use table format.
- Enhanced readability and consistency in search tips and formatting logic.
- Introduced token estimation for observations and improved time formatting.

* refactor: update documentation and API references for version bump and search functionalities

* Refactor code structure for improved readability and maintainability

* chore: change default model from haiku to sonnet

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: unify timeline formatting across search and context services

Extract shared timeline formatting utilities into reusable module to align
MCP search output format with context-generator's date/file-grouped format.

Changes:
- Create src/shared/timeline-formatting.ts with reusable utilities
  (parseJsonArray, formatDateTime, formatTime, formatDate, toRelativePath,
  extractFirstFile, groupByDate)
- Refactor context-generator.ts to use shared utilities
- Update SearchManager.search() to use date/file grouping
- Add search-specific row formatters to FormattingService
- Fix timeline methods to extract actual file paths from metadata
  instead of hardcoding 'General'
- Remove Work column from search output (kept in context output)

Result: Consistent date/file-grouped markdown formatting across both
systems while maintaining their different column requirements.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* refactor: remove redundant legend from search output

Remove legend from search/timeline results since it's already shown
in SessionStart context. Saves ~30 tokens per search result.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Refactor session summary rendering to remove links

- Removed link generation for session summaries in context generation and search manager.
- Updated output formatting to exclude links while maintaining the session summary structure.
- Adjusted related components in TimelineService to ensure consistency across the application.

* fix: move skillPath declaration outside try block to fix scoping bug

The skillPath variable was declared inside the try block but referenced
in the catch block for error logging. Since const is block-scoped, this
would cause a ReferenceError when the error handler executes.

Moved skillPath declaration before the try block so it's accessible in
both try and catch scopes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: address PR #317 code review feedback

**Critical Fixes:**
- Replace happy_path_error__with_fallback debug calls with proper logger methods in mcp-server.ts
- All HTTP API calls now use logger.debug/error for consistent logging

**Code Quality Improvements:**
- Extract 90-day recency window magic numbers to named constants
- Added RECENCY_WINDOW_DAYS and RECENCY_WINDOW_MS constants in SearchManager

**Documentation:**
- Document model cost implications of Haiku → Sonnet upgrade in CHANGELOG
- Provide clear migration path for users who want to revert to Haiku

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* refactor: simplify CHANGELOG - remove cost documentation

Removed model cost comparison documentation per user feedback.
Kept only the technical code quality improvements.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
2025-12-14 21:58:11 -05:00
Copilot e7380adb2f Fix 404 error on /api/context/inject during worker startup (#310)
* Initial plan

* Fix worker service connection failed error by adding early context/inject route

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Add integration test for context inject early access

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Fix import path and improve test code style

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Add clarifying comment about intentional code duplication

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* build: compile fix for /api/context/inject 404 error

Compiled worker service and MCP server with the initialization race condition fix.
Validation results: All tests passing, route available immediately on restart.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
Co-authored-by: Alex Newman <thedotmack@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-14 14:33:13 -05:00
Alex Newman 52d2f72a82 Standardize and enhance error handling across hooks and worker service (#295)
* Enhance error logging in hooks

- Added detailed error logging in context-hook, new-hook, save-hook, and summary-hook to capture status, project, port, and relevant session information on failures.
- Improved error messages thrown in save-hook and summary-hook to include specific context about the failure.

* Refactor migration logging to use console.log instead of console.error

- Updated SessionSearch and SessionStore classes to replace console.error with console.log for migration-related messages.
- Added notes in the documentation to clarify the use of console.log for migration messages due to the unavailability of the structured logger during constructor execution.

* Refactor SDKAgent and silent-debug utility to simplify error handling

- Updated SDKAgent to use direct defaults instead of happy_path_error__with_fallback for non-critical fields such as last_user_message, last_assistant_message, title, filesRead, filesModified, concepts, and summary.request.
- Enhanced silent-debug documentation to clarify appropriate use cases for happy_path_error__with_fallback, emphasizing its role in handling unexpected null/undefined values while discouraging its use for nullable fields with valid defaults.

* fix: correct happy_path_error__with_fallback usage to prevent false errors

Fixes false "Missing cwd" and "Missing transcript_path" errors that were
flooding silent.log even when values were present.

Root cause: happy_path_error__with_fallback was being called unconditionally
instead of only when the value was actually missing.

Pattern changed from:
  value: happy_path_error__with_fallback('Missing', {}, value || '')

To correct usage:
  value: value || happy_path_error__with_fallback('Missing', {}, '')

Fixed in:
- src/hooks/save-hook.ts (PostToolUse hook)
- src/hooks/summary-hook.ts (Stop hook)
- src/services/worker/http/routes/SessionRoutes.ts (2 instances)

Impact: Eliminates false error noise, making actual errors visible.

Addresses issue #260 - users were seeing "Missing cwd" errors despite
Claude Code correctly passing all required fields.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Enhance error logging and handling across services

- Improved error messages in SessionStore to include project context when fetching boundary observations and timestamps.
- Updated ChromaSync error handling to provide more informative messages regarding client initialization failures, including the project context.
- Enhanced error logging in WorkerService to include the package path when reading version fails.
- Added detailed error logging in worker-utils to capture expected and running versions during health checks.
- Extended WorkerErrorMessageOptions to include actualError for more informative restart instructions.

* Refactor error handling in hooks to use standardized fetch error handler

- Introduced a new error handler `handleFetchError` in `shared/error-handler.ts` to standardize logging and user-facing error messages for fetch failures across hooks.
- Updated `context-hook.ts`, `new-hook.ts`, `save-hook.ts`, and `summary-hook.ts` to utilize the new error handler, improving consistency and maintainability.
- Removed redundant imports and error handling logic related to worker restart instructions from the hooks.

* feat: add comprehensive error handling tests for hooks and ChromaSync client

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-13 23:25:43 -05:00
Alex Newman f41579b4d0 feat: auto-cleanup orphaned chroma-mcp processes on worker startup
Enhancement to process leak fix from v7.1.9 - automatically detects and
kills orphaned chroma-mcp processes when the worker starts.

Changes:
- Added cleanupOrphanedProcesses() method to WorkerService
- Scans for existing chroma-mcp processes on startup
- Kills all found processes before creating new ones
- Logs cleanup activity (process count and PIDs)
- Non-fatal error handling (continues on cleanup failure)

Benefits:
- Automatically recovers from pre-7.1.9 process leaks
- Ensures clean slate on every worker restart
- No manual intervention needed to cleanup orphans
- Prevents accumulation even if v7.1.9 close() fails

Verified working in logs:
[INFO] [SYSTEM] Cleaning up orphaned chroma-mcp processes {count=2, pids=33753,33750}
[INFO] [SYSTEM] Orphaned processes cleaned up {count=2}

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-13 20:21:21 -05:00
Alex Newman 4c44a65877 fix: remove Windows process.type workaround causing libuv crashes
Removed the process.type = 'renderer' workaround that was causing libuv
assertion failures on Windows. This hack was attempting to hide console
windows but resulted in crashes when process.title was accessed.

Prioritizing stability over cosmetics - console windows may briefly appear
on Windows until the MCP SDK provides proper window hiding support.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-13 17:48:47 -05:00
Alex Newman 490ba182d5 feat: automatic worker restart on version updates
Critical improvement for seamless upgrades across all versions.

Changes:
1. Added /api/version endpoint to worker service
   - Returns current worker version from package.json

2. Added version checking in worker-utils.ts
   - getPluginVersion() - reads plugin's package.json version
   - getWorkerVersion() - fetches version from worker API
   - ensureWorkerVersionMatches() - compares and restarts if needed

3. Modified ensureWorkerRunning()
   - Now calls ensureWorkerVersionMatches() after health check
   - Automatically restarts worker when version mismatch detected
   - Logs version mismatch for debugging

Impact:
- Users no longer need to manually restart worker after upgrades
- Eliminates connection errors from running old worker code
- Critical for v7.1.1 (Bun auto-install) and all future releases
- Fixes the issue where PR #236 changes weren't applied until manual restart

Testing:
- Version endpoint working: returns {"version":"7.1.0"}
- Worker health check: passing
- Auto-restart logic: triggers on version mismatch

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-12 22:39:24 -05:00
Alex Newman 1cd545c36c Merge main into feature/localhost-only-binding - rebuild plugin files 2025-12-12 22:17:19 -05:00
7Sageer b8a9f366e7 feat(security): default worker to localhost-only binding
BREAKING: Worker now binds to 127.0.0.1 by default.
Set CLAUDE_MEM_WORKER_HOST=0.0.0.0 for remote access.
2025-12-11 22:01:31 +08:00
Alex Newman 83b0f9551b feat: add admin endpoints for process management and improve error handling
- Introduced `/api/admin/restart` and `/api/admin/shutdown` endpoints in WorkerService for restarting and shutting down the service.
- Updated error message in hook-error-handler to provide clearer instructions for restarting the worker.
- Refactored worker-utils to remove PM2 dependency and implement ProcessManager for starting the worker service.
- Cleaned up legacy PM2 references and provided new manual start instructions.
2025-12-10 23:46:17 -05:00
Alex Newman bafc86832c Update src/services/worker-service.ts
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-12-09 20:19:06 -05:00
CrystallDEV 5f36d2bf9a fix(windows): hide terminal windows when spawning child processes 2025-12-10 00:15:14 +01:00
kat-bell 1f2e5f1a9c fix(windows): Comprehensive fixes for Windows plugin installation
This PR addresses issue #193 affecting Windows installations of claude-mem.

## Bug 1: Missing ecosystem.config.cjs in packaged plugin

**Problem**: The ecosystem.config.cjs file was not included in the plugin
package, causing PM2 to fail when trying to start the worker from cache.

**Fix**: Added `plugin/ecosystem.config.cjs` with correct path for packaged
structure (`./scripts/worker-service.cjs` instead of `./plugin/scripts/`).

## Bug 2: Incorrect MCP Server Path (src/services/worker-service.ts)

**Problem**: Path `__dirname, '..', '..', 'plugin', 'scripts', 'mcp-server.cjs'`
only worked in dev structure, failed in packaged plugin.

**Error produced**:
```
Error: Cannot find module 'C:\Users\...\claude-mem\plugin\scripts\mcp-server.cjs'
[ERROR] [SYSTEM] Background initialization failed MCP error -32000: Connection closed
```

**Fix**: Changed to `path.join(__dirname, 'mcp-server.cjs')` since mcp-server.cjs
is in the same directory as worker-service.cjs after bundling.

## Bug 3: Missing smart-install.js in plugin package

**Problem**: smart-install.js was referenced in hooks.json but not included
in the plugin/ directory for cache deployment.

**Fix**: Added `plugin/scripts/smart-install.js` that uses `createRequire()`
to resolve modules from MARKETPLACE_ROOT.

## Bug 4: hooks.json incorrect path

**Problem**: Referenced `/../scripts/smart-install.js` but CLAUDE_PLUGIN_ROOT
points to the plugin/ directory.

**Fix**: Changed to `/scripts/smart-install.js`.

## Bug 5: Windows Worker Startup - Visible Console Windows

**Problem**: PM2 ignores windowsHide option on Windows, opening visible
console windows when starting the worker service.

**Fix**: Use PowerShell `Start-Process -WindowStyle Hidden` on Windows while
keeping PM2 for Unix systems (src/shared/worker-utils.ts).

## Additional Improvements

- Increased worker startup timeouts for Windows (500ms health check, 1000ms
  wait between retries, 15 retries = 15s total vs previous 5s)
- Added `windowsHide: true` to root ecosystem.config.cjs for PM2

## Note on Assertion Failure

The Windows libuv assertion failure `!(handle->flags & UV_HANDLE_CLOSING)`
at `src\win\async.c:76` is a known upstream issue in Claude Code (Issue #7579),
triggered by fetch() calls on Windows. This is NOT caused by worker spawning
and cannot be fixed in claude-mem.

Tested on Windows 11 with Node.js v24.

Fixes #193

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-09 05:02:56 -06:00
Alex Newman 54c53fda04 feat: Introduce SessionEventBroadcaster and SessionCompletionHandler for improved session management
- Added SessionEventBroadcaster to handle broadcasting of session lifecycle events, consolidating SSE broadcasting and processing status updates.
- Refactored SessionRoutes to utilize SessionEventBroadcaster for broadcasting events related to new prompts, session starts, and completions.
- Created SessionCompletionHandler to centralize session completion logic, reducing duplication across multiple endpoints.
- Updated WorkerService to initialize SessionEventBroadcaster and pass it to SessionRoutes.
2025-12-07 22:35:31 -05:00
Alex Newman b1fb135d9c refactor: remove obsolete uvx cleanup code from worker service
Remove cleanupOrphanedProcesses() method that was solving an old problem and is no longer needed. This method was platform-specific (crashes on Windows) and adds unnecessary complexity.

Changes:
- Delete cleanupOrphanedProcesses() method (28 lines)
- Remove call from initializeBackground()
- Worker starts cleanly without attempting process cleanup

Benefits:
- Simpler startup sequence
- Cross-platform compatibility (no pgrep/pkill)
- Reduced code complexity (YAGNI principle)
- No functional impact (orphaned processes indicate separate issues)

Phase 4 of Worker Service Quality Improvements Plan

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-07 21:32:50 -05:00
Alex Newman efcc557e4f Refactor WorkerService to allow searchRoutes to be nullable
- Changed searchRoutes property to be of type SearchRoutes | null.
- Initialized searchRoutes to null instead of using a temporary type assertion.
- Removed conditional setup for searchRoutes in setupRoutes method, as it will be handled after database initialization.
2025-12-07 21:16:42 -05:00
Alex Newman 7175b527a6 Refactor search functionality to utilize SearchManager
- Introduced SearchManager to handle search operations directly instead of proxying to MCP server.
- Updated WorkerService to initialize SearchManager after database setup.
- Modified SearchRoutes to call SearchManager methods for search operations.
- Adjusted SearchManager to manage timeline and formatting services.
- Enhanced error handling and logging for search operations.
2025-12-07 19:31:15 -05:00
Alex Newman c415ff5120 feat(timeline): implement TimelineService for building and formatting timeline items
- Extracted timeline-related functionality from mcp-server.ts to a dedicated TimelineService class.
- Added methods to build, filter, and format timeline items based on observations, sessions, and prompts.
- Introduced interfaces for TimelineItem and TimelineData to standardize data structures.
- Implemented sorting and grouping of timeline items by date, with markdown formatting for output.
- Included utility methods for date and time formatting, as well as token estimation.
2025-12-07 19:14:18 -05:00
Alex Newman 83b4806718 Refactor logging and improve service initialization
- Updated logging in `search-server.ts` to use `silentDebug` for non-critical messages, reducing console noise.
- Added a health check endpoint in `worker-service.ts` to improve service monitoring.
- Modified the worker service startup process to start the HTTP server immediately and perform slow initialization in the background.
- Refactored database initialization in `Database.ts` to use the correct `Database` class.
- Implemented a port availability check in `context-hook.ts` to ensure the worker service is ready before making requests.
2025-12-07 18:13:07 -05:00
Alex Newman 3aaee6f13a refactor: Organize worker into clean route-based HTTP architecture
Major architectural improvements to the worker service:

- Extracted monolithic WorkerService (~1900 lines) into organized route classes
- New HTTP layer with dedicated route handlers:
  - SessionRoutes: Session lifecycle operations
  - DataRoutes: Data retrieval endpoints
  - SearchRoutes: Search/MCP proxy operations
  - SettingsRoutes: Settings and configuration
  - ViewerRoutes: Health, UI, and SSE streaming
- Added comprehensive README documenting worker architecture
- Improved build script to handle worker service compilation
- Added context-generator for hook context operations

This is Phase 1 of worker refactoring - pure code reorganization with zero
functional changes. All existing behavior preserved while improving
maintainability and code organization.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-05 20:27:49 -05:00
Alex Newman d3aaef926b feat: Add worker HTTP endpoints for hook operations
- Add /api/context/inject endpoint (uses context-generator service)
- Add /api/sessions/observations endpoint (handles observations by claudeSessionId)
- Add /api/sessions/summarize endpoint (handles summaries by claudeSessionId)
- Add /api/sessions/complete endpoint (handles cleanup by claudeSessionId)
- Import stripMemoryTagsFromJson for privacy tag handling
- All endpoints accept claudeSessionId in request body (not path params)
- Enables hooks to become pure HTTP clients with zero database dependencies
2025-12-05 19:10:38 -05:00
Alex Newman e1d2ffeb02 fix: Hide console window on Windows when spawning child processes (#166)
* fix: Hide console window on Windows when spawning child processes

Add windowsHide: true to spawnSync and execSync calls to prevent
empty console windows from appearing on Windows when hooks execute.

Fixes two spawn points:
- worker-utils.ts: PM2 spawn when starting worker service
- user-message-hook.ts: Node spawn for context display

Reference: https://nodejs.org/api/child_process.html (windowsHide option)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Add windowsHide to remaining execSync calls for complete Windows console window hiding

This completes the Windows console window fix by adding `windowsHide: true` to all remaining `execSync` calls:

- src/services/worker-service.ts:220 - pgrep command for orphaned process detection
- src/services/worker-service.ts:226 - pkill command for process cleanup
- src/services/worker/SDKAgent.ts:414 - where/which claude command for finding executable

These operations are less frequent than the user-prompt hook, but should still avoid spawning console windows on Windows for a complete fix.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
2025-12-04 16:02:35 -05:00
Alex Newman 375dd1c3d6 feat: Add Context Settings Modal with Terminal Preview and UI Enhancements (#161)
* feat: Add Context Injection Settings modal with terminal preview

Adds a new settings modal accessible from the viewer UI header that allows users to configure context injection parameters with a live terminal preview showing how observations will appear.

Changes:
- New ContextSettingsModal component with auto-saving settings
- TerminalPreview component for live context visualization
- useContextPreview hook for fetching preview data
- Modal positioned to left of color mode button
- Settings sync with backend via worker service API

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: Add demo data and modify contextHook for cm_demo_content project

- Introduced DEMO_OBSERVATIONS and DEMO_SUMMARIES for the cm_demo_content project to provide mock data for testing and demonstration purposes.
- Updated contextHook to utilize demo data when the project is cm_demo_content, filtering observations based on configured types and concepts.
- Adjusted the worker service to use the contextHook with demo data, ensuring ANSI rendering for terminal output.
- Enhanced error handling and ensured proper closure of database connections.

* feat: add GitHub stars button with dynamic star count

- Implemented a new GitHubStarsButton component that fetches and displays the star count for a specified GitHub repository.
- Added useGitHubStars hook to handle API requests and state management for star count.
- Created formatStarCount utility function to format the star count into compact notation (k/M suffixes).
- Styled the GitHub stars button to match existing UI components, including hover and active states.
- Updated Header component to include the new GitHubStarsButton, replacing the static GitHub link.
- Added responsive styles to hide the GitHub stars button on mobile devices.

* feat: add API endpoint to fetch distinct projects and update context settings modal

- Implemented a new API endpoint `/api/projects` in `worker-service.ts` to retrieve a list of distinct projects from the observations.
- Modified `ContextSettingsModal.tsx` to replace the current project display with a dropdown for selecting projects, utilizing the fetched project list.
- Updated `useContextPreview.ts` to fetch projects on mount and manage the selected project state.
- Removed the `currentProject` prop from `ContextSettingsModal` and `App` components as it is now managed internally within the modal.

* Enhance Context Settings Modal and Terminal Preview

- Updated the styling of the Context Settings Modal for a modern clean design, including improved backdrop, header, and body layout.
- Introduced responsive design adjustments for smaller screens.
- Added custom scrollbar styles for better user experience.
- Refactored the TerminalPreview component to utilize `ansi-to-html` for rendering ANSI content, improving text display.
- Implemented new font variables for terminal styling across the application.
- Enhanced checkbox and input styles in the settings panel for better usability and aesthetics.
- Improved the layout and structure of settings groups and chips for a more organized appearance.

* Refactor UI components for compact design and enhance MCP toggle functionality

- Updated grid layout in viewer.html and viewer-template.html for better space utilization.
- Reduced padding and font sizes in settings groups, filter chips, and form controls for a more compact appearance.
- Implemented MCP toggle state management in ContextSettingsModal with API integration for status fetching and toggling.
- Reorganized settings groups for clarity, renaming and consolidating sections for improved user experience.
- Added feedback mechanism for MCP toggle status to inform users of changes and errors.

* feat: add collapsible sections, chip groups, form fields with tooltips, and toggle switches in settings modal

- Implemented collapsible sections for better organization of settings.
- Added chip groups with select all/none functionality for observation types and concepts.
- Enhanced form fields with optional tooltips for better user guidance.
- Introduced toggle switches for various settings, improving user interaction.
- Updated styles for new components to ensure consistency and responsiveness.
- Refactored ContextSettingsModal to utilize new components and improve readability.
- Improved TerminalPreview component styling for better layout and usability.

* Refactor modal header and preview selector styles; enhance terminal preview functionality

- Updated modal header padding and added gap for better spacing.
- Introduced a new header-controls section to include a project preview selector.
- Enhanced the preview selector styles for improved usability and aesthetics.
- Adjusted the preview column styles for a cleaner look.
- Implemented word wrap toggle functionality in the TerminalPreview component, allowing users to switch between wrapped and scrollable text.
- Improved scroll position handling in TerminalPreview to maintain user experience during content updates.

* feat: enhance modal settings with new icon links and update header controls

- Added new modal icon links for documentation and social media in ContextSettingsModal.
- Updated the header to remove sidebar toggle functionality and replaced it with context preview toggle.
- Refactored styles for modal icon links to improve UI/UX.
- Removed sidebar component from App and adjusted related state management.

* chore: remove abandoned cm_demo_content demo data approach

The demo data feature was prototyped but didn't work out. Removes:
- DEMO_OBSERVATIONS and DEMO_SUMMARIES arrays
- Conditional logic that bypassed DB for demo project
- Demo mode check in prior message extraction

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-03 21:34:41 -05:00
Alex Newman c768a80bf0 Refactor context configuration and settings handling
- Updated context configuration loading path from ~/.claude/settings.json to ~/.claude-mem/settings.json.
- Modified the extractPriorMessages function to focus on retrieving the last assistant message only, removing user message extraction.
- Enhanced output formatting for displaying prior assistant messages in the context hook.
- Added new settings related to token economics and observation filtering in the useSettings hook.
2025-12-01 19:26:33 -05:00
Alex Newman d1876cb6e0 Refactor observation handling: centralize constants and improve context settings
- Introduced `observation-metadata.ts` to define valid observation types and concepts, along with their corresponding emoji mappings.
- Updated `context-hook.ts` to utilize new constants for observation types and concepts, enhancing maintainability.
- Refactored `worker-service.ts` to validate observation types and concepts against the new centralized constants.
- Consolidated settings management in `Sidebar.tsx` to streamline state handling for context settings.
- Improved error handling and logging for context loading failures.
2025-12-01 17:29:48 -05:00
Alex Newman e1017b483b feat: Enhance context settings with validation and UI options
- Added ContextConfig interface and loadContextConfig function to manage context settings.
- Implemented validation for context settings in WorkerService.
- Updated Sidebar component to include new context settings for token economics, observation filtering, display configuration, and feature toggles.
- Introduced default settings for new context features.
- Adjusted types to accommodate new settings in the application state.
2025-12-01 16:53:35 -05:00
Alex Newman 8caf159d99 feat: Add branch-based beta toggle for switching between stable and beta versions
Adds Version Channel section to Settings sidebar allowing users to:
- See current branch (main or beta/7.0) and stability status
- Switch to beta branch to access Endless Mode features
- Switch back to stable for production use
- Pull updates for current branch

Implementation:
- BranchManager.ts: Git operations for branch detection/switching
- worker-service.ts: /api/branch/* endpoints (status, switch, update)
- Sidebar.tsx: Version Channel UI with branch state and handlers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-25 14:12:49 -05:00
Alex Newman c5e68a17c8 refactor: Clean up search architecture, remove experimental contextualize endpoint (#133)
* Refactor code structure for improved readability and maintainability

* Add test results for search API and related functionalities

- Created test result files for various search-related functionalities, including:
  - test-11-search-server-changes.json
  - test-12-context-hook-changes.json
  - test-13-worker-service-changes.json
  - test-14-patterns.json
  - test-15-gotchas.json
  - test-16-discoveries.json
  - test-17-all-bugfixes.json
  - test-18-all-features.json
  - test-19-all-decisions.json
  - test-20-session-search.json
  - test-21-prompt-search.json
  - test-22-decisions-endpoint.json
  - test-23-changes-endpoint.json
  - test-24-how-it-works-endpoint.json
  - test-25-contextualize-endpoint.json
  - test-26-timeline-around-observation.json
  - test-27-multi-param-combo.json
  - test-28-file-type-combo.json

- Each test result file captures specific search failures or outcomes, including issues with undefined properties and successful execution of search queries.
- Enhanced documentation of search architecture and testing strategies, ensuring compliance with established guidelines and improving overall search functionality.

* feat: Enhance unified search API with catch-all parameters and backward compatibility

- Implemented a unified search API at /api/search that accepts catch-all parameters for filtering by type, observation type, concepts, and files.
- Maintained backward compatibility by keeping granular endpoints functional while routing through the same infrastructure.
- Completed comprehensive testing of search capabilities with real-world query scenarios.

fix: Address missing debug output in search API query tests

- Flushed PM2 logs and executed search queries to verify functionality.
- Diagnosed absence of "Raw Chroma" debug messages in worker logs, indicating potential issues with logging or query processing.

refactor: Improve build and deployment pipeline for claude-mem plugin

- Successfully built and synced all hooks and services to the marketplace directory.
- Ensured all dependencies are installed and up-to-date in the deployment location.

feat: Implement hybrid search filters with 90-day recency window

- Enhanced search server to apply a 90-day recency filter to Chroma results before categorizing by document type.

fix: Correct parameter handling in searchUserPrompts method

- Added support for filter-only queries and improved dual-path logic for clarity.

refactor: Rename FTS5 method to clarify fallback status

- Renamed escapeFTS5 to escapeFTS5_fallback_when_chroma_unavailable to indicate its temporary usage.

feat: Introduce contextualize tool for comprehensive project overview

- Added a new tool to fetch recent observations, sessions, and user prompts, providing a quick project overview.

feat: Add semantic shortcut tools for common search patterns

- Implemented 'decisions', 'changes', and 'how_it_works' tools for convenient access to frequently searched observation categories.

feat: Unified timeline tool supports anchor and query modes

- Combined get_context_timeline and get_timeline_by_query into a single interface for timeline exploration.

feat: Unified search tool added to MCP server

- New tool queries all memory types simultaneously, providing combined chronological results for improved search efficiency.

* Refactor search functionality to clarify FTS5 fallback usage

- Updated `worker-service.cjs` to replace FTS5 fallback function with a more descriptive name and improved error handling.
- Enhanced documentation in `SKILL.md` to specify the unified API endpoint and clarify the behavior of the search engine, including the conditions under which FTS5 is used.
- Modified `search-server.ts` to provide clearer logging and descriptions regarding the fallback to FTS5 when UVX/Python is unavailable.
- Renamed and updated the `SessionSearch.ts` methods to reflect the conditions for using FTS5, emphasizing the lack of semantic understanding in fallback scenarios.

* feat: Add ID-based fetch endpoints and simplify mem-search skill

**Problem:**
- Search returns IDs but no way to fetch by ID
- Skill documentation was bloated with too many options
- Claude wasn't using IDs because we didn't tell it how

**Solution:**
1. Added three new HTTP endpoints:
   - GET /api/observation/:id
   - GET /api/session/:id
   - GET /api/prompt/:id

2. Completely rewrote SKILL.md:
   - Stripped complexity down to essentials
   - Clear 3-step prescriptive workflow: Search → Review IDs → Fetch by ID
   - Emphasized ID usage: "The IDs are there for a reason - USE THEM"
   - Removed confusing multi-endpoint documentation
   - Kept only unified search with filters

**Impact:**
- Token efficiency: Claude can now fetch full details only for relevant IDs
- Clarity: One clear workflow instead of 10+ options to choose from
- Usability: IDs are no longer wasted context - they're actionable

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: Move internal docs to private directory

Moved POSTMORTEM and planning docs to ./private to exclude from PR reviews.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Remove experimental contextualize endpoint

- Removed contextualize MCP tool from search-server (saves ~4KB)
- Disabled FTS5 fallback paths in SessionSearch (now vector-first)
- Cleaned up CLAUDE.md documentation
- Removed contextualize-rewrite-plan.md doc

Rationale:
- Contextualize is better suited as a skill (LLM-powered) than an endpoint
- Search API already provides vector search with configurable limits
- Created issue #132 to track future contextualize skill implementation

Changes:
- src/servers/search-server.ts: Removed contextualize tool definition
- src/services/sqlite/SessionSearch.ts: Disabled FTS5 fallback, added deprecation warnings
- CLAUDE.md: Cleaned up outdated skill documentation
- docs/: Removed contextualize plan document

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Complete FTS5 cleanup - remove all deprecated search code

This completes the FTS5 cleanup work by removing all commented-out
FTS5 search code while preserving database tables for backward compatibility.

Changes:
- Removed 200+ lines of commented FTS5 search code from SessionSearch.ts
- Removed deprecated degraded_search_query__when_uvx_unavailable method
- Updated all method documentation to clarify vector-first architecture
- Updated class documentation to reflect filter-only query support
- Updated CLAUDE.md to remove FTS5 search references
- Clarified that FTS5 tables exist for backward compatibility only
- Updated "Why SQLite FTS5" section to "Why Vector-First Search"

Database impact: NONE - FTS5 tables remain intact for existing installations

Search architecture:
- ChromaDB: All text-based vector search queries
- SQLite: Filter-only queries (date ranges, metadata, no query text)
- FTS5 tables: Maintained but unused (backward compatibility)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Remove all FTS5 fallback execution code from search-server

Completes the FTS5 cleanup by removing all fallback execution paths
that attempted to use FTS5 when ChromaDB was unavailable.

Changes:
- Removed all FTS5 fallback code execution paths
- When ChromaDB fails or is unavailable, return empty results with helpful error messages
- Updated all deprecated tool descriptions (search_observations, search_sessions, search_user_prompts)
- Changed error messages to indicate FTS5 fallback has been removed
- Added installation instructions for UVX/Python when vector search is unavailable
- Updated comments from "hybrid search" to "vector-first search"
- Removed ~100 lines of dead FTS5 fallback code

Database impact: NONE - FTS5 tables remain intact (backward compatibility)

Search behavior when ChromaDB unavailable:
- Text queries: Return empty results with error explaining ChromaDB is required
- Filter-only queries (no text): Continue to work via direct SQLite

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Address PR 133 review feedback

Critical fixes:
- Remove contextualize endpoint from worker-service (route + handler)
- Fix build script logging to show correct .cjs extension (was .mjs)

Documentation improvements:
- Add comprehensive FTS5 retention rationale documentation
- Include v7.0.0 removal TODO for future cleanup

Testing:
- Build succeeds with correct output logging
- Worker restarts successfully (30th restart)
- Contextualize endpoint properly removed (404 response)
- Search endpoint verified working

This addresses all critical review feedback from PR 133.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-21 18:59:23 -05:00
Alex Newman 74c8afd0e0 feat: Add real-time queue depth indicator to viewer UI
Implements a visual badge that displays the count of active work items (queued + currently processing) in the worker service. The badge appears next to the claude-mem logo and updates in real-time via SSE.

Features:
- Shows count of pending messages + active SDK generators
- Only displays when queueDepth > 0
- Subtle pulse animation for visual feedback
- Theme-aware styling

Backend changes:
- Added getTotalActiveWork() method to SessionManager
- Updated worker-service to broadcast queueDepth via SSE
- Enhanced processing status API endpoint

Frontend changes:
- Updated Header component to display queue bubble
- Enhanced useSSE hook to track queueDepth state
- Added CSS styling with pulse animation

Closes #122
Closes #96
Closes #97

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 15:11:37 -05:00
Alex Newman 02fef487e7 feat: add cleanup for orphaned MCP server processes on startup
- Implemented a new method `cleanupOrphanedProcesses` to identify and terminate orphaned `uvx` processes from previous sessions.
- Integrated the cleanup method into the `start` process of the WorkerService to ensure a clean environment at startup.
- Added logging for process cleanup actions and handled potential errors gracefully without failing the service startup.
2025-11-16 22:36:39 -05:00
Copilot c46e4a341a Fix memory leaks from orphaned uvx/python processes (#120)
This fixes memory leak, will remove one unnecessary MCP after this in a new PR but this is mission critical fix

* Initial plan

* Fix memory leaks: Add proper cleanup for ChromaSync and search server processes

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Add comprehensive process cleanup and PM2 configuration improvements

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Add comprehensive summary and recommendations for memory leak fixes

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-16 22:16:41 -05:00
Alex Newman 68290a9121 Performance improvements: Token reduction and enhanced summaries (#101)
* refactor: Reduce continuation prompt token usage by 95 lines

Removed redundant instructions from continuation prompt that were originally
added to mitigate a session continuity issue. That issue has since been
resolved, making these detailed instructions unnecessary on every continuation.

Changes:
- Reduced continuation prompt from ~106 lines to ~11 lines (~95 line reduction)
- Changed "User's Goal:" to "Next Prompt in Session:" (more accurate framing)
- Removed redundant WHAT TO RECORD, WHEN TO SKIP, and OUTPUT FORMAT sections
- Kept concise reminder: "Continue generating observations and progress summaries..."
- Initial prompt still contains all detailed instructions

Impact:
- Significant token savings on every continuation prompt
- Faster context injection with no loss of functionality
- Instructions remain comprehensive in initial prompt

Files modified:
- src/sdk/prompts.ts (buildContinuationPrompt function)
- plugin/scripts/worker-service.cjs (compiled output)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Enhance observation and summary prompts for clarity and token efficiency

* Enhance prompt clarity and instructions in prompts.ts

- Added a reminder to think about instructions before starting work.
- Simplified the continuation prompt instruction by removing "for this ongoing session."

* feat: Enhance settings.json with permissions and deny access to sensitive files

refactor: Remove PLAN-full-observation-display.md and PR_SUMMARY.md as they are no longer needed

chore: Delete SECURITY_SUMMARY.md since it is redundant after recent changes

fix: Update worker-service.cjs to streamline observation generation instructions

cleanup: Remove src-analysis.md and src-tree.md for a cleaner codebase

refactor: Modify prompts.ts to clarify instructions for memory processing

* refactor: Remove legacy worker service implementation

* feat: Enhance summary hook to extract last assistant message and improve logging

- Added function to extract the last assistant message from the transcript.
- Updated summary hook to include last assistant message in the summary request.
- Modified SDKSession interface to store last assistant message.
- Adjusted buildSummaryPrompt to utilize last assistant message for generating summaries.
- Updated worker service and session manager to handle last assistant message in summarize requests.
- Introduced silentDebug utility for improved logging and diagnostics throughout the summary process.

* docs: Add comprehensive implementation plan for ROI metrics feature

Added detailed implementation plan covering:
- Token usage capture from Agent SDK
- Database schema changes (migration #8)
- Discovery cost tracking per observation
- Context hook display with ROI metrics
- Testing and rollout strategy

Timeline: ~20 hours over 4 days
Goal: Empirical data for YC application amendment

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: Add transcript processing scripts for analysis and formatting

- Implemented `dump-transcript-readable.ts` to generate a readable markdown dump of transcripts, excluding certain entry types.
- Created `extract-rich-context-examples.ts` to extract and showcase rich context examples from transcripts, highlighting user requests and assistant reasoning.
- Developed `format-transcript-context.ts` to format transcript context into a structured markdown format for improved observation generation.
- Added `test-transcript-parser.ts` for validating data extraction from transcript JSONL files, including statistics and error reporting.
- Introduced `transcript-to-markdown.ts` for a complete representation of transcript data in markdown format, showing all context data.
- Enhanced type definitions in `transcript.ts` to support new features and ensure type safety.
- Built `transcript-parser.ts` to handle parsing of transcript JSONL files, including error handling and data extraction methods.

* Refactor hooks and SDKAgent for improved observation handling

- Updated `new-hook.ts` to clean user prompts by stripping leading slashes for better semantic clarity.
- Enhanced `save-hook.ts` to include additional tools in the SKIP_TOOLS set, preventing unnecessary observations from certain command invocations.
- Modified `prompts.ts` to change the structure of observation prompts, emphasizing the observational role and providing a detailed XML output format for observations.
- Adjusted `SDKAgent.ts` to enforce stricter tool usage restrictions, ensuring the memory agent operates solely as an observer without any tool access.

* feat: Enhance session initialization to accept user prompts and prompt numbers

- Updated `handleSessionInit` in `worker-service.ts` to extract `userPrompt` and `promptNumber` from the request body and pass them to `initializeSession`.
- Modified `initializeSession` in `SessionManager.ts` to handle optional `currentUserPrompt` and `promptNumber` parameters.
- Added logic to update the existing session's `userPrompt` and `lastPromptNumber` if a `currentUserPrompt` is provided.
- Implemented debug logging for session initialization and updates to track user prompts and prompt numbers.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-13 18:22:44 -05:00
Alex Newman 39fedfc5fc feat: Enhance summary hook to include last user message from transcript (#95)
* feat: Enhance summary hook to include last user message from transcript

- Added functionality to extract the last user message from a JSONL transcript file in the summary hook.
- Updated the summary hook to send the last user message along with the summary request.
- Modified the SDKSession interface to include an optional last_user_message field.
- Updated the summary prompt to incorporate the last user message in the output format.
- Refactored worker service to handle the last user message in the summarize queue.
- Enhanced session manager to track and broadcast processing status based on active sessions and queue depth.
- Improved error handling and logging for better traceability during transcript reading and processing.

* feat(worker): enhance processing status broadcasting and session management

- Added immediate broadcasting of processing status when a prompt is received.
- Implemented logging for generator completion in multiple locations.
- Updated `broadcastProcessingStatus` to include queue depth and active session count in logs.
- Modified session iterator to stop yielding messages after a summary is yielded, with appropriate logging.
2025-11-11 17:38:22 -05:00
basher83 97d565e3cd Replace search skill with mem-search (#91)
* feat: add mem-search skill with progressive disclosure architecture

Add comprehensive mem-search skill for accessing claude-mem's persistent
cross-session memory database. Implements progressive disclosure workflow
and token-efficient search patterns.

Features:
- 12 search operations (observations, sessions, prompts, by-type, by-concept, by-file, timelines, etc.)
- Progressive disclosure principles to minimize token usage
- Anti-patterns documentation to guide LLM behavior
- HTTP API integration for all search functionality
- Common workflows with composition examples

Structure:
- SKILL.md: Entry point with temporal trigger patterns
- principles/: Progressive disclosure + anti-patterns
- operations/: 12 search operation files

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* docs: add CHANGELOG entry for mem-search skill

Document mem-search skill addition in Unreleased section with:
- 100% effectiveness compliance metrics
- Comparison to previous search skill implementation
- Progressive disclosure architecture details
- Reference to audit report documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* docs: add mem-search skill audit report

Add comprehensive audit report validating mem-search skill against
Anthropic's official skill-creator documentation.

Report includes:
- Effectiveness metrics comparison (search vs mem-search)
- Critical issues analysis for production readiness
- Compliance validation across 6 key dimensions
- Reference implementation guidance

Result: mem-search achieves 100% compliance vs search's 67%

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: Add comprehensive search architecture analysis document

- Document current state of dual search architectures (HTTP API and MCP)
- Analyze HTTP endpoints and MCP search server architectures
- Identify DRY violations across search implementations
- Evaluate the use of curl as the optimal approach for search
- Provide architectural recommendations for immediate and long-term improvements
- Outline action plan for cleanup, feature parity, DRY refactoring

* refactor: Remove deprecated search skill documentation and operations

* refactor: Reorganize documentation into public and context directories

Changes:
- Created docs/public/ for Mintlify documentation (.mdx files)
- Created docs/context/ for internal planning and implementation docs
- Moved all .mdx files and assets to docs/public/
- Moved all internal .md files to docs/context/
- Added CLAUDE.md to both directories explaining their purpose
- Updated docs.json paths to work with new structure

Benefits:
- Clear separation between user-facing and internal documentation
- Easier to maintain Mintlify docs in dedicated directory
- Internal context files organized separately

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Enhance session management and continuity in hooks

- Updated new-hook.ts to clarify session_id threading and idempotent session creation.
- Modified prompts.ts to require claudeSessionId for continuation prompts, ensuring session context is maintained.
- Improved SessionStore.ts documentation on createSDKSession to emphasize idempotent behavior and session connection.
- Refined SDKAgent.ts to detail continuation prompt logic and its reliance on session.claudeSessionId for unified session handling.

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Alex Newman <thedotmack@gmail.com>
2025-11-11 16:15:07 -05:00
Alex Newman 3529f9274b feat: Enhanced logging and SDK prompt improvements (#94)
* Initial plan

* Initial analysis: Found root cause of double entries bug

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Fix double entries by assigning generatorPromise in handleSessionInit

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* feat(logging): Enhance HTTP request logging and session management

- Added middleware for logging HTTP requests and responses, excluding static assets and health checks.
- Introduced a method to summarize request bodies for specific endpoints.
- Improved logging for user prompt synchronization with Chroma, including duration tracking.
- Enhanced session initialization logging to include additional session details.
- Updated observation and summary logging to provide more context and error handling during Chroma synchronization.
- Refactored tool name formatting for logging in the SessionManager.
- Expanded logger component types to include 'HTTP', 'SESSION', and 'CHROMA'.

* Refactor SDK prompts and logging for improved clarity and functionality

- Updated buildInitPrompt to clarify the observer's role and what to record.
- Enhanced buildSummaryPrompt with clearer instructions for summarizing ongoing sessions.
- Improved buildContinuationPrompt to emphasize the focus on deliverables and capabilities.
- Refactored WorkerService to utilize a centralized tool formatting function for logging.
- Added truncation for logged responses and observations to improve readability.
- Updated SessionManager to log the queuing of summarize actions with session details.
- Enhanced App and Sidebar components to support refreshing stats on sidebar open.
- Refactored useStats hook to allow manual refreshing of stats while maintaining automatic loading on mount.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-11 13:49:00 -05:00
Copilot 1bb203cbb5 Fix duplicate entries in viewer caused by untracked SDK agent promise (#86)
* Initial plan

* Initial analysis: Found root cause of double entries bug

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Fix double entries by assigning generatorPromise in handleSessionInit

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
Co-authored-by: Alex Newman <thedotmack@gmail.com>
2025-11-10 17:30:02 -05:00
copilot-swe-agent[bot] 730c420a13 Add CWD propagation through hook, worker, and SDK agent
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-10 19:23:06 +00:00
Alex Newman 9e235b5b57 feat: Add MCP search server toggle with dedicated API architecture (#85)
* feat: add MCP search server toggle functionality

- Introduced `CLAUDE_MEM_MCP_ENABLED` setting to manage the MCP search server state.
- Updated `WorkerService` to handle MCP enabling/disabling based on the new setting.
- Enhanced `Sidebar` component to include a checkbox for toggling MCP search server.
- Modified `useSettings` hook to incorporate the new MCP setting.
- Updated default settings to include `CLAUDE_MEM_MCP_ENABLED` with a default value of true.
- Adjusted TypeScript types to include the new MCP setting.

* feat: Implement MCP toggle functionality in WorkerService and Sidebar

- Added API endpoints for MCP status retrieval and toggling in WorkerService.
- Updated Sidebar component to manage MCP toggle state and display status messages.
- Removed MCP_ENABLED from settings state management and default settings.
- Adjusted settings interface and related hooks to reflect the removal of MCP_ENABLED.
2025-11-10 13:47:25 -05:00
Alex Newman ca4f046777 feat: Add search skill with progressive disclosure and refactor existing skills
Enhancements:
- Added search skill with 10 HTTP API endpoints for memory queries
- Refactored version-bump and troubleshoot skills using progressive disclosure pattern
- Added operations/ subdirectories for detailed skill documentation
- Updated CLAUDE.md with skill-based search architecture
- Enhanced worker service with search API endpoints
- Updated CHANGELOG.md with v5.4.0 migration details

Technical changes:
- New plugin/skills/search/ directory with SKILL.md
- New .claude/skills/version-bump/operations/ (workflow.md, scenarios.md)
- New plugin/skills/troubleshoot/operations/ (common-issues.md, worker.md)
- Modified src/services/worker-service.ts (added search endpoints)
- Modified plugin/scripts/worker-service.cjs (rebuilt with search API)
- Reduced main skill files by 89% using progressive disclosure
- Token savings: ~2,250 tokens per session start

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-09 18:41:53 -05:00
Alex Newman 740d65b5a5 Add TypeScript Agent SDK reference documentation
- Introduced comprehensive API reference for the TypeScript Agent SDK.
- Documented installation instructions for the SDK.
- Detailed the main functions: `query()`, `tool()`, and `createSdkMcpServer()`.
- Defined various types including `Options`, `Query`, `AgentDefinition`, and more.
- Included message types and their structures, such as `SDKMessage`, `SDKAssistantMessage`, and `SDKUserMessage`.
- Explained hook types and their usage within the SDK.
- Provided detailed documentation for tool input and output types.
- Added sections on permission types and other relevant types for better clarity.
2025-11-07 15:05:31 -05:00
Alex Newman 4bc467f7ed feat: Implement Worker Service for long-running HTTP service with PM2 management
- Introduced WorkerService class to handle HTTP requests and manage sessions.
- Added endpoints for health check, session management, and data retrieval.
- Integrated ChromaSync for background data synchronization.
- Implemented SSE for real-time updates to connected clients.
- Added error handling and logging throughout the service.
- Cached Claude executable path for improved performance.
- Included settings management for user configuration.
- Established database interactions for session and observation management.
2025-11-07 13:26:13 -05:00
Alex Newman 13643a5b18 Release v5.1.3: Fix PostToolUse hook field name bug
Bug Fix:
- Changed tool_output to tool_response throughout PostToolUse hook chain
- PostToolUse events provide tool_response field, not tool_output
- Updated save-hook.ts to send tool_response
- Updated worker-service.ts endpoint to accept tool_response
- Updated worker-service-v2.ts for consistency
- Updated worker-types.ts interfaces (PendingMessage, ObservationData)
- Updated SessionManager.ts message queue
- Updated SDKAgent.ts observation prompt builder

Impact: Fixes observation capture for PostToolUse events. Previous version was sending undefined for tool responses, causing incomplete observations.

Files changed:
- src/hooks/save-hook.ts (interface + destructuring + fetch body)
- src/services/worker-service.ts (ObservationMessage interface + handler + SDK prompt)
- src/services/worker-service-v2.ts (handler)
- src/services/worker-types.ts (PendingMessage + ObservationData interfaces)
- src/services/worker/SessionManager.ts (queue push)
- src/services/worker/SDKAgent.ts (observation prompt)
- plugin/scripts/*.js (rebuilt)
- Version bumped to 5.1.3 (patch)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-07 11:47:03 -05:00
Alex Newman 3bdf0b41bc chore: Remove unused imports and variables
Cleanup:
- Removed unused SDKSystemMessage import
- Removed unused ensureAllDataDirs import
- Removed unused userPrompt destructuring in handleInit

All TypeScript diagnostics now clear.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-06 22:03:11 -05:00
Alex Newman 3030f518b5 refactor: Complete rewrite of worker-utils.ts and cleanup of worker-service.ts
- Removed fragile PM2 string parsing and replaced with direct PM2 restart logic.
- Eliminated silent error handling in worker-utils.ts for better error visibility.
- Extracted duplicated session auto-creation logic into a new helper method getOrCreateSession() in worker-service.ts.
- Centralized configuration values and replaced magic numbers with named constants.
- Updated health check logic to ensure worker is restarted if unhealthy.
- Removed unnecessary getWorkerPort() wrapper function.
- Improved overall code quality and maintainability by applying DRY and YAGNI principles.
2025-11-06 22:00:07 -05:00
Alex Newman f8dc7f940f uploade 2025-11-06 15:47:16 -05:00