214 Commits

Author SHA1 Message Date
Alex Newman 01b3103567 chore: Bump version to 6.3.4
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-30 16:08:03 -05:00
Alex Newman 03b85d2af2 chore: Bump version to 6.3.3 2025-11-30 15:01:02 -05:00
Alex Newman 03fefca884 Enhance memory search functionality with timeline context retrieval (#151)
- Introduced optional timeline context retrieval step in memory search flow to provide users with better understanding of previous sessions.
- Updated SKILL.md to reflect new flow, including timeline context commands and usage scenarios.
- Refactored timeline retrieval commands in timeline-by-query.md and timeline.md to utilize new MCP tools for streamlined access.
- Implemented filtering logic in search-server.ts to respect depth_before and depth_after parameters when displaying timeline items.
- Improved response formatting to include filtered item counts and enhanced user guidance for timeline queries.
2025-11-30 14:58:01 -05:00
Alex Newman 01be3156fb fix: context hook updates and cleanup (#150)
* fix(context-hook): update savings message to reference mem-search skill

Changed "Use claude-mem search" to "Use the mem-search skill" for clarity.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: delete outdated docs, experiments, and test results

Removed:
- docs/context/ (moved to private/)
- experiment/ (obsolete)
- test-results/ (stale)
- tests/ (outdated)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(user-message-hook): update support link to Discord community

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-28 20:17:44 -05:00
Alex Newman 0a70bcecc5 chore: Bump version to 6.3.2
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-25 16:38:06 -05:00
Alex Newman 4e5913611a feat(search-server): enhance decision search with optional semantic query support
- Updated the 'decisions' tool to accept an optional 'query' parameter for semantic filtering.
- Implemented logic to handle semantic search using Chroma when a query is provided.
- Preserved ranking order of results based on Chroma's output.
- Added fallback to metadata-first search when no query is present.
2025-11-25 16:37:08 -05:00
Alex Newman 93de6d97f5 chore: Bump version to 6.3.1
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-25 15:47:47 -05:00
Alex Newman 6ffa4cfde5 chore: Bump version to 6.3.0
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-25 14:13:32 -05:00
Alex Newman 8caf159d99 feat: Add branch-based beta toggle for switching between stable and beta versions
Adds Version Channel section to Settings sidebar allowing users to:
- See current branch (main or beta/7.0) and stability status
- Switch to beta branch to access Endless Mode features
- Switch back to stable for production use
- Pull updates for current branch

Implementation:
- BranchManager.ts: Git operations for branch detection/switching
- worker-service.ts: /api/branch/* endpoints (status, switch, update)
- Sidebar.tsx: Version Channel UI with branch state and handlers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-25 14:12:49 -05:00
Alex Newman 634033b730 chore: Bump version to 6.2.1 2025-11-23 16:18:58 -05:00
Alex Newman 54ef1496d2 fix: Refresh in-memory session project when updated in database
**Problem:**
- Summaries created with empty project names even after database fix
- 13 summaries had empty project, couldn't appear in context-hook
- In-memory sessions cached stale empty project value

**Root Cause:**
SessionManager caches ActiveSession objects in memory. When reusing
an existing session, it never refreshed the project field from the
database. Even though createSDKSession() now updates the DB with the
correct project, the in-memory session.project remained empty, causing
summaries to be stored with empty project.

**Flow:**
1. Session created with empty project (cached in memory)
2. new-hook UPDATE fixes project in database
3. SessionManager reuses cached session (stale project="")
4. Summary stored with session.project (empty)
5. Context-hook can't find summary (WHERE project = 'claude-mem')

**Solution:**
When SessionManager.initializeSession() reuses an existing session,
refresh the project field from the database. This ensures summaries
and observations always use the latest project value.

**Impact:**
- Future summaries will have correct project name
- Backfilled 13 historical summaries with project='claude-mem'
- Context injection will now show recent summaries
- Both observations AND summaries fixed

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-23 16:14:53 -05:00
Alex Newman 5d23c60b76 fix: Update project name when session already exists in createSDKSession
**Problem:**
- Sessions created with empty project names couldn't be updated
- INSERT OR IGNORE skipped updates when session already existed
- Context-hook couldn't find observations (WHERE project = 'claude-mem')
- 364 observations had empty project names

**Root Cause:**
createSDKSession() used INSERT OR IGNORE for idempotency, but never
updated project/prompt fields when session already existed. If SAVE hook
created session first (with empty project), NEW hook couldn't fix it.

**Solution:**
When INSERT is ignored (session exists), UPDATE project and user_prompt
if we have non-empty values. This ensures project name gets set even
when session was created by SAVE hook or with incomplete data.

**Impact:**
- New sessions will always have correct project name
- Backfilled 364 historical observations with project='claude-mem'
- Context injection will now work (finds observations by project)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-23 16:06:46 -05:00
Alex Newman 58f4053a61 chore: Bump version to 6.2.0 2025-11-22 00:27:34 -05:00
Alex Newman c5e68a17c8 refactor: Clean up search architecture, remove experimental contextualize endpoint (#133)
* Refactor code structure for improved readability and maintainability

* Add test results for search API and related functionalities

- Created test result files for various search-related functionalities, including:
  - test-11-search-server-changes.json
  - test-12-context-hook-changes.json
  - test-13-worker-service-changes.json
  - test-14-patterns.json
  - test-15-gotchas.json
  - test-16-discoveries.json
  - test-17-all-bugfixes.json
  - test-18-all-features.json
  - test-19-all-decisions.json
  - test-20-session-search.json
  - test-21-prompt-search.json
  - test-22-decisions-endpoint.json
  - test-23-changes-endpoint.json
  - test-24-how-it-works-endpoint.json
  - test-25-contextualize-endpoint.json
  - test-26-timeline-around-observation.json
  - test-27-multi-param-combo.json
  - test-28-file-type-combo.json

- Each test result file captures specific search failures or outcomes, including issues with undefined properties and successful execution of search queries.
- Enhanced documentation of search architecture and testing strategies, ensuring compliance with established guidelines and improving overall search functionality.

* feat: Enhance unified search API with catch-all parameters and backward compatibility

- Implemented a unified search API at /api/search that accepts catch-all parameters for filtering by type, observation type, concepts, and files.
- Maintained backward compatibility by keeping granular endpoints functional while routing through the same infrastructure.
- Completed comprehensive testing of search capabilities with real-world query scenarios.

fix: Address missing debug output in search API query tests

- Flushed PM2 logs and executed search queries to verify functionality.
- Diagnosed absence of "Raw Chroma" debug messages in worker logs, indicating potential issues with logging or query processing.

refactor: Improve build and deployment pipeline for claude-mem plugin

- Successfully built and synced all hooks and services to the marketplace directory.
- Ensured all dependencies are installed and up-to-date in the deployment location.

feat: Implement hybrid search filters with 90-day recency window

- Enhanced search server to apply a 90-day recency filter to Chroma results before categorizing by document type.

fix: Correct parameter handling in searchUserPrompts method

- Added support for filter-only queries and improved dual-path logic for clarity.

refactor: Rename FTS5 method to clarify fallback status

- Renamed escapeFTS5 to escapeFTS5_fallback_when_chroma_unavailable to indicate its temporary usage.

feat: Introduce contextualize tool for comprehensive project overview

- Added a new tool to fetch recent observations, sessions, and user prompts, providing a quick project overview.

feat: Add semantic shortcut tools for common search patterns

- Implemented 'decisions', 'changes', and 'how_it_works' tools for convenient access to frequently searched observation categories.

feat: Unified timeline tool supports anchor and query modes

- Combined get_context_timeline and get_timeline_by_query into a single interface for timeline exploration.

feat: Unified search tool added to MCP server

- New tool queries all memory types simultaneously, providing combined chronological results for improved search efficiency.

* Refactor search functionality to clarify FTS5 fallback usage

- Updated `worker-service.cjs` to replace FTS5 fallback function with a more descriptive name and improved error handling.
- Enhanced documentation in `SKILL.md` to specify the unified API endpoint and clarify the behavior of the search engine, including the conditions under which FTS5 is used.
- Modified `search-server.ts` to provide clearer logging and descriptions regarding the fallback to FTS5 when UVX/Python is unavailable.
- Renamed and updated the `SessionSearch.ts` methods to reflect the conditions for using FTS5, emphasizing the lack of semantic understanding in fallback scenarios.

* feat: Add ID-based fetch endpoints and simplify mem-search skill

**Problem:**
- Search returns IDs but no way to fetch by ID
- Skill documentation was bloated with too many options
- Claude wasn't using IDs because we didn't tell it how

**Solution:**
1. Added three new HTTP endpoints:
   - GET /api/observation/:id
   - GET /api/session/:id
   - GET /api/prompt/:id

2. Completely rewrote SKILL.md:
   - Stripped complexity down to essentials
   - Clear 3-step prescriptive workflow: Search → Review IDs → Fetch by ID
   - Emphasized ID usage: "The IDs are there for a reason - USE THEM"
   - Removed confusing multi-endpoint documentation
   - Kept only unified search with filters

**Impact:**
- Token efficiency: Claude can now fetch full details only for relevant IDs
- Clarity: One clear workflow instead of 10+ options to choose from
- Usability: IDs are no longer wasted context - they're actionable

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: Move internal docs to private directory

Moved POSTMORTEM and planning docs to ./private to exclude from PR reviews.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Remove experimental contextualize endpoint

- Removed contextualize MCP tool from search-server (saves ~4KB)
- Disabled FTS5 fallback paths in SessionSearch (now vector-first)
- Cleaned up CLAUDE.md documentation
- Removed contextualize-rewrite-plan.md doc

Rationale:
- Contextualize is better suited as a skill (LLM-powered) than an endpoint
- Search API already provides vector search with configurable limits
- Created issue #132 to track future contextualize skill implementation

Changes:
- src/servers/search-server.ts: Removed contextualize tool definition
- src/services/sqlite/SessionSearch.ts: Disabled FTS5 fallback, added deprecation warnings
- CLAUDE.md: Cleaned up outdated skill documentation
- docs/: Removed contextualize plan document

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Complete FTS5 cleanup - remove all deprecated search code

This completes the FTS5 cleanup work by removing all commented-out
FTS5 search code while preserving database tables for backward compatibility.

Changes:
- Removed 200+ lines of commented FTS5 search code from SessionSearch.ts
- Removed deprecated degraded_search_query__when_uvx_unavailable method
- Updated all method documentation to clarify vector-first architecture
- Updated class documentation to reflect filter-only query support
- Updated CLAUDE.md to remove FTS5 search references
- Clarified that FTS5 tables exist for backward compatibility only
- Updated "Why SQLite FTS5" section to "Why Vector-First Search"

Database impact: NONE - FTS5 tables remain intact for existing installations

Search architecture:
- ChromaDB: All text-based vector search queries
- SQLite: Filter-only queries (date ranges, metadata, no query text)
- FTS5 tables: Maintained but unused (backward compatibility)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Remove all FTS5 fallback execution code from search-server

Completes the FTS5 cleanup by removing all fallback execution paths
that attempted to use FTS5 when ChromaDB was unavailable.

Changes:
- Removed all FTS5 fallback code execution paths
- When ChromaDB fails or is unavailable, return empty results with helpful error messages
- Updated all deprecated tool descriptions (search_observations, search_sessions, search_user_prompts)
- Changed error messages to indicate FTS5 fallback has been removed
- Added installation instructions for UVX/Python when vector search is unavailable
- Updated comments from "hybrid search" to "vector-first search"
- Removed ~100 lines of dead FTS5 fallback code

Database impact: NONE - FTS5 tables remain intact (backward compatibility)

Search behavior when ChromaDB unavailable:
- Text queries: Return empty results with error explaining ChromaDB is required
- Filter-only queries (no text): Continue to work via direct SQLite

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Address PR 133 review feedback

Critical fixes:
- Remove contextualize endpoint from worker-service (route + handler)
- Fix build script logging to show correct .cjs extension (was .mjs)

Documentation improvements:
- Add comprehensive FTS5 retention rationale documentation
- Include v7.0.0 removal TODO for future cleanup

Testing:
- Build succeeds with correct output logging
- Worker restarts successfully (30th restart)
- Contextualize endpoint properly removed (404 response)
- Search endpoint verified working

This addresses all critical review feedback from PR 133.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-21 18:59:23 -05:00
Alex Newman 6cd904a81c chore: Bump version to 6.0.9
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 15:17:23 -05:00
Alex Newman 74c8afd0e0 feat: Add real-time queue depth indicator to viewer UI
Implements a visual badge that displays the count of active work items (queued + currently processing) in the worker service. The badge appears next to the claude-mem logo and updates in real-time via SSE.

Features:
- Shows count of pending messages + active SDK generators
- Only displays when queueDepth > 0
- Subtle pulse animation for visual feedback
- Theme-aware styling

Backend changes:
- Added getTotalActiveWork() method to SessionManager
- Updated worker-service to broadcast queueDepth via SSE
- Enhanced processing status API endpoint

Frontend changes:
- Updated Header component to display queue bubble
- Enhanced useSSE hook to track queueDepth state
- Added CSS styling with pulse animation

Closes #122
Closes #96
Closes #97

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 15:11:37 -05:00
Alex Newman d175f1759c chore: Bump version to 6.0.8
Critical hotfix for PM2 worker startup issue. The worker now correctly
starts from the marketplace directory automatically.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 14:42:16 -05:00
Alex Newman d55db524f0 Refactor worker scripts to improve logging and error handling
- Updated `new-hook.js` and `save-hook.js` to enhance logging functionality, including better error messages and structured logging.
- Improved the handling of worker startup in `summary-hook.js` and `worker-utils.ts`, ensuring proper directory context for PM2.
- Added checks for worker health and streamlined session management across hooks.
2025-11-17 14:38:47 -05:00
Alex Newman 7d44fdb289 Refactor worker-utils to use getPackageRoot for plugin path resolution 2025-11-17 14:12:50 -05:00
Alex Newman 812de2940d feat: Implement Endless Mode Token Economics Calculator
- Added a new script to simulate token savings from Endless Mode using real observation data.
- Introduced functions to calculate token costs with and without Endless Mode, showcasing the differences in context handling.
- Enhanced output to display detailed token usage statistics and potential savings at scale.
- Integrated assumptions for user activity to estimate weekly and annual token savings.
- Updated worker-utils to automatically start the worker using PM2 if not running, improving service reliability.
2025-11-17 14:04:47 -05:00
Alex Newman 047914d087 chore: Bump version to 6.0.7
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 13:46:12 -05:00
Alex Newman bdf79a439b fix: Change discovery_tokens migration from version 7 to 11
Root cause: The ensureDiscoveryTokensColumn migration was using version 7,
which was already taken by removeSessionSummariesUniqueConstraint. This
duplicate version number caused migration tracking issues in some databases.

Changes:
- Updated migration version from 7 to 11 (next available)
- Added schema_versions check to prevent unnecessary re-runs
- Updated comments to clarify the version number conflict
- Added error propagation (already present, but now more reliable)

This resolves issue #121 where users were seeing "no such column: discovery_tokens"
errors after upgrading to v6.0.6.

Affected users can manually add the columns:
  ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0;
  ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0;

Or wait for v6.0.7 release which includes this fix.

Fixes #121

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 13:43:34 -05:00
Alex Newman 798dec972e chore: Bump version to 6.0.6
Critical bugfix for database migration issue (Issue #121)

Changes:
- Fix migration logic to always verify column existence
- Remove early return that trusted schema_versions alone
- Ensures discovery_tokens columns exist before queries
- Prevents "no such column" errors for all users

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 13:18:49 -05:00
Alex Newman cf1c966409 chore: Bump version to 6.0.5
Automatic cleanup of orphaned MCP server processes on worker startup
Removed manual cleanup notice from session context
Self-healing maintenance on every worker restart

Generated with Claude Code
2025-11-16 22:39:06 -05:00
Alex Newman 02fef487e7 feat: add cleanup for orphaned MCP server processes on startup
- Implemented a new method `cleanupOrphanedProcesses` to identify and terminate orphaned `uvx` processes from previous sessions.
- Integrated the cleanup method into the `start` process of the WorkerService to ensure a clean environment at startup.
- Added logging for process cleanup actions and handled potential errors gracefully without failing the service startup.
2025-11-16 22:36:39 -05:00
Alex Newman 4f1cd309fd chore: Bump version to 6.0.4
Fix memory leaks from orphaned uvx/python processes

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-16 22:20:57 -05:00
Copilot c46e4a341a Fix memory leaks from orphaned uvx/python processes (#120)
This fixes memory leak, will remove one unnecessary MCP after this in a new PR but this is mission critical fix

* Initial plan

* Fix memory leaks: Add proper cleanup for ChromaSync and search server processes

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Add comprehensive process cleanup and PM2 configuration improvements

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Add comprehensive summary and recommendations for memory leak fixes

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-16 22:16:41 -05:00
Alex Newman 60d5f8fbf1 chore: Bump version to 6.0.3
Version bump for patch release.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-16 13:37:30 -05:00
Alex Newman c0778bef00 docs: Align search documentation with hybrid ChromaDB architecture (#116)
* feat: Add discovery_tokens for ROI tracking in observations and session summaries

- Introduced `discovery_tokens` column in `observations` and `session_summaries` tables to track token costs associated with discovering and creating each observation and summary.
- Updated relevant services and hooks to calculate and display ROI metrics based on discovery tokens.
- Enhanced context economics reporting to include savings from reusing previous observations.
- Implemented migration to ensure the new column is added to existing tables.
- Adjusted data models and sync processes to accommodate the new `discovery_tokens` field.

* refactor: streamline context hook by removing unused functions and updating terminology

- Removed the estimateTokens and getObservations helper functions as they were not utilized.
- Updated the legend and output messages to replace "discovery" with "work" for clarity.
- Changed the emoji representation for different observation types to better reflect their purpose.
- Enhanced output formatting for improved readability and understanding of token usage.

* Refactor user-message-hook and context-hook for improved clarity and functionality

- Updated user-message-hook.js to enhance error messaging and improve variable naming for clarity.
- Modified context-hook.ts to include a new column key section, improved context index instructions, and added emoji icons for observation types.
- Adjusted footer messages in context-hook.ts to emphasize token savings and access to past research.
- Changed user-message-hook.ts to update the feedback and support message for clarity.

* fix: Critical ROI tracking fixes from PR review

Addresses critical findings from PR #111 review:

1. **Fixed incorrect discovery token calculation** (src/services/worker/SDKAgent.ts)
   - Changed from passing cumulative total to per-response delta
   - Now correctly tracks token cost for each observation/summary
   - Captures token state before/after response processing
   - Prevents all observations getting inflated cumulative values

2. **Fixed schema version mismatch** (src/services/sqlite/SessionStore.ts)
   - Changed ensureDiscoveryTokensColumn() from version 11 to version 7
   - Now matches migration007 definition in migrations.ts
   - Ensures consistent version tracking across migration system

These fixes ensure ROI metrics accurately reflect token costs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Update search documentation to reflect hybrid ChromaDB architecture

The backend correctly implements ChromaDB-first semantic search with SQLite
temporal ordering and FTS5 fallback, but documentation incorrectly described
it as "FTS5 full-text search". This fix aligns all skill guides and tool
descriptions with the actual implementation.

Changes:
- Update SKILL.md to describe hybrid architecture with ChromaDB primary
- Update observations.md title and query parameter descriptions
- Update all three search tool descriptions in search-server.ts:
  * search_observations
  * search_sessions
  * search_user_prompts

All tools now correctly document:
- ChromaDB semantic search (primary ranking)
- 90-day recency filter
- SQLite temporal ordering
- FTS5 fallback (when ChromaDB unavailable)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Add discovery_tokens column to observations and session_summaries tables

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-16 13:36:17 -05:00
Alex Newman 3cbc041c8b feat: Add ROI tracking with discovery_tokens for observations and session summaries (#111)
* feat: Add discovery_tokens for ROI tracking in observations and session summaries

- Introduced `discovery_tokens` column in `observations` and `session_summaries` tables to track token costs associated with discovering and creating each observation and summary.
- Updated relevant services and hooks to calculate and display ROI metrics based on discovery tokens.
- Enhanced context economics reporting to include savings from reusing previous observations.
- Implemented migration to ensure the new column is added to existing tables.
- Adjusted data models and sync processes to accommodate the new `discovery_tokens` field.

* refactor: streamline context hook by removing unused functions and updating terminology

- Removed the estimateTokens and getObservations helper functions as they were not utilized.
- Updated the legend and output messages to replace "discovery" with "work" for clarity.
- Changed the emoji representation for different observation types to better reflect their purpose.
- Enhanced output formatting for improved readability and understanding of token usage.

* Refactor user-message-hook and context-hook for improved clarity and functionality

- Updated user-message-hook.js to enhance error messaging and improve variable naming for clarity.
- Modified context-hook.ts to include a new column key section, improved context index instructions, and added emoji icons for observation types.
- Adjusted footer messages in context-hook.ts to emphasize token savings and access to past research.
- Changed user-message-hook.ts to update the feedback and support message for clarity.

* fix: Critical ROI tracking fixes from PR review

Addresses critical findings from PR #111 review:

1. **Fixed incorrect discovery token calculation** (src/services/worker/SDKAgent.ts)
   - Changed from passing cumulative total to per-response delta
   - Now correctly tracks token cost for each observation/summary
   - Captures token state before/after response processing
   - Prevents all observations getting inflated cumulative values

2. **Fixed schema version mismatch** (src/services/sqlite/SessionStore.ts)
   - Changed ensureDiscoveryTokensColumn() from version 11 to version 7
   - Now matches migration007 definition in migrations.ts
   - Ensures consistent version tracking across migration system

These fixes ensure ROI metrics accurately reflect token costs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-15 19:34:53 -05:00
Alex Newman 9fb7383ab3 chore: bump version to 6.0.2
Updated user message hook with Claude-Mem community discussion link.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 15:44:02 -05:00
Alex Newman e3283c2a1d chore: bump version to 6.0.1 2025-11-14 15:06:11 -05:00
Alex Newman 581e940659 Add new SVG icons for "learned" and "next steps" features (#109)
- Introduced icon-thin-learned.svg with detailed path definitions and color styling.
- Added icon-thin-next-steps.svg featuring a unique design and color scheme.
2025-11-14 15:04:29 -05:00
Alex Newman 68290a9121 Performance improvements: Token reduction and enhanced summaries (#101)
* refactor: Reduce continuation prompt token usage by 95 lines

Removed redundant instructions from continuation prompt that were originally
added to mitigate a session continuity issue. That issue has since been
resolved, making these detailed instructions unnecessary on every continuation.

Changes:
- Reduced continuation prompt from ~106 lines to ~11 lines (~95 line reduction)
- Changed "User's Goal:" to "Next Prompt in Session:" (more accurate framing)
- Removed redundant WHAT TO RECORD, WHEN TO SKIP, and OUTPUT FORMAT sections
- Kept concise reminder: "Continue generating observations and progress summaries..."
- Initial prompt still contains all detailed instructions

Impact:
- Significant token savings on every continuation prompt
- Faster context injection with no loss of functionality
- Instructions remain comprehensive in initial prompt

Files modified:
- src/sdk/prompts.ts (buildContinuationPrompt function)
- plugin/scripts/worker-service.cjs (compiled output)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Enhance observation and summary prompts for clarity and token efficiency

* Enhance prompt clarity and instructions in prompts.ts

- Added a reminder to think about instructions before starting work.
- Simplified the continuation prompt instruction by removing "for this ongoing session."

* feat: Enhance settings.json with permissions and deny access to sensitive files

refactor: Remove PLAN-full-observation-display.md and PR_SUMMARY.md as they are no longer needed

chore: Delete SECURITY_SUMMARY.md since it is redundant after recent changes

fix: Update worker-service.cjs to streamline observation generation instructions

cleanup: Remove src-analysis.md and src-tree.md for a cleaner codebase

refactor: Modify prompts.ts to clarify instructions for memory processing

* refactor: Remove legacy worker service implementation

* feat: Enhance summary hook to extract last assistant message and improve logging

- Added function to extract the last assistant message from the transcript.
- Updated summary hook to include last assistant message in the summary request.
- Modified SDKSession interface to store last assistant message.
- Adjusted buildSummaryPrompt to utilize last assistant message for generating summaries.
- Updated worker service and session manager to handle last assistant message in summarize requests.
- Introduced silentDebug utility for improved logging and diagnostics throughout the summary process.

* docs: Add comprehensive implementation plan for ROI metrics feature

Added detailed implementation plan covering:
- Token usage capture from Agent SDK
- Database schema changes (migration #8)
- Discovery cost tracking per observation
- Context hook display with ROI metrics
- Testing and rollout strategy

Timeline: ~20 hours over 4 days
Goal: Empirical data for YC application amendment

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: Add transcript processing scripts for analysis and formatting

- Implemented `dump-transcript-readable.ts` to generate a readable markdown dump of transcripts, excluding certain entry types.
- Created `extract-rich-context-examples.ts` to extract and showcase rich context examples from transcripts, highlighting user requests and assistant reasoning.
- Developed `format-transcript-context.ts` to format transcript context into a structured markdown format for improved observation generation.
- Added `test-transcript-parser.ts` for validating data extraction from transcript JSONL files, including statistics and error reporting.
- Introduced `transcript-to-markdown.ts` for a complete representation of transcript data in markdown format, showing all context data.
- Enhanced type definitions in `transcript.ts` to support new features and ensure type safety.
- Built `transcript-parser.ts` to handle parsing of transcript JSONL files, including error handling and data extraction methods.

* Refactor hooks and SDKAgent for improved observation handling

- Updated `new-hook.ts` to clean user prompts by stripping leading slashes for better semantic clarity.
- Enhanced `save-hook.ts` to include additional tools in the SKIP_TOOLS set, preventing unnecessary observations from certain command invocations.
- Modified `prompts.ts` to change the structure of observation prompts, emphasizing the observational role and providing a detailed XML output format for observations.
- Adjusted `SDKAgent.ts` to enforce stricter tool usage restrictions, ensuring the memory agent operates solely as an observer without any tool access.

* feat: Enhance session initialization to accept user prompts and prompt numbers

- Updated `handleSessionInit` in `worker-service.ts` to extract `userPrompt` and `promptNumber` from the request body and pass them to `initializeSession`.
- Modified `initializeSession` in `SessionManager.ts` to handle optional `currentUserPrompt` and `promptNumber` parameters.
- Added logic to update the existing session's `userPrompt` and `lastPromptNumber` if a `currentUserPrompt` is provided.
- Implemented debug logging for session initialization and updates to track user prompts and prompt numbers.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-13 18:22:44 -05:00
Alex Newman 5f7aa0710e Bump version to 5.5.1
This patch release includes:
- Summary hook enhancements to capture last user message from transcripts
- Activity indicator improvements for better user feedback
- Worker service enhancements for queue depth tracking

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-11 17:41:22 -05:00
Alex Newman 39fedfc5fc feat: Enhance summary hook to include last user message from transcript (#95)
* feat: Enhance summary hook to include last user message from transcript

- Added functionality to extract the last user message from a JSONL transcript file in the summary hook.
- Updated the summary hook to send the last user message along with the summary request.
- Modified the SDKSession interface to include an optional last_user_message field.
- Updated the summary prompt to incorporate the last user message in the output format.
- Refactored worker service to handle the last user message in the summarize queue.
- Enhanced session manager to track and broadcast processing status based on active sessions and queue depth.
- Improved error handling and logging for better traceability during transcript reading and processing.

* feat(worker): enhance processing status broadcasting and session management

- Added immediate broadcasting of processing status when a prompt is received.
- Implemented logging for generator completion in multiple locations.
- Updated `broadcastProcessingStatus` to include queue depth and active session count in logs.
- Modified session iterator to stop yielding messages after a summary is yielded, with appropriate logging.
2025-11-11 17:38:22 -05:00
Alex Newman 6ddef1093a Release v5.5.0: Enhanced mem-search skill
Breaking Changes: None (minor version)

Improvements:
- Merged PR #91: Replace generic "search" skill with enhanced "mem-search" skill
- Improved skill effectiveness from 67% to 100% (Anthropic standards)
- Enhanced scope differentiation to prevent confusion with native conversation memory
- Increased concrete triggers from 44% to 85%
- Added 5+ unique identifiers and explicit exclusion patterns
- Comprehensive documentation reorganization (17 total files)

Technical Changes:
- New mem-search skill with system-specific naming
- Explicit temporal keywords ("previous sessions", "weeks/months ago")
- Technical anchors referencing FTS5 full-text index and typed observations
- Documentation moved from /context/ to /docs/context/
- Detailed technical architecture documentation added
- 12 operation guides + 2 principle directories

Credits:
- Skill design and enhancement by @basher83

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-11 16:18:42 -05:00
basher83 97d565e3cd Replace search skill with mem-search (#91)
* feat: add mem-search skill with progressive disclosure architecture

Add comprehensive mem-search skill for accessing claude-mem's persistent
cross-session memory database. Implements progressive disclosure workflow
and token-efficient search patterns.

Features:
- 12 search operations (observations, sessions, prompts, by-type, by-concept, by-file, timelines, etc.)
- Progressive disclosure principles to minimize token usage
- Anti-patterns documentation to guide LLM behavior
- HTTP API integration for all search functionality
- Common workflows with composition examples

Structure:
- SKILL.md: Entry point with temporal trigger patterns
- principles/: Progressive disclosure + anti-patterns
- operations/: 12 search operation files

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* docs: add CHANGELOG entry for mem-search skill

Document mem-search skill addition in Unreleased section with:
- 100% effectiveness compliance metrics
- Comparison to previous search skill implementation
- Progressive disclosure architecture details
- Reference to audit report documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* docs: add mem-search skill audit report

Add comprehensive audit report validating mem-search skill against
Anthropic's official skill-creator documentation.

Report includes:
- Effectiveness metrics comparison (search vs mem-search)
- Critical issues analysis for production readiness
- Compliance validation across 6 key dimensions
- Reference implementation guidance

Result: mem-search achieves 100% compliance vs search's 67%

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: Add comprehensive search architecture analysis document

- Document current state of dual search architectures (HTTP API and MCP)
- Analyze HTTP endpoints and MCP search server architectures
- Identify DRY violations across search implementations
- Evaluate the use of curl as the optimal approach for search
- Provide architectural recommendations for immediate and long-term improvements
- Outline action plan for cleanup, feature parity, DRY refactoring

* refactor: Remove deprecated search skill documentation and operations

* refactor: Reorganize documentation into public and context directories

Changes:
- Created docs/public/ for Mintlify documentation (.mdx files)
- Created docs/context/ for internal planning and implementation docs
- Moved all .mdx files and assets to docs/public/
- Moved all internal .md files to docs/context/
- Added CLAUDE.md to both directories explaining their purpose
- Updated docs.json paths to work with new structure

Benefits:
- Clear separation between user-facing and internal documentation
- Easier to maintain Mintlify docs in dedicated directory
- Internal context files organized separately

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Enhance session management and continuity in hooks

- Updated new-hook.ts to clarify session_id threading and idempotent session creation.
- Modified prompts.ts to require claudeSessionId for continuation prompts, ensuring session context is maintained.
- Improved SessionStore.ts documentation on createSDKSession to emphasize idempotent behavior and session connection.
- Refined SDKAgent.ts to detail continuation prompt logic and its reliance on session.claudeSessionId for unified session handling.

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Alex Newman <thedotmack@gmail.com>
2025-11-11 16:15:07 -05:00
Alex Newman eafdd6a7be Bump version to 5.4.5 2025-11-11 13:51:19 -05:00
Alex Newman 3529f9274b feat: Enhanced logging and SDK prompt improvements (#94)
* Initial plan

* Initial analysis: Found root cause of double entries bug

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Fix double entries by assigning generatorPromise in handleSessionInit

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* feat(logging): Enhance HTTP request logging and session management

- Added middleware for logging HTTP requests and responses, excluding static assets and health checks.
- Introduced a method to summarize request bodies for specific endpoints.
- Improved logging for user prompt synchronization with Chroma, including duration tracking.
- Enhanced session initialization logging to include additional session details.
- Updated observation and summary logging to provide more context and error handling during Chroma synchronization.
- Refactored tool name formatting for logging in the SessionManager.
- Expanded logger component types to include 'HTTP', 'SESSION', and 'CHROMA'.

* Refactor SDK prompts and logging for improved clarity and functionality

- Updated buildInitPrompt to clarify the observer's role and what to record.
- Enhanced buildSummaryPrompt with clearer instructions for summarizing ongoing sessions.
- Improved buildContinuationPrompt to emphasize the focus on deliverables and capabilities.
- Refactored WorkerService to utilize a centralized tool formatting function for logging.
- Added truncation for logged responses and observations to improve readability.
- Updated SessionManager to log the queuing of summarize actions with session details.
- Enhanced App and Sidebar components to support refreshing stats on sidebar open.
- Refactored useStats hook to allow manual refreshing of stats while maintaining automatic loading on mount.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-11 13:49:00 -05:00
Alex Newman 30ebe92a53 Release v5.4.4: Fix duplicate entries in viewer
Bugfix:
- Fixed duplicate observations and summaries appearing in viewer
- Root cause: handleSessionInit spawned SDK agent but didn't save promise to session.generatorPromise
- Second agent would spawn when handleObservations ran, causing duplicates
- Fix: Assign generatorPromise in handleSessionInit (matches handleSummarize pattern)

Technical changes:
- Modified src/services/worker-service.ts:265
- Now tracks promise to prevent duplicate agent spawning
- Guard condition in handleObservations (line 301) now works correctly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-10 17:30:58 -05:00
Copilot 1bb203cbb5 Fix duplicate entries in viewer caused by untracked SDK agent promise (#86)
* Initial plan

* Initial analysis: Found root cause of double entries bug

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Fix double entries by assigning generatorPromise in handleSessionInit

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
Co-authored-by: Alex Newman <thedotmack@gmail.com>
2025-11-10 17:30:02 -05:00
Alex Newman 9f8499fe54 Bump version to 5.4.3
Release v5.4.3: Fix PM2 race condition in worker restart

Changes:
- Removed PM2 restart logic from ensureWorkerRunning()
- PM2 watch mode now exclusively handles worker restarts
- Eliminated race condition causing TypeError
- Reduced unnecessary worker restarts (39+ → minimal)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-10 16:53:23 -05:00
Alex Newman cda12d95c9 Merge main into copilot/fix-pm2-race-condition
Resolved conflicts in built files by rebuilding from source
2025-11-10 16:51:26 -05:00
Alex Newman bbd6113f69 Release v5.4.2: CWD spatial awareness fix
Bugfix release:
- Fixed SDK agent spatial awareness by propagating CWD context from tool execution
- Prevents false "file not found" reports when working across multiple repositories
- Added comprehensive test suite for CWD propagation (8 passing tests)
- Security analysis: Zero vulnerabilities, CodeQL approved

Technical changes:
- Hook extracts CWD from PostToolUseInput and forwards to worker service
- Worker types define optional CWD fields in PendingMessage and ObservationData
- SessionManager passes CWD to SDKAgent's addObservation
- SDK agent includes CWD in tool observation objects sent to Claude API
- Prompts render tool_cwd XML elements with spatial awareness guidance
- All changes present in compiled outputs (save-hook.js, worker-service.cjs)

Documentation:
- Technical documentation: context/CWD_CONTEXT_FIX.md
- PR summary: PR_SUMMARY.md
- Security review: SECURITY_SUMMARY.md
- CHANGELOG entry added

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-10 15:33:14 -05:00
copilot-swe-agent[bot] 730c420a13 Add CWD propagation through hook, worker, and SDK agent
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-10 19:23:06 +00:00
copilot-swe-agent[bot] a60ef6eacb Remove PM2 restart logic from ensureWorkerRunning to fix race condition
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-10 19:05:17 +00:00
Alex Newman b4b90faa1e Release v5.4.1: MCP search server toggle functionality
New Features:
- REST API endpoints for MCP server status and toggle control
- UI toggle in viewer sidebar for enabling/disabling MCP search server
- File-based persistence mechanism (.mcp.json ↔ .mcp.json.disabled)
- Independent state management for MCP toggle

Technical changes:
- Added GET /api/mcp/status endpoint (returns mcpEnabled boolean)
- Added POST /api/mcp/toggle endpoint (toggles MCP server state)
- Updated Settings interface with mcpEnabled property
- Enhanced Sidebar component with MCP toggle UI and state management
- Updated worker service with MCP control logic
- Bumped version to 5.4.1 in all metadata files

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-10 13:48:52 -05:00
Alex Newman 9e235b5b57 feat: Add MCP search server toggle with dedicated API architecture (#85)
* feat: add MCP search server toggle functionality

- Introduced `CLAUDE_MEM_MCP_ENABLED` setting to manage the MCP search server state.
- Updated `WorkerService` to handle MCP enabling/disabling based on the new setting.
- Enhanced `Sidebar` component to include a checkbox for toggling MCP search server.
- Modified `useSettings` hook to incorporate the new MCP setting.
- Updated default settings to include `CLAUDE_MEM_MCP_ENABLED` with a default value of true.
- Adjusted TypeScript types to include the new MCP setting.

* feat: Implement MCP toggle functionality in WorkerService and Sidebar

- Added API endpoints for MCP status retrieval and toggling in WorkerService.
- Updated Sidebar component to manage MCP toggle state and display status messages.
- Removed MCP_ENABLED from settings state management and default settings.
- Adjusted settings interface and related hooks to reflect the removal of MCP_ENABLED.
2025-11-10 13:47:25 -05:00
Alex Newman a367436a78 Release v5.4.0: Skill-Based Search Migration & Progressive Disclosure
Version Bump:
- package.json: 5.3.0 → 5.4.0
- .claude-plugin/marketplace.json: 5.3.0 → 5.4.0
- plugin/.claude-plugin/plugin.json: 5.3.0 → 5.4.0
- CLAUDE.md: 5.3.0 → 5.4.0

Breaking Changes:
- MCP search tools removed (replaced with skill-based search)
- No user action required - migration is transparent

Major Features:
- Skill-based search architecture (~2,250 token savings per session)
- Progressive disclosure pattern (skill frontmatter ~250 tokens)
- 10 HTTP API endpoints for search operations
- Natural language queries replace MCP tool syntax

Technical Changes:
- Removed MCP search server registration
- Added search skill with comprehensive documentation
- Updated all documentation to reflect new architecture
- HTTP API endpoints in worker service

Migration Notes:
- Users: No action required, searches work identically
- Developers: MCP server source kept for reference

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-09 19:15:04 -05:00