Compare commits

...

83 Commits

Author SHA1 Message Date
Alex Newman 60d5f8fbf1 chore: Bump version to 6.0.3
Version bump for patch release.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-16 13:37:30 -05:00
Alex Newman c0778bef00 docs: Align search documentation with hybrid ChromaDB architecture (#116)
* feat: Add discovery_tokens for ROI tracking in observations and session summaries

- Introduced `discovery_tokens` column in `observations` and `session_summaries` tables to track token costs associated with discovering and creating each observation and summary.
- Updated relevant services and hooks to calculate and display ROI metrics based on discovery tokens.
- Enhanced context economics reporting to include savings from reusing previous observations.
- Implemented migration to ensure the new column is added to existing tables.
- Adjusted data models and sync processes to accommodate the new `discovery_tokens` field.

* refactor: streamline context hook by removing unused functions and updating terminology

- Removed the estimateTokens and getObservations helper functions as they were not utilized.
- Updated the legend and output messages to replace "discovery" with "work" for clarity.
- Changed the emoji representation for different observation types to better reflect their purpose.
- Enhanced output formatting for improved readability and understanding of token usage.

* Refactor user-message-hook and context-hook for improved clarity and functionality

- Updated user-message-hook.js to enhance error messaging and improve variable naming for clarity.
- Modified context-hook.ts to include a new column key section, improved context index instructions, and added emoji icons for observation types.
- Adjusted footer messages in context-hook.ts to emphasize token savings and access to past research.
- Changed user-message-hook.ts to update the feedback and support message for clarity.

* fix: Critical ROI tracking fixes from PR review

Addresses critical findings from PR #111 review:

1. **Fixed incorrect discovery token calculation** (src/services/worker/SDKAgent.ts)
   - Changed from passing cumulative total to per-response delta
   - Now correctly tracks token cost for each observation/summary
   - Captures token state before/after response processing
   - Prevents all observations getting inflated cumulative values

2. **Fixed schema version mismatch** (src/services/sqlite/SessionStore.ts)
   - Changed ensureDiscoveryTokensColumn() from version 11 to version 7
   - Now matches migration007 definition in migrations.ts
   - Ensures consistent version tracking across migration system

These fixes ensure ROI metrics accurately reflect token costs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Update search documentation to reflect hybrid ChromaDB architecture

The backend correctly implements ChromaDB-first semantic search with SQLite
temporal ordering and FTS5 fallback, but documentation incorrectly described
it as "FTS5 full-text search". This fix aligns all skill guides and tool
descriptions with the actual implementation.

Changes:
- Update SKILL.md to describe hybrid architecture with ChromaDB primary
- Update observations.md title and query parameter descriptions
- Update all three search tool descriptions in search-server.ts:
  * search_observations
  * search_sessions
  * search_user_prompts

All tools now correctly document:
- ChromaDB semantic search (primary ranking)
- 90-day recency filter
- SQLite temporal ordering
- FTS5 fallback (when ChromaDB unavailable)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Add discovery_tokens column to observations and session_summaries tables

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-16 13:36:17 -05:00
Alex Newman 3cbc041c8b feat: Add ROI tracking with discovery_tokens for observations and session summaries (#111)
* feat: Add discovery_tokens for ROI tracking in observations and session summaries

- Introduced `discovery_tokens` column in `observations` and `session_summaries` tables to track token costs associated with discovering and creating each observation and summary.
- Updated relevant services and hooks to calculate and display ROI metrics based on discovery tokens.
- Enhanced context economics reporting to include savings from reusing previous observations.
- Implemented migration to ensure the new column is added to existing tables.
- Adjusted data models and sync processes to accommodate the new `discovery_tokens` field.

* refactor: streamline context hook by removing unused functions and updating terminology

- Removed the estimateTokens and getObservations helper functions as they were not utilized.
- Updated the legend and output messages to replace "discovery" with "work" for clarity.
- Changed the emoji representation for different observation types to better reflect their purpose.
- Enhanced output formatting for improved readability and understanding of token usage.

* Refactor user-message-hook and context-hook for improved clarity and functionality

- Updated user-message-hook.js to enhance error messaging and improve variable naming for clarity.
- Modified context-hook.ts to include a new column key section, improved context index instructions, and added emoji icons for observation types.
- Adjusted footer messages in context-hook.ts to emphasize token savings and access to past research.
- Changed user-message-hook.ts to update the feedback and support message for clarity.

* fix: Critical ROI tracking fixes from PR review

Addresses critical findings from PR #111 review:

1. **Fixed incorrect discovery token calculation** (src/services/worker/SDKAgent.ts)
   - Changed from passing cumulative total to per-response delta
   - Now correctly tracks token cost for each observation/summary
   - Captures token state before/after response processing
   - Prevents all observations getting inflated cumulative values

2. **Fixed schema version mismatch** (src/services/sqlite/SessionStore.ts)
   - Changed ensureDiscoveryTokensColumn() from version 11 to version 7
   - Now matches migration007 definition in migrations.ts
   - Ensures consistent version tracking across migration system

These fixes ensure ROI metrics accurately reflect token costs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-15 19:34:53 -05:00
Copilot 0f96476987 Fix documentation links to point to docs.claude-mem.ai (#114)
* Initial plan

* Fix documentation links to point to docs.claude-mem.ai

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-15 16:29:11 -05:00
Alex Newman cd6f883020 docs: update CHANGELOG.md for v6.0.2 2025-11-14 15:44:46 -05:00
Alex Newman 9fb7383ab3 chore: bump version to 6.0.2
Updated user message hook with Claude-Mem community discussion link.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 15:44:02 -05:00
Alex Newman e8e7fc81af docs: update CHANGELOG.md for v6.0.1 2025-11-14 15:06:54 -05:00
Alex Newman e3283c2a1d chore: bump version to 6.0.1 2025-11-14 15:06:11 -05:00
Alex Newman 581e940659 Add new SVG icons for "learned" and "next steps" features (#109)
- Introduced icon-thin-learned.svg with detailed path definitions and color styling.
- Added icon-thin-next-steps.svg featuring a unique design and color scheme.
2025-11-14 15:04:29 -05:00
Alex Newman 915dbc1aa9 fix: restore jsx option in tsconfig.json 2025-11-14 13:06:49 -05:00
Alex Newman 5a84198529 fix: correct image paths to use absolute GitHub URLs
- Changed relative paths (docs/) to absolute GitHub raw URLs
- Fixes broken social sharing images
- Images now load correctly on GitHub and external sites

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-13 21:06:30 -05:00
Alex Newman f5b25e8fc4 docs: update user-facing documentation for v6.0.0
- Updated version badge in README.md to 6.0.0
- Updated 'What's New' sections in README.md and introduction.mdx
- Highlighted major session management and transcript processing improvements
- Removed restrictive permissions from .claude/settings.json

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-13 20:53:59 -05:00
Alex Newman 7d1e6af5c5 docs: update CHANGELOG.md for v6.0.0
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-13 18:26:02 -05:00
Alex Newman 8b4cc4f6bf chore: bump version to 6.0.0
Major version bump reflecting significant improvements:
- Enhanced session initialization with live userPrompt updates
- Improved transcript processing and analysis capabilities
- Refactored hooks and SDKAgent for better observation handling

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-13 18:24:59 -05:00
Alex Newman 68290a9121 Performance improvements: Token reduction and enhanced summaries (#101)
* refactor: Reduce continuation prompt token usage by 95 lines

Removed redundant instructions from continuation prompt that were originally
added to mitigate a session continuity issue. That issue has since been
resolved, making these detailed instructions unnecessary on every continuation.

Changes:
- Reduced continuation prompt from ~106 lines to ~11 lines (~95 line reduction)
- Changed "User's Goal:" to "Next Prompt in Session:" (more accurate framing)
- Removed redundant WHAT TO RECORD, WHEN TO SKIP, and OUTPUT FORMAT sections
- Kept concise reminder: "Continue generating observations and progress summaries..."
- Initial prompt still contains all detailed instructions

Impact:
- Significant token savings on every continuation prompt
- Faster context injection with no loss of functionality
- Instructions remain comprehensive in initial prompt

Files modified:
- src/sdk/prompts.ts (buildContinuationPrompt function)
- plugin/scripts/worker-service.cjs (compiled output)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Enhance observation and summary prompts for clarity and token efficiency

* Enhance prompt clarity and instructions in prompts.ts

- Added a reminder to think about instructions before starting work.
- Simplified the continuation prompt instruction by removing "for this ongoing session."

* feat: Enhance settings.json with permissions and deny access to sensitive files

refactor: Remove PLAN-full-observation-display.md and PR_SUMMARY.md as they are no longer needed

chore: Delete SECURITY_SUMMARY.md since it is redundant after recent changes

fix: Update worker-service.cjs to streamline observation generation instructions

cleanup: Remove src-analysis.md and src-tree.md for a cleaner codebase

refactor: Modify prompts.ts to clarify instructions for memory processing

* refactor: Remove legacy worker service implementation

* feat: Enhance summary hook to extract last assistant message and improve logging

- Added function to extract the last assistant message from the transcript.
- Updated summary hook to include last assistant message in the summary request.
- Modified SDKSession interface to store last assistant message.
- Adjusted buildSummaryPrompt to utilize last assistant message for generating summaries.
- Updated worker service and session manager to handle last assistant message in summarize requests.
- Introduced silentDebug utility for improved logging and diagnostics throughout the summary process.

* docs: Add comprehensive implementation plan for ROI metrics feature

Added detailed implementation plan covering:
- Token usage capture from Agent SDK
- Database schema changes (migration #8)
- Discovery cost tracking per observation
- Context hook display with ROI metrics
- Testing and rollout strategy

Timeline: ~20 hours over 4 days
Goal: Empirical data for YC application amendment

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: Add transcript processing scripts for analysis and formatting

- Implemented `dump-transcript-readable.ts` to generate a readable markdown dump of transcripts, excluding certain entry types.
- Created `extract-rich-context-examples.ts` to extract and showcase rich context examples from transcripts, highlighting user requests and assistant reasoning.
- Developed `format-transcript-context.ts` to format transcript context into a structured markdown format for improved observation generation.
- Added `test-transcript-parser.ts` for validating data extraction from transcript JSONL files, including statistics and error reporting.
- Introduced `transcript-to-markdown.ts` for a complete representation of transcript data in markdown format, showing all context data.
- Enhanced type definitions in `transcript.ts` to support new features and ensure type safety.
- Built `transcript-parser.ts` to handle parsing of transcript JSONL files, including error handling and data extraction methods.

* Refactor hooks and SDKAgent for improved observation handling

- Updated `new-hook.ts` to clean user prompts by stripping leading slashes for better semantic clarity.
- Enhanced `save-hook.ts` to include additional tools in the SKIP_TOOLS set, preventing unnecessary observations from certain command invocations.
- Modified `prompts.ts` to change the structure of observation prompts, emphasizing the observational role and providing a detailed XML output format for observations.
- Adjusted `SDKAgent.ts` to enforce stricter tool usage restrictions, ensuring the memory agent operates solely as an observer without any tool access.

* feat: Enhance session initialization to accept user prompts and prompt numbers

- Updated `handleSessionInit` in `worker-service.ts` to extract `userPrompt` and `promptNumber` from the request body and pass them to `initializeSession`.
- Modified `initializeSession` in `SessionManager.ts` to handle optional `currentUserPrompt` and `promptNumber` parameters.
- Added logic to update the existing session's `userPrompt` and `lastPromptNumber` if a `currentUserPrompt` is provided.
- Implemented debug logging for session initialization and updates to track user prompts and prompt numbers.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-13 18:22:44 -05:00
Alex Newman ab5d78717f Merge remote-tracking branch 'refs/remotes/origin/main' 2025-11-12 15:59:02 -05:00
Alex Newman cb4aea57a8 feat: Add comprehensive documentation for Language Model Tool API and related resources 2025-11-12 15:58:44 -05:00
Copilot 7bdf6dbfe1 Align user-facing documentation with v5.5.1 codebase state (#99)
* Initial plan

* Update documentation to reflect v5.5.1 state and mem-search skill

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Update hooks documentation to clarify 6 hooks + pre-hook architecture

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Complete documentation alignment with mem-search skill naming

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Fix remaining old skill path references in troubleshooting docs

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Documentation alignment complete - all tests pass

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Fix hallucinated /skill command references - skills are auto-invoked

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-12 14:59:04 -05:00
Alex Newman 1c9da73d5f feat: Add VSCode Language Model Tool API documentation 2025-11-12 13:39:46 -05:00
Alex Newman 5f7aa0710e Bump version to 5.5.1
This patch release includes:
- Summary hook enhancements to capture last user message from transcripts
- Activity indicator improvements for better user feedback
- Worker service enhancements for queue depth tracking

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-11 17:41:22 -05:00
Alex Newman 39fedfc5fc feat: Enhance summary hook to include last user message from transcript (#95)
* feat: Enhance summary hook to include last user message from transcript

- Added functionality to extract the last user message from a JSONL transcript file in the summary hook.
- Updated the summary hook to send the last user message along with the summary request.
- Modified the SDKSession interface to include an optional last_user_message field.
- Updated the summary prompt to incorporate the last user message in the output format.
- Refactored worker service to handle the last user message in the summarize queue.
- Enhanced session manager to track and broadcast processing status based on active sessions and queue depth.
- Improved error handling and logging for better traceability during transcript reading and processing.

* feat(worker): enhance processing status broadcasting and session management

- Added immediate broadcasting of processing status when a prompt is received.
- Implemented logging for generator completion in multiple locations.
- Updated `broadcastProcessingStatus` to include queue depth and active session count in logs.
- Modified session iterator to stop yielding messages after a summary is yielded, with appropriate logging.
2025-11-11 17:38:22 -05:00
Alex Newman ecb8b39f6d Add auto-generated CHANGELOG from GitHub releases
New Features:
- Created scripts/generate-changelog.js to auto-generate CHANGELOG.md
- Fetches all GitHub releases and formats into Keep a Changelog format
- Added npm run changelog:generate command

Version-Bump Skill Updates:
- Added Step 10: Generate CHANGELOG to workflow
- Updated verification checklist to include CHANGELOG generation
- Updated skill description and critical rules
- Single source of truth: GitHub releases

Technical Details:
- Script fetches releases via gh CLI
- Parses release bodies and formats to markdown
- Removes duplicate headers and Claude Code footers
- Sorts releases by date (newest first)
- Generates clean, consistent changelog

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-11 16:26:10 -05:00
Alex Newman fe0902b48f Remove empty Unreleased section from CHANGELOG 2025-11-11 16:22:12 -05:00
Alex Newman 4ab9739e4c Update CHANGELOG.md for v5.5.0 release 2025-11-11 16:20:32 -05:00
Alex Newman 6ddef1093a Release v5.5.0: Enhanced mem-search skill
Breaking Changes: None (minor version)

Improvements:
- Merged PR #91: Replace generic "search" skill with enhanced "mem-search" skill
- Improved skill effectiveness from 67% to 100% (Anthropic standards)
- Enhanced scope differentiation to prevent confusion with native conversation memory
- Increased concrete triggers from 44% to 85%
- Added 5+ unique identifiers and explicit exclusion patterns
- Comprehensive documentation reorganization (17 total files)

Technical Changes:
- New mem-search skill with system-specific naming
- Explicit temporal keywords ("previous sessions", "weeks/months ago")
- Technical anchors referencing FTS5 full-text index and typed observations
- Documentation moved from /context/ to /docs/context/
- Detailed technical architecture documentation added
- 12 operation guides + 2 principle directories

Credits:
- Skill design and enhancement by @basher83

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-11 16:18:42 -05:00
basher83 97d565e3cd Replace search skill with mem-search (#91)
* feat: add mem-search skill with progressive disclosure architecture

Add comprehensive mem-search skill for accessing claude-mem's persistent
cross-session memory database. Implements progressive disclosure workflow
and token-efficient search patterns.

Features:
- 12 search operations (observations, sessions, prompts, by-type, by-concept, by-file, timelines, etc.)
- Progressive disclosure principles to minimize token usage
- Anti-patterns documentation to guide LLM behavior
- HTTP API integration for all search functionality
- Common workflows with composition examples

Structure:
- SKILL.md: Entry point with temporal trigger patterns
- principles/: Progressive disclosure + anti-patterns
- operations/: 12 search operation files

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* docs: add CHANGELOG entry for mem-search skill

Document mem-search skill addition in Unreleased section with:
- 100% effectiveness compliance metrics
- Comparison to previous search skill implementation
- Progressive disclosure architecture details
- Reference to audit report documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* docs: add mem-search skill audit report

Add comprehensive audit report validating mem-search skill against
Anthropic's official skill-creator documentation.

Report includes:
- Effectiveness metrics comparison (search vs mem-search)
- Critical issues analysis for production readiness
- Compliance validation across 6 key dimensions
- Reference implementation guidance

Result: mem-search achieves 100% compliance vs search's 67%

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: Add comprehensive search architecture analysis document

- Document current state of dual search architectures (HTTP API and MCP)
- Analyze HTTP endpoints and MCP search server architectures
- Identify DRY violations across search implementations
- Evaluate the use of curl as the optimal approach for search
- Provide architectural recommendations for immediate and long-term improvements
- Outline action plan for cleanup, feature parity, DRY refactoring

* refactor: Remove deprecated search skill documentation and operations

* refactor: Reorganize documentation into public and context directories

Changes:
- Created docs/public/ for Mintlify documentation (.mdx files)
- Created docs/context/ for internal planning and implementation docs
- Moved all .mdx files and assets to docs/public/
- Moved all internal .md files to docs/context/
- Added CLAUDE.md to both directories explaining their purpose
- Updated docs.json paths to work with new structure

Benefits:
- Clear separation between user-facing and internal documentation
- Easier to maintain Mintlify docs in dedicated directory
- Internal context files organized separately

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Enhance session management and continuity in hooks

- Updated new-hook.ts to clarify session_id threading and idempotent session creation.
- Modified prompts.ts to require claudeSessionId for continuation prompts, ensuring session context is maintained.
- Improved SessionStore.ts documentation on createSDKSession to emphasize idempotent behavior and session connection.
- Refined SDKAgent.ts to detail continuation prompt logic and its reliance on session.claudeSessionId for unified session handling.

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Alex Newman <thedotmack@gmail.com>
2025-11-11 16:15:07 -05:00
Alex Newman eafdd6a7be Bump version to 5.4.5 2025-11-11 13:51:19 -05:00
Alex Newman 3529f9274b feat: Enhanced logging and SDK prompt improvements (#94)
* Initial plan

* Initial analysis: Found root cause of double entries bug

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Fix double entries by assigning generatorPromise in handleSessionInit

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* feat(logging): Enhance HTTP request logging and session management

- Added middleware for logging HTTP requests and responses, excluding static assets and health checks.
- Introduced a method to summarize request bodies for specific endpoints.
- Improved logging for user prompt synchronization with Chroma, including duration tracking.
- Enhanced session initialization logging to include additional session details.
- Updated observation and summary logging to provide more context and error handling during Chroma synchronization.
- Refactored tool name formatting for logging in the SessionManager.
- Expanded logger component types to include 'HTTP', 'SESSION', and 'CHROMA'.

* Refactor SDK prompts and logging for improved clarity and functionality

- Updated buildInitPrompt to clarify the observer's role and what to record.
- Enhanced buildSummaryPrompt with clearer instructions for summarizing ongoing sessions.
- Improved buildContinuationPrompt to emphasize the focus on deliverables and capabilities.
- Refactored WorkerService to utilize a centralized tool formatting function for logging.
- Added truncation for logged responses and observations to improve readability.
- Updated SessionManager to log the queuing of summarize actions with session details.
- Enhanced App and Sidebar components to support refreshing stats on sidebar open.
- Refactored useStats hook to allow manual refreshing of stats while maintaining automatic loading on mount.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-11 13:49:00 -05:00
Alex Newman 30ebe92a53 Release v5.4.4: Fix duplicate entries in viewer
Bugfix:
- Fixed duplicate observations and summaries appearing in viewer
- Root cause: handleSessionInit spawned SDK agent but didn't save promise to session.generatorPromise
- Second agent would spawn when handleObservations ran, causing duplicates
- Fix: Assign generatorPromise in handleSessionInit (matches handleSummarize pattern)

Technical changes:
- Modified src/services/worker-service.ts:265
- Now tracks promise to prevent duplicate agent spawning
- Guard condition in handleObservations (line 301) now works correctly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-10 17:30:58 -05:00
Copilot 1bb203cbb5 Fix duplicate entries in viewer caused by untracked SDK agent promise (#86)
* Initial plan

* Initial analysis: Found root cause of double entries bug

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Fix double entries by assigning generatorPromise in handleSessionInit

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
Co-authored-by: Alex Newman <thedotmack@gmail.com>
2025-11-10 17:30:02 -05:00
Alex Newman 9f8499fe54 Bump version to 5.4.3
Release v5.4.3: Fix PM2 race condition in worker restart

Changes:
- Removed PM2 restart logic from ensureWorkerRunning()
- PM2 watch mode now exclusively handles worker restarts
- Eliminated race condition causing TypeError
- Reduced unnecessary worker restarts (39+ → minimal)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-10 16:53:23 -05:00
Alex Newman 40d105d7de Merge pull request #87 from thedotmack/copilot/fix-pm2-race-condition
Remove PM2 restart logic from hooks to fix race condition with watch mode
2025-11-10 16:52:25 -05:00
Alex Newman cda12d95c9 Merge main into copilot/fix-pm2-race-condition
Resolved conflicts in built files by rebuilding from source
2025-11-10 16:51:26 -05:00
Alex Newman bbd6113f69 Release v5.4.2: CWD spatial awareness fix
Bugfix release:
- Fixed SDK agent spatial awareness by propagating CWD context from tool execution
- Prevents false "file not found" reports when working across multiple repositories
- Added comprehensive test suite for CWD propagation (8 passing tests)
- Security analysis: Zero vulnerabilities, CodeQL approved

Technical changes:
- Hook extracts CWD from PostToolUseInput and forwards to worker service
- Worker types define optional CWD fields in PendingMessage and ObservationData
- SessionManager passes CWD to SDKAgent's addObservation
- SDK agent includes CWD in tool observation objects sent to Claude API
- Prompts render tool_cwd XML elements with spatial awareness guidance
- All changes present in compiled outputs (save-hook.js, worker-service.cjs)

Documentation:
- Technical documentation: context/CWD_CONTEXT_FIX.md
- PR summary: PR_SUMMARY.md
- Security review: SECURITY_SUMMARY.md
- CHANGELOG entry added

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-10 15:33:14 -05:00
Alex Newman 51d1315562 Merge pull request #90 from thedotmack/copilot/fix-sdk-agent-cwd-issue
Fix: Pass CWD context to SDK agent for spatial awareness
2025-11-10 15:32:00 -05:00
Alex Newman 85763d575d Move CWD documentation to context directory
Per project conventions, docs/ is reserved for user-facing Mintlify .mdx files.
Technical documentation should live in context/ directory.

- Moved docs/CWD_CONTEXT_FIX.md → context/CWD_CONTEXT_FIX.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-10 15:31:05 -05:00
Alex Newman c76ddc2f83 Add TypeScript Agent SDK reference documentation
- Introduced comprehensive API reference for the TypeScript Agent SDK.
- Documented key functions including `query()`, `tool()`, and `createSdkMcpServer()`.
- Detailed types such as `Options`, `Query`, `AgentDefinition`, and various tool input/output types.
- Included examples and descriptions for configuration options and permission handling.
- Added sections for hook types, tool input/output types, and permission types.
2025-11-10 15:04:44 -05:00
copilot-swe-agent[bot] 348cc7f7ac Add security summary documentation
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-10 19:31:57 +00:00
copilot-swe-agent[bot] ed19e92f75 Add PR summary and final verification
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-10 19:30:13 +00:00
copilot-swe-agent[bot] 2a96214456 Add documentation for CWD context fix
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-10 19:28:39 +00:00
copilot-swe-agent[bot] eaa2268bf9 Add CWD propagation tests and verify build output
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-10 19:25:08 +00:00
copilot-swe-agent[bot] 730c420a13 Add CWD propagation through hook, worker, and SDK agent
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-10 19:23:06 +00:00
copilot-swe-agent[bot] d0bdc6ae9b Initial exploration - understanding the codebase
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-10 19:20:41 +00:00
copilot-swe-agent[bot] d46fbb7166 Initial plan 2025-11-10 19:17:13 +00:00
copilot-swe-agent[bot] a60ef6eacb Remove PM2 restart logic from ensureWorkerRunning to fix race condition
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-10 19:05:17 +00:00
copilot-swe-agent[bot] 13941a1e72 Initial analysis: Identify PM2 race condition in ensureWorkerRunning
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-10 19:03:08 +00:00
copilot-swe-agent[bot] bd809c860e Initial plan 2025-11-10 19:00:05 +00:00
Alex Newman b4b90faa1e Release v5.4.1: MCP search server toggle functionality
New Features:
- REST API endpoints for MCP server status and toggle control
- UI toggle in viewer sidebar for enabling/disabling MCP search server
- File-based persistence mechanism (.mcp.json ↔ .mcp.json.disabled)
- Independent state management for MCP toggle

Technical changes:
- Added GET /api/mcp/status endpoint (returns mcpEnabled boolean)
- Added POST /api/mcp/toggle endpoint (toggles MCP server state)
- Updated Settings interface with mcpEnabled property
- Enhanced Sidebar component with MCP toggle UI and state management
- Updated worker service with MCP control logic
- Bumped version to 5.4.1 in all metadata files

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-10 13:48:52 -05:00
Alex Newman 9e235b5b57 feat: Add MCP search server toggle with dedicated API architecture (#85)
* feat: add MCP search server toggle functionality

- Introduced `CLAUDE_MEM_MCP_ENABLED` setting to manage the MCP search server state.
- Updated `WorkerService` to handle MCP enabling/disabling based on the new setting.
- Enhanced `Sidebar` component to include a checkbox for toggling MCP search server.
- Modified `useSettings` hook to incorporate the new MCP setting.
- Updated default settings to include `CLAUDE_MEM_MCP_ENABLED` with a default value of true.
- Adjusted TypeScript types to include the new MCP setting.

* feat: Implement MCP toggle functionality in WorkerService and Sidebar

- Added API endpoints for MCP status retrieval and toggling in WorkerService.
- Updated Sidebar component to manage MCP toggle state and display status messages.
- Removed MCP_ENABLED from settings state management and default settings.
- Adjusted settings interface and related hooks to reflect the removal of MCP_ENABLED.
2025-11-10 13:47:25 -05:00
Alex Newman 5fdf25d60f docs: Add JIT context filtering post-mortem
Comprehensive analysis of the 3.5-hour JIT context filtering experiment.
Documents architectural learnings, implementation attempts, and why the
complex persistent-session approach was reverted in favor of a simpler
future implementation.

Key learnings:
- Hooks cannot run Agent SDK queries (no claudePath access)
- All AI processing must happen in worker service
- Dynamic timeline search is the right approach
- Infrastructure (SQLite FTS5 + Chroma) validated and ready
- Simple per-request approach should be tried before optimization

The vision is sound, execution was overcomplicated. This was productive
R&D that validated constraints and documented patterns for future work.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-09 23:31:03 -05:00
Alex Newman a367436a78 Release v5.4.0: Skill-Based Search Migration & Progressive Disclosure
Version Bump:
- package.json: 5.3.0 → 5.4.0
- .claude-plugin/marketplace.json: 5.3.0 → 5.4.0
- plugin/.claude-plugin/plugin.json: 5.3.0 → 5.4.0
- CLAUDE.md: 5.3.0 → 5.4.0

Breaking Changes:
- MCP search tools removed (replaced with skill-based search)
- No user action required - migration is transparent

Major Features:
- Skill-based search architecture (~2,250 token savings per session)
- Progressive disclosure pattern (skill frontmatter ~250 tokens)
- 10 HTTP API endpoints for search operations
- Natural language queries replace MCP tool syntax

Technical Changes:
- Removed MCP search server registration
- Added search skill with comprehensive documentation
- Updated all documentation to reflect new architecture
- HTTP API endpoints in worker service

Migration Notes:
- Users: No action required, searches work identically
- Developers: MCP server source kept for reference

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-09 19:15:04 -05:00
Alex Newman 5d64df2ba5 docs: Update all documentation to reflect v5.4.0 skill-based search
Documentation Updates:
- README.md: Updated version badge, What's New, and search section
- docs/usage/search-tools.mdx: Rewrote for skill-based natural language approach
- docs/architecture/mcp-search.mdx → search-architecture.mdx: Complete rewrite
- docs/architecture/overview.mdx: Updated components and search pipeline
- docs/usage/getting-started.mdx: Added skill-based search section
- docs/configuration.mdx: Updated search configuration for v5.4.0
- docs/introduction.mdx: Updated key features
- docs/docs.json: Updated navigation to search-architecture

Key Changes:
- Emphasized ~2,250 token savings per session start
- Converted all examples to natural language queries
- Documented 10 HTTP API endpoints
- Explained progressive disclosure pattern
- Added migration notes (transparent, no user action required)
- Removed outdated MCP references

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-09 19:14:05 -05:00
Alex Newman 22a04ac461 Merge pull request #78 from thedotmack/feature/skill-search
v5.4.0: Skill-Based Search Migration & Progressive Disclosure Pattern
2025-11-09 18:58:22 -05:00
Alex Newman 170b66f623 feat: Add documentation for project-level skills and their distinction from plugin skills 2025-11-09 18:55:09 -05:00
Alex Newman ca4f046777 feat: Add search skill with progressive disclosure and refactor existing skills
Enhancements:
- Added search skill with 10 HTTP API endpoints for memory queries
- Refactored version-bump and troubleshoot skills using progressive disclosure pattern
- Added operations/ subdirectories for detailed skill documentation
- Updated CLAUDE.md with skill-based search architecture
- Enhanced worker service with search API endpoints
- Updated CHANGELOG.md with v5.4.0 migration details

Technical changes:
- New plugin/skills/search/ directory with SKILL.md
- New .claude/skills/version-bump/operations/ (workflow.md, scenarios.md)
- New plugin/skills/troubleshoot/operations/ (common-issues.md, worker.md)
- Modified src/services/worker-service.ts (added search endpoints)
- Modified plugin/scripts/worker-service.cjs (rebuilt with search API)
- Reduced main skill files by 89% using progressive disclosure
- Token savings: ~2,250 tokens per session start

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-09 18:41:53 -05:00
Alex Newman 0475a57fb1 feat: Migrate to Skill-Based Search and enhance context handling
- Implemented a new skill-based search approach, replacing MCP search tools to reduce session start context from ~2,500 tokens to ~250 tokens.
- Added 10 new HTTP API endpoints to the worker service for various search functionalities.
- Created a new search skill with detailed instructions and examples for users.
- Removed MCP search server references and updated documentation to reflect the new skill-based approach.
- Enhanced context handling in the context hook to conditionally display the most recent summary based on observations.
- Introduced comprehensive documentation for Agent Skills, including creation, management, and best practices.
2025-11-09 17:34:00 -05:00
Alex Newman fda069bb07 Fix GitHub issues #76, #74, #75 + session lifecycle improvements (#77)
* Refactor context-hook: Fix anti-patterns and improve maintainability

This refactor addresses all anti-patterns documented in CLAUDE.md, improving code quality without changing behavior.

**Dead Code Removed:**
- Eliminated unused useIndexView parameter throughout
- Cleaned up entry point logic

**Magic Numbers → Named Constants:**
- CHARS_PER_TOKEN_ESTIMATE = 4 (token estimation)
- SUMMARY_LOOKAHEAD = 1 (explains +1 in query)
- Added clarifying comment for DISPLAY_SESSION_COUNT

**Code Duplication Eliminated:**
- Reduced 34 lines to 4 lines with renderSummaryField() helper
- Replaced 4 identical summary field rendering blocks

**Error Handling Added:**
- parseJsonArray() now catches JSON.parse exceptions
- Prevents session crashes from malformed data

**Type Safety Improved:**
- Added SessionSummary interface (replaced inline type cast)
- Added SummaryTimelineItem for timeline items
- Proper Map typing: Map<string, TimelineItem[]>

**Variable Naming Clarity:**
- summariesWithOffset → summariesForTimeline
- isMostRecent → shouldShowLink (explains purpose)
- dayTimelines → itemsByDay
- nextSummary → olderSummary (correct chronology)

**Better Documentation:**
- Explained confusing timeline offset logic
- Removed apologetic comments, added clarifying ones

**Impact:**
- 28 lines saved from duplication elimination
- Zero behavioral changes (output identical)
- Improved maintainability and type safety

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix context-hook to respect settings.json contextDepth

The context-hook was ignoring the user's contextDepth setting from the web UI (stored in ~/.claude-mem/settings.json) and always using the default of 50 observations.

**Problem:**
- Web UI sets contextDepth in ~/.claude-mem/settings.json
- Context-hook only read from process.env.CLAUDE_MEM_CONTEXT_OBSERVATIONS
- User's preference of 7 observations was ignored, always showing 50

**Solution:**
- Added getContextDepth() function following same pattern as getWorkerPort()
- Priority: settings.json > env var > default (50)
- Validates contextDepth is a positive number

**Testing:**
- Verified with contextDepth: 7 → shows 7 observations ✓
- Verified with contextDepth: 3 → shows 3 observations ✓
- Settings properly respected on every session start

**Files Changed:**
- src/hooks/context-hook.ts: Added getContextDepth() + imports
- plugin/scripts/context-hook.js: Built output

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix context-hook to read from correct settings file location

**Critical Bug Fix:**
Previous commit read from completely wrong location with wrong field name.

**What was wrong:**
- Reading from: ~/.claude-mem/settings.json (doesn't exist)
- Looking for: contextDepth (wrong field)
- Result: Always falling back to default of 50

**What's fixed:**
- Reading from: ~/.claude/settings.json (correct location)
- Looking for: env.CLAUDE_MEM_CONTEXT_OBSERVATIONS (correct field)
- Matches pattern used in worker-service.ts

**Testing:**
- With CLAUDE_MEM_CONTEXT_OBSERVATIONS: "15" → shows 15 observations ✓
- With CLAUDE_MEM_CONTEXT_OBSERVATIONS: "5" → shows 5 observations ✓
- Web UI settings now properly respected

**Files Changed:**
- src/hooks/context-hook.ts: Fixed path and field name in getContextDepth()
- plugin/scripts/context-hook.js: Built output

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix GitHub issues #76, #74, #75 + session lifecycle improvements

Bug Fixes:
- Fix PM2 'Process 0 not found' error (#76): Changed pm2 restart to pm2 start (idempotent)
- Fix troubleshooting skill distribution (#74, #75): Moved from .claude/skills/ to plugin/skills/

Session Lifecycle Improvements:
- Added session lifecycle context to SDK agent prompt
- Changed summary framing from "final report" to "progress checkpoint"
- Updated summary prompts to use progressive tense ("so far", "actively working on")
- Added buildContinuationPrompt() for prompt #2+ to avoid re-initialization
- SessionManager now restores prompt counter from database
- SDKAgent conditionally uses init vs continuation prompt based on prompt number

These changes improve context-loading task handling and reduce incorrect "file not found" reports in summaries (partial fix for #73 - awaiting user feedback).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Release v5.3.0: Session lifecycle improvements and bug fixes

Improvements:
- Session prompt counter now restored from DB on worker restart
- Continuation prompts for prompt #2+ (lightweight, avoids re-init)
- Summary framing changed from "final report" to "progress checkpoint"
- PM2 start command (idempotent, fixes "Process 0 not found" error)
- Troubleshooting skill moved to plugin/skills/ for proper distribution

Technical changes:
- SessionManager loads prompt_counter from DB on initialization
- SDKAgent uses buildContinuationPrompt() for requests #2+
- Updated summary prompt to clarify mid-session checkpoints
- Fixed worker-utils.ts to use pm2 start instead of pm2 restart
- Moved .claude/skills/troubleshoot → plugin/skills/troubleshoot

Fixes:
- GitHub issue #76: PM2 "Process 0 not found" error
- GitHub issue #74, #75: Troubleshooting skill not distributed
- GitHub issue #73 (partial): Context-loading tasks reported as failed

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-09 16:44:58 -05:00
Alex Newman 4c1f7b4ff1 Release v5.2.3: Added troubleshooting skill
Improvements:
- Added troubleshooting slash command skill for diagnosing claude-mem installation issues
- Comprehensive diagnostic workflow covering PM2, worker health, database, dependencies, logs, and viewer UI
- Automated fix sequences and common issue resolutions
- Full system diagnostic report generation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-08 19:29:26 -05:00
claude[bot] 35b98c53f0 Remove unnecessary troubleshooting checks from skill
Simplified troubleshooting diagnostics by removing checks that are:
- Too brittle (300+ packages requirement)
- Not relevant to v5.2.2 issues (version cache)
- Redundant (Node.js version checks)

Changes:
- Removed "300+ packages installed" expectation
- Removed entire "Step 5: Check Version Cache" section
- Removed dependency count check (ls node_modules/ | wc -l)
- Removed Node.js version checks from multiple sections
- Renumbered steps from 8 to 7

Result: More focused diagnostics for reported issues (memory persistence,
viewer state, observation freshness) without false positives.

Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
2025-11-08 23:21:52 +00:00
copilot-swe-agent[bot] e029903a66 Add troubleshooting slash command skill
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-08 21:47:03 +00:00
copilot-swe-agent[bot] 5666f7dd1c Initial exploration and planning for troubleshooting slash command
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-08 21:43:06 +00:00
copilot-swe-agent[bot] 57f0c9fd92 Initial plan 2025-11-08 21:39:44 +00:00
Alex Newman 5482052c16 Release v5.2.2: Context hook now displays investigated and learned fields
Improvements:
- Context hook now displays 'investigated' and 'learned' fields from session summaries
- Enhanced SQL query to SELECT these fields from database
- Added color-coded display formatting (blue for investigated, yellow for learned)
- Updated TypeScript types to include nullable investigated and learned fields

Technical changes:
- Updated src/hooks/context-hook.ts to query and display new fields
- Updated built plugin/scripts/context-hook.js
- Bumped version to 5.2.2 in all metadata files

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-08 12:55:53 -05:00
Alex Newman ccb7001ff0 fix: update README to enhance image linking and improve layout consistency 2025-11-07 22:02:17 -05:00
Alex Newman edc20a12a1 fix: remove unnecessary horizontal rule from README for cleaner layout 2025-11-07 22:00:49 -05:00
Alex Newman 45c35c6bdd fix: adjust spacing and alignment in README for improved readability 2025-11-07 22:00:28 -05:00
Alex Newman 47ee0838e3 fix: update README to correct image placement for Claude-Mem preview 2025-11-07 21:52:14 -05:00
Alex Newman 809c9e6639 Implement code changes to enhance functionality and improve performance 2025-11-07 21:51:04 -05:00
Alex Newman 9646527d66 Release v5.2.1: Fix project filter synchronization bugs
This patch release fixes critical race conditions and state synchronization
issues in the viewer UI's project filtering system.

**Bug Fixes:**
- Fixed race condition where offset wasn't reset when filter changed
- Fixed state ref synchronization causing stale hasMore values
- Fixed batched state updates mixing data from different projects
- Fixed useEffect dependency cycle causing double renders
- Combined useEffect hooks for guaranteed execution order

**Technical Changes:**
- Updated App.tsx: Fixed filter change detection and data reset logic
- Updated usePagination.ts: Improved offset and state ref handling
- Updated data.ts: Simplified mergeAndDeduplicateByProject validation
- Updated SessionStore.ts: Filter NULL/empty projects from dropdown
- Added investigated field to Summary interface

**Files Updated:**
- package.json: version 5.2.0 → 5.2.1
- .claude-plugin/marketplace.json: version 5.2.0 → 5.2.1
- plugin/.claude-plugin/plugin.json: version 5.2.0 → 5.2.1
- CLAUDE.md: version 5.2.0 → 5.2.1

All changes follow CLAUDE.md coding standards (DRY, YAGNI, fail-fast).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-07 20:29:24 -05:00
Alex Newman f27c73b469 Fix project filter synchronization issues in viewer UI (#70)
* feat: Enhance session and summary handling

- Update SQL query in SessionStore to exclude null or empty projects.
- Add 'investigated' field to Summary interface for better tracking.
- Modify App component to handle pagination more efficiently based on current filters.
- Update SummaryCard to display the 'investigated' field if present.
- Refactor usePagination hook to reset pagination state when filters change.
- Adjust mergeAndDeduplicateByProject function to ensure it only merges unfiltered data.

* refactor: address PR feedback - remove redundancies and fix dependency cycles

Fixes based on PR #70 review feedback:

Required:
- Fixed useEffect dependency cycle in App.tsx (removed handleLoadMore from deps)

Recommended:
- Removed redundant filter detection from App.tsx (usePagination handles this)
- Removed redundant stateRef update effect from usePagination.ts
- Simplified mergeAndDeduplicateByProject by removing defensive validation
- Removed unused imports (useRef, useEffect)

All changes follow CLAUDE.md principles: DRY, fail-fast, no defensive programming.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Reset paginated data arrays when filter changes

Critical fix for data mixing bug identified in PR review follow-up.

Problem:
When filter changes, usePagination correctly resets its offset to 0, but
the paginated state arrays (observations, summaries, prompts) were NOT
being reset. This caused data from different projects to mix together
because setState appends to the existing array.

Example:
- User views Project A: [A1, A2, A3]
- User switches to Project B
- API fetches [B1, B2, B3]
- setState does: [...prev, ...new] = [A1, A2, A3, B1, B2, B3] 

Solution:
Added a separate useEffect that resets all three paginated arrays when
currentFilter changes. This happens BEFORE handleLoadMore fetches new
data, ensuring clean state for the new filter.

Files changed:
- src/ui/viewer/App.tsx: Added useEffect to reset arrays on filter change
- plugin/ui/viewer-bundle.js: Built UI bundle

Testing:
1. Select Project A, verify data loads
2. Switch to Project B
3. Verify ONLY Project B data is shown (no mixing)
4. Switch back to "All Projects"
5. Verify all data appears correctly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Combine filter change useEffect hooks for guaranteed order

Merged two separate useEffect hooks into one to:
1. Guarantee execution order (reset THEN load)
2. Reduce complexity (one hook instead of two)
3. Make intent clearer with single comment

Before:
- useEffect #1: handleLoadMore() on filter change
- useEffect #2: Reset arrays on filter change
- React could run these in any order

After:
- Single useEffect: Reset arrays THEN handleLoadMore()
- Execution order is now guaranteed

Files changed:
- src/ui/viewer/App.tsx: Combined useEffect hooks
- plugin/ui/viewer-bundle.js: Built UI bundle

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-07 20:28:12 -05:00
Alex Newman 7f9959fdb7 Release v5.2.0: Major Worker Service Refactor & UI Improvements
This release merges PR #69, delivering a comprehensive architectural refactor
of the worker service, extensive UI enhancements, and significant code cleanup.

🏗️ Architecture Changes (Worker Service v2)

**Modular Rewrite**: Extracted monolithic worker-service.ts into focused modules:
- DatabaseManager.ts (111 lines): Centralized database initialization
- SessionManager.ts (204 lines): Complete session lifecycle management
- SDKAgent.ts (309 lines): Claude SDK interactions & observation compression
- SSEBroadcaster.ts (86 lines): Server-Sent Events broadcast management
- PaginationHelper.ts (196 lines): Reusable pagination logic
- SettingsManager.ts (68 lines): Viewer settings persistence
- worker-types.ts (176 lines): Shared TypeScript types

**Key Improvements**:
- Eliminated duplicated session logic (4 instances → 1 helper)
- Replaced magic numbers with named constants
- Removed fragile PM2 string parsing
- Fail-fast error handling instead of silent failures
- Fixed SDK agent narrative assignment (obs.title → obs.narrative)

🎨 UI/UX Improvements

**ScrollToTop Component**: GPU-accelerated smooth scrolling button
**ObservationCard Refactor**: Fixed facts toggle, improved metadata display
**Pagination Enhancements**: Better loading states, error recovery, deduplication
**Card Consistency**: Unified layout patterns across all card types

📚 Documentation

**New Files** (7,542 lines):
- context/agent-sdk-ref.md (1,797 lines): Complete Agent SDK reference
- docs/worker-service-architecture.md (1,174 lines): v2 architecture docs
- docs/worker-service-rewrite-outline.md (1,069 lines): Refactor plan
- docs/worker-service-overhead.md (959 lines): Performance analysis
- docs/processing-indicator-*.md (980 lines): Processing status docs
- docs/typescript-errors.md (180 lines): Error reference
- PLAN-full-observation-display.md (468 lines): Future UI roadmap

🧹 Code Cleanup

**Deleted Dead Code** (~2,000 lines):
- src/shared/{config.ts,storage.ts,types.ts}
- src/utils/{platform.ts,usage-logger.ts}
- src/hooks/index.ts, src/sdk/index.ts
- docs/{VIEWER.md,worker-server-architecture.md}

**Files Changed**: 70 total (11 new, 7 deleted, 52 modified)
**Net Impact**: +7,470 lines (11,105 additions, 3,635 deletions)

🐛 Bug Fixes

- Fixed SDK agent narrative assignment (e22edad)
- Corrected PostToolUse hook field name (13643a5)
- Removed unnecessary worker startup from smart-install (6204fe9)
- Simplified context-hook worker management (6204fe9)

 Testing

All systems verified:
- Worker service starts successfully
- All hooks function correctly
- Viewer UI renders properly
- Build pipeline compiles without errors

📖 Reference

PR: #69
Previous Version: 5.1.4
Semantic Version: MINOR (backward compatible features & improvements)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-07 18:33:28 -05:00
claude[bot] 012774b83c docs: add CLAUDE.md explaining docs folder structure
Clarifies that docs/ is a Mintlify documentation site for official
user-facing documentation (.mdx files), while context/ should contain
planning documents, design docs, and internal references.

Lists files currently in docs/ that should be moved to context/:
- typescript-errors.md
- worker-service-*.md files
- processing-indicator-*.md files
- CHROMA.md and chroma-search-completion-plan.md

Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
2025-11-07 23:23:28 +00:00
Alex Newman e22edaddf4 fix: update narrative assignment in SDKAgent to use obs.narrative 2025-11-07 18:04:57 -05:00
Alex Newman 6204fe9b9d refactor: remove startWorker function and adjust installation flow 2025-11-07 18:00:42 -05:00
Alex Newman 30a42036aa feat: add scroll-to-top button and improve pagination handling
- Implemented a scroll-to-top button in the viewer UI for better navigation.
- Added styles for the scroll-to-top button in viewer.html and viewer-template.html.
- Created a new ScrollToTop component to manage visibility and scrolling behavior.
- Updated Feed component to include the ScrollToTop component.
- Enhanced pagination logic in usePagination hook to prevent stale closures and improve performance.
- Modified SDKAgent to include additional observation fields for better data handling.
2025-11-07 17:57:54 -05:00
Alex Newman d6f1237283 Refactor ObservationCard to improve facts toggle logic and metadata display
- Introduced hasFactsContent to determine if facts, concepts, or files are present.
- Updated view-mode toggles to conditionally render based on hasFactsContent.
- Modified content rendering to show subtitle only when facts and narrative are off.
- Enhanced metadata footer to display concepts and files only when facts toggle is active, with improved styling for concepts.
2025-11-07 17:42:45 -05:00
Alex Newman 700e3253fa Refactor card components for improved layout and functionality
- Updated card styles in viewer.html and viewer-template.html to enhance padding, margins, and overall layout.
- Introduced new header structure with left-aligned type and project name, and added view mode toggle buttons for facts and narrative.
- Simplified content rendering logic in ObservationCard, allowing for toggling between facts and narrative.
- Updated metadata display in ObservationCard, PromptCard, and SummaryCard to include formatted date and improved layout.
- Removed unnecessary verbose content sections and streamlined the presentation of facts and narrative.
2025-11-07 17:03:05 -05:00
Alex Newman 740d65b5a5 Add TypeScript Agent SDK reference documentation
- Introduced comprehensive API reference for the TypeScript Agent SDK.
- Documented installation instructions for the SDK.
- Detailed the main functions: `query()`, `tool()`, and `createSdkMcpServer()`.
- Defined various types including `Options`, `Query`, `AgentDefinition`, and more.
- Included message types and their structures, such as `SDKMessage`, `SDKAssistantMessage`, and `SDKUserMessage`.
- Explained hook types and their usage within the SDK.
- Provided detailed documentation for tool input and output types.
- Added sections on permission types and other relevant types for better clarity.
2025-11-07 15:05:31 -05:00
Alex Newman 4bc467f7ed feat: Implement Worker Service for long-running HTTP service with PM2 management
- Introduced WorkerService class to handle HTTP requests and manage sessions.
- Added endpoints for health check, session management, and data retrieval.
- Integrated ChromaSync for background data synchronization.
- Implemented SSE for real-time updates to connected clients.
- Added error handling and logging throughout the service.
- Cached Claude executable path for improved performance.
- Included settings management for user configuration.
- Established database interactions for session and observation management.
2025-11-07 13:26:13 -05:00
Alex Newman 7fdfdd5d5e Release v5.1.4: Bugfix for PostToolUse hook schema compliance
Changes:
- Renamed tool_output to tool_response in save-hook.ts to match Claude Code PostToolUse API schema
- Updated worker-service.ts to use tool_response field consistently
- Rebuilt all hooks and worker service with corrected parameter names

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-07 11:53:57 -05:00
Alex Newman e872c2da38 refactor: rename tool_output to tool_response in save-hook and worker-service 2025-11-07 11:50:14 -05:00
Copilot 0b476e971a Release v5.1.3: Version bump for npm install fix (#66)
* Initial plan

* Release v5.1.3: Fix npm install failures with smart caching

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-06 18:05:43 -05:00
Copilot f7b51a963e Fix npm install failures with automatic retry and silent output (#64)
* Initial plan

* Initial investigation: identified npm install failure issue

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Add retry logic and better error handling to smart-install

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Fix spacing inconsistency in log messages

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Refactor worker startup check for better readability

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

* Simplify npm install: use plain npm install without flags

Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thedotmack <683968+thedotmack@users.noreply.github.com>
2025-11-06 17:21:44 -05:00
182 changed files with 25047 additions and 4801 deletions
+1 -1
View File
@@ -10,7 +10,7 @@
"plugins": [
{
"name": "claude-mem",
"version": "5.1.3",
"version": "6.0.3",
"source": "./plugin",
"description": "Persistent memory system for Claude Code - context compression across sessions"
}
+8 -1
View File
@@ -1,3 +1,10 @@
{
"env": {}
"env": {},
"permissions": {
"deny": [
"Read(./package-lock.json)",
"Read(./node_modules/**)",
"Read(./.DS_Store)"
]
}
}
+29
View File
@@ -0,0 +1,29 @@
# Project-Level Skills
This directory contains skills **for developing and maintaining the claude-mem project itself**, not skills that are released as part of the plugin.
## Distinction
**Project Skills** (`.claude/skills/`):
- Used by developers working on claude-mem
- Not included in the plugin distribution
- Project-specific workflows (version bumps, release management, etc.)
- Not synced to `~/.claude/plugins/marketplaces/thedotmack/`
**Plugin Skills** (`plugin/skills/`):
- Released as part of the claude-mem plugin
- Available to all users who install the plugin
- General-purpose memory search functionality
- Synced to user installations via `npm run sync-marketplace`
## Skills in This Directory
### version-bump
Manages semantic versioning for the claude-mem project itself. Handles updating all four version files (package.json, marketplace.json, plugin.json, CLAUDE.md), creating git tags, and GitHub releases.
**Usage**: Only for claude-mem maintainers releasing new versions.
## Adding New Skills
**For claude-mem development** → Add to `.claude/skills/`
**For end users** → Add to `plugin/skills/` (gets distributed with plugin)
+56 -170
View File
@@ -1,212 +1,98 @@
---
name: version-bump
description: Manage semantic version updates for claude-mem project. Handles patch, minor, and major version increments following semantic versioning. Updates package.json, marketplace.json, plugin.json, and CLAUDE.md version number (NOT version history). Creates git tags.
description: Manage semantic version updates for claude-mem project. Handles patch, minor, and major version increments following semantic versioning. Updates package.json, marketplace.json, plugin.json, and CLAUDE.md version number (NOT version history). Creates git tags and GitHub releases. Auto-generates CHANGELOG.md from releases.
---
# Version Bump Skill
IMPORTANT: This skill manages semantic versioning across the claude-mem project. YOU MUST update all FOUR version-tracked files consistently and create a git tag.
Manage semantic versioning across the claude-mem project with consistent updates to all version-tracked files.
## Quick Reference
**Files requiring updates:**
**Files requiring updates (ALL FOUR):**
1. `package.json` (line 3)
2. `.claude-plugin/marketplace.json` (line 13)
3. `plugin/.claude-plugin/plugin.json` (line 3)
4. `CLAUDE.md` (line 9 ONLY - version number, NOT version history)
**Semantic versioning:**
- PATCH (x.y.Z): Bugfixes only
- MINOR (x.Y.0): New features, backward compatible
- MAJOR (X.0.0): Breaking changes
- **PATCH** (x.y.Z): Bugfixes only
- **MINOR** (x.Y.0): New features, backward compatible
- **MAJOR** (X.0.0): Breaking changes
## Workflow
## Quick Decision Guide
When invoked, follow this process:
**What changed?**
- "Fixed a bug" → PATCH (5.3.0 → 5.3.1)
- "Added new feature" → MINOR (5.3.0 → 5.4.0)
- "Breaking change" → MAJOR (5.3.0 → 6.0.0)
### 1. Analyze Changes
First, understand what changed:
```bash
git log --oneline -5
git diff HEAD~1
```
**If unclear, ASK THE USER explicitly.**
### 2. Determine Version Type
Ask yourself:
- Breaking changes? → MAJOR
- New features? → MINOR
- Bugfixes only? → PATCH
## Standard Workflow
If unclear, ASK THE USER explicitly.
See [operations/workflow.md](operations/workflow.md) for detailed step-by-step process.
### 3. Calculate New Version
From current version in `package.json`:
```bash
grep '"version"' package.json
```
Apply semantic versioning rules:
- Patch: increment Z (4.2.8 → 4.2.9)
- Minor: increment Y, reset Z (4.2.8 → 4.3.0)
- Major: increment X, reset Y and Z (4.2.8 → 5.0.0)
### 4. Preview Changes
BEFORE making changes, show the user:
```
Current version: 4.2.8
New version: 4.2.9 (PATCH)
Reason: Fixed database query bug
Files to update:
- package.json: "version": "4.2.9"
- marketplace.json: "version": "4.2.9"
- plugin.json: "version": "4.2.9"
- CLAUDE.md line 9: "**Current Version**: 4.2.9" (version number ONLY)
- Git tag: v4.2.9
Proceed? (yes/no)
```
### 5. Update Files
**Update package.json:**
```json
{
"name": "claude-mem",
"version": "4.2.9",
...
}
```
**Update .claude-plugin/marketplace.json:**
```json
{
"name": "claude-mem",
"version": "4.2.9",
...
}
```
**Update plugin/.claude-plugin/plugin.json:**
```json
{
"name": "claude-mem",
"version": "4.2.9",
...
}
```
**Update CLAUDE.md:**
ONLY update line 9 with the version number:
```markdown
**Current Version**: 4.2.9
```
**CRITICAL**: DO NOT add version history entries to CLAUDE.md. Version history is managed separately outside this skill.
### 6. Verify Consistency
```bash
# Check all versions match
grep -n '"version"' package.json .claude-plugin/marketplace.json plugin/.claude-plugin/plugin.json
# Should show same version in all three files
```
### 7. Test
```bash
# Verify the plugin loads correctly
npm run build
```
### 8. Commit and Tag
```bash
# Stage all version files
git add package.json .claude-plugin/marketplace.json plugin/.claude-plugin/plugin.json CLAUDE.md plugin/scripts/
# Commit with descriptive message
git commit -m "Release vX.Y.Z: [Brief description]"
# Create annotated git tag
git tag vX.Y.Z -m "Release vX.Y.Z: [Brief description]"
# Push commit and tags
git push && git push --tags
```
### 9. Create GitHub Release
```bash
# Create GitHub release from the tag
gh release create vX.Y.Z --title "vX.Y.Z" --notes "[Brief release notes]"
# Or generate notes automatically from commits
gh release create vX.Y.Z --title "vX.Y.Z" --generate-notes
```
**IMPORTANT**: Always create the GitHub release immediately after pushing the tag. This makes the release discoverable to users and triggers any automated workflows.
**Quick version:**
1. Determine version type (PATCH/MINOR/MAJOR)
2. Calculate new version from current
3. Preview changes to user
4. Update ALL FOUR files
5. Verify consistency
6. Build and test
7. Commit and create git tag
8. Push and create GitHub release
9. Generate CHANGELOG.md from releases and commit
## Common Scenarios
**Scenario 1: Bug fix after testing**
```
User: "Fixed the memory leak in the search function"
You: Determine → PATCH
Calculate → 4.2.8 → 4.2.9
Update all four files (version numbers only)
Build and commit
Create git tag v4.2.9
Push commit and tags
Create GitHub release v4.2.9
```
See [operations/scenarios.md](operations/scenarios.md) for examples:
- Bug fix releases
- New feature releases
- Breaking change releases
**Scenario 2: New MCP tool added**
```
User: "Added web search MCP integration"
You: Determine → MINOR (new feature)
Calculate → 4.2.8 → 4.3.0
Update all four files (version numbers only)
Build and commit
Create git tag v4.3.0
Push commit and tags
Create GitHub release v4.3.0
```
## Critical Rules
**Scenario 3: Database schema redesign**
```
User: "Rewrote storage layer, old data needs migration"
You: Determine → MAJOR (breaking change)
Calculate → 4.2.8 → 5.0.0
Update all four files (version numbers only)
Build and commit
Create git tag v5.0.0
Push commit and tags
Create GitHub release v5.0.0
```
## Error Prevention
**ALWAYS verify:**
- [ ] All FOUR files have matching version numbers (package.json, marketplace.json, plugin.json, CLAUDE.md)
- [ ] Git tag created with format vX.Y.Z
- [ ] GitHub release created from the tag
- [ ] CLAUDE.md: ONLY updated line 9 (version number), did NOT touch version history
- [ ] Commit and tags pushed to remote
**ALWAYS:**
- Update ALL FOUR files with matching version numbers
- Create git tag with format `vX.Y.Z`
- Create GitHub release from the tag
- Generate CHANGELOG.md from releases after creating release
- Ask user if version type is unclear
**NEVER:**
- Update only one, two, or three files - ALL FOUR must be updated
- Update only one, two, or three files
- Skip the verification step
- Forget to create git tag
- Forget to create GitHub release
- Forget to ask user if version type is unclear
- Forget to create git tag or GitHub release
- Add version history entries to CLAUDE.md (that's managed separately)
## Verification Checklist
Before considering the task complete:
- [ ] All FOUR files have matching version numbers
- [ ] `npm run build` succeeds
- [ ] Git commit created with all version files
- [ ] Git tag created (format: vX.Y.Z)
- [ ] Commit and tags pushed to remote
- [ ] GitHub release created from the tag
- [ ] CHANGELOG.md generated and committed
- [ ] CLAUDE.md: ONLY line 9 updated (version number), NOT version history
## Reference Commands
```bash
# View current version
cat package.json | grep version
grep '"version"' package.json
# Verify consistency across all version files
grep '"version"' package.json .claude-plugin/marketplace.json plugin/.claude-plugin/plugin.json
# View git tags
git tag -l -n1
# Check what will be committed
git status
git diff package.json .claude-plugin/marketplace.json plugin/.claude-plugin/plugin.json CLAUDE.md
```
For more commands, see [operations/reference.md](operations/reference.md).
@@ -0,0 +1,275 @@
# Version Bump Reference
Quick reference for version bump commands and file locations.
## File Locations
### Version-Tracked Files (ALL FOUR)
1. **package.json**
- Path: `package.json`
- Line: 3
- Format: `"version": "X.Y.Z",`
2. **marketplace.json**
- Path: `.claude-plugin/marketplace.json`
- Line: 13
- Format: `"version": "X.Y.Z",`
3. **plugin.json**
- Path: `plugin/.claude-plugin/plugin.json`
- Line: 3
- Format: `"version": "X.Y.Z",`
4. **CLAUDE.md**
- Path: `CLAUDE.md`
- Line: 9
- Format: `**Current Version**: X.Y.Z`
## Essential Commands
### View Current Version
```bash
# From package.json
grep '"version"' package.json
# Extract just the version number
grep '"version"' package.json | head -1 | sed 's/.*"version": "\(.*\)".*/\1/'
# From all version files
grep '"version"' package.json .claude-plugin/marketplace.json plugin/.claude-plugin/plugin.json
grep "Current Version" CLAUDE.md
```
### Verify Version Consistency
```bash
# Check all JSON files match
grep '"version"' package.json .claude-plugin/marketplace.json plugin/.claude-plugin/plugin.json
# Should output identical version in all three:
# package.json:3: "version": "5.3.0",
# .claude-plugin/marketplace.json:13: "version": "5.3.0",
# plugin/.claude-plugin/plugin.json:3: "version": "5.3.0",
# Check CLAUDE.md
grep "Current Version" CLAUDE.md
# Should output: **Current Version**: 5.3.0
```
### Git Commands
```bash
# View recent commits
git log --oneline -5
# View changes since last tag
LAST_TAG=$(git describe --tags --abbrev=0)
git log $LAST_TAG..HEAD --oneline
git diff $LAST_TAG..HEAD
# List all tags
git tag -l
# View tag details
git show vX.Y.Z
# List tags with messages
git tag -l -n1
```
### Build and Test
```bash
# Build plugin
npm run build
# Sync to marketplace
npm run sync-marketplace
# Run tests (if available)
npm test
```
### Commit and Tag
```bash
# Stage version files
git add package.json .claude-plugin/marketplace.json plugin/.claude-plugin/plugin.json CLAUDE.md plugin/scripts/
# Commit
git commit -m "Release vX.Y.Z: [Description]"
# Create tag
git tag vX.Y.Z -m "Release vX.Y.Z: [Description]"
# Push
git push && git push --tags
```
### GitHub Release
```bash
# Create release
gh release create vX.Y.Z --title "vX.Y.Z" --notes "[Release notes]"
# Create with auto-generated notes
gh release create vX.Y.Z --title "vX.Y.Z" --generate-notes
# View release
gh release view vX.Y.Z
# List all releases
gh release list
# Delete release (if needed)
gh release delete vX.Y.Z
```
## Semantic Versioning Rules
### Version Format: MAJOR.MINOR.PATCH
**MAJOR (X.0.0):**
- Breaking changes
- Incompatible API changes
- Schema changes requiring migration
- Removes features
**MINOR (x.Y.0):**
- New features (backward compatible)
- New functionality
- Deprecations (but not removals)
- Resets PATCH to 0
**PATCH (x.y.Z):**
- Bug fixes
- Performance improvements
- Documentation fixes
- No new features
### Incrementing Rules
```
PATCH: 5.3.2 → 5.3.3
MINOR: 5.3.2 → 5.4.0 (resets patch)
MAJOR: 5.3.2 → 6.0.0 (resets minor and patch)
```
## Common Patterns
### Bug Fix Release
```bash
# Example: 5.3.0 → 5.3.1
# 1. Update all four files to 5.3.1
# 2. Build and test
npm run build
# 3. Commit and tag
git add package.json .claude-plugin/marketplace.json plugin/.claude-plugin/plugin.json CLAUDE.md plugin/scripts/
git commit -m "Release v5.3.1: Fixed observer crash"
git tag v5.3.1 -m "Release v5.3.1: Fixed observer crash"
git push && git push --tags
# 4. Create release
gh release create v5.3.1 --title "v5.3.1" --notes "Fixed observer crash on empty content"
```
### Feature Release
```bash
# Example: 5.3.0 → 5.4.0
# 1. Update all four files to 5.4.0
# 2. Build and test
npm run build
# 3. Commit and tag
git add package.json .claude-plugin/marketplace.json plugin/.claude-plugin/plugin.json CLAUDE.md plugin/scripts/
git commit -m "Release v5.4.0: Added dark mode support"
git tag v5.4.0 -m "Release v5.4.0: Added dark mode support"
git push && git push --tags
# 4. Create release
gh release create v5.4.0 --title "v5.4.0" --generate-notes
```
### Breaking Change Release
```bash
# Example: 5.3.0 → 6.0.0
# 1. Update all four files to 6.0.0
# 2. Build and test
npm run build
# 3. Commit and tag
git add package.json .claude-plugin/marketplace.json plugin/.claude-plugin/plugin.json CLAUDE.md plugin/scripts/
git commit -m "Release v6.0.0: Storage layer redesign"
git tag v6.0.0 -m "Release v6.0.0: Storage layer redesign"
git push && git push --tags
# 4. Create release with warning
gh release create v6.0.0 --title "v6.0.0" --notes "⚠️ Breaking change: Storage layer redesigned. Migration required."
```
## Rollback Commands
### Delete Tag
```bash
# Delete local tag
git tag -d vX.Y.Z
# Delete remote tag
git push origin :refs/tags/vX.Y.Z
# Or: git push --delete origin vX.Y.Z
```
### Delete Release
```bash
# Delete GitHub release
gh release delete vX.Y.Z
# Confirm deletion
gh release delete vX.Y.Z --yes
```
### Revert Commit
```bash
# Revert last commit (creates new commit)
git revert HEAD
# Reset to previous commit (destructive)
git reset --hard HEAD~1
git push --force # Dangerous! Only if not shared
```
## Error Prevention
### Pre-commit Checks
```bash
# Check all versions match before committing
V1=$(grep '"version"' package.json | head -1 | sed 's/.*"\([^"]*\)".*/\1/')
V2=$(grep '"version"' .claude-plugin/marketplace.json | sed 's/.*"\([^"]*\)".*/\1/')
V3=$(grep '"version"' plugin/.claude-plugin/plugin.json | head -1 | sed 's/.*"\([^"]*\)".*/\1/')
if [ "$V1" = "$V2" ] && [ "$V2" = "$V3" ]; then
echo "✓ All versions match: $V1"
else
echo "✗ Version mismatch!"
echo " package.json: $V1"
echo " marketplace.json: $V2"
echo " plugin.json: $V3"
fi
```
### Pre-push Checks
```bash
# Check tag exists
git tag -l | grep vX.Y.Z || echo "Warning: Tag not created"
# Check build succeeds
npm run build || echo "Error: Build failed"
# Check no uncommitted changes
git status --porcelain | grep -q . && echo "Warning: Uncommitted changes"
```
@@ -0,0 +1,218 @@
# Common Version Bump Scenarios
Real-world examples of version bumps with decision rationale.
## Scenario 1: Bug Fix After Testing
**User request:**
> "Fixed the memory leak in the search function"
**Analysis:**
- What changed: Bug fix
- Breaking changes: No
- New features: No
- **Decision: PATCH**
**Workflow:**
```
Current: 4.2.8
New: 4.2.9 (PATCH)
Steps:
1. Update all four files to 4.2.9
2. npm run build
3. git commit -m "Release v4.2.9: Fixed memory leak in search"
4. git tag v4.2.9 -m "Release v4.2.9: Fixed memory leak in search"
5. git push && git push --tags
6. gh release create v4.2.9 --title "v4.2.9" --notes "Fixed memory leak in search function"
```
## Scenario 2: New Feature Added
**User request:**
> "Added web search MCP integration"
**Analysis:**
- What changed: New feature (MCP integration)
- Breaking changes: No
- Backward compatible: Yes
- **Decision: MINOR**
**Workflow:**
```
Current: 4.2.8
New: 4.3.0 (MINOR - reset patch to 0)
Steps:
1. Update all four files to 4.3.0
2. npm run build
3. git commit -m "Release v4.3.0: Added web search MCP integration"
4. git tag v4.3.0 -m "Release v4.3.0: Added web search MCP integration"
5. git push && git push --tags
6. gh release create v4.3.0 --title "v4.3.0" --generate-notes
```
## Scenario 3: Database Schema Redesign
**User request:**
> "Rewrote storage layer, old data needs migration"
**Analysis:**
- What changed: Storage layer rewrite
- Breaking changes: Yes (requires migration)
- Backward compatible: No
- **Decision: MAJOR**
**Workflow:**
```
Current: 4.2.8
New: 5.0.0 (MAJOR - reset minor and patch to 0)
Steps:
1. Update all four files to 5.0.0
2. npm run build
3. git commit -m "Release v5.0.0: Storage layer redesign with migration required"
4. git tag v5.0.0 -m "Release v5.0.0: Storage layer redesign"
5. git push && git push --tags
6. gh release create v5.0.0 --title "v5.0.0" --notes "⚠️ Breaking change: Storage layer redesigned. Migration required."
```
## Scenario 4: Multiple Small Bug Fixes
**User request:**
> "Fixed three bugs: observer crash, viewer pagination, and date formatting"
**Analysis:**
- What changed: Multiple bug fixes
- Breaking changes: No
- New features: No
- **Decision: PATCH** (one patch covers all fixes)
**Workflow:**
```
Current: 4.2.8
New: 4.2.9 (PATCH)
Steps:
1. Update all four files to 4.2.9
2. npm run build
3. git commit -m "Release v4.2.9: Multiple bug fixes
- Fixed observer crash on empty content
- Fixed viewer pagination edge case
- Fixed date formatting in timeline"
4. git tag v4.2.9 -m "Release v4.2.9: Multiple bug fixes"
5. git push && git push --tags
6. gh release create v4.2.9 --title "v4.2.9" --generate-notes
```
## Scenario 5: Feature + Bug Fix
**User request:**
> "Added dark mode support and fixed the viewer crash bug"
**Analysis:**
- What changed: New feature + bug fix
- Breaking changes: No
- **Decision: MINOR** (feature trumps bug fix)
**Workflow:**
```
Current: 5.1.0
New: 5.2.0 (MINOR)
Steps:
1. Update all four files to 5.2.0
2. npm run build
3. git commit -m "Release v5.2.0: Dark mode support + bug fixes
Features:
- Added dark mode toggle to viewer UI
Bug fixes:
- Fixed viewer crash on empty database"
4. git tag v5.2.0 -m "Release v5.2.0: Dark mode support"
5. git push && git push --tags
6. gh release create v5.2.0 --title "v5.2.0" --generate-notes
```
## Scenario 6: Documentation Only
**User request:**
> "Updated README with new installation instructions"
**Analysis:**
- What changed: Documentation only
- Breaking changes: No
- Code changes: No
- **Decision: PATCH** (or skip version bump if no code changes)
**Workflow:**
```
Option 1: PATCH (if you want to tag doc improvements)
Current: 4.2.8
New: 4.2.9
Option 2: No version bump (documentation-only changes don't require versioning)
Just commit without bumping version
```
**Recommendation:** Skip version bump for documentation-only changes unless it's a significant documentation overhaul.
## Scenario 7: Configuration Change
**User request:**
> "Changed default observation count from 50 to 30"
**Analysis:**
- What changed: Default configuration
- Breaking changes: Yes (behavior changes)
- Users might notice different context size
- **Decision: MINOR or MAJOR** (ask user)
**Workflow:**
```
Ask user:
"This changes default behavior (context size). Users will see different results.
Is this:
- MINOR (acceptable behavior change): 4.2.8 → 4.3.0
- MAJOR (breaking change): 4.2.8 → 5.0.0
Which should I use?"
```
## Scenario 8: Dependency Update
**User request:**
> "Updated Claude SDK from 1.2.0 to 1.3.0"
**Analysis:**
- What changed: Dependency version
- Breaking changes: Depends on SDK changes
- **Decision: Ask user or check SDK changelog**
**Workflow:**
```
1. Check SDK changelog for breaking changes
2. If SDK has breaking changes → MAJOR
3. If SDK adds features → MINOR
4. If SDK only fixes bugs → PATCH
Typical: PATCH (unless SDK breaks compatibility)
```
## Decision Tree
```
Is there a breaking change?
├─ Yes → MAJOR (X.0.0)
└─ No
├─ Is there a new feature?
│ ├─ Yes → MINOR (x.Y.0)
│ └─ No
│ ├─ Is there a bug fix?
│ │ ├─ Yes → PATCH (x.y.Z)
│ │ └─ No → Don't bump version (docs only, etc.)
│ └─ Configuration change? → Ask user (MINOR or MAJOR)
└─ Multiple changes? → Use highest level (MAJOR > MINOR > PATCH)
```
@@ -0,0 +1,251 @@
# Detailed Version Bump Workflow
Step-by-step process for bumping versions in the claude-mem project.
## Step 1: Analyze Changes
First, understand what changed:
```bash
# View recent commits
git log --oneline -5
# See what changed in last commit
git diff HEAD~1
# Or see all changes since last tag
LAST_TAG=$(git describe --tags --abbrev=0)
git log $LAST_TAG..HEAD --oneline
git diff $LAST_TAG..HEAD
```
## Step 2: Determine Version Type
Ask yourself:
- **Breaking changes?** → MAJOR
- **New features?** → MINOR
- **Bugfixes only?** → PATCH
**If unclear, ASK THE USER explicitly.**
### Decision Matrix
| Change Type | Version Bump | Example |
|------------|--------------|---------|
| Bug fix | PATCH | 4.2.8 → 4.2.9 |
| New feature (backward compatible) | MINOR | 4.2.8 → 4.3.0 |
| Breaking change | MAJOR | 4.2.8 → 5.0.0 |
| Multiple features | MINOR | 4.2.8 → 4.3.0 |
| Feature + breaking change | MAJOR | 4.2.8 → 5.0.0 |
## Step 3: Calculate New Version
From current version in `package.json`:
```bash
grep '"version"' package.json
```
Apply semantic versioning rules:
- **Patch:** increment Z (4.2.8 → 4.2.9)
- **Minor:** increment Y, reset Z (4.2.8 → 4.3.0)
- **Major:** increment X, reset Y and Z (4.2.8 → 5.0.0)
## Step 4: Preview Changes
**BEFORE making changes, show the user:**
```
Current version: 4.2.8
New version: 4.2.9 (PATCH)
Reason: Fixed database query bug
Files to update:
- package.json: "version": "4.2.9"
- marketplace.json: "version": "4.2.9"
- plugin.json: "version": "4.2.9"
- CLAUDE.md line 9: "**Current Version**: 4.2.9" (version number ONLY)
- Git tag: v4.2.9
Proceed? (yes/no)
```
Wait for user confirmation before proceeding.
## Step 5: Update Files
### Update package.json
File: `package.json`
```json
{
"name": "claude-mem",
"version": "4.2.9",
...
}
```
Update line 3 with new version.
### Update marketplace.json
File: `.claude-plugin/marketplace.json`
```json
{
"name": "claude-mem",
"version": "4.2.9",
...
}
```
Update line 13 with new version.
### Update plugin.json
File: `plugin/.claude-plugin/plugin.json`
```json
{
"name": "claude-mem",
"version": "4.2.9",
...
}
```
Update line 3 with new version.
### Update CLAUDE.md
File: `CLAUDE.md`
**ONLY update line 9 with the version number:**
```markdown
**Current Version**: 4.2.9
```
**CRITICAL:** DO NOT add version history entries to CLAUDE.md. Version history is managed separately outside this skill.
## Step 6: Verify Consistency
```bash
# Check all versions match
grep -n '"version"' package.json .claude-plugin/marketplace.json plugin/.claude-plugin/plugin.json
# Should show same version in all three files:
# package.json:3: "version": "4.2.9",
# .claude-plugin/marketplace.json:13: "version": "4.2.9",
# plugin/.claude-plugin/plugin.json:3: "version": "4.2.9",
```
All three must match exactly.
## Step 7: Test
```bash
# Verify the plugin loads correctly
npm run build
```
Build must succeed before proceeding.
## Step 8: Commit and Tag
```bash
# Stage all version files
git add package.json .claude-plugin/marketplace.json plugin/.claude-plugin/plugin.json CLAUDE.md plugin/scripts/
# Commit with descriptive message
git commit -m "Release vX.Y.Z: [Brief description]
[Optional detailed description]
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>"
# Create annotated git tag
git tag vX.Y.Z -m "Release vX.Y.Z: [Brief description]"
# Push commit and tags
git push && git push --tags
```
Replace `X.Y.Z` with actual version (e.g., `4.2.9`).
## Step 9: Create GitHub Release
```bash
# Create GitHub release from the tag
gh release create vX.Y.Z --title "vX.Y.Z" --notes "[Brief release notes]"
# Or generate notes automatically from commits
gh release create vX.Y.Z --title "vX.Y.Z" --generate-notes
```
**IMPORTANT:** Always create the GitHub release immediately after pushing the tag. This makes the release discoverable to users and triggers any automated workflows.
## Step 10: Generate CHANGELOG
After creating the GitHub release, regenerate CHANGELOG.md from all releases:
```bash
# Generate CHANGELOG.md from all GitHub releases
npm run changelog:generate
# Review the generated changelog
git diff CHANGELOG.md
# Commit and push the updated changelog
git add CHANGELOG.md
git commit -m "Update CHANGELOG.md for vX.Y.Z release"
git push
```
**Why this step:**
- CHANGELOG.md is auto-generated from GitHub releases
- Keeps the changelog in sync with release notes
- No manual editing required
- Single source of truth: GitHub releases
## Verification
After completing all steps, verify:
```bash
# Check git tag created
git tag -l | grep vX.Y.Z
# Check remote has tag
git ls-remote --tags origin | grep vX.Y.Z
# Check GitHub release exists
gh release view vX.Y.Z
# Verify versions match
grep '"version"' package.json .claude-plugin/marketplace.json plugin/.claude-plugin/plugin.json
```
All checks should pass.
## Rollback (If Needed)
If you made a mistake:
```bash
# Delete local tag
git tag -d vX.Y.Z
# Delete remote tag (if already pushed)
git push origin :refs/tags/vX.Y.Z
# Delete GitHub release (if created)
gh release delete vX.Y.Z
# Revert commits if needed
git revert HEAD
```
Then restart the workflow with correct version.
+1448 -343
View File
File diff suppressed because it is too large Load Diff
+46 -273
View File
@@ -6,13 +6,17 @@ Claude-mem is a Claude Code plugin providing persistent memory across sessions.
**Your Role**: You are working on the plugin itself. When users interact with Claude Code with this plugin installed, your observations get captured and become their persistent memory.
**Current Version**: 5.1.3
**Current Version**: 6.0.3
## IMPORTANT: Skills Are Auto-Invoked
**There is no `/skill` command.** Skills auto-invoke based on description metadata matching user queries. Don't document manual invocation (e.g., "Run `/skill troubleshoot`"). Instead: "The troubleshoot skill auto-activates when issues are detected."
## Critical Architecture Knowledge
### The Lifecycle Flow
1. **SessionStart**`context-hook.ts` runs
1. **SessionStart** smart-install.js runs first (pre-hook), then `context-hook.ts` runs
- Smart installer checks dependencies (cached, only runs on version changes)
- Starts PM2 worker if not healthy
- Injects context from previous sessions (configurable observation count)
@@ -31,6 +35,8 @@ Claude-mem is a Claude Code plugin providing persistent memory across sessions.
- Marks session complete (graceful, not DELETE)
- Skips on `/clear` to preserve ongoing sessions
**Note**: smart-install.js is a pre-hook script (not a lifecycle hook). It's called before context-hook via command chaining in hooks.json and only runs when dependencies need updating.
### Key Components
**Hooks** (`src/hooks/*.ts`)
@@ -50,10 +56,11 @@ Claude-mem is a Claude Code plugin providing persistent memory across sessions.
- FTS5 virtual tables for full-text search
- `SessionStore` = CRUD, `SessionSearch` = FTS5 queries
**MCP Search Server** (`src/servers/search-server.ts`)
- Exposes 9 search tools to Claude Code
- Configured in `plugin/.mcp.json`
- Built to `plugin/search-server.mjs` (ESM format)
**Search Skill** (`plugin/skills/mem-search/SKILL.md`)
- Provides access to all search functionality via HTTP API + skill
- Auto-invoked when users ask about past work, decisions, or history
- Uses HTTP endpoints instead of MCP tools (~2,250 token savings per session)
- 10 search operations: observations, sessions, prompts, by-type, by-file, by-concept, timelines, etc.
**Chroma Vector Database** (`src/services/sync/ChromaSync.ts`)
- Hybrid semantic + keyword search architecture
@@ -86,12 +93,11 @@ npm run worker:restart
```
Must restart PM2 worker for changes to take effect.
### When You Modify MCP Server
### When You Modify Search Skill
```bash
npm run build
npm run sync-marketplace
# Restart Claude Code for MCP changes
```
Skill changes take effect immediately on next Claude Code session. No build or restart needed (skills are markdown).
### When You Modify Viewer UI
```bash
@@ -104,137 +110,27 @@ Changes to React components, styles, or viewer logic require rebuilding and rest
### Build Pipeline
1. `npm run build` → Compiles TypeScript, outputs to `plugin/`
2. `npm run sync-marketplace` → Syncs to `~/.claude/plugins/marketplaces/thedotmack/`
3. Changes are live for next session (hooks/MCP) or after restart (worker)
3. Changes are live for next session (hooks/skills) or after restart (worker)
## Coding Standards: DRY, YAGNI, and Anti-Patterns
## Coding Standards
**Philosophy**: Write the dumb, obvious thing first. Add complexity only when you actually hit the problem.
**Philosophy**: Write the dumb, obvious thing first. Add complexity only when you hit the problem.
### Common Anti-Patterns to Avoid
**1. Wrapper Functions for Constants**
```typescript
// ❌ DON'T: Ceremonial wrapper that adds zero value
export function getWorkerPort(): number {
return FIXED_PORT;
}
// ✅ DO: Export the constant directly
export const WORKER_PORT = parseInt(process.env.CLAUDE_MEM_WORKER_PORT || "37777", 10);
```
**2. Unused Default Parameters**
```typescript
// ❌ DON'T: Defaults that are never actually used
async function isHealthy(timeout: number = 3000) { ... }
// Every call: isHealthy(1000) - the default is dead code
// ✅ DO: Remove the default if no one uses it
async function isHealthy(timeout: number) { ... }
```
**3. Magic Numbers Everywhere**
```typescript
// ❌ DON'T: Unexplained magic numbers scattered throughout
if (await isWorkerHealthy(1000)) { ... }
await waitForHealth(10000);
setTimeout(resolve, 100);
// ✅ DO: Named constants with context
const HEALTH_CHECK_TIMEOUT_MS = 1000;
const HEALTH_CHECK_MAX_WAIT_MS = 10000;
const HEALTH_CHECK_POLL_INTERVAL_MS = 100;
```
**4. Overengineered Error Handling**
```typescript
// ❌ DON'T: Silent failures and defensive programming for ghosts
checkProcess.on("close", (code) => {
// PM2 list can fail, but we should still continue - just assume worker isn't running
resolve(); // <- Silent failure!
});
// ✅ DO: Fail fast with clear errors
checkProcess.on("close", (code) => {
if (code !== 0) {
reject(new Error(`PM2 not found - install dependencies first`));
}
resolve();
});
```
**5. Fragile String Parsing**
```typescript
// ❌ DON'T: Parse human-readable output with string matching
const isRunning = output.includes("claude-mem-worker") && output.includes("online");
// ✅ DO: Use structured output (JSON)
const processes = JSON.parse(execSync('pm2 jlist'));
const isRunning = processes.some(p => p.name === 'claude-mem-worker' && p.pm2_env.status === 'online');
```
**6. Duplicated Promise Wrappers**
```typescript
// ❌ DON'T: Copy-paste the same promise pattern multiple times
await new Promise((resolve, reject) => {
process1.on("error", reject);
process1.on("close", (code) => { /* ... */ });
});
// ... later ...
await new Promise((resolve, reject) => {
process2.on("error", reject);
process2.on("close", (code) => { /* ... same pattern */ });
});
// ✅ DO: Extract a helper function
async function waitForProcess(process: ChildProcess, validateExitCode = false): Promise<void> {
return new Promise((resolve, reject) => {
process.on("error", reject);
process.on("close", (code) => {
if (validateExitCode && code !== 0 && code !== null) {
reject(new Error(`Process failed with exit code ${code}`));
} else {
resolve();
}
});
});
}
```
**7. YAGNI Violations - Solving Problems You Don't Have**
```typescript
// ❌ DON'T: 50+ lines checking PM2 status before starting
const checkProcess = spawn(pm2Path, ["list", "--no-color"]);
// ... parse output ...
// ... check if running ...
// ... then maybe start it ...
// ✅ DO: Just start it (PM2 start is idempotent)
if (!await isWorkerHealthy()) {
await startWorker(); // PM2 handles "already running" gracefully
if (!await waitForWorkerHealth()) {
throw new Error("Worker failed to become healthy");
}
}
```
### Why These Patterns Appear
These anti-patterns often emerge from:
- **Training bias**: Code that looks "professional" is often overengineered
- **Risk aversion**: Optimizing for "what could go wrong" instead of "what do you actually need"
- **Pattern matching**: Seeing a problem and immediately scaffolding a framework
- **No real-world pain**: Not debugging at 2am means not feeling the cost of complexity
### The Actual Standard
1. **YAGNI (You Aren't Gonna Need It)**: Don't build it until you need it
2. **DRY (Don't Repeat Yourself)**: Extract patterns after the second duplication, not before
**Key Principles:**
1. **YAGNI**: Don't build it until you need it
2. **DRY**: Extract patterns after second duplication, not before
3. **Fail Fast**: Explicit errors beat silent failures
4. **Simple First**: Write the obvious solution, then optimize only if needed
4. **Simple First**: Write the obvious solution, optimize only if needed
5. **Delete Aggressively**: Less code = fewer bugs
**Reference**: See worker-utils.ts critique (conversation 2025-11-05) for detailed examples.
**Common anti-patterns to avoid:**
- Ceremonial wrapper functions for constants (just export the constant)
- Unused default parameters (remove if never used)
- Magic numbers without named constants
- Silent failures instead of explicit errors
- Fragile string parsing (use structured JSON output)
- Copy-pasted promise wrappers (extract helper functions)
- Overengineered "defensive" code for problems you don't have
## Common Tasks
@@ -244,6 +140,8 @@ These anti-patterns often emerge from:
3. Add configuration to `plugin/hooks/hooks.json`
4. Build and sync: `npm run build && npm run sync-marketplace`
**Note**: smart-install.js is not a hook - it's a pre-hook dependency checker that runs before context-hook via command chaining.
### Modifying Database Schema
1. Update schema in `src/services/sqlite/schema.ts`
2. Update SessionStore/SessionSearch classes
@@ -263,148 +161,24 @@ pm2 delete claude-mem-worker # Force clean start
2. `npm run build && npm run sync-marketplace`
3. Start new Claude Code session (hooks) or restart worker (worker changes)
4. Check `~/.claude-mem/claude-mem.db` for database state
5. Use MCP search tools to verify behavior
5. Use mem-search skill to verify behavior (auto-invoked when asking about past work)
### Version Bumps
Use the version-bump skill:
```bash
/skill version-bump
```
Choose patch/minor/major. Updates package.json, marketplace.json, plugin.json, and CLAUDE.md.
Use the `version-bump` skill (auto-invokes when requesting version updates). It handles:
- Semantic version increments (patch/minor/major)
- Updates all version references (package.json, plugin.json, CLAUDE.md, marketplace.json)
- Creates git tags and GitHub releases
- Auto-generates CHANGELOG.md from releases
## Investigation Best Practices
**When investigations are failing persistently**, use Task agents for comprehensive file analysis instead of grep/search:
When investigations fail persistently, use Task agents for comprehensive file analysis instead of repeated grep/search. Deploy agents to read full files and answer specific questions - more efficient than multiple rounds of searching.
**❌ Don't:** Repeatedly grep and search for patterns when failing to find the issue
## Environment Variables
**✅ Do:** Deploy a Task agent to read files in full and answer specific questions
```
"Read these files in full and answer: [specific questions about the implementation]"
- Reduces token usage by delegating to a specialized agent
- Provides comprehensive analysis in one pass
- Finds issues that grep might miss due to poor query formulation
- More efficient than multiple rounds of searching
```
**Example:**
```
Deploy a general-purpose Task agent to:
1. Read src/hooks/context-hook.ts in full
2. Read src/servers/search-server.ts in full
3. Answer: How do these files work together? What's the current implementation state?
4. Find any bugs or inconsistencies between them
```
Use this when:
- Investigating how multiple files interact
- Search queries aren't finding what you expect
- Need complete implementation context
- Issue might be a subtle inconsistency between files
## Recent Changes
### v5.1.2 - Theme Toggle
**Theme Support**: Light/dark mode for viewer UI
- User-selectable theme with persistent settings
- Automatic system preference detection
- Smooth transitions between themes
- Settings stored in browser localStorage
### v5.1.0 - Web-Based Viewer UI
**Major Feature**: Web-Based Viewer UI for Real-Time Memory Stream
- Production-ready viewer accessible at http://localhost:37777
- Real-time visualization via Server-Sent Events (SSE) - see observations, sessions, and prompts as they happen
- Infinite scroll pagination with automatic deduplication
- Project filtering to focus on specific codebases
- Settings persistence (sidebar state, selected project)
- Auto-reconnection with exponential backoff
- GPU-accelerated animations for smooth interactions
**Worker Service API Endpoints** (14 HTTP/SSE endpoints total):
*Viewer & Health:*
- `GET /` - Serves viewer HTML (self-contained React app)
- `GET /health` - Health check endpoint
- `GET /stream` - Server-Sent Events for real-time updates
*Data Retrieval:*
- `GET /api/prompts` - Paginated user prompts with project filtering
- `GET /api/observations` - Paginated observations with project filtering
- `GET /api/summaries` - Paginated session summaries with project filtering
- `GET /api/stats` - Database statistics (total counts by project)
*Settings:*
- `GET /api/settings` - Get current viewer settings
- `POST /api/settings` - Update viewer settings
*Session Management:*
- `POST /sessions/:sessionDbId/init` - Initialize new session
- `POST /sessions/:sessionDbId/observations` - Add observations to session
- `POST /sessions/:sessionDbId/summarize` - Generate session summary
- `GET /sessions/:sessionDbId/status` - Get session status
- `DELETE /sessions/:sessionDbId` - Delete session (graceful cleanup)
**Database Enhancements** (+98 lines in SessionStore):
- `getRecentPrompts()` - Paginated prompts with OFFSET/LIMIT
- `getRecentObservations()` - Paginated observations with OFFSET/LIMIT
- `getRecentSummaries()` - Paginated summaries with OFFSET/LIMIT
- `getStats()` - Aggregated statistics by project
- `getUniqueProjects()` - Distinct project names
**Complete React UI** (17 new files, 1,500+ lines):
- Components: Header, Sidebar, Feed, Cards (Observation, Prompt, Summary, Skeleton)
- Hooks: useSSE, usePagination, useSettings, useStats
- Utils: Data merging, formatters, constants
- Assets: Monaspace Radon font, logos (dark mode + logomark)
- Build: esbuild pipeline for self-contained HTML bundle
**Why This Matters**: Users can now visualize their memory stream in real-time. See exactly what claude-mem is capturing as you work, filter by project, and understand the context being injected into sessions.
### v5.0.3 - Smart Install Caching
**Smart Caching Installer for Windows Compatibility**:
- Eliminated redundant npm install on every SessionStart (2-5s → 10ms)
- Caches version in `.install-version` file
- Only runs npm install when actually needed (first time, version change, missing deps)
- 200x performance improvement for cached installations
### v5.0.0 - Hybrid Search Architecture
**Major Feature**: Chroma Vector Database Integration
- Hybrid semantic + keyword search combining ChromaDB with SQLite FTS5
- ChromaSync service for automatic vector embedding synchronization (738 lines)
- 90-day recency filtering for contextually relevant results
- New MCP tools: `get_context_timeline` and `get_timeline_by_query`
- Performance: Semantic search <200ms with 8,000+ vector documents
- Enhanced all 9 MCP search tools with hybrid search capabilities
## Configuration Users Can Set
**Model Selection** (`~/.claude/settings.json`):
```json
{
"env": {
"CLAUDE_MEM_MODEL": "claude-haiku-4-5" // or sonnet-4-5, opus-4, etc.
}
}
```
**Context Observation Count** (`~/.claude/settings.json`):
```json
{
"env": {
"CLAUDE_MEM_CONTEXT_OBSERVATIONS": "50" // default, adjust based on needs
}
}
```
**Worker Port** (`~/.claude/settings.json`):
```json
{
"env": {
"CLAUDE_MEM_WORKER_PORT": "37777" // default
}
}
```
- `CLAUDE_MEM_MODEL` - Model for observations/summaries (default: claude-haiku-4-5)
- `CLAUDE_MEM_CONTEXT_OBSERVATIONS` - Observations injected at SessionStart (default: 50)
- `CLAUDE_MEM_WORKER_PORT` - Worker service port (default: 37777)
## Key Design Decisions
@@ -414,13 +188,13 @@ Hooks have strict timeout limits. PM2 manages a persistent background worker, al
### Why SQLite FTS5
Enables instant full-text search across thousands of observations without external dependencies. Automatic sync triggers keep FTS5 tables synchronized.
### Why Graceful Cleanup (v4.1.0)
### Why Graceful Cleanup
Changed from aggressive DELETE requests to marking sessions complete. Prevents interrupting summary generation and other async operations.
### Why Smart Install Caching (v5.0.3)
### Why Smart Install Caching
npm install is expensive (2-5s). Caching version state and only installing on changes makes SessionStart nearly instant (10ms).
### Why Web-Based Viewer UI (v5.1.0)
### Why Web-Based Viewer UI
Real-time visibility into memory stream helps users understand what's being captured and how context is being built. SSE provides instant updates without polling. Self-contained HTML bundle (esbuild) eliminates deployment complexity - everything served from a single file.
## File Locations
@@ -438,6 +212,5 @@ Real-time visibility into memory stream helps users understand what's being capt
**Sync**: `npm run sync-marketplace`
**Worker Restart**: `npm run worker:restart`
**Worker Logs**: `npm run worker:logs`
**Version Bump**: `/skill version-bump`
**Usage Analysis**: `npm run usage:today`
**Viewer UI**: http://localhost:37777 (auto-starts with worker)
+434
View File
@@ -0,0 +1,434 @@
# Hybrid Search Architecture: Problem-Solution Document
**Date:** 2025-01-15
**Author:** Claude Code (Session handoff document)
**Purpose:** Comprehensive fix guide for hybrid search architecture documentation and implementation
---
## Executive Summary
The claude-mem hybrid search architecture is **correctly implemented in code** but **incorrectly documented** in skill guides. Additionally, the workflow is missing the final "instant context timeline" step that completes the human memory analogy.
**Quick Status:**
- ✅ Backend code (`search-server.ts`): ChromaDB first, SQLite temporal sort
- ❌ Skill operation guides: Describe FTS5 as primary search method
- ❌ Missing feature: Automatic timeline context retrieval (before/after observations)
- ✅ Landing page: Recently corrected
- ⚠️ Documentation: Needs validation and potential refinement
---
## The Intended Architecture (User's Vision)
### Storage Flow
```
User Action
1. SQLite Insert (FAST, synchronous)
- Immediate persistence
- Available for querying instantly
2. ChromaDB Sync (BACKGROUND, asynchronous)
- Worker generates embeddings
- Takes time but doesn't block user
- Uses OpenAI text-embedding-3-small
```
**Why this design:**
- Users don't wait for embedding generation
- SQLite provides immediate access
- ChromaDB catches up in background for semantic search
### Search Flow (3-Layer Sequential Architecture)
```
User Query: "How did we implement authentication?"
LAYER 1: Semantic Retrieval (ChromaDB)
- Vector similarity search
- Returns observation IDs (not full records)
- Top 100 semantic matches
- 90-day recency filter applied
LAYER 2: Temporal Ordering (SQLite)
- Takes IDs from Layer 1
- Hydrates full records from SQLite
- Sorts by created_at_epoch DESC
- Returns NEWEST relevant observation
LAYER 3: Instant Context Timeline (SQLite) [MISSING IN CURRENT IMPLEMENTATION]
- Takes top observation ID from Layer 2
- Retrieves N observations BEFORE that point
- Retrieves N observations AFTER that point
- Provides temporal context: "what led here" + "what happened next"
Present to User
- Most relevant observation
- Timeline showing before/after context
- Mimics human memory
```
**Why ChromaDB can't do it alone:**
- ChromaDB doesn't efficiently support date range queries sorted by time
- SQLite excels at temporal operations (ORDER BY created_at_epoch)
- Need both: ChromaDB for semantic, SQLite for temporal
**Why the timeline matters:**
> LLMs don't experience time linearly like humans do. Humans remember: "I did X, which led to Y, then Z happened." The instant context timeline gives LLMs this temporal awareness that humans experience naturally.
### Fallback Behavior
```
IF ChromaDB unavailable OR no results:
FTS5 Keyword Search (SQLite)
- Full-text search on observations_fts
- Basic keyword matching
- Ensures backward compatibility
- Fallback for older systems
```
**FTS5 is NOT "optional"** - it's the fallback mechanism for when ChromaDB isn't available or returns no results.
---
## Current State Analysis
### ✅ What's Correct: Backend Implementation
**File:** `/Users/alexnewman/Scripts/claude-mem/src/servers/search-server.ts`
**Lines:** 360-396 (search_observations handler)
The code DOES implement Layers 1 & 2 correctly:
```typescript
// Step 1: ChromaDB semantic search (top 100)
if (chromaClient) {
const chromaResults = await queryChroma(query, 100);
// Step 2: Filter by 90-day recency
const ninetyDaysAgo = Date.now() - (90 * 24 * 60 * 60 * 1000);
const recentIds = chromaResults.ids.filter((_id, idx) => {
const meta = chromaResults.metadatas[idx];
return meta && meta.created_at_epoch > ninetyDaysAgo;
});
// Step 3: Hydrate from SQLite with temporal ordering
results = store.getObservationsByIds(recentIds, {
orderBy: 'date_desc',
limit
});
}
// Fallback to FTS5 if ChromaDB unavailable
if (results.length === 0) {
results = search.searchObservations(query, options); // FTS5
}
```
**What this gets right:**
- ChromaDB semantic search FIRST (not FTS5)
- 90-day recency filter
- SQLite temporal ordering (`orderBy: 'date_desc'`)
- FTS5 fallback for reliability
### ❌ What's Wrong: Skill Operation Guides
**File:** `/Users/alexnewman/Scripts/claude-mem/plugin/skills/mem-search/operations/observations.md`
**Current Title:** "Search Observations (Full-Text)"
**Current Description:** "Search all observations using natural language queries."
**Current Line 351:** `query: z.string().describe('Search query for FTS5 full-text search')`
**The Problem:**
- Describes FTS5 as the search method
- No mention of ChromaDB semantic search
- Misleading title "Full-Text" implies keyword-only
- Examples don't show the ChromaDB → SQLite flow
**Impact:**
- Claude thinks it's doing FTS5 keyword search
- Doesn't understand it's semantic vector search
- Can't explain the architecture to users correctly
### ⚠️ What's Missing: Layer 3 (Instant Context Timeline)
The current implementation stops at Layer 2 (temporal ordering). It doesn't automatically:
1. Identify the MOST relevant observation (it returns a sorted list)
2. Retrieve observations BEFORE that point in time
3. Retrieve observations AFTER that point in time
4. Present the timeline context to the user
**Why this matters:**
The timeline is the **killer feature** that mimics human memory. Without it, users get:
- ❌ A sorted list of relevant observations
- ❌ No context about what led there
- ❌ No context about what happened next
With timeline, users get:
- ✅ The MOST relevant observation
- ✅ Context: "You did A and B before this"
- ✅ Context: "After this, you did C and D"
- ✅ Complete narrative like human memory
### 📋 Documentation Status
**Recently Fixed (✅):**
- `/Users/alexnewman/Scripts/claude-mem/docs/context/mem-search-technical-architecture.md`
- Now describes 3-layer sequential flow
- Includes human memory analogy
- Positions ChromaDB as primary
**Landing Page (✅):**
- `/Users/alexnewman/Scripts/claude-mem-pro/src/components/landing/Features.tsx`
- `/Users/alexnewman/Scripts/claude-mem-pro/src/components/landing/QuickBenefits.tsx`
- `/Users/alexnewman/Scripts/claude-mem-pro/src/components/landing/Architecture.tsx`
- All updated to describe ChromaDB-first architecture
- "Remember Like a Human" messaging added
- Timeline feature highlighted
**Needs Review:**
- SKILL.md technical notes (line 172)
- All operation guides in `/operations/` directory
- Common workflows documentation
---
## Required Fixes
### Fix 1: Update Skill Operation Guides
**Files to modify:**
- `/Users/alexnewman/Scripts/claude-mem/plugin/skills/mem-search/operations/observations.md`
- `/Users/alexnewman/Scripts/claude-mem/plugin/skills/mem-search/operations/common-workflows.md`
**Changes needed:**
1. **observations.md:**
- Change title: "Search Observations (Full-Text)" → "Search Observations (Semantic + Temporal)"
- Update description: Explain ChromaDB semantic search as primary
- Update command examples to explain hybrid flow
- Add note: "Uses ChromaDB vector search with SQLite temporal ordering. FTS5 used as fallback."
2. **common-workflows.md:**
- Update "Workflow 2: Finding Specific Bug Fixes" to explain ChromaDB → SQLite flow
- Add new workflow: "Workflow N: Getting Timeline Context Around Relevant Observations"
**Example of corrected observations.md header:**
```markdown
# Search Observations (Semantic + Temporal)
Search observations using ChromaDB vector similarity with SQLite temporal ordering.
## Architecture
**3-Layer Hybrid Search:**
1. **ChromaDB semantic retrieval** - Finds what's semantically relevant (vector similarity)
2. **90-day recency filter** - Prioritizes recent work
3. **SQLite temporal ordering** - Sorts by time, returns newest relevant
**Fallback:** If ChromaDB unavailable, falls back to FTS5 keyword search.
## When to Use
- User asks: "How did we implement authentication?"
- User asks: "What bugs did we fix?"
- Looking for past work by meaning/topic (not just keywords)
```
### Fix 2: Implement Layer 3 (Instant Context Timeline)
**Option A: Add to existing search_observations handler**
Modify `/Users/alexnewman/Scripts/claude-mem/src/servers/search-server.ts` line ~396:
```typescript
// After getting sorted results, if user wants timeline context
if (results.length > 0 && options.includeTimeline) {
const topObservation = results[0];
const depth_before = options.timelineDepthBefore || 5;
const depth_after = options.timelineDepthAfter || 5;
// Get observations before and after
const timeline = store.getTimelineContext(
topObservation.id,
depth_before,
depth_after
);
return {
topResult: topObservation,
timeline: timeline,
format: format
};
}
```
**Option B: Use existing timeline-by-query operation**
The `/api/timeline/by-query` endpoint already implements search + timeline. Could:
1. Make it the DEFAULT recommended operation in skill guides
2. Update operation guides to emphasize this as primary workflow
3. Position observations search as "timeline-less" alternative
**Recommendation:** Option B is faster - leverage existing `timeline-by-query` endpoint and update skill guides to make it the primary workflow.
### Fix 3: Update SKILL.md Technical Notes
**File:** `/Users/alexnewman/Scripts/claude-mem/plugin/skills/mem-search/SKILL.md`
**Line 172:**
**Current:**
```markdown
- **Search engine:** FTS5 full-text search + structured filters
```
**Change to:**
```markdown
- **Search engine:** ChromaDB vector search (primary) + SQLite temporal ordering + instant context timeline (3-layer sequential architecture)
```
### Fix 4: Update search_observations Description
**File:** `/Users/alexnewman/Scripts/claude-mem/src/servers/search-server.ts`
**Line 349:**
**Current:**
```typescript
description: 'Search observations using full-text search across titles, narratives...'
```
**Change to:**
```typescript
description: 'Search observations using hybrid semantic search (ChromaDB vector similarity + SQLite temporal ordering). Falls back to FTS5 keyword search if ChromaDB unavailable. IMPORTANT: Always use index format first...'
```
**Line 351:**
**Current:**
```typescript
query: z.string().describe('Search query for FTS5 full-text search'),
```
**Change to:**
```typescript
query: z.string().describe('Search query (semantic vector search via ChromaDB, falls back to FTS5 if unavailable)'),
```
---
## Implementation Checklist
Use this checklist when executing fixes:
### Phase 1: Core Documentation
- [ ] Update `observations.md` title and description
- [ ] Update `observations.md` architecture explanation
- [ ] Update `observations.md` examples to mention ChromaDB
- [ ] Update `common-workflows.md` to explain hybrid flow
- [ ] Update `SKILL.md` line 172 technical notes
- [ ] Verify all operation guides mention ChromaDB correctly
### Phase 2: Backend Updates
- [ ] Update `search-server.ts` search_observations description (line 349)
- [ ] Update `search-server.ts` query parameter description (line 351)
- [ ] Add code comments explaining 3-layer flow
- [ ] Consider adding `includeTimeline` option to search_observations
### Phase 3: Timeline Integration
- [ ] Review timeline-by-query operation
- [ ] Update skill guides to recommend timeline-by-query as primary workflow
- [ ] Add example: "When you need context, use timeline-by-query instead of observations search"
- [ ] Update quick reference table in SKILL.md to highlight timeline-by-query
### Phase 4: Validation
- [ ] Test search behavior with ChromaDB enabled
- [ ] Test fallback behavior with ChromaDB disabled
- [ ] Verify skill guides accurately describe behavior
- [ ] Ensure landing page messaging aligns with skill guides
- [ ] Check that human memory analogy is consistent everywhere
---
## Key Messaging (Use Consistently)
### Value Proposition
"3-layer hybrid search mimics human memory: ChromaDB semantic retrieval finds what's relevant → SQLite temporal ordering identifies when → instant context timeline shows what led there and what came next."
### Technical Architecture
"ChromaDB vector search handles semantic understanding (what's relevant), SQLite handles temporal queries (when it happened, what's newest), and timeline context provides before/after observations (what led there, what happened next)."
### Why It Matters
"LLMs don't experience time linearly like humans do. Claude-mem gives them temporal context: not just 'you implemented authentication,' but 'you researched OAuth libraries, then implemented JWT auth, then fixed a token expiration bug.' Complete narrative, like human memory."
### ChromaDB Role
"ChromaDB is the PRIMARY search mechanism for semantic understanding. FTS5 is the FALLBACK for backward compatibility and reliability when ChromaDB is unavailable."
---
## Files Reference
**Skill Guides (Primary Fixes):**
- `/Users/alexnewman/Scripts/claude-mem/plugin/skills/mem-search/SKILL.md`
- `/Users/alexnewman/Scripts/claude-mem/plugin/skills/mem-search/operations/observations.md`
- `/Users/alexnewman/Scripts/claude-mem/plugin/skills/mem-search/operations/timeline-by-query.md`
- `/Users/alexnewman/Scripts/claude-mem/plugin/skills/mem-search/operations/common-workflows.md`
**Backend Code (Minor Updates):**
- `/Users/alexnewman/Scripts/claude-mem/src/servers/search-server.ts`
**Documentation (Validation):**
- `/Users/alexnewman/Scripts/claude-mem/docs/context/mem-search-technical-architecture.md`
**Landing Page (Already Fixed):**
- `/Users/alexnewman/Scripts/claude-mem-pro/src/components/landing/Features.tsx`
- `/Users/alexnewman/Scripts/claude-mem-pro/src/components/landing/QuickBenefits.tsx`
- `/Users/alexnewman/Scripts/claude-mem-pro/src/components/landing/Architecture.tsx`
---
## Questions for User (If Needed)
1. **Timeline Integration Approach:**
- Option A: Modify search_observations to add `includeTimeline` parameter
- Option B: Emphasize timeline-by-query as primary workflow in guides
- User preference?
2. **Backward Compatibility:**
- Should FTS5 fallback be MORE prominent in docs for older systems?
- Or keep it as "implementation detail"?
3. **Progressive Disclosure:**
- Should timeline context ALWAYS be included?
- Or only when user explicitly asks for context?
---
## Success Criteria
When these fixes are complete:
1. ✅ Skill operation guides accurately describe ChromaDB-first architecture
2. ✅ No references to "FTS5 as primary search method"
3. ✅ Timeline feature integrated into standard workflow
4. ✅ Human memory analogy present in key documentation
5. ✅ Consistent messaging across skill guides, docs, and landing page
6. ✅ Backend code comments explain 3-layer flow clearly
7. ✅ Users understand: "This is semantic search with temporal context, not just keyword search"
---
## Notes for Next Claude
- The user has already clarified the architecture thoroughly
- Backend code is already correct - focus on documentation/guides
- Landing page recently updated - validate for consistency
- Timeline-by-query endpoint already exists - leverage it
- Key insight: This mimics human memory through temporal context
- ChromaDB is PRIMARY, not optional. FTS5 is FALLBACK, not primary.
**Start with:** Reading this document fully, then update skill operation guides first (highest impact).
+614
View File
@@ -0,0 +1,614 @@
# Implementation Plan: ROI Metrics & Discovery Cost Tracking
**Feature**: Display token discovery costs alongside observations to demonstrate knowledge reuse ROI
**Branch**: `enhancement/roi`
**Issue**: #104
**Priority**: HIGH (needed for YC application amendment)
---
## Executive Summary
Capture token usage from Agent SDK, store as "discovery cost" with each observation, and display metrics in SessionStart context to prove that claude-mem reduces token consumption by 50-75% through knowledge reuse.
### The Value Proposition
**Session 1**: Claude spends 4,000 tokens discovering "how Stop hooks work"
**Sessions 2-5**: Claude reads 163-token observation instead of re-discovering
**Savings**: 15,348 tokens (77% reduction) over 5 sessions
This feature makes that ROI **visible and measurable** for both users and Claude.
---
## Architecture Overview
```
Agent SDK Messages (with usage)
SDKAgent captures usage data
ActiveSession tracks cumulative tokens
Observations stored with discovery_tokens
Context hook displays metrics
User/Claude sees ROI
```
---
## Implementation Steps
### Phase 1: Capture Token Usage from Agent SDK
**File**: `src/services/worker/SDKAgent.ts`
**Changes**:
1. Extract usage data from assistant messages (lines 64-86)
2. Track cumulative session tokens in ActiveSession
3. Pass cumulative tokens when storing observations
**Code Changes**:
```typescript
// Line ~70: After extracting textContent, add:
const usage = message.message.usage;
if (usage) {
session.cumulativeInputTokens += usage.input_tokens || 0;
session.cumulativeOutputTokens += usage.output_tokens || 0;
// Cache creation counts as discovery, cache read doesn't
if (usage.cache_creation_input_tokens) {
session.cumulativeInputTokens += usage.cache_creation_input_tokens;
}
logger.debug('SDK', 'Token usage captured', {
sessionId: session.sessionDbId,
inputTokens: usage.input_tokens,
outputTokens: usage.output_tokens,
cumulativeInput: session.cumulativeInputTokens,
cumulativeOutput: session.cumulativeOutputTokens
});
}
```
```typescript
// Line ~213-218: Pass discovery tokens when storing
const { id: obsId, createdAtEpoch } = this.dbManager.getSessionStore().storeObservation(
session.claudeSessionId,
session.project,
obs,
session.lastPromptNumber,
session.cumulativeInputTokens + session.cumulativeOutputTokens // Add discovery cost
);
```
**Edge Cases**:
- Handle missing usage data (default to 0)
- Cache tokens: `cache_creation_input_tokens` counts as discovery, `cache_read_input_tokens` doesn't
- Multiple observations per response: Each gets snapshot of cumulative tokens at creation time
---
### Phase 2: Update ActiveSession Type
**File**: `src/services/worker-types.ts`
**Changes**: Add token tracking fields to ActiveSession interface
```typescript
export interface ActiveSession {
sessionDbId: number;
sdkSessionId: string | null;
claudeSessionId: string;
project: string;
userPrompt: string;
lastPromptNumber: number;
pendingMessages: PendingMessage[];
abortController: AbortController;
startTime: number;
cumulativeInputTokens: number; // NEW: Track input tokens
cumulativeOutputTokens: number; // NEW: Track output tokens
}
```
**Initialization**: When creating new session in SessionManager.initializeSession, set:
```typescript
cumulativeInputTokens: 0,
cumulativeOutputTokens: 0
```
---
### Phase 3: Database Schema Migration
**File**: `src/services/sqlite/migrations.ts`
**Add Migration**: Create migration #8 (next available number)
```typescript
{
version: 8,
name: 'add_discovery_tokens',
up: (db: Database) => {
// Add discovery_tokens to observations
db.exec(`
ALTER TABLE observations
ADD COLUMN discovery_tokens INTEGER DEFAULT 0;
`);
// Add discovery_tokens to summaries
db.exec(`
ALTER TABLE summaries
ADD COLUMN discovery_tokens INTEGER DEFAULT 0;
`);
logger.info('DB', 'Migration 8: Added discovery_tokens columns');
}
}
```
**Why summaries too?**: Summaries represent accumulated session work, so they should also show total discovery cost.
---
### Phase 4: Update SessionStore
**File**: `src/services/sqlite/SessionStore.ts`
**Changes**:
1. Update `storeObservation` signature (around line ~1000):
```typescript
storeObservation(
sessionId: string,
project: string,
observation: ParsedObservation,
promptNumber: number,
discoveryTokens: number = 0 // NEW parameter
): { id: number; createdAtEpoch: number }
```
2. Update INSERT statement to include discovery_tokens:
```typescript
const stmt = this.db.prepare(`
INSERT INTO observations (
session_id,
project,
type,
title,
subtitle,
narrative,
facts,
concepts,
files_read,
files_modified,
prompt_number,
discovery_tokens, -- NEW
created_at_epoch
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`);
const result = stmt.run(
sessionId,
project,
observation.type,
observation.title,
observation.subtitle || '',
observation.narrative || '',
JSON.stringify(observation.facts || []),
JSON.stringify(observation.concepts || []),
JSON.stringify(observation.files || []),
JSON.stringify([]),
promptNumber,
discoveryTokens, // NEW
createdAtEpoch
);
```
3. Update `storeSummary` similarly (around line ~1150):
```typescript
storeSummary(
sessionId: string,
project: string,
summary: ParsedSummary,
promptNumber: number,
discoveryTokens: number = 0 // NEW parameter
): { id: number; createdAtEpoch: number }
```
---
### Phase 5: Update Database Types
**File**: `src/services/sqlite/types.ts`
**Changes**: Add discovery_tokens to DBObservation and DBSummary interfaces
```typescript
export interface DBObservation {
id: number;
session_id: string;
project: string;
type: 'decision' | 'bugfix' | 'feature' | 'refactor' | 'discovery' | 'change';
title: string;
subtitle: string;
narrative: string | null;
facts: string; // JSON array
concepts: string; // JSON array
files_read: string; // JSON array
files_modified: string; // JSON array
prompt_number: number;
discovery_tokens: number; // NEW
created_at_epoch: number;
}
export interface DBSummary {
id: number;
session_id: string;
request: string;
investigated: string | null;
learned: string | null;
completed: string | null;
next_steps: string | null;
notes: string | null;
project: string;
prompt_number: number;
discovery_tokens: number; // NEW
created_at_epoch: number;
}
```
---
### Phase 6: Update Search Queries
**File**: `src/services/sqlite/SessionSearch.ts`
**Changes**: Ensure all SELECT queries include discovery_tokens
Example (around line ~50, searchObservations):
```typescript
SELECT
o.id,
o.session_id,
o.project,
o.type,
o.title,
o.subtitle,
o.narrative,
o.facts,
o.concepts,
o.files_read,
o.files_modified,
o.prompt_number,
o.discovery_tokens, -- NEW
o.created_at_epoch,
...
```
**Affected methods**:
- `searchObservations`
- `getRecentObservations`
- `getObservationsByType`
- `getObservationsByConcept`
- `getObservationsByFile`
- All other observation query methods
---
### Phase 7: Update Context Hook Display
**File**: `src/hooks/context-hook.ts`
**Changes**: Display discovery costs and ROI metrics in SessionStart context
**Section 1: Add Aggregate Metrics** (insert after line ~250, before observation table)
```typescript
// Calculate aggregate metrics
const totalObservations = observations.length;
const totalReadTokens = observations.reduce((sum, obs) => {
// Estimate read tokens from observation size
const obsSize = (obs.title?.length || 0) +
(obs.subtitle?.length || 0) +
(obs.narrative?.length || 0) +
JSON.stringify(obs.facts || []).length;
return sum + Math.ceil(obsSize / 4); // Rough token estimate
}, 0);
const totalDiscoveryTokens = observations.reduce((sum, obs) => sum + (obs.discovery_tokens || 0), 0);
const savings = totalDiscoveryTokens - totalReadTokens;
const savingsPercent = totalDiscoveryTokens > 0
? Math.round((savings / totalDiscoveryTokens) * 100)
: 0;
output += `\n📊 **Context Economics**:\n`;
output += ` - Loading: ${totalObservations} observations (${totalReadTokens.toLocaleString()} tokens to read)\n`;
output += ` - Discovery investment: ${totalDiscoveryTokens.toLocaleString()} tokens spent by previous sessions\n`;
output += ` - Your savings: ${savings.toLocaleString()} tokens (${savingsPercent}% reduction from reuse)\n\n`;
```
**Section 2: Update Table Format** (around line ~280)
```typescript
output += `| ID | Time | T | Title | Read | Discovery |\n`;
output += `|----|------|---|-------|------|-----------||\n`;
for (const obs of observations) {
const time = formatTime(obs.created_at_epoch);
const emoji = getTypeEmoji(obs.type);
const title = truncate(obs.title, 50);
// Estimate read tokens (observation size in tokens)
const obsSize = (obs.title?.length || 0) +
(obs.subtitle?.length || 0) +
(obs.narrative?.length || 0) +
JSON.stringify(obs.facts || []).length;
const readTokens = Math.ceil(obsSize / 4);
const discoveryTokens = obs.discovery_tokens || 0;
const discoveryDisplay = discoveryTokens > 0
? `🔍 ${discoveryTokens.toLocaleString()}`
: '-';
output += `| #${obs.id} | ${time} | ${emoji} | ${title} | ~${readTokens} | ${discoveryDisplay} |\n`;
}
```
**Section 3: Add Footer Explanation** (after table)
```typescript
output += `\n💡 **Column Key**:\n`;
output += ` - **Read**: Tokens to read this observation (cost to learn it now)\n`;
output += ` - **Discovery**: Tokens Previous Claude spent exploring/researching this topic\n`;
output += `\n**ROI**: Reading these learnings instead of re-discovering saves ${savingsPercent}% tokens\n`;
```
**Edge Case**: Handle old observations without discovery_tokens (show '-' or 0)
---
### Phase 8: Update Chroma Sync (Optional)
**File**: `src/services/sync/ChromaSync.ts`
**Changes**: Include discovery_tokens in vector metadata
```typescript
// Around line ~100, syncObservation metadata
metadata: {
session_id: sessionId,
project: project,
type: observation.type,
title: observation.title,
prompt_number: promptNumber,
discovery_tokens: discoveryTokens, // NEW
created_at_epoch: createdAtEpoch,
...
}
```
**Why?**: Enables semantic search to factor in discovery cost for relevance scoring (future enhancement)
---
## Testing Plan
### Unit Tests
1. **Token Capture Test**:
- Mock Agent SDK response with usage data
- Verify ActiveSession.cumulativeTokens increments correctly
- Test cache token handling (creation counts, read doesn't)
2. **Storage Test**:
- Create observation with discovery_tokens
- Verify database stores correctly
- Query back and verify field present
3. **Display Test**:
- Create test observations with varying discovery costs
- Run context-hook
- Verify metrics calculate correctly
- Verify table displays both Read and Discovery columns
### Integration Tests
1. **Full Session Flow**:
- Start new session
- Trigger multiple tool executions
- Generate observations
- Verify cumulative tokens accumulate
- Check context displays metrics
2. **Migration Test**:
- Backup existing database
- Run migration #8
- Verify columns added
- Verify existing data intact (discovery_tokens = 0)
- Test new observations store correctly
### Manual Testing
1. **Real Usage Scenario**:
- Start fresh Claude Code session
- Perform research task (read files, search codebase)
- Generate observations via claude-mem
- Check database for discovery_tokens values
- Start new session, verify context shows metrics
2. **YC Demo Data**:
- Run 5 sessions on same topic
- Collect token data for each session
- Calculate actual ROI (Session 1 cost vs Sessions 2-5)
- Screenshot metrics for YC application
---
## Rollout Plan
### Phase 1: Data Collection (Week 1)
- Deploy migration and token capture
- Run without displaying metrics yet
- Verify data quality and accuracy
- Fix any issues with token tracking
### Phase 2: Display Metrics (Week 2)
- Enable context hook display
- Gather user feedback
- Iterate on presentation format
- Document any edge cases
### Phase 3: YC Application (Week 2-3)
- Collect empirical data from real usage
- Generate charts/graphs showing ROI
- Write case study with actual numbers
- Amend YC application with proof
### Phase 4: Public Launch (Week 4)
- Blog post explaining the feature
- Update README with ROI metrics
- Submit to HN/Reddit with data
- Reach out to Anthropic with findings
---
## Success Metrics
**Technical Success**:
- ✅ Token capture accuracy: >95% of SDK responses captured
- ✅ Database migration: 0 data loss, all observations migrated
- ✅ Display accuracy: Metrics match raw data within 5%
**Business Success**:
- ✅ Demonstrate 50-75% token reduction across 10+ sessions
- ✅ YC application strengthened with empirical data
- ✅ User/Claude understanding of ROI improves (survey/feedback)
**Strategic Success**:
- ✅ Proof that memory optimization reduces infrastructure needs
- ✅ Data compelling enough for Anthropic partnership discussion
- ✅ Foundation for enterprise licensing ROI calculator
---
## Open Questions
1. **Token Attribution**:
- Should each observation get cumulative session tokens, or split proportionally?
- **Decision**: Use cumulative (simpler, shows total cost at that point)
2. **Cache Tokens**:
- How to handle cache_read_input_tokens in ROI calculation?
- **Decision**: Don't count cache reads as discovery (they're already discovered)
3. **Display Format**:
- Show raw token counts or human-readable format (K, M)?
- **Decision**: Use toLocaleString() for readability (e.g., "4,000" not "4K")
4. **Pricing Display**:
- Should we show dollar costs too, or just tokens?
- **Decision**: Tokens only initially. Pricing varies by model/plan, adds complexity
5. **Historical Data**:
- What to do with old observations without discovery_tokens?
- **Decision**: Show as 0 or '-', document limitation
---
## Files Modified Summary
**Core Implementation**:
- `src/services/worker/SDKAgent.ts` - Capture usage, pass to storage
- `src/services/worker-types.ts` - Add cumulative token fields
- `src/services/sqlite/migrations.ts` - Migration #8 for discovery_tokens
- `src/services/sqlite/SessionStore.ts` - Store discovery tokens
- `src/services/sqlite/types.ts` - Update interfaces
- `src/services/sqlite/SessionSearch.ts` - Include in queries
- `src/hooks/context-hook.ts` - Display metrics
**Optional**:
- `src/services/sync/ChromaSync.ts` - Include in vector metadata
- `src/services/worker/SessionManager.ts` - Initialize cumulative tokens
**Documentation**:
- `CLAUDE.md` - Update with new feature
- `README.md` - Add ROI metrics section
- Issue #104 - Track implementation progress
---
## Timeline Estimate
**Day 1** (Tomorrow):
- [ ] Create branch ✅
- [ ] Write implementation plan ✅
- [ ] Phase 1: Capture token usage (2 hours)
- [ ] Phase 2: Update types (30 min)
- [ ] Phase 3: Database migration (1 hour)
**Day 2**:
- [ ] Phase 4: Update SessionStore (1 hour)
- [ ] Phase 5: Update types (30 min)
- [ ] Phase 6: Update search queries (1 hour)
- [ ] Testing: Unit tests (2 hours)
**Day 3**:
- [ ] Phase 7: Update context hook display (2 hours)
- [ ] Testing: Integration tests (2 hours)
- [ ] Manual testing and iteration (2 hours)
**Day 4**:
- [ ] Collect real usage data (ongoing throughout day)
- [ ] Generate YC metrics/charts (2 hours)
- [ ] Amend YC application (2 hours)
- [ ] Documentation updates (1 hour)
**Total**: ~20 hours of development over 4 days
---
## Risk Mitigation
**Risk 1**: Agent SDK usage data incomplete or missing
**Mitigation**: Default to 0, log warnings, don't break existing functionality
**Risk 2**: Migration fails on large databases
**Mitigation**: Test on database copy first, add rollback mechanism
**Risk 3**: Token estimates inaccurate
**Mitigation**: Document methodology, provide "rough estimate" disclaimer
**Risk 4**: Display too noisy/overwhelming
**Mitigation**: Make display configurable via settings, start collapsed
**Risk 5**: YC data not compelling enough
**Mitigation**: Run on diverse projects, cherry-pick best examples, be honest about limitations
---
## Next Steps
1. ✅ Create branch `enhancement/roi`
2. ✅ Write implementation plan
3. Start Phase 1: Implement token capture in SDKAgent.ts
4. Run manual test to verify usage data captured
5. Continue through phases sequentially
6. Collect data for YC application by end of week
---
## Notes for Tomorrow
**Start here**: `src/services/worker/SDKAgent.ts` line 64-86
**Key insight**: `message.message.usage` contains the token data
**Don't forget**: Initialize cumulative tokens to 0 in SessionManager
**Test with**: Simple session that reads a few files and creates 1-2 observations
**The goal**: By end of week, have real numbers showing 50-75% token savings to prove the hypothesis and strengthen YC application.
---
*This plan represents ~20 hours of focused development. Prioritize getting Phase 1-7 working correctly over perfection. The YC data is the critical deliverable.*
+107 -71
View File
@@ -2,9 +2,9 @@
<br>
<a href="https://github.com/thedotmack/claude-mem">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/claude-mem-logo-for-dark-mode.webp">
<source media="(prefers-color-scheme: light)" srcset="docs/claude-mem-logo-for-light-mode.webp">
<img src="docs/claude-mem-logo-for-light-mode.webp" alt="Claude-Mem" width="400">
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/thedotmack/claude-mem/main/docs/public/claude-mem-logo-for-dark-mode.webp">
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/thedotmack/claude-mem/main/docs/public/claude-mem-logo-for-light-mode.webp">
<img src="https://raw.githubusercontent.com/thedotmack/claude-mem/main/docs/public/claude-mem-logo-for-light-mode.webp" alt="Claude-Mem" width="400">
</picture>
</a>
<br>
@@ -17,7 +17,7 @@
<img src="https://img.shields.io/badge/License-AGPL%203.0-blue.svg" alt="License">
</a>
<a href="package.json">
<img src="https://img.shields.io/badge/version-5.1.2-green.svg" alt="Version">
<img src="https://img.shields.io/badge/version-6.0.0-green.svg" alt="Version">
</a>
<a href="package.json">
<img src="https://img.shields.io/badge/node-%3E%3D18.0.0-brightgreen.svg" alt="Node">
@@ -27,6 +27,16 @@
</a>
</p>
<br>
<p align="center">
<a href="https://github.com/thedotmack/claude-mem">
<picture>
<img src="https://raw.githubusercontent.com/thedotmack/claude-mem/main/docs/public/cm-preview.gif" alt="Claude-Mem Preview" width="800">
</picture>
</a>
</p>
<p align="center">
<a href="#quick-start">Quick Start</a> •
<a href="#how-it-works">How It Works</a> •
@@ -56,9 +66,10 @@ Start a new Claude Code session in the terminal and enter the following commands
Restart Claude Code. Context from previous sessions will automatically appear in new sessions.
**Key Features:**
- 🧠 **Persistent Memory** - Context survives across sessions
- 📊 **Progressive Disclosure** - Layered memory retrieval with token cost visibility
- 🔍 **9 Search Tools** - Query your project history via MCP
- 🔍 **Skill-Based Search** - Query your project history with mem-search skill (~2,250 token savings)
- 🖥️ **Web Viewer UI** - Real-time memory stream at http://localhost:37777
- 🤖 **Automatic Operation** - No manual intervention required
- 🔗 **Citations** - Reference past decisions with `claude-mem://` URIs
@@ -70,34 +81,38 @@ Restart Claude Code. Context from previous sessions will automatically appear in
📚 **[View Full Documentation](docs/)** - Browse markdown docs on GitHub
💻 **Local Preview**: Run Mintlify docs locally:
```bash
cd docs
npx mintlify dev
```
### Getting Started
- **[Installation Guide](docs/installation.mdx)** - Quick start & advanced installation
- **[Usage Guide](docs/usage/getting-started.mdx)** - How Claude-Mem works automatically
- **[MCP Search Tools](docs/usage/search-tools.mdx)** - Query your project history
- **[Installation Guide](https://docs.claude-mem.ai/installation)** - Quick start & advanced installation
- **[Usage Guide](https://docs.claude-mem.ai/usage/getting-started)** - How Claude-Mem works automatically
- **[Search Tools](https://docs.claude-mem.ai/usage/search-tools)** - Query your project history with natural language
### Best Practices
- **[Context Engineering](docs/context-engineering.mdx)** - AI agent context optimization principles
- **[Progressive Disclosure](docs/progressive-disclosure.mdx)** - Philosophy behind Claude-Mem's context priming strategy
- **[Context Engineering](https://docs.claude-mem.ai/context-engineering)** - AI agent context optimization principles
- **[Progressive Disclosure](https://docs.claude-mem.ai/progressive-disclosure)** - Philosophy behind Claude-Mem's context priming strategy
### Architecture
- **[Overview](docs/architecture/overview.mdx)** - System components & data flow
- **[Architecture Evolution](docs/architecture-evolution.mdx)** - The journey from v3 to v5
- **[Hooks Architecture](docs/hooks-architecture.mdx)** - How Claude-Mem uses lifecycle hooks
- **[Hooks Reference](docs/architecture/hooks.mdx)** - 7 hook scripts explained
- **[Worker Service](docs/architecture/worker-service.mdx)** - HTTP API & PM2 management
- **[Database](docs/architecture/database.mdx)** - SQLite schema & FTS5 search
- **[MCP Search](docs/architecture/mcp-search.mdx)** - 9 search tools & examples
- **[Viewer UI](docs/VIEWER.md)** - Web-based memory stream visualization
- **[Overview](https://docs.claude-mem.ai/architecture/overview)** - System components & data flow
- **[Architecture Evolution](https://docs.claude-mem.ai/architecture-evolution)** - The journey from v3 to v5
- **[Hooks Architecture](https://docs.claude-mem.ai/hooks-architecture)** - How Claude-Mem uses lifecycle hooks
- **[Hooks Reference](https://docs.claude-mem.ai/architecture/hooks)** - 7 hook scripts explained
- **[Worker Service](https://docs.claude-mem.ai/architecture/worker-service)** - HTTP API & PM2 management
- **[Database](https://docs.claude-mem.ai/architecture/database)** - SQLite schema & FTS5 search
- **[Search Architecture](https://docs.claude-mem.ai/architecture/search-architecture)** - Hybrid search with Chroma vector database
### Configuration & Development
- **[Configuration](docs/configuration.mdx)** - Environment variables & settings
- **[Development](docs/development.mdx)** - Building, testing, contributing
- **[Troubleshooting](docs/troubleshooting.mdx)** - Common issues & solutions
- **[Configuration](https://docs.claude-mem.ai/configuration)** - Environment variables & settings
- **[Development](https://docs.claude-mem.ai/development)** - Building, testing, contributing
- **[Troubleshooting](https://docs.claude-mem.ai/troubleshooting)** - Common issues & solutions
---
@@ -126,69 +141,77 @@ npx mintlify dev
```
**Core Components:**
1. **7 Lifecycle Hook Scripts** - smart-install, context-hook, user-message-hook, new-hook, save-hook, summary-hook, cleanup-hook
2. **Worker Service** - HTTP API on port 37777 with web viewer UI, managed by PM2
3. **SQLite Database** - Stores sessions, observations, summaries with FTS5 full-text search
4. **9 MCP Search Tools** - Query historical context with citations and timeline analysis
5. **Chroma Vector Database** - Hybrid semantic + keyword search for intelligent context retrieval
See [Architecture Overview](docs/architecture/overview.mdx) for details.
1. **6 Lifecycle Hooks** - context-hook, user-message-hook, new-hook, save-hook, summary-hook, cleanup-hook
2. **Smart Install** - Cached dependency checker (pre-hook script, not a lifecycle hook)
3. **Worker Service** - HTTP API on port 37777 with web viewer UI and 10 search endpoints, managed by PM2
4. **SQLite Database** - Stores sessions, observations, summaries with FTS5 full-text search
5. **mem-search Skill** - Natural language queries with progressive disclosure (~2,250 token savings vs MCP)
6. **Chroma Vector Database** - Hybrid semantic + keyword search for intelligent context retrieval
See [Architecture Overview](https://docs.claude-mem.ai/architecture/overview) for details.
---
## MCP Search Tools
## mem-search Skill
Claude-Mem provides 9 specialized search tools:
Claude-Mem provides intelligent search through the mem-search skill that auto-invokes when you ask about past work:
1. **search_observations** - Full-text search across observations
2. **search_sessions** - Full-text search across session summaries
3. **search_user_prompts** - Search raw user requests
4. **find_by_concept** - Find by concept tags (discovery, problem-solution, pattern, etc.)
5. **find_by_file** - Find observations referencing specific files
6. **find_by_type** - Find by type (decision, bugfix, feature, refactor, discovery, change)
7. **get_recent_context** - Get recent session context for a project
8. **get_context_timeline** - Get unified timeline of context around a specific point in time
9. **get_timeline_by_query** - Search for observations and get timeline context around best match
**How It Works:**
- Just ask naturally: *"What did we do last session?"* or *"Did we fix this bug before?"*
- Claude automatically invokes the mem-search skill to find relevant context
- ~2,250 token savings per session start vs MCP approach
**Available Search Operations:**
1. **Search Observations** - Full-text search across observations
2. **Search Sessions** - Full-text search across session summaries
3. **Search Prompts** - Search raw user requests
4. **By Concept** - Find by concept tags (discovery, problem-solution, pattern, etc.)
5. **By File** - Find observations referencing specific files
6. **By Type** - Find by type (decision, bugfix, feature, refactor, discovery, change)
7. **Recent Context** - Get recent session context for a project
8. **Timeline** - Get unified timeline of context around a specific point in time
9. **Timeline by Query** - Search for observations and get timeline context around best match
10. **API Help** - Get search API documentation
**Example Natural Language Queries:**
**Example Queries:**
```
search_observations with query="authentication" and type="decision"
find_by_file with filePath="worker-service.ts"
search_user_prompts with query="add dark mode"
get_recent_context with limit=5
get_context_timeline with anchor="S890" depth_before=10 depth_after=10
get_timeline_by_query with query="viewer UI implementation" mode="auto"
"What bugs did we fix last session?"
"How did we implement authentication?"
"What changes were made to worker-service.ts?"
"Show me recent work on this project"
"What was happening when we added the viewer UI?"
```
See [MCP Search Tools Guide](docs/usage/search-tools.mdx) for detailed examples.
See [Search Tools Guide](https://docs.claude-mem.ai/usage/search-tools) for detailed examples.
---
## What's New in v5.1.2
## What's New in v6.0.0
**🎨 Theme Toggle (v5.1.2):**
- Light/dark mode support in viewer UI
- System preference detection
- Persistent theme settings across sessions
- Smooth transitions between themes
**🚀 Major Session Management & Transcript Processing Improvements:**
**🖥️ Web-Based Viewer UI (v5.1.0):**
- Real-time memory stream visualization at http://localhost:37777
- Server-Sent Events (SSE) for instant updates
- Infinite scroll pagination with automatic deduplication
- Project filtering to focus on specific codebases
- Settings persistence (sidebar state, selected project)
- Auto-reconnection with exponential backoff
- **Enhanced Session Initialization**: Accept userPrompt and promptNumber for better context tracking
- **Live UserPrompt Updates**: Multi-turn conversation support with real-time prompt tracking
- **Improved SessionManager**: Better context handling and observation processing
- **Comprehensive Transcript Processing**: New scripts and utilities for analyzing Claude Code transcripts
- **Rich Context Extraction**: Advanced parsing utilities for extracting meaningful context from sessions
- **Refactored Architecture**: Improved hooks and SDKAgent for more reliable observation handling
- **Silent Debug Logging**: Better debugging capabilities without cluttering output
- **Enhanced Error Handling**: More robust error recovery and debugging tools
**⚡ Smart Install Caching (v5.0.3):**
- Eliminated redundant npm installs on every session (2-5s → 10ms)
- Caches version in `.install-version` file
- Only runs npm install when needed (first time, version change, missing deps)
**Breaking Changes**: Significant architectural changes in session management and observation handling. Existing sessions continue to work, but internal APIs have evolved.
**🔍 Hybrid Search Architecture (v5.0.0):**
- Chroma vector database for semantic search
- Combined with FTS5 keyword search
- Intelligent context retrieval with 90-day recency filtering
**Previous Highlights:**
- **v5.5.0**: mem-search skill enhancement with 100% effectiveness rate
- **v5.4.0**: Skill-based search architecture (~2,250 tokens saved per session)
- **v5.1.2**: Theme toggle for light/dark mode in viewer UI
- **v5.1.0**: Web-based viewer UI with real-time updates
- **v5.0.3**: Smart install caching (2-5s → 10ms)
- **v5.0.0**: Hybrid search with Chroma vector database
See [CHANGELOG.md](CHANGELOG.md) for complete version history.
@@ -206,6 +229,7 @@ See [CHANGELOG.md](CHANGELOG.md) for complete version history.
## Key Benefits
### Progressive Disclosure Context
- **Layered memory retrieval** mirrors human memory patterns
- **Layer 1 (Index)**: See what observations exist with token costs at session start
- **Layer 2 (Details)**: Fetch full narratives on-demand via MCP search
@@ -214,21 +238,25 @@ See [CHANGELOG.md](CHANGELOG.md) for complete version history.
- **Type indicators**: Visual cues (🔴 critical, 🟤 decision, 🔵 informational) highlight observation importance
### Automatic Memory
- Context automatically injected when Claude starts
- No manual commands or configuration needed
- Works transparently in the background
### Full History Search
- Search across all sessions and observations
- FTS5 full-text search for fast queries
- Citations link back to specific observations
### Structured Observations
- AI-powered extraction of learnings
- Categorized by type (decision, bugfix, feature, etc.)
- Tagged with concepts and file references
### Multi-Prompt Sessions
- Sessions span multiple user prompts
- Context preserved across `/clear` commands
- Track entire conversation threads
@@ -238,16 +266,18 @@ See [CHANGELOG.md](CHANGELOG.md) for complete version history.
## Configuration
**Model Selection:**
```bash
./claude-mem-settings.sh
```
**Environment Variables:**
- `CLAUDE_MEM_MODEL` - AI model for processing (default: claude-sonnet-4-5)
- `CLAUDE_MEM_WORKER_PORT` - Worker port (default: 37777)
- `CLAUDE_MEM_DATA_DIR` - Data directory override (dev only)
See [Configuration Guide](docs/configuration.mdx) for details.
See [Configuration Guide](https://docs.claude-mem.ai/configuration) for details.
---
@@ -270,19 +300,24 @@ npm run worker:start
npm run worker:logs
```
See [Development Guide](docs/development.mdx) for detailed instructions.
See [Development Guide](https://docs.claude-mem.ai/development) for detailed instructions.
---
## Troubleshooting
**Quick Diagnostic:**
If you're experiencing issues, describe the problem to Claude and the troubleshoot skill will automatically activate to diagnose and provide fixes.
**Common Issues:**
- Worker not starting → `npm run worker:restart`
- No context appearing → `npm run test:context`
- Database issues → `sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;"`
- Search not working → Check FTS5 tables exist
See [Troubleshooting Guide](docs/troubleshooting.mdx) for complete solutions.
See [Troubleshooting Guide](https://docs.claude-mem.ai/troubleshooting) for complete solutions.
---
@@ -296,7 +331,7 @@ Contributions are welcome! Please:
4. Update documentation
5. Submit a Pull Request
See [Development Guide](docs/development.mdx) for contribution workflow.
See [Development Guide](https://docs.claude-mem.ai/development) for contribution workflow.
---
@@ -309,6 +344,7 @@ Copyright (C) 2025 Alex Newman (@thedotmack). All rights reserved.
See the [LICENSE](LICENSE) file for full details.
**What This Means:**
- You can use, modify, and distribute this software freely
- If you modify and deploy on a network server, you must make your source code available
- Derivative works must also be licensed under AGPL-3.0
-447
View File
@@ -1,447 +0,0 @@
---
title: "MCP Search Server"
description: "9 search tools with examples and usage patterns"
---
# MCP Search Server
Claude-Mem includes a Model Context Protocol (MCP) server that exposes 9 specialized search tools for querying stored observations and sessions.
## Overview
- **Location**: `src/servers/search-server.ts`
- **Built Output**: `plugin/scripts/search-server.mjs`
- **Configuration**: `plugin/.mcp.json`
- **Transport**: stdio
- **Tools**: 9 specialized search functions
- **Citations**: All results use `claude-mem://` URI scheme
## Configuration
The MCP server is automatically registered via `plugin/.mcp.json`:
```json
{
"mcpServers": {
"claude-mem-search": {
"type": "stdio",
"command": "${CLAUDE_PLUGIN_ROOT}/scripts/search-server.mjs"
}
}
}
```
This registers the `claude-mem-search` server with Claude Code, making the 9 search tools available in all sessions. The server is automatically started when Claude Code launches and communicates via stdio transport.
## Search Tools
### 1. search_observations
Full-text search across observation titles, narratives, facts, and concepts.
**Parameters**:
- `query` (required): Search query for FTS5 full-text search
- `type`: Filter by observation type(s) (decision, bugfix, feature, refactor, discovery, change)
- `concepts`: Filter by concept tags
- `files`: Filter by file paths (partial match)
- `project`: Filter by project name
- `dateRange`: Filter by date range (`{start, end}`)
- `orderBy`: Sort order (relevance, date_desc, date_asc)
- `limit`: Maximum results (default: 20, max: 100)
- `offset`: Number of results to skip
- `format`: Output format ("index" for titles/dates only, "full" for complete details)
**Example**:
```
search_observations with query="build system" and type="decision"
```
### 2. search_sessions
Full-text search across session summaries, requests, and learnings.
**Parameters**:
- `query` (required): Search query for FTS5 full-text search
- `project`: Filter by project name
- `dateRange`: Filter by date range
- `orderBy`: Sort order (relevance, date_desc, date_asc)
- `limit`: Maximum results (default: 20, max: 100)
- `offset`: Number of results to skip
- `format`: Output format ("index" or "full")
**Example**:
```
search_sessions with query="hooks implementation"
```
### 3. search_user_prompts
Search raw user prompts with full-text search. Use this to find what the user actually said/requested across all sessions.
**Parameters**:
- `query` (required): Search query for FTS5 full-text search
- `project`: Filter by project name
- `dateRange`: Filter by date range
- `orderBy`: Sort order (relevance, date_desc, date_asc)
- `limit`: Maximum results (default: 20, max: 100)
- `offset`: Number of results to skip
- `format`: Output format ("index" for truncated prompts/dates, "full" for complete prompt text)
**Example**:
```
search_user_prompts with query="authentication feature"
```
**Benefits**:
- Full context reconstruction from user intent → Claude actions → outcomes
- Pattern detection for repeated requests
- Improved debugging by tracing from original user words to final implementation
### 4. find_by_concept
Find observations tagged with specific concepts.
**Parameters**:
- `concept` (required): Concept tag to search for
- `project`: Filter by project name
- `dateRange`: Filter by date range
- `orderBy`: Sort order (relevance, date_desc, date_asc)
- `limit`: Maximum results (default: 20, max: 100)
- `offset`: Number of results to skip
- `format`: Output format ("index" or "full")
**Example**:
```
find_by_concept with concept="architecture"
```
### 5. find_by_file
Find observations and sessions that reference specific file paths.
**Parameters**:
- `filePath` (required): File path to search for (supports partial matching)
- `project`: Filter by project name
- `dateRange`: Filter by date range
- `orderBy`: Sort order (relevance, date_desc, date_asc)
- `limit`: Maximum results (default: 20, max: 100)
- `offset`: Number of results to skip
- `format`: Output format ("index" or "full")
**Example**:
```
find_by_file with filePath="worker-service.ts"
```
### 6. find_by_type
Find observations by type (decision, bugfix, feature, refactor, discovery, change).
**Parameters**:
- `type` (required): Observation type(s) to filter by (single type or array)
- `project`: Filter by project name
- `dateRange`: Filter by date range
- `orderBy`: Sort order (relevance, date_desc, date_asc)
- `limit`: Maximum results (default: 20, max: 100)
- `offset`: Number of results to skip
- `format`: Output format ("index" or "full")
**Example**:
```
find_by_type with type=["decision", "feature"]
```
### 7. get_recent_context
Get recent session context including summaries and observations for a project.
**Parameters**:
- `project`: Project name (defaults to current working directory basename)
- `limit`: Number of recent sessions to retrieve (default: 3, max: 10)
**Example**:
```
get_recent_context with limit=5
```
### 8. get_context_timeline
Get a unified timeline of context (observations, sessions, and prompts) around a specific point in time. All record types are interleaved chronologically.
**Parameters**:
- `anchor` (required): Anchor point - observation ID, session ID (e.g., "S123"), or ISO timestamp
- `depth_before` (default: 10): Number of records to retrieve before anchor (max: 50)
- `depth_after` (default: 10): Number of records to retrieve after anchor (max: 50)
- `project`: Filter by project name
**Return Format**:
Returns `depth_before` records + anchor + `depth_after` records, all interleaved chronologically. Total records: `depth_before + 1 + depth_after`.
**Use Case**: Understanding "what was happening when X occurred"
**Example**:
```
# Timeline around observation #123
get_context_timeline with anchor=123 and depth_before=5 and depth_after=5
# Timeline around a session
get_context_timeline with anchor="S456" and depth_before=10 and depth_after=10
# Timeline around a timestamp
get_context_timeline with anchor="2025-11-06T10:30:00Z" and depth_before=15 and depth_after=5
```
**Response Structure**:
```json
{
"timeline": [
{
"type": "observation",
"id": 120,
"title": "Context before",
"created_at": "2025-11-06T10:25:00Z"
},
{
"type": "user-prompt",
"id": 45,
"prompt": "User request",
"created_at": "2025-11-06T10:28:00Z"
},
{
"type": "observation",
"id": 123,
"title": "Anchor observation",
"created_at": "2025-11-06T10:30:00Z",
"isAnchor": true
},
{
"type": "session",
"id": "S456",
"request": "Session summary",
"created_at": "2025-11-06T10:32:00Z"
}
],
"anchor": {
"type": "observation",
"id": 123
}
}
```
### 9. get_timeline_by_query
Search for observations using natural language and get timeline context around the best match. Combines search + timeline into a single operation.
**Parameters**:
- `query` (required): Natural language search query to find relevant observations
- `mode` (default: "auto"): Operation mode
- `"auto"`: Automatically use top search result as timeline anchor
- `"interactive"`: Return top N search results for manual anchor selection
- `depth_before` (default: 10): Number of timeline records before anchor (max: 50)
- `depth_after` (default: 10): Number of timeline records after anchor (max: 50)
- `limit` (default: 5): For interactive mode - number of top search results to display (max: 20)
- `project`: Filter by project name
**Use Case**: Faster context discovery - "show me what happened around when we fixed the authentication bug"
**Example - Auto Mode**:
```
# Automatically find and show timeline for "authentication bug"
get_timeline_by_query with query="authentication bug" and mode="auto" and depth_before=10 and depth_after=10
```
**Example - Interactive Mode**:
```
# Show top 5 matches, let user choose anchor
get_timeline_by_query with query="authentication bug" and mode="interactive" and limit=5
```
**Auto Mode Response**:
```json
{
"search_result": {
"id": 123,
"title": "Fix authentication bug",
"relevance": 0.95
},
"timeline": [
/* timeline records before and after observation 123 */
]
}
```
**Interactive Mode Response**:
```json
{
"top_results": [
{
"id": 123,
"title": "Fix authentication bug",
"relevance": 0.95,
"created_at": "2025-11-06T10:30:00Z"
},
{
"id": 98,
"title": "Authentication refactor",
"relevance": 0.82,
"created_at": "2025-11-05T14:20:00Z"
}
],
"message": "Select an observation ID to view its timeline context"
}
```
## Output Formats
All search tools support two output formats:
### Index Format (Default)
Returns titles, dates, and source URIs only. Uses ~10x fewer tokens than full format.
**Always use index format first** to get an overview and identify relevant results.
**Example Output**:
```
1. [decision] Implement graceful session cleanup
Date: 2025-10-21 14:23:45
Source: claude-mem://observation/123
2. [feature] Add FTS5 full-text search
Date: 2025-10-21 13:15:22
Source: claude-mem://observation/124
```
### Full Format
Returns complete observation/summary details including narrative, facts, concepts, files, etc.
**Only use after reviewing index results** to dive deep into specific items of interest.
## Search Strategy
**Recommended Workflow**:
1. **Initial search**: Use default (index) format to see titles, dates, and sources
2. **Review results**: Identify which items are most relevant to your needs
3. **Deep dive**: Only then use `format: "full"` on specific items of interest
4. **Narrow down**: Use filters (type, dateRange, concepts, files) to refine results
**Token Efficiency**:
- Index format: ~50-100 tokens per result
- Full format: ~500-1000 tokens per result
- Start with 3-5 results to avoid MCP token limits
## Citations
All search results use the `claude-mem://` URI scheme for citations:
- `claude-mem://observation/{id}` - References specific observations
- `claude-mem://session/{id}` - References specific sessions
- `claude-mem://user-prompt/{id}` - References specific user prompts
These citations allow Claude to reference specific historical context in responses.
## FTS5 Query Syntax
The `query` parameter supports SQLite FTS5 full-text search syntax:
- **Simple**: `"error handling"`
- **AND**: `"error" AND "handling"`
- **OR**: `"bug" OR "fix"`
- **NOT**: `"bug" NOT "feature"`
- **Phrase**: `"'exact phrase'"`
- **Column**: `title:"authentication"`
## Security
As of v4.2.3, all FTS5 queries are properly escaped to prevent SQL injection attacks:
- Double quotes are escaped: `query.replace(/"/g, '""')`
- Comprehensive test suite with 332 injection attack tests
- Affects: `search_observations`, `search_sessions`, `search_user_prompts`
## Example Queries
```
# Find all decisions about build system
search_observations with query="build system" and type="decision"
# Show everything related to worker-service.ts
find_by_file with filePath="worker-service.ts"
# Search what we learned about hooks
search_sessions with query="hooks"
# Show observations tagged with 'architecture'
find_by_concept with concept="architecture"
# Find what user asked about authentication
search_user_prompts with query="authentication"
# Get recent context for debugging
get_recent_context with limit=5
# Timeline around a specific observation
get_context_timeline with anchor=123 and depth_before=10 and depth_after=10
# Quick timeline search for authentication work
get_timeline_by_query with query="authentication bug" and mode="auto"
```
## Implementation
The MCP search server is implemented using:
- `@modelcontextprotocol/sdk` (v1.20.1)
- `SessionSearch` service for FTS5 queries
- `SessionStore` for database access
- `zod-to-json-schema` for parameter validation
**Source Code**: `src/servers/search-server.ts`
## Troubleshooting
### Tool Not Available
If search tools are not available in Claude Code sessions:
1. Check MCP configuration:
```bash
cat plugin/.mcp.json
```
2. Verify search server is built:
```bash
ls -l plugin/scripts/search-server.mjs
```
3. Rebuild if needed:
```bash
npm run build
```
### Search Returns No Results
1. Check database has data:
```bash
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations;"
```
2. Verify FTS5 tables exist:
```bash
sqlite3 ~/.claude-mem/claude-mem.db "SELECT name FROM sqlite_master WHERE type='table' AND name LIKE '%_fts';"
```
3. Test query syntax:
```bash
# Simple query should work
search_observations with query="test"
```
### Token Limit Errors
If you hit MCP token limits:
1. Use `format: "index"` instead of `format: "full"`
2. Reduce `limit` parameter (try 3-5 instead of 20)
3. Use more specific filters to narrow results
4. Use `offset` to paginate through results
+104
View File
@@ -0,0 +1,104 @@
# Claude-Mem Context Documentation
## What This Folder Is
This `docs/context/` folder contains **internal documentation** - planning documents, design references, audits, and work-in-progress materials that support development but are NOT user-facing.
## Folder Structure
```
docs/
├── public/ ← User-facing Mintlify docs (DO NOT put internal docs there)
│ └── *.mdx - Official documentation
└── context/ ← You are here (Internal documentation)
├── *.md - Planning docs, audits, references
├── *-plan.md - Implementation plans
├── *-audit.md - Code audits and reviews
├── agent-sdk-*.md - SDK reference materials
└── subdirs/ - Organized by topic
```
## What Belongs Here
**Internal Documentation** (`.md` format):
- Planning documents (`*-plan.md`, `*-outline.md`)
- Implementation analysis (`*-audit.md`, `*-code-reference.md`)
- Error tracking (`typescript-errors.md`)
- Design documents not ready for public docs
- PR review responses
- Reference materials (like `agent-sdk-ref.md`)
- Work-in-progress documentation
- Technical investigations and postmortems
- Architecture analysis documents
**Examples from this folder:**
- `mem-search-technical-architecture.md` - Deep technical reference
- `search-architecture-analysis.md` - Implementation analysis
- `agent-sdk-ref.md` - SDK reference for developers
- `typescript-errors.md` - Error tracking during development
- `worker-service-architecture.md` - Internal architecture notes
- `processing-indicator-audit.md` - Code audit document
## What Does NOT Belong Here
**User-Facing Documentation** goes in `/docs/public/`:
- User guides and tutorials
- Official architecture documentation
- Installation instructions
- Configuration guides
- Best practices for users
- Troubleshooting guides
**Rule of Thumb:**
- If a user would read it → `/docs/public/` (as `.mdx`)
- If only developers/contributors need it → `/docs/context/` (as `.md`)
## File Organization
### By Type
- `*-plan.md` - Implementation plans for features
- `*-audit.md` - Code audits and reviews
- `*-postmortem.md` - Analysis of issues or incidents
- `*-reference.md` - Technical reference materials
- `*-analysis.md` - Architecture or design analysis
### By Topic
- Create subdirectories for related documents
- Example: `claude-code/` for Claude Code specific docs
- Example: `architecture/` for internal architecture notes
## Development Workflow
### When to Create Context Docs
1. **Planning Phase** - Before implementing a feature
- Create `feature-name-plan.md`
- Outline implementation steps
- Document decisions and tradeoffs
2. **During Development** - Track issues and decisions
- Create `feature-name-audit.md` for code reviews
- Update `typescript-errors.md` for build issues
- Document gotchas in topic-specific files
3. **After Implementation** - Preserve knowledge
- Create `feature-name-postmortem.md` if issues occurred
- Update architecture analysis documents
- Archive plan docs (don't delete - useful for history)
### Graduating to Public Docs
When internal docs are polished enough for users:
1. Convert `.md` to `.mdx` format
2. Add Mintlify frontmatter
3. Move to appropriate `/docs/public/` subdirectory
4. Add to `docs.json` navigation
5. Keep original in `/docs/context/` for reference
## Summary
**Simple Rule**:
- `/docs/context/` = Internal docs, plans, references, audits ← YOU ARE HERE
- `/docs/public/` = Official user documentation (Mintlify .mdx files)
**Purpose**: This folder preserves development context, design decisions, and technical knowledge that helps contributors understand WHY things work the way they do, even if users don't need those details.
+164
View File
@@ -0,0 +1,164 @@
# CWD Context Fix - Technical Documentation
## Overview
This fix adds working directory (CWD) context propagation through the entire claude-mem pipeline, enabling the SDK agent to have spatial awareness of which directory/repository it's observing.
## Problem Statement
Previously, the SDK agent would:
- Search wrong repositories when analyzing file operations
- Report "file not found" for files that actually exist
- Lack context about which project was being worked on
- Generate inaccurate observations due to spatial confusion
## Solution
The CWD information now flows through the entire system:
```
Hook Input (cwd) → Worker API (cwd) → SessionManager (cwd) → SDK Agent (tool_cwd)
```
## Data Flow
### 1. Hook Layer (`save-hook.ts`)
```typescript
export interface PostToolUseInput {
session_id: string;
cwd: string; // ← Captured from Claude Code
tool_name: string;
tool_input: any;
tool_response: any;
}
```
The hook extracts `cwd` and includes it in the worker API request:
```typescript
body: JSON.stringify({
tool_name,
tool_input,
tool_response,
prompt_number,
cwd: cwd || '' // ← Passed to worker
})
```
### 2. Worker Service (`worker-service.ts`)
```typescript
const { tool_name, tool_input, tool_response, prompt_number, cwd } = req.body;
this.sessionManager.queueObservation(sessionDbId, {
tool_name,
tool_input,
tool_response,
prompt_number,
cwd // ← Forwarded to queue
});
```
### 3. Session Manager (`SessionManager.ts`)
```typescript
session.pendingMessages.push({
type: 'observation',
tool_name: data.tool_name,
tool_input: data.tool_input,
tool_response: data.tool_response,
prompt_number: data.prompt_number,
cwd: data.cwd // ← Included in message queue
});
```
### 4. SDK Agent (`SDKAgent.ts`)
```typescript
content: buildObservationPrompt({
id: 0,
tool_name: message.tool_name!,
tool_input: JSON.stringify(message.tool_input),
tool_output: JSON.stringify(message.tool_response),
created_at_epoch: Date.now(),
cwd: message.cwd // ← Passed to prompt builder
})
```
### 5. Prompt Generation (`prompts.ts`)
```typescript
return `<tool_used>
<tool_name>${obs.tool_name}</tool_name>
<tool_time>${new Date(obs.created_at_epoch).toISOString()}</tool_time>${obs.cwd ? `
<tool_cwd>${obs.cwd}</tool_cwd>` : ''} // ← Included in XML
<tool_input>${JSON.stringify(toolInput, null, 2)}</tool_input>
<tool_output>${JSON.stringify(toolOutput, null, 2)}</tool_output>
</tool_used>`;
```
## SDK Agent Prompt Changes
The init prompt now includes a "SPATIAL AWARENESS" section:
```
SPATIAL AWARENESS: Tool executions include the working directory (tool_cwd) to help you understand:
- Which repository/project is being worked on
- Where files are located relative to the project root
- How to match requested paths to actual execution paths
```
## Example Usage
When a user executes a read operation in `/home/user/my-project`:
```xml
<tool_used>
<tool_name>ReadTool</tool_name>
<tool_time>2025-11-10T19:18:03.065Z</tool_time>
<tool_cwd>/home/user/my-project</tool_cwd>
<tool_input>
{
"path": "src/index.ts"
}
</tool_input>
<tool_output>
{
"content": "export default..."
}
</tool_output>
</tool_used>
```
The SDK agent now knows:
1. The operation happened in `/home/user/my-project`
2. The file `src/index.ts` is relative to that directory
3. Which repository context to search when generating observations
## Testing
8 comprehensive tests validate the CWD propagation:
```bash
npx tsx --test tests/cwd-propagation.test.ts
```
All tests verify:
- Type interfaces include `cwd` fields
- Hook extracts and passes `cwd`
- Worker accepts and forwards `cwd`
- SDK agent includes `cwd` in prompts
- End-to-end flow is correct
## Benefits
1. **Spatial Awareness**: SDK agent knows which directory/repository it's observing
2. **Accurate Path Matching**: Can verify if requested paths match executed paths
3. **Better Summaries**: Won't search wrong repositories or report false negatives
4. **Works with All Models**: Even Haiku benefits from correct context (no need for Opus workaround)
## Backward Compatibility
- `cwd` is optional in all interfaces (`cwd?: string`)
- Missing `cwd` values are handled gracefully (defaults to empty string)
- Existing observations without `cwd` continue to work
- No database migration required (CWD is transient, not persisted)
## Related Issues
Fixes issue #73 (CWD context missing from SDK agent)
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+302
View File
@@ -0,0 +1,302 @@
# Agent Skills in the SDK
> Extend Claude with specialized capabilities using Agent Skills in the Claude Agent SDK
## Overview
Agent Skills extend Claude with specialized capabilities that Claude autonomously invokes when relevant. Skills are packaged as `SKILL.md` files containing instructions, descriptions, and optional supporting resources.
For comprehensive information about Skills, including benefits, architecture, and authoring guidelines, see the [Agent Skills overview](/en/docs/agents-and-tools/agent-skills/overview).
## How Skills Work with the SDK
When using the Claude Agent SDK, Skills are:
1. **Defined as filesystem artifacts**: Created as `SKILL.md` files in specific directories (`.claude/skills/`)
2. **Loaded from filesystem**: Skills are loaded from configured filesystem locations. You must specify `settingSources` (TypeScript) or `setting_sources` (Python) to load Skills from the filesystem
3. **Automatically discovered**: Once filesystem settings are loaded, Skill metadata is discovered at startup from user and project directories; full content loaded when triggered
4. **Model-invoked**: Claude autonomously chooses when to use them based on context
5. **Enabled via allowed\_tools**: Add `"Skill"` to your `allowed_tools` to enable Skills
Unlike subagents (which can be defined programmatically), Skills must be created as filesystem artifacts. The SDK does not provide a programmatic API for registering Skills.
<Note>
**Default behavior**: By default, the SDK does not load any filesystem settings. To use Skills, you must explicitly configure `settingSources: ['user', 'project']` (TypeScript) or `setting_sources=["user", "project"]` (Python) in your options.
</Note>
## Using Skills with the SDK
To use Skills with the SDK, you need to:
1. Include `"Skill"` in your `allowed_tools` configuration
2. Configure `settingSources`/`setting_sources` to load Skills from the filesystem
Once configured, Claude automatically discovers Skills from the specified directories and invokes them when relevant to the user's request.
<CodeGroup>
```python Python theme={null}
import asyncio
from claude_agent_sdk import query, ClaudeAgentOptions
async def main():
options = ClaudeAgentOptions(
cwd="/path/to/project", # Project with .claude/skills/
setting_sources=["user", "project"], # Load Skills from filesystem
allowed_tools=["Skill", "Read", "Write", "Bash"] # Enable Skill tool
)
async for message in query(
prompt="Help me process this PDF document",
options=options
):
print(message)
asyncio.run(main())
```
```typescript TypeScript theme={null}
import { query } from "@anthropic-ai/claude-agent-sdk";
for await (const message of query({
prompt: "Help me process this PDF document",
options: {
cwd: "/path/to/project", // Project with .claude/skills/
settingSources: ["user", "project"], // Load Skills from filesystem
allowedTools: ["Skill", "Read", "Write", "Bash"] // Enable Skill tool
}
})) {
console.log(message);
}
```
</CodeGroup>
## Skill Locations
Skills are loaded from filesystem directories based on your `settingSources`/`setting_sources` configuration:
* **Project Skills** (`.claude/skills/`): Shared with your team via git - loaded when `setting_sources` includes `"project"`
* **User Skills** (`~/.claude/skills/`): Personal Skills across all projects - loaded when `setting_sources` includes `"user"`
* **Plugin Skills**: Bundled with installed Claude Code plugins
## Creating Skills
Skills are defined as directories containing a `SKILL.md` file with YAML frontmatter and Markdown content. The `description` field determines when Claude invokes your Skill.
**Example directory structure**:
```bash theme={null}
.claude/skills/processing-pdfs/
└── SKILL.md
```
For complete guidance on creating Skills, including SKILL.md structure, multi-file Skills, and examples, see:
* [Agent Skills in Claude Code](https://code.claude.com/docs/skills): Complete guide with examples
* [Agent Skills Best Practices](/en/docs/agents-and-tools/agent-skills/best-practices): Authoring guidelines and naming conventions
## Tool Restrictions
<Note>
The `allowed-tools` frontmatter field in SKILL.md is only supported when using Claude Code CLI directly. **It does not apply when using Skills through the SDK**.
When using the SDK, control tool access through the main `allowedTools` option in your query configuration.
</Note>
To restrict tools for Skills in SDK applications, use the `allowedTools` option:
<Note>
Import statements from the first example are assumed in the following code snippets.
</Note>
<CodeGroup>
```python Python theme={null}
options = ClaudeAgentOptions(
setting_sources=["user", "project"], # Load Skills from filesystem
allowed_tools=["Skill", "Read", "Grep", "Glob"] # Restricted toolset
)
async for message in query(
prompt="Analyze the codebase structure",
options=options
):
print(message)
```
```typescript TypeScript theme={null}
// Skills can only use Read, Grep, and Glob tools
for await (const message of query({
prompt: "Analyze the codebase structure",
options: {
settingSources: ["user", "project"], // Load Skills from filesystem
allowedTools: ["Skill", "Read", "Grep", "Glob"] // Restricted toolset
}
})) {
console.log(message);
}
```
</CodeGroup>
## Discovering Available Skills
To see which Skills are available in your SDK application, simply ask Claude:
<CodeGroup>
```python Python theme={null}
options = ClaudeAgentOptions(
setting_sources=["user", "project"], # Load Skills from filesystem
allowed_tools=["Skill"]
)
async for message in query(
prompt="What Skills are available?",
options=options
):
print(message)
```
```typescript TypeScript theme={null}
for await (const message of query({
prompt: "What Skills are available?",
options: {
settingSources: ["user", "project"], // Load Skills from filesystem
allowedTools: ["Skill"]
}
})) {
console.log(message);
}
```
</CodeGroup>
Claude will list the available Skills based on your current working directory and installed plugins.
## Testing Skills
Test Skills by asking questions that match their descriptions:
<CodeGroup>
```python Python theme={null}
options = ClaudeAgentOptions(
cwd="/path/to/project",
setting_sources=["user", "project"], # Load Skills from filesystem
allowed_tools=["Skill", "Read", "Bash"]
)
async for message in query(
prompt="Extract text from invoice.pdf",
options=options
):
print(message)
```
```typescript TypeScript theme={null}
for await (const message of query({
prompt: "Extract text from invoice.pdf",
options: {
cwd: "/path/to/project",
settingSources: ["user", "project"], // Load Skills from filesystem
allowedTools: ["Skill", "Read", "Bash"]
}
})) {
console.log(message);
}
```
</CodeGroup>
Claude automatically invokes the relevant Skill if the description matches your request.
## Troubleshooting
### Skills Not Found
**Check settingSources configuration**: Skills are only loaded when you explicitly configure `settingSources`/`setting_sources`. This is the most common issue:
<CodeGroup>
```python Python theme={null}
# Wrong - Skills won't be loaded
options = ClaudeAgentOptions(
allowed_tools=["Skill"]
)
# Correct - Skills will be loaded
options = ClaudeAgentOptions(
setting_sources=["user", "project"], # Required to load Skills
allowed_tools=["Skill"]
)
```
```typescript TypeScript theme={null}
// Wrong - Skills won't be loaded
const options = {
allowedTools: ["Skill"]
};
// Correct - Skills will be loaded
const options = {
settingSources: ["user", "project"], // Required to load Skills
allowedTools: ["Skill"]
};
```
</CodeGroup>
For more details on `settingSources`/`setting_sources`, see the [TypeScript SDK reference](/en/docs/agent-sdk/typescript#settingsource) or [Python SDK reference](/en/docs/agent-sdk/python#settingsource).
**Check working directory**: The SDK loads Skills relative to the `cwd` option. Ensure it points to a directory containing `.claude/skills/`:
<CodeGroup>
```python Python theme={null}
# Ensure your cwd points to the directory containing .claude/skills/
options = ClaudeAgentOptions(
cwd="/path/to/project", # Must contain .claude/skills/
setting_sources=["user", "project"], # Required to load Skills
allowed_tools=["Skill"]
)
```
```typescript TypeScript theme={null}
// Ensure your cwd points to the directory containing .claude/skills/
const options = {
cwd: "/path/to/project", // Must contain .claude/skills/
settingSources: ["user", "project"], // Required to load Skills
allowedTools: ["Skill"]
};
```
</CodeGroup>
See the "Using Skills with the SDK" section above for the complete pattern.
**Verify filesystem location**:
```bash theme={null}
# Check project Skills
ls .claude/skills/*/SKILL.md
# Check personal Skills
ls ~/.claude/skills/*/SKILL.md
```
### Skill Not Being Used
**Check the Skill tool is enabled**: Confirm `"Skill"` is in your `allowedTools`.
**Check the description**: Ensure it's specific and includes relevant keywords. See [Agent Skills Best Practices](/en/docs/agents-and-tools/agent-skills/best-practices#writing-effective-descriptions) for guidance on writing effective descriptions.
### Additional Troubleshooting
For general Skills troubleshooting (YAML syntax, debugging, etc.), see the [Claude Code Skills troubleshooting section](https://code.claude.com/docs/skills#troubleshooting).
## Related Documentation
### Skills Guides
* [Agent Skills in Claude Code](https://code.claude.com/docs/skills): Complete Skills guide with creation, examples, and troubleshooting
* [Agent Skills Overview](/en/docs/agents-and-tools/agent-skills/overview): Conceptual overview, benefits, and architecture
* [Agent Skills Best Practices](/en/docs/agents-and-tools/agent-skills/best-practices): Authoring guidelines for effective Skills
* [Agent Skills Cookbook](https://github.com/anthropics/claude-cookbooks/tree/main/skills): Example Skills and templates
### SDK Resources
* [Subagents in the SDK](/en/docs/agent-sdk/subagents): Similar filesystem-based agents with programmatic options
* [Slash Commands in the SDK](/en/docs/agent-sdk/slash-commands): User-invoked commands
* [SDK Overview](/en/docs/agent-sdk/overview): General SDK concepts
* [TypeScript SDK Reference](/en/docs/agent-sdk/typescript): Complete API documentation
* [Python SDK Reference](/en/docs/agent-sdk/python): Complete API documentation
+607
View File
@@ -0,0 +1,607 @@
# Agent Skills
> Create, manage, and share Skills to extend Claude's capabilities in Claude Code.
This guide shows you how to create, use, and manage Agent Skills in Claude Code. Skills are modular capabilities that extend Claude's functionality through organized folders containing instructions, scripts, and resources.
## Prerequisites
* Claude Code version 1.0 or later
* Basic familiarity with [Claude Code](/en/quickstart)
## What are Agent Skills?
Agent Skills package expertise into discoverable capabilities. Each Skill consists of a `SKILL.md` file with instructions that Claude reads when relevant, plus optional supporting files like scripts and templates.
**How Skills are invoked**: Skills are **model-invoked**—Claude autonomously decides when to use them based on your request and the Skill's description. This is different from slash commands, which are **user-invoked** (you explicitly type `/command` to trigger them).
**Benefits**:
* Extend Claude's capabilities for your specific workflows
* Share expertise across your team via git
* Reduce repetitive prompting
* Compose multiple Skills for complex tasks
Learn more in the [Agent Skills overview](https://docs.claude.com/en/docs/agents-and-tools/agent-skills/overview).
<Note>
For a deep dive into the architecture and real-world applications of Agent Skills, read our engineering blog: [Equipping agents for the real world with Agent Skills](https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills).
</Note>
## Create a Skill
Skills are stored as directories containing a `SKILL.md` file.
### Personal Skills
Personal Skills are available across all your projects. Store them in `~/.claude/skills/`:
```bash theme={null}
mkdir -p ~/.claude/skills/my-skill-name
```
**Use personal Skills for**:
* Your individual workflows and preferences
* Experimental Skills you're developing
* Personal productivity tools
### Project Skills
Project Skills are shared with your team. Store them in `.claude/skills/` within your project:
```bash theme={null}
mkdir -p .claude/skills/my-skill-name
```
**Use project Skills for**:
* Team workflows and conventions
* Project-specific expertise
* Shared utilities and scripts
Project Skills are checked into git and automatically available to team members.
### Plugin Skills
Skills can also come from [Claude Code plugins](/en/plugins). Plugins may bundle Skills that are automatically available when the plugin is installed. These Skills work the same way as personal and project Skills.
## Write SKILL.md
Create a `SKILL.md` file with YAML frontmatter and Markdown content:
```yaml theme={null}
---
name: your-skill-name
description: Brief description of what this Skill does and when to use it
---
# Your Skill Name
## Instructions
Provide clear, step-by-step guidance for Claude.
## Examples
Show concrete examples of using this Skill.
```
**Field requirements**:
* `name`: Must use lowercase letters, numbers, and hyphens only (max 64 characters)
* `description`: Brief description of what the Skill does and when to use it (max 1024 characters)
The `description` field is critical for Claude to discover when to use your Skill. It should include both what the Skill does and when Claude should use it.
See the [best practices guide](https://docs.claude.com/en/docs/agents-and-tools/agent-skills/best-practices) for complete authoring guidance including validation rules.
## Add supporting files
Create additional files alongside SKILL.md:
```
my-skill/
├── SKILL.md (required)
├── reference.md (optional documentation)
├── examples.md (optional examples)
├── scripts/
│ └── helper.py (optional utility)
└── templates/
└── template.txt (optional template)
```
Reference these files from SKILL.md:
````markdown theme={null}
For advanced usage, see [reference.md](reference.md).
Run the helper script:
```bash
python scripts/helper.py input.txt
```
````
Claude reads these files only when needed, using progressive disclosure to manage context efficiently.
## Restrict tool access with allowed-tools
Use the `allowed-tools` frontmatter field to limit which tools Claude can use when a Skill is active:
```yaml theme={null}
---
name: safe-file-reader
description: Read files without making changes. Use when you need read-only file access.
allowed-tools: Read, Grep, Glob
---
# Safe File Reader
This Skill provides read-only file access.
## Instructions
1. Use Read to view file contents
2. Use Grep to search within files
3. Use Glob to find files by pattern
```
When this Skill is active, Claude can only use the specified tools (Read, Grep, Glob) without needing to ask for permission. This is useful for:
* Read-only Skills that shouldn't modify files
* Skills with limited scope (e.g., only data analysis, no file writing)
* Security-sensitive workflows where you want to restrict capabilities
If `allowed-tools` is not specified, Claude will ask for permission to use tools as normal, following the standard permission model.
<Note>
`allowed-tools` is only supported for Skills in Claude Code.
</Note>
## View available Skills
Skills are automatically discovered by Claude from three sources:
* Personal Skills: `~/.claude/skills/`
* Project Skills: `.claude/skills/`
* Plugin Skills: bundled with installed plugins
**To view all available Skills**, ask Claude directly:
```
What Skills are available?
```
or
```
List all available Skills
```
This will show all Skills from all sources, including plugin Skills.
**To inspect a specific Skill**, you can also check the filesystem:
```bash theme={null}
# List personal Skills
ls ~/.claude/skills/
# List project Skills (if in a project directory)
ls .claude/skills/
# View a specific Skill's content
cat ~/.claude/skills/my-skill/SKILL.md
```
## Test a Skill
After creating a Skill, test it by asking questions that match your description.
**Example**: If your description mentions "PDF files":
```
Can you help me extract text from this PDF?
```
Claude autonomously decides to use your Skill if it matches the request—you don't need to explicitly invoke it. The Skill activates automatically based on the context of your question.
## Debug a Skill
If Claude doesn't use your Skill, check these common issues:
### Make description specific
**Too vague**:
```yaml theme={null}
description: Helps with documents
```
**Specific**:
```yaml theme={null}
description: Extract text and tables from PDF files, fill forms, merge documents. Use when working with PDF files or when the user mentions PDFs, forms, or document extraction.
```
Include both what the Skill does and when to use it in the description.
### Verify file path
**Personal Skills**: `~/.claude/skills/skill-name/SKILL.md`
**Project Skills**: `.claude/skills/skill-name/SKILL.md`
Check the file exists:
```bash theme={null}
# Personal
ls ~/.claude/skills/my-skill/SKILL.md
# Project
ls .claude/skills/my-skill/SKILL.md
```
### Check YAML syntax
Invalid YAML prevents the Skill from loading. Verify the frontmatter:
```bash theme={null}
cat SKILL.md | head -n 10
```
Ensure:
* Opening `---` on line 1
* Closing `---` before Markdown content
* Valid YAML syntax (no tabs, correct indentation)
### View errors
Run Claude Code with debug mode to see Skill loading errors:
```bash theme={null}
claude --debug
```
## Share Skills with your team
**Recommended approach**: Distribute Skills through [plugins](/en/plugins).
To share Skills via plugin:
1. Create a plugin with Skills in the `skills/` directory
2. Add the plugin to a marketplace
3. Team members install the plugin
For complete instructions, see [Add Skills to your plugin](/en/plugins#add-skills-to-your-plugin).
You can also share Skills directly through project repositories:
### Step 1: Add Skill to your project
Create a project Skill:
```bash theme={null}
mkdir -p .claude/skills/team-skill
# Create SKILL.md
```
### Step 2: Commit to git
```bash theme={null}
git add .claude/skills/
git commit -m "Add team Skill for PDF processing"
git push
```
### Step 3: Team members get Skills automatically
When team members pull the latest changes, Skills are immediately available:
```bash theme={null}
git pull
claude # Skills are now available
```
## Update a Skill
Edit SKILL.md directly:
```bash theme={null}
# Personal Skill
code ~/.claude/skills/my-skill/SKILL.md
# Project Skill
code .claude/skills/my-skill/SKILL.md
```
Changes take effect the next time you start Claude Code. If Claude Code is already running, restart it to load the updates.
## Remove a Skill
Delete the Skill directory:
```bash theme={null}
# Personal
rm -rf ~/.claude/skills/my-skill
# Project
rm -rf .claude/skills/my-skill
git commit -m "Remove unused Skill"
```
## Best practices
### Keep Skills focused
One Skill should address one capability:
**Focused**:
* "PDF form filling"
* "Excel data analysis"
* "Git commit messages"
**Too broad**:
* "Document processing" (split into separate Skills)
* "Data tools" (split by data type or operation)
### Write clear descriptions
Help Claude discover when to use Skills by including specific triggers in your description:
**Clear**:
```yaml theme={null}
description: Analyze Excel spreadsheets, create pivot tables, and generate charts. Use when working with Excel files, spreadsheets, or analyzing tabular data in .xlsx format.
```
**Vague**:
```yaml theme={null}
description: For files
```
### Test with your team
Have teammates use Skills and provide feedback:
* Does the Skill activate when expected?
* Are the instructions clear?
* Are there missing examples or edge cases?
### Document Skill versions
You can document Skill versions in your SKILL.md content to track changes over time. Add a version history section:
```markdown theme={null}
# My Skill
## Version History
- v2.0.0 (2025-10-01): Breaking changes to API
- v1.1.0 (2025-09-15): Added new features
- v1.0.0 (2025-09-01): Initial release
```
This helps team members understand what changed between versions.
## Troubleshooting
### Claude doesn't use my Skill
**Symptom**: You ask a relevant question but Claude doesn't use your Skill.
**Check**: Is the description specific enough?
Vague descriptions make discovery difficult. Include both what the Skill does and when to use it, with key terms users would mention.
**Too generic**:
```yaml theme={null}
description: Helps with data
```
**Specific**:
```yaml theme={null}
description: Analyze Excel spreadsheets, generate pivot tables, create charts. Use when working with Excel files, spreadsheets, or .xlsx files.
```
**Check**: Is the YAML valid?
Run validation to check for syntax errors:
```bash theme={null}
# View frontmatter
cat .claude/skills/my-skill/SKILL.md | head -n 15
# Check for common issues
# - Missing opening or closing ---
# - Tabs instead of spaces
# - Unquoted strings with special characters
```
**Check**: Is the Skill in the correct location?
```bash theme={null}
# Personal Skills
ls ~/.claude/skills/*/SKILL.md
# Project Skills
ls .claude/skills/*/SKILL.md
```
### Skill has errors
**Symptom**: The Skill loads but doesn't work correctly.
**Check**: Are dependencies available?
Claude will automatically install required dependencies (or ask for permission to install them) when it needs them.
**Check**: Do scripts have execute permissions?
```bash theme={null}
chmod +x .claude/skills/my-skill/scripts/*.py
```
**Check**: Are file paths correct?
Use forward slashes (Unix style) in all paths:
**Correct**: `scripts/helper.py`
**Wrong**: `scripts\helper.py` (Windows style)
### Multiple Skills conflict
**Symptom**: Claude uses the wrong Skill or seems confused between similar Skills.
**Be specific in descriptions**: Help Claude choose the right Skill by using distinct trigger terms in your descriptions.
Instead of:
```yaml theme={null}
# Skill 1
description: For data analysis
# Skill 2
description: For analyzing data
```
Use:
```yaml theme={null}
# Skill 1
description: Analyze sales data in Excel files and CRM exports. Use for sales reports, pipeline analysis, and revenue tracking.
# Skill 2
description: Analyze log files and system metrics data. Use for performance monitoring, debugging, and system diagnostics.
```
## Examples
### Simple Skill (single file)
```
commit-helper/
└── SKILL.md
```
```yaml theme={null}
---
name: generating-commit-messages
description: Generates clear commit messages from git diffs. Use when writing commit messages or reviewing staged changes.
---
# Generating Commit Messages
## Instructions
1. Run `git diff --staged` to see changes
2. I'll suggest a commit message with:
- Summary under 50 characters
- Detailed description
- Affected components
## Best practices
- Use present tense
- Explain what and why, not how
```
### Skill with tool permissions
```
code-reviewer/
└── SKILL.md
```
```yaml theme={null}
---
name: code-reviewer
description: Review code for best practices and potential issues. Use when reviewing code, checking PRs, or analyzing code quality.
allowed-tools: Read, Grep, Glob
---
# Code Reviewer
## Review checklist
1. Code organization and structure
2. Error handling
3. Performance considerations
4. Security concerns
5. Test coverage
## Instructions
1. Read the target files using Read tool
2. Search for patterns using Grep
3. Find related files using Glob
4. Provide detailed feedback on code quality
```
### Multi-file Skill
```
pdf-processing/
├── SKILL.md
├── FORMS.md
├── REFERENCE.md
└── scripts/
├── fill_form.py
└── validate.py
```
**SKILL.md**:
````yaml theme={null}
---
name: pdf-processing
description: Extract text, fill forms, merge PDFs. Use when working with PDF files, forms, or document extraction. Requires pypdf and pdfplumber packages.
---
# PDF Processing
## Quick start
Extract text:
```python
import pdfplumber
with pdfplumber.open("doc.pdf") as pdf:
text = pdf.pages[0].extract_text()
```
For form filling, see [FORMS.md](FORMS.md).
For detailed API reference, see [REFERENCE.md](REFERENCE.md).
## Requirements
Packages must be installed in your environment:
```bash
pip install pypdf pdfplumber
```
````
<Note>
List required packages in the description. Packages must be installed in your environment before Claude can use them.
</Note>
Claude loads additional files only when needed.
## Next steps
<CardGroup cols={2}>
<Card title="Authoring best practices" icon="lightbulb" href="https://docs.claude.com/en/docs/agents-and-tools/agent-skills/best-practices">
Write Skills that Claude can use effectively
</Card>
<Card title="Agent Skills overview" icon="book" href="https://docs.claude.com/en/docs/agents-and-tools/agent-skills/overview">
Learn how Skills work across Claude products
</Card>
<Card title="Use Skills in the Agent SDK" icon="cube" href="https://docs.claude.com/en/docs/agent-sdk/skills">
Use Skills programmatically with TypeScript and Python
</Card>
<Card title="Get started with Agent Skills" icon="rocket" href="https://docs.claude.com/en/docs/agents-and-tools/agent-skills/quickstart">
Create your first Skill
</Card>
</CardGroup>
+424
View File
@@ -0,0 +1,424 @@
"""Pydantic models for Claude Code transcript JSON structures.
Enhanced to leverage official Anthropic types where beneficial.
"""
from typing import Any, List, Union, Optional, Dict, Literal, cast
from pydantic import BaseModel
from anthropic.types import Message as AnthropicMessage
from anthropic.types import StopReason
from anthropic.types import Usage as AnthropicUsage
from anthropic.types.content_block import ContentBlock
class TodoItem(BaseModel):
id: str
content: str
status: Literal["pending", "in_progress", "completed"]
priority: Literal["high", "medium", "low"]
class UsageInfo(BaseModel):
"""Token usage information that extends Anthropic's Usage type to handle optional fields."""
input_tokens: Optional[int] = None
cache_creation_input_tokens: Optional[int] = None
cache_read_input_tokens: Optional[int] = None
output_tokens: Optional[int] = None
service_tier: Optional[str] = None
server_tool_use: Optional[Dict[str, Any]] = None
def to_anthropic_usage(self) -> Optional[AnthropicUsage]:
"""Convert to Anthropic Usage type if both required fields are present."""
if self.input_tokens is not None and self.output_tokens is not None:
return AnthropicUsage(
input_tokens=self.input_tokens,
output_tokens=self.output_tokens,
cache_creation_input_tokens=self.cache_creation_input_tokens,
cache_read_input_tokens=self.cache_read_input_tokens,
service_tier=self.service_tier, # type: ignore
server_tool_use=self.server_tool_use, # type: ignore
)
return None
@classmethod
def from_anthropic_usage(cls, usage: AnthropicUsage) -> "UsageInfo":
"""Create UsageInfo from Anthropic Usage."""
return cls(
input_tokens=usage.input_tokens,
output_tokens=usage.output_tokens,
cache_creation_input_tokens=usage.cache_creation_input_tokens,
cache_read_input_tokens=usage.cache_read_input_tokens,
service_tier=usage.service_tier,
server_tool_use=usage.server_tool_use.model_dump()
if usage.server_tool_use
else None,
)
class TextContent(BaseModel):
type: Literal["text"]
text: str
class ToolUseContent(BaseModel):
type: Literal["tool_use"]
id: str
name: str
input: Dict[str, Any]
class ToolResultContent(BaseModel):
type: Literal["tool_result"]
tool_use_id: str
content: Union[str, List[Dict[str, Any]]]
is_error: Optional[bool] = None
class ThinkingContent(BaseModel):
type: Literal["thinking"]
thinking: str
signature: Optional[str] = None
class ImageSource(BaseModel):
type: Literal["base64"]
media_type: str
data: str
class ImageContent(BaseModel):
type: Literal["image"]
source: ImageSource
# Enhanced ContentItem to include official Anthropic ContentBlock types
ContentItem = Union[
TextContent,
ToolUseContent,
ToolResultContent,
ThinkingContent,
ImageContent,
ContentBlock, # Official Anthropic content block types
]
class UserMessage(BaseModel):
role: Literal["user"]
content: Union[str, List[ContentItem]]
class AssistantMessage(BaseModel):
"""Assistant message model compatible with Anthropic's Message type."""
id: str
type: Literal["message"]
role: Literal["assistant"]
model: str
content: List[ContentItem]
stop_reason: Optional[StopReason] = None
stop_sequence: Optional[str] = None
usage: Optional[UsageInfo] = None
@classmethod
def from_anthropic_message(
cls, anthropic_msg: AnthropicMessage
) -> "AssistantMessage":
"""Create AssistantMessage from official Anthropic Message."""
# Convert Anthropic Message to our format, preserving official types where possible
return cls(
id=anthropic_msg.id,
type=anthropic_msg.type,
role=anthropic_msg.role,
model=anthropic_msg.model,
content=list(
anthropic_msg.content
), # Convert to list for ContentItem compatibility
stop_reason=anthropic_msg.stop_reason,
stop_sequence=anthropic_msg.stop_sequence,
usage=normalize_usage_info(anthropic_msg.usage),
)
class FileInfo(BaseModel):
filePath: str
content: str
numLines: int
startLine: int
totalLines: int
class FileReadResult(BaseModel):
type: Literal["text"]
file: FileInfo
class CommandResult(BaseModel):
stdout: str
stderr: str
interrupted: bool
isImage: bool
class TodoResult(BaseModel):
oldTodos: List[TodoItem]
newTodos: List[TodoItem]
class EditResult(BaseModel):
oldString: Optional[str] = None
newString: Optional[str] = None
replaceAll: Optional[bool] = None
originalFile: Optional[str] = None
structuredPatch: Optional[Any] = None
userModified: Optional[bool] = None
ToolUseResult = Union[
str,
List[TodoItem],
FileReadResult,
CommandResult,
TodoResult,
EditResult,
List[ContentItem],
]
class BaseTranscriptEntry(BaseModel):
parentUuid: Optional[str]
isSidechain: bool
userType: str
cwd: str
sessionId: str
version: str
uuid: str
timestamp: str
isMeta: Optional[bool] = None
class UserTranscriptEntry(BaseTranscriptEntry):
type: Literal["user"]
message: UserMessage
toolUseResult: Optional[ToolUseResult] = None
class AssistantTranscriptEntry(BaseTranscriptEntry):
type: Literal["assistant"]
message: AssistantMessage
requestId: Optional[str] = None
class SummaryTranscriptEntry(BaseModel):
type: Literal["summary"]
summary: str
leafUuid: str
cwd: Optional[str] = None
class SystemTranscriptEntry(BaseTranscriptEntry):
"""System messages like warnings, notifications, etc."""
type: Literal["system"]
content: str
level: Optional[str] = None # e.g., "warning", "info", "error"
class QueueOperationTranscriptEntry(BaseModel):
"""Queue operations (enqueue/dequeue) for message queueing tracking.
These are internal operations that track when messages are queued and dequeued.
They are parsed but not rendered, as the content duplicates actual user messages.
"""
type: Literal["queue-operation"]
operation: Literal["enqueue", "dequeue"]
timestamp: str
sessionId: str
content: Optional[List[ContentItem]] = None # Only present for enqueue operations
TranscriptEntry = Union[
UserTranscriptEntry,
AssistantTranscriptEntry,
SummaryTranscriptEntry,
SystemTranscriptEntry,
QueueOperationTranscriptEntry,
]
def normalize_usage_info(usage_data: Any) -> Optional[UsageInfo]:
"""Normalize usage data to be compatible with both custom and Anthropic formats."""
if usage_data is None:
return None
# If it's already a UsageInfo instance, return as-is
if isinstance(usage_data, UsageInfo):
return usage_data
# If it's an Anthropic Usage instance, convert using our method
if isinstance(usage_data, AnthropicUsage):
return UsageInfo.from_anthropic_usage(usage_data)
# If it has the shape of an Anthropic Usage, try to construct it first
if hasattr(usage_data, "input_tokens") and hasattr(usage_data, "output_tokens"):
try:
# Try to create an Anthropic Usage first
anthropic_usage = AnthropicUsage.model_validate(usage_data)
return UsageInfo.from_anthropic_usage(anthropic_usage)
except Exception:
# Fall back to direct conversion
return UsageInfo(
input_tokens=getattr(usage_data, "input_tokens", None),
cache_creation_input_tokens=getattr(
usage_data, "cache_creation_input_tokens", None
),
cache_read_input_tokens=getattr(
usage_data, "cache_read_input_tokens", None
),
output_tokens=getattr(usage_data, "output_tokens", None),
service_tier=getattr(usage_data, "service_tier", None),
server_tool_use=getattr(usage_data, "server_tool_use", None),
)
# If it's a dict, validate and convert to our format
if isinstance(usage_data, dict):
return UsageInfo.model_validate(usage_data)
return None
def parse_content_item(item_data: Dict[str, Any]) -> ContentItem:
"""Parse a content item using enhanced approach with Anthropic types."""
try:
content_type = item_data.get("type", "")
# Try official Anthropic types first for better future compatibility
if content_type == "text":
try:
from anthropic.types.text_block import TextBlock
return TextBlock.model_validate(item_data)
except Exception:
return TextContent.model_validate(item_data)
elif content_type == "tool_use":
try:
from anthropic.types.tool_use_block import ToolUseBlock
return ToolUseBlock.model_validate(item_data)
except Exception:
return ToolUseContent.model_validate(item_data)
elif content_type == "thinking":
try:
from anthropic.types.thinking_block import ThinkingBlock
return ThinkingBlock.model_validate(item_data)
except Exception:
return ThinkingContent.model_validate(item_data)
elif content_type == "tool_result":
return ToolResultContent.model_validate(item_data)
elif content_type == "image":
return ImageContent.model_validate(item_data)
else:
# Fallback to text content for unknown types
return TextContent(type="text", text=str(item_data))
except Exception:
return TextContent(type="text", text=str(item_data))
def parse_message_content(content_data: Any) -> Union[str, List[ContentItem]]:
"""Parse message content, handling both string and list formats."""
if isinstance(content_data, str):
return content_data
elif isinstance(content_data, list):
content_list = cast(List[Dict[str, Any]], content_data)
return [parse_content_item(item) for item in content_list]
else:
return str(content_data)
def parse_transcript_entry(data: Dict[str, Any]) -> TranscriptEntry:
"""
Parse a JSON dictionary into the appropriate TranscriptEntry type.
Enhanced to optionally use official Anthropic types for assistant messages.
Args:
data: Dictionary parsed from JSON
Returns:
The appropriate TranscriptEntry subclass
Raises:
ValueError: If the data doesn't match any known transcript entry type
"""
entry_type = data.get("type")
if entry_type == "user":
# Parse message content if present
data_copy = data.copy()
if "message" in data_copy and "content" in data_copy["message"]:
data_copy["message"] = data_copy["message"].copy()
data_copy["message"]["content"] = parse_message_content(
data_copy["message"]["content"]
)
# Parse toolUseResult if present and it's a list of content items
if "toolUseResult" in data_copy and isinstance(
data_copy["toolUseResult"], list
):
# Check if it's a list of content items (MCP tool results)
tool_use_result = cast(List[Any], data_copy["toolUseResult"])
if (
tool_use_result
and isinstance(tool_use_result[0], dict)
and "type" in tool_use_result[0]
):
data_copy["toolUseResult"] = [
parse_content_item(cast(Dict[str, Any], item))
for item in tool_use_result
if isinstance(item, dict)
]
return UserTranscriptEntry.model_validate(data_copy)
elif entry_type == "assistant":
# Enhanced assistant message parsing with optional Anthropic types
data_copy = data.copy()
# Validate compatibility with official Anthropic Message type
if "message" in data_copy:
try:
message_data = data_copy["message"]
AnthropicMessage.model_validate(message_data)
# Successfully validated - our data is compatible with official Anthropic types
except Exception:
# Validation failed - continue with standard parsing
pass
# Standard parsing path (works for all cases)
if "message" in data_copy and "content" in data_copy["message"]:
message_copy = data_copy["message"].copy()
message_copy["content"] = parse_message_content(message_copy["content"])
# Normalize usage data to support both Anthropic and custom formats
if "usage" in message_copy:
message_copy["usage"] = normalize_usage_info(message_copy["usage"])
data_copy["message"] = message_copy
return AssistantTranscriptEntry.model_validate(data_copy)
elif entry_type == "summary":
return SummaryTranscriptEntry.model_validate(data)
elif entry_type == "system":
return SystemTranscriptEntry.model_validate(data)
elif entry_type == "queue-operation":
# Parse content if present (only in enqueue operations)
data_copy = data.copy()
if "content" in data_copy and isinstance(data_copy["content"], list):
data_copy["content"] = parse_message_content(data_copy["content"])
return QueueOperationTranscriptEntry.model_validate(data_copy)
else:
raise ValueError(f"Unknown transcript entry type: {entry_type}")
+616
View File
@@ -0,0 +1,616 @@
# JIT Context Filtering: Post-Mortem
**Date:** November 9, 2025
**Duration:** 3.5 hours (7:45 PM - 11:11 PM)
**Branches:** `feature/jit-context`, `failed/jit-context`
**Status:** Failed, reverted to main
**Commits:**
- `3ac0790` - feat: Implement JIT context hook for user prompt submission
- `adf7bf4` - Refactor JIT context handling in SDKAgent and WorkerService
## Executive Summary
Attempted to implement JIT (Just-In-Time) context filtering—a feature that would dynamically generate relevant context timelines on every user prompt, potentially replacing the static session-start context entirely. After multiple architectural iterations spanning 3.5 hours and adding ~2,850 lines of code, the implementation was abandoned and reverted. The revert was not due to lack of vision (the feature aligns with long-term architectural goals), but due to implementation complexity and the need for a simpler initial approach. Significant architectural knowledge was gained about hook limitations, worker patterns, and proper separation of concerns.
## What We Tried to Build
### Goal
When a user submits a prompt, dynamically generate a relevant context timeline instead of the static session-start context. Use the fast search infrastructure (SQLite FTS5 + ChromaDB) to fetch precisely relevant context on-demand.
### The Vision
**Current approach:** SessionStart hook loads 50 recent observations blindly, displays them all.
**Proposed approach:** UserPromptSubmit hook analyzes the prompt, queries the timeline search API, and loads only the relevant context window dynamically.
**Why this makes sense:**
- We already have fast search: SQLite FTS5 + Chroma semantic search
- Dynamic context timeline search is implemented and tested
- Search results come back in <200ms
- Could **replace** session-start context entirely with smarter, prompt-specific context
### User Experience
```
User types: "How did we fix the authentication bug?"
Behind the scenes:
1. Analyze prompt: "authentication bug fix"
2. Query timeline search for relevant period
3. Load 5-10 observations from that specific timeline
4. Inject as context
5. Claude answers with precisely relevant historical context
vs. Current:
Load 50 most recent observations regardless of relevance
```
### Why Checkbox Settings Became Less Important
Originally asked for checkboxes to customize session-start context display. But if JIT context could replace session-start context with intelligent, prompt-specific timelines, the display customization became a non-issue.
## Architectural Attempts
### Attempt 1: Hook-Based Filtering (7:45 PM - 9:30 PM)
**Approach:** Call Agent SDK `query()` directly in `new-hook.ts` during UserPromptSubmit event.
**Implementation:**
- Created `jit-context-hook.ts` (~432 lines)
- Added `generateJitContext()` function in hook
- Called SDK `query()` with observation list and user prompt
- Expected hook to block for ~1-2s while Haiku filters
**Failure:**
```
Error: Claude Code executable not found at
/Users/alexnewman/.claude/plugins/marketplaces/thedotmack/plugin/scripts/cli.js
```
**Root Cause:** Hooks run in sandboxed environment without access to `claudePath` (path to Claude Code executable). The Agent SDK requires this path, which is only available in the worker service.
**Architectural Violation:** This broke the established pattern where hooks handle orchestration and workers handle AI processing. The `save-hook` sets the precedent: hooks capture data, send to worker, worker runs SDK queries asynchronously.
### Attempt 2: Worker-Based with Simple Queries (9:30 PM - 10:30 PM)
**Approach:** Move JIT filtering to worker service, keep it simple with per-request SDK queries.
**Implementation:**
- Documented architecture fix plan in `docs/jit-context-architecture-fix.md`
- Moved `generateJitContext()` to worker (considered creating `src/services/worker/JitContext.ts`)
- Modified `/sessions/:id/init` endpoint to accept `jitEnabled` flag
- Worker would run one-shot SDK query per prompt
**Architecture:**
```
UserPromptSubmit → new-hook → POST /sessions/:id/init { jitEnabled: true }
Worker spawns Claude Haiku
Filters 50 obs → 3-5 IDs
Returns { context: [...] }
Hook injects context → Claude
```
**Issues Identified:**
- Each filter request spawns a new Claude subprocess (~200-500ms overhead)
- Observation list re-sent on every prompt (~5-10KB per request)
- No token caching between requests
- Performance worse than just loading all observations directly
**Decision:** Pivoted to persistent sessions to solve performance issues.
### Attempt 3: Persistent JIT Sessions (10:30 PM - 11:11 PM)
**Approach:** Create a long-lived Agent SDK session that persists throughout user session, similar to main memory session pattern.
**Implementation (291 new lines in SDKAgent.ts):**
1. **Session Lifecycle:**
- Added `jitSessionId`, `jitAbortController`, `jitGeneratorPromise` to `ActiveSession` interface
- `startJitSession()`: Creates persistent SDK session at session init
- `cleanupJitSession()`: Terminates JIT session at session end
2. **Request Queue Architecture:**
- `jitFilterQueues` Map: Per-session request queues
- `JITFilterRequest` interface: `{ userPrompt, resolve, reject }`
- EventEmitter coordination: Wake generator when new requests arrive
3. **Message Generator Pattern:**
- `createJitMessageGenerator()`: Async generator that yields filter requests
- Initial prompt: Load 50 observations, wait for "READY" response
- Loop: Wait for EventEmitter signal → yield user prompt → parse response → resolve promise
- Pattern: Persistent session stays alive between requests
4. **Filter Query Flow:**
```typescript
runFilterQuery(sessionDbId, userPrompt) {
// Queue request
queue.requests.push({ userPrompt, resolve, reject });
queue.emitter.emit('request');
// Wait for response (30s timeout)
return Promise.race([
new Promise((resolve, reject) => { /* queued */ }),
timeout(30000)
]);
}
```
5. **Response Processing:**
- `processJitFilterResponse()`: Accumulate streaming text
- Parse IDs: "1,5,23,41" or "NONE"
- Resolve queued promise with ID array
**Added Files:**
- `src/services/worker/SDKAgent.ts`: +291 lines
- `src/services/worker-types.ts`: +3 fields (jit state tracking)
- `src/services/worker/SessionManager.ts`: +26 lines (JIT cleanup)
- `src/services/worker-service.ts`: +102 lines (JIT initialization)
- `src/shared/settings.ts`: +65 lines (JIT config)
- `src/hooks/jit-context-hook.ts`: +208 lines (orchestration)
- `docs/jit-context-architecture-fix.md`: +265 lines
- `context/session-pattern-parity.md`: +298 lines
**Total Changes:** 18 files, +2,852 lines, -133 lines
**Final Status at Revert:** Implementation was complete and likely functional, but...
## Why It Failed
### 1. Architectural Complexity Explosion
**Problem:** The persistent session pattern added enormous complexity for marginal benefit.
**Evidence:**
- Parallel session management: Regular + JIT sessions running concurrently
- Complex coordination: EventEmitter + promise queues + generator pattern
- Lifecycle coupling: Session init, request handling, cleanup all intertwined
- State explosion: 3 new fields per session (`jitSessionId`, `jitAbortController`, `jitGeneratorPromise`)
**Code Smell:** When the "optimization" requires 300 lines of coordination code, it's probably not an optimization.
### 2. Premature Optimization
**YAGNI Violation:** Built elaborate token caching and persistent session architecture before proving the feature provided value.
**Reality Check:**
- **Current approach:** Load 50 observations = ~25KB context, works fine
- **JIT overhead:** Haiku query = 1-2s latency + coordination complexity
- **User benefit:** Unclear—users haven't complained about context relevance
- **Token savings:** Marginal—Claude caches long contexts efficiently anyway
**Quote from CLAUDE.md:**
> "Write the dumb, obvious thing first. Add complexity only when you actually hit the problem."
We didn't hit a problem. We invented one.
### 3. Implementation Complexity, Not Vision
**The Vision is Sound:**
- Dynamic context is better than static context
- Timeline search API exists and is fast
- Infrastructure (SQLite + Chroma) can support this
- Replacing session-start context with prompt-specific context makes sense
**The Problem:**
We jumped to the complex persistent-session approach without trying the simple per-request approach first.
**What We Should Have Done:**
```typescript
// Simple version (not tried):
app.post('/sessions/:id/init', async (req, res) => {
const { userPrompt } = req.body;
// Query timeline search API (already exists, fast)
const timeline = await timelineSearch(project, userPrompt, depth=10);
// Return observations
return res.json({ context: timeline });
});
```
**This would have:**
- Validated the feature's value quickly
- Used existing infrastructure
- Avoided all the persistence complexity
- Taken 30 minutes instead of 3.5 hours
### 4. Pattern Divergence
**Inconsistency:** JIT sessions work fundamentally differently from memory sessions.
**Memory Session Pattern:**
```typescript
// One-shot: Init → Process observations → Complete
startSession() → yield prompts → parse responses → complete
```
**JIT Session Pattern:**
```typescript
// Persistent: Init → Wait indefinitely → Process on-demand → Complete
startJitSession() → yield initial load → LOOP:
- Wait for EventEmitter signal
- Yield filter request
- Parse response
- Resolve promise
- GOTO LOOP
```
**Maintenance Burden:** Two completely different session patterns means:
- Doubled testing complexity
- Increased cognitive load for contributors
- Higher risk of subtle bugs in lifecycle management
**Session Pattern Parity Document:** The 298-line `session-pattern-parity.md` was created to document the differences—a sign that maybe they shouldn't be different.
### 5. Blocking I/O in Critical Path
**Performance Impact:** Every user prompt now blocks for 1-2s waiting for Haiku filtering.
**Current Flow:**
```
User types prompt → 10ms → Claude responds
```
**JIT Flow:**
```
User types prompt → 10ms init → 1-2s Haiku filter → Claude responds
```
**User Experience:** We added 1-2 seconds of latency to every interaction for questionable benefit.
**Alternative:** If context filtering is valuable, do it asynchronously and apply to next prompt.
### 6. Missing the Forest for the Trees
**Real Issue:** We focused on technical implementation without asking strategic questions:
- **Is context relevance actually a problem?** No evidence.
- **Do users want this?** No feedback requested.
- **Is 50 observations too many?** Not proven.
- **Does filtering improve responses?** Not tested.
**Anti-Pattern:** Solution in search of a problem.
## What We Should Have Done
### Option 1: Don't Build It
**Justification:** No validated user need. Current system works fine.
**Next Step:** Wait for user feedback indicating context relevance is an issue.
### Option 2: Simple MVP
If we really wanted to explore this:
1. **Week 1:** Add basic filtering in worker with one-shot queries
- Accept slight performance hit (~500ms overhead)
- Measure filter accuracy and user impact
- Gather feedback
2. **Week 2:** If proven valuable, optimize
- Add token caching only if needed
- Consider persistent sessions only if performance is bottleneck
3. **Week 3:** If still valuable, scale
- Polish error handling
- Add configuration options
- Document patterns
**Philosophy:** Incremental validation, not big-bang architecture.
### Option 3: Different Approach Entirely
**Alternative:** Pre-computed relevance scores
Instead of on-demand filtering:
- Score observations at creation time (save-hook)
- Store relevance embeddings in Chroma
- At session start, query Chroma with user's first prompt
- Load top 10-20 most relevant observations
- No runtime latency, better accuracy, simpler architecture
**Benefit:** Leverages existing Chroma infrastructure, avoids runtime overhead.
## Technical Lessons Learned
### 1. EventEmitter Coordination Anti-Pattern
**Code:**
```typescript
queue.emitter.on('request', () => {
// Wake up generator to process request
});
```
**Issue:** Complex async coordination using event-driven wakeup signals is hard to reason about.
**Better:** Use async queues or channels (e.g., `async-queue` package) that handle coordination internally.
### 2. Generator Pattern Complexity
**Pattern:**
```typescript
async *createJitMessageGenerator() {
yield initialPrompt;
while (!aborted) {
await waitForEvent(); // Blocks here
yield nextRequest;
}
}
```
**Tradeoff:** Generators are great for iteration, but terrible for event-driven request/response patterns.
**Better:** Use explicit session object with `sendMessage()/waitForResponse()` methods.
### 3. Dual Session Management
**Complexity:** Managing two concurrent SDK sessions per user session is inherently complex.
**Alternatives Considered:**
- Single session handling both observations and filtering (rejected: tight coupling)
- Separate service for filtering (rejected: too much infrastructure)
- Pre-computed filtering (not considered: should have been)
**Lesson:** When parallel state management feels hard, question whether you need parallel state.
### 4. Promise Queue Pattern
**Implementation:**
```typescript
interface QueuedRequest {
resolve: (result: T) => void;
reject: (error: Error) => void;
}
queue.push({ resolve, reject });
// Later...
queue[0].resolve(result);
```
**Good:** Clean async API for callers
**Bad:** Easy to leak promises if error handling isn't perfect
**Improvement:** Use libraries like `p-queue` that handle edge cases
## Process Lessons Learned
### 1. No Incremental Validation
**Mistake:** Went from "idea" to "complete architecture" without validation points.
**Better Process:**
1. Write one-pager explaining user value
2. Build simplest possible version (2 hours max)
3. Test with real usage
4. Measure impact
5. Decide: kill, iterate, or scale
**Checkpoint Questions:**
- After 1 hour: "Does this solve a real problem?"
- After 2 hours: "Is this getting too complex?"
- After 3 hours: "Should I just ship the simple version?"
### 2. Architecture Astronomy
**Definition:** Designing elaborate systems without building/testing them.
**Evidence:**
- 265-line architecture doc written before any code
- 298-line session pattern parity analysis
- Multiple complete rewrites of the same feature
**Better:** Code first, document later. Spike solutions, learn from implementation.
### 3. Sunk Cost Fallacy
**Timeline:**
- **Hour 1:** "This seems complex but achievable"
- **Hour 2:** "We're halfway done, can't stop now"
- **Hour 3:** "Just need to fix this one coordination issue"
- **Hour 4:** "It's working, but... this feels wrong"
**Correct Decision:** Revert. Took courage to throw away 4 hours of work.
**Learning:** Time invested is not a reason to continue. Quality of outcome matters more.
### 4. Missing User Feedback Loop
**No User Input:**
- Didn't ask: "Is context relevance a problem for you?"
- Didn't test: "Does filtered context improve your responses?"
- Didn't measure: "Are you hitting context limits?"
**Engineering Theater:** Building impressive-sounding features without user validation.
## What We Actually Learned (The Real Value)
Despite reverting, this was productive R&D:
### 1. Deep Understanding of Hook Architecture
**Critical Discovery:** Hooks run in sandboxed environment without `claudePath`.
- Hooks cannot call Agent SDK `query()` directly
- All AI processing must happen in worker service
- This architectural constraint is now documented
**Learned Pattern:**
```
Hook (orchestration) → Worker (AI processing)
✓ save-hook: Captures data → Worker processes with SDK
✓ new-hook: Creates session → Worker returns confirmation
✗ jit-hook: Tried SDK in hook → Failed, no claudePath
```
**Value:** Future features will avoid this mistake. We now know the boundary.
### 2. Worker Architecture Patterns
**Blocking vs. Non-Blocking:**
- SessionStart: Can be non-blocking (context loads async)
- UserPromptSubmit: Must be blocking (session must exist before processing)
- JIT Context: Must be blocking (context needed before prompt processed)
**Established Pattern:**
```typescript
// Worker endpoint for features requiring AI
app.post('/sessions/:id/operation', async (req, res) => {
const { operationData } = req.body;
const result = await sdkAgent.performOperation(operationData);
return res.json({ result });
});
```
### 3. Persistent Session Management
**Architecture Knowledge Gained:**
- How to maintain long-lived SDK sessions
- EventEmitter coordination patterns for request/response
- Promise queue management for async operations
- Proper cleanup with AbortControllers
**Pattern Documented:**
- Dual session management (regular + JIT)
- Generator-based message loops
- Request queuing with timeouts
**Value:** When we build the simpler version, we'll know these patterns.
### 4. Configuration Infrastructure
`src/shared/settings.ts` (65 lines) provides reusable configuration patterns:
```typescript
export function getConfigValue(key: string, defaultValue: string): string {
// Priority: settings.json → env var → default
}
```
**Kept After Revert:** This module is useful for other features.
### 5. Key Architectural Decisions Made
**Decisions that will guide future implementation:**
1. JIT context filtering must happen in worker (proven via failed hook attempt)
2. Context must be blocking on UserPromptSubmit (session needs context before processing)
3. Dynamic timeline search is the right approach (fast, precise, leverages existing infrastructure)
4. Simple per-request queries should be tried before persistent sessions
### 6. Documentation Quality
- `jit-context-architecture-fix.md`: Documents why hooks can't run SDK queries
- `session-pattern-parity.md`: Reference for implementing dual sessions
- Hooks reference: Comprehensive hook documentation added
**Value:** These docs help future contributors understand the system constraints.
### 7. Infrastructure Validation
**Confirmed that our search stack is ready:**
- SQLite FTS5: Fast full-text search (<50ms)
- ChromaDB: Semantic search (<200ms with 8,000+ vectors)
- Timeline search API: Already implemented and tested
- Worker service: Can handle synchronous AI operations
**The infrastructure exists. We just need a simpler integration.**
## Recommendations
### Immediate Actions
1. **Archive the work:**
- Keep `failed/jit-context` branch for reference
- Extract reusable components (settings.ts)
- Save architecture docs for future features
2. **Document the anti-patterns:**
- Add this post-mortem to CLAUDE.md references
- Update coding standards with lessons learned
3. **Reset focus:**
- Return to validated user needs
- Prioritize features with clear value propositions
### Future Feature Development
**Gating Questions (Answer before coding):**
1. **User Value:** What specific user problem does this solve?
2. **Evidence:** Have users requested this or reported the underlying issue?
3. **Measurement:** How will we know if it's successful?
4. **Simplicity:** What's the dumbest version that could work?
5. **Time Limit:** If we can't prove value in 2 hours, should we build it?
**Process:**
```
VALIDATE → BUILD SIMPLE → TEST → MEASURE → DECIDE
↑ ↓
└──────────── ITERATE OR KILL ────────────┘
```
### If Context Filtering Returns
Should we revisit this idea in the future:
**Prerequisites:**
- User feedback requesting better context relevance
- Metrics showing current context is too broad
- Evidence that filtering improves response quality
**Simple Approach:**
```typescript
// In worker-service.ts /sessions/:id/init
if (jitEnabled) {
const observations = await db.getRecentObservations(project, 50);
const filtered = await simpleFilter(observations, userPrompt); // One-shot query
return { context: filtered };
}
```
**Acceptance Criteria:**
- <100 lines of code
- <500ms latency impact
- No new session types
- Degrades gracefully on errors
**If that works:** Then consider optimization.
## Conclusion
JIT context filtering failed not because the vision was wrong, but because we jumped to the complex implementation without validating the simple one first. The feature aligns with long-term goals (dynamic, prompt-specific context using our fast search infrastructure), but the persistent-session architecture was premature optimization.
**The right call:** Revert the complex implementation. Build the simple version when ready.
**Key Takeaway:** The vision is sound. The execution was overcomplicated. We now have:
- Deep knowledge of hook/worker architecture constraints
- Documented patterns for persistent SDK sessions
- Validated fast search infrastructure
- Clear understanding of what to build next time (simple timeline search API integration)
**This was R&D, not failure.** We learned what doesn't work (SDK in hooks), what does work (worker-based AI processing), and how to approach it next time (simple API calls before persistent sessions).
**Next Implementation:**
When we revisit this (and we should), start with:
1. Worker endpoint that accepts prompt
2. Queries existing timeline search API
3. Returns context
4. Hook injects context
5. Validate it improves responses
6. Then optimize if needed
**Final Thought:** Sometimes you have to build the wrong thing to understand the right thing. That's R&D.
---
**Branch Status:**
- `feature/jit-context`: Abandoned
- `failed/jit-context`: Archived for reference
- `main`: Stable at v5.4.0
**Files to Keep:**
- `src/shared/settings.ts`: Reusable config utilities
**Files Discarded:**
- Everything else (+2,850 lines)
**Emotional State:** Relieved. Dodged a maintenance nightmare.
File diff suppressed because it is too large Load Diff
+156
View File
@@ -0,0 +1,156 @@
# Rich Context Examples
This document shows what contextual data is available in transcripts
that could improve observation generation quality.
## Statistics
- Total entries: 369
- User messages: 74
- Assistant messages: 133
- Token usage: 67,465 total
- Cache efficiency: 6,979,410 tokens read from cache
## Conversation Flow
This shows how user requests, assistant reasoning, and tool executions flow together.
This is the rich context currently missing from individual tool observations.
---
### Example 1
#### 👤 User Request
```
Thank you for that. So now that you have a very deep understanding of what we are doing here, I'd like you to begin working on the enhancements to our prompts that leverage data using the transcript model we discovered
```
#### 🔧 Tools Executed (1)
**TodoWrite**
```json
{
"todos": [
{
"content": "Read the ROI implementation plan to understand full scope",
"status": "in_progress",
"activeForm": "Reading ROI implementation plan"
},
{
```
**📊 Data Available for This Exchange:**
- User intent: ✅ (218 chars)
- Assistant reasoning: ✅ (0 chars)
- Thinking process: ❌
- Tool executions: ✅ (1 tools)
- **Currently sent to memory worker:** Tool inputs/outputs only (no context!) ❌
---
### Example 2
#### 👤 User Request
```
Thank you for that. So now that you have a very deep understanding of what we are doing here, I'd like you to begin working on the enhancements to our prompts that leverage data using the transcript model we discovered
```
#### 🔧 Tools Executed (1)
**Glob**
- Pattern: `**/*roi*`
**📊 Data Available for This Exchange:**
- User intent: ✅ (218 chars)
- Assistant reasoning: ✅ (0 chars)
- Thinking process: ❌
- Tool executions: ✅ (1 tools)
- **Currently sent to memory worker:** Tool inputs/outputs only (no context!) ❌
---
### Example 3
#### 👤 User Request
```
Thank you for that. So now that you have a very deep understanding of what we are doing here, I'd like you to begin working on the enhancements to our prompts that leverage data using the transcript model we discovered
```
#### 🔧 Tools Executed (1)
**Glob**
- Pattern: `**/*implementation*plan*`
**📊 Data Available for This Exchange:**
- User intent: ✅ (218 chars)
- Assistant reasoning: ✅ (0 chars)
- Thinking process: ❌
- Tool executions: ✅ (1 tools)
- **Currently sent to memory worker:** Tool inputs/outputs only (no context!) ❌
---
### Example 4
#### 👤 User Request
```
Thank you for that. So now that you have a very deep understanding of what we are doing here, I'd like you to begin working on the enhancements to our prompts that leverage data using the transcript model we discovered
```
#### 🔧 Tools Executed (1)
**Read**
- Reading: `/Users/alexnewman/Scripts/claude-mem/docs/context/transcript-data-discovery.md`
**📊 Data Available for This Exchange:**
- User intent: ✅ (218 chars)
- Assistant reasoning: ✅ (0 chars)
- Thinking process: ❌
- Tool executions: ✅ (1 tools)
- **Currently sent to memory worker:** Tool inputs/outputs only (no context!) ❌
---
### Example 5
#### 👤 User Request
```
Thank you for that. So now that you have a very deep understanding of what we are doing here, I'd like you to begin working on the enhancements to our prompts that leverage data using the transcript model we discovered
```
#### 🔧 Tools Executed (1)
**Read**
- Reading: `/Users/alexnewman/Scripts/claude-mem/IMPLEMENTATION_PLAN_ROI_METRICS.md`
**📊 Data Available for This Exchange:**
- User intent: ✅ (218 chars)
- Assistant reasoning: ✅ (0 chars)
- Thinking process: ❌
- Tool executions: ✅ (1 tools)
- **Currently sent to memory worker:** Tool inputs/outputs only (no context!) ❌
---
## Key Insight
Currently, the memory worker receives **isolated tool executions** via save-hook:
- tool_name: "Read"
- tool_input: {"file_path": "src/foo.ts"}
- tool_output: {file contents}
But the transcript contains **rich contextual data**:
- WHY the tool was used (user's request)
- WHAT the assistant planned to accomplish
- HOW it fits into the broader task
- The assistant's reasoning/thinking
- Multiple related tools used together
This context would help the memory worker:
1. Understand if a tool use is meaningful or routine
2. Generate observations that capture WHY, not just WHAT
3. Group related tools into coherent actions
4. Avoid "investigating" - the context is already present
@@ -0,0 +1,857 @@
# Search Architecture Analysis
**Date:** 2025-11-11 **Scope:** HTTP API endpoints, MCP search server, DRY violations, architectural recommendations
---
## Current State: Dual Search Architectures
### Architecture Overview
```
┌─────────────────────────────────────────────────────────────┐
│ Claude Code Session │
│ │
│ ┌────────────────────────────────────────────────────┐ │
│ │ mem-search Skill (ACTIVE) │ │
│ │ - Uses HTTP API via curl commands │ │
│ │ - 10 search operations │ │
│ │ - Progressive disclosure workflow │ │
│ └────────────────────────────────────────────────────┘ │
│ │ │
│ │ HTTP GET │
│ ▼ │
│ ┌────────────────────────────────────────────────────┐ │
│ │ MCP Search Server (DEPRECATED but BUILT) │ │
│ │ - .mcp.json configured │ │
│ │ - search-server.mjs exists (74KB) │ │
│ │ - 9 MCP tools defined │ │
│ │ - Not used by skill │ │
│ └────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
┌───────────┴───────────┐
▼ ▼
┌──────────────────────────┐ ┌──────────────────────────┐
│ Worker Service │ │ MCP Server │
│ (worker-service.ts) │ │ (search-server.ts) │
│ │ │ │
│ 10 HTTP Endpoints: │ │ 9 MCP Tools: │
│ ├─ /api/search/ │ │ ├─ search_observations │
│ │ observations │ │ ├─ search_sessions │
│ ├─ /api/search/ │ │ ├─ search_user_prompts │
│ │ sessions │ │ ├─ find_by_concept │
│ ├─ /api/search/ │ │ ├─ find_by_file │
│ │ prompts │ │ ├─ find_by_type │
│ ├─ /api/search/ │ │ ├─ get_recent_context │
│ │ by-concept │ │ ├─ get_context_timeline │
│ ├─ /api/search/ │ │ └─ get_timeline_by_query│
│ │ by-file │ │ │
│ ├─ /api/search/ │ │ Built: ✅ │
│ │ by-type │ │ Used: ❌ │
│ ├─ /api/context/recent │ │ Configured: ✅ │
│ ├─ /api/context/ │ │ Status: DEPRECATED │
│ │ timeline │ │ │
│ ├─ /api/timeline/ │ │ │
│ │ by-query │ │ │
│ └─ /api/search/help │ │ │
│ │ │ │
│ Built: ✅ │ │ │
│ Used: ✅ │ │ │
│ Status: ACTIVE │ │ │
└──────────────────────────┘ └──────────────────────────┘
│ │
└─────────┬─────────────────┘
┌────────────────────────────────┐
│ SessionSearch (Shared Layer) │
│ - FTS5 queries │
│ - SQLite operations │
│ - Common data access │
└────────────────────────────────┘
┌────────────────────────────────┐
│ SQLite Database │
│ ~/.claude-mem/claude-mem.db │
└────────────────────────────────┘
```
---
## HTTP Endpoints Architecture
### Location
`src/services/worker-service.ts` (lines 108-118, 748-1174)
### Endpoints (10 total)
| Endpoint | Method | Purpose | Used By |
| -------------------------- | ------ | ----------------------------------- | ---------------- |
| `/api/search/observations` | GET | Full-text search observations | mem-search skill |
| `/api/search/sessions` | GET | Full-text search session summaries | mem-search skill |
| `/api/search/prompts` | GET | Full-text search user prompts | mem-search skill |
| `/api/search/by-concept` | GET | Find observations by concept tag | mem-search skill |
| `/api/search/by-file` | GET | Find work related to specific files | mem-search skill |
| `/api/search/by-type` | GET | Find observations by type | mem-search skill |
| `/api/context/recent` | GET | Get recent session context | mem-search skill |
| `/api/context/timeline` | GET | Get timeline around point in time | mem-search skill |
| `/api/timeline/by-query` | GET | Search + timeline in one call | mem-search skill |
| `/api/search/help` | GET | API documentation | mem-search skill |
### Implementation Pattern
**Example: Search Observations**
```typescript
// src/services/worker-service.ts:748-781
private handleSearchObservations(req: Request, res: Response): void {
try {
// 1. Parse query parameters
const query = req.query.query as string;
const format = (req.query.format as string) || 'full';
const limit = parseInt(req.query.limit as string, 10) || 20;
const project = req.query.project as string | undefined;
// 2. Validate required parameters
if (!query) {
res.status(400).json({ error: 'Missing required parameter: query' });
return;
}
// 3. Call SessionSearch (shared data layer)
const sessionSearch = this.dbManager.getSessionSearch();
const results = sessionSearch.searchObservations(query, { limit, project });
// 4. Format response based on format parameter
res.json({
query,
count: results.length,
format,
results: format === 'index' ? results.map(r => ({
id: r.id,
type: r.type,
title: r.title,
subtitle: r.subtitle,
created_at_epoch: r.created_at_epoch,
project: r.project,
score: r.score
})) : results
});
} catch (error) {
logger.failure('WORKER', 'Search observations failed', {}, error as Error);
res.status(500).json({ error: (error as Error).message });
}
}
```
### Characteristics
**Pros:**
- ✅ Simple HTTP GET requests (curl-friendly)
- ✅ Standard REST API pattern
- ✅ Easy to test and debug
- ✅ No MCP protocol overhead
- ✅ Works with any HTTP client
**Cons:**
- ⚠️ Parameter parsing duplicated across 10 endpoints
- ⚠️ Format conversion logic duplicated
- ⚠️ Error handling pattern repeated
---
## MCP Search Server Architecture
### Location
`src/servers/search-server.ts` (1,781 lines)
### Status
- **Built:** ✅ Yes (`plugin/scripts/search-server.mjs`, 74KB)
- **Configured:** ✅ Yes (`.mcp.json` line 3-6)
- **Used:** ❌ No (deprecated in v5.4.0)
- **Maintained:** ⚠️ Source kept for reference
### Tools (9 total)
| Tool Name | Purpose | Line |
| ----------------------- | -------------------------------------- | -------- |
| `search_observations` | Search observations with FTS5 + Chroma | 348-422 |
| `search_sessions` | Search session summaries | 438-490 |
| `search_user_prompts` | Search user prompts | 506-558 |
| `find_by_concept` | Find by concept tag | 574-626 |
| `find_by_file` | Find by file path | 642-694 |
| `find_by_type` | Find by observation type | 710-762 |
| `get_recent_context` | Get recent sessions | 778-830 |
| `get_context_timeline` | Get timeline context | 846-950 |
| `get_timeline_by_query` | Search + timeline | 966-1064 |
### Implementation Pattern
**Example: Search Observations (MCP)**
```typescript
// src/servers/search-server.ts:348-422
{
name: 'search_observations',
description: 'Search observations using full-text search across titles, narratives, facts, and concepts...',
inputSchema: z.object({
query: z.string().describe('Search query for FTS5 full-text search'),
format: z.enum(['index', 'full']).default('index').describe('...'),
...filterSchema.shape
}),
handler: async (args: any) => {
try {
const { query, format = 'index', ...options } = args;
let results: ObservationSearchResult[] = [];
// Hybrid search: Try Chroma semantic search first, fall back to FTS5
if (chromaClient) {
try {
// Step 1: Chroma semantic search (top 100)
const chromaResults = await queryChroma(query, 100);
if (chromaResults.ids.length > 0) {
// Step 2: Filter by recency (90 days)
const ninetyDaysAgo = Date.now() - (90 * 24 * 60 * 60 * 1000);
const recentIds = chromaResults.ids.filter((_id, idx) => {
const meta = chromaResults.metadatas[idx];
return meta && meta.created_at_epoch > ninetyDaysAgo;
});
// Step 3: Hydrate from SQLite
if (recentIds.length > 0) {
const limit = options.limit || 20;
results = store.getObservationsByIds(recentIds, { orderBy: 'date_desc', limit });
}
}
} catch (chromaError: any) {
console.error('[search-server] Chroma query failed, falling back to FTS5:', chromaError.message);
}
}
// Fall back to FTS5 if Chroma unavailable or returned no results
if (results.length === 0) {
results = search.searchObservations(query, options);
}
// Format results
if (format === 'index') {
return {
content: [{
type: 'text',
text: results.map((r, i) => formatObservationIndex(r, i)).join('\n\n') + formatSearchTips()
}]
};
} else {
return {
content: results.map(r => ({
type: 'resource',
resource: {
uri: `claude-mem://observation/${r.id}`,
mimeType: 'text/markdown',
text: formatObservationResult(r)
}
}))
};
}
} catch (error: any) {
return { content: [{ type: 'text', text: `Error: ${error.message}` }] };
}
}
}
```
### Characteristics
**Pros:**
- ✅ MCP protocol support
- ✅ Hybrid search (Chroma + FTS5)
- ✅ Rich formatting (markdown, resources)
- ✅ Comprehensive error handling
**Cons:**
- ❌ Not used by skill (deprecated)
- ❌ ~2,500 token overhead for tool definitions
- ❌ More complex than HTTP
- ❌ Still being built despite deprecation
---
## DRY Violation Analysis
### Areas of Duplication
#### 1. **Parameter Parsing** (10 HTTP endpoints + 9 MCP tools)
**HTTP Endpoints:**
```typescript
// Repeated in each endpoint handler
const query = req.query.query as string;
const format = (req.query.format as string) || "full";
const limit = parseInt(req.query.limit as string, 10) || 20;
const project = req.query.project as string | undefined;
if (!query) {
res.status(400).json({ error: "Missing required parameter: query" });
return;
}
```
**MCP Tools:**
```typescript
// Repeated in each tool handler
const { query, format = "index", ...options } = args;
if (!query) {
throw new Error("Missing required parameter: query");
}
```
**Violation:** Parameter parsing logic duplicated 19 times (10 + 9)
#### 2. **Format Conversion** (Index vs Full)
**HTTP Endpoints:**
```typescript
results: format === "index"
? results.map((r) => ({
id: r.id,
type: r.type,
title: r.title,
subtitle: r.subtitle,
created_at_epoch: r.created_at_epoch,
project: r.project,
score: r.score,
}))
: results;
```
**MCP Tools:**
```typescript
if (format === "index") {
return {
content: [
{
type: "text",
text: results.map((r, i) => formatObservationIndex(r, i)).join("\n\n"),
},
],
};
} else {
return {
content: results.map((r) => ({
type: "resource",
resource: {
uri: `claude-mem://observation/${r.id}`,
mimeType: "text/markdown",
text: formatObservationResult(r),
},
})),
};
}
```
**Violation:** Format conversion logic duplicated with different output formats
#### 3. **Search Logic Duplication**
**HTTP Endpoints:**
```typescript
const sessionSearch = this.dbManager.getSessionSearch();
const results = sessionSearch.searchObservations(query, { limit, project });
```
**MCP Tools:**
```typescript
// Hybrid search with Chroma fallback
if (chromaClient) {
const chromaResults = await queryChroma(query, 100);
// ... complex hybrid logic ...
}
if (results.length === 0) {
results = search.searchObservations(query, options);
}
```
**Violation:** MCP has hybrid Chroma+FTS5 search, HTTP only has FTS5
#### 4. **Error Handling**
**HTTP Endpoints:**
```typescript
try {
// ... handler logic ...
} catch (error) {
logger.failure("WORKER", "Search observations failed", {}, error as Error);
res.status(500).json({ error: (error as Error).message });
}
```
**MCP Tools:**
```typescript
try {
// ... handler logic ...
} catch (error: any) {
return { content: [{ type: "text", text: `Error: ${error.message}` }] };
}
```
**Violation:** Different error handling patterns
### DRY Compliance at Data Layer ✅
**Good news:** Both architectures use the **same data layer**:
```
HTTP Endpoints → SessionSearch → SQLite
MCP Tools → SessionSearch → SQLite
```
The `SessionSearch` class is the **single source of truth** for data access. No duplication there.
---
## Is curl the Best Approach?
### Current Approach: curl Commands
**Example from skill:**
```bash
curl -s "http://localhost:37777/api/search/observations?query=authentication&format=index&limit=5"
```
### Alternative Approaches
#### 1. **MCP Tools** (Deprecated)
**Pros:**
- Native Claude Code protocol
- Rich type definitions
- Better error handling
- Resource formatting
**Cons:**
- ❌ ~2,500 token overhead per session
- ❌ More complex to implement
- ❌ Requires MCP server process
- ❌ Less accessible for external tools
**Verdict:** MCP was deprecated for good reasons (token overhead). curl is better.
#### 2. **Direct Database Access** (Not feasible)
**Pros:**
- No HTTP overhead
- No worker process needed
**Cons:**
- ❌ Skills can't access files directly
- ❌ No way to execute TypeScript/SQLite from skill
- ❌ Would require building native bindings
**Verdict:** Not possible with current skill architecture.
#### 3. **HTTP API via curl** (Current) ✅
**Pros:**
- ✅ Simple, standard protocol
- ✅ Works with skill architecture
- ✅ Easy to test (curl in terminal)
- ✅ Language-agnostic
- ✅ No MCP token overhead
- ✅ RESTful design
**Cons:**
- ⚠️ Requires worker service running
- ⚠️ HTTP parsing overhead (minimal)
**Verdict:** **Best approach given constraints.**
### Why curl is Optimal
1. **Skill Constraints:** Skills can only execute shell commands. curl is the standard HTTP client.
2. **Token Efficiency:** No tool definitions loaded into context (~2,250 token savings).
3. **Progressive Disclosure:** Skill loads gradually, HTTP requests are made only when needed.
4. **Debuggability:** Easy to test endpoints manually with curl.
5. **Cross-platform:** curl available on all platforms.
---
### Question: "Is it routing into the search-service MCP file or is it a DRY violation?"
**Answer:** Both architectures exist, creating a DRY violation:
1. **HTTP Endpoints** (worker-service.ts) ← **Used by skill**
2. **MCP Server** (search-server.ts) ← **Deprecated but still built**
### Current State
```
mem-search skill → HTTP API (worker-service.ts) → SessionSearch → SQLite
MCP search server (deprecated) → SessionSearch ──────────────────────┘
```
Both use the same data layer (SessionSearch), but:
- ❌ Parameter parsing duplicated
- ❌ Format conversion duplicated
- ❌ MCP has hybrid Chroma search, HTTP doesn't
- ❌ MCP still being built despite deprecation
**You said:** "We are intentionally exposing API search endpoints
```
┌─────────────────────────────────────────────────────────────┐
│ - Web UI │
│ - Mobile app │
│ - VS Code extension │
│ - CLI tools │
└─────────────────────────────────────────────────────────────┘
│ HTTP API
┌─────────────────────────────────────────────────────────────┐
│ Worker Service HTTP API │
│ localhost:37777/api/search/* │
│ │
│ - Standard REST endpoints │
│ - JSON responses │
│ - Query parameter API │
│ - format=index/full support │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ SessionSearch + ChromaSync │
│ (Shared data layer) │
└─────────────────────────────────────────────────────────────┘
```
- Standard REST API
- Easy to consume from any language/platform
- Already supports format=index/full for token efficiency
- Well-documented in skill operation guides
- Clean JSON responses
---
## Architectural Recommendations
### Immediate Actions
#### 1. **Remove MCP Search Server** (Reduce Maintenance Burden)
**Problem:**
- MCP server is deprecated but still being built
- Adds 1,781 lines of maintenance burden
- Creates confusion about which search to use
- DRY violation with HTTP endpoints
**Recommendation:**
```bash
# Remove from build pipeline
# scripts/build-hooks.js - already commented out, make permanent
# Delete configuration
rm plugin/.mcp.json
# Archive source (don't delete, keep for reference)
git mv src/servers/search-server.ts archive/search-server.ts.archived
# Remove built file
rm plugin/scripts/search-server.mjs
```
**Impact:**
- ✅ Reduces build time
- ✅ Eliminates confusion
- ✅ Reduces maintenance burden
- ✅ Removes DRY violation
- ⚠️ Loses hybrid Chroma search in MCP (but HTTP doesn't have it anyway)
#### 2. **Add Hybrid Search to HTTP Endpoints** (Feature Parity)
**Problem:** MCP server has Chroma hybrid search, HTTP endpoints don't
**Recommendation:**
```typescript
// src/services/worker-service.ts
private async handleSearchObservations(req: Request, res: Response): Promise<void> {
try {
const { query, format, limit, project } = this.parseSearchParams(req);
// Try hybrid search first if Chroma available
let results = await this.hybridSearch(query, { limit, project });
// Fallback to FTS5 if Chroma unavailable
if (results.length === 0) {
const sessionSearch = this.dbManager.getSessionSearch();
results = sessionSearch.searchObservations(query, { limit, project });
}
res.json(this.formatSearchResponse(query, results, format));
} catch (error) {
this.handleSearchError(res, 'Search observations failed', error);
}
}
// Extract shared methods
private parseSearchParams(req: Request): SearchParams { /* ... */ }
private async hybridSearch(query: string, options: SearchOptions): Promise<any[]> { /* ... */ }
private formatSearchResponse(query: string, results: any[], format: string): any { /* ... */ }
private handleSearchError(res: Response, message: string, error: any): void { /* ... */ }
```
**Impact:**
- ✅ Adds Chroma semantic search to HTTP API
- ✅ Makes HTTP API feature-complete
#### 3. **Extract Shared Search Logic** (DRY Refactoring)
**Problem:** 10 HTTP endpoints have duplicated parameter parsing and formatting
**Recommendation:**
```typescript
// src/services/search/SearchController.ts (new file)
export class SearchController {
constructor(private sessionSearch: SessionSearch, private chromaSync: ChromaSync) {}
async searchObservations(params: SearchParams): Promise<SearchResponse> {
// Shared logic for observations search
const results = await this.hybridSearch(params);
return this.formatResponse(results, params.format);
}
async searchSessions(params: SearchParams): Promise<SearchResponse> {
// Shared logic for sessions search
}
// ... other search methods
private async hybridSearch(params: SearchParams): Promise<any[]> {
// Shared hybrid search logic
}
private formatResponse(results: any[], format: "index" | "full"): SearchResponse {
// Shared formatting logic
}
private parseParams(req: Request): SearchParams {
// Shared parameter parsing
}
}
```
**Usage in worker-service.ts:**
```typescript
private searchController: SearchController;
private handleSearchObservations(req: Request, res: Response): void {
try {
const params = this.searchController.parseParams(req);
const response = await this.searchController.searchObservations(params);
res.json(response);
} catch (error) {
this.handleSearchError(res, error);
}
}
```
**Impact:**
- ✅ Eliminates 90% of duplication across 10 endpoints
- ✅ Single source of truth for search logic
- ✅ Easier to test (test controller, not HTTP layer)
- ✅ Easier to maintain
- ✅ Easier to add new search endpoints
### Long-term Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Clients │
│ ┌──────────────┬──────────────┬──────────────────────┐ │
│ │ Skill │ Frontend │ (CLI, IDE plugins) │ │
│ └──────────────┴──────────────┴──────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│ HTTP API (REST)
┌─────────────────────────────────────────────────────────────┐
│ WorkerService (Express.js) │
│ │
│ Route Layer (thin) │
│ ├─ GET /api/search/observations │
│ ├─ GET /api/search/sessions │
│ └─ ... (delegates to controller) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ SearchController (business logic) │
│ │
│ ├─ searchObservations() │
│ ├─ searchSessions() │
│ ├─ hybridSearch() - Chroma + FTS5 │
│ ├─ formatResponse() - index/full conversion │
│ └─ parseParams() - parameter validation │
└─────────────────────────────────────────────────────────────┘
┌───────────┴───────────┐
▼ ▼
┌──────────────────────────┐ ┌──────────────────────────┐
│ SessionSearch (FTS5) │ │ ChromaSync (Vectors) │
│ - searchObservations() │ │ - queryByEmbedding() │
│ - searchSessions() │ │ - 90-day recency filter │
│ - searchPrompts() │ │ - Hydrate from SQLite │
└──────────────────────────┘ └──────────────────────────┘
│ │
└─────────┬─────────────────┘
┌────────────────────────────────┐
│ SQLite Database │
│ ~/.claude-mem/claude-mem.db │
└────────────────────────────────┘
```
---
## Summary
### Current Architecture Issues
1.**Dual search implementations** (HTTP + deprecated MCP)
2.**DRY violations** across 19 search handlers
3.**MCP server still built** despite deprecation
4.**HTTP missing hybrid Chroma search** (MCP has it)
5.**No shared controller layer** for search logic
### Is curl the Best Approach?
**Yes.**
Given the constraints:
- Skills can only execute shell commands
- Token efficiency vs MCP (~2,250 token savings)
- Standard REST pattern, easy to consume
curl + HTTP API is the optimal architecture.
### Is it Routing into search-service or DRY Violation?
**DRY violation.**
Both architectures exist and duplicate logic:
- HTTP endpoints (worker-service.ts) ← ACTIVE
- MCP server (search-server.ts) ← DEPRECATED but BUILT
They share the data layer (SessionSearch) but duplicate:
- Parameter parsing
- Format conversion
- Error handling
- Search orchestration (MCP has Chroma, HTTP doesn't)
### Recommendations Priority
**High Priority:**
1. ✅ Remove MCP search server entirely (archive source)
2. ✅ Add hybrid Chroma search to HTTP endpoints
3. ✅ Extract SearchController for shared logic
**Medium Priority:**
5. Add API versioning (/api/v1/search/\*)
6. Add rate limiting for external access
**Low Priority:** 7. OpenAPI/Swagger documentation
9. WebSocket support for real-time search
### Action Plan
**Phase 1: Cleanup (1 day)**
- Remove .mcp.json
- Archive search-server.ts
- Update CLAUDE.md to reflect removal
- Update build scripts to skip MCP server
**Phase 2: Feature Parity (2 days)**
- Port hybrid Chroma search from MCP to HTTP
- Test all 10 endpoints with hybrid search
- Update skill documentation
**Phase 3: DRY Refactoring (3 days)**
- Create SearchController class
- Extract shared logic (parsing, formatting, errors)
- Refactor 10 HTTP handlers to use controller
- Add comprehensive tests
- Document API for external consumption
- Add authentication/authorization (if needed)
- Add rate limiting
- Create OpenAPI spec
---
## Files Referenced
**Active:**
- `src/services/worker-service.ts` - HTTP endpoints (1,338 lines)
- `src/services/sqlite/SessionSearch.ts` - FTS5 search
- `src/services/sync/ChromaSync.ts` - Vector search
- `plugin/skills/mem-search/SKILL.md` - Skill using HTTP API
**Deprecated:**
- `src/servers/search-server.ts` - MCP tools (1,781 lines)
- `plugin/.mcp.json` - MCP configuration
- `plugin/scripts/search-server.mjs` - Built MCP server (74KB)
**Configuration:**
- `CLAUDE.md` line 314 - Deprecation notice
- `CHANGELOG.md` line 32-52 - v5.4.0 migration
- `scripts/build-hooks.js` - Build pipeline (MCP commented out)
+160
View File
@@ -0,0 +1,160 @@
# Skill Audit Report
**Date:** 2025-11-10
**Validation:** Anthropic's official skill-creator documentation
**Skills Audited:** mem-search, search
## Executive Summary
The mem-search skill achieves 100% compliance across all dimensions. The search skill meets technical requirements but fails effectiveness metrics critical for auto-invocation.
**mem-search:** Production-ready. No changes required.
**search:** Requires three critical fixes before Claude reliably discovers and invokes this skill.
## mem-search Skill Results
**Status:** ✅ PASS
**Compliance:** 100% technical, 100% effectiveness
**Files:** 17 (202-line SKILL.md + 13 operations + 2 principles)
### Strengths
The skill demonstrates exemplary effectiveness engineering:
1. **Trigger Design (85% concrete)**
- Five unique identifiers: claude-mem, PM2-managed database, cross-session memory, session summaries, observations
- Nine scope differentiation keywords
- Explicit boundary: "NOT in the current conversation context"
- Minimal overlap with Claude's native capabilities
2. **Capability Visibility (100%)**
- All nine operations include inline "Use when" examples
- Decision guide reduces complexity from nine operations to five common cases
- No navigation friction
3. **Structure**
- 202 lines (60% under limit)
- Perfect progressive disclosure with token cost documentation
- Clean file organization: operations/ and principles/ directories
- No content duplication
### Issues
**One false positive:** Line 152 contains backslashes in regex notation `(bugfix\|feature\|decision)`. This documents parameter syntax, not Windows paths. No action required.
## search Skill Results
**Status:** ⚠️ NEEDS IMPROVEMENT
**Compliance:** 100% technical, 67% effectiveness
**Files:** 13 (96-line SKILL.md + 12 operations)
### Critical Effectiveness Issues
Three failures prevent reliable auto-invocation:
#### Issue 1: Insufficient Scope Differentiation
**Problem:** Description contains only two differentiation keywords (threshold: ≥3). Claude cannot distinguish this skill from native conversation memory.
**Current description:**
```text
Search claude-mem persistent memory for past sessions, observations, bugs
fixed, features implemented, decisions made, code changes, and previous work.
Use when answering questions about history, finding past decisions, or
researching previous implementations.
```
**Domain overlap analysis:**
- Claude answers natively: "What bugs did we fix?" (current conversation)
- Claude needs skill: "What bugs did we fix last week?" (external database)
**Fix required:**
```text
Search claude-mem's external database of past sessions, observations, and
work from previous conversations. Accesses persistent memory stored outside
current session context - NOT information from today's conversation. Use when
users ask about: (1) previous sessions ("what did we do last week?"),
(2) historical work ("bugs we fixed months ago"), (3) cross-session patterns
("how have we approached this before?"), (4) work already stored in claude-mem
("what's in the database about X?"). Searches FTS5 full-text index across
typed observations (bugfix/feature/refactor/decision/discovery). For current
session memory, use native conversation context instead.
```
This adds eight differentiation keywords: "external database", "past sessions", "previous conversations", "outside current session", "NOT information from today's", "last week", "months ago", "already stored in claude-mem".
#### Issue 2: Weak Trigger Specificity
**Problem:** Only 44% concrete triggers (threshold: >50%). Only one unique identifier (threshold: ≥2).
**Abstract triggers (low specificity):**
- "history" (could mean git history, browser history)
- "past work" (could mean files, commits, documents)
- "decisions" (could mean any decision tracking)
- "previous work" (could mean current session earlier)
- "implementations" (could mean code in current conversation)
**Concrete triggers (high specificity):**
- "claude-mem" (unique system name)
- "persistent memory" (system-specific)
- "sessions" (cross-session concept)
- "observations" (system-specific)
**Concrete ratio:** 4/9 = 44% (fails 50% threshold)
**Fix required:** Add system-specific terminology: "HTTP API", "port 37777", "FTS5 full-text index", "typed observations". See combined description in Issue 1 fix.
#### Issue 3: Wasted Content in Body
**Problem:** Lines 10-22 contain "When to Use This Skill" section in SKILL.md body. This loads AFTER triggering, wastes ~200 tokens, provides no value.
**Reference:** [Anthropic's skill-creator documentation](https://github.com/anthropics/anthropic-quickstarts/tree/main/skill-creator) states: "The body is only loaded after triggering, so 'When to Use This Skill' sections in the body are not helpful to Claude."
**Fix required:** Delete lines 10-22 entirely. Move triggering examples to description field (already included in Issue 1 fix).
### Strengths
The skill demonstrates strong structure:
- Excellent progressive disclosure (96-line navigation hub)
- Strong decision guide (reduces 10 operations to common cases)
- 100% capability visibility (all operations show purpose inline)
- No forbidden files or content duplication
- Clean operations/ directory structure
### Warning
**Minor:** Description uses imperative "Use when" instead of third person. Change to "Useful for" or "Invoked when" for consistency with skill-creator best practices.
## Comparison
| Metric | mem-search | search | Impact |
|--------|-----------|---------|--------|
| Concrete triggers | 85% | 44% | search harder to discover |
| Unique identifiers | 5+ | 1 | search less distinct |
| Scope differentiation | 9 keywords | 2 keywords | **search conflicts with native memory** |
| Body optimization | Clean | Wasted section | search wastes tokens |
| Overall effectiveness | 100% | 67% | search needs fixes |
## Critical Recommendations
The search skill requires three changes before production use:
1. **Rewrite description** to add scope differentiation and concrete triggers (see Issue 1 fix)
2. **Delete lines 10-22** from SKILL.md body
3. **Convert to third person** - change "Use when" to "Useful for"
**Why this matters:** Without scope differentiation, Claude assumes "What bugs did we fix?" refers to current conversation, not the external claude-mem database. This causes systematic under-invocation.
## Reference Implementation
The mem-search skill serves as a reference implementation for:
- Trigger design with explicit scope boundaries
- Progressive disclosure with token efficiency documentation
- Inline capability visibility eliminating navigation friction
- Decision guides reducing cognitive load
Study mem-search when creating skills that overlap with Claude's native capabilities.
@@ -0,0 +1,467 @@
# Transcript: 57dcc12f-4751-46bb-82b4-2aa96a3e226d.jsonl
**Generated:** 11/13/2025, 3:58:53 PM
**Total Entries:** 448
**Entry Types:** {
"summary": 149,
"file-history-snapshot": 20,
"user": 98,
"assistant": 181
}
**Showing:** First 10 conversation turns
---
## Turn 1
### 👤 User
**Timestamp:** 2025-11-13T17:09:47.231Z
**UUID:** 5c4b246c-2100-4e6b-a275-f0bb62bba020
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
**CWD:** /Users/alexnewman/Scripts/claude-mem
I have a question regarding the context timeline that we made. and it's a bit of a thinker...
when i first started out working on claude-mem my idea was using a fast index of memories with links to access them on demand. I thought that claude-mem would be searching for all the things it needed to get the job done, but what I found is that the description of the work done + the files touched actually improved performance tremendously, and without actively searching for things constantly.
I just had a thought that maybe claude isn't searching, because when it reads what it did, it already "knows" how it did that, for the most part. Would you say I'm on the right track? I want you to ultrathink about WHY claude-mem's session start context works so well, from your OWN perspective
### 🤖 Assistant
**Timestamp:** 2025-11-13T17:10:28.576Z
**UUID:** 1dd9591b-c98b-4031-ac0a-2f7e067b427a
**Model:** claude-sonnet-4-5-20250929
**Stop Reason:** tool_use
**💭 Thinking:**
> This is a fascinating question about the fundamental mechanics of how LLMs process context and why claude-mem's approach is working so well. The user is asking me to think deeply about WHY the session start context (descriptions + file paths) works better than active searching.
>
> Let me think about this from my own perspective as Claude...
>
> The user's hypothesis is: "Claude isn't searching because when it reads what it did, it already 'knows' how it did that."
>
> This is a really interesting observation. Let me think about what happens when I see context at session start vs when I actively search:
>
> 1. **Session Start Context**: I see summaries like "Fixed authentication bug in src/auth/login.ts by updating token validation logic" with file paths
> 2. **Active Search**: I use grep/search tools to find specific code or patterns
>
> The key insight the user is pointing to: When I see the session summary, it's triggering my training knowledge about how these kinds of problems are typically solved. The description + file path combo is enough to reconstruct the likely solution pattern.
>
> Let me ultrathink about this...
>
> **Information Density & Compression**:
> - A summary like "Added JWT token refresh logic to src/auth/token-manager.ts" is highly compressed but information-rich
> - It contains: the pattern (JWT refresh), the location (token-manager), and the action type (added)
> - This maps to my training data about how JWT refresh is typically implemented
> - The file path gives architectural context about where this lives in the codebase
>
> **Pattern Recognition vs Literal Recall**:
> - I don't need to see the exact code to "know" what was probably done
> - JWT token refresh follows common patterns that I've seen thousands of times in training
> - The summary activates these patterns in my understanding
> - It's like how a human developer reading "fixed the N+1 query problem" immediately knows the general shape of the solution without seeing the code
>
> **Contextual Scaffolding**:
> - File paths provide architectural scaffolding: "src/hooks/context-hook.ts" tells me this is a hook in a hooks directory
>
>
> - The summary provides semantic scaffolding: what was accomplished
> - Together they form a mental model of the codebase without needing to read every line
>
> **Search is for Unknowns, Summaries are for Knowns**:
> - I search when I don't know something: "Where is error handling done?"
> - I don't need to search when the summary tells me: "Enhanced error handling in src/services/worker-service.ts"
> - The summary has already done the work of locating and describing the relevant information
>
> **The Compression-Decompression Cycle**:
> - The AI compression (creating observations) is like creating a highly optimized index
> - When I encounter a summary, I'm essentially decompressing that index into actionable knowledge
> - This process is far more efficient than searching through raw code or logs
>
> The summaries act like cognitive shortcuts, instantly reconstructing complex contextual understanding through minimal, strategic information.
>
> **Why This Works Better Than Constant Searching**:
> 1. Drastically reduces mental overhead of navigating complex systems
> 2. Enables rapid pattern recognition across different code segments
> 3. Provides architectural insights without deep diving into implementation details
> 4. Minimizes computational and cognitive resources spent on exploration
> 5. Delivers pre-filtered, contextually relevant information
>
> **The "Already Knows" Hypothesis**:
> I recognize complex technical changes instantly through pattern matching. When encountering a concise code modification description, my training allows immediate comprehension by activating relevant technical knowledge domains. The summary triggers an intuitive understanding of systemic implications, similar to how experienced developers quickly grasp architectural shifts through minimal context.
>
> **The Real Magic**:
> Leveraging deep pattern recognition capabilities, I can reconstruct technical landscapes from minimal descriptive signals. File paths and succinct change descriptions become powerful activation keys that unlock comprehensive understanding without exhaustive code examination.
>
> **Why Links-On-Demand Falls Short**:
> Retrieving information through multiple steps disrupts cognitive flow. Contextual understanding emerges more naturally through continuous narrative, where each detail builds upon previous insights. Searching and parsing interrupt the organic comprehension process, making knowledge integration less fluid and intuitive.
>
> **The Timeline Innovation**:
> Project evolution unfolds like a narrative, with each session representing a chapter in technological development. Temporal relationships between changes reveal deeper systemic transformations, mirroring how humans conceptualize complex system growth through interconnected stories of incremental progress.
>
> The essence lies in providing compressed, meaningful representations that leverage my inherent training knowledge. By presenting contextual frameworks that enable rapid pattern recognition, understanding emerges more elegantly than through fragmented, manually retrieved information.
**📊 Token Usage:**
- Input: 10
- Output: 1495
- Cache creation: 28036
---
## Turn 2
### 👤 User
**Timestamp:** 2025-11-13T17:10:31.963Z
**UUID:** 364676a7-51c3-4036-afc3-7ff8f7301a8f
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
**CWD:** /Users/alexnewman/Scripts/claude-mem
**Tool Results Submitted (1):**
- **Tool Use ID:** `toolu_01T477WUra1sDR6gHaqZHhKT`
```json
{
"thoughtNumber": 1,
"totalThoughts": 8,
"nextThoughtNeeded": true,
"branches": [],
"thoughtHistoryLength": 1
}
```
### 🤖 Assistant
**Timestamp:** 2025-11-13T17:10:35.939Z
**UUID:** 201d5077-aae8-4e26-9c5e-6a63f1ca88fa
**Model:** claude-sonnet-4-5-20250929
**Stop Reason:** tool_use
**💭 Thinking:**
> Good, I'm using the sequential thinking tool to really dig into this. Let me continue with the next thought about information density.
**📊 Token Usage:**
- Input: 12
- Output: 256
- Cache creation: 1529
- Cache read: 28036
---
## Turn 3
### 👤 User
**Timestamp:** 2025-11-13T17:10:38.354Z
**UUID:** 6c2f5da2-6a5a-4742-a46d-6dd5d193ad39
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
**CWD:** /Users/alexnewman/Scripts/claude-mem
**Tool Results Submitted (1):**
- **Tool Use ID:** `toolu_01QMMWEJm8yH25VtBmAzYbmy`
```json
{
"thoughtNumber": 2,
"totalThoughts": 8,
"nextThoughtNeeded": true,
"branches": [],
"thoughtHistoryLength": 2
}
```
### 🤖 Assistant
**Timestamp:** 2025-11-13T17:10:41.841Z
**UUID:** 3e6f3e78-5c12-4739-a7aa-a8c6fc9d2a33
**Model:** claude-sonnet-4-5-20250929
**Stop Reason:** tool_use
**💭 Thinking:**
> Good, now let me think about the difference between pattern recognition and literal recall.
**📊 Token Usage:**
- Input: 11
- Output: 262
- Cache creation: 323
- Cache read: 29565
---
## Turn 4
### 👤 User
**Timestamp:** 2025-11-13T17:10:45.430Z
**UUID:** f99751a0-b13a-446a-9b3b-dfc08ffb8e5b
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
**CWD:** /Users/alexnewman/Scripts/claude-mem
**Tool Results Submitted (1):**
- **Tool Use ID:** `toolu_01JwNsDV2HcPnDE2eFJdG7zF`
```json
{
"thoughtNumber": 3,
"totalThoughts": 8,
"nextThoughtNeeded": true,
"branches": [],
"thoughtHistoryLength": 3
}
```
### 🤖 Assistant
**Timestamp:** 2025-11-13T17:10:48.954Z
**UUID:** 11c28930-0797-4f9e-ada8-5eaa27c70b47
**Model:** claude-sonnet-4-5-20250929
**Stop Reason:** tool_use
**💭 Thinking:**
> Now let me think about the architectural scaffolding aspect.
**📊 Token Usage:**
- Input: 10
- Output: 260
- Cache creation: 329
- Cache read: 29888
---
## Turn 5
### 👤 User
**Timestamp:** 2025-11-13T17:10:53.302Z
**UUID:** 6f013ecd-a0f0-4bb9-8eca-e6e8d3f80a53
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
**CWD:** /Users/alexnewman/Scripts/claude-mem
**Tool Results Submitted (1):**
- **Tool Use ID:** `toolu_016nFXyGw4SsdKfhUb9eBtkv`
```json
{
"thoughtNumber": 4,
"totalThoughts": 8,
"nextThoughtNeeded": true,
"branches": [],
"thoughtHistoryLength": 4
}
```
### 🤖 Assistant
**Timestamp:** 2025-11-13T17:10:56.583Z
**UUID:** c6d275b3-7f28-4bdd-8a81-80f1a73ffbc7
**Model:** claude-sonnet-4-5-20250929
**Stop Reason:** tool_use
**💭 Thinking:**
> Now let me think about why this works better than active searching.
**📊 Token Usage:**
- Input: 9
- Output: 258
- Cache creation: 327
- Cache read: 30217
---
## Turn 6
### 👤 User
**Timestamp:** 2025-11-13T17:11:00.306Z
**UUID:** 885fbc9f-e767-4a16-a8aa-b8bb54849d67
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
**CWD:** /Users/alexnewman/Scripts/claude-mem
**Tool Results Submitted (1):**
- **Tool Use ID:** `toolu_013QydhPuiBKVRHh1vbD8eL8`
```json
{
"thoughtNumber": 5,
"totalThoughts": 8,
"nextThoughtNeeded": true,
"branches": [],
"thoughtHistoryLength": 5
}
```
### 🤖 Assistant
**Timestamp:** 2025-11-13T17:11:04.430Z
**UUID:** 780354e4-abbf-4d97-b1e2-4f3abc5aef8f
**Model:** claude-sonnet-4-5-20250929
**Stop Reason:** tool_use
**💭 Thinking:**
> Now let me think about the compression-decompression cycle - this is a key insight.
**📊 Token Usage:**
- Input: 8
- Output: 264
- Cache creation: 325
- Cache read: 30544
---
## Turn 7
### 👤 User
**Timestamp:** 2025-11-13T17:11:07.748Z
**UUID:** 462efec2-17c3-47f7-9171-622e00f6bbf7
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
**CWD:** /Users/alexnewman/Scripts/claude-mem
**Tool Results Submitted (1):**
- **Tool Use ID:** `toolu_01Wa5zzkzrYqf5Cfb5UZi5H6`
```json
{
"thoughtNumber": 6,
"totalThoughts": 8,
"nextThoughtNeeded": true,
"branches": [],
"thoughtHistoryLength": 6
}
```
### 🤖 Assistant
**Timestamp:** 2025-11-13T17:11:12.931Z
**UUID:** 325a8179-8291-4198-a661-eae8f02d57b2
**Model:** claude-sonnet-4-5-20250929
**Stop Reason:** tool_use
**💭 Thinking:**
> The user wants me to continue with the sequential thinking about why claude-mem's session start context works so well. I should continue the thinking process and not mention the todo reminder since this is a theoretical discussion, not a task execution.
**📊 Token Usage:**
- Input: 10
- Output: 328
- Cache creation: 1883
- Cache read: 28036
---
## Turn 8
### 👤 User
**Timestamp:** 2025-11-13T17:11:17.712Z
**UUID:** 189d903f-a901-465f-bd6b-8addf44cd5f5
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
**CWD:** /Users/alexnewman/Scripts/claude-mem
**Tool Results Submitted (1):**
- **Tool Use ID:** `toolu_015VKanT9FxnCSsZiTduHd2y`
```json
{
"thoughtNumber": 7,
"totalThoughts": 9,
"nextThoughtNeeded": true,
"branches": [],
"thoughtHistoryLength": 7
}
```
### 🤖 Assistant
**Timestamp:** 2025-11-13T17:11:21.820Z
**UUID:** 96ca3f03-f889-454d-a13a-96e7a36a3daa
**Model:** claude-sonnet-4-5-20250929
**Stop Reason:** tool_use
**💭 Thinking:**
> I want to add one more thought about the temporal/chronological aspect and how that creates a narrative structure that aids understanding.
**📊 Token Usage:**
- Input: 12
- Output: 290
- Cache creation: 392
- Cache read: 29919
---
## Turn 9
### 👤 User
**Timestamp:** 2025-11-13T17:11:26.591Z
**UUID:** cfd56a68-8628-4811-9df5-75a8e557bec6
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
**CWD:** /Users/alexnewman/Scripts/claude-mem
**Tool Results Submitted (1):**
- **Tool Use ID:** `toolu_01EWtaz38nageXXEjYzzm2hu`
```json
{
"thoughtNumber": 8,
"totalThoughts": 9,
"nextThoughtNeeded": true,
"branches": [],
"thoughtHistoryLength": 8
}
```
### 🤖 Assistant
**Timestamp:** 2025-11-13T17:11:30.749Z
**UUID:** bb106bb5-9696-4538-b810-d9fa56bcffff
**Model:** claude-sonnet-4-5-20250929
**Stop Reason:** tool_use
**💭 Thinking:**
> Now for the synthesis - bringing all these threads together to answer the user's question about WHY it works so well from my own perspective.
**📊 Token Usage:**
- Input: 11
- Output: 360
- Cache creation: 357
- Cache read: 30311
---
## Turn 10
### 👤 User
**Timestamp:** 2025-11-13T17:11:36.692Z
**UUID:** 0aa4f55d-9f5a-485f-9aef-dd90f9e80391
**Session ID:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d
**CWD:** /Users/alexnewman/Scripts/claude-mem
**Tool Results Submitted (1):**
- **Tool Use ID:** `toolu_01R5Sy5zet7JCPmXhvDrkMAR`
```json
{
"thoughtNumber": 9,
"totalThoughts": 9,
"nextThoughtNeeded": false,
"branches": [],
"thoughtHistoryLength": 9
}
```
### 🤖 Assistant
**Timestamp:** 2025-11-13T17:11:43.435Z
**UUID:** 2a5c5c37-9124-441f-ac1f-3f9bff08281c
**Model:** claude-sonnet-4-5-20250929
**Stop Reason:** end_turn
**💭 Thinking:**
> This is a fascinating philosophical question about how I process information. The user is asking me to think deeply about WHY claude-mem's session start context works so well from MY perspective as an LLM. They've noticed something important: I don't seem to need to actively search when I have good summaries, and they're hypothesizing it's because the summaries trigger pattern recognition rather than requiring literal recall.
>
> This is not a task-oriented request - it's a meta-cognitive question about how I think. I should not use TodoWrite here as this is a philosophical discussion, not a task to track.
>
> The sequential thinking tool has helped me organize my thoughts. Let me now synthesize this into a clear response about my actual experience.
**📊 Token Usage:**
- Input: 10
- Output: 936
- Cache creation: 1140
- Cache read: 29919
---
*... 87 more turns not shown*
+632
View File
@@ -0,0 +1,632 @@
# Transcript Data Analysis: Available Context for Memory Worker
**Generated:** 2025-11-13
**Purpose:** Document what contextual data exists in Claude Code transcripts and identify opportunities to improve memory worker observation generation.
---
## Executive Summary
**Current State:** The memory worker receives isolated tool executions via `save-hook.ts`:
- Tool name
- Tool input (parameters)
- Tool output (results)
**Available in Transcripts:** Rich contextual data that could dramatically improve observation quality:
- User's original request/intent
- Assistant's reasoning (thinking blocks)
- Full conversation context
- Tool result data
- Token usage and performance metrics
- Session metadata (timestamps, UUIDs, CWD)
**Recommendation:** Enhance the memory worker to receive full conversation context for each tool execution, not just isolated tool data.
---
## Transcript Structure
### Entry Types
The transcript file (`~/.claude/projects/-{project}/session-id.jsonl`) contains:
```
- summary entries (149 in sample)
- file-history-snapshot entries (18 in sample)
- user entries (86 in sample)
- assistant entries (155 in sample)
```
### Conversation Turn Pattern
Each conversation turn consists of:
1. **User Entry** - User's request
2. **Assistant Entry** - Assistant's response
3. **User Entry** - Tool results submitted back (automatic)
4. **Assistant Entry** - Assistant processes results and continues
This creates a pattern: User → Assistant → User (tool results) → Assistant (continues) → ...
---
## Available Data by Entry Type
### 1. User Entries
**Current Save-Hook Access:**
- Tool name
- Tool input
- Tool output
**Additional Data Available in User Entries:**
```typescript
interface UserTranscriptEntry {
type: 'user';
timestamp: string; // ISO timestamp
uuid: string; // Unique entry ID
sessionId: string; // Session identifier
cwd: string; // Working directory
parentUuid?: string; // Parent entry reference
isSidechain: boolean; // Is this a side conversation?
userType: string; // 'human' or 'system'
version: string; // Claude Code version
message: {
role: 'user';
content: string | ContentItem[]; // Can be text or structured
};
toolUseResult?: ToolUseResult; // Legacy field, may contain results
}
```
**When `content` is an array, it contains:**
- Text blocks with user's actual request
- Tool result blocks with complete output data
**Example Structure:**
```json
{
"type": "user",
"timestamp": "2025-11-13T17:10:31.963Z",
"uuid": "364676a7-51c3-4036-afc3-7ff8f7301a8f",
"sessionId": "57dcc12f-4751-46bb-82b4-2aa96a3e226d",
"cwd": "/Users/alexnewman/Scripts/claude-mem",
"message": {
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": "toolu_01T477WUra1sDR6gHaqZHhKT",
"content": "[actual tool output data]"
}
]
}
}
```
### 2. Assistant Entries
**Current Save-Hook Access:**
- Nothing from assistant entries (they happen after tool execution)
**Available Data in Assistant Entries:**
```typescript
interface AssistantTranscriptEntry {
type: 'assistant';
timestamp: string;
uuid: string;
sessionId: string;
cwd: string;
parentUuid?: string;
isSidechain: boolean;
userType: string;
version: string;
requestId?: string; // API request ID
message: {
id: string;
type: 'message';
role: 'assistant';
model: string; // e.g., "claude-sonnet-4-5-20250929"
content: ContentItem[]; // Array of content blocks
stop_reason?: string; // 'tool_use' | 'end_turn' | etc.
stop_sequence?: string;
usage?: UsageInfo; // Token usage stats
};
}
```
**Content Block Types in `message.content`:**
1. **Thinking Blocks** - Internal reasoning before acting
```typescript
{
type: 'thinking';
thinking: string; // Full reasoning text
signature?: string;
}
```
2. **Text Blocks** - Assistant's visible response
```typescript
{
type: 'text';
text: string; // Response text
}
```
3. **Tool Use Blocks** - Tool invocations
```typescript
{
type: 'tool_use';
id: string; // Tool use ID
name: string; // Tool name (e.g., 'Read', 'Edit')
input: Record<string, any>; // Complete tool parameters
}
```
**Token Usage Data:**
```typescript
interface UsageInfo {
input_tokens?: number;
output_tokens?: number;
cache_creation_input_tokens?: number;
cache_read_input_tokens?: number;
service_tier?: string;
}
```
### 3. Summary Entries
```typescript
interface SummaryTranscriptEntry {
type: 'summary';
summary: string; // Generated summary text
leafUuid: string; // UUID of summarized entry
cwd?: string;
}
```
These appear frequently (149 in sample) and provide high-level summaries of work done.
---
## Data Flow: Current vs Potential
### Current Flow (Save-Hook Only)
```
User: "Fix the bug in login.ts"
Assistant: [uses Edit tool]
Tool Execution: Edit(file_path: "login.ts", old_string: "...", new_string: "...")
Save-Hook receives:
- toolName: "Edit"
- toolInput: { file_path: "login.ts", old_string: "...", new_string: "..." }
- toolOutput: { success: true }
Memory Worker generates observation from ONLY tool data
- No user intent
- No assistant reasoning
- No context about WHY this change was made
```
### Enhanced Flow (With Transcript Context)
```
User: "Fix the authentication bug - users getting logged out randomly"
Assistant (thinking): "This sounds like a token expiration issue.
Let me check the JWT handling in login.ts..."
Assistant (uses Edit tool)
Save-Hook receives:
- toolName: "Edit"
- toolInput: { file_path: "login.ts", ... }
- toolOutput: { success: true }
- PLUS:
- userRequest: "Fix the authentication bug - users getting logged out randomly"
- assistantReasoning: "This sounds like a token expiration issue..."
- conversationContext: Previous 2-3 turns
- sessionMetadata: { cwd, timestamp, sessionId }
Memory Worker generates richer observation:
- "Fixed authentication bug causing random logouts"
- "Problem: JWT tokens expiring too quickly"
- "Solution: Updated token expiration to 24h in login.ts"
- "Files: src/auth/login.ts"
- "Concepts: authentication, token-management, bugfix"
```
---
## Specific Opportunities
### 1. User Intent Extraction
**Problem:** Current observations lack user intent.
**Solution:** Parse the most recent user text entry before the tool execution.
**Implementation:**
- Walk backward from tool execution entry
- Find first user entry with text content
- Extract text blocks (filter out tool_result blocks)
**Example:**
```typescript
// In save-hook.ts
const userEntries = parser.getUserEntries();
const recentUserMessage = findUserMessageBeforeTool(userEntries, toolExecutionTimestamp);
const userIntent = extractTextFromContent(recentUserMessage.content);
```
### 2. Assistant Reasoning
**Problem:** We don't capture WHY the assistant chose to use a tool.
**Solution:** Extract thinking blocks from assistant entry immediately before tool use.
**Implementation:**
- Find assistant entry that contains the tool_use block
- Extract thinking blocks from same entry
- Include first ~500 chars of thinking in observation context
**Example:**
```typescript
const assistantEntry = findAssistantEntryWithToolUse(toolUseId);
const thinkingBlocks = assistantEntry.message.content.filter(c => c.type === 'thinking');
const reasoning = thinkingBlocks.map(b => b.thinking).join('\n');
```
### 3. Tool Results Context
**Problem:** Tool output alone doesn't show what was found or changed.
**Solution:** Access full tool result content from next user entry.
**Implementation:**
- Tool execution happens in assistant entry
- Results come back in next user entry as tool_result content
- Save-hook can access both
**Current Structure:**
```
Assistant Entry:
{ type: 'tool_use', id: 'toolu_123', name: 'Read', input: {...} }
User Entry (automatic):
{ type: 'tool_result', tool_use_id: 'toolu_123', content: "file contents..." }
```
**Opportunity:** Match tool_use_id to tool_result and include full result content.
### 4. Conversation Context
**Problem:** Isolated tool executions miss the larger conversation flow.
**Solution:** Include last N conversation turns (2-3 turns is usually sufficient).
**Implementation:**
- Get entries from transcript within time window (e.g., last 5 minutes)
- Include user messages and assistant text responses
- Exclude thinking blocks to save tokens
**Example Context:**
```
Turn 1:
User: "I need to add dark mode support"
Assistant: "I'll help you add dark mode. Let me start by..."
Turn 2:
User: [tool results]
Assistant: "Now I'll update the theme configuration..."
Turn 3: [current tool execution]
```
### 5. Session Metadata
**Problem:** Observations lack temporal and project context.
**Solution:** Include session metadata in observation generation.
**Available Fields:**
- `cwd` - Working directory (project path)
- `timestamp` - Exact time of execution
- `sessionId` - Session identifier
- `uuid` - Entry identifier
- `version` - Claude Code version
**Use Case:** Helps with project-specific context and temporal queries.
### 6. Token Usage Metrics
**Problem:** No visibility into performance and cost.
**Solution:** Track token usage per observation.
**Available Data:**
- Input tokens
- Output tokens
- Cache creation tokens
- Cache read tokens
**Use Case:**
- Performance monitoring
- Cost attribution
- Cache effectiveness analysis
---
## Recommended Implementation Strategy
### Phase 1: User Intent (High Impact, Low Effort)
**Change:** Modify save-hook to extract user's most recent message.
**Implementation:**
```typescript
// In save-hook.ts
import { TranscriptParser } from '../utils/transcript-parser';
const parser = new TranscriptParser(transcriptPath);
const userIntent = parser.getLastUserMessage();
// Send to worker
await workerService.saveToolExecution({
...existingData,
userIntent, // NEW
});
```
**Impact:** Observations now include "what the user wanted to do".
### Phase 2: Assistant Reasoning (High Impact, Medium Effort)
**Change:** Extract thinking blocks from assistant entry containing tool use.
**Implementation:**
```typescript
const assistantEntries = parser.getAssistantEntries();
const toolUseEntry = findEntryWithToolUse(assistantEntries, toolUseId);
const thinking = extractThinkingBlocks(toolUseEntry);
await workerService.saveToolExecution({
...existingData,
userIntent,
assistantReasoning: thinking, // NEW
});
```
**Impact:** Observations include "why the assistant chose this approach".
### Phase 3: Conversation Context (Medium Impact, High Effort)
**Change:** Include last 2-3 conversation turns.
**Implementation:**
```typescript
const recentTurns = getRecentConversationTurns(parser, 3);
await workerService.saveToolExecution({
...existingData,
userIntent,
assistantReasoning: thinking,
conversationContext: recentTurns, // NEW
});
```
**Impact:** Observations understand multi-turn workflows.
### Phase 4: Enhanced Metadata (Low Impact, Low Effort)
**Change:** Include session and performance metadata.
**Implementation:**
```typescript
await workerService.saveToolExecution({
...existingData,
userIntent,
assistantReasoning: thinking,
conversationContext: recentTurns,
metadata: { // NEW
cwd: entry.cwd,
timestamp: entry.timestamp,
sessionId: entry.sessionId,
tokenUsage: entry.message.usage,
},
});
```
**Impact:** Better analytics and debugging.
---
## Example: Before and After
### Current Observation (Tool Data Only)
```json
{
"type": "feature",
"title": "Updated login.ts",
"narrative": "Modified authentication logic in src/auth/login.ts",
"files": ["src/auth/login.ts"],
"concepts": ["authentication"],
"facts": []
}
```
### Enhanced Observation (With Transcript Context)
```json
{
"type": "bugfix",
"title": "Fixed authentication bug causing random logouts",
"narrative": "Users were experiencing random logouts due to JWT token expiration. Updated token expiration from 1h to 24h in token validation logic. Modified src/auth/login.ts to use longer-lived tokens and improved error handling for expired tokens.",
"files": ["src/auth/login.ts"],
"concepts": ["authentication", "jwt", "token-management", "bugfix"],
"facts": [
"JWT token expiration was too short (1h)",
"Updated expiration to 24h",
"Added error handling for expired tokens"
]
}
```
**Improvement:**
- Clear problem statement
- Explicit solution
- Specific technical details
- Better concept tagging
- Actionable facts
---
## Technical Considerations
### 1. Performance
**Concern:** Parsing entire transcript on every tool execution.
**Solution:**
- TranscriptParser already loads full file (unavoidable)
- Use caching for transcript parsing within same session
- Only parse once per session, reuse parsed entries
**Benchmark:**
- Current: ~10ms to parse 408-line transcript
- Impact: Negligible (save-hook already reads transcript)
### 2. Token Usage
**Concern:** Sending more context to worker increases tokens.
**Solution:**
- Thinking blocks: Limit to first 500 chars
- Conversation context: Only last 2-3 turns
- Tool results: Truncate large outputs to 500 chars
- User intent: Full text (usually short)
**Estimate:**
- Current: ~200 tokens per observation generation
- Enhanced: ~500 tokens per observation generation
- Increase: ~150%
- Cost: Still < $0.001 per observation with Haiku
### 3. Implementation Complexity
**Concern:** Matching tool executions to transcript entries.
**Solution:**
- Tool use IDs are in both places
- Timestamps provide ordering
- UUID chains provide parent-child relationships
**Example Matching:**
```typescript
function findToolContext(parser: TranscriptParser, toolUseId: string) {
// 1. Find assistant entry with tool_use block
const assistantEntry = parser.getAssistantEntries()
.find(entry =>
entry.message.content.some(c =>
c.type === 'tool_use' && c.id === toolUseId
)
);
// 2. Find next user entry with tool_result
const userEntry = parser.getUserEntries()
.find(entry =>
entry.message.content.some(c =>
c.type === 'tool_result' && c.tool_use_id === toolUseId
)
);
return { assistantEntry, userEntry };
}
```
---
## Next Steps
1. **Validate Approach**
- Review this analysis with project team
- Confirm data availability in all transcript scenarios
- Identify any privacy concerns
2. **Implement Phase 1**
- Update save-hook.ts to extract user intent
- Modify worker service to accept new fields
- Update observation prompt to use user intent
3. **Test and Measure**
- Compare observation quality before/after
- Measure token usage increase
- Validate performance impact
4. **Iterate**
- Roll out Phase 2 (assistant reasoning)
- Roll out Phase 3 (conversation context)
- Monitor improvements at each phase
---
## Appendix: Data Samples
### Complete Markdown Representation
See `/Users/alexnewman/Scripts/claude-mem/docs/context/transcript-complete-readable.md` for a full 1:1 markdown representation of the first 10 conversation turns from the sample transcript, including:
- Complete user messages
- Full assistant responses
- Thinking blocks (truncated to 2000 chars)
- Tool uses with complete input JSON
- Tool results with actual output data (truncated to 500 chars)
- Token usage stats
- All metadata (timestamps, UUIDs, session IDs, CWD)
### Sample Tool Result Structure
```typescript
// User entry containing tool result
{
"type": "user",
"message": {
"content": [
{
"type": "tool_result",
"tool_use_id": "toolu_01T477WUra1sDR6gHaqZHhKT",
"content": [
{
"type": "text",
"text": "{\n \"thoughtNumber\": 1,\n \"totalThoughts\": 8,\n \"nextThoughtNeeded\": true,\n \"branches\": [],\n \"thoughtHistoryLength\": 1\n}"
}
]
}
]
}
}
```
---
## Conclusion
The Claude Code transcript files contain a wealth of contextual data that is currently unused by the memory worker. By extracting:
1. User intent (the "what" and "why")
2. Assistant reasoning (the "how" and "because")
3. Tool results (the "outcome")
4. Conversation context (the "flow")
5. Session metadata (the "when" and "where")
We can generate significantly richer, more useful observations that better capture the intent, decisions, and outcomes of each coding session.
**The data is already there - we just need to read it.**
+234
View File
@@ -0,0 +1,234 @@
# Claude Code Transcript Data Discovery
## Executive Summary
This document details findings from implementing a validated transcript parser for Claude Code JSONL transcripts. The parser enables extraction of rich contextual data that can optimize prompt generation and track token usage for ROI metrics.
## Transcript Structure
### File Location
```
~/.claude/projects/<encoded-project-path>/<session-id>.jsonl
```
Example:
```
~/.claude/projects/-Users-alexnewman-Scripts-claude-mem/2933cff9-f0a7-4f0b-8296-0a030e7658a6.jsonl
```
### Entry Types
Discovered 5 transcript entry types:
1. **`file-history-snapshot`** (NEW - not in Python model)
- Purpose: Track file state snapshots
- Frequency: ~10 entries per session
2. **`user`** - User messages and tool results
- Contains actual user text messages OR tool result data
- Can have string content or array of ContentItems
3. **`assistant`** - Assistant responses and tool uses
- Contains text responses, tool uses, and thinking blocks
- **Critical**: Contains usage data with token counts
4. **`summary`** (not yet observed in test data)
- Session summaries
5. **`system`** (not yet observed in test data)
- System messages/warnings
6. **`queue-operation`** (not yet observed in test data)
- Queue tracking for message flow
## Key Findings
### 1. Message Extraction Complexity
**Problem**: Naively getting the "last" entry doesn't work because:
- Last user entry might be a tool result, not a text message
- Last assistant entry might only contain tool uses, no text
**Solution**: Iterate backward through entries to find the last entry with actual text content.
### 2. Tool Use Tracking
**Discovery**: Tool uses are in **assistant** messages, not user messages.
**Data Available**:
```typescript
{
name: string; // Tool name (e.g., "Bash", "Read", "TodoWrite")
timestamp: string; // When the tool was used
input: any; // Full tool input parameters
}
```
**Test Session Results** (168 entries):
- 42 tool uses across 7 different tool types
- Most used: Bash (24x), TodoWrite (5x), Edit (4x)
### 3. Token Usage Data (ROI Foundation)
**Critical Discovery**: Every assistant message contains complete token usage data:
```typescript
interface UsageInfo {
input_tokens?: number; // Total input tokens (includes context)
cache_creation_input_tokens?: number; // Tokens used to create cache
cache_read_input_tokens?: number; // Cached tokens read (discounted cost)
output_tokens?: number; // Model output tokens
}
```
**Test Session Token Analysis**:
```
Input tokens: 858
Output tokens: 44,165
Cache creation tokens: 469,650
Cache read tokens: 5,294,101 ← 5.29M tokens saved by caching!
Total tokens: 45,023
```
**ROI Implication**: This validates our ROI implementation plan. We can track:
- Discovery cost = sum of all input + output tokens across session
- Context savings = cache_read_input_tokens (tokens NOT paid for in full)
- ROI = Discovery cost / Context savings
### 4. Parse Reliability
**Result**: 0.00% parse failure rate on production transcript with 168 entries.
**Conclusion**: The JSONL format is stable and well-formed. No need for extensive error handling.
## Implementation Files
### Created Files
1. **`src/types/transcript.ts`** - TypeScript types matching Python Pydantic model
- All entry types, content types, usage info
- Drop-in compatible with Python model structure
2. **`src/utils/transcript-parser.ts`** - Robust transcript parsing class
- Handles all entry types
- Smart message extraction (finds last text message, not just last entry)
- Tool use history extraction
- Token usage aggregation
- Parse statistics and error tracking
3. **`scripts/test-transcript-parser.ts`** - Validation script
- Tests all extraction methods
- Reports parse statistics
- Shows token usage breakdown
- Lists tool use history
### Usage Example
```typescript
import { TranscriptParser } from '../src/utils/transcript-parser.js';
const parser = new TranscriptParser('/path/to/transcript.jsonl');
// Extract messages
const lastUserMsg = parser.getLastUserMessage();
const lastAssistantMsg = parser.getLastAssistantMessage();
// Get tool history
const tools = parser.getToolUseHistory();
// => [{name: 'Bash', timestamp: '...', input: {...}}, ...]
// Get token usage
const tokens = parser.getTotalTokenUsage();
// => {inputTokens: 858, outputTokens: 44165, cacheReadTokens: 5294101, ...}
// Parse statistics
const stats = parser.getParseStats();
// => {totalLines: 168, parsedEntries: 168, failedLines: 0, ...}
```
## Next Steps for PR Review
### Addressing "Drops Unknown Lines" Concern
**Original Issue**: Summary hook silently skipped malformed lines without visibility.
**Root Cause**: We didn't understand the full transcript model. The "skip malformed lines" was a band-aid.
**Solution**: Replace ad-hoc parsing in `summary-hook.ts` with validated `TranscriptParser` class:
**Before** (summary-hook.ts:38-117):
```typescript
// Manually parsing with try/catch, no type safety
for (let i = lines.length - 1; i >= 0; i--) {
try {
const line = JSON.parse(lines[i]);
if (line.type === 'user' && line.message?.content) {
// ... extraction logic
}
} catch (parseError) {
// Skip malformed lines ← BLACK HOLE
continue;
}
}
```
**After** (using TranscriptParser):
```typescript
import { TranscriptParser } from '../utils/transcript-parser.js';
const parser = new TranscriptParser(transcriptPath);
const lastUserMessage = parser.getLastUserMessage();
const lastAssistantMessage = parser.getLastAssistantMessage();
// Parse errors are tracked in parser.getParseErrors()
```
**Benefits**:
1. ✅ Type-safe extraction based on validated model
2. ✅ No silent failures - parse errors are tracked
3. ✅ Smart extraction (finds last TEXT message, not last entry)
4. ✅ Reusable across all hooks and scripts
5. ✅ Enables token usage tracking (ROI metrics)
6. ✅ Enables tool use tracking (prompt optimization)
## Prompt Optimization Opportunities
With rich transcript data available, we can enhance prompts with:
### 1. Tool Use Patterns
- "In this session you've used: Bash (24x), TodoWrite (5x), Edit (4x)"
- Helps Claude understand what kind of work is being done
### 2. Token Economics Awareness
- "Cache read tokens: 5.29M (context savings)"
- Reinforces value of memory system
### 3. Session Flow Understanding
- Number of user/assistant exchanges
- Tools used per exchange
- Session complexity metrics
### 4. File History Snapshots
- Track which files were modified during session
- Provide file change context to summaries
## Testing
Run the validation script:
```bash
# Find your current session transcript
ls -lt ~/.claude/projects/-Users-alexnewman-Scripts-claude-mem/*.jsonl | head -1
# Test the parser
npx tsx scripts/test-transcript-parser.ts <path-to-transcript.jsonl>
```
## Conclusion
The transcript parser implementation:
1. ✅ Addresses PR review concern about dropped lines
2. ✅ Validates the ROI metrics implementation plan
3. ✅ Enables prompt optimization with rich context
4. ✅ Provides foundation for future enhancements
**Recommendation**: Replace ad-hoc transcript parsing in hooks with `TranscriptParser` class for improved reliability and feature richness.
+22
View File
@@ -0,0 +1,22 @@
# Transcript Dump
Total entries: 384
---
## Entry 151: USER
**Timestamp:** 2025-11-13T17:09:47.231Z
**Content:**
```
I have a question regarding the context timeline that we made. and it's a bit of a thinker...
when i first started out working on claude-mem my idea was using a fast index of memories with links to access them on demand. I thought that claude-mem would be searching for all the things it needed to get the job done, but what I found is that the description of the work done + the files touched actually improved performance tremendously, and without actively searching for things constantly.
I just had a thought that maybe claude isn't searching, because when it reads what it did, it already "knows" how it did that, for the most part. Would you say I'm on the right track? I want you to ultrathink about WHY claude-mem's session start context works so well, from your OWN perspective
```
---
_Remaining 364 entries omitted for brevity_
@@ -0,0 +1,106 @@
# Transcript Context Analysis
**File:** 57dcc12f-4751-46bb-82b4-2aa96a3e226d.jsonl
**Parsed:** 11/13/2025, 12:34:19 PM
## Statistics
- Total entries: 281
- Successfully parsed: 281
- Failed lines: 0
- Conversation turns: 43
## Token Usage
- Input tokens: 6,805
- Output tokens: 38,209
- Cache creation: 224,078
- Cache read: 3,320,726
- Total: 45,014
---
# Conversation Turns
## Turn 1
---
## Turn 2
---
## Turn 3
---
## Turn 4
---
## Turn 5
---
## Turn 6
---
## Turn 7
---
## Turn 8
---
## Turn 9
---
## Turn 10
---
## Turn 11
---
## Turn 12
---
## Turn 13
---
## Turn 14
---
## Turn 15
---
## Turn 16
---
## Turn 17
---
## Turn 18
---
## Turn 19
---
## Turn 20
---
_... 23 more turns omitted for brevity_
+67
View File
@@ -0,0 +1,67 @@
# VS Code API Language Model (`lm`) Namespace
Excerpt captured from `https://code.visualstudio.com/api/references/vscode-api#lm` using markitdown MCP on 2025-11-12.
## Overview
The `vscode.lm` namespace exposes APIs for interacting with language models inside Visual Studio Code. It allows extensions to register tools, select chat models, invoke tools, and surface MCP servers so that agent mode can compose complex responses.
### Available Tools
- `vscode.lm.tools`: Readonly array of [`LanguageModelToolInformation`](https://code.visualstudio.com/api/references/vscode-api#LanguageModelToolInformation).
- Lists all tools registered via `vscode.lm.registerTool`.
- Tools can be invoked programmatically with `vscode.lm.invokeTool` when their inputs satisfy the declared schema.
### Events
- `vscode.lm.onDidChangeChatModels`: Fires when the set of available chat models changes. Extensions should re-query models after this event.
### Functions
#### `vscode.lm.invokeTool(name, options, token?)`
Invokes a tool by name with a given input payload.
- Validates input against the schema declared by the tool.
- When called from a chat participant, pass the `toolInvocationToken` so the chat UI associates results with the correct conversation.
- Returns a [`LanguageModelToolResult`](https://code.visualstudio.com/api/references/vscode-api#LanguageModelToolResult) composed of text and optional prompt-tsx parts.
- Tool results can be preserved across turns by storing them in `ChatResult.metadata` and retrieving them later from `ChatResponseTurn.result`.
#### `vscode.lm.registerLanguageModelChatProvider(vendor, provider)`
Registers a [`LanguageModelChatProvider`](https://code.visualstudio.com/api/references/vscode-api#LanguageModelChatProvider).
- Requires a matching `languageModelChatProviders` contribution in `package.json`.
- `vendor` must be globally unique (for example `copilot` or `openai`).
- Returns a `Disposable` to unregister the provider.
#### `vscode.lm.registerMcpServerDefinitionProvider(id, provider)`
Publishes Model Context Protocol servers for the editor.
- Requires a `contributes.mcpServerDefinitionProviders` entry in `package.json`.
- Enables dynamic discovery of MCP servers and tools when users submit chat messages.
- Returns a `Disposable` that unregisters the provider.
#### `vscode.lm.registerTool(name, tool)`
Registers a [`LanguageModelTool`](https://code.visualstudio.com/api/references/vscode-api#LanguageModelTool) implementation with the runtime.
- Tool must also appear in `package.json -> contributes.languageModelTools`.
- Registered tools appear in `vscode.lm.tools` and can be used by any extension.
#### `vscode.lm.selectChatModels(selector?)`
Returns an array of [`LanguageModelChat`](https://code.visualstudio.com/api/references/vscode-api#LanguageModelChat) instances matching a selector.
- Selector can be broad (by vendor or family) or narrow (by exact model ID).
- Handle scenarios where zero models are available.
- Persisted model references should be refreshed when `onDidChangeChatModels` fires.
## Usage Notes
- Extensions should gracefully handle missing models or tools.
- Tool invocation responses can include multiple parts; integrate them using prompt-tsx or by constructing `LanguageModelToolResultPart` objects.
- When providing MCP servers or tools, ensure proper contribution points exist in `package.json`.
_Fetched on 2025-11-12 via markitdown MCP._
@@ -0,0 +1,49 @@
# VS Code Copilot Extension Integration Plan
## 1. Groundwork
- Audit existing claude-mem hook scripts (`context-hook`, `user-message-hook`, `new-hook`, `save-hook`, `summary-hook`, `cleanup-hook`) and their worker-service payloads.
- Document REST endpoints, request bodies, and SessionStore schema fields used today so the extension mirrors them exactly.
- Confirm worker service availability workflow (`ensureWorkerRunning`, port resolution) and decide how extension error reporting will surface issues to Copilot chat users.
## 2. Project Scaffold
- Clone the VS Code `chat-sample` starter, convert to a TypeScript-only extension, and align lint/tsconfig with repo standards.
- Add build pipeline (esbuild or webpack) plus npm scripts that match the existing `scripts/build-hooks.js` release flow.
- Wire extension activation events for chat participation and ensure packaging metadata (publisher, categories) is in place.
## 3. Shared Worker Client
- Extract reusable worker-service client utilities from `plugin/scripts/*.js` (port discovery, session init, observation uploads).
- Publish TypeScript definitions by re-exporting from `src/services/worker-types.ts` to keep contracts synchronized.
- Centralize HTTP calls (timeouts, retries, logging) so every tool implementation uses the same helper layer.
## 4. Language Model Tool Contracts
- Add `contributes.languageModelTools` entries in `package.json` for lifecycle parity:
- `mem_session_init`, `mem_user_prompt_log`, `mem_observation_record`, `mem_summary_finalize`, `mem_session_cleanup`.
- Provide detailed JSON schemas mirroring hook input structures (session IDs, cwd, prompt text, tool payload metadata).
- Supply descriptive `modelDescription`, `userDescription`, icons, tags, and enable `canBeReferencedInPrompt` where appropriate.
## 5. Tool Implementations
- Register each tool via `vscode.lm.registerTool` inside `activate`.
- Implement `prepareInvocation` to show user confirmations (especially for cleanup/stop actions) and tailor messages to match existing CLI prompts.
- In `invoke`, call the shared worker client, translate successes into `LanguageModelToolResult` text parts, and craft error messages that guide the LLM toward recovery (retry, alternate parameters).
- Ensure telemetry/logging records tool usage for debugging without leaking sensitive data.
## 6. Chat Orchestration
- Implement a chat participant based on the sample that maps Copilot threads to claude-mem session IDs stored in turn metadata.
- On conversation start, auto-run `mem_session_init`; before each user prompt, dispatch `mem_user_prompt_log`; when Copilot signals stop, run `mem_summary_finalize` (with fallbacks if the worker is unavailable).
- Capture tool events emitted by Copilot (file edits, terminal runs) and forward them through `mem_observation_record` with matching payload structure.
- Handle conversation disposal or model changes by calling `mem_session_cleanup` to mirror `SessionEnd` hooks.
## 7. Settings and UX
- Read `.claude-mem/settings.json` overrides (worker port, observation depth) and surface VS Code settings for Copilot-specific toggles (auto-sync enabled, max observations per prompt).
- Add status bar indicator/commands for worker health, quick restart instructions, and opening the viewer UI (`http://localhost:37777`).
- Provide inline notifications when the worker is unreachable, including guidance to restart via PM2.
## 8. Testing and QA
- Draft manual validation checklist: initial session, prompt logging, observation capture, summary completion, worker-down handling.
- Add integration tests using `@vscode/test-electron` to simulate chat turns and assert database side effects in a temporary claude-mem data directory.
- Build mocks for worker endpoints to enable unit tests of tool invocation logic without hitting the real service.
## 9. Release Readiness
- Document installation and usage in `README.md`, including architecture diagrams showing Copilot → tool → worker flow.
- Update CHANGELOG and marketing copy to announce Copilot support and list prerequisites (worker running, settings file placement).
- Prepare Marketplace assets (icon, gallery text) and extend existing publish scripts to package and ship the new extension.
@@ -0,0 +1,8 @@
# VS Code Extension Chat Sample Repository (GitHub Snapshot)
The markitdown MCP fetch for `https://github.com/microsoft/vscode-extension-samples/tree/main/chat-sample` returned only the public navigation scaffolding for GitHub. No repository-specific content or README data was captured because the site requires client-side execution that the fetcher cannot perform.
You can browse the repository directly for full details:
- https://github.com/microsoft/vscode-extension-samples/tree/main/chat-sample
_Fetched on 2025-11-12 via markitdown MCP. Content retrieval was limited to GitHub's static navigation shell._
@@ -0,0 +1,267 @@
# Language Model Tool API
Language model tools enable you to extend the functionality of a large language model (LLM) in chat with domain-specific capabilities. To process a user's chat prompt, [agent mode](/docs/copilot/chat/chat-agent-mode) in VS Code can automatically invoke these tools to perform specialized tasks as part of the conversation.
By contributing a language model tool in your VS Code extension, you can extend the agentic coding workflow while also providing deep integration with the editor. Extension tools are one of three types of tools available in VS Code, alongside [built-in tools and MCP tools](/docs/copilot/chat/chat-tools.md#types-of-tools).
In this extension guide, you learn how to create a language model tool by using the Language Model Tools API and how to implement tool calling in a chat extension.
You can also extend the chat experience with specialized tools by contributing an [MCP server](/api/extension-guides/ai/mcp). See the [AI Extensibility Overview](/api/extension-guides/ai/ai-extensibility-overview) for details on the different options and how to decide which approach to use.
> **Tip**
> For information about using tools as an end user, see [Use tools in chat](/docs/copilot/chat/chat-tools.md).
## What is tool calling in an LLM?
A language model tool is a function that can be invoked as part of a language model request. For example, you might have a function that retrieves information from a database, performs some calculation, or calls an online API. When you contribute a tool in a VS Code extension, agent mode can then invoke the tool based on the context of the conversation.
The LLM never actually executes the tool itself, instead the LLM generates the parameters that are used to call your tool. It's important to clearly describe the tool's purpose, functionality, and input parameters so that the tool can be invoked in the right context.
The following diagram shows the tool-calling flow in agent mode in VS Code. See [Tool-calling flow](#tool-calling-flow) for details about the specific steps involved.
![Diagram that shows the Copilot tool-calling flow](/assets/api/extension-guides/ai/tools/copilot-tool-calling-flow.png)
Read more about [function calling](https://platform.openai.com/docs/guides/function-calling) in the OpenAI documentation.
## Why implement a language model tool in your extension?
Implementing a language model tool in your extension has several benefits:
- **Extend agent mode** with specialized, domain-specific tools that are automatically invoked as part of responding to a user prompt. For example, enable database scaffolding and querying to dynamically provide the LLM with relevant context.
- **Deeply integrate with VS Code** by using the broad set of extension APIs. For example, use the [debug APIs](/api/extension-guides/debugger-extension) to get the current debugging context and use it as part of the tool's functionality.
- **Distribute and deploy** tools via the Visual Studio Marketplace, providing a reliable and seamless experience for users. Users don't need a separate installation and update process for your tool.
You might consider implementing a language model tool with an [MCP server](/api/extension-guides/ai/mcp) in the following scenarios:
- You already have an MCP server implementation and also want to use it in VS Code.
- You want to reuse the same tool across different development environments and platforms.
- Your tool is hosted remotely as a service.
- You don't need access to VS Code APIs.
Learn more about the [differences between tool types](/docs/copilot/chat/chat-tools.md#types-of-tools).
## Create a language model tool
Implementing a language model tool consists of two main parts:
1. Define the tool's configuration in the `package.json` file of your extension.
2. Implement the tool in your extension code by using the [Language Model API reference](/api/references/vscode-api#lm)
You can get started with a [basic example project](https://github.com/microsoft/vscode-extension-samples/tree/main/chat-sample).
### 1. Static configuration in `package.json`
The first step to define a language model tool in your extension is to define it in the `package.json` file of your extension. This configuration includes the tool name, description, input schema, and other metadata:
1. Add an entry for your tool in the `contributes.languageModelTools` section of your extension's `package.json` file.
2. Give the tool a unique name:
| Property | Description |
| --- | --- |
| `name` | The unique name of the tool, used to reference the tool in the extension implementation code. Format the name in the format `{verb}_{noun}`. See [naming guidelines](#guidelines-and-conventions). |
| `displayName` | The user-friendly name of the tool, used for displaying in the UI. |
3. If the tool can be used in [agent mode](/docs/copilot/chat/chat-agent-mode) or referenced in a chat prompt with `#`, add the following properties:
Users can enable or disable the tool in the Chat view, similar to how this is done for [Model Context Protocol (MCP) tools](/docs/copilot/chat/chat-tools.md#mcp-tools).
| Property | Description |
| --- | --- |
| `canBeReferencedInPrompt` | Set to `true` if the tool can be used in [agent mode](/docs/copilot/chat/chat-agent-mode) or referenced in chat. |
| `toolReferenceName` | The name for users to reference the tool in a chat prompt via `#`. |
| `icon` | The icon to display for the tool in the UI. |
| `userDescription` | User-friendly description of the tool, used for displaying in the UI. |
4. Add a detailed description in `modelDescription`. This information is used by the LLM to determine in which context your tool should be used.
- What exactly does the tool do?
- What kind of information does it return?
- When should and shouldn't it be used?
- Describe important limitations or constraints of the tool.
5. If the tool takes input parameters, add an `inputSchema` property that describes the tool's input parameters.
This JSON schema describes an object with the properties that the tool takes as input, and whether they are required. File paths should be absolute paths.
Describe what each parameter does and how it relates to the tool's functionality.
6. Add a `when` clause to control when the tool is available.
The `languageModelTools` contribution point lets you restrict when a tool is available for agent mode or can be referenced in a prompt by using a [when clause](/api/references/when-clause-contexts). For example, a tool that gets the debug call stack information should only be available when the user is debugging.
```json
"contributes": {
"languageModelTools": [
{
"name": "chat-tools-sample_tabCount",
...
"when": "debugState == 'running'"
}
]
}
```
**Example tool definition**
The following example shows how to define a tool that counts the number of active tabs in a tab group.
```json
"contributes": {
"languageModelTools": [
{
"name": "chat-tools-sample_tabCount",
"tags": [
"editors",
"chat-tools-sample"
],
"toolReferenceName": "tabCount",
"displayName": "Tab Count",
"modelDescription": "The number of active tabs in a tab group in VS Code.",
"userDescription": "Count the number of active tabs in a tab group.",
"canBeReferencedInPrompt": true,
"icon": "$(files)",
"inputSchema": {
"type": "object",
"properties": {
"tabGroup": {
"type": "number",
"description": "The index of the tab group to check. This is optional- if not specified, the active tab group will be checked.",
"default": 0
}
}
}
}
]
}
```
### 2. Tool implementation
Implement the language model tool by using the [Language Model API](/api/references/vscode-api#lm). This consists of the following steps:
1. On activation of the extension, register the tool with [`vscode.lm.registerTool`](/api/references/vscode-api#lm.registerTool).
Provide the name of the tool as you specified it in the `name` property in `package.json`.
If you want the tool to be private to your extension, skip the tool registration step.
```ts
export function registerChatTools(context: vscode.ExtensionContext) {
context.subscriptions.push(
vscode.lm.registerTool('chat-tools-sample_tabCount', new TabCountTool())
);
}
```
2. Create a class that implements the [`vscode.LanguageModelTool<>`](/api/references/vscode-api#LanguageModelTool%3CT%3E) interface.
3. Add tool confirmation messages in the `prepareInvocation` method.
A generic confirmation dialog will always be shown for tools from extensions, but the tool can customize the confirmation message. Give enough context to the user to understand what the tool is doing. The message can be a `MarkdownString` containing a code block.
The following example shows how to provide a confirmation message for the tab count tool.
```ts
async prepareInvocation(
options: vscode.LanguageModelToolInvocationPrepareOptions<ITabCountParameters>,
_token: vscode.CancellationToken
) {
const confirmationMessages = {
title: 'Count the number of open tabs',
message: new vscode.MarkdownString(
`Count the number of open tabs?` +
(options.input.tabGroup !== undefined
? ` in tab group ${options.input.tabGroup}`
: '')
),
};
return {
invocationMessage: 'Counting the number of tabs',
confirmationMessages,
};
}
```
If `prepareInvocation` returned `undefined`, the generic confirmation message will be shown. Note that the user can also select to "Always Allow" a certain tool.
4. Define an interface that describes the tool input parameters.
The interface is used in the `invoke` method of the `vscode.LanguageModelTool` class. The input parameters are validated against the JSON schema you defined in the `inputSchema` in `package.json`.
The following example shows the interface for the tab count tool.
```ts
export interface ITabCountParameters {
tabGroup?: number;
}
```
5. Implement the `invoke` method. This method is called when the language model tool is invoked while processing a chat prompt.
The `invoke` method receives the tool input parameters in the `options` parameter. The parameters are validated against the JSON schema defined in `inputSchema` in `package.json`.
When an error occurs, throw an error with a message that makes sense to the LLM. Optionally, provide instructions on what the LLM should do next, such as retrying with different parameters, or performing a different action.
The following example shows the implementation of the tab count tool. The result of the tool is an instance of type `vscode.LanguageModelToolResult`.
```ts
async invoke(
options: vscode.LanguageModelToolInvocationOptions<ITabCountParameters>,
_token: vscode.CancellationToken
) {
const params = options.input;
if (typeof params.tabGroup === 'number') {
const group = vscode.window.tabGroups.all[Math.max(params.tabGroup - 1, 0)];
const nth =
params.tabGroup === 1
? '1st'
: params.tabGroup === 2
? '2nd'
: params.tabGroup === 3
? '3rd'
: `${params.tabGroup}th`;
return new vscode.LanguageModelToolResult([new vscode.LanguageModelTextPart(`There are ${group.tabs.length} tabs open in the ${nth} tab group.`)]);
} else {
const group = vscode.window.tabGroups.activeTabGroup;
return new vscode.LanguageModelToolResult([new vscode.LanguageModelTextPart(`There are ${group.tabs.length} tabs open.`)]);
}
}
```
View the full source code for implementing a [language model tool](https://github.com/microsoft/vscode-extension-samples/blob/main/chat-sample/src/tools.ts) in the VS Code Extension Samples repository.
## Tool-calling flow
When a user sends a chat prompt, the following steps occur:
1. Copilot determines the list of available tools based on the user's configuration.
The list of tools consists of built-in tools, tools registered by extensions, and tools from [MCP servers](/docs/copilot/chat/mcp-servers). You can contribute to agent mode via extensions or MCP servers (shown in green in the diagram).
2. Copilot sends the request to the LLM and provides it with the prompt, chat context, and the list of tool definitions to consider.
The LLM generates a response, which might include one or more requests to invoke a tool.
3. If needed, Copilot invokes the suggested tool(s) with the parameter values provided by the LLM.
A tool response might result in more requests for tool invocations.
4. If there are errors or follow-up tool requests, Copilot iterates over the tool-calling flow until all tool requests are resolved.
5. Copilot returns the final response to the user, which might include responses from multiple tools.
## Guidelines and conventions
- **Naming**: write clear and descriptive names for tools and parameters.
- **Tool name**: should be unique, and clearly describe their intent. Structure the tool name in the format `{verb}_{noun}`. For example, `get_weather`, `get_azure_deployment`, or `get_terminal_output`.
- **Parameter name**: should describe the parameter's purpose. Structure the parameter name in the format `{noun}`. For example, `destination_location`, `ticker`, or `file_name`.
- **Descriptions**: write detailed descriptions for tools and parameters.
- Describe what the tool does and when it should and shouldn't be used. For example, "This tool retrieves the weather for a given location."
- Describe what each parameter does and how it relates to the tool's functionality. For example, "The `destination_location` parameter specifies the location for which to retrieve the weather. It should be a valid location name or coordinates."
- Describe important limitations or constraints of the tool. For example, "This tool only retrieves weather data for locations in the United States. It might not work for other regions."
- **User confirmation**: provide a confirmation message for the tool invocation. A generic confirmation dialog will always be shown for tools from extensions, but the tool can customize the confirmation message. Give enough context to the user to understand what the tool is doing.
- **Error handling**: when an error occurs, throw an error with a message that makes sense to the LLM. Optionally, provide instructions on what the LLM should do next, such as retrying with different parameters, or performing a different action.
Get more best practices for creating tools in the [OpenAI documentation](https://platform.openai.com/docs/guides/function-calling?api-mode=chat#best-practices-for-defining-functions) and [Anthropic documentation](https://docs.anthropic.com/en/docs/build-with-claude/tool-use/overview).
## Related content
- [Language Model API reference](/api/references/vscode-api#lm)
- [Register an MCP server in a VS Code extension](/api/extension-guides/ai/mcp)
- [Use MCP tools in agent mode](/docs/copilot/chat/mcp-servers)
_Fetched on 2025-11-12 via markitdown MCP._
@@ -0,0 +1,13 @@
VSCode Language Model Tool API
Local snapshots fetched via the markitdown MCP on 2025-11-12:
- `docs/context/vscode-language-model-tool-api.md`
- `docs/context/vscode-extension-chat-sample.md`
- `docs/context/vscode-api-lm.md`
Original sources for reference:
- http://code.visualstudio.com/api/extension-guides/ai/tools
- https://github.com/microsoft/vscode-extension-samples/tree/main/chat-sample
- https://code.visualstudio.com/api/references/vscode-api#lm
+88
View File
@@ -0,0 +1,88 @@
# Claude-Mem Public Documentation
## What This Folder Is
This `docs/public/` folder contains the **Mintlify documentation site** - the official user-facing documentation for claude-mem. It's a structured documentation platform with a specific file format and organization.
## Folder Structure
```
docs/
├── public/ ← You are here (Mintlify MDX files)
│ ├── *.mdx - User-facing documentation pages
│ ├── docs.json - Mintlify configuration and navigation
│ ├── architecture/ - Technical architecture docs
│ ├── usage/ - User guides and workflows
│ └── *.webp, *.gif - Assets (logos, screenshots)
└── context/ ← Internal documentation (DO NOT put here)
└── *.md - Planning docs, audits, references
```
## File Requirements
### Mintlify Documentation Files (.mdx)
All official documentation files must be:
- Written in `.mdx` format (Markdown with JSX support)
- Listed in `docs.json` navigation structure
- Follow Mintlify's schema and conventions
The documentation is organized into these sections:
- **Get Started**: Introduction, installation, usage guides
- **Best Practices**: Context engineering, progressive disclosure
- **Configuration & Development**: Settings, dev workflow, troubleshooting
- **Architecture**: System design, components, technical details
### Configuration File
`docs.json` defines:
- Site metadata (name, description, theme)
- Navigation structure
- Branding (logos, colors)
- Footer links and social media
## What Does NOT Belong Here
**Planning documents, design docs, and reference materials go in `/docs/context/` instead:**
Files that belong in `/docs/context/` (NOT here):
- Planning documents (`*-plan.md`, `*-outline.md`)
- Implementation analysis (`*-audit.md`, `*-code-reference.md`)
- Error tracking (`typescript-errors.md`)
- Internal design documents
- PR review responses
- Reference materials (like `agent-sdk-ref.md`)
- Work-in-progress documentation
## How to Add Official Documentation
1. Create a new `.mdx` file in the appropriate subdirectory
2. Add the file path to `docs.json` navigation
3. Use Mintlify's frontmatter and components
4. Follow the existing documentation style
5. Test locally: `npx mintlify dev`
## Development Workflow
**For contributors working on claude-mem:**
- Read `/CLAUDE.md` in the project root for development instructions
- Place planning/design docs in `/docs/context/`
- Only add user-facing documentation to `/docs/public/`
- Test documentation locally with Mintlify CLI before committing
## Testing Documentation
```bash
# Validate docs structure
npx mintlify validate
# Check for broken links
npx mintlify broken-links
# Run local dev server
npx mintlify dev
```
## Summary
**Simple Rule**:
- `/docs/public/` = Official user documentation (Mintlify .mdx files) ← YOU ARE HERE
- `/docs/context/` = Internal docs, plans, references, audits
@@ -1,17 +1,17 @@
---
title: "Plugin Hooks"
description: "7 hook scripts that power Claude-Mem"
description: "6 lifecycle hooks that power Claude-Mem"
---
# Plugin Hooks
Claude-Mem integrates with Claude Code through 7 hook scripts across 5 lifecycle events that capture events and inject context.
Claude-Mem integrates with Claude Code through 6 hook scripts across 5 lifecycle events that capture events and inject context. Additionally, a smart-install pre-hook script manages dependencies.
## Hook Overview
| Hook Name | Purpose | Timeout | Script |
| Hook Type | Purpose | Timeout | Script |
|---------------------|--------------------------------------|---------|-------------------------|
| SessionStart | Smart dependency installation | 300s | smart-install.js |
| Pre-Hook | Smart dependency installation | 300s | smart-install.js* |
| SessionStart | Inject context from previous sessions| 300s | context-hook.js |
| SessionStart | Display first-time setup message | 10s | user-message-hook.js |
| UserPromptSubmit | Create/track new sessions | 120s | new-hook.js |
@@ -19,6 +19,8 @@ Claude-Mem integrates with Claude Code through 7 hook scripts across 5 lifecycle
| Stop | Generate session summaries | 120s | summary-hook.js |
| SessionEnd | Mark sessions complete | 120s | cleanup-hook.js |
*smart-install.js is a pre-hook script (not a lifecycle hook). It's called before context-hook via command chaining in hooks.json.
## Hook Configuration
Hooks are configured in `plugin/hooks/hooks.json`:
@@ -72,10 +74,12 @@ Hooks are configured in `plugin/hooks/hooks.json`:
}
```
## 1. SessionStart Hook - Smart Install (`smart-install.js`)
## 1. Pre-Hook Script - Smart Install (`smart-install.js`)
**Purpose**: Intelligently manage dependencies and ensure worker service is running.
**Note**: This is NOT a lifecycle hook - it's a pre-hook script executed via command chaining before context-hook.js runs.
**Behavior**:
- Checks if dependencies need installation using version marker (`.install-version`)
- Only runs npm install when:
@@ -95,7 +99,7 @@ Hooks are configured in `plugin/hooks/hooks.json`:
}
```
**Implementation**: `scripts/smart-install.js`
**Implementation**: `scripts/smart-install.js` (standalone script, not in src/hooks/)
**Key Features**:
- Version caching prevents redundant installs
@@ -9,11 +9,12 @@ description: "System components and data flow in Claude-Mem"
Claude-Mem operates as a Claude Code plugin with five core components:
1. **Plugin Hooks** - Capture lifecycle events (7 hook files)
2. **Worker Service** - Process observations via Claude Agent SDK
3. **Database Layer** - Store sessions and observations (SQLite + FTS5)
4. **MCP Search Server** - Query historical context (9 search tools)
5. **Viewer UI** - Web-based real-time memory stream visualization
1. **Plugin Hooks** - Capture lifecycle events (6 hook files)
2. **Smart Install** - Cached dependency checker (pre-hook script, runs before context-hook)
3. **Worker Service** - Process observations via Claude Agent SDK + HTTP API (10 search endpoints)
4. **Database Layer** - Store sessions and observations (SQLite + FTS5 + ChromaDB)
5. **mem-search Skill** - Skill-based search with progressive disclosure (v5.4.0+)
6. **Viewer UI** - Web-based real-time memory stream visualization
## Technology Stack
@@ -44,23 +45,27 @@ Hook (stdin) → Database → Worker Service → SDK Processor → Database →
4. **Output**: Processed summaries written back to database
5. **Retrieval**: Next session's context hook reads summaries from database
### Search Pipeline
### Search Pipeline (v5.4.0+)
```
Claude Request → MCP Server → SessionSearch Service → FTS5 Database → Search Results → Claude
User Query → mem-search Skill Invoked → HTTP API → SessionSearch Service → FTS5 Database → Search Results → Claude
```
1. **Query**: Claude uses MCP search tools (e.g., `search_observations`)
2. **Search**: MCP server calls SessionSearch service with query parameters
3. **FTS5**: Full-text search executes against FTS5 virtual tables
4. **Format**: Results formatted as `search_result` blocks with citations
5. **Return**: Claude receives citable search results for analysis
1. **User Query**: User asks naturally: "What bugs did we fix?"
2. **Skill Invoked**: Claude recognizes intent and invokes mem-search skill
3. **HTTP API**: Skill uses curl to call HTTP endpoint (e.g., `/api/search/observations`)
4. **SessionSearch**: Worker service queries FTS5 virtual tables
5. **Format**: Results formatted and returned to skill
6. **Return**: Claude presents formatted results to user
**Token Savings**: ~2,250 tokens per session vs MCP approach through progressive disclosure
## Session Lifecycle
```
┌─────────────────────────────────────────────────────────────────┐
│ 0. Smart Install Hook Fires
│ 0. Smart Install Pre-Hook Fires │
│ Checks dependencies (cached), only runs on version changes │
│ Not a lifecycle hook - runs before context-hook starts │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
@@ -101,17 +106,14 @@ Claude Request → MCP Server → SessionSearch Service → FTS5 Database → Se
```
claude-mem/
├── src/
│ ├── hooks/ # Hook implementations (7 hooks)
│ │ ├── smart-install.ts # Dependency check (cached)
│ ├── hooks/ # Hook implementations (6 hooks)
│ │ ├── context-hook.ts # SessionStart
│ │ ├── user-message-hook.ts # UserMessage (for debugging)
│ │ ├── new-hook.ts # UserPromptSubmit
│ │ ├── save-hook.ts # PostToolUse
│ │ ├── summary-hook.ts # Stop
│ │ ── cleanup-hook.ts # SessionEnd
│ │
│ ├── servers/ # MCP servers
│ │ └── search-server.ts # MCP search tools server (9 tools)
│ │ ── cleanup-hook.ts # SessionEnd
│ │ └── hook-response.ts # Hook response utilities
│ │
│ ├── sdk/ # Claude Agent SDK integration
│ │ ├── prompts.ts # XML prompt builders
@@ -143,22 +145,32 @@ claude-mem/
│ ├── platform.ts
│ └── port-allocator.ts
├── scripts/ # Build and utility scripts
│ └── smart-install.js # Cached dependency checker (pre-hook)
├── plugin/ # Plugin distribution
│ ├── .claude-plugin/
│ │ └── plugin.json
│ ├── .mcp.json # MCP server configuration
│ ├── hooks/
│ │ └── hooks.json
│ ├── scripts/ # Built executables
│ │ ├── smart-install.js
│ │ ├── context-hook.js
│ │ ├── user-message-hook.js
│ │ ├── new-hook.js
│ │ ├── save-hook.js
│ │ ├── summary-hook.js
│ │ ├── cleanup-hook.js
│ │ ── worker-service.cjs # Background worker
│ │ └── search-server.mjs # MCP search server
│ │ ── worker-service.cjs # Background worker + HTTP API
│ │
│ ├── skills/ # Agent skills (v5.4.0+)
│ │ ├── mem-search/ # Search skill with progressive disclosure (v5.5.0)
│ │ │ ├── SKILL.md # Skill frontmatter (~250 tokens)
│ │ │ ├── operations/ # 12 detailed operation docs
│ │ │ └── principles/ # 2 principle guides
│ │ ├── troubleshoot/ # Troubleshooting skill
│ │ │ ├── SKILL.md
│ │ │ └── operations/ # 6 operation docs
│ │ └── version-bump/ # Version management skill (deprecated)
│ │
│ └── ui/ # Built viewer UI
│ └── viewer.html # Self-contained bundle
@@ -170,8 +182,7 @@ claude-mem/
## Component Details
### 1. Plugin Hooks (7 Hooks)
- **smart-install.js** - Cached dependency checker (only runs on version changes)
### 1. Plugin Hooks (6 Hooks)
- **context-hook.js** - SessionStart: Starts PM2 worker, injects context
- **user-message-hook.js** - UserMessage: Debugging hook
- **new-hook.js** - UserPromptSubmit: Creates session, saves prompt
@@ -179,11 +190,14 @@ claude-mem/
- **summary-hook.js** - Stop: Generates session summary
- **cleanup-hook.js** - SessionEnd: Marks session complete
**Note**: smart-install.js is a pre-hook dependency checker (not a lifecycle hook). It's called before context-hook via command chaining in hooks.json and only runs when dependencies need updating.
See [Plugin Hooks](/architecture/hooks) for detailed hook documentation.
### 2. Worker Service
Express.js HTTP server on port 37777 (configurable) with:
- 8 HTTP/SSE endpoints for viewer UI
- 10 search HTTP API endpoints (v5.4.0+)
- 8 viewer UI HTTP/SSE endpoints
- Async observation processing via Claude Agent SDK
- Real-time updates via Server-Sent Events
- Auto-managed by PM2 process manager
@@ -199,13 +213,21 @@ SQLite3 with better-sqlite3 driver featuring:
See [Database Architecture](/architecture/database) for schema and FTS5 search.
### 4. MCP Search Server (9 Tools)
Provides 9 specialized search tools:
- search_observations, search_sessions, search_user_prompts
- find_by_concept, find_by_file, find_by_type
- get_recent_context, get_context_timeline, get_timeline_by_query
### 4. mem-search Skill (v5.4.0+)
Skill-based search with progressive disclosure providing 10 search operations:
- Search observations, sessions, prompts (full-text FTS5)
- Filter by type, concept, file
- Get recent context, timeline, timeline by query
- API help documentation
See [MCP Search Server](/architecture/mcp-search) for search tools and examples.
**Token Savings**: ~2,250 tokens per session vs MCP approach
- Skill frontmatter: ~250 tokens (loaded at session start)
- Full instructions: ~2,500 tokens (loaded on-demand when invoked)
- HTTP API endpoints instead of MCP tools
**Skill Enhancement (v5.5.0)**: Renamed from "search" to "mem-search" for better scope differentiation. Effectiveness increased from 67% to 100% with enhanced triggers and comprehensive documentation.
See [Search Architecture](/architecture/search-architecture) for technical details and examples.
### 5. Viewer UI
React + TypeScript web interface at http://localhost:37777 featuring:
@@ -0,0 +1,448 @@
---
title: "Search Architecture"
description: "mem-search skill with HTTP API and progressive disclosure"
---
# Search Architecture
Claude-Mem uses a skill-based search architecture that provides intelligent memory retrieval through natural language queries. This replaced the MCP-based approach in v5.4.0, saving ~2,250 tokens per session start. The skill was enhanced and renamed to "mem-search" in v5.5.0 for better scope differentiation.
## Overview
**Architecture**: Skill-Based Search + HTTP API + Progressive Disclosure
**Key Components**:
1. **mem-search Skill** (`plugin/skills/mem-search/SKILL.md`) - Auto-invoked when users ask about past work
2. **HTTP API Endpoints** (10 routes) - Fast, efficient search operations on port 37777
3. **Worker Service** - Express.js server with FTS5 full-text search
4. **SQLite Database** - Persistent storage with FTS5 virtual tables
5. **Chroma Vector DB** - Semantic search with hybrid retrieval
**v5.5.0 Enhancement**: Renamed from "search" to "mem-search" with:
- Effectiveness increased from 67% to 100%
- Concrete triggers increased from 44% to 85%
- 5+ unique identifiers for better scope differentiation
- Comprehensive documentation (17 files, 12 operation guides)
## How It Works
### 1. User Query (Natural Language)
```
User: "What bugs did we fix last session?"
```
### 2. Skill Invocation
Claude recognizes the intent and invokes the mem-search skill:
- Skill frontmatter (~250 tokens) loaded at session start
- Full skill instructions loaded on-demand when skill is invoked
- Progressive disclosure pattern minimizes context overhead
- "mem-search" naming provides clear scope differentiation from native memory
### 3. HTTP API Call
The skill uses `curl` to call the HTTP API:
```bash
curl "http://localhost:37777/api/search/observations?query=bugs&type=bugfix&limit=5"
```
### 4. FTS5 Search
Worker service queries SQLite FTS5 virtual tables:
```sql
SELECT * FROM observations_fts
WHERE observations_fts MATCH ?
AND type = 'bugfix'
ORDER BY rank
LIMIT 5
```
### 5. Results Formatted
Skill formats results and returns to Claude:
```
## Recent Bugfixes
1. [bugfix] Fixed authentication token expiry
Date: 2025-11-08 14:23:45
Files: src/auth/jwt.ts
2. [bugfix] Resolved database connection leak
Date: 2025-11-08 13:15:22
Files: src/services/database.ts
```
### 6. User Sees Answer
Claude presents the formatted results naturally in conversation.
## Architecture Change (v5.4.0)
### Before: MCP-Based Search
**Approach**: 9 MCP tools registered at session start
**Token Cost**: ~2,500 tokens in tool definitions per session
- Each tool's schema, parameters, descriptions loaded
- All 9 tools available whether needed or not
- No progressive disclosure
**Example MCP Tool**:
```json
{
"name": "search_observations",
"description": "Full-text search across observations...",
"inputSchema": {
"type": "object",
"properties": {
"query": { "type": "string", "description": "..." },
"type": { "type": "array", "items": { "enum": [...] } },
"format": { "enum": ["index", "full"] },
// ... many more parameters
}
}
}
```
### After: Skill-Based Search
**Approach**: 1 mem-search skill with progressive disclosure
**Token Cost**: ~250 tokens in skill frontmatter per session
- Only skill description loaded at session start
- Full instructions loaded on-demand when skill is invoked
- HTTP API endpoints instead of MCP protocol
**Example Skill Frontmatter**:
```markdown
# Claude-Mem mem-search Skill
Access claude-mem's persistent memory through a comprehensive HTTP API.
Search for past work, understand context, and learn from previous decisions.
## When to Use This Skill
Invoke this skill when users ask about:
- Past work: "What did we do last session?"
- Bug fixes: "Did we fix this before?"
- Features: "How did we implement authentication?"
...
```
**Token Savings**: ~2,250 tokens per session start (90% reduction)
## HTTP API Endpoints
The worker service exposes 10 search endpoints:
### Full-Text Search
```
GET /api/search/observations
GET /api/search/sessions
GET /api/search/prompts
```
**Parameters**:
- `query` - FTS5 search query (required)
- `type` - Filter by type (bugfix, feature, refactor, etc.)
- `project` - Filter by project name
- `limit` - Maximum results (default: 20)
- `offset` - Pagination offset
- `format` - Response format (index or full)
**Example**:
```bash
curl "http://localhost:37777/api/search/observations?query=authentication&type=decision&limit=5"
```
### Filtered Search
```
GET /api/search/by-type
GET /api/search/by-concept
GET /api/search/by-file
```
**Parameters**:
- `type` / `concept` / `filePath` - Filter criteria (required)
- `project` - Filter by project
- `limit` - Maximum results
- `format` - Response format
**Example**:
```bash
curl "http://localhost:37777/api/search/by-file?filePath=worker-service.ts&limit=10"
```
### Context Retrieval
```
GET /api/context/recent
GET /api/context/timeline
GET /api/timeline/by-query
```
**Parameters**:
- `project` - Filter by project
- `limit` - Number of sessions/records
- `anchor` - Timeline anchor point (ID or timestamp)
- `depth_before` - Records before anchor
- `depth_after` - Records after anchor
**Example**:
```bash
curl "http://localhost:37777/api/context/recent?project=claude-mem&limit=5"
```
### Documentation
```
GET /api/search/help
```
Returns API documentation in JSON format.
## Progressive Disclosure Pattern
The mem-search skill uses progressive disclosure to minimize token usage:
### Layer 1: Skill Frontmatter (Session Start)
**What's Loaded**: Skill description and when to use it (~250 tokens)
**Purpose**: Claude can recognize when to invoke the skill
**Example**:
```markdown
# Claude-Mem mem-search Skill
Access claude-mem's persistent memory through a comprehensive HTTP API.
## When to Use This Skill
Invoke this skill when users ask about:
- Past work: "What did we do last session?"
- Bug fixes: "Did we fix this before?"
...
```
### Layer 2: Full Skill Instructions (On-Demand)
**What's Loaded**: Complete operation documentation (~2,500 tokens)
**Purpose**: Detailed instructions for each search operation
**When Loaded**: Only when Claude invokes the skill
**Example Structure**:
```
/skills/search/
├── SKILL.md (main frontmatter)
├── operations/
│ ├── observations.md (detailed instructions)
│ ├── sessions.md
│ ├── prompts.md
│ ├── by-type.md
│ ├── by-concept.md
│ ├── by-file.md
│ ├── recent-context.md
│ ├── timeline.md
│ ├── timeline-by-query.md
│ ├── help.md
│ ├── formatting.md
│ └── common-workflows.md
```
### Layer 3: API Response
**What's Returned**: Search results in requested format
**Format Options**:
- `index` - Titles, dates, IDs only (~50-100 tokens per result)
- `full` - Complete details (~500-1000 tokens per result)
**Progressive Usage**: Start with `index`, drill down with `full` as needed
## Implementation Details
### mem-search Skill Structure
```
plugin/skills/mem-search/
├── SKILL.md # Main frontmatter (~250 tokens)
├── operations/
│ ├── observations.md # Search observations
│ ├── sessions.md # Search sessions
│ ├── prompts.md # Search prompts
│ ├── by-type.md # Filter by type
│ ├── by-concept.md # Filter by concept
│ ├── by-file.md # Filter by file
│ ├── recent-context.md # Get recent context
│ ├── timeline.md # Timeline around point
│ ├── timeline-by-query.md # Search + timeline
│ ├── help.md # API documentation
│ ├── formatting.md # Result formatting guide
│ └── common-workflows.md # Usage patterns
```
### Worker Service Integration
**File**: `src/services/worker-service.ts`
**Search Routes**:
```typescript
// Full-text search
app.get('/api/search/observations', handleSearchObservations);
app.get('/api/search/sessions', handleSearchSessions);
app.get('/api/search/prompts', handleSearchPrompts);
// Filtered search
app.get('/api/search/by-type', handleSearchByType);
app.get('/api/search/by-concept', handleSearchByConcept);
app.get('/api/search/by-file', handleSearchByFile);
// Context retrieval
app.get('/api/context/recent', handleRecentContext);
app.get('/api/context/timeline', handleTimeline);
app.get('/api/timeline/by-query', handleTimelineByQuery);
// Documentation
app.get('/api/search/help', handleHelp);
```
**Database Access**:
- Uses `SessionSearch` service for FTS5 queries
- Uses `SessionStore` for structured queries
- Hybrid search with ChromaDB for semantic similarity
### Security
**FTS5 Injection Prevention** (v4.2.3):
```typescript
function escapeFTS5Query(query: string): string {
return query.replace(/"/g, '""');
}
```
All user-provided search queries are properly escaped to prevent SQL injection.
**Comprehensive Testing**: 332 injection attack tests covering:
- Special characters
- SQL keywords
- Quote escaping
- Boolean operators
## Benefits
### 1. Token Efficiency
**Before (MCP)**:
- Session start: ~2,500 tokens for tool definitions
- Every session pays this cost
- No progressive disclosure
**After (Skill)**:
- Session start: ~250 tokens for skill frontmatter
- Full instructions: ~2,500 tokens (only when invoked)
- Net savings: ~2,250 tokens per session (~90% reduction)
### 2. Natural Language Interface
**Before**: Users needed to learn MCP tool syntax
```
search_observations with query="authentication" and type="decision"
```
**After**: Users ask naturally
```
"What decisions did we make about authentication?"
```
Claude translates to appropriate API call.
### 3. Flexibility
**HTTP API Benefits**:
- Can be called from skills, MCP tools, or other clients
- Easy to test with curl
- Standard REST conventions
- JSON responses
**Progressive Disclosure**:
- Loads only what's needed
- Can add more operations without increasing base cost
- Documentation co-located with operations
### 4. Performance
**Fast Queries**: FTS5 full-text search <10ms for typical queries
**Caching**: HTTP layer allows response caching
**Pagination**: Efficient result pagination with offset/limit
## Migration Notes
### For Users
**No Action Required**: The migration from MCP to skill-based search is transparent.
**Same Questions Work**: Natural language queries work exactly the same way.
**Invisible Change**: Users won't notice any difference except better performance.
### For Developers
**Deprecated**: MCP search server (`src/servers/search-server.ts`)
- Source file kept for reference
- No longer built or registered
- MCP configuration removed from `plugin/.mcp.json`
**New Implementation**: Skill-based search
- Skill files: `plugin/skills/mem-search/`
- HTTP endpoints: `src/services/worker-service.ts` (lines 200-400)
- Build script: `npm run build` includes skill files
- Sync script: `npm run sync-marketplace` copies to plugin directory
## Troubleshooting
### Worker Service Not Running
If searches fail, check worker service:
```bash
pm2 list # Check status
npm run worker:restart # Restart worker
npm run worker:logs # View logs
```
### HTTP Endpoints Not Responding
Test endpoints directly:
```bash
# Health check
curl http://localhost:37777/health
# Search test
curl "http://localhost:37777/api/search/observations?query=test&limit=1"
```
### Skill Not Invoking
If Claude doesn't invoke the mem-search skill automatically:
1. Check skill files exist: `ls ~/.claude/plugins/marketplaces/thedotmack/plugin/skills/mem-search/`
2. Restart Claude Code session to reload skill definitions
3. Try more explicit phrasing: "Search past sessions for bug fixes" or "What did we do in yesterday's session?"
4. Ensure your question is about previous sessions (not current conversation context)
## Next Steps
- [Search Tools Usage](/usage/search-tools) - User guide with examples
- [Worker Service Architecture](/architecture/worker-service) - HTTP API details
- [Database Schema](/architecture/database) - FTS5 tables and indexes

Before

Width:  |  Height:  |  Size: 78 KiB

After

Width:  |  Height:  |  Size: 78 KiB

Before

Width:  |  Height:  |  Size: 78 KiB

After

Width:  |  Height:  |  Size: 78 KiB

Before

Width:  |  Height:  |  Size: 42 KiB

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.1 MiB

@@ -137,22 +137,19 @@ Hooks are configured in `plugin/hooks/hooks.json`:
}
```
### MCP Server Configuration
### Search Configuration (v5.4.0+)
The MCP search server is configured in `plugin/.mcp.json`:
**Migration Note**: As of v5.4.0, Claude-Mem uses skill-based search instead of MCP tools. As of v5.5.0, the skill was renamed to "mem-search" for better scope differentiation.
```json
{
"mcpServers": {
"claude-mem-search": {
"type": "stdio",
"command": "${CLAUDE_PLUGIN_ROOT}/scripts/search-server.mjs"
}
}
}
```
**Previous (v5.3.x and earlier)**: MCP search server with 9 tools (~2,500 tokens per session)
**Current (v5.4.0+)**: mem-search skill with HTTP API (~250 tokens per session)
This registers the `claude-mem-search` server with Claude Code, making the 9 search tools available in all sessions.
**No configuration required** - the mem-search skill is automatically available in Claude Code sessions.
Search operations are now provided via:
- **Skill**: `plugin/skills/mem-search/SKILL.md` (auto-invoked when users ask about past work)
- **HTTP API**: 10 endpoints on worker service port 37777
- **Progressive Disclosure**: Full instructions loaded on-demand only when needed
## PM2 Configuration
@@ -526,11 +526,14 @@ The `release` script:
6. Publish to NPM
```bash
# Use the version bump skill (recommended as of v4.3.0)
# In Claude Code, run: /skill version-bump
# This updates package.json, marketplace.json, and CLAUDE.md
# Manual version bump:
# 1. Update version in package.json
# 2. Update version in plugin/.claude-plugin/plugin.json
# 3. Update version at top of CLAUDE.md
# 4. Update version badge in README.md
# 5. Run: npm run build && npm run sync-marketplace
# Or manually:
# Or use npm version command:
npm version 4.3.2
# Update changelog
+1 -1
View File
@@ -66,7 +66,7 @@
"architecture/hooks",
"architecture/worker-service",
"architecture/database",
"architecture/mcp-search"
"architecture/search-architecture"
]
}
]
@@ -68,14 +68,16 @@ Claude Code's hook system provides exactly what we need:
---
## The Seven Hook Scripts
## The Six Hook Scripts + Pre-Hook
Claude-Mem uses 7 hook scripts across 5 lifecycle events. SessionStart runs 3 hooks in sequence.
Claude-Mem uses 6 lifecycle hook scripts across 5 lifecycle events, plus 1 pre-hook script for dependency management. SessionStart runs 2 hooks in sequence (after the pre-hook script).
### Hook 1: SessionStart - Smart Install
### Pre-Hook: Smart Install (Before SessionStart)
**Purpose:** Intelligently manage dependencies and start worker service
**Note:** This is NOT a lifecycle hook - it's a pre-hook script executed via command chaining before context-hook runs.
**When:** Claude Code starts (startup, clear, or compact)
**What it does:**
@@ -115,11 +117,11 @@ Claude-Mem uses 7 hook scripts across 5 lifecycle events. SessionStart runs 3 ho
---
### Hook 2: SessionStart - Context Injection
### Hook 1: SessionStart - Context Injection
**Purpose:** Inject relevant context from previous sessions
**When:** Claude Code starts (runs after smart-install)
**When:** Claude Code starts (runs after smart-install pre-hook)
**What it does:**
1. Extracts project name from current working directory
@@ -147,14 +149,14 @@ Claude-Mem uses 7 hook scripts across 5 lifecycle events. SessionStart runs 3 ho
|----|------|---|-------|--------|
| #2586 | 12:58 AM | 🔵 | Context hook file empty | ~51 |
*Use claude-mem MCP search to access full details*
*Use mem-search skill to access full details*
```
**Source:** `src/hooks/context-hook.ts` → `plugin/scripts/context-hook.js`
---
### Hook 3: SessionStart - User Message
### Hook 2: SessionStart - User Message
**Purpose:** Display helpful user messages during first-time setup
@@ -203,7 +205,7 @@ Claude-Mem uses 7 hook scripts across 5 lifecycle events. SessionStart runs 3 ho
---
### Hook 4: UserPromptSubmit (New Session Hook)
### Hook 3: UserPromptSubmit (New Session Hook)
**Purpose:** Initialize session tracking when user submits a prompt
@@ -250,7 +252,7 @@ VALUES (?, ?, ?, ...)
---
### Hook 5: PostToolUse (Save Observation Hook)
### Hook 4: PostToolUse (Save Observation Hook)
**Purpose:** Capture tool execution observations for later processing
@@ -311,11 +313,11 @@ VALUES (?, ?, ?, ?, ...)
---
### Hook 6: Summary Hook (Mid-Session Checkpoint)
### Hook 5: Stop Hook (Summary Generation)
**Purpose:** Generate AI-powered session summaries during the session
**When:** Triggered programmatically by the worker service
**When:** When Claude stops (triggered by Stop lifecycle event)
**What it does:**
1. Gathers session observations from database
@@ -327,7 +329,7 @@ VALUES (?, ?, ?, ?, ...)
```json
{
"hooks": {
"Summary": [{
"Stop": [{
"hooks": [{
"type": "command",
"command": "${CLAUDE_PLUGIN_ROOT}/scripts/summary-hook.js"
@@ -338,7 +340,7 @@ VALUES (?, ?, ?, ?, ...)
```
**Key decisions:**
- ✅ Triggered by worker, not by Claude Code lifecycle
- ✅ Triggered by Stop lifecycle event
- ✅ Multiple summaries per session (v4.2.0+)
- ✅ Summaries are checkpoints, not endings
- ✅ Uses Claude Agent SDK for AI compression
@@ -366,7 +368,7 @@ VALUES (?, ?, ?, ?, ...)
---
### Hook 7: SessionEnd (Cleanup Hook)
### Hook 6: SessionEnd (Cleanup Hook)
**Purpose:** Mark sessions as completed when they end
@@ -23,7 +23,7 @@ Restart Claude Code. Context from previous sessions will automatically appear in
## Key Features
- 🧠 **Persistent Memory** - Context survives across sessions
- 🔍 **9 Search Tools** - Query your project history via MCP
- 🔍 **mem-search Skill** - Query your project history with natural language (~2,250 token savings)
- 🌐 **Web Viewer UI** - Real-time memory stream visualization at http://localhost:37777
- 🎨 **Theme Toggle** - Light, dark, and system preference themes
- 🤖 **Automatic Operation** - No manual intervention required
@@ -55,11 +55,12 @@ Restart Claude Code. Context from previous sessions will automatically appear in
```
**Core Components:**
1. **5 Lifecycle Hooks** - SessionStart, UserPromptSubmit, PostToolUse, Stop, SessionEnd
2. **Worker Service** - HTTP API on port 37777 managed by PM2
3. **SQLite Database** - Stores sessions, observations, summaries with FTS5 search
4. **9 MCP Search Tools** - Query historical context with citations
5. **Web Viewer UI** - Real-time visualization with SSE and infinite scroll
1. **6 Lifecycle Hooks** - SessionStart, UserPromptSubmit, PostToolUse, Stop, SessionEnd, UserMessage
2. **Smart Install** - Cached dependency checker (pre-hook script)
3. **Worker Service** - HTTP API on port 37777 managed by PM2
4. **SQLite Database** - Stores sessions, observations, summaries with FTS5 search
5. **mem-search Skill** - Query historical context with natural language
6. **Web Viewer UI** - Real-time visualization with SSE and infinite scroll
See [Architecture Overview](architecture/overview) for details.
@@ -70,22 +71,25 @@ See [Architecture Overview](architecture/overview) for details.
- **PM2**: Process manager (bundled - no global install required)
- **SQLite 3**: For persistent storage (bundled)
## What's New in v5.1.2
## What's New in v6.0.0
**Latest Updates (v5.1.2):**
- Theme toggle for light, dark, and system preferences in viewer UI
- Improved visual design with theme-aware components
**🚀 Major Session Management & Transcript Processing Improvements:**
**Recent Updates (v5.1.0):**
- Web-based viewer UI for real-time memory stream visualization
- Server-Sent Events (SSE) for instant updates
- Infinite scroll pagination with project filtering
- 8 new HTTP/SSE endpoints in worker service
- **Enhanced Session Initialization**: Accept userPrompt and promptNumber for better context tracking
- **Live UserPrompt Updates**: Multi-turn conversation support with real-time prompt tracking
- **Improved SessionManager**: Better context handling and observation processing
- **Comprehensive Transcript Processing**: New scripts and utilities for analyzing Claude Code transcripts
- **Rich Context Extraction**: Advanced parsing utilities for extracting meaningful context from sessions
- **Refactored Architecture**: Improved hooks and SDKAgent for more reliable observation handling
- **Silent Debug Logging**: Better debugging capabilities without cluttering output
- **Enhanced Error Handling**: More robust error recovery and debugging tools
**Previous Updates (v4.3.1):**
- Fixed SessionStart hook context injection
- Smart install caching for Windows compatibility
- Progressive disclosure context with observation timelines
**Breaking Changes**: Significant architectural changes in session management and observation handling. Existing sessions continue to work, but internal APIs have evolved.
**Previous Highlights:**
- **v5.5.0**: mem-search skill enhancement with 100% effectiveness rate
- **v5.4.0**: Skill-based search architecture (~2,250 tokens saved per session)
- **v5.1.2**: Theme toggle for light/dark mode in viewer UI
## Next Steps
@@ -5,6 +5,22 @@ description: "Common issues and solutions for Claude-Mem"
# Troubleshooting Guide
## Quick Diagnostic Tool
Describe any issues you're experiencing to Claude, and the troubleshoot skill will automatically activate to provide diagnosis and fixes.
The troubleshoot skill will:
- ✅ Check PM2 worker status and health
- ✅ Verify database existence and integrity
- ✅ Test worker service connectivity
- ✅ Validate dependencies installation
- ✅ Check port configuration and availability
- ✅ Provide automated fixes for common issues
The skill includes comprehensive diagnostics, automated repair sequences, and detailed troubleshooting workflows for all common issues. Simply describe the problem naturally to invoke it.
---
## v5.x Specific Issues
### Viewer UI Not Loading
@@ -54,10 +54,21 @@ When Claude finishes responding (triggering the Stop hook), a summary is automat
When you start a new Claude Code session, the SessionStart hook:
1. Queries the database for recent sessions in your project
2. Retrieves the last 10 session summaries
3. Formats them with three-tier verbosity (most recent = most detail)
4. Injects them into Claude's initial context
1. Queries the database for recent observations in your project (default: 50)
2. Retrieves recent session summaries for context
3. Displays observations in a chronological timeline with session markers
4. Shows full summary details (Investigated, Learned, Completed, Next Steps) **only if the summary was generated after the last observation**
5. Injects formatted context into Claude's initial context
**Summary Display Logic:**
The most recent summary's full details appear at the end of the context display **only when** the summary was generated after the most recent observation. This ensures you see summary details when they represent the latest state of your project, but not when new observations have been captured since the last summary.
For example:
- ✅ **Shows summary**: Last observation at 2:00 PM, summary generated at 2:05 PM → Summary details appear
- ❌ **Hides summary**: Summary generated at 2:00 PM, new observation at 2:05 PM → Summary details hidden (outdated)
This prevents showing stale summaries when new work has been captured but not yet summarized.
This means Claude "remembers" what happened in previous sessions!
@@ -138,26 +149,31 @@ FROM observations
WHERE session_id = 'YOUR_SESSION_ID';
```
## Understanding Verbosity Levels
## Understanding Progressive Disclosure
Context injection uses three-tier verbosity for efficient token usage:
Context injection uses progressive disclosure for efficient token usage:
### Tier 1 (Most Recent Session)
- Full summary with all details
- Request, investigated, learned, completed, next_steps, notes
- ~500-1000 tokens
### Layer 1: Index Display (Session Start)
- Shows observation titles with token cost estimates
- Displays session markers in chronological timeline
- Groups observations by file for visual clarity
- Shows full summary details **only if** generated after last observation
- Token cost: ~50-200 tokens for index view
### Tier 2 (Sessions 2-5)
- Medium detail
- Request, learned, completed
- ~200-400 tokens
### Layer 2: On-Demand Details (mem-search Skill)
- Ask naturally: "What bugs did we fix?" or "How did we implement X?"
- Claude auto-invokes mem-search skill to fetch full details
- Search by concept, file, type, or keyword
- Timeline context around specific observations
- Token cost: ~100-500 tokens per observation fetched
- Skill uses HTTP API (v5.4.0+) for efficient retrieval
### Tier 3 (Sessions 6-10)
- Brief summary
- Request and completed only
- ~100-200 tokens
### Layer 3: Perfect Recall (Code Access)
- Read source files directly when needed
- Access original transcripts and raw data
- Full context available on-demand
This ensures you get maximum detail for recent work while still having context from older sessions.
This ensures efficient token usage while maintaining access to complete history when needed.
## Multi-Prompt Sessions & `/clear` Behavior
@@ -177,8 +193,23 @@ When you use `/clear`, the session doesn't end - it continues with a new prompt
The `/clear` command clears the conversation context visible to Claude AND re-injects fresh context from recent sessions, while the underlying session continues tracking observations.
## Searching Your History (v5.4.0+)
Claude-Mem uses the mem-search skill for querying your project history. Simply ask naturally:
```
"What bugs did we fix last session?"
"How did we implement authentication?"
"What changes were made to worker-service.ts?"
"Show me recent work on this project"
```
Claude automatically recognizes your intent and invokes the mem-search skill, which uses HTTP API endpoints to query your memory efficiently.
**Token Savings**: ~2,250 tokens per session start vs previous MCP approach
## Next Steps
- [MCP Search Tools](/usage/search-tools) - Learn how to search your project history
- [Skill-Based Search](/usage/search-tools) - Learn how to search your project history
- [Architecture Overview](/architecture/overview) - Understand how it works
- [Troubleshooting](/troubleshooting) - Common issues and solutions
+404
View File
@@ -0,0 +1,404 @@
---
title: "mem-search Skill"
description: "Query your project history with natural language"
---
# mem-search Skill Usage
Once claude-mem is installed as a plugin, you can search your project history using natural language. Claude automatically invokes the mem-search skill when you ask about past work.
## How It Works
**v5.5.0 Enhancement**: The search skill was renamed to "mem-search" for better scope differentiation, with effectiveness increased from 67% to 100% and enhanced concrete triggers (85% vs 44%).
**v5.4.0 Architecture**: Claude-Mem uses a skill-based search architecture instead of MCP tools, saving ~2,250 tokens per session start through progressive disclosure.
**Simple Usage:**
- Just ask naturally: *"What did we do last session?"*
- Claude recognizes the intent and invokes the mem-search skill
- The skill uses HTTP API endpoints to query your memory
- Results are formatted and presented to you
**Benefits:**
- **Token Efficient**: ~250 tokens (skill frontmatter) vs ~2,500 tokens (MCP tool definitions)
- **Natural Language**: No need to learn specific tool syntax
- **Progressive Disclosure**: Only loads detailed instructions when needed
- **Auto-Invoked**: Claude knows when to search based on your questions
- **Scope Differentiation**: "mem-search" clearly distinguishes from native conversation memory
## Quick Reference
| Operation | Purpose |
|-------------------------|----------------------------------------------|
| Search Observations | Full-text search across observations |
| Search Sessions | Full-text search across session summaries |
| Search Prompts | Full-text search across raw user prompts |
| By Concept | Find observations tagged with concepts |
| By File | Find observations referencing files |
| By Type | Find observations by type |
| Recent Context | Get recent session context |
| Timeline | Get unified timeline around a specific point |
| Timeline by Query | Search and get timeline context in one step |
| API Help | Get search API documentation |
## Example Queries
### Natural Language Queries
**Search Observations:**
```
"What bugs did we fix related to authentication?"
"Show me all decisions about the build system"
"Find refactoring work on the database"
```
**Search Sessions:**
```
"What did we learn about hooks?"
"What was accomplished in the API implementation?"
"Show me recent work on this project"
```
**Search Prompts:**
```
"When did I ask about authentication features?"
"Find all my requests about dark mode"
```
**Note**: Claude automatically translates your natural language queries into the appropriate search operations.
### Search by File
```
"Show me everything related to worker-service.ts"
"What changes were made to migrations.ts?"
"Find all work on the database file"
```
### Search by Concept
```
"Show observations tagged with architecture"
"Find all security-related observations"
"What patterns have we used?"
```
### Search by Type
```
"Find all feature implementations"
"Show me all decisions and discoveries"
"What bugs have we fixed?"
```
### Recent Context
```
"Show me what we've been working on"
"Get context from the last 5 sessions"
"What happened recently on this project?"
```
### Timeline Queries
**Get timeline around a specific point:**
```
"What was happening when we implemented authentication?"
"Show me the context around that bug fix"
"What led to the decision to refactor the database?"
```
**Timeline by query:**
```
"Find when we added the viewer UI and show what happened around that time"
"Search for authentication work and show the timeline"
```
**Benefits:**
- See the complete narrative arc around key events
- All record types (observations, sessions, prompts) in chronological view
- Understand what was happening before and after important changes
## Search Strategy
The mem-search skill uses a progressive disclosure pattern to efficiently retrieve information:
### 1. Ask Naturally
Start with a natural language question:
```
"What bugs did we fix related to authentication?"
```
### 2. Claude Invokes mem-search Skill
Claude recognizes your intent and loads the mem-search skill (~250 tokens for skill frontmatter).
### 3. Skill Uses HTTP API
The skill calls the appropriate HTTP endpoint (e.g., `/api/search/observations`) with the query.
### 4. Results Formatted
Results are formatted and presented to you, usually starting with an index/summary format.
### 5. Deep Dive if Needed
If you need more details, ask follow-up questions:
```
"Tell me more about observation #123"
"Show me the full details of that decision"
```
**Benefits of This Approach:**
- **Token Efficient**: Only loads what you need, when you need it
- **Natural**: No syntax to learn
- **Progressive**: Start with overview, drill down as needed
- **Automatic**: Claude handles the search invocation
## Advanced Filtering
You can refine searches using natural language filters:
### Date Ranges
```
"What bugs did we fix in October?"
"Show me work from last week"
"Find decisions made between October 1-31"
```
### Multiple Types
```
"Show me all decisions and features"
"Find bugfixes and refactorings"
```
### Concepts
```
"Find database work related to architecture and performance"
"Show security observations"
```
### File-Specific
```
"Show refactoring work that touched worker-service.ts"
"Find changes to auth files"
```
### Project Filtering
```
"Show authentication work on my-app project"
"What have we done on this codebase?"
```
**Note**: Claude translates your natural language into the appropriate API filters automatically.
## Under the Hood: HTTP API
The mem-search skill uses HTTP endpoints on the worker service (port 37777):
- `GET /api/search/observations` - Full-text search observations
- `GET /api/search/sessions` - Full-text search session summaries
- `GET /api/search/prompts` - Full-text search user prompts
- `GET /api/search/by-concept` - Find observations by concept tag
- `GET /api/search/by-file` - Find work related to specific files
- `GET /api/search/by-type` - Find observations by type
- `GET /api/context/recent` - Get recent session context
- `GET /api/context/timeline` - Get timeline around specific point
- `GET /api/timeline/by-query` - Search + timeline in one call
- `GET /api/search/help` - API documentation
These endpoints use FTS5 full-text search with support for:
- Boolean operators (AND, OR, NOT)
- Phrase searches
- Column-specific searches
- Date range filtering
- Project filtering
## Result Metadata
All results include rich metadata:
```
## JWT authentication decision
**Type**: decision
**Date**: 2025-10-21 14:23:45
**Concepts**: authentication, security, architecture
**Files Read**: src/auth/middleware.ts, src/utils/jwt.ts
**Files Modified**: src/auth/jwt-strategy.ts
**Narrative**:
Decided to implement JWT-based authentication instead of session-based
authentication for better scalability and stateless design...
**Facts**:
• JWT tokens expire after 1 hour
• Refresh tokens stored in httpOnly cookies
• Token signing uses RS256 algorithm
• Public keys rotated every 30 days
```
## Citations
All search results include citations using the `claude-mem://` URI scheme:
- `claude-mem://observation/123` - Specific observation
- `claude-mem://session/abc-456` - Specific session
- `claude-mem://user-prompt/789` - Specific user prompt
These citations enable referencing specific historical context in your work.
## Token Management
### Token Efficiency Tips
1. **Start with index format**: ~50-100 tokens per result
2. **Use small limits**: Start with 3-5 results
3. **Apply filters**: Narrow results before searching
4. **Paginate**: Use offset to browse results in batches
### Token Estimates
| Format | Tokens per Result |
|--------|-------------------|
| Index | 50-100 |
| Full | 500-1000 |
**Example**:
- 20 results in index format: ~1,000-2,000 tokens
- 20 results in full format: ~10,000-20,000 tokens
## Common Use Cases
### 1. Debugging Issues
Find what went wrong:
```
search_observations with query="error database connection" and type="bugfix"
```
### 2. Understanding Decisions
Review architectural choices:
```
find_by_type with type="decision" and format="index"
```
Then deep dive on specific decisions:
```
search_observations with query="[DECISION TITLE]" and format="full"
```
### 3. Code Archaeology
Find when a file was modified:
```
find_by_file with filePath="worker-service.ts"
```
### 4. Feature History
Track feature development:
```
search_sessions with query="authentication feature"
search_user_prompts with query="add authentication"
```
### 5. Learning from Past Work
Review refactoring patterns:
```
find_by_type with type="refactor" and limit=10
```
### 6. Context Recovery
Restore context after time away:
```
get_recent_context with limit=5
search_sessions with query="[YOUR PROJECT NAME]" and orderBy="date_desc"
```
## Best Practices
1. **Index first, full later**: Always start with index format
2. **Small limits**: Start with 3-5 results to avoid token limits
3. **Use filters**: Narrow results before searching
4. **Specific queries**: More specific = better results
5. **Review citations**: Use citations to reference past decisions
6. **Date filtering**: Use date ranges for time-based searches
7. **Type filtering**: Use types to categorize searches
8. **Concept tags**: Use concepts for thematic searches
## Troubleshooting
### No Results Found
1. Check database has data:
```bash
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations;"
```
2. Try broader natural language query:
```
"Show me anything about authentication" # Broader
vs
"Find exact JWT authentication implementation" # Too specific
```
3. Ask without filters first:
```
"What do we have about auth?"
# Then narrow down
"Show me auth-related decisions"
```
### Worker Service Not Running
If search isn't working, check the worker service:
```bash
pm2 list # Check worker status
npm run worker:restart # Restart if needed
npm run worker:logs # View logs
```
Or describe the issue to Claude and the troubleshoot skill will automatically activate to provide diagnosis.
### Performance Issues
If searches seem slow:
1. Be more specific in your queries
2. Ask for recent work (naturally filters by date)
3. Specify the project you're interested in
4. Ask for fewer results initially
## Technical Details
**Architecture Change (v5.4.0)**:
- **Before**: 9 MCP tools (~2,500 tokens in tool definitions per session start)
- **After**: 1 mem-search skill (~250 tokens in frontmatter, full instructions loaded on-demand)
- **Savings**: ~2,250 tokens per session start
- **Migration**: Transparent - users don't need to change how they ask questions
**v5.5.0 Enhancement**: Renamed from "search" to "mem-search" with improved effectiveness (67% → 100%) and enhanced triggers (44% → 85%).
**How the Skill Works:**
1. User asks a question about past work
2. Claude recognizes the intent matches the mem-search skill description
3. Skill loads full instructions from `plugin/skills/mem-search/SKILL.md`
4. Skill uses `curl` to call HTTP API endpoints
5. Results formatted and returned to Claude
6. Claude presents results to user
## Next Steps
- [Architecture Overview](/architecture/overview) - System components
- [Database Schema](/architecture/database) - Understanding the data
- [Getting Started](/usage/getting-started) - Automatic operation
-492
View File
@@ -1,492 +0,0 @@
---
title: "MCP Search Tools"
description: "Query your project history with 9 specialized search tools"
---
# MCP Search Tools Usage
Once claude-mem is installed as a plugin, 9 search tools become available in your Claude Code sessions for querying project history.
## Quick Reference
| Tool | Purpose |
|-------------------------|----------------------------------------------|
| search_observations | Full-text search across observations |
| search_sessions | Full-text search across session summaries |
| search_user_prompts | Full-text search across raw user prompts |
| find_by_concept | Find observations tagged with concepts |
| find_by_file | Find observations referencing files |
| find_by_type | Find observations by type |
| get_recent_context | Get recent session context |
| get_context_timeline | Get unified timeline around a specific point |
| get_timeline_by_query | Search and get timeline context in one step |
## Example Queries
### search_observations
Find all decisions about the build system:
```
Use search_observations to find all decisions about the build system
```
Find bugs related to authentication:
```
search_observations with query="authentication" and type="bugfix"
```
Search for refactoring work:
```
search_observations with query="refactor database" and type="refactor"
```
### search_sessions
Find what we learned about hooks:
```
Use search_sessions to find what we learned about hooks
```
Search for completed work on the API:
```
search_sessions with query="API implementation"
```
### search_user_prompts
Find when user asked about authentication:
```
search_user_prompts with query="authentication feature"
```
Trace user requests for a specific feature:
```
search_user_prompts with query="dark mode"
```
**Benefits**:
- See exactly what the user asked for (vs what was implemented)
- Detect patterns in repeated requests
- Debug miscommunications between user intent and implementation
### find_by_file
Show everything related to worker-service.ts:
```
Use find_by_file to show me everything related to worker-service.ts
```
Find all work on the database migration file:
```
find_by_file with filePath="migrations.ts"
```
### find_by_concept
Show observations tagged with 'architecture':
```
Use find_by_concept to show observations tagged with 'architecture'
```
Find all 'security' related observations:
```
find_by_concept with concept="security"
```
### find_by_type
Find all feature implementations:
```
find_by_type with type="feature"
```
Find all decisions and discoveries:
```
find_by_type with type=["decision", "discovery"]
```
### get_recent_context
Get the last 5 sessions for context:
```
get_recent_context with limit=5
```
Get recent context for debugging:
```
Use get_recent_context to show me what we've been working on
```
### get_context_timeline
Get a unified timeline of context around a specific point in time. This tool interleaves observations, sessions, and user prompts chronologically to show what was happening before and after a specific moment.
**Anchor by observation ID:**
```
get_context_timeline with anchor=12345 and depth_before=10 and depth_after=10
```
**Anchor by session ID:**
```
get_context_timeline with anchor="S123" and depth_before=5 and depth_after=5
```
**Anchor by ISO timestamp:**
```
get_context_timeline with anchor="2025-10-21T14:30:00Z" and depth_before=15 and depth_after=15
```
**Use cases:**
- Understand what was happening when a specific observation occurred
- See the full context around a bug fix or decision
- Trace the events leading up to and following a specific change
- View chronological sequence of related work
**Benefits:**
- All record types (observations, sessions, prompts) in one chronological view
- Configurable depth before/after anchor point
- Flexible anchoring by ID or timestamp
- See the complete narrative arc around key events
### get_timeline_by_query
Search for observations using natural language and get timeline context around the best match. This combines search + timeline into a single operation for faster context discovery.
**Auto mode (default):**
```
get_timeline_by_query with query="authentication implementation"
```
Automatically uses the top search result as timeline anchor and returns surrounding context.
**Interactive mode:**
```
get_timeline_by_query with query="authentication" and mode="interactive" and limit=5
```
Shows top 5 search results for you to manually choose which to use as timeline anchor.
**Customize timeline depth:**
```
get_timeline_by_query with query="bug fix" and depth_before=20 and depth_after=10
```
**Use cases:**
- Quick context discovery: "What was happening when we implemented X?"
- Investigate issues: Find a bug fix and see what led to it
- Decision archaeology: Search for a decision and understand the context
- Feature timeline: See the complete story of a feature implementation
**Benefits:**
- Single-step operation (no need to search, then timeline separately)
- Auto mode provides instant context
- Interactive mode gives you control over anchor selection
- Natural language search makes it easy to find relevant moments
## Search Strategy
### 1. Start with Index Format
**Always use index format first** to get an overview:
```
search_observations with query="authentication" and format="index"
```
**Why?**
- Index format uses ~10x fewer tokens than full format
- See titles, dates, and sources to identify relevant results
- Avoid hitting MCP token limits
### 2. Review Results
Look at the index results to identify items of interest:
```
1. [decision] Implement JWT authentication
Date: 2025-10-21 14:23:45
Source: claude-mem://observation/123
2. [feature] Add user authentication endpoints
Date: 2025-10-21 13:15:22
Source: claude-mem://observation/124
3. [bugfix] Fix authentication token expiry
Date: 2025-10-20 16:45:30
Source: claude-mem://observation/125
```
### 3. Deep Dive with Full Format
Only use full format for specific items:
```
search_observations with query="JWT authentication" and format="full" and limit=3
```
### 4. Use Filters to Narrow Results
Combine filters for precise searches:
```
search_observations with query="authentication" and type="decision" and dateRange={start: "2025-10-20", end: "2025-10-21"}
```
## Advanced Filtering
### Date Ranges
Search within specific time periods:
```json
{
"dateRange": {
"start": "2025-10-01",
"end": "2025-10-31"
}
}
```
Or use epoch timestamps:
```json
{
"dateRange": {
"start": 1729449600,
"end": 1732128000
}
}
```
### Multiple Types
Search across multiple observation types:
```
find_by_type with type=["decision", "feature", "refactor"]
```
### Multiple Concepts
Search observations with specific concepts:
```
search_observations with query="database" and concepts=["architecture", "performance"]
```
### File Filtering
Search observations that touched specific files:
```
search_observations with query="refactor" and files="worker-service.ts"
```
### Project Filtering
Search within specific projects:
```
search_observations with query="authentication" and project="my-app"
```
## FTS5 Query Syntax
The `query` parameter supports SQLite FTS5 full-text search syntax:
### Simple Queries
```
"authentication" # Single word
"error handling" # Multiple words (OR)
```
### Boolean Operators
```
"error" AND "handling" # Both terms required
"bug" OR "fix" # Either term
"bug" NOT "feature" # First term, not second
```
### Phrase Searches
```
"'exact phrase'" # Exact phrase match
```
### Column Searches
```
title:"authentication" # Search specific column
narrative:"bug fix" # Search narrative field
```
## Result Metadata
All results include rich metadata:
```
## JWT authentication decision
**Type**: decision
**Date**: 2025-10-21 14:23:45
**Concepts**: authentication, security, architecture
**Files Read**: src/auth/middleware.ts, src/utils/jwt.ts
**Files Modified**: src/auth/jwt-strategy.ts
**Narrative**:
Decided to implement JWT-based authentication instead of session-based
authentication for better scalability and stateless design...
**Facts**:
• JWT tokens expire after 1 hour
• Refresh tokens stored in httpOnly cookies
• Token signing uses RS256 algorithm
• Public keys rotated every 30 days
```
## Citations
All search results include citations using the `claude-mem://` URI scheme:
- `claude-mem://observation/123` - Specific observation
- `claude-mem://session/abc-456` - Specific session
- `claude-mem://user-prompt/789` - Specific user prompt
These citations enable referencing specific historical context in your work.
## Token Management
### Token Efficiency Tips
1. **Start with index format**: ~50-100 tokens per result
2. **Use small limits**: Start with 3-5 results
3. **Apply filters**: Narrow results before searching
4. **Paginate**: Use offset to browse results in batches
### Token Estimates
| Format | Tokens per Result |
|--------|-------------------|
| Index | 50-100 |
| Full | 500-1000 |
**Example**:
- 20 results in index format: ~1,000-2,000 tokens
- 20 results in full format: ~10,000-20,000 tokens
## Common Use Cases
### 1. Debugging Issues
Find what went wrong:
```
search_observations with query="error database connection" and type="bugfix"
```
### 2. Understanding Decisions
Review architectural choices:
```
find_by_type with type="decision" and format="index"
```
Then deep dive on specific decisions:
```
search_observations with query="[DECISION TITLE]" and format="full"
```
### 3. Code Archaeology
Find when a file was modified:
```
find_by_file with filePath="worker-service.ts"
```
### 4. Feature History
Track feature development:
```
search_sessions with query="authentication feature"
search_user_prompts with query="add authentication"
```
### 5. Learning from Past Work
Review refactoring patterns:
```
find_by_type with type="refactor" and limit=10
```
### 6. Context Recovery
Restore context after time away:
```
get_recent_context with limit=5
search_sessions with query="[YOUR PROJECT NAME]" and orderBy="date_desc"
```
## Best Practices
1. **Index first, full later**: Always start with index format
2. **Small limits**: Start with 3-5 results to avoid token limits
3. **Use filters**: Narrow results before searching
4. **Specific queries**: More specific = better results
5. **Review citations**: Use citations to reference past decisions
6. **Date filtering**: Use date ranges for time-based searches
7. **Type filtering**: Use types to categorize searches
8. **Concept tags**: Use concepts for thematic searches
## Troubleshooting
### No Results Found
1. Check database has data:
```bash
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations;"
```
2. Try broader query:
```
search_observations with query="authentication" # Good
vs
search_observations with query="'exact JWT authentication implementation'" # Too specific
```
3. Remove filters:
```
# Start broad
search_observations with query="auth"
# Then add filters
search_observations with query="auth" and type="decision"
```
### Token Limit Errors
1. Use index format:
```
search_observations with query="..." and format="index"
```
2. Reduce limit:
```
search_observations with query="..." and limit=3
```
3. Use pagination:
```
# First page
search_observations with query="..." and limit=5 and offset=0
# Second page
search_observations with query="..." and limit=5 and offset=5
```
### Search Too Slow
1. Use more specific queries
2. Add date range filters
3. Add type/concept filters
4. Reduce result limit
## Next Steps
- [MCP Search Architecture](/architecture/mcp-search) - Technical details
- [Database Schema](/architecture/database) - Understanding the data
- [Getting Started](/usage/getting-started) - Automatic operation
+25 -23
View File
@@ -10,27 +10,29 @@
*/
module.exports = {
apps: [{
name: 'claude-mem-worker',
script: './plugin/scripts/worker-service.cjs',
// INTENTIONAL: Watch mode enables auto-restart on plugin updates
//
// Why this is enabled:
// - When you run `npm run sync-marketplace` or rebuild the plugin,
// files in ~/.claude/plugins/marketplaces/thedotmack/ change
// - Watch mode detects these changes and auto-restarts the worker
// - Users get the latest code without manually running `pm2 restart`
//
// This is a feature, not a bug - it ensures users always run the
// latest version after plugin updates.
watch: true,
ignore_watch: [
'node_modules',
'logs',
'*.log',
'*.db',
'*.db-*',
'.git'
]
}]
apps: [
{
name: 'claude-mem-worker',
script: './plugin/scripts/worker-service.cjs',
// INTENTIONAL: Watch mode enables auto-restart on plugin updates
//
// Why this is enabled:
// - When you run `npm run sync-marketplace` or rebuild the plugin,
// files in ~/.claude/plugins/marketplaces/thedotmack/ change
// - Watch mode detects these changes and auto-restarts the worker
// - Users get the latest code without manually running `pm2 restart`
//
// This is a feature, not a bug - it ensures users always run the
// latest version after plugin updates.
watch: true,
ignore_watch: [
'node_modules',
'logs',
'*.log',
'*.db',
'*.db-*',
'.git'
]
}
]
};
+2 -6
View File
@@ -1,12 +1,12 @@
{
"name": "claude-mem",
"version": "5.1.2",
"version": "5.5.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "claude-mem",
"version": "5.1.2",
"version": "5.5.1",
"license": "AGPL-3.0",
"dependencies": {
"@anthropic-ai/claude-agent-sdk": "^0.1.27",
@@ -1484,7 +1484,6 @@
"integrity": "sha512-RFA/bURkcKzx/X9oumPG9Vp3D3JUgus/d0b67KB0t5S/raciymilkOa66olh78MUI92QLbEJevO7rvqU/kjwKA==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@types/prop-types": "*",
"csstype": "^3.0.2"
@@ -2338,7 +2337,6 @@
"resolved": "https://registry.npmjs.org/express/-/express-4.21.2.tgz",
"integrity": "sha512-28HqgMZAmih1Czt9ny7qr6ek2qddF4FclbMzwhCREB6OFfH+rXAnuNCwo1/wFvrtbgsQDb4kSbX9de9lFbrXnA==",
"license": "MIT",
"peer": true,
"dependencies": {
"accepts": "~1.3.8",
"array-flatten": "1.1.1",
@@ -3851,7 +3849,6 @@
"resolved": "https://registry.npmjs.org/react/-/react-18.3.1.tgz",
"integrity": "sha512-wS+hAgJShR0KhEvPJArfuPVN1+Hz1t0Y6n5jLrGQbkb4urgPE/0Rve+1kMB1v/oWgHgm4WIcV+i7F2pTVj+2iQ==",
"license": "MIT",
"peer": true,
"dependencies": {
"loose-envify": "^1.1.0"
},
@@ -4853,7 +4850,6 @@
"node_modules/zod": {
"version": "3.25.76",
"license": "MIT",
"peer": true,
"funding": {
"url": "https://github.com/sponsors/colinhacks"
}
+2 -1
View File
@@ -1,6 +1,6 @@
{
"name": "claude-mem",
"version": "5.1.3",
"version": "6.0.3",
"description": "Memory compression system for Claude Code - persist context across sessions",
"keywords": [
"claude",
@@ -40,6 +40,7 @@
"worker:stop": "pm2 stop claude-mem-worker",
"worker:restart": "pm2 restart claude-mem-worker",
"worker:logs": "pm2 logs claude-mem-worker",
"changelog:generate": "node scripts/generate-changelog.js",
"usage:analyze": "node scripts/analyze-usage.js",
"usage:today": "node scripts/analyze-usage.js $(date +%Y-%m-%d)"
},
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "claude-mem",
"version": "5.1.3",
"version": "6.0.3",
"description": "Persistent memory system for Claude Code - seamlessly preserve context across sessions",
"author": {
"name": "Alex Newman"
+19 -22
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env node
import{stdin as I}from"process";import w from"better-sqlite3";import{join as E,dirname as k,basename as W}from"path";import{homedir as O}from"os";import{existsSync as K,mkdirSync as x}from"fs";import{fileURLToPath as U}from"url";function M(){return typeof __dirname<"u"?__dirname:k(U(import.meta.url))}var q=M(),u=process.env.CLAUDE_MEM_DATA_DIR||E(O(),".claude-mem"),R=process.env.CLAUDE_CONFIG_DIR||E(O(),".claude"),J=E(u,"archives"),Q=E(u,"logs"),z=E(u,"trash"),Z=E(u,"backups"),ee=E(u,"settings.json"),f=E(u,"claude-mem.db"),se=E(u,"vector-db"),te=E(R,"settings.json"),re=E(R,"commands"),ne=E(R,"CLAUDE.md");function L(c){x(c,{recursive:!0})}var h=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(h||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=h[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=h[e].padEnd(5),d=s.padEnd(6),_="";r?.correlationId?_=`[${r.correlationId}] `:r?.sessionId&&(_=`[session-${r.sessionId}] `);let m="";n!=null&&(this.level===0&&typeof n=="object"?m=`
`+JSON.stringify(n,null,2):m=" "+this.formatData(n));let T="";if(r){let{sessionId:l,sdkSessionId:S,correlationId:p,...a}=r;Object.keys(a).length>0&&(T=` {${Object.entries(a).map(([y,D])=>`${y}=${D}`).join(", ")}}`)}let b=`[${o}] [${i}] [${d}] ${_}${t}${T}${m}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},A=new N;var g=class{db;constructor(){L(u),this.db=new w(f),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
import{stdin as I}from"process";import w from"better-sqlite3";import{join as E,dirname as k,basename as G}from"path";import{homedir as O}from"os";import{existsSync as K,mkdirSync as x}from"fs";import{fileURLToPath as U}from"url";function M(){return typeof __dirname<"u"?__dirname:k(U(import.meta.url))}var q=M(),m=process.env.CLAUDE_MEM_DATA_DIR||E(O(),".claude-mem"),R=process.env.CLAUDE_CONFIG_DIR||E(O(),".claude"),J=E(m,"archives"),Q=E(m,"logs"),z=E(m,"trash"),Z=E(m,"backups"),ee=E(m,"settings.json"),f=E(m,"claude-mem.db"),se=E(m,"vector-db"),te=E(R,"settings.json"),re=E(R,"commands"),ne=E(R,"CLAUDE.md");function L(c){x(c,{recursive:!0})}var h=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(h||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=h[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=h[e].padEnd(5),d=s.padEnd(6),_="";r?.correlationId?_=`[${r.correlationId}] `:r?.sessionId&&(_=`[session-${r.sessionId}] `);let T="";n!=null&&(this.level===0&&typeof n=="object"?T=`
`+JSON.stringify(n,null,2):T=" "+this.formatData(n));let u="";if(r){let{sessionId:l,sdkSessionId:S,correlationId:p,...a}=r;Object.keys(a).length>0&&(u=` {${Object.entries(a).map(([y,D])=>`${y}=${D}`).join(", ")}}`)}let b=`[${o}] [${i}] [${d}] ${_}${t}${u}${T}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},A=new N;var g=class{db;constructor(){L(m),this.db=new w(f),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable(),this.ensureDiscoveryTokensColumn()}initializeSchema(){try{this.db.exec(`
CREATE TABLE IF NOT EXISTS schema_versions (
id INTEGER PRIMARY KEY,
version INTEGER UNIQUE NOT NULL,
@@ -166,7 +166,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}ensureDiscoveryTokensColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;this.db.pragma("table_info(observations)").some(o=>o.name==="discovery_tokens")||(this.db.exec("ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to observations table")),this.db.pragma("table_info(session_summaries)").some(o=>o.name==="discovery_tokens")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString())}catch(e){console.error("[SessionStore] Discovery tokens migration error:",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
SELECT
request, investigated, learned, completed, next_steps,
files_read, files_edited, notes, prompt_number, created_at
@@ -216,6 +216,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
`).all(e)}getAllProjects(){return this.db.prepare(`
SELECT DISTINCT project
FROM sdk_sessions
WHERE project IS NOT NULL AND project != ''
ORDER BY project ASC
`).all().map(t=>t.project)}getRecentSessionsWithStatus(e,s=3){return this.db.prepare(`
SELECT * FROM (
@@ -311,29 +312,29 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
INSERT INTO user_prompts
(claude_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?)
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r,n=0){let o=new Date,i=o.getTime();this.db.prepare(`
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
`).get(e)||(this.db.prepare(`
INSERT INTO sdk_sessions
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
VALUES (?, ?, ?, ?, ?, 'active')
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let m=this.db.prepare(`
`).run(e,e,s,o.toISOString(),i),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
INSERT INTO observations
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
files_read, files_modified, prompt_number, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(m.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n,o.toISOString(),i);return{id:Number(u.lastInsertRowid),createdAtEpoch:i}}storeSummary(e,s,t,r,n=0){let o=new Date,i=o.getTime();this.db.prepare(`
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
`).get(e)||(this.db.prepare(`
INSERT INTO sdk_sessions
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
VALUES (?, ?, ?, ?, ?, 'active')
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let m=this.db.prepare(`
`).run(e,e,s,o.toISOString(),i),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
INSERT INTO session_summaries
(sdk_session_id, project, request, investigated, learned, completed,
next_steps, notes, prompt_number, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(m.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
next_steps, notes, prompt_number, discovery_tokens, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n,o.toISOString(),i);return{id:Number(u.lastInsertRowid),createdAtEpoch:i}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
UPDATE sdk_sessions
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
WHERE id = ?
@@ -341,11 +342,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
UPDATE sdk_sessions
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
WHERE id = ?
`).run(s.toISOString(),t,e)}cleanupOrphanedSessions(){let e=new Date,s=e.getTime();return this.db.prepare(`
UPDATE sdk_sessions
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
WHERE status = 'active'
`).run(e.toISOString(),s).changes}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
SELECT * FROM session_summaries
WHERE id IN (${i})
ORDER BY created_at_epoch ${n}
@@ -384,12 +381,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
WHERE created_at_epoch >= ? ${o}
ORDER BY created_at_epoch ASC
LIMIT ?
`;try{let p=this.db.prepare(l).all(s,...i,t),a=this.db.prepare(S).all(s,...i,r+1);if(p.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=p.length>0?p[p.length-1].created_at_epoch:s,_=a.length>0?a[a.length-1].created_at_epoch:s}catch(p){return console.error("[SessionStore] Error getting boundary timestamps:",p.message),{observations:[],sessions:[],prompts:[]}}}let m=`
`;try{let p=this.db.prepare(l).all(s,...i,t),a=this.db.prepare(S).all(s,...i,r+1);if(p.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=p.length>0?p[p.length-1].created_at_epoch:s,_=a.length>0?a[a.length-1].created_at_epoch:s}catch(p){return console.error("[SessionStore] Error getting boundary timestamps:",p.message),{observations:[],sessions:[],prompts:[]}}}let T=`
SELECT *
FROM observations
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
ORDER BY created_at_epoch ASC
`,T=`
`,u=`
SELECT *
FROM session_summaries
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
@@ -400,5 +397,5 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${o.replace("project","s.project")}
ORDER BY up.created_at_epoch ASC
`;try{let l=this.db.prepare(m).all(d,_,...i),S=this.db.prepare(T).all(d,_,...i),p=this.db.prepare(b).all(d,_,...i);return{observations:l,sessions:S.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:p.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(l){return console.error("[SessionStore] Error querying timeline records:",l.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};import X from"path";import{homedir as F}from"os";import{existsSync as B,readFileSync as H}from"fs";function C(){try{let c=X.join(F(),".claude-mem","settings.json");if(B(c)){let e=JSON.parse(H(c,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function v(c){console.error("[claude-mem cleanup] Hook fired",{input:c?{session_id:c.session_id,cwd:c.cwd,reason:c.reason}:null}),c||(console.log("No input provided - this script is designed to run as a Claude Code SessionEnd hook"),console.log(`
Expected input format:`),console.log(JSON.stringify({session_id:"string",cwd:"string",transcript_path:"string",hook_event_name:"SessionEnd",reason:"exit"},null,2)),process.exit(0));let{session_id:e,reason:s}=c;console.error("[claude-mem cleanup] Searching for active SDK session",{session_id:e,reason:s});let t=new g,r=t.findActiveSDKSession(e);r||(console.error("[claude-mem cleanup] No active SDK session found",{session_id:e}),t.close(),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)),console.error("[claude-mem cleanup] Active SDK session found",{session_id:r.id,sdk_session_id:r.sdk_session_id,project:r.project,worker_port:r.worker_port}),t.markSessionCompleted(r.id),console.error("[claude-mem cleanup] Session marked as completed in database"),t.close();try{let n=r.worker_port||C();await fetch(`http://127.0.0.1:${n}/sessions/${r.id}/complete`,{method:"POST",signal:AbortSignal.timeout(1e3)}),console.error("[claude-mem cleanup] Worker notified to stop processing indicator")}catch(n){console.error("[claude-mem cleanup] Failed to notify worker (non-critical):",n)}console.error("[claude-mem cleanup] Cleanup completed successfully"),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)}if(I.isTTY)v(void 0);else{let c="";I.on("data",e=>c+=e),I.on("end",async()=>{let e=c?JSON.parse(c):void 0;await v(e)})}
`;try{let l=this.db.prepare(T).all(d,_,...i),S=this.db.prepare(u).all(d,_,...i),p=this.db.prepare(b).all(d,_,...i);return{observations:l,sessions:S.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:p.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(l){return console.error("[SessionStore] Error querying timeline records:",l.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};import F from"path";import{homedir as X}from"os";import{existsSync as B,readFileSync as H}from"fs";function v(){try{let c=F.join(X(),".claude-mem","settings.json");if(B(c)){let e=JSON.parse(H(c,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function C(c){console.error("[claude-mem cleanup] Hook fired",{input:c?{session_id:c.session_id,cwd:c.cwd,reason:c.reason}:null}),c||(console.log("No input provided - this script is designed to run as a Claude Code SessionEnd hook"),console.log(`
Expected input format:`),console.log(JSON.stringify({session_id:"string",cwd:"string",transcript_path:"string",hook_event_name:"SessionEnd",reason:"exit"},null,2)),process.exit(0));let{session_id:e,reason:s}=c;console.error("[claude-mem cleanup] Searching for active SDK session",{session_id:e,reason:s});let t=new g,r=t.findActiveSDKSession(e);r||(console.error("[claude-mem cleanup] No active SDK session found",{session_id:e}),t.close(),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)),console.error("[claude-mem cleanup] Active SDK session found",{session_id:r.id,sdk_session_id:r.sdk_session_id,project:r.project,worker_port:r.worker_port}),t.markSessionCompleted(r.id),console.error("[claude-mem cleanup] Session marked as completed in database"),t.close();try{let n=r.worker_port||v();await fetch(`http://127.0.0.1:${n}/sessions/${r.id}/complete`,{method:"POST",signal:AbortSignal.timeout(1e3)}),console.error("[claude-mem cleanup] Worker notified to stop processing indicator")}catch(n){console.error("[claude-mem cleanup] Failed to notify worker (non-critical):",n)}console.error("[claude-mem cleanup] Cleanup completed successfully"),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)}if(I.isTTY)C(void 0);else{let c="";I.on("data",e=>c+=e),I.on("end",async()=>{let e=c?JSON.parse(c):void 0;await C(e)})}
File diff suppressed because one or more lines are too long
+29 -29
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env node
import z from"path";import{stdin as U}from"process";import j from"better-sqlite3";import{join as u,dirname as X,basename as te}from"path";import{homedir as L}from"os";import{existsSync as ie,mkdirSync as F}from"fs";import{fileURLToPath as P}from"url";function H(){return typeof __dirname<"u"?__dirname:X(P(import.meta.url))}var B=H(),m=process.env.CLAUDE_MEM_DATA_DIR||u(L(),".claude-mem"),R=process.env.CLAUDE_CONFIG_DIR||u(L(),".claude"),pe=u(m,"archives"),de=u(m,"logs"),ce=u(m,"trash"),_e=u(m,"backups"),ue=u(m,"settings.json"),A=u(m,"claude-mem.db"),Ee=u(m,"vector-db"),me=u(R,"settings.json"),le=u(R,"commands"),Te=u(R,"CLAUDE.md");function C(a){F(a,{recursive:!0})}function v(){return u(B,"..","..")}var h=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(h||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=h[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=h[e].padEnd(5),d=s.padEnd(6),c="";r?.correlationId?c=`[${r.correlationId}] `:r?.sessionId&&(c=`[session-${r.sessionId}] `);let E="";n!=null&&(this.level===0&&typeof n=="object"?E=`
`+JSON.stringify(n,null,2):E=" "+this.formatData(n));let T="";if(r){let{sessionId:l,sdkSessionId:b,correlationId:_,...p}=r;Object.keys(p).length>0&&(T=` {${Object.entries(p).map(([M,w])=>`${M}=${w}`).join(", ")}}`)}let S=`[${o}] [${i}] [${d}] ${c}${t}${T}${E}`;e===3?console.error(S):console.log(S)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},y=new N;var g=class{db;constructor(){C(m),this.db=new j(A),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
import Y from"path";import{stdin as D}from"process";import X from"better-sqlite3";import{join as m,dirname as U,basename as J}from"path";import{homedir as f}from"os";import{existsSync as ee,mkdirSync as M}from"fs";import{fileURLToPath as w}from"url";function F(){return typeof __dirname<"u"?__dirname:U(w(import.meta.url))}var te=F(),l=process.env.CLAUDE_MEM_DATA_DIR||m(f(),".claude-mem"),h=process.env.CLAUDE_CONFIG_DIR||m(f(),".claude"),re=m(l,"archives"),ne=m(l,"logs"),oe=m(l,"trash"),ie=m(l,"backups"),ae=m(l,"settings.json"),L=m(l,"claude-mem.db"),de=m(l,"vector-db"),pe=m(h,"settings.json"),ce=m(h,"commands"),_e=m(h,"CLAUDE.md");function A(d){M(d,{recursive:!0})}var N=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(N||{}),O=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=N[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=N[e].padEnd(5),p=s.padEnd(6),u="";r?.correlationId?u=`[${r.correlationId}] `:r?.sessionId&&(u=`[session-${r.sessionId}] `);let c="";n!=null&&(this.level===0&&typeof n=="object"?c=`
`+JSON.stringify(n,null,2):c=" "+this.formatData(n));let E="";if(r){let{sessionId:T,sdkSessionId:S,correlationId:_,...a}=r;Object.keys(a).length>0&&(E=` {${Object.entries(a).map(([k,x])=>`${k}=${x}`).join(", ")}}`)}let b=`[${o}] [${i}] [${p}] ${u}${t}${E}${c}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},v=new O;var R=class{db;constructor(){A(l),this.db=new X(L),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable(),this.ensureDiscoveryTokensColumn()}initializeSchema(){try{this.db.exec(`
CREATE TABLE IF NOT EXISTS schema_versions (
id INTEGER PRIMARY KEY,
version INTEGER UNIQUE NOT NULL,
@@ -63,7 +63,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
CREATE INDEX IF NOT EXISTS idx_session_summaries_sdk_session ON session_summaries(sdk_session_id);
CREATE INDEX IF NOT EXISTS idx_session_summaries_project ON session_summaries(project);
CREATE INDEX IF NOT EXISTS idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
`),this.db.prepare("INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)").run(4,new Date().toISOString()),console.error("[SessionStore] Migration004 applied successfully"))}catch(e){throw console.error("[SessionStore] Schema initialization error:",e.message),e}}ensureWorkerPortColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(5))return;this.db.pragma("table_info(sdk_sessions)").some(r=>r.name==="worker_port")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN worker_port INTEGER"),console.error("[SessionStore] Added worker_port column to sdk_sessions table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(5,new Date().toISOString())}catch(e){console.error("[SessionStore] Migration error:",e.message)}}ensurePromptTrackingColumns(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(6))return;this.db.pragma("table_info(sdk_sessions)").some(d=>d.name==="prompt_counter")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN prompt_counter INTEGER DEFAULT 0"),console.error("[SessionStore] Added prompt_counter column to sdk_sessions table")),this.db.pragma("table_info(observations)").some(d=>d.name==="prompt_number")||(this.db.exec("ALTER TABLE observations ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to observations table")),this.db.pragma("table_info(session_summaries)").some(d=>d.name==="prompt_number")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(6,new Date().toISOString())}catch(e){console.error("[SessionStore] Prompt tracking migration error:",e.message)}}removeSessionSummariesUniqueConstraint(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;if(!this.db.pragma("index_list(session_summaries)").some(r=>r.unique===1)){this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString());return}console.error("[SessionStore] Removing UNIQUE constraint from session_summaries.sdk_session_id..."),this.db.exec("BEGIN TRANSACTION");try{this.db.exec(`
`),this.db.prepare("INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)").run(4,new Date().toISOString()),console.error("[SessionStore] Migration004 applied successfully"))}catch(e){throw console.error("[SessionStore] Schema initialization error:",e.message),e}}ensureWorkerPortColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(5))return;this.db.pragma("table_info(sdk_sessions)").some(r=>r.name==="worker_port")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN worker_port INTEGER"),console.error("[SessionStore] Added worker_port column to sdk_sessions table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(5,new Date().toISOString())}catch(e){console.error("[SessionStore] Migration error:",e.message)}}ensurePromptTrackingColumns(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(6))return;this.db.pragma("table_info(sdk_sessions)").some(p=>p.name==="prompt_counter")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN prompt_counter INTEGER DEFAULT 0"),console.error("[SessionStore] Added prompt_counter column to sdk_sessions table")),this.db.pragma("table_info(observations)").some(p=>p.name==="prompt_number")||(this.db.exec("ALTER TABLE observations ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to observations table")),this.db.pragma("table_info(session_summaries)").some(p=>p.name==="prompt_number")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(6,new Date().toISOString())}catch(e){console.error("[SessionStore] Prompt tracking migration error:",e.message)}}removeSessionSummariesUniqueConstraint(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;if(!this.db.pragma("index_list(session_summaries)").some(r=>r.unique===1)){this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString());return}console.error("[SessionStore] Removing UNIQUE constraint from session_summaries.sdk_session_id..."),this.db.exec("BEGIN TRANSACTION");try{this.db.exec(`
CREATE TABLE session_summaries_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
sdk_session_id TEXT NOT NULL,
@@ -166,7 +166,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}ensureDiscoveryTokensColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;this.db.pragma("table_info(observations)").some(o=>o.name==="discovery_tokens")||(this.db.exec("ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to observations table")),this.db.pragma("table_info(session_summaries)").some(o=>o.name==="discovery_tokens")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString())}catch(e){console.error("[SessionStore] Discovery tokens migration error:",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
SELECT
request, investigated, learned, completed, next_steps,
files_read, files_edited, notes, prompt_number, created_at
@@ -216,6 +216,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
`).all(e)}getAllProjects(){return this.db.prepare(`
SELECT DISTINCT project
FROM sdk_sessions
WHERE project IS NOT NULL AND project != ''
ORDER BY project ASC
`).all().map(t=>t.project)}getRecentSessionsWithStatus(e,s=3){return this.db.prepare(`
SELECT * FROM (
@@ -261,7 +262,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
SELECT files_read, files_modified
FROM observations
WHERE sdk_session_id = ?
`).all(e),r=new Set,n=new Set;for(let o of t){if(o.files_read)try{let i=JSON.parse(o.files_read);Array.isArray(i)&&i.forEach(d=>r.add(d))}catch{}if(o.files_modified)try{let i=JSON.parse(o.files_modified);Array.isArray(i)&&i.forEach(d=>n.add(d))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
`).all(e),r=new Set,n=new Set;for(let o of t){if(o.files_read)try{let i=JSON.parse(o.files_read);Array.isArray(i)&&i.forEach(p=>r.add(p))}catch{}if(o.files_modified)try{let i=JSON.parse(o.files_modified);Array.isArray(i)&&i.forEach(p=>n.add(p))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
SELECT id, claude_session_id, sdk_session_id, project, user_prompt
FROM sdk_sessions
WHERE id = ?
@@ -298,7 +299,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
UPDATE sdk_sessions
SET sdk_session_id = ?
WHERE id = ? AND sdk_session_id IS NULL
`).run(s,e).changes===0?(y.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
`).run(s,e).changes===0?(v.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
UPDATE sdk_sessions
SET worker_port = ?
WHERE id = ?
@@ -311,29 +312,29 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
INSERT INTO user_prompts
(claude_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?)
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r,n=0){let o=new Date,i=o.getTime();this.db.prepare(`
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
`).get(e)||(this.db.prepare(`
INSERT INTO sdk_sessions
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
VALUES (?, ?, ?, ?, ?, 'active')
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
`).run(e,e,s,o.toISOString(),i),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
INSERT INTO observations
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
files_read, files_modified, prompt_number, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n,o.toISOString(),i);return{id:Number(E.lastInsertRowid),createdAtEpoch:i}}storeSummary(e,s,t,r,n=0){let o=new Date,i=o.getTime();this.db.prepare(`
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
`).get(e)||(this.db.prepare(`
INSERT INTO sdk_sessions
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
VALUES (?, ?, ?, ?, ?, 'active')
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
`).run(e,e,s,o.toISOString(),i),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
INSERT INTO session_summaries
(sdk_session_id, project, request, investigated, learned, completed,
next_steps, notes, prompt_number, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
next_steps, notes, prompt_number, discovery_tokens, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n,o.toISOString(),i);return{id:Number(E.lastInsertRowid),createdAtEpoch:i}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
UPDATE sdk_sessions
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
WHERE id = ?
@@ -341,11 +342,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
UPDATE sdk_sessions
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
WHERE id = ?
`).run(s.toISOString(),t,e)}cleanupOrphanedSessions(){let e=new Date,s=e.getTime();return this.db.prepare(`
UPDATE sdk_sessions
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
WHERE status = 'active'
`).run(e.toISOString(),s).changes}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
SELECT * FROM session_summaries
WHERE id IN (${i})
ORDER BY created_at_epoch ${n}
@@ -360,44 +357,47 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
WHERE up.id IN (${i})
ORDER BY up.created_at_epoch ${n}
${o}
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],d,c;if(e!==null){let l=`
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],p,u;if(e!==null){let T=`
SELECT id, created_at_epoch
FROM observations
WHERE id <= ? ${o}
ORDER BY id DESC
LIMIT ?
`,b=`
`,S=`
SELECT id, created_at_epoch
FROM observations
WHERE id >= ? ${o}
ORDER BY id ASC
LIMIT ?
`;try{let _=this.db.prepare(l).all(e,...i,t+1),p=this.db.prepare(b).all(e,...i,r+1);if(_.length===0&&p.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,c=p.length>0?p[p.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let l=`
`;try{let _=this.db.prepare(T).all(e,...i,t+1),a=this.db.prepare(S).all(e,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
SELECT created_at_epoch
FROM observations
WHERE created_at_epoch <= ? ${o}
ORDER BY created_at_epoch DESC
LIMIT ?
`,b=`
`,S=`
SELECT created_at_epoch
FROM observations
WHERE created_at_epoch >= ? ${o}
ORDER BY created_at_epoch ASC
LIMIT ?
`;try{let _=this.db.prepare(l).all(s,...i,t),p=this.db.prepare(b).all(s,...i,r+1);if(_.length===0&&p.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,c=p.length>0?p[p.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let E=`
`;try{let _=this.db.prepare(T).all(s,...i,t),a=this.db.prepare(S).all(s,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let c=`
SELECT *
FROM observations
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
ORDER BY created_at_epoch ASC
`,T=`
`,E=`
SELECT *
FROM session_summaries
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
ORDER BY created_at_epoch ASC
`,S=`
`,b=`
SELECT up.*, s.project, s.sdk_session_id
FROM user_prompts up
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${o.replace("project","s.project")}
ORDER BY up.created_at_epoch ASC
`;try{let l=this.db.prepare(E).all(d,c,...i),b=this.db.prepare(T).all(d,c,...i),_=this.db.prepare(S).all(d,c,...i);return{observations:l,sessions:b.map(p=>({id:p.id,sdk_session_id:p.sdk_session_id,project:p.project,request:p.request,completed:p.completed,next_steps:p.next_steps,created_at:p.created_at,created_at_epoch:p.created_at_epoch})),prompts:_.map(p=>({id:p.id,claude_session_id:p.claude_session_id,project:p.project,prompt:p.prompt_text,created_at:p.created_at,created_at_epoch:p.created_at_epoch}))}}catch(l){return console.error("[SessionStore] Error querying timeline records:",l.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function $(a,e,s){return a==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:a==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:a==="UserPromptSubmit"||a==="PostToolUse"?{continue:!0,suppressOutput:!0}:a==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function D(a,e,s={}){let t=$(a,e,s);return JSON.stringify(t)}import O from"path";import{homedir as W}from"os";import{existsSync as G,readFileSync as Y}from"fs";import{execSync as K}from"child_process";var V=100,q=100,J=1e4;function f(){try{let a=O.join(W(),".claude-mem","settings.json");if(G(a)){let e=JSON.parse(Y(a,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function k(){try{let a=f();return(await fetch(`http://127.0.0.1:${a}/health`,{signal:AbortSignal.timeout(V)})).ok}catch{return!1}}async function Q(){let a=Date.now();for(;Date.now()-a<J;){if(await k())return!0;await new Promise(e=>setTimeout(e,q))}return!1}async function x(){if(await k())return;let a=v(),e=O.join(a,"node_modules",".bin","pm2"),s=O.join(a,"ecosystem.config.cjs");if(K(`"${e}" restart "${s}"`,{cwd:a,stdio:"pipe"}),!await Q())throw new Error("Worker failed to become healthy after restart")}async function Z(a){if(!a)throw new Error("newHook requires input");let{session_id:e,cwd:s,prompt:t}=a,r=z.basename(s);await x();let n=new g,o=n.createSDKSession(e,r,t),i=n.incrementPromptCounter(o);n.saveUserPrompt(e,i,t),console.error(`[new-hook] Session ${o}, prompt #${i}`),n.close();let d=f();try{let c=await fetch(`http://127.0.0.1:${d}/sessions/${o}/init`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({project:r,userPrompt:t}),signal:AbortSignal.timeout(5e3)});if(!c.ok){let E=await c.text();throw new Error(`Failed to initialize session: ${c.status} ${E}`)}}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(D("UserPromptSubmit",!0))}var I="";U.on("data",a=>I+=a);U.on("end",async()=>{let a=I?JSON.parse(I):void 0;await Z(a)});
`;try{let T=this.db.prepare(c).all(p,u,...i),S=this.db.prepare(E).all(p,u,...i),_=this.db.prepare(b).all(p,u,...i);return{observations:T,sessions:S.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:_.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function H(d,e,s){return d==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:d==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:d==="UserPromptSubmit"||d==="PostToolUse"?{continue:!0,suppressOutput:!0}:d==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function C(d,e,s={}){let t=H(d,e,s);return JSON.stringify(t)}import P from"path";import{homedir as B}from"os";import{existsSync as j,readFileSync as $}from"fs";var G=100;function g(){try{let d=P.join(B(),".claude-mem","settings.json");if(j(d)){let e=JSON.parse($(d,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function W(){try{let d=g();return(await fetch(`http://127.0.0.1:${d}/health`,{signal:AbortSignal.timeout(G)})).ok}catch{return!1}}async function y(){if(await W())return;let d=g();throw new Error(`Worker service is not responding on port ${d}.
If you just updated the plugin, PM2's watch mode should restart automatically.
If the problem persists, run: pm2 restart claude-mem-worker`)}async function K(d){if(!d)throw new Error("newHook requires input");let{session_id:e,cwd:s,prompt:t}=d,r=Y.basename(s);await y();let n=new R,o=n.createSDKSession(e,r,t),i=n.incrementPromptCounter(o);n.saveUserPrompt(e,i,t),console.error(`[new-hook] Session ${o}, prompt #${i}`),n.close();let p=g(),u=t.startsWith("/")?t.substring(1):t;try{let c=await fetch(`http://127.0.0.1:${p}/sessions/${o}/init`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({project:r,userPrompt:u,promptNumber:i}),signal:AbortSignal.timeout(5e3)});if(!c.ok){let E=await c.text();throw new Error(`Failed to initialize session: ${c.status} ${E}`)}}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(C("UserPromptSubmit",!0))}var I="";D.on("data",d=>I+=d);D.on("end",async()=>{let d=I?JSON.parse(I):void 0;await K(d)});
+50 -50
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env node
import{stdin as U}from"process";import j from"better-sqlite3";import{join as E,dirname as F,basename as te}from"path";import{homedir as C}from"os";import{existsSync as ie,mkdirSync as X}from"fs";import{fileURLToPath as H}from"url";function P(){return typeof __dirname<"u"?__dirname:F(H(import.meta.url))}var B=P(),l=process.env.CLAUDE_MEM_DATA_DIR||E(C(),".claude-mem"),h=process.env.CLAUDE_CONFIG_DIR||E(C(),".claude"),de=E(l,"archives"),pe=E(l,"logs"),ce=E(l,"trash"),_e=E(l,"backups"),ue=E(l,"settings.json"),v=E(l,"claude-mem.db"),Ee=E(l,"vector-db"),me=E(h,"settings.json"),le=E(h,"commands"),Te=E(h,"CLAUDE.md");function y(a){X(a,{recursive:!0})}function D(){return E(B,"..","..")}var N=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(N||{}),O=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=N[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=N[e].padEnd(5),p=s.padEnd(6),u="";r?.correlationId?u=`[${r.correlationId}] `:r?.sessionId&&(u=`[session-${r.sessionId}] `);let c="";n!=null&&(this.level===0&&typeof n=="object"?c=`
`+JSON.stringify(n,null,2):c=" "+this.formatData(n));let m="";if(r){let{sessionId:T,sdkSessionId:S,correlationId:_,...d}=r;Object.keys(d).length>0&&(m=` {${Object.entries(d).map(([M,w])=>`${M}=${w}`).join(", ")}}`)}let g=`[${o}] [${i}] [${p}] ${u}${t}${m}${c}`;e===3?console.error(g):console.log(g)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},b=new O;var R=class{db;constructor(){y(l),this.db=new j(v),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
import{stdin as D}from"process";import X from"better-sqlite3";import{join as m,dirname as U,basename as J}from"path";import{homedir as A}from"os";import{existsSync as ee,mkdirSync as M}from"fs";import{fileURLToPath as w}from"url";function F(){return typeof __dirname<"u"?__dirname:U(w(import.meta.url))}var te=F(),T=process.env.CLAUDE_MEM_DATA_DIR||m(A(),".claude-mem"),N=process.env.CLAUDE_CONFIG_DIR||m(A(),".claude"),re=m(T,"archives"),oe=m(T,"logs"),ne=m(T,"trash"),ie=m(T,"backups"),ae=m(T,"settings.json"),v=m(T,"claude-mem.db"),de=m(T,"vector-db"),pe=m(N,"settings.json"),ce=m(N,"commands"),_e=m(N,"CLAUDE.md");function C(d){M(d,{recursive:!0})}var O=(o=>(o[o.DEBUG=0]="DEBUG",o[o.INFO=1]="INFO",o[o.WARN=2]="WARN",o[o.ERROR=3]="ERROR",o[o.SILENT=4]="SILENT",o))(O||{}),I=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=O[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,o){if(e<this.level)return;let i=new Date().toISOString().replace("T"," ").substring(0,23),n=O[e].padEnd(5),p=s.padEnd(6),u="";r?.correlationId?u=`[${r.correlationId}] `:r?.sessionId&&(u=`[session-${r.sessionId}] `);let E="";o!=null&&(this.level===0&&typeof o=="object"?E=`
`+JSON.stringify(o,null,2):E=" "+this.formatData(o));let c="";if(r){let{sessionId:b,sdkSessionId:g,correlationId:_,...a}=r;Object.keys(a).length>0&&(c=` {${Object.entries(a).map(([k,x])=>`${k}=${x}`).join(", ")}}`)}let l=`[${i}] [${n}] [${p}] ${u}${t}${c}${E}`;e===3?console.error(l):console.log(l)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},S=new I;var R=class{db;constructor(){C(T),this.db=new X(v),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable(),this.ensureDiscoveryTokensColumn()}initializeSchema(){try{this.db.exec(`
CREATE TABLE IF NOT EXISTS schema_versions (
id INTEGER PRIMARY KEY,
version INTEGER UNIQUE NOT NULL,
@@ -166,7 +166,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}ensureDiscoveryTokensColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;this.db.pragma("table_info(observations)").some(i=>i.name==="discovery_tokens")||(this.db.exec("ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to observations table")),this.db.pragma("table_info(session_summaries)").some(i=>i.name==="discovery_tokens")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString())}catch(e){console.error("[SessionStore] Discovery tokens migration error:",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
SELECT
request, investigated, learned, completed, next_steps,
files_read, files_edited, notes, prompt_number, created_at
@@ -216,6 +216,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
`).all(e)}getAllProjects(){return this.db.prepare(`
SELECT DISTINCT project
FROM sdk_sessions
WHERE project IS NOT NULL AND project != ''
ORDER BY project ASC
`).all().map(t=>t.project)}getRecentSessionsWithStatus(e,s=3){return this.db.prepare(`
SELECT * FROM (
@@ -243,12 +244,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
SELECT *
FROM observations
WHERE id = ?
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,o=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",n=e.map(()=>"?").join(",");return this.db.prepare(`
SELECT *
FROM observations
WHERE id IN (${i})
ORDER BY created_at_epoch ${n}
${o}
WHERE id IN (${n})
ORDER BY created_at_epoch ${o}
${i}
`).all(...e)}getSummaryForSession(e){return this.db.prepare(`
SELECT
request, investigated, learned, completed, next_steps,
@@ -261,7 +262,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
SELECT files_read, files_modified
FROM observations
WHERE sdk_session_id = ?
`).all(e),r=new Set,n=new Set;for(let o of t){if(o.files_read)try{let i=JSON.parse(o.files_read);Array.isArray(i)&&i.forEach(p=>r.add(p))}catch{}if(o.files_modified)try{let i=JSON.parse(o.files_modified);Array.isArray(i)&&i.forEach(p=>n.add(p))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
`).all(e),r=new Set,o=new Set;for(let i of t){if(i.files_read)try{let n=JSON.parse(i.files_read);Array.isArray(n)&&n.forEach(p=>r.add(p))}catch{}if(i.files_modified)try{let n=JSON.parse(i.files_modified);Array.isArray(n)&&n.forEach(p=>o.add(p))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(o)}}getSessionById(e){return this.db.prepare(`
SELECT id, claude_session_id, sdk_session_id, project, user_prompt
FROM sdk_sessions
WHERE id = ?
@@ -288,17 +289,17 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
`).get(e)?.prompt_counter||1}getPromptCounter(e){return this.db.prepare(`
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
`).get(e)?.prompt_counter||0}createSDKSession(e,s,t){let r=new Date,n=r.getTime(),i=this.db.prepare(`
`).get(e)?.prompt_counter||0}createSDKSession(e,s,t){let r=new Date,o=r.getTime(),n=this.db.prepare(`
INSERT OR IGNORE INTO sdk_sessions
(claude_session_id, sdk_session_id, project, user_prompt, started_at, started_at_epoch, status)
VALUES (?, ?, ?, ?, ?, ?, 'active')
`).run(e,e,s,t,r.toISOString(),n);return i.lastInsertRowid===0||i.changes===0?this.db.prepare(`
`).run(e,e,s,t,r.toISOString(),o);return n.lastInsertRowid===0||n.changes===0?this.db.prepare(`
SELECT id FROM sdk_sessions WHERE claude_session_id = ? LIMIT 1
`).get(e).id:i.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
`).get(e).id:n.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
UPDATE sdk_sessions
SET sdk_session_id = ?
WHERE id = ? AND sdk_session_id IS NULL
`).run(s,e).changes===0?(b.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
`).run(s,e).changes===0?(S.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
UPDATE sdk_sessions
SET worker_port = ?
WHERE id = ?
@@ -307,33 +308,33 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
FROM sdk_sessions
WHERE id = ?
LIMIT 1
`).get(e)?.worker_port||null}saveUserPrompt(e,s,t){let r=new Date,n=r.getTime();return this.db.prepare(`
`).get(e)?.worker_port||null}saveUserPrompt(e,s,t){let r=new Date,o=r.getTime();return this.db.prepare(`
INSERT INTO user_prompts
(claude_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?)
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
`).run(e,s,t,r.toISOString(),o).lastInsertRowid}storeObservation(e,s,t,r,o=0){let i=new Date,n=i.getTime();this.db.prepare(`
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
`).get(e)||(this.db.prepare(`
INSERT INTO sdk_sessions
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
VALUES (?, ?, ?, ?, ?, 'active')
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
`).run(e,e,s,i.toISOString(),n),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
INSERT INTO observations
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
files_read, files_modified, prompt_number, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(c.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,o,i.toISOString(),n);return{id:Number(c.lastInsertRowid),createdAtEpoch:n}}storeSummary(e,s,t,r,o=0){let i=new Date,n=i.getTime();this.db.prepare(`
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
`).get(e)||(this.db.prepare(`
INSERT INTO sdk_sessions
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
VALUES (?, ?, ?, ?, ?, 'active')
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
`).run(e,e,s,i.toISOString(),n),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
INSERT INTO session_summaries
(sdk_session_id, project, request, investigated, learned, completed,
next_steps, notes, prompt_number, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(c.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
next_steps, notes, prompt_number, discovery_tokens, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,o,i.toISOString(),n);return{id:Number(c.lastInsertRowid),createdAtEpoch:n}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
UPDATE sdk_sessions
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
WHERE id = ?
@@ -341,63 +342,62 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
UPDATE sdk_sessions
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
WHERE id = ?
`).run(s.toISOString(),t,e)}cleanupOrphanedSessions(){let e=new Date,s=e.getTime();return this.db.prepare(`
UPDATE sdk_sessions
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
WHERE status = 'active'
`).run(e.toISOString(),s).changes}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,o=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",n=e.map(()=>"?").join(",");return this.db.prepare(`
SELECT * FROM session_summaries
WHERE id IN (${i})
ORDER BY created_at_epoch ${n}
${o}
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
WHERE id IN (${n})
ORDER BY created_at_epoch ${o}
${i}
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,o=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",n=e.map(()=>"?").join(",");return this.db.prepare(`
SELECT
up.*,
s.project,
s.sdk_session_id
FROM user_prompts up
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
WHERE up.id IN (${i})
ORDER BY up.created_at_epoch ${n}
${o}
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],p,u;if(e!==null){let T=`
WHERE up.id IN (${n})
ORDER BY up.created_at_epoch ${o}
${i}
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,o){let i=o?"AND project = ?":"",n=o?[o]:[],p,u;if(e!==null){let b=`
SELECT id, created_at_epoch
FROM observations
WHERE id <= ? ${o}
WHERE id <= ? ${i}
ORDER BY id DESC
LIMIT ?
`,S=`
`,g=`
SELECT id, created_at_epoch
FROM observations
WHERE id >= ? ${o}
WHERE id >= ? ${i}
ORDER BY id ASC
LIMIT ?
`;try{let _=this.db.prepare(T).all(e,...i,t+1),d=this.db.prepare(S).all(e,...i,r+1);if(_.length===0&&d.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=d.length>0?d[d.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
`;try{let _=this.db.prepare(b).all(e,...n,t+1),a=this.db.prepare(g).all(e,...n,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let b=`
SELECT created_at_epoch
FROM observations
WHERE created_at_epoch <= ? ${o}
WHERE created_at_epoch <= ? ${i}
ORDER BY created_at_epoch DESC
LIMIT ?
`,S=`
`,g=`
SELECT created_at_epoch
FROM observations
WHERE created_at_epoch >= ? ${o}
WHERE created_at_epoch >= ? ${i}
ORDER BY created_at_epoch ASC
LIMIT ?
`;try{let _=this.db.prepare(T).all(s,...i,t),d=this.db.prepare(S).all(s,...i,r+1);if(_.length===0&&d.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=d.length>0?d[d.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let c=`
`;try{let _=this.db.prepare(b).all(s,...n,t),a=this.db.prepare(g).all(s,...n,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let E=`
SELECT *
FROM observations
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${i}
ORDER BY created_at_epoch ASC
`,m=`
`,c=`
SELECT *
FROM session_summaries
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${i}
ORDER BY created_at_epoch ASC
`,g=`
`,l=`
SELECT up.*, s.project, s.sdk_session_id
FROM user_prompts up
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${o.replace("project","s.project")}
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${i.replace("project","s.project")}
ORDER BY up.created_at_epoch ASC
`;try{let T=this.db.prepare(c).all(p,u,...i),S=this.db.prepare(m).all(p,u,...i),_=this.db.prepare(g).all(p,u,...i);return{observations:T,sessions:S.map(d=>({id:d.id,sdk_session_id:d.sdk_session_id,project:d.project,request:d.request,completed:d.completed,next_steps:d.next_steps,created_at:d.created_at,created_at_epoch:d.created_at_epoch})),prompts:_.map(d=>({id:d.id,claude_session_id:d.claude_session_id,project:d.project,prompt:d.prompt_text,created_at:d.created_at,created_at_epoch:d.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function $(a,e,s){return a==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:a==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:a==="UserPromptSubmit"||a==="PostToolUse"?{continue:!0,suppressOutput:!0}:a==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function f(a,e,s={}){let t=$(a,e,s);return JSON.stringify(t)}import I from"path";import{homedir as W}from"os";import{existsSync as G,readFileSync as Y}from"fs";import{execSync as K}from"child_process";var V=100,q=100,J=1e4;function L(){try{let a=I.join(W(),".claude-mem","settings.json");if(G(a)){let e=JSON.parse(Y(a,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function k(){try{let a=L();return(await fetch(`http://127.0.0.1:${a}/health`,{signal:AbortSignal.timeout(V)})).ok}catch{return!1}}async function Q(){let a=Date.now();for(;Date.now()-a<J;){if(await k())return!0;await new Promise(e=>setTimeout(e,q))}return!1}async function x(){if(await k())return;let a=D(),e=I.join(a,"node_modules",".bin","pm2"),s=I.join(a,"ecosystem.config.cjs");if(K(`"${e}" restart "${s}"`,{cwd:a,stdio:"pipe"}),!await Q())throw new Error("Worker failed to become healthy after restart")}var z=new Set(["ListMcpResourcesTool"]);async function Z(a){if(!a)throw new Error("saveHook requires input");let{session_id:e,tool_name:s,tool_input:t,tool_response:r}=a;if(z.has(s)){console.log(f("PostToolUse",!0));return}await x();let n=new R,o=n.createSDKSession(e,"",""),i=n.getPromptCounter(o);n.close();let p=b.formatTool(s,t),u=L();b.dataIn("HOOK",`PostToolUse: ${p}`,{sessionId:o,workerPort:u});try{let c=await fetch(`http://127.0.0.1:${u}/sessions/${o}/observations`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({tool_name:s,tool_input:t!==void 0?JSON.stringify(t):"{}",tool_response:r!==void 0?JSON.stringify(r):"{}",prompt_number:i}),signal:AbortSignal.timeout(2e3)});if(!c.ok){let m=await c.text();throw b.failure("HOOK","Failed to send observation",{sessionId:o,status:c.status},m),new Error(`Failed to send observation to worker: ${c.status} ${m}`)}b.debug("HOOK","Observation sent successfully",{sessionId:o,toolName:s})}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(f("PostToolUse",!0))}var A="";U.on("data",a=>A+=a);U.on("end",async()=>{let a=A?JSON.parse(A):void 0;await Z(a)});
`;try{let b=this.db.prepare(E).all(p,u,...n),g=this.db.prepare(c).all(p,u,...n),_=this.db.prepare(l).all(p,u,...n);return{observations:b,sessions:g.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:_.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(b){return console.error("[SessionStore] Error querying timeline records:",b.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function H(d,e,s){return d==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:d==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:d==="UserPromptSubmit"||d==="PostToolUse"?{continue:!0,suppressOutput:!0}:d==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function f(d,e,s={}){let t=H(d,e,s);return JSON.stringify(t)}import P from"path";import{homedir as B}from"os";import{existsSync as j,readFileSync as G}from"fs";var W=100;function h(){try{let d=P.join(B(),".claude-mem","settings.json");if(j(d)){let e=JSON.parse(G(d,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function $(){try{let d=h();return(await fetch(`http://127.0.0.1:${d}/health`,{signal:AbortSignal.timeout(W)})).ok}catch{return!1}}async function y(){if(await $())return;let d=h();throw new Error(`Worker service is not responding on port ${d}.
If you just updated the plugin, PM2's watch mode should restart automatically.
If the problem persists, run: pm2 restart claude-mem-worker`)}var Y=new Set(["ListMcpResourcesTool","SlashCommand","Skill","TodoWrite","AskUserQuestion"]);async function K(d){if(!d)throw new Error("saveHook requires input");let{session_id:e,cwd:s,tool_name:t,tool_input:r,tool_response:o}=d;if(Y.has(t)){console.log(f("PostToolUse",!0));return}await y();let i=new R,n=i.createSDKSession(e,"",""),p=i.getPromptCounter(n);i.close();let u=S.formatTool(t,r),E=h();S.dataIn("HOOK",`PostToolUse: ${u}`,{sessionId:n,workerPort:E});try{let c=await fetch(`http://127.0.0.1:${E}/sessions/${n}/observations`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({tool_name:t,tool_input:r!==void 0?JSON.stringify(r):"{}",tool_response:o!==void 0?JSON.stringify(o):"{}",prompt_number:p,cwd:s||""}),signal:AbortSignal.timeout(2e3)});if(!c.ok){let l=await c.text();throw S.failure("HOOK","Failed to send observation",{sessionId:n,status:c.status},l),new Error(`Failed to send observation to worker: ${c.status} ${l}`)}S.debug("HOOK","Observation sent successfully",{sessionId:n,toolName:t})}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(f("PostToolUse",!0))}var L="";D.on("data",d=>L+=d);D.on("end",async()=>{let d=L?JSON.parse(L):void 0;await K(d)});
+2 -5
View File
@@ -352,6 +352,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
`).all(e)}getAllProjects(){return this.db.prepare(`
SELECT DISTINCT project
FROM sdk_sessions
WHERE project IS NOT NULL AND project != ''
ORDER BY project ASC
`).all().map(r=>r.project)}getRecentSessionsWithStatus(e,s=3){return this.db.prepare(`
SELECT * FROM (
@@ -477,11 +478,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
UPDATE sdk_sessions
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
WHERE id = ?
`).run(s.toISOString(),r,e)}cleanupOrphanedSessions(){let e=new Date,s=e.getTime();return this.db.prepare(`
UPDATE sdk_sessions
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
WHERE status = 'active'
`).run(e.toISOString(),s).changes}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:r="date_desc",limit:t}=s,n=r==="date_asc"?"ASC":"DESC",o=t?`LIMIT ${t}`:"",a=e.map(()=>"?").join(",");return this.db.prepare(`
`).run(s.toISOString(),r,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:r="date_desc",limit:t}=s,n=r==="date_asc"?"ASC":"DESC",o=t?`LIMIT ${t}`:"",a=e.map(()=>"?").join(",");return this.db.prepare(`
SELECT * FROM session_summaries
WHERE id IN (${a})
ORDER BY created_at_epoch ${n}
+62 -47
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env node
import{stdin as U}from"process";import j from"better-sqlite3";import{join as _,dirname as F,basename as se}from"path";import{homedir as A}from"os";import{existsSync as oe,mkdirSync as X}from"fs";import{fileURLToPath as H}from"url";function P(){return typeof __dirname<"u"?__dirname:F(H(import.meta.url))}var B=P(),E=process.env.CLAUDE_MEM_DATA_DIR||_(A(),".claude-mem"),h=process.env.CLAUDE_CONFIG_DIR||_(A(),".claude"),ae=_(E,"archives"),de=_(E,"logs"),pe=_(E,"trash"),ce=_(E,"backups"),_e=_(E,"settings.json"),C=_(E,"claude-mem.db"),ue=_(E,"vector-db"),me=_(h,"settings.json"),Ee=_(h,"commands"),le=_(h,"CLAUDE.md");function y(a){X(a,{recursive:!0})}function v(){return _(B,"..","..")}var N=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(N||{}),O=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=N[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=N[e].padEnd(5),p=s.padEnd(6),u="";r?.correlationId?u=`[${r.correlationId}] `:r?.sessionId&&(u=`[session-${r.sessionId}] `);let m="";n!=null&&(this.level===0&&typeof n=="object"?m=`
`+JSON.stringify(n,null,2):m=" "+this.formatData(n));let T="";if(r){let{sessionId:l,sdkSessionId:b,correlationId:c,...d}=r;Object.keys(d).length>0&&(T=` {${Object.entries(d).map(([M,w])=>`${M}=${w}`).join(", ")}}`)}let g=`[${o}] [${i}] [${p}] ${u}${t}${T}${m}`;e===3?console.error(g):console.log(g)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},S=new O;var R=class{db;constructor(){y(E),this.db=new j(C),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
import{stdin as k}from"process";import{readFileSync as x,existsSync as U}from"fs";import j from"better-sqlite3";import{join as m,dirname as F,basename as ne}from"path";import{homedir as y}from"os";import{existsSync as de,mkdirSync as X}from"fs";import{fileURLToPath as H}from"url";function B(){return typeof __dirname<"u"?__dirname:F(H(import.meta.url))}var ce=B(),l=process.env.CLAUDE_MEM_DATA_DIR||m(y(),".claude-mem"),f=process.env.CLAUDE_CONFIG_DIR||m(y(),".claude"),ue=m(l,"archives"),_e=m(l,"logs"),me=m(l,"trash"),Ee=m(l,"backups"),le=m(l,"settings.json"),A=m(l,"claude-mem.db"),Te=m(l,"vector-db"),ge=m(f,"settings.json"),be=m(f,"commands"),Se=m(f,"CLAUDE.md");function v(a){X(a,{recursive:!0})}var O=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(O||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=O[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let i=new Date().toISOString().replace("T"," ").substring(0,23),o=O[e].padEnd(5),d=s.padEnd(6),c="";r?.correlationId?c=`[${r.correlationId}] `:r?.sessionId&&(c=`[session-${r.sessionId}] `);let u="";n!=null&&(this.level===0&&typeof n=="object"?u=`
`+JSON.stringify(n,null,2):u=" "+this.formatData(n));let E="";if(r){let{sessionId:T,sdkSessionId:S,correlationId:_,...p}=r;Object.keys(p).length>0&&(E=` {${Object.entries(p).map(([M,w])=>`${M}=${w}`).join(", ")}}`)}let b=`[${i}] [${o}] [${d}] ${c}${t}${E}${u}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},g=new N;var R=class{db;constructor(){v(l),this.db=new j(A),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable(),this.ensureDiscoveryTokensColumn()}initializeSchema(){try{this.db.exec(`
CREATE TABLE IF NOT EXISTS schema_versions (
id INTEGER PRIMARY KEY,
version INTEGER UNIQUE NOT NULL,
@@ -63,7 +63,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
CREATE INDEX IF NOT EXISTS idx_session_summaries_sdk_session ON session_summaries(sdk_session_id);
CREATE INDEX IF NOT EXISTS idx_session_summaries_project ON session_summaries(project);
CREATE INDEX IF NOT EXISTS idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
`),this.db.prepare("INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)").run(4,new Date().toISOString()),console.error("[SessionStore] Migration004 applied successfully"))}catch(e){throw console.error("[SessionStore] Schema initialization error:",e.message),e}}ensureWorkerPortColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(5))return;this.db.pragma("table_info(sdk_sessions)").some(r=>r.name==="worker_port")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN worker_port INTEGER"),console.error("[SessionStore] Added worker_port column to sdk_sessions table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(5,new Date().toISOString())}catch(e){console.error("[SessionStore] Migration error:",e.message)}}ensurePromptTrackingColumns(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(6))return;this.db.pragma("table_info(sdk_sessions)").some(p=>p.name==="prompt_counter")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN prompt_counter INTEGER DEFAULT 0"),console.error("[SessionStore] Added prompt_counter column to sdk_sessions table")),this.db.pragma("table_info(observations)").some(p=>p.name==="prompt_number")||(this.db.exec("ALTER TABLE observations ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to observations table")),this.db.pragma("table_info(session_summaries)").some(p=>p.name==="prompt_number")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(6,new Date().toISOString())}catch(e){console.error("[SessionStore] Prompt tracking migration error:",e.message)}}removeSessionSummariesUniqueConstraint(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;if(!this.db.pragma("index_list(session_summaries)").some(r=>r.unique===1)){this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString());return}console.error("[SessionStore] Removing UNIQUE constraint from session_summaries.sdk_session_id..."),this.db.exec("BEGIN TRANSACTION");try{this.db.exec(`
`),this.db.prepare("INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)").run(4,new Date().toISOString()),console.error("[SessionStore] Migration004 applied successfully"))}catch(e){throw console.error("[SessionStore] Schema initialization error:",e.message),e}}ensureWorkerPortColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(5))return;this.db.pragma("table_info(sdk_sessions)").some(r=>r.name==="worker_port")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN worker_port INTEGER"),console.error("[SessionStore] Added worker_port column to sdk_sessions table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(5,new Date().toISOString())}catch(e){console.error("[SessionStore] Migration error:",e.message)}}ensurePromptTrackingColumns(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(6))return;this.db.pragma("table_info(sdk_sessions)").some(d=>d.name==="prompt_counter")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN prompt_counter INTEGER DEFAULT 0"),console.error("[SessionStore] Added prompt_counter column to sdk_sessions table")),this.db.pragma("table_info(observations)").some(d=>d.name==="prompt_number")||(this.db.exec("ALTER TABLE observations ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to observations table")),this.db.pragma("table_info(session_summaries)").some(d=>d.name==="prompt_number")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(6,new Date().toISOString())}catch(e){console.error("[SessionStore] Prompt tracking migration error:",e.message)}}removeSessionSummariesUniqueConstraint(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;if(!this.db.pragma("index_list(session_summaries)").some(r=>r.unique===1)){this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString());return}console.error("[SessionStore] Removing UNIQUE constraint from session_summaries.sdk_session_id..."),this.db.exec("BEGIN TRANSACTION");try{this.db.exec(`
CREATE TABLE session_summaries_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
sdk_session_id TEXT NOT NULL,
@@ -166,7 +166,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
INSERT INTO user_prompts_fts(rowid, prompt_text)
VALUES (new.id, new.prompt_text);
END;
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}ensureDiscoveryTokensColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;this.db.pragma("table_info(observations)").some(i=>i.name==="discovery_tokens")||(this.db.exec("ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to observations table")),this.db.pragma("table_info(session_summaries)").some(i=>i.name==="discovery_tokens")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString())}catch(e){console.error("[SessionStore] Discovery tokens migration error:",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
SELECT
request, investigated, learned, completed, next_steps,
files_read, files_edited, notes, prompt_number, created_at
@@ -216,6 +216,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
`).all(e)}getAllProjects(){return this.db.prepare(`
SELECT DISTINCT project
FROM sdk_sessions
WHERE project IS NOT NULL AND project != ''
ORDER BY project ASC
`).all().map(t=>t.project)}getRecentSessionsWithStatus(e,s=3){return this.db.prepare(`
SELECT * FROM (
@@ -243,12 +244,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
SELECT *
FROM observations
WHERE id = ?
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
SELECT *
FROM observations
WHERE id IN (${i})
WHERE id IN (${o})
ORDER BY created_at_epoch ${n}
${o}
${i}
`).all(...e)}getSummaryForSession(e){return this.db.prepare(`
SELECT
request, investigated, learned, completed, next_steps,
@@ -261,7 +262,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
SELECT files_read, files_modified
FROM observations
WHERE sdk_session_id = ?
`).all(e),r=new Set,n=new Set;for(let o of t){if(o.files_read)try{let i=JSON.parse(o.files_read);Array.isArray(i)&&i.forEach(p=>r.add(p))}catch{}if(o.files_modified)try{let i=JSON.parse(o.files_modified);Array.isArray(i)&&i.forEach(p=>n.add(p))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
`).all(e),r=new Set,n=new Set;for(let i of t){if(i.files_read)try{let o=JSON.parse(i.files_read);Array.isArray(o)&&o.forEach(d=>r.add(d))}catch{}if(i.files_modified)try{let o=JSON.parse(i.files_modified);Array.isArray(o)&&o.forEach(d=>n.add(d))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
SELECT id, claude_session_id, sdk_session_id, project, user_prompt
FROM sdk_sessions
WHERE id = ?
@@ -288,17 +289,17 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
`).get(e)?.prompt_counter||1}getPromptCounter(e){return this.db.prepare(`
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
`).get(e)?.prompt_counter||0}createSDKSession(e,s,t){let r=new Date,n=r.getTime(),i=this.db.prepare(`
`).get(e)?.prompt_counter||0}createSDKSession(e,s,t){let r=new Date,n=r.getTime(),o=this.db.prepare(`
INSERT OR IGNORE INTO sdk_sessions
(claude_session_id, sdk_session_id, project, user_prompt, started_at, started_at_epoch, status)
VALUES (?, ?, ?, ?, ?, ?, 'active')
`).run(e,e,s,t,r.toISOString(),n);return i.lastInsertRowid===0||i.changes===0?this.db.prepare(`
`).run(e,e,s,t,r.toISOString(),n);return o.lastInsertRowid===0||o.changes===0?this.db.prepare(`
SELECT id FROM sdk_sessions WHERE claude_session_id = ? LIMIT 1
`).get(e).id:i.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
`).get(e).id:o.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
UPDATE sdk_sessions
SET sdk_session_id = ?
WHERE id = ? AND sdk_session_id IS NULL
`).run(s,e).changes===0?(S.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
`).run(s,e).changes===0?(g.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
UPDATE sdk_sessions
SET worker_port = ?
WHERE id = ?
@@ -311,29 +312,29 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
INSERT INTO user_prompts
(claude_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?)
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r,n=0){let i=new Date,o=i.getTime();this.db.prepare(`
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
`).get(e)||(this.db.prepare(`
INSERT INTO sdk_sessions
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
VALUES (?, ?, ?, ?, ?, 'active')
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let m=this.db.prepare(`
`).run(e,e,s,i.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
INSERT INTO observations
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
files_read, files_modified, prompt_number, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(m.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n,i.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r,n=0){let i=new Date,o=i.getTime();this.db.prepare(`
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
`).get(e)||(this.db.prepare(`
INSERT INTO sdk_sessions
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
VALUES (?, ?, ?, ?, ?, 'active')
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let m=this.db.prepare(`
`).run(e,e,s,i.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
INSERT INTO session_summaries
(sdk_session_id, project, request, investigated, learned, completed,
next_steps, notes, prompt_number, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(m.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
next_steps, notes, prompt_number, discovery_tokens, created_at, created_at_epoch)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n,i.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
UPDATE sdk_sessions
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
WHERE id = ?
@@ -341,63 +342,77 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
UPDATE sdk_sessions
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
WHERE id = ?
`).run(s.toISOString(),t,e)}cleanupOrphanedSessions(){let e=new Date,s=e.getTime();return this.db.prepare(`
UPDATE sdk_sessions
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
WHERE status = 'active'
`).run(e.toISOString(),s).changes}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
SELECT * FROM session_summaries
WHERE id IN (${i})
WHERE id IN (${o})
ORDER BY created_at_epoch ${n}
${o}
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
${i}
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
SELECT
up.*,
s.project,
s.sdk_session_id
FROM user_prompts up
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
WHERE up.id IN (${i})
WHERE up.id IN (${o})
ORDER BY up.created_at_epoch ${n}
${o}
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],p,u;if(e!==null){let l=`
${i}
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let i=n?"AND project = ?":"",o=n?[n]:[],d,c;if(e!==null){let T=`
SELECT id, created_at_epoch
FROM observations
WHERE id <= ? ${o}
WHERE id <= ? ${i}
ORDER BY id DESC
LIMIT ?
`,b=`
`,S=`
SELECT id, created_at_epoch
FROM observations
WHERE id >= ? ${o}
WHERE id >= ? ${i}
ORDER BY id ASC
LIMIT ?
`;try{let c=this.db.prepare(l).all(e,...i,t+1),d=this.db.prepare(b).all(e,...i,r+1);if(c.length===0&&d.length===0)return{observations:[],sessions:[],prompts:[]};p=c.length>0?c[c.length-1].created_at_epoch:s,u=d.length>0?d[d.length-1].created_at_epoch:s}catch(c){return console.error("[SessionStore] Error getting boundary observations:",c.message),{observations:[],sessions:[],prompts:[]}}}else{let l=`
`;try{let _=this.db.prepare(T).all(e,...o,t+1),p=this.db.prepare(S).all(e,...o,r+1);if(_.length===0&&p.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,c=p.length>0?p[p.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
SELECT created_at_epoch
FROM observations
WHERE created_at_epoch <= ? ${o}
WHERE created_at_epoch <= ? ${i}
ORDER BY created_at_epoch DESC
LIMIT ?
`,b=`
`,S=`
SELECT created_at_epoch
FROM observations
WHERE created_at_epoch >= ? ${o}
WHERE created_at_epoch >= ? ${i}
ORDER BY created_at_epoch ASC
LIMIT ?
`;try{let c=this.db.prepare(l).all(s,...i,t),d=this.db.prepare(b).all(s,...i,r+1);if(c.length===0&&d.length===0)return{observations:[],sessions:[],prompts:[]};p=c.length>0?c[c.length-1].created_at_epoch:s,u=d.length>0?d[d.length-1].created_at_epoch:s}catch(c){return console.error("[SessionStore] Error getting boundary timestamps:",c.message),{observations:[],sessions:[],prompts:[]}}}let m=`
`;try{let _=this.db.prepare(T).all(s,...o,t),p=this.db.prepare(S).all(s,...o,r+1);if(_.length===0&&p.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,c=p.length>0?p[p.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let u=`
SELECT *
FROM observations
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${i}
ORDER BY created_at_epoch ASC
`,T=`
`,E=`
SELECT *
FROM session_summaries
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${i}
ORDER BY created_at_epoch ASC
`,g=`
`,b=`
SELECT up.*, s.project, s.sdk_session_id
FROM user_prompts up
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${o.replace("project","s.project")}
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${i.replace("project","s.project")}
ORDER BY up.created_at_epoch ASC
`;try{let l=this.db.prepare(m).all(p,u,...i),b=this.db.prepare(T).all(p,u,...i),c=this.db.prepare(g).all(p,u,...i);return{observations:l,sessions:b.map(d=>({id:d.id,sdk_session_id:d.sdk_session_id,project:d.project,request:d.request,completed:d.completed,next_steps:d.next_steps,created_at:d.created_at,created_at_epoch:d.created_at_epoch})),prompts:c.map(d=>({id:d.id,claude_session_id:d.claude_session_id,project:d.project,prompt:d.prompt_text,created_at:d.created_at,created_at_epoch:d.created_at_epoch}))}}catch(l){return console.error("[SessionStore] Error querying timeline records:",l.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function W(a,e,s){return a==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:a==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:a==="UserPromptSubmit"||a==="PostToolUse"?{continue:!0,suppressOutput:!0}:a==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function D(a,e,s={}){let t=W(a,e,s);return JSON.stringify(t)}import f from"path";import{homedir as $}from"os";import{existsSync as G,readFileSync as Y}from"fs";import{execSync as K}from"child_process";var q=100,V=100,J=1e4;function I(){try{let a=f.join($(),".claude-mem","settings.json");if(G(a)){let e=JSON.parse(Y(a,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function k(){try{let a=I();return(await fetch(`http://127.0.0.1:${a}/health`,{signal:AbortSignal.timeout(q)})).ok}catch{return!1}}async function Q(){let a=Date.now();for(;Date.now()-a<J;){if(await k())return!0;await new Promise(e=>setTimeout(e,V))}return!1}async function x(){if(await k())return;let a=v(),e=f.join(a,"node_modules",".bin","pm2"),s=f.join(a,"ecosystem.config.cjs");if(K(`"${e}" restart "${s}"`,{cwd:a,stdio:"pipe"}),!await Q())throw new Error("Worker failed to become healthy after restart")}async function z(a){if(!a)throw new Error("summaryHook requires input");let{session_id:e}=a;await x();let s=new R,t=s.createSDKSession(e,"",""),r=s.getPromptCounter(t);s.close();let n=I();S.dataIn("HOOK","Stop: Requesting summary",{sessionId:t,workerPort:n,promptNumber:r});try{let o=await fetch(`http://127.0.0.1:${n}/sessions/${t}/summarize`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({prompt_number:r}),signal:AbortSignal.timeout(2e3)});if(!o.ok){let i=await o.text();throw S.failure("HOOK","Failed to generate summary",{sessionId:t,status:o.status},i),new Error(`Failed to request summary from worker: ${o.status} ${i}`)}S.debug("HOOK","Summary request sent successfully",{sessionId:t})}catch(o){throw o.cause?.code==="ECONNREFUSED"||o.name==="TimeoutError"||o.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):o}console.log(D("Stop",!0))}var L="";U.on("data",a=>L+=a);U.on("end",async()=>{let a=L?JSON.parse(L):void 0;await z(a)});
`;try{let T=this.db.prepare(u).all(d,c,...o),S=this.db.prepare(E).all(d,c,...o),_=this.db.prepare(b).all(d,c,...o);return{observations:T,sessions:S.map(p=>({id:p.id,sdk_session_id:p.sdk_session_id,project:p.project,request:p.request,completed:p.completed,next_steps:p.next_steps,created_at:p.created_at,created_at_epoch:p.created_at_epoch})),prompts:_.map(p=>({id:p.id,claude_session_id:p.claude_session_id,project:p.project,prompt:p.prompt_text,created_at:p.created_at,created_at_epoch:p.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function P(a,e,s){return a==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:a==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:a==="UserPromptSubmit"||a==="PostToolUse"?{continue:!0,suppressOutput:!0}:a==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function C(a,e,s={}){let t=P(a,e,s);return JSON.stringify(t)}import $ from"path";import{homedir as G}from"os";import{existsSync as W,readFileSync as Y}from"fs";var K=100;function h(){try{let a=$.join(G(),".claude-mem","settings.json");if(W(a)){let e=JSON.parse(Y(a,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function q(){try{let a=h();return(await fetch(`http://127.0.0.1:${a}/health`,{signal:AbortSignal.timeout(K)})).ok}catch{return!1}}async function D(){if(await q())return;let a=h();throw new Error(`Worker service is not responding on port ${a}.
If you just updated the plugin, PM2's watch mode should restart automatically.
If the problem persists, run: pm2 restart claude-mem-worker`)}import{appendFileSync as V}from"fs";import{homedir as J}from"os";import{join as Q}from"path";var z=Q(J(),".claude-mem","silent.log");function I(a,e,s=""){let t=new Date().toISOString(),o=((new Error().stack||"").split(`
`)[2]||"").match(/at\s+(?:.*\s+)?\(?([^:]+):(\d+):(\d+)\)?/),d=o?`${o[1].split("/").pop()}:${o[2]}`:"unknown",c=`[${t}] [${d}] ${a}`;if(e!==void 0)try{c+=` ${JSON.stringify(e)}`}catch(u){c+=` [stringify error: ${u}]`}c+=`
`;try{V(z,c)}catch(u){console.error("[silent-debug] Failed to write to log:",u)}return s}function Z(a){if(!a||!U(a))return"";try{let e=x(a,"utf-8").trim();if(!e)return"";let s=e.split(`
`);for(let t=s.length-1;t>=0;t--)try{let r=JSON.parse(s[t]);if(r.type==="user"&&r.message?.content){let n=r.message.content;if(typeof n=="string")return n;if(Array.isArray(n))return n.filter(o=>o.type==="text").map(o=>o.text).join(`
`)}}catch{continue}}catch(e){g.error("HOOK","Failed to read transcript",{transcriptPath:a},e)}return""}function ee(a){if(!a||!U(a))return"";try{let e=x(a,"utf-8").trim();if(!e)return"";let s=e.split(`
`);for(let t=s.length-1;t>=0;t--)try{let r=JSON.parse(s[t]);if(r.type==="assistant"&&r.message?.content){let n="",i=r.message.content;return typeof i=="string"?n=i:Array.isArray(i)&&(n=i.filter(d=>d.type==="text").map(d=>d.text).join(`
`)),n=n.replace(/<system-reminder>[\s\S]*?<\/system-reminder>/g,""),n=n.replace(/\n{3,}/g,`
`).trim(),n}}catch{continue}}catch(e){g.error("HOOK","Failed to read transcript",{transcriptPath:a},e)}return""}async function se(a){if(!a)throw new Error("summaryHook requires input");let{session_id:e}=a;await D();let s=new R,t=s.createSDKSession(e,"",""),r=s.getPromptCounter(t),n=s.db.prepare(`
SELECT id, claude_session_id, sdk_session_id, project
FROM sdk_sessions WHERE id = ?
`).get(t),i=s.db.prepare(`
SELECT COUNT(*) as count
FROM observations
WHERE sdk_session_id = ?
`).get(n?.sdk_session_id);I("[summary-hook] Session diagnostics",{claudeSessionId:e,sessionDbId:t,sdkSessionId:n?.sdk_session_id,project:n?.project,promptNumber:r,observationCount:i?.count||0,transcriptPath:a.transcript_path}),s.close();let o=h(),d=Z(a.transcript_path||""),c=ee(a.transcript_path||"");I("[summary-hook] Extracted messages",{hasLastUserMessage:!!d,hasLastAssistantMessage:!!c,lastAssistantPreview:c.substring(0,200),lastAssistantLength:c.length}),g.dataIn("HOOK","Stop: Requesting summary",{sessionId:t,workerPort:o,promptNumber:r,hasLastUserMessage:!!d,hasLastAssistantMessage:!!c});try{let u=await fetch(`http://127.0.0.1:${o}/sessions/${t}/summarize`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({prompt_number:r,last_user_message:d,last_assistant_message:c}),signal:AbortSignal.timeout(2e3)});if(!u.ok){let E=await u.text();throw g.failure("HOOK","Failed to generate summary",{sessionId:t,status:u.status},E),new Error(`Failed to request summary from worker: ${u.status} ${E}`)}g.debug("HOOK","Summary request sent successfully",{sessionId:t})}catch(u){throw u.cause?.code==="ECONNREFUSED"||u.name==="TimeoutError"||u.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):u}finally{await fetch(`http://127.0.0.1:${o}/api/processing`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({isProcessing:!1})})}console.log(C("Stop",!0))}var L="";k.on("data",a=>L+=a);k.on("end",async()=>{let a=L?JSON.parse(L):void 0;await se(a)});
+8 -5
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env node
import{execSync as _}from"child_process";import{join as i}from"path";import{homedir as p}from"os";import{existsSync as D}from"fs";import l from"path";import{homedir as f}from"os";import{existsSync as g,readFileSync as h}from"fs";import{join as e,dirname as m,basename as A}from"path";import{homedir as c}from"os";import{fileURLToPath as u}from"url";function d(){return typeof __dirname<"u"?__dirname:m(u(import.meta.url))}var w=d(),t=process.env.CLAUDE_MEM_DATA_DIR||e(c(),".claude-mem"),s=process.env.CLAUDE_CONFIG_DIR||e(c(),".claude"),P=e(t,"archives"),R=e(t,"logs"),S=e(t,"trash"),I=e(t,"backups"),b=e(t,"settings.json"),v=e(t,"claude-mem.db"),H=e(t,"vector-db"),L=e(s,"settings.json"),M=e(s,"commands"),U=e(s,"CLAUDE.md");function a(){try{let o=l.join(f(),".claude-mem","settings.json");if(g(o)){let n=JSON.parse(h(o,"utf-8")),r=parseInt(n.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(r))return r}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}var x=i(p(),".claude","plugins","marketplaces","thedotmack"),k=i(x,"node_modules");D(k)||(console.error(`
import{execSync as u}from"child_process";import{join as r}from"path";import{homedir as s}from"os";import{existsSync as l}from"fs";import i from"path";import{homedir as a}from"os";import{existsSync as c,readFileSync as p}from"fs";function n(){try{let e=i.join(a(),".claude-mem","settings.json");if(c(e)){let t=JSON.parse(p(e,"utf-8")),o=parseInt(t.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(o))return o}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}var m=r(s(),".claude","plugins","marketplaces","thedotmack"),d=r(m,"node_modules");l(d)||(console.error(`
---
\u{1F389} Note: This appears under Plugin Hook Error, but it's not an error. That's the only option for
user messages in Claude Code UI until a better method is provided.
@@ -17,12 +17,15 @@ Dependencies have been installed in the background. This only happens once.
Thank you for installing Claude-Mem!
This message was not added to your startup context, so you can continue working as normal.
`),process.exit(3));try{let o=i(p(),".claude","plugins","marketplaces","thedotmack","plugin","scripts","context-hook.js"),n=_(`node "${o}" --colors`,{encoding:"utf8"}),r=a();console.error(`
`),process.exit(3));try{let e=r(s(),".claude","plugins","marketplaces","thedotmack","plugin","scripts","context-hook.js"),t=u(`node "${e}" --colors`,{encoding:"utf8"}),o=n();console.error(`
\u{1F4DD} Claude-Mem Context Loaded
\u2139\uFE0F Note: This appears as stderr but is informational only
`+n+`
`+t+`
\u{1F4FA} Watch live in browser http://localhost:${r}/ (New! v5.1)
`)}catch(o){console.error(`\u274C Failed to load context display: ${o}`)}process.exit(3);
\u{1F4AC} Feedback & Support
https://github.com/thedotmack/claude-mem/discussions/110
\u{1F4FA} Watch live in browser http://localhost:${o}/
`)}catch(e){console.error(`\u274C Failed to load context display: ${e}`)}process.exit(3);
File diff suppressed because one or more lines are too long
+196
View File
@@ -0,0 +1,196 @@
---
name: mem-search
description: Search claude-mem's persistent cross-session memory database to find work from previous conversations days, weeks, or months ago. Access past session summaries, bug fixes, feature implementations, and decisions that are NOT in the current conversation context. Use when user asks "did we already solve this?", "how did we do X last time?", "what happened in last week's session?", or needs information from previous sessions stored in the PM2-managed database. Searches observations, session summaries, and user prompts across entire project history.
---
# Memory Search
Access claude-mem's persistent cross-session memory through HTTP API. Find past work, understand context across sessions, and learn from previous decisions.
## When to Use This Skill
**Use when users ask about work from PREVIOUS sessions** (not current conversation):
### Temporal Triggers (Key Indicators)
- "Did we **already** fix this bug?" or "Have we seen this error **before**?"
- "How did we solve X **last time**?" or "What approach did we take **previously**?"
- "What did we do in **yesterday's/last week's/last month's** session?"
- "**When** did we last work on this?" or "What's the **history** of this file?"
### Cross-Session Queries
- "Show me all authentication-related changes **across all sessions**"
- "What features did we add **last month**?" (not "today" or "this session")
- "Why did we choose this approach **before**?" (decisions from past sessions)
- "What files did we modify **when we added X**?" (historical context)
**Do NOT use** for current session work, future planning, or questions Claude can answer from current conversation context.
## Common Trigger Phrases
This skill activates when detecting phrases about **cross-session history**:
- "Did we already solve this?" / "Have we done this before?"
- "How did we implement X last time?"
- "What did we work on yesterday/last week/last month?"
- "Show me the history of [file/feature/decision]"
- "When did we fix/add/change X?"
- "What was happening around [date/time]?"
- "Catch me up on recent sessions" / "What have we been doing?"
- "What changes to [filename] across all sessions?"
**Unique identifiers:** claude-mem, persistent memory, cross-session database, session history, PM2-managed database
## Available Operations
### Full-Text Search
- **observations** - Search all observations by keyword (bugs, features, decisions, discoveries, changes)
- Use when: "How did we implement X?" or "What bugs did we fix?"
- Example: Search for "authentication JWT" to find auth-related work
- **sessions** - Search session summaries to find what was accomplished when
- Use when: "What did we accomplish last time?" or "What was the goal of that session?"
- Example: Find sessions where "added login feature"
- **prompts** - Find what users have asked about in previous sessions
- Use when: "Did I ask about this before?" or "What did I request last week?"
- Example: Search for "database migration" in past user prompts
### Filtered Search
- **by-type** - Filter observations by type (bugfix, feature, refactor, decision, discovery, change)
- Use when: "Show me all bug fixes" or "List features we added"
- Example: Get all observations with type=bugfix from last month
- **by-concept** - Find observations tagged with specific concepts (problem-solution, how-it-works, gotcha)
- Use when: "What patterns did we discover?" or "Show me gotchas"
- Example: Find all observations tagged with concept "gotcha"
- **by-file** - Find all work related to a specific file path across all sessions
- Use when: "What changes to auth.ts?" or "History of this file"
- Example: Get all work related to "src/auth/login.ts"
### Timeline & Context
- **recent-context** - Get last N sessions with summaries and observations
- Use when: "What's been happening?" or "Catch me up on recent work"
- Example: Get last 3 sessions with limit=3
- **timeline** - Get chronological context around a specific point in time (before/after window)
- Use when: "What was happening around [date]?" or "Show me context from that time"
- Example: Timeline around session 123 with depth 5 before and after
- **timeline-by-query** - Search first, then get timeline context around best match
- Use when: "When did we implement auth?" combined with "show me context around that time"
- Example: Search for "OAuth implementation" then get surrounding timeline
For detailed instructions on any operation, read the corresponding file in [operations/](operations/).
## Quick Decision Guide
**What is the user asking about?**
1. **Recent work** (last 3-5 sessions) → Use **recent-context** with limit=3-5
2. **Specific topic/keyword** (bugs, features, decisions) → Use **observations** search
3. **Specific file history** (changes to a file) → Use **by-file** search
4. **Timeline/chronology** (what happened when) → Use **timeline** or **timeline-by-query**
5. **Type-specific** (all bug fixes, all features) → Use **by-type** filter
**Most common:** Use **observations** search for general "how did we..." questions.
## Progressive Disclosure Workflow (Token Efficiency)
**Core Principle**: Find high-signal items in index format FIRST (~50-100 tokens each), then request full details ONLY for relevant items (~500-1000 tokens each).
**4-Step Workflow:**
1. **Start with Index Format**
- Always use `format=index` initially
- Set `limit=3-5` (not 10-20)
- Review titles and dates to assess relevance
- Token cost: ~50-100 per result
2. **Identify Relevant Items**
- Scan index results
- Discard irrelevant items from list
- Keep only 1-3 most relevant
3. **Request Full Details Selectively**
- Use `format=full` ONLY for specific relevant items
- Token cost: ~500-1000 per result
- **10x cost difference** - be selective
4. **Refine with Filters**
- Use type, dateRange, concepts, files filters
- Paginate with offset if needed
- Narrow scope before expanding limits
**DO:**
- ✅ Start with `format=index` and `limit=3-5`
- ✅ Use filters (type, dateRange, concepts, files) to narrow results
- ✅ Request `format=full` ONLY for specific relevant items
- ✅ Use offset for pagination instead of large limits
**DON'T:**
- ❌ Jump straight to `format=full`
- ❌ Request `limit=20` without assessing index results first
- ❌ Load full details for all results upfront
- ❌ Skip index format to "save a step" (costs 10x more tokens)
See [principles/progressive-disclosure.md](principles/progressive-disclosure.md) for complete workflow with examples and token calculations.
## Quick Reference Table
| Need | Operation | Key Parameters |
|------|-----------|----------------|
| Recent context | recent-context | limit=3-5 |
| Search observations | observations | query, format=index, limit=5 |
| Search sessions | sessions | query, format=index, limit=5 |
| Find by type | by-type | type=(bugfix\|feature\|decision), format=index |
| Find by file | by-file | filePath, format=index |
| Timeline around event | timeline | anchor=(sessionDbId), depth_before=5, depth_after=5 |
| Search + timeline | timeline-by-query | query, mode=auto |
## Common Workflows
For step-by-step guides on typical user requests, see [operations/common-workflows.md](operations/common-workflows.md):
- Understanding past work from previous sessions
- Finding specific bug fixes from history
- Understanding file history across sessions
- Timeline investigation workflows
- Search composition patterns
## Response Formatting
For guidelines on presenting search results to users, see [operations/formatting.md](operations/formatting.md):
- Index format responses (compact lists with titles/dates)
- Full format responses (complete observation details)
- Timeline responses (chronologically grouped)
- Error handling and user-friendly messages
## Technical Notes
- **Port:** Default 37777 (configurable via `CLAUDE_MEM_WORKER_PORT`)
- **Response format:** Always JSON
- **Search engine:** ChromaDB semantic search (primary ranking) + SQLite FTS5 (fallback) + 90-day recency filter + temporal ordering (hybrid architecture)
- **All operations:** HTTP GET with query parameters
- **Worker:** PM2-managed background process
## Error Handling
If HTTP request fails:
1. Inform user the claude-mem search service isn't available
2. Suggest checking if worker is running: `pm2 list` or `pm2 status claude-mem-worker`
3. Offer to help troubleshoot using the troubleshoot skill
## Resources
**Principles:**
- [principles/progressive-disclosure.md](principles/progressive-disclosure.md) - Complete 4-step workflow with token calculations
- [principles/anti-patterns.md](principles/anti-patterns.md) - 5 anti-patterns to avoid with LLM behavior insights
**Operations:**
- [operations/](operations/) - Detailed instructions for each operation (9 operations + help)
- [operations/common-workflows.md](operations/common-workflows.md) - Step-by-step workflow guides
- [operations/formatting.md](operations/formatting.md) - Response formatting templates
---
**Remember:** This skill searches **cross-session persistent memory**, NOT current conversation. Start with index format for token efficiency. Use temporal triggers to differentiate from native Claude memory.
@@ -0,0 +1,124 @@
# Search by Concept
Find observations tagged with specific concepts.
## When to Use
- User asks: "What discoveries did we make?"
- User asks: "What patterns did we identify?"
- User asks: "What gotchas did we encounter?"
- Looking for observations with semantic tags
## Command
```bash
curl -s "http://localhost:37777/api/search/by-concept?concept=discovery&format=index&limit=5"
```
## Parameters
- **concept** (required): Concept tag to search for
- `discovery` - New discoveries and insights
- `problem-solution` - Problems and their solutions
- `what-changed` - Change descriptions
- `how-it-works` - Explanations of mechanisms
- `pattern` - Identified patterns
- `gotcha` - Edge cases and gotchas
- `change` - General changes
- **format**: "index" (summary) or "full" (complete details). Default: "full"
- **limit**: Number of results (default: 20, max: 100)
- **project**: Filter by project name (optional)
- **dateRange**: Filter by date range (optional)
## When to Use Each Format
**Use format=index for:**
- Quick overviews of observations by concept
- Finding IDs for deeper investigation
- Listing multiple results
- **Token cost: ~50-100 per result**
**Use format=full for:**
- Complete details including narrative, facts, files, concepts
- Understanding the full context of specific observations
- **Token cost: ~500-1000 per result**
## Example Response (format=index)
```json
{
"concept": "discovery",
"count": 3,
"format": "index",
"results": [
{
"id": 1240,
"type": "discovery",
"title": "Worker service uses PM2 for process management",
"subtitle": "Discovered persistent background worker pattern",
"created_at_epoch": 1699564800000,
"project": "claude-mem",
"concepts": ["discovery", "how-it-works"]
}
]
}
```
## How to Present Results
For format=index, present as a compact list:
```markdown
Found 3 observations tagged with "discovery":
🔵 **#1240** Worker service uses PM2 for process management
> Discovered persistent background worker pattern
> Nov 9, 2024 • claude-mem
> Tags: discovery, how-it-works
🔵 **#1241** FTS5 full-text search enables instant searches
> SQLite FTS5 virtual tables provide sub-100ms search
> Nov 9, 2024 • claude-mem
> Tags: discovery, pattern
```
For complete formatting guidelines, see [formatting.md](formatting.md).
## Available Concepts
| Concept | Description | When to Use |
|---------|-------------|-------------|
| `discovery` | New discoveries and insights | Finding what was learned |
| `problem-solution` | Problems and their solutions | Finding how issues were resolved |
| `what-changed` | Change descriptions | Understanding what changed |
| `how-it-works` | Explanations of mechanisms | Learning how things work |
| `pattern` | Identified patterns | Finding design patterns |
| `gotcha` | Edge cases and gotchas | Learning about pitfalls |
| `change` | General changes | Tracking modifications |
## Error Handling
**Missing concept parameter:**
```json
{"error": "Missing required parameter: concept"}
```
Fix: Add the concept parameter
**Invalid concept:**
```json
{"error": "Invalid concept: foobar. Valid concepts: discovery, problem-solution, what-changed, how-it-works, pattern, gotcha, change"}
```
Fix: Use one of the valid concept values
## Tips
1. Use format=index first to see overview
2. Start with limit=5-10 to avoid token overload
3. Combine concepts with type filtering for precision
4. Use `discovery` for learning what was found during investigation
5. Use `problem-solution` for finding how past issues were resolved
**Token Efficiency:**
- Start with format=index (~50-100 tokens per result)
- Use format=full only for relevant items (~500-1000 tokens per result)
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
@@ -0,0 +1,127 @@
# Search by File
Find all work related to a specific file path.
## When to Use
- User asks: "What changes to auth/login.ts?"
- User asks: "What work was done on this file?"
- User asks: "Show me the history of src/services/worker.ts"
- Looking for all observations that reference a file
## Command
```bash
curl -s "http://localhost:37777/api/search/by-file?filePath=src/services/worker-service.ts&format=index&limit=10"
```
## Parameters
- **filePath** (required): File path to search for (supports partial matching)
- Full path: `src/services/worker-service.ts`
- Partial path: `worker-service.ts`
- Directory: `src/hooks/`
- **format**: "index" (summary) or "full" (complete details). Default: "full"
- **limit**: Number of results (default: 20, max: 100)
- **project**: Filter by project name (optional)
- **dateRange**: Filter by date range (optional)
## When to Use Each Format
**Use format=index for:**
- Quick overviews of work on a file
- Finding IDs for deeper investigation
- Listing multiple changes
- **Token cost: ~50-100 per result**
**Use format=full for:**
- Complete details including narrative, facts, files, concepts
- Understanding the full context of specific changes
- **Token cost: ~500-1000 per result**
## Example Response (format=index)
```json
{
"filePath": "src/services/worker-service.ts",
"count": 8,
"format": "index",
"results": [
{
"id": 1245,
"type": "refactor",
"title": "Simplified worker health check logic",
"subtitle": "Removed redundant PM2 status check",
"created_at_epoch": 1699564800000,
"project": "claude-mem",
"files": ["src/services/worker-service.ts", "src/services/worker-utils.ts"]
}
]
}
```
## How to Present Results
For format=index, present as a compact list:
```markdown
Found 8 observations related to "src/services/worker-service.ts":
🔄 **#1245** Simplified worker health check logic
> Removed redundant PM2 status check
> Nov 9, 2024 • claude-mem
> Files: worker-service.ts, worker-utils.ts
🟣 **#1246** Added SSE endpoint for real-time updates
> Implemented Server-Sent Events for viewer UI
> Nov 8, 2024 • claude-mem
> Files: worker-service.ts
```
For complete formatting guidelines, see [formatting.md](formatting.md).
## Partial Path Matching
The file path parameter supports partial matching:
```bash
# These all match "src/services/worker-service.ts"
curl -s "http://localhost:37777/api/search/by-file?filePath=worker-service.ts&format=index"
curl -s "http://localhost:37777/api/search/by-file?filePath=services/worker&format=index"
curl -s "http://localhost:37777/api/search/by-file?filePath=worker-service&format=index"
```
## Directory Searches
Search for all work in a directory:
```bash
curl -s "http://localhost:37777/api/search/by-file?filePath=src/hooks/&format=index&limit=20"
```
## Error Handling
**Missing filePath parameter:**
```json
{"error": "Missing required parameter: filePath"}
```
Fix: Add the filePath parameter
**No results found:**
```json
{"filePath": "nonexistent.ts", "count": 0, "results": []}
```
Response: "No observations found for 'nonexistent.ts'. Try a partial path or check the spelling."
## Tips
1. Use format=index first to see overview of all changes
2. Start with partial paths (e.g., filename only) for broader matches
3. Use full paths when you need specific file matches
4. Combine with dateRange to see recent changes: `?filePath=worker.ts&dateRange[start]=2024-11-01`
5. Use directory searches to see all work in a module
**Token Efficiency:**
- Start with format=index (~50-100 tokens per result)
- Use format=full only for relevant items (~500-1000 tokens per result)
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
@@ -0,0 +1,123 @@
# Search by Type
Find observations by type: bugfix, feature, refactor, decision, discovery, or change.
## When to Use
- User asks: "What bugs did we fix?"
- User asks: "What features did we add?"
- User asks: "What decisions did we make?"
- Looking for specific types of work
## Command
```bash
curl -s "http://localhost:37777/api/search/by-type?type=bugfix&format=index&limit=5"
```
## Parameters
- **type** (required): One or more types (comma-separated)
- `bugfix` - Bug fixes
- `feature` - New features
- `refactor` - Code refactoring
- `decision` - Architectural/design decisions
- `discovery` - Discoveries and insights
- `change` - General changes
- **format**: "index" (summary) or "full" (complete details). Default: "full"
- **limit**: Number of results (default: 20, max: 100)
- **project**: Filter by project name (optional)
- **dateRange**: Filter by date range (optional)
## When to Use Each Format
**Use format=index for:**
- Quick overviews of work by type
- Finding IDs for deeper investigation
- Listing multiple results
- **Token cost: ~50-100 per result**
**Use format=full for:**
- Complete details including narrative, facts, files, concepts
- Understanding the full context of specific observations
- **Token cost: ~500-1000 per result**
## Example Response (format=index)
```json
{
"type": "bugfix",
"count": 5,
"format": "index",
"results": [
{
"id": 1235,
"type": "bugfix",
"title": "Fixed token expiration edge case",
"subtitle": "Handled race condition in refresh flow",
"created_at_epoch": 1699564800000,
"project": "api-server"
}
]
}
```
## How to Present Results
For format=index, present as a compact list with type emojis:
```markdown
Found 5 bugfixes:
🔴 **#1235** Fixed token expiration edge case
> Handled race condition in refresh flow
> Nov 9, 2024 • api-server
🔴 **#1236** Resolved memory leak in worker
> Fixed event listener cleanup
> Nov 8, 2024 • worker-service
```
**Type Emojis:**
- 🔴 bugfix
- 🟣 feature
- 🔄 refactor
- 🔵 discovery
- 🧠 decision
- ✅ change
For complete formatting guidelines, see [formatting.md](formatting.md).
## Multiple Types
To search for multiple types:
```bash
curl -s "http://localhost:37777/api/search/by-type?type=bugfix,feature&format=index&limit=10"
```
## Error Handling
**Missing type parameter:**
```json
{"error": "Missing required parameter: type"}
```
Fix: Add the type parameter
**Invalid type:**
```json
{"error": "Invalid type: foobar. Valid types: bugfix, feature, refactor, decision, discovery, change"}
```
Fix: Use one of the valid type values
## Tips
1. Use format=index first to see overview
2. Start with limit=5-10 to avoid token overload
3. Combine with dateRange for recent work: `?type=bugfix&dateRange[start]=2024-11-01`
4. Use project filtering when working on one codebase
**Token Efficiency:**
- Start with format=index (~50-100 tokens per result)
- Use format=full only for relevant items (~500-1000 tokens per result)
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
@@ -0,0 +1,251 @@
# Common Workflows
Step-by-step guides for typical user requests using the search API.
## Workflow 1: Understanding Past Work
**User asks:** "What did we do last session?" or "Catch me up on recent work"
**Steps:**
1. **Get recent context** (fastest path):
```bash
curl -s "http://localhost:37777/api/context/recent?limit=3"
```
2. **Present as narrative:**
```markdown
## Recent Work
### Session #545 - Nov 9, 2024
Implemented JWT authentication system
**Completed:**
- Added token-based auth with refresh tokens
- Created JWT signing and verification logic
**Key Learning:** JWT expiration requires careful handling of refresh race conditions
```
**Why this workflow:**
- Single request gets both sessions and observations
- Optimized for "catch me up" questions
- ~1,500-2,500 tokens for 3 sessions
---
## Workflow 2: Finding Specific Bug Fixes
**User asks:** "What bugs did we fix?" or "Show me recent bug fixes"
**Steps:**
1. **Search by type** (index format first):
```bash
curl -s "http://localhost:37777/api/search/by-type?type=bugfix&format=index&limit=5"
```
2. **Review index results**, identify relevant items
3. **Get full details** for specific bugs:
```bash
curl -s "http://localhost:37777/api/search/by-type?type=bugfix&format=full&limit=1&offset=2"
```
4. **Present findings:**
```markdown
Found 5 bug fixes:
🔴 **#1235** Fixed token expiration edge case
> Handled race condition in refresh flow
> Nov 9, 2024 • api-server
[Click for full details on #1235]
```
**Why this workflow:**
- Progressive disclosure: index first, full details selectively
- Type-specific search is more efficient than generic search
- ~250-500 tokens for index, ~750-1000 per full detail
---
## Workflow 3: Understanding File History
**User asks:** "What changes to auth/login.ts?" or "Show me work on this file"
**Steps:**
1. **Search by file** (index format):
```bash
curl -s "http://localhost:37777/api/search/by-file?filePath=auth/login.ts&format=index&limit=10"
```
2. **Review chronological changes**
3. **Get full details** for specific changes:
```bash
curl -s "http://localhost:37777/api/search/by-file?filePath=auth/login.ts&format=full&limit=1&offset=3"
```
4. **Present as file timeline:**
```markdown
## History of auth/login.ts
🟣 **#1230** Added JWT authentication (Nov 9)
🔴 **#1235** Fixed token expiration bug (Nov 9)
🔄 **#1240** Refactored auth flow (Nov 8)
```
**Why this workflow:**
- File-specific search finds all related work
- Index format shows chronological overview
- Selective full details for deep dives
---
## Workflow 4: Timeline Investigation
**User asks:** "What was happening when we deployed?" or "Show me context around that bug fix"
**Steps:**
1. **Find the event** using search:
```bash
curl -s "http://localhost:37777/api/search/observations?query=deployment&format=index&limit=5"
```
2. **Note observation ID** (e.g., #1234)
3. **Get timeline context**:
```bash
curl -s "http://localhost:37777/api/timeline/context?anchor=1234&depth_before=10&depth_after=10"
```
4. **Present as chronological narrative:**
```markdown
## Timeline: Deployment
### Before (10 records)
**2:45 PM** - 🟣 Prepared deployment scripts
**2:50 PM** - 💬 User asked: "Are we ready to deploy?"
### ⭐ Anchor Point (2:55 PM)
🎯 **Observation #1234**: Deployed to production
### After (10 records)
**3:00 PM** - 🔴 Fixed post-deployment routing issue
```
**Why this workflow:**
- Timeline shows temporal context (what happened before/after)
- Captures causality between events
- All record types (observations, sessions, prompts) interleaved
---
## Workflow 5: Quick Timeline (One Request)
**User asks:** "Timeline of authentication work"
**Steps:**
1. **Use timeline-by-query** (auto mode):
```bash
curl -s "http://localhost:37777/api/timeline/by-query?query=authentication&mode=auto&depth_before=10&depth_after=10"
```
2. **Present timeline directly:**
```markdown
## Timeline: Authentication
**Best Match:** 🟣 Observation #1234 - Implemented JWT authentication
### Context (21 records)
[... timeline around best match ...]
```
**Why this workflow:**
- Single request combines search + timeline
- Fastest path when query is specific
- Auto mode uses top result as anchor
**Alternative:** Use interactive mode for broad queries:
```bash
curl -s "http://localhost:37777/api/timeline/by-query?query=auth&mode=interactive&limit=5"
```
Then choose anchor manually.
---
## Workflow 6: Search Composition
**User asks:** "What features did we add to the authentication system recently?"
**Steps:**
1. **Combine filters** for precision:
```bash
curl -s "http://localhost:37777/api/search/observations?query=authentication&type=feature&dateRange[start]=2024-11-01&format=index&limit=10"
```
2. **Review filtered results**
3. **Get full details** for relevant features:
```bash
curl -s "http://localhost:37777/api/search/observations?query=authentication&type=feature&format=full&limit=1&offset=2"
```
4. **Present findings:**
```markdown
Found 10 authentication features added in November:
🟣 **#1234** Implemented JWT authentication (Nov 9)
🟣 **#1236** Added refresh token rotation (Nov 9)
🟣 **#1238** Implemented OAuth2 flow (Nov 7)
```
**Why this workflow:**
- Multiple filters narrow results before requesting full details
- Type + query + dateRange = precise targeting
- Progressive disclosure: index first, full details selectively
---
## Workflow Selection Guide
| User Request | Workflow | Operation | Token Cost |
|--------------|----------|-----------|------------|
| "What did we do last session?" | #1 | recent-context | 1,500-2,500 |
| "What bugs did we fix?" | #2 | by-type | 500-3,000 |
| "What changes to file.ts?" | #3 | by-file | 500-3,000 |
| "What was happening then?" | #4 | search → timeline | 3,500-6,000 |
| "Timeline of X work" | #5 | timeline-by-query | 3,000-4,000 |
| "Recent features added?" | #6 | observations + filters | 500-3,000 |
## General Principles
1. **Start with index format** - Always use `format=index` first
2. **Use specialized tools** - by-type, by-file, by-concept when applicable
3. **Compose operations** - Combine search + timeline for investigations
4. **Filter early** - Use type, dateRange, project to narrow before expanding
5. **Progressive disclosure** - Load full details only for relevant items
## Token Budget Awareness
**Quick queries** (500-1,500 tokens):
- Recent context (limit=3)
- Index search (limit=5-10)
- Filtered searches
**Medium queries** (1,500-4,000 tokens):
- Recent context (limit=5-10)
- Full details (3-5 items)
- Timeline (depth 10/10)
**Deep queries** (4,000-8,000 tokens):
- Timeline (depth 20/20)
- Full details (10+ items)
- Multiple composed operations
Always start with minimal token investment, expand only when needed.
@@ -0,0 +1,403 @@
# Response Formatting Guidelines
How to present search results to users for maximum clarity and usefulness.
## General Principles
1. **Progressive disclosure** - Show index results first, full details on demand
2. **Visual hierarchy** - Use emojis, bold, and structure for scannability
3. **Context-aware** - Tailor presentation to user's question
4. **Actionable** - Include IDs for follow-up queries
5. **Token-efficient** - Balance detail with token budget
---
## Format: Index Results
**When to use:** First response to searches, overviews, multiple results
**Structure:**
```markdown
Found {count} results for "{query}":
{emoji} **#{id}** {title}
> {subtitle}
> {date} • {project}
```
**Example:**
```markdown
Found 5 results for "authentication":
🟣 **#1234** Implemented JWT authentication
> Added token-based auth with refresh tokens
> Nov 9, 2024 • api-server
🔴 **#1235** Fixed token expiration edge case
> Handled race condition in refresh flow
> Nov 9, 2024 • api-server
```
**Type Emojis:**
- 🔴 bugfix
- 🟣 feature
- 🔄 refactor
- 🔵 discovery
- 🧠 decision
- ✅ change
- 🎯 session
- 💬 prompt
**What to include:**
- ✅ ID (for follow-up)
- ✅ Type emoji
- ✅ Title
- ✅ Subtitle (if available)
- ✅ Date (human-readable)
- ✅ Project name
- ❌ Don't include full narrative/facts/files in index format
---
## Format: Full Results
**When to use:** User requests details, specific items selected from index
**Structure:**
```markdown
## {emoji} {type} #{id}: {title}
**Summary:** {subtitle}
**What happened:**
{narrative}
**Key Facts:**
- {fact1}
- {fact2}
**Files modified:**
- {file1}
- {file2}
**Concepts:** {concepts}
**Date:** {human_readable_date}
**Project:** {project}
```
**Example:**
```markdown
## 🟣 Feature #1234: Implemented JWT authentication
**Summary:** Added token-based auth with refresh tokens
**What happened:**
Implemented a complete JWT authentication system with access and refresh tokens. Access tokens expire after 15 minutes, refresh tokens after 7 days. Added token signing with RS256 algorithm and proper key rotation infrastructure.
**Key Facts:**
- Access tokens use 15-minute expiration
- Refresh tokens stored in httpOnly cookies
- RS256 algorithm with key rotation support
- Token refresh endpoint handles race conditions gracefully
**Files modified:**
- src/auth/jwt.ts (created)
- src/auth/refresh.ts (created)
- src/middleware/auth.ts (modified)
**Concepts:** how-it-works, pattern
**Date:** November 9, 2024 at 2:55 PM
**Project:** api-server
```
**What to include:**
- ✅ Full title with emoji and ID
- ✅ Summary/subtitle
- ✅ Complete narrative
- ✅ All key facts
- ✅ All files (with status: created/modified/deleted)
- ✅ Concepts/tags
- ✅ Precise timestamp
- ✅ Project name
---
## Format: Timeline Results
**When to use:** Temporal investigations, "what was happening" questions
**Structure:**
```markdown
## Timeline: {anchor_description}
### Before ({count} records)
**{time}** - {emoji} {type} #{id}: {title}
**{time}** - {emoji} {type} #{id}: {title}
### ⭐ Anchor Point ({time})
{emoji} **{type} #{id}**: {title}
### After ({count} records)
**{time}** - {emoji} {type} #{id}: {title}
**{time}** - {emoji} {type} #{id}: {title}
```
**Example:**
```markdown
## Timeline: Deployment
### Before (10 records)
**2:30 PM** - 🟣 #1230: Prepared deployment scripts
**2:45 PM** - 🔄 #1232: Updated configuration files
**2:50 PM** - 💬 User asked: "Are we ready to deploy?"
### ⭐ Anchor Point (2:55 PM)
🎯 **Session #545**: Deployed to production
### After (10 records)
**3:00 PM** - 🔴 #1235: Fixed post-deployment routing issue
**3:10 PM** - 🔵 #1236: Discovered caching behavior in production
**3:15 PM** - 🧠 #1237: Decided to add health check endpoint
```
**What to include:**
- ✅ Chronological ordering (oldest to newest)
- ✅ Human-readable times (not epochs)
- ✅ Clear anchor point marker (⭐)
- ✅ Mix of all record types (observations, sessions, prompts)
- ✅ Concise titles (not full narratives)
- ✅ Type emojis for quick scanning
---
## Format: Session Summaries
**When to use:** Recent context, "what did we do" questions
**Structure:**
```markdown
## Recent Work on {project}
### 🎯 Session #{id} - {date}
**Request:** {user_request}
**Completed:**
- {completion1}
- {completion2}
**Key Learning:** {learning}
**Observations:**
- {emoji} **#{obs_id}** {obs_title}
- Files: {file_list}
```
**Example:**
```markdown
## Recent Work on api-server
### 🎯 Session #545 - November 9, 2024
**Request:** Add JWT authentication with refresh tokens
**Completed:**
- Implemented token-based auth with refresh logic
- Added JWT signing and verification
- Created refresh token rotation
**Key Learning:** JWT expiration requires careful handling of refresh race conditions
**Observations:**
- 🟣 **#1234** Implemented JWT authentication
- Files: jwt.ts, refresh.ts, middleware/auth.ts
- 🔴 **#1235** Fixed token expiration edge case
- Files: refresh.ts
```
**What to include:**
- ✅ Session ID and date
- ✅ Original user request
- ✅ What was completed (bulleted list)
- ✅ Key learnings/insights
- ✅ Linked observations with file lists
- ✅ Clear hierarchy (session → observations)
---
## Format: User Prompts
**When to use:** "What did I ask" questions, prompt searches
**Structure:**
```markdown
Found {count} user prompts:
💬 **Prompt #{id}** (Session #{session_id})
> "{preview_text}"
> {date} • {project}
```
**Example:**
```markdown
Found 5 user prompts about "authentication":
💬 **Prompt #1250** (Session #545)
> "How do I implement JWT authentication with refresh tokens? I need to handle token expiration..."
> Nov 9, 2024 • api-server
💬 **Prompt #1251** (Session #546)
> "The auth tokens are expiring too quickly. Can you help debug the refresh flow?"
> Nov 8, 2024 • api-server
```
**What to include:**
- ✅ Prompt ID
- ✅ Session ID (for context linking)
- ✅ Preview text (200 chars for index, full text for full format)
- ✅ Date and project
- ✅ Quote formatting for prompt text
---
## Error Responses
**No results found:**
```markdown
No results found for "{query}". Try:
- Different search terms
- Broader keywords
- Checking spelling
- Using partial paths (for file searches)
```
**Service unavailable:**
```markdown
The search service isn't available. Check if the worker is running:
```bash
pm2 list
```
If the worker is stopped, restart it:
```bash
npm run worker:restart
```
```
**Invalid parameters:**
```markdown
Invalid search parameters:
- {parameter}: {error_message}
See the [API help](help.md) for valid parameter options.
```
---
## Context-Aware Presentation
Tailor formatting to user's question:
**"What bugs did we fix?"**
→ Use index format, emphasize date/type, group by recency
**"How did we implement X?"**
→ Use full format for best match, include complete narrative and files
**"What was happening when..."**
→ Use timeline format, emphasize chronology and causality
**"Catch me up on recent work"**
→ Use session summary format, focus on high-level accomplishments
---
## Token Budget Guidelines
**Minimal presentation (~100-200 tokens):**
- Index format with 3-5 results
- Compact list structure
- Essential metadata only
**Standard presentation (~500-1,000 tokens):**
- Index format with 10-15 results
- Include subtitles and context
- Clear formatting and emojis
**Detailed presentation (~1,500-3,000 tokens):**
- Full format for 2-3 items
- Complete narratives and facts
- Timeline with 20-30 records
**Comprehensive presentation (~5,000+ tokens):**
- Multiple full results
- Deep timelines (40+ records)
- Session summaries with observations
Always start minimal, expand only when needed.
---
## Markdown Best Practices
1. **Use headers (##, ###)** for hierarchy
2. **Bold important elements** (IDs, titles, dates)
3. **Quote user text** (prompts, questions)
4. **Bullet lists** for facts and files
5. **Code blocks** for commands and examples
6. **Emojis** for type indicators
7. **Horizontal rules (---)** for section breaks
8. **Blockquotes (>)** for subtitles and previews
---
## Examples by Use Case
### Use Case 1: Quick Overview
User: "What did we do last session?"
```markdown
## Recent Work
### 🎯 Session #545 - November 9, 2024
Implemented JWT authentication system
**Key accomplishment:** Added token-based auth with refresh tokens
**Key learning:** JWT expiration requires careful handling of refresh race conditions
```
### Use Case 2: Specific Investigation
User: "How did we implement JWT authentication?"
```markdown
## 🟣 Feature #1234: Implemented JWT authentication
**What happened:**
Implemented a complete JWT authentication system with access and refresh tokens. Access tokens expire after 15 minutes, refresh tokens after 7 days. Added token signing with RS256 algorithm.
**Files:**
- src/auth/jwt.ts (created)
- src/auth/refresh.ts (created)
- src/middleware/auth.ts (modified)
**Key insight:** Refresh race conditions require atomic token exchange logic.
```
### Use Case 3: Timeline Investigation
User: "What was happening around the deployment?"
```markdown
## Timeline: Deployment
[... chronological timeline with before/after context ...]
```
Choose presentation style based on user's question and information needs.
+175
View File
@@ -0,0 +1,175 @@
# API Help
Get comprehensive API documentation for all search endpoints.
## When to Use
- User asks: "What search operations are available?"
- User asks: "How do I use the search API?"
- Need reference documentation for endpoints
- Want to see all available parameters
## Command
```bash
curl -s "http://localhost:37777/api/help"
```
## Response Structure
Returns complete API documentation:
```json
{
"version": "5.4.0",
"base_url": "http://localhost:37777/api",
"endpoints": [
{
"path": "/search/observations",
"method": "GET",
"description": "Search observations using full-text search",
"parameters": [
{
"name": "query",
"required": true,
"type": "string",
"description": "Search terms"
},
{
"name": "format",
"required": false,
"type": "string",
"default": "full",
"options": ["index", "full"],
"description": "Response format"
}
],
"example": "curl -s \"http://localhost:37777/api/search/observations?query=authentication&format=index&limit=5\""
}
]
}
```
## How to Present Results
Present as reference documentation:
```markdown
## claude-mem Search API Reference (v5.4.0)
Base URL: `http://localhost:37777/api`
### Search Operations
**1. Search Observations**
- **Endpoint:** `GET /search/observations`
- **Description:** Search observations using full-text search
- **Parameters:**
- `query` (required, string): Search terms
- `format` (optional, string): "index" or "full" (default: "full")
- `limit` (optional, number): Max results (default: 20, max: 100)
- **Example:**
```bash
curl -s "http://localhost:37777/api/search/observations?query=authentication&format=index&limit=5"
```
[... continue for all endpoints ...]
```
## Endpoint Categories
The API help response organizes endpoints by category:
1. **Full-Text Search**
- `/search/observations`
- `/search/sessions`
- `/search/prompts`
2. **Filtered Search**
- `/search/by-type`
- `/search/by-concept`
- `/search/by-file`
3. **Context Retrieval**
- `/context/recent`
- `/timeline/context`
- `/timeline/by-query`
4. **Utilities**
- `/help`
## Common Parameters
Many endpoints share these parameters:
- **format**: "index" (summary) or "full" (complete details)
- **limit**: Number of results to return
- **offset**: Number of results to skip (for pagination)
- **project**: Filter by project name
- **dateRange**: Filter by date range
- `dateRange[start]`: Start date (ISO string or epoch)
- `dateRange[end]`: End date (ISO string or epoch)
## Error Handling
**Worker not running:**
Connection refused error. Response: "The search API isn't available. Check if worker is running: `pm2 list`"
**Invalid endpoint:**
```json
{"error": "Not found"}
```
Response: "Invalid API endpoint. Use /api/help to see available endpoints."
## Tips
1. Save help response for reference during investigation
2. Use examples as starting point for your queries
3. Check required parameters before making requests
4. Refer to format options for each endpoint
5. All endpoints use GET method with query parameters
**Token Efficiency:**
- Help response: ~2,000-3,000 tokens (complete API reference)
- Use sparingly - refer to operation-specific docs instead
- Keep help response cached for repeated reference
## When to Use Help
**Use help when:**
- Starting to use the search API
- Need complete parameter reference
- Forgot which endpoints are available
- Want to see all options at once
**Don't use help when:**
- You know which operation you need (use operation-specific docs)
- Just need examples (use common-workflows.md)
- Token budget is limited (help is comprehensive)
## Alternative to Help Endpoint
Instead of calling `/api/help`, you can:
1. **Use SKILL.md** - Quick decision guide with operation links
2. **Use operation docs** - Detailed guides for specific endpoints
3. **Use common-workflows.md** - Step-by-step examples
4. **Use formatting.md** - Response presentation templates
The help endpoint is most useful when you need complete API reference in one response.
## API Versioning
The help response includes version information:
```json
{
"version": "5.4.0",
"skill_migration": true,
"deprecated": {
"mcp_tools": "Replaced by HTTP API in v5.4.0"
}
}
```
Check version to ensure compatibility with documentation.
@@ -0,0 +1,103 @@
# Search Observations (Semantic + Full-Text Hybrid)
Search all observations using natural language queries.
## When to Use
- User asks: "How did we implement authentication?"
- User asks: "What bugs did we fix?"
- User asks: "What features did we add?"
- Looking for past work by keyword or topic
## Command
```bash
curl -s "http://localhost:37777/api/search/observations?query=authentication&format=index&limit=5"
```
## Parameters
- **query** (required): Natural language search query - uses semantic search (ChromaDB) for ranking with SQLite FTS5 fallback (e.g., "authentication", "bug fix", "database migration")
- **format**: "index" (summary) or "full" (complete details). Default: "full"
- **limit**: Number of results (default: 20, max: 100)
- **project**: Filter by project name (optional)
## When to Use Each Format
**Use format=index for:**
- Quick overviews
- Finding IDs for deeper investigation
- Listing multiple results
- **Token cost: ~50-100 per result**
**Use format=full for:**
- Complete details including narrative, facts, files, concepts
- Understanding the full context of specific observations
- **Token cost: ~500-1000 per result**
## Example Response (format=index)
```json
{
"query": "authentication",
"count": 5,
"format": "index",
"results": [
{
"id": 1234,
"type": "feature",
"title": "Implemented JWT authentication",
"subtitle": "Added token-based auth with refresh tokens",
"created_at_epoch": 1699564800000,
"project": "api-server",
"score": 0.95
}
]
}
```
## How to Present Results
For format=index, present as a compact list:
```markdown
Found 5 results for "authentication":
1. **#1234** [feature] Implemented JWT authentication
> Added token-based auth with refresh tokens
> Nov 9, 2024 • api-server
2. **#1235** [bugfix] Fixed token expiration edge case
> Handled race condition in refresh flow
> Nov 9, 2024 • api-server
```
**Include:** ID (for follow-up), type emoji (🔴 bugfix, 🟣 feature, 🔄 refactor, 🔵 discovery, 🧠 decision, ✅ change), title, subtitle, date, project.
For complete formatting guidelines, see formatting.md (documentation coming soon).
## Error Handling
**Missing query parameter:**
```json
{"error": "Missing required parameter: query"}
```
Fix: Add the query parameter
**No results found:**
```json
{"query": "foobar", "count": 0, "results": []}
```
Response: "No results found for 'foobar'. Try different search terms."
## Tips
1. Be specific: "authentication JWT" > "auth"
2. Start with format=index and limit=5-10
3. Use project filtering when working on one codebase
4. If no results, try broader terms or check spelling
**Token Efficiency:**
- Start with format=index (~50-100 tokens per result)
- Use format=full only for relevant items (~500-1000 tokens per result)
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)

Some files were not shown because too many files have changed in this diff Show More