Alex Newman
68290a9121
* refactor: Reduce continuation prompt token usage by 95 lines Removed redundant instructions from continuation prompt that were originally added to mitigate a session continuity issue. That issue has since been resolved, making these detailed instructions unnecessary on every continuation. Changes: - Reduced continuation prompt from ~106 lines to ~11 lines (~95 line reduction) - Changed "User's Goal:" to "Next Prompt in Session:" (more accurate framing) - Removed redundant WHAT TO RECORD, WHEN TO SKIP, and OUTPUT FORMAT sections - Kept concise reminder: "Continue generating observations and progress summaries..." - Initial prompt still contains all detailed instructions Impact: - Significant token savings on every continuation prompt - Faster context injection with no loss of functionality - Instructions remain comprehensive in initial prompt Files modified: - src/sdk/prompts.ts (buildContinuationPrompt function) - plugin/scripts/worker-service.cjs (compiled output) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: Enhance observation and summary prompts for clarity and token efficiency * Enhance prompt clarity and instructions in prompts.ts - Added a reminder to think about instructions before starting work. - Simplified the continuation prompt instruction by removing "for this ongoing session." * feat: Enhance settings.json with permissions and deny access to sensitive files refactor: Remove PLAN-full-observation-display.md and PR_SUMMARY.md as they are no longer needed chore: Delete SECURITY_SUMMARY.md since it is redundant after recent changes fix: Update worker-service.cjs to streamline observation generation instructions cleanup: Remove src-analysis.md and src-tree.md for a cleaner codebase refactor: Modify prompts.ts to clarify instructions for memory processing * refactor: Remove legacy worker service implementation * feat: Enhance summary hook to extract last assistant message and improve logging - Added function to extract the last assistant message from the transcript. - Updated summary hook to include last assistant message in the summary request. - Modified SDKSession interface to store last assistant message. - Adjusted buildSummaryPrompt to utilize last assistant message for generating summaries. - Updated worker service and session manager to handle last assistant message in summarize requests. - Introduced silentDebug utility for improved logging and diagnostics throughout the summary process. * docs: Add comprehensive implementation plan for ROI metrics feature Added detailed implementation plan covering: - Token usage capture from Agent SDK - Database schema changes (migration #8) - Discovery cost tracking per observation - Context hook display with ROI metrics - Testing and rollout strategy Timeline: ~20 hours over 4 days Goal: Empirical data for YC application amendment 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * feat: Add transcript processing scripts for analysis and formatting - Implemented `dump-transcript-readable.ts` to generate a readable markdown dump of transcripts, excluding certain entry types. - Created `extract-rich-context-examples.ts` to extract and showcase rich context examples from transcripts, highlighting user requests and assistant reasoning. - Developed `format-transcript-context.ts` to format transcript context into a structured markdown format for improved observation generation. - Added `test-transcript-parser.ts` for validating data extraction from transcript JSONL files, including statistics and error reporting. - Introduced `transcript-to-markdown.ts` for a complete representation of transcript data in markdown format, showing all context data. - Enhanced type definitions in `transcript.ts` to support new features and ensure type safety. - Built `transcript-parser.ts` to handle parsing of transcript JSONL files, including error handling and data extraction methods. * Refactor hooks and SDKAgent for improved observation handling - Updated `new-hook.ts` to clean user prompts by stripping leading slashes for better semantic clarity. - Enhanced `save-hook.ts` to include additional tools in the SKIP_TOOLS set, preventing unnecessary observations from certain command invocations. - Modified `prompts.ts` to change the structure of observation prompts, emphasizing the observational role and providing a detailed XML output format for observations. - Adjusted `SDKAgent.ts` to enforce stricter tool usage restrictions, ensuring the memory agent operates solely as an observer without any tool access. * feat: Enhance session initialization to accept user prompts and prompt numbers - Updated `handleSessionInit` in `worker-service.ts` to extract `userPrompt` and `promptNumber` from the request body and pass them to `initializeSession`. - Modified `initializeSession` in `SessionManager.ts` to handle optional `currentUserPrompt` and `promptNumber` parameters. - Added logic to update the existing session's `userPrompt` and `lastPromptNumber` if a `currentUserPrompt` is provided. - Implemented debug logging for session initialization and updates to track user prompts and prompt numbers. --------- Co-authored-by: Claude <noreply@anthropic.com>
Persistent memory compression system built for Claude Code.
Quick Start • How It Works • Search Tools • Documentation • Configuration • Troubleshooting • License
Claude-Mem seamlessly preserves context across sessions by automatically capturing tool usage observations, generating semantic summaries, and making them available to future sessions. This enables Claude to maintain continuity of knowledge about projects even after sessions end or reconnect.
Quick Start
Start a new Claude Code session in the terminal and enter the following commands:
> /plugin marketplace add thedotmack/claude-mem
> /plugin install claude-mem
Restart Claude Code. Context from previous sessions will automatically appear in new sessions.
Key Features:
- 🧠 Persistent Memory - Context survives across sessions
- 📊 Progressive Disclosure - Layered memory retrieval with token cost visibility
- 🔍 Skill-Based Search - Query your project history with mem-search skill (~2,250 token savings)
- 🖥️ Web Viewer UI - Real-time memory stream at http://localhost:37777
- 🤖 Automatic Operation - No manual intervention required
- 🔗 Citations - Reference past decisions with
claude-mem://URIs
Documentation
📚 View Full Documentation - Browse markdown docs on GitHub
💻 Local Preview: Run Mintlify docs locally:
cd docs
npx mintlify dev
Getting Started
- Installation Guide - Quick start & advanced installation
- Usage Guide - How Claude-Mem works automatically
- Search Tools - Query your project history with natural language
Best Practices
- Context Engineering - AI agent context optimization principles
- Progressive Disclosure - Philosophy behind Claude-Mem's context priming strategy
Architecture
- Overview - System components & data flow
- Architecture Evolution - The journey from v3 to v5
- Hooks Architecture - How Claude-Mem uses lifecycle hooks
- Hooks Reference - 7 hook scripts explained
- Worker Service - HTTP API & PM2 management
- Database - SQLite schema & FTS5 search
- MCP Search - 9 search tools & examples
- Viewer UI - Web-based memory stream visualization
Configuration & Development
- Configuration - Environment variables & settings
- Development - Building, testing, contributing
- Troubleshooting - Common issues & solutions
How It Works
┌─────────────────────────────────────────────────────────────┐
│ Session Start → Inject recent observations as context │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ User Prompts → Create session, save user prompts │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Tool Executions → Capture observations (Read, Write, etc.) │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Worker Processes → Extract learnings via Claude Agent SDK │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Session Ends → Generate summary, ready for next session │
└─────────────────────────────────────────────────────────────┘
Core Components:
- 6 Lifecycle Hooks - context-hook, user-message-hook, new-hook, save-hook, summary-hook, cleanup-hook
- Smart Install - Cached dependency checker (pre-hook script, not a lifecycle hook)
- Worker Service - HTTP API on port 37777 with web viewer UI and 10 search endpoints, managed by PM2
- SQLite Database - Stores sessions, observations, summaries with FTS5 full-text search
- mem-search Skill - Natural language queries with progressive disclosure (~2,250 token savings vs MCP)
- Chroma Vector Database - Hybrid semantic + keyword search for intelligent context retrieval
See Architecture Overview for details.
mem-search Skill
Claude-Mem provides intelligent search through the mem-search skill that auto-invokes when you ask about past work:
How It Works:
- Just ask naturally: "What did we do last session?" or "Did we fix this bug before?"
- Claude automatically invokes the mem-search skill to find relevant context
- ~2,250 token savings per session start vs MCP approach
Available Search Operations:
- Search Observations - Full-text search across observations
- Search Sessions - Full-text search across session summaries
- Search Prompts - Search raw user requests
- By Concept - Find by concept tags (discovery, problem-solution, pattern, etc.)
- By File - Find observations referencing specific files
- By Type - Find by type (decision, bugfix, feature, refactor, discovery, change)
- Recent Context - Get recent session context for a project
- Timeline - Get unified timeline of context around a specific point in time
- Timeline by Query - Search for observations and get timeline context around best match
- API Help - Get search API documentation
Example Natural Language Queries:
"What bugs did we fix last session?"
"How did we implement authentication?"
"What changes were made to worker-service.ts?"
"Show me recent work on this project"
"What was happening when we added the viewer UI?"
See Search Tools Guide for detailed examples.
What's New in v5.5.1
🎯 mem-search Skill Enhancement (v5.5.0):
- Improved Effectiveness: Skill success rate increased from 67% to 100%
- Better Scope Differentiation: Clear distinction from native conversation memory
- Enhanced Triggers: Concrete triggers increased from 44% to 85%
- System-Specific Naming: "mem-search" replaces generic "search" for clarity
- Comprehensive Documentation: 17 total files with detailed operation guides
🔍 Skill-Based Search Architecture (v5.4.0):
- Token Savings: ~2,250 tokens per session start
- Progressive Disclosure: Skill frontmatter (~250 tokens) vs MCP tool definitions (~2,500 tokens)
- Natural Language: Just ask about past work - Claude auto-invokes the mem-search skill
- 10 HTTP API Endpoints: Fast, efficient search operations
- No User Action Required: Migration is transparent
🎨 Theme Toggle (v5.1.2):
- Light/dark mode support in viewer UI
- System preference detection
- Persistent theme settings across sessions
🖥️ Web-Based Viewer UI (v5.1.0):
- Real-time memory stream visualization at http://localhost:37777
- Server-Sent Events (SSE) for instant updates
- Infinite scroll pagination with project filtering
⚡ Smart Install Caching (v5.0.3):
- Eliminated redundant npm installs (2-5s → 10ms)
- Caches version state, only installs when needed
🔍 Hybrid Search Architecture (v5.0.0):
- Chroma vector database for semantic search
- Combined with FTS5 keyword search
- 90-day recency filtering
See CHANGELOG.md for complete version history.
System Requirements
- Node.js: 18.0.0 or higher
- Claude Code: Latest version with plugin support
- PM2: Process manager (bundled - no global install required)
- SQLite 3: For persistent storage (bundled)
Key Benefits
Progressive Disclosure Context
- Layered memory retrieval mirrors human memory patterns
- Layer 1 (Index): See what observations exist with token costs at session start
- Layer 2 (Details): Fetch full narratives on-demand via MCP search
- Layer 3 (Perfect Recall): Access source code and original transcripts
- Smart decision-making: Token counts help Claude choose between fetching details or reading code
- Type indicators: Visual cues (🔴 critical, 🟤 decision, 🔵 informational) highlight observation importance
Automatic Memory
- Context automatically injected when Claude starts
- No manual commands or configuration needed
- Works transparently in the background
Full History Search
- Search across all sessions and observations
- FTS5 full-text search for fast queries
- Citations link back to specific observations
Structured Observations
- AI-powered extraction of learnings
- Categorized by type (decision, bugfix, feature, etc.)
- Tagged with concepts and file references
Multi-Prompt Sessions
- Sessions span multiple user prompts
- Context preserved across
/clearcommands - Track entire conversation threads
Configuration
Model Selection:
./claude-mem-settings.sh
Environment Variables:
CLAUDE_MEM_MODEL- AI model for processing (default: claude-sonnet-4-5)CLAUDE_MEM_WORKER_PORT- Worker port (default: 37777)CLAUDE_MEM_DATA_DIR- Data directory override (dev only)
See Configuration Guide for details.
Development
# Clone and build
git clone https://github.com/thedotmack/claude-mem.git
cd claude-mem
npm install
npm run build
# Run tests
npm test
# Start worker
npm run worker:start
# View logs
npm run worker:logs
See Development Guide for detailed instructions.
Troubleshooting
Quick Diagnostic:
If you're experiencing issues, describe the problem to Claude and the troubleshoot skill will automatically activate to diagnose and provide fixes.
Common Issues:
- Worker not starting →
npm run worker:restart - No context appearing →
npm run test:context - Database issues →
sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;" - Search not working → Check FTS5 tables exist
See Troubleshooting Guide for complete solutions.
Contributing
Contributions are welcome! Please:
- Fork the repository
- Create a feature branch
- Make your changes with tests
- Update documentation
- Submit a Pull Request
See Development Guide for contribution workflow.
License
This project is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0).
Copyright (C) 2025 Alex Newman (@thedotmack). All rights reserved.
See the LICENSE file for full details.
What This Means:
- You can use, modify, and distribute this software freely
- If you modify and deploy on a network server, you must make your source code available
- Derivative works must also be licensed under AGPL-3.0
- There is NO WARRANTY for this software
Support
- Documentation: docs/
- Issues: GitHub Issues
- Repository: github.com/thedotmack/claude-mem
- Author: Alex Newman (@thedotmack)
Built with Claude Agent SDK | Powered by Claude Code | Made with TypeScript