Comprehensive analysis of the 3.5-hour JIT context filtering experiment.
Documents architectural learnings, implementation attempts, and why the
complex persistent-session approach was reverted in favor of a simpler
future implementation.
Key learnings:
- Hooks cannot run Agent SDK queries (no claudePath access)
- All AI processing must happen in worker service
- Dynamic timeline search is the right approach
- Infrastructure (SQLite FTS5 + Chroma) validated and ready
- Simple per-request approach should be tried before optimization
The vision is sound, execution was overcomplicated. This was productive
R&D that validated constraints and documented patterns for future work.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Implemented a new skill-based search approach, replacing MCP search tools to reduce session start context from ~2,500 tokens to ~250 tokens.
- Added 10 new HTTP API endpoints to the worker service for various search functionalities.
- Created a new search skill with detailed instructions and examples for users.
- Removed MCP search server references and updated documentation to reflect the new skill-based approach.
- Enhanced context handling in the context hook to conditionally display the most recent summary based on observations.
- Introduced comprehensive documentation for Agent Skills, including creation, management, and best practices.
- Introduced comprehensive API reference for the TypeScript Agent SDK.
- Documented installation instructions for the SDK.
- Detailed the main functions: `query()`, `tool()`, and `createSdkMcpServer()`.
- Defined various types including `Options`, `Query`, `AgentDefinition`, and more.
- Included message types and their structures, such as `SDKMessage`, `SDKAssistantMessage`, and `SDKUserMessage`.
- Explained hook types and their usage within the SDK.
- Provided detailed documentation for tool input and output types.
- Added sections on permission types and other relevant types for better clarity.
- Document intentional watch mode in ecosystem config for auto-restart on plugin updates.
- Implement auto-session creation in worker service to preserve observation data, ensuring no loss of valuable information during hook failures.
- Address race condition concerns in worker startup and improve error handling in hooks.
- Removed ensureWorkerRunning calls from multiple hooks (cleanup, context, new, save, summary) to streamline code and avoid unnecessary checks.
- Introduced fixed port usage for worker communication across hooks.
- Enhanced error handling in newHook, saveHook, and summaryHook to provide clearer messages for worker connection issues.
- Updated worker service to start without health checks, relying on PM2 for management.
- Cached Claude executable path to optimize repeated calls.
- Improved logging for better traceability of worker actions and errors.
Documentation:
- v5-reddit-post.md: v5.0-specific post focusing on hybrid search breakthrough
- v5-reddit-FINAL-DRAFT.md: General claude-mem post with timeline examples
- v5-reddit-post-story.md: Architecture evolution narrative
- v5-reddit-post-draft.md: Early draft with search examples
- v5-linkedin-post.md: Professional LinkedIn announcement
- reddit-posts.md: Research and reference materials
These are working drafts for community announcements.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Features:
- New UsageLogger utility that writes usage metrics to daily JSONL files
- Captures token counts, costs, timing, and cache metrics from SDK result messages
- Usage logs stored in ~/.claude-mem/usage-logs/ (one file per day)
- Added analyze-usage.js script for analyzing usage patterns
Usage data captured:
- Token counts (input, output, cache creation, cache read)
- Total cost in USD per API call
- Duration metrics (total and API time)
- Number of turns per session
- Session and project attribution
Analysis script features:
- Aggregates totals by project and model
- Shows cache hit rates and savings
- Displays cost breakdowns and averages
- npm scripts: usage:analyze and usage:today
Files:
- src/utils/usage-logger.ts (new)
- src/services/worker-service.ts (modified - captures SDK result messages)
- scripts/analyze-usage.js (new)
- package.json (added usage:* npm scripts)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Created docs structure including introduction, installation, troubleshooting, and usage guides.
- Added detailed installation instructions with quick start and advanced options.
- Documented the automatic operation of Claude-Mem and its session management features.
- Introduced MCP search tools with usage examples and query strategies.
- Provided troubleshooting steps for common issues related to worker service, hooks, database, and search tools.
- Included system requirements and upgrade notes for transitioning from v3.x to v4.0.0.
Root cause: Hooks provide session_id as the source of truth. We were adding
unnecessary validation (checking if sessions exist, checking status, etc.)
which caused 409 conflicts when continuing sessions after /exit.
Changes:
1. worker-service.ts: Removed 409 "Session already exists" check in handleInit
2. SessionStore.ts: Made createSDKSession idempotent using INSERT OR IGNORE
3. new-hook.ts: Simplified to just call createSDKSession - no findActiveSDKSession,
no reactivateSession logic, no status management
4. save-hook.ts: Removed session validation, use fixed port instead of session.worker_port
5. summary-hook.ts: Removed session validation, use fixed port instead of session.worker_port
Philosophy: Hooks manage lifecycle, we just save data with whatever session_id
they give us. No validation, no status checks, no guessing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
All 4 hook entry point scripts were missing process.exit(0) after successful
execution, causing Node processes to hang indefinitely instead of returning
control to Claude Code with exit code 0.
Root cause: In commit 6f62a56, process.exit(0) calls were removed from the
hook functions but were never added to the entry point scripts that wrap them.
Fixed files:
- src/bin/hooks/save-hook.ts (PostToolUse)
- src/bin/hooks/new-hook.ts (UserPromptSubmit)
- src/bin/hooks/summary-hook.ts (Stop)
- src/bin/hooks/context-hook.ts (SessionStart)
This restores proper hook exit behavior and prevents Claude Code from waiting
indefinitely for hook completion.
- Implemented context-hook.ts for handling session start events.
- Created new-hook.ts for user prompt submission events.
- Developed save-hook.ts for post tool use events.
- Added summary-hook.ts for handling stop events.
- Introduced worker.ts as a standalone background process for the SDK agent.
- Each hook reads input from stdin, processes it, and handles errors gracefully.
- Created new tables for SDK sessions and observations
- Implemented HooksDatabase for CRUD operations
- Developed four hook functions: context, new, save, and summary
- Added CLI commands for each hook
- Established comprehensive test suite with all tests passing
feat: Complete Phase 2 implementation with SDK worker process and XML parsing
- Developed SDK prompts for initializing and processing observations
- Implemented XML parser for SDK responses
- Created SDK worker process to handle background observation processing
- Verified integration with HooksDatabase and added tests for all components
feat: Complete Phase 3 integration with comprehensive testing and validation
- Verified all hook functions with database integration
- Created integration and end-to-end tests for session lifecycle
- Ensured non-blocking operations and performance requirements met
- Updated CLI commands for hook architecture and installation flow
- Documented success criteria and next steps for real-world testing
- Introduced a detailed reference for implementing hooks in Claude Code, covering configuration, project-specific scripts, plugin hooks, and various hook events.
- Explained the input modes available in the Claude Agent SDK, emphasizing the benefits of streaming input mode and providing implementation examples for both streaming and single message input.
- Highlighted security considerations and best practices for writing hooks, along with debugging tips and execution details.
Alex Newman