Compare commits
60 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 0a70bcecc5 | |||
| 4e5913611a | |||
| 73982dc709 | |||
| 93de6d97f5 | |||
| 29e6e026b6 | |||
| 1b394cdf4e | |||
| 6ffa4cfde5 | |||
| 8caf159d99 | |||
| e3a63c0294 | |||
| 634033b730 | |||
| 54ef1496d2 | |||
| 5d23c60b76 | |||
| 9314ede6e9 | |||
| 58f4053a61 | |||
| a08e5068c8 | |||
| 9f1b653f2f | |||
| c5d89142b9 | |||
| b44d4702b1 | |||
| 4988b4b317 | |||
| c5e68a17c8 | |||
| f35764aa14 | |||
| 601bded789 | |||
| c231e8d076 | |||
| b62e93577c | |||
| 0787e47b1a | |||
| f9843fe593 | |||
| 5b772ee768 | |||
| baabf14bfa | |||
| c126a7083f | |||
| f1170512d6 | |||
| d64939c379 | |||
| 97d8bd3e62 | |||
| 6cd904a81c | |||
| 74c8afd0e0 | |||
| 02444392da | |||
| d175f1759c | |||
| d55db524f0 | |||
| 7d44fdb289 | |||
| 812de2940d | |||
| 95edf31c14 | |||
| a9ae89a198 | |||
| 047914d087 | |||
| bdf79a439b | |||
| 99b6b85d67 | |||
| 798dec972e | |||
| 286343fef6 | |||
| 9285826547 | |||
| ce3b3733fa | |||
| cf1c966409 | |||
| 02fef487e7 | |||
| 20d45006c0 | |||
| 4f1cd309fd | |||
| c46e4a341a | |||
| 60d5f8fbf1 | |||
| c0778bef00 | |||
| 3cbc041c8b | |||
| 0f96476987 | |||
| cd6f883020 | |||
| 9fb7383ab3 | |||
| e8e7fc81af |
@@ -10,7 +10,7 @@
|
||||
"plugins": [
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "6.0.1",
|
||||
"version": "6.3.2",
|
||||
"source": "./plugin",
|
||||
"description": "Persistent memory system for Claude Code - context compression across sessions"
|
||||
}
|
||||
|
||||
+340
@@ -4,6 +4,346 @@ All notable changes to this project will be documented in this file.
|
||||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
||||
|
||||
## [6.3.1] - 2025-11-25
|
||||
|
||||
## What's New
|
||||
|
||||
- Add script to help estimate token savings from on-the-fly replacements
|
||||
|
||||
## [6.3.0] - 2025-11-25
|
||||
|
||||
## What's New
|
||||
|
||||
### Branch-Based Beta Toggle
|
||||
Added Version Channel section to Settings sidebar allowing users to switch between stable and beta versions directly from the UI.
|
||||
|
||||
**Features:**
|
||||
- See current branch (main or beta/7.0) and stability status
|
||||
- Switch to beta branch to access Endless Mode features
|
||||
- Switch back to stable for production use
|
||||
- Pull updates for current branch
|
||||
|
||||
**Implementation:**
|
||||
- `BranchManager.ts`: Git operations for branch detection/switching
|
||||
- `worker-service.ts`: `/api/branch/*` endpoints (status, switch, update)
|
||||
- `Sidebar.tsx`: Version Channel UI with branch state and handlers
|
||||
|
||||
## Installation
|
||||
To update, restart Claude Code or run the plugin installer.
|
||||
|
||||
## [6.2.1] - 2025-11-23
|
||||
|
||||
## 🐛 Bug Fixes
|
||||
|
||||
### Critical: Empty Project Names Breaking Context Injection
|
||||
|
||||
**Problem:**
|
||||
- Observations and summaries created with empty project names
|
||||
- Context-hook couldn't find recent context (queries `WHERE project = 'claude-mem'`)
|
||||
- Users saw no observations or summaries in SessionStart since Nov 22
|
||||
|
||||
**Root Causes:**
|
||||
|
||||
1. **Sessions:** `createSDKSession()` used `INSERT OR IGNORE` for idempotency, but never updated project field when session already existed
|
||||
2. **In-Memory Cache:** `SessionManager` cached sessions with stale empty project values, even after database was updated
|
||||
|
||||
**Fixes:**
|
||||
|
||||
- `5d23c60` - fix: Update project name when session already exists in createSDKSession
|
||||
- `54ef149` - fix: Refresh in-memory session project when updated in database
|
||||
|
||||
**Impact:**
|
||||
- ✅ 364 observations backfilled with correct project names
|
||||
- ✅ 13 summaries backfilled with correct project names
|
||||
- ✅ Context injection now works (shows recent observations and summaries)
|
||||
- ✅ Future sessions will always have correct project names
|
||||
|
||||
## 📦 Full Changelog
|
||||
|
||||
**Commits since v6.2.0:**
|
||||
- `634033b` - chore: Bump version to 6.2.1
|
||||
- `54ef149` - fix: Refresh in-memory session project when updated in database
|
||||
- `5d23c60` - fix: Update project name when session already exists in createSDKSession
|
||||
|
||||
## [6.2.0] - 2025-11-22
|
||||
|
||||
## Major Features
|
||||
|
||||
### Unified Search API (#145, #133)
|
||||
- **Vector-first search architecture**: All text queries now use ChromaDB semantic search
|
||||
- **Unified /api/search endpoint**: Single endpoint with filter parameters (type, concepts, files)
|
||||
- **ID-based fetch endpoints**: New GET /api/observation/:id, /api/session/:id, /api/prompt/:id
|
||||
- **90-day recency filter**: Automatic relevance filtering for search results
|
||||
- **Backward compatibility**: Legacy endpoints still functional, routing through unified infrastructure
|
||||
|
||||
### Search Architecture Cleanup
|
||||
- **Removed FTS5 fallback code**: Eliminated ~300 lines of deprecated full-text search code
|
||||
- **Removed experimental contextualize endpoint**: Will be reimplemented as LLM-powered skill (see #132)
|
||||
- **Simplified mem-search skill**: Streamlined to prescriptive 3-step workflow (Search → Review IDs → Fetch by ID)
|
||||
- **Better error messages**: Clear guidance when ChromaDB/UVX unavailable
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
### Search Improvements
|
||||
- Fixed parameter handling in searchUserPrompts method
|
||||
- Improved dual-path logic for filter-only vs text queries
|
||||
- Corrected missing debug output in search API
|
||||
|
||||
## Documentation
|
||||
|
||||
- Updated CLAUDE.md to reflect vector-first architecture
|
||||
- Clarified FTS5 tables maintained for backward compatibility only (removal planned for v7.0.0)
|
||||
- Enhanced mem-search skill documentation with clearer usage patterns
|
||||
- Added comprehensive test results for search functionality
|
||||
|
||||
## Breaking Changes
|
||||
|
||||
None - all changes maintain backward compatibility.
|
||||
|
||||
## Installation
|
||||
|
||||
Users with auto-update enabled will receive this update automatically. To manually update:
|
||||
|
||||
\`\`\`bash
|
||||
# Restart Claude Code or run:
|
||||
npm run sync-marketplace
|
||||
\`\`\`
|
||||
|
||||
## [6.1.1] - 2025-11-21
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
### Dynamic Project Name Detection (#142)
|
||||
- Fixed hardcoded "claude-mem" project name in ChromaSync and search-server
|
||||
- Now uses `getCurrentProjectName()` to dynamically detect the project based on working directory
|
||||
- Resolves #140 where all observations were incorrectly tagged with "claude-mem"
|
||||
|
||||
### Viewer UI Scrolling
|
||||
- Simplified overflow CSS to enable proper scrolling in viewer UI
|
||||
- Removed overcomplicated nested overflow containers
|
||||
- Fixed issue where feed content wouldn't scroll
|
||||
|
||||
## Installation
|
||||
|
||||
Users with auto-update enabled will receive this patch automatically. To manually update:
|
||||
|
||||
\`\`\`bash
|
||||
# Restart Claude Code or run:
|
||||
npm run sync-marketplace
|
||||
\`\`\`
|
||||
|
||||
## [6.1.0] - 2025-11-19
|
||||
|
||||
## Viewer UI: Responsive Layout Improvements
|
||||
|
||||
The viewer UI now handles narrow screens better with responsive breakpoints:
|
||||
|
||||
- Community button relocates to sidebar below 600px width
|
||||
- Projects dropdown relocates to sidebar below 480px width
|
||||
- Sidebar constrained to 400px max width
|
||||
|
||||
Makes the viewer usable on phones and narrow browser windows.
|
||||
|
||||
## [6.0.9] - 2025-11-17
|
||||
|
||||
## Queue Depth Indicator Feature
|
||||
|
||||
Added a real-time queue depth indicator to the viewer UI that displays the count of active work items (queued + currently processing).
|
||||
|
||||
### Features
|
||||
- Visual badge next to claude-mem logo
|
||||
- Shows count of pending messages + active SDK generators
|
||||
- Only displays when queueDepth > 0
|
||||
- Subtle pulse animation for visual feedback
|
||||
- Theme-aware styling
|
||||
- Real-time updates via SSE
|
||||
|
||||
### Implementation
|
||||
- Backend: Added `getTotalActiveWork()` method to SessionManager
|
||||
- Backend: Updated worker-service to broadcast queueDepth via SSE
|
||||
- Frontend: Enhanced Header component to display queue bubble
|
||||
- Frontend: Updated useSSE hook to track queueDepth state
|
||||
- Frontend: Added CSS styling with pulse animation
|
||||
|
||||
### Closes
|
||||
- #122 - Implement queue depth indicator feature
|
||||
- #96 - Add real-time queue depth indicator to viewer UI
|
||||
- #97 - Fix inconsistent queue depth calculation
|
||||
|
||||
### Credit
|
||||
Original implementation by @thedotmack in PR #96
|
||||
Bug fix by @copilot-swe-agent in PR #97
|
||||
|
||||
## [6.0.8] - 2025-11-17
|
||||
|
||||
## Critical Fix
|
||||
|
||||
This patch release fixes a critical bug where the PM2 worker process would start from the wrong directory (development folder instead of marketplace folder), causing the plugin to malfunction when installed via the marketplace.
|
||||
|
||||
### What's Fixed
|
||||
|
||||
- **Worker Startup Path Resolution** (`src/shared/worker-utils.ts:61`)
|
||||
Added `cwd: pluginRoot` option to `execSync` when starting PM2
|
||||
|
||||
This ensures the worker always starts from the correct marketplace directory (`~/.claude/plugins/marketplaces/thedotmack/`), regardless of where the hook is invoked from.
|
||||
|
||||
### Impact
|
||||
|
||||
Users will no longer experience issues with the worker starting from the wrong location. The plugin now works correctly when installed via marketplace without manual intervention.
|
||||
|
||||
### Verification
|
||||
|
||||
Run `pm2 info claude-mem-worker` to verify:
|
||||
- **exec cwd** should be: `/Users/[username]/.claude/plugins/marketplaces/thedotmack`
|
||||
- **script path** should be: `/Users/[username]/.claude/plugins/marketplaces/thedotmack/plugin/scripts/worker-service.cjs`
|
||||
|
||||
## [6.0.7] - 2025-11-17
|
||||
|
||||
## Critical Hotfix: Database Migration Issue (#121)
|
||||
|
||||
This is an emergency hotfix addressing a critical database migration bug that prevented claude-mem from loading for some users.
|
||||
|
||||
### What was fixed
|
||||
|
||||
**Issue**: Users were seeing `SqliteError: no such column: discovery_tokens` when starting Claude Code.
|
||||
|
||||
**Root Cause**: The `ensureDiscoveryTokensColumn` migration was using version number 7, which was already taken by another migration (`removeSessionSummariesUniqueConstraint`). This duplicate version number caused migration tracking issues in databases that were upgraded through multiple versions.
|
||||
|
||||
**Fix**:
|
||||
- Changed migration version from 7 to 11 (next available)
|
||||
- Added explicit schema_versions check to prevent unnecessary re-runs
|
||||
- Improved error propagation and documentation
|
||||
|
||||
### Upgrade Instructions
|
||||
|
||||
**If you're experiencing the error:**
|
||||
|
||||
Option 1 - Manual fix (preserves history):
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0; ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0;"
|
||||
```
|
||||
|
||||
Option 2 - Delete and recreate (loses history):
|
||||
```bash
|
||||
rm ~/.claude-mem/claude-mem.db
|
||||
# Restart Claude Code - database will recreate with correct schema
|
||||
```
|
||||
|
||||
Option 3 - Fresh install:
|
||||
Just upgrade to v6.0.7 and the migration will work correctly.
|
||||
|
||||
### Changes
|
||||
|
||||
- **Fixed**: Database migration version conflict (migration 7 → 11) (#121)
|
||||
- **Improved**: Migration error handling and schema_versions tracking
|
||||
|
||||
### Full Changelog
|
||||
|
||||
See [CHANGELOG.md](https://github.com/thedotmack/claude-mem/blob/main/CHANGELOG.md) for complete version history.
|
||||
|
||||
---
|
||||
|
||||
**Affected Users**: @liadtigloo @notmyself - this release fixes your reported issue. Please try one of the upgrade options above and let me know if the issue persists.
|
||||
|
||||
Thanks to everyone who reported this issue with detailed error logs! 🙏
|
||||
|
||||
## [6.0.6] - 2025-11-17
|
||||
|
||||
## Critical Bugfix Release
|
||||
|
||||
### Fixed
|
||||
- **Database Migration**: Fixed critical bug where `discovery_tokens` migration logic trusted `schema_versions` table without verifying actual column existence (#121)
|
||||
- Migration now always checks if columns exist before queries, preventing "no such column" errors
|
||||
- Safe for all users - auto-migrates on next Claude Code session without data loss
|
||||
|
||||
### Technical Details
|
||||
- Removed early return based on `schema_versions` check that could skip actual column verification
|
||||
- Migration now uses `PRAGMA table_info()` to verify column existence before every query
|
||||
- Ensures idempotent, safe schema migrations for SQLite databases
|
||||
|
||||
### Impact
|
||||
- Users experiencing "SqliteError: no such column: discovery_tokens" will be automatically fixed
|
||||
- No manual intervention or database backup required
|
||||
- Update to v6.0.6 via marketplace or `git pull` and restart Claude Code
|
||||
|
||||
**Affected Users**: All users who upgraded to v6.0.5 and experienced the migration error
|
||||
|
||||
## [6.0.5] - 2025-11-17
|
||||
|
||||
## Changes
|
||||
|
||||
### Automatic MCP Server Cleanup
|
||||
- Automatic cleanup of orphaned MCP server processes on worker startup
|
||||
- Self-healing maintenance runs on every worker restart
|
||||
- Prevents orphaned process accumulation and resource leaks
|
||||
|
||||
### Improvements
|
||||
- Removed manual cleanup notice from session context
|
||||
- Streamlined worker initialization process
|
||||
|
||||
## What's Fixed
|
||||
- Memory leaks from orphaned uvx/python processes are now prevented automatically
|
||||
- Workers self-heal on every restart without manual intervention
|
||||
|
||||
---
|
||||
|
||||
**Release Date**: November 16, 2025
|
||||
**Plugin Version**: 6.0.5
|
||||
|
||||
## [6.0.4] - 2025-11-17
|
||||
|
||||
**Patch Release**
|
||||
|
||||
Fixes memory leaks from orphaned uvx/python processes that could accumulate during ChromaDB operations.
|
||||
|
||||
**Changes:**
|
||||
- Fixed process cleanup in ChromaDB sync operations to prevent orphaned processes
|
||||
- Improved resource management for external process spawning
|
||||
|
||||
**Full Changelog:** https://github.com/thedotmack/claude-mem/compare/v6.0.3...v6.0.4
|
||||
|
||||
## [6.0.3] - 2025-11-16
|
||||
|
||||
## What's Changed
|
||||
|
||||
Documentation alignment release - merged PR #116 fixing hybrid search architecture documentation.
|
||||
|
||||
### Documentation Updates
|
||||
- Added comprehensive guide
|
||||
- Updated technical architecture documentation to reflect hybrid ChromaDB + SQLite + timeline context flow
|
||||
- Fixed skill operation guides to accurately describe semantic search capabilities
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v6.0.2...v6.0.3
|
||||
|
||||
## [6.0.2] - 2025-11-14
|
||||
|
||||
## Changes
|
||||
|
||||
- Updated user message hook with Claude-Mem community discussion link for better user engagement and support
|
||||
|
||||
## What's Changed
|
||||
- Enhanced startup context messaging with community connection information
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v6.0.1...v6.0.2
|
||||
|
||||
## [6.0.1] - 2025-11-14
|
||||
|
||||
## UI Enhancements
|
||||
|
||||
### Changes
|
||||
- Refined color theme with warmer tones for better visual hierarchy
|
||||
- New observation card blue/teal theme with distinct light/dark mode values
|
||||
- Added 8 SVG icon assets for summary card sections (thick and thin variants)
|
||||
- Enhanced summary card component with icon support for completed, investigated, learned, and next-steps sections
|
||||
- Updated build system to handle icon asset copying
|
||||
|
||||
### Visual Improvements
|
||||
- Unified color palette refinements across all UI components
|
||||
- Improved card type differentiation: gold/amber for summaries, purple for prompts, blue/teal for observations
|
||||
- Better visual consistency in viewer UI
|
||||
|
||||
Full changelog: https://github.com/thedotmack/claude-mem/compare/v6.0.0...v6.0.1
|
||||
|
||||
## [6.0.0] - 2025-11-13
|
||||
|
||||
## What's New
|
||||
|
||||
@@ -1,178 +1,38 @@
|
||||
/* To @claude: be vigilant about only leaving evergreen context in this file, claude-mem handles working context separately. */
|
||||
|
||||
# Claude-Mem: AI Development Instructions
|
||||
|
||||
## What This Project Is
|
||||
|
||||
Claude-mem is a Claude Code plugin providing persistent memory across sessions. It captures tool usage, compresses observations using the Claude Agent SDK, and injects relevant context into future sessions.
|
||||
|
||||
**Your Role**: You are working on the plugin itself. When users interact with Claude Code with this plugin installed, your observations get captured and become their persistent memory.
|
||||
**Current Version**: 6.3.2
|
||||
|
||||
**Current Version**: 6.0.1
|
||||
## Architecture
|
||||
|
||||
## IMPORTANT: Skills Are Auto-Invoked
|
||||
**5 Lifecycle Hooks**: SessionStart → UserPromptSubmit → PostToolUse → Summary → SessionEnd
|
||||
|
||||
**There is no `/skill` command.** Skills auto-invoke based on description metadata matching user queries. Don't document manual invocation (e.g., "Run `/skill troubleshoot`"). Instead: "The troubleshoot skill auto-activates when issues are detected."
|
||||
**Hooks** (`src/hooks/*.ts`) - TypeScript → ESM, built to `plugin/scripts/*-hook.js`
|
||||
|
||||
## Critical Architecture Knowledge
|
||||
**Worker Service** (`src/services/worker-service.ts`) - Express API on port 37777, PM2-managed, handles AI processing asynchronously
|
||||
|
||||
### The Lifecycle Flow
|
||||
**Database** (`src/services/sqlite/`) - SQLite3 at `~/.claude-mem/claude-mem.db` with FTS5 full-text search
|
||||
|
||||
1. **SessionStart** → smart-install.js runs first (pre-hook), then `context-hook.ts` runs
|
||||
- Smart installer checks dependencies (cached, only runs on version changes)
|
||||
- Starts PM2 worker if not healthy
|
||||
- Injects context from previous sessions (configurable observation count)
|
||||
**Search Skill** (`plugin/skills/mem-search/SKILL.md`) - HTTP API for searching past work, auto-invoked when users ask about history
|
||||
|
||||
2. **UserPromptSubmit** → `new-hook.ts` runs
|
||||
- Creates session record in SQLite
|
||||
- Saves raw user prompt for FTS5 search
|
||||
**Chroma** (`src/services/sync/ChromaSync.ts`) - Vector embeddings for semantic search
|
||||
|
||||
3. **PostToolUse** → `save-hook.ts` runs
|
||||
- Captures your tool executions
|
||||
- Sends to worker service for AI compression
|
||||
**Viewer UI** (`src/ui/viewer/`) - React interface at http://localhost:37777, built to `plugin/ui/viewer.html`
|
||||
|
||||
4. **Summary** → Summary hook generates session summaries
|
||||
## Build Commands
|
||||
|
||||
5. **SessionEnd** → `cleanup-hook.ts` runs
|
||||
- Marks session complete (graceful, not DELETE)
|
||||
- Skips on `/clear` to preserve ongoing sessions
|
||||
**Hooks only**: `npm run build && npm run sync-marketplace`
|
||||
|
||||
**Note**: smart-install.js is a pre-hook script (not a lifecycle hook). It's called before context-hook via command chaining in hooks.json and only runs when dependencies need updating.
|
||||
**Worker changes**: `npm run build && npm run sync-marketplace && npm run worker:restart`
|
||||
|
||||
### Key Components
|
||||
**Skills only**: `npm run sync-marketplace`
|
||||
|
||||
**Hooks** (`src/hooks/*.ts`)
|
||||
- Built to `plugin/scripts/*-hook.js` (ESM format)
|
||||
- Must output valid JSON to `hookSpecificOutput` field
|
||||
- Called by Claude Code lifecycle events
|
||||
|
||||
**Worker Service** (`src/services/worker-service.ts`)
|
||||
- Express.js API on port 37777 (configurable via `CLAUDE_MEM_WORKER_PORT`)
|
||||
- Managed by PM2 (auto-started by hooks)
|
||||
- Built to `plugin/worker-service.cjs` (CJS format)
|
||||
- Handles AI processing asynchronously to avoid hook timeouts
|
||||
|
||||
**Database** (`src/services/sqlite/`)
|
||||
- SQLite3 with better-sqlite3 (NOT bun:sqlite - that's legacy)
|
||||
- Location: `~/.claude-mem/claude-mem.db`
|
||||
- FTS5 virtual tables for full-text search
|
||||
- `SessionStore` = CRUD, `SessionSearch` = FTS5 queries
|
||||
|
||||
**Search Skill** (`plugin/skills/mem-search/SKILL.md`)
|
||||
- Provides access to all search functionality via HTTP API + skill
|
||||
- Auto-invoked when users ask about past work, decisions, or history
|
||||
- Uses HTTP endpoints instead of MCP tools (~2,250 token savings per session)
|
||||
- 10 search operations: observations, sessions, prompts, by-type, by-file, by-concept, timelines, etc.
|
||||
|
||||
**Chroma Vector Database** (`src/services/sync/ChromaSync.ts`)
|
||||
- Hybrid semantic + keyword search architecture
|
||||
- Automatic vector embedding synchronization
|
||||
- 90-day recency filtering for relevant results
|
||||
- Combined with SQLite FTS5 for optimal search performance
|
||||
|
||||
**Viewer UI** (`src/ui/viewer/`)
|
||||
- React + TypeScript web interface accessible at http://localhost:37777
|
||||
- Real-time memory stream visualization via Server-Sent Events (SSE)
|
||||
- Infinite scroll pagination for observations, sessions, and user prompts
|
||||
- Project filtering and settings persistence
|
||||
- Built to `plugin/ui/viewer.html` (self-contained bundle via esbuild)
|
||||
- Auto-reconnection and error recovery
|
||||
|
||||
## How to Make Changes
|
||||
|
||||
### When You Modify Hooks
|
||||
```bash
|
||||
npm run build
|
||||
npm run sync-marketplace
|
||||
```
|
||||
Changes take effect on next Claude Code session. No worker restart needed.
|
||||
|
||||
### When You Modify Worker Service
|
||||
```bash
|
||||
npm run build
|
||||
npm run sync-marketplace
|
||||
npm run worker:restart
|
||||
```
|
||||
Must restart PM2 worker for changes to take effect.
|
||||
|
||||
### When You Modify Search Skill
|
||||
```bash
|
||||
npm run sync-marketplace
|
||||
```
|
||||
Skill changes take effect immediately on next Claude Code session. No build or restart needed (skills are markdown).
|
||||
|
||||
### When You Modify Viewer UI
|
||||
```bash
|
||||
npm run build
|
||||
npm run sync-marketplace
|
||||
npm run worker:restart
|
||||
```
|
||||
Changes to React components, styles, or viewer logic require rebuilding and restarting the worker. Refresh browser to see changes.
|
||||
|
||||
### Build Pipeline
|
||||
1. `npm run build` → Compiles TypeScript, outputs to `plugin/`
|
||||
2. `npm run sync-marketplace` → Syncs to `~/.claude/plugins/marketplaces/thedotmack/`
|
||||
3. Changes are live for next session (hooks/skills) or after restart (worker)
|
||||
|
||||
## Coding Standards
|
||||
|
||||
**Philosophy**: Write the dumb, obvious thing first. Add complexity only when you hit the problem.
|
||||
|
||||
**Key Principles:**
|
||||
1. **YAGNI**: Don't build it until you need it
|
||||
2. **DRY**: Extract patterns after second duplication, not before
|
||||
3. **Fail Fast**: Explicit errors beat silent failures
|
||||
4. **Simple First**: Write the obvious solution, optimize only if needed
|
||||
5. **Delete Aggressively**: Less code = fewer bugs
|
||||
|
||||
**Common anti-patterns to avoid:**
|
||||
- Ceremonial wrapper functions for constants (just export the constant)
|
||||
- Unused default parameters (remove if never used)
|
||||
- Magic numbers without named constants
|
||||
- Silent failures instead of explicit errors
|
||||
- Fragile string parsing (use structured JSON output)
|
||||
- Copy-pasted promise wrappers (extract helper functions)
|
||||
- Overengineered "defensive" code for problems you don't have
|
||||
|
||||
## Common Tasks
|
||||
|
||||
### Adding a New Hook
|
||||
1. Create `src/hooks/new-hook.ts`
|
||||
2. Add to `scripts/build-hooks.js` build list
|
||||
3. Add configuration to `plugin/hooks/hooks.json`
|
||||
4. Build and sync: `npm run build && npm run sync-marketplace`
|
||||
|
||||
**Note**: smart-install.js is not a hook - it's a pre-hook dependency checker that runs before context-hook via command chaining.
|
||||
|
||||
### Modifying Database Schema
|
||||
1. Update schema in `src/services/sqlite/schema.ts`
|
||||
2. Update SessionStore/SessionSearch classes
|
||||
3. Migration strategy: The plugin currently recreates on schema changes (acceptable for alpha)
|
||||
4. TODO: Add proper migrations for production
|
||||
|
||||
### Debugging Worker Issues
|
||||
```bash
|
||||
pm2 list # Check worker status
|
||||
npm run worker:logs # View logs
|
||||
npm run worker:restart # Restart if needed
|
||||
pm2 delete claude-mem-worker # Force clean start
|
||||
```
|
||||
|
||||
### Testing Changes Locally
|
||||
1. Make changes in `src/`
|
||||
2. `npm run build && npm run sync-marketplace`
|
||||
3. Start new Claude Code session (hooks) or restart worker (worker changes)
|
||||
4. Check `~/.claude-mem/claude-mem.db` for database state
|
||||
5. Use mem-search skill to verify behavior (auto-invoked when asking about past work)
|
||||
|
||||
### Version Bumps
|
||||
Use the `version-bump` skill (auto-invokes when requesting version updates). It handles:
|
||||
- Semantic version increments (patch/minor/major)
|
||||
- Updates all version references (package.json, plugin.json, CLAUDE.md, marketplace.json)
|
||||
- Creates git tags and GitHub releases
|
||||
- Auto-generates CHANGELOG.md from releases
|
||||
|
||||
## Investigation Best Practices
|
||||
|
||||
When investigations fail persistently, use Task agents for comprehensive file analysis instead of repeated grep/search. Deploy agents to read full files and answer specific questions - more efficient than multiple rounds of searching.
|
||||
**Viewer UI**: `npm run build && npm run sync-marketplace && npm run worker:restart`
|
||||
|
||||
## Environment Variables
|
||||
|
||||
@@ -180,37 +40,24 @@ When investigations fail persistently, use Task agents for comprehensive file an
|
||||
- `CLAUDE_MEM_CONTEXT_OBSERVATIONS` - Observations injected at SessionStart (default: 50)
|
||||
- `CLAUDE_MEM_WORKER_PORT` - Worker service port (default: 37777)
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
### Why PM2 Instead of Direct Process
|
||||
Hooks have strict timeout limits. PM2 manages a persistent background worker, allowing AI processing to continue after hooks complete.
|
||||
|
||||
### Why SQLite FTS5
|
||||
Enables instant full-text search across thousands of observations without external dependencies. Automatic sync triggers keep FTS5 tables synchronized.
|
||||
|
||||
### Why Graceful Cleanup
|
||||
Changed from aggressive DELETE requests to marking sessions complete. Prevents interrupting summary generation and other async operations.
|
||||
|
||||
### Why Smart Install Caching
|
||||
npm install is expensive (2-5s). Caching version state and only installing on changes makes SessionStart nearly instant (10ms).
|
||||
|
||||
### Why Web-Based Viewer UI
|
||||
Real-time visibility into memory stream helps users understand what's being captured and how context is being built. SSE provides instant updates without polling. Self-contained HTML bundle (esbuild) eliminates deployment complexity - everything served from a single file.
|
||||
|
||||
## File Locations
|
||||
|
||||
**Source**: `<project-root>/src/` - TypeScript source files
|
||||
**Built Plugin**: `<project-root>/plugin/` - Compiled JavaScript outputs
|
||||
**Installed Plugin**: `~/.claude/plugins/marketplaces/thedotmack/` - User's installed plugin location
|
||||
**Database**: `~/.claude-mem/claude-mem.db` - SQLite database with observations, sessions, summaries
|
||||
**Chroma Database**: `~/.claude-mem/chroma/` - Vector embeddings for semantic search
|
||||
**Usage Logs**: `~/.claude-mem/usage-logs/usage-YYYY-MM-DD.jsonl` - Daily API usage tracking
|
||||
- **Source**: `<project-root>/src/`
|
||||
- **Built Plugin**: `<project-root>/plugin/`
|
||||
- **Installed Plugin**: `~/.claude/plugins/marketplaces/thedotmack/`
|
||||
- **Database**: `~/.claude-mem/claude-mem.db`
|
||||
- **Chroma**: `~/.claude-mem/chroma/`
|
||||
- **Usage Logs**: `~/.claude-mem/usage-logs/usage-YYYY-MM-DD.jsonl`
|
||||
|
||||
## Quick Reference
|
||||
|
||||
**Build**: `npm run build`
|
||||
**Sync**: `npm run sync-marketplace`
|
||||
**Worker Restart**: `npm run worker:restart`
|
||||
**Worker Logs**: `npm run worker:logs`
|
||||
**Usage Analysis**: `npm run usage:today`
|
||||
**Viewer UI**: http://localhost:37777 (auto-starts with worker)
|
||||
```bash
|
||||
npm run build # Compile TypeScript
|
||||
npm run sync-marketplace # Copy to ~/.claude/plugins
|
||||
npm run worker:restart # Restart PM2 worker
|
||||
npm run worker:logs # View worker logs
|
||||
pm2 list # Check worker status
|
||||
pm2 delete claude-mem-worker # Force clean start
|
||||
```
|
||||
|
||||
**Viewer UI**: http://localhost:37777
|
||||
|
||||
@@ -1,614 +0,0 @@
|
||||
# Implementation Plan: ROI Metrics & Discovery Cost Tracking
|
||||
|
||||
**Feature**: Display token discovery costs alongside observations to demonstrate knowledge reuse ROI
|
||||
**Branch**: `enhancement/roi`
|
||||
**Issue**: #104
|
||||
**Priority**: HIGH (needed for YC application amendment)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Capture token usage from Agent SDK, store as "discovery cost" with each observation, and display metrics in SessionStart context to prove that claude-mem reduces token consumption by 50-75% through knowledge reuse.
|
||||
|
||||
### The Value Proposition
|
||||
|
||||
**Session 1**: Claude spends 4,000 tokens discovering "how Stop hooks work"
|
||||
**Sessions 2-5**: Claude reads 163-token observation instead of re-discovering
|
||||
**Savings**: 15,348 tokens (77% reduction) over 5 sessions
|
||||
|
||||
This feature makes that ROI **visible and measurable** for both users and Claude.
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
Agent SDK Messages (with usage)
|
||||
↓
|
||||
SDKAgent captures usage data
|
||||
↓
|
||||
ActiveSession tracks cumulative tokens
|
||||
↓
|
||||
Observations stored with discovery_tokens
|
||||
↓
|
||||
Context hook displays metrics
|
||||
↓
|
||||
User/Claude sees ROI
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Capture Token Usage from Agent SDK
|
||||
|
||||
**File**: `src/services/worker/SDKAgent.ts`
|
||||
|
||||
**Changes**:
|
||||
1. Extract usage data from assistant messages (lines 64-86)
|
||||
2. Track cumulative session tokens in ActiveSession
|
||||
3. Pass cumulative tokens when storing observations
|
||||
|
||||
**Code Changes**:
|
||||
|
||||
```typescript
|
||||
// Line ~70: After extracting textContent, add:
|
||||
const usage = message.message.usage;
|
||||
if (usage) {
|
||||
session.cumulativeInputTokens += usage.input_tokens || 0;
|
||||
session.cumulativeOutputTokens += usage.output_tokens || 0;
|
||||
|
||||
// Cache creation counts as discovery, cache read doesn't
|
||||
if (usage.cache_creation_input_tokens) {
|
||||
session.cumulativeInputTokens += usage.cache_creation_input_tokens;
|
||||
}
|
||||
|
||||
logger.debug('SDK', 'Token usage captured', {
|
||||
sessionId: session.sessionDbId,
|
||||
inputTokens: usage.input_tokens,
|
||||
outputTokens: usage.output_tokens,
|
||||
cumulativeInput: session.cumulativeInputTokens,
|
||||
cumulativeOutput: session.cumulativeOutputTokens
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
```typescript
|
||||
// Line ~213-218: Pass discovery tokens when storing
|
||||
const { id: obsId, createdAtEpoch } = this.dbManager.getSessionStore().storeObservation(
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
obs,
|
||||
session.lastPromptNumber,
|
||||
session.cumulativeInputTokens + session.cumulativeOutputTokens // Add discovery cost
|
||||
);
|
||||
```
|
||||
|
||||
**Edge Cases**:
|
||||
- Handle missing usage data (default to 0)
|
||||
- Cache tokens: `cache_creation_input_tokens` counts as discovery, `cache_read_input_tokens` doesn't
|
||||
- Multiple observations per response: Each gets snapshot of cumulative tokens at creation time
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Update ActiveSession Type
|
||||
|
||||
**File**: `src/services/worker-types.ts`
|
||||
|
||||
**Changes**: Add token tracking fields to ActiveSession interface
|
||||
|
||||
```typescript
|
||||
export interface ActiveSession {
|
||||
sessionDbId: number;
|
||||
sdkSessionId: string | null;
|
||||
claudeSessionId: string;
|
||||
project: string;
|
||||
userPrompt: string;
|
||||
lastPromptNumber: number;
|
||||
pendingMessages: PendingMessage[];
|
||||
abortController: AbortController;
|
||||
startTime: number;
|
||||
cumulativeInputTokens: number; // NEW: Track input tokens
|
||||
cumulativeOutputTokens: number; // NEW: Track output tokens
|
||||
}
|
||||
```
|
||||
|
||||
**Initialization**: When creating new session in SessionManager.initializeSession, set:
|
||||
```typescript
|
||||
cumulativeInputTokens: 0,
|
||||
cumulativeOutputTokens: 0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Database Schema Migration
|
||||
|
||||
**File**: `src/services/sqlite/migrations.ts`
|
||||
|
||||
**Add Migration**: Create migration #8 (next available number)
|
||||
|
||||
```typescript
|
||||
{
|
||||
version: 8,
|
||||
name: 'add_discovery_tokens',
|
||||
up: (db: Database) => {
|
||||
// Add discovery_tokens to observations
|
||||
db.exec(`
|
||||
ALTER TABLE observations
|
||||
ADD COLUMN discovery_tokens INTEGER DEFAULT 0;
|
||||
`);
|
||||
|
||||
// Add discovery_tokens to summaries
|
||||
db.exec(`
|
||||
ALTER TABLE summaries
|
||||
ADD COLUMN discovery_tokens INTEGER DEFAULT 0;
|
||||
`);
|
||||
|
||||
logger.info('DB', 'Migration 8: Added discovery_tokens columns');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Why summaries too?**: Summaries represent accumulated session work, so they should also show total discovery cost.
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Update SessionStore
|
||||
|
||||
**File**: `src/services/sqlite/SessionStore.ts`
|
||||
|
||||
**Changes**:
|
||||
|
||||
1. Update `storeObservation` signature (around line ~1000):
|
||||
```typescript
|
||||
storeObservation(
|
||||
sessionId: string,
|
||||
project: string,
|
||||
observation: ParsedObservation,
|
||||
promptNumber: number,
|
||||
discoveryTokens: number = 0 // NEW parameter
|
||||
): { id: number; createdAtEpoch: number }
|
||||
```
|
||||
|
||||
2. Update INSERT statement to include discovery_tokens:
|
||||
```typescript
|
||||
const stmt = this.db.prepare(`
|
||||
INSERT INTO observations (
|
||||
session_id,
|
||||
project,
|
||||
type,
|
||||
title,
|
||||
subtitle,
|
||||
narrative,
|
||||
facts,
|
||||
concepts,
|
||||
files_read,
|
||||
files_modified,
|
||||
prompt_number,
|
||||
discovery_tokens, -- NEW
|
||||
created_at_epoch
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
const result = stmt.run(
|
||||
sessionId,
|
||||
project,
|
||||
observation.type,
|
||||
observation.title,
|
||||
observation.subtitle || '',
|
||||
observation.narrative || '',
|
||||
JSON.stringify(observation.facts || []),
|
||||
JSON.stringify(observation.concepts || []),
|
||||
JSON.stringify(observation.files || []),
|
||||
JSON.stringify([]),
|
||||
promptNumber,
|
||||
discoveryTokens, // NEW
|
||||
createdAtEpoch
|
||||
);
|
||||
```
|
||||
|
||||
3. Update `storeSummary` similarly (around line ~1150):
|
||||
```typescript
|
||||
storeSummary(
|
||||
sessionId: string,
|
||||
project: string,
|
||||
summary: ParsedSummary,
|
||||
promptNumber: number,
|
||||
discoveryTokens: number = 0 // NEW parameter
|
||||
): { id: number; createdAtEpoch: number }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Update Database Types
|
||||
|
||||
**File**: `src/services/sqlite/types.ts`
|
||||
|
||||
**Changes**: Add discovery_tokens to DBObservation and DBSummary interfaces
|
||||
|
||||
```typescript
|
||||
export interface DBObservation {
|
||||
id: number;
|
||||
session_id: string;
|
||||
project: string;
|
||||
type: 'decision' | 'bugfix' | 'feature' | 'refactor' | 'discovery' | 'change';
|
||||
title: string;
|
||||
subtitle: string;
|
||||
narrative: string | null;
|
||||
facts: string; // JSON array
|
||||
concepts: string; // JSON array
|
||||
files_read: string; // JSON array
|
||||
files_modified: string; // JSON array
|
||||
prompt_number: number;
|
||||
discovery_tokens: number; // NEW
|
||||
created_at_epoch: number;
|
||||
}
|
||||
|
||||
export interface DBSummary {
|
||||
id: number;
|
||||
session_id: string;
|
||||
request: string;
|
||||
investigated: string | null;
|
||||
learned: string | null;
|
||||
completed: string | null;
|
||||
next_steps: string | null;
|
||||
notes: string | null;
|
||||
project: string;
|
||||
prompt_number: number;
|
||||
discovery_tokens: number; // NEW
|
||||
created_at_epoch: number;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 6: Update Search Queries
|
||||
|
||||
**File**: `src/services/sqlite/SessionSearch.ts`
|
||||
|
||||
**Changes**: Ensure all SELECT queries include discovery_tokens
|
||||
|
||||
Example (around line ~50, searchObservations):
|
||||
```typescript
|
||||
SELECT
|
||||
o.id,
|
||||
o.session_id,
|
||||
o.project,
|
||||
o.type,
|
||||
o.title,
|
||||
o.subtitle,
|
||||
o.narrative,
|
||||
o.facts,
|
||||
o.concepts,
|
||||
o.files_read,
|
||||
o.files_modified,
|
||||
o.prompt_number,
|
||||
o.discovery_tokens, -- NEW
|
||||
o.created_at_epoch,
|
||||
...
|
||||
```
|
||||
|
||||
**Affected methods**:
|
||||
- `searchObservations`
|
||||
- `getRecentObservations`
|
||||
- `getObservationsByType`
|
||||
- `getObservationsByConcept`
|
||||
- `getObservationsByFile`
|
||||
- All other observation query methods
|
||||
|
||||
---
|
||||
|
||||
### Phase 7: Update Context Hook Display
|
||||
|
||||
**File**: `src/hooks/context-hook.ts`
|
||||
|
||||
**Changes**: Display discovery costs and ROI metrics in SessionStart context
|
||||
|
||||
**Section 1: Add Aggregate Metrics** (insert after line ~250, before observation table)
|
||||
|
||||
```typescript
|
||||
// Calculate aggregate metrics
|
||||
const totalObservations = observations.length;
|
||||
const totalReadTokens = observations.reduce((sum, obs) => {
|
||||
// Estimate read tokens from observation size
|
||||
const obsSize = (obs.title?.length || 0) +
|
||||
(obs.subtitle?.length || 0) +
|
||||
(obs.narrative?.length || 0) +
|
||||
JSON.stringify(obs.facts || []).length;
|
||||
return sum + Math.ceil(obsSize / 4); // Rough token estimate
|
||||
}, 0);
|
||||
const totalDiscoveryTokens = observations.reduce((sum, obs) => sum + (obs.discovery_tokens || 0), 0);
|
||||
const savings = totalDiscoveryTokens - totalReadTokens;
|
||||
const savingsPercent = totalDiscoveryTokens > 0
|
||||
? Math.round((savings / totalDiscoveryTokens) * 100)
|
||||
: 0;
|
||||
|
||||
output += `\n📊 **Context Economics**:\n`;
|
||||
output += ` - Loading: ${totalObservations} observations (${totalReadTokens.toLocaleString()} tokens to read)\n`;
|
||||
output += ` - Discovery investment: ${totalDiscoveryTokens.toLocaleString()} tokens spent by previous sessions\n`;
|
||||
output += ` - Your savings: ${savings.toLocaleString()} tokens (${savingsPercent}% reduction from reuse)\n\n`;
|
||||
```
|
||||
|
||||
**Section 2: Update Table Format** (around line ~280)
|
||||
|
||||
```typescript
|
||||
output += `| ID | Time | T | Title | Read | Discovery |\n`;
|
||||
output += `|----|------|---|-------|------|-----------||\n`;
|
||||
|
||||
for (const obs of observations) {
|
||||
const time = formatTime(obs.created_at_epoch);
|
||||
const emoji = getTypeEmoji(obs.type);
|
||||
const title = truncate(obs.title, 50);
|
||||
|
||||
// Estimate read tokens (observation size in tokens)
|
||||
const obsSize = (obs.title?.length || 0) +
|
||||
(obs.subtitle?.length || 0) +
|
||||
(obs.narrative?.length || 0) +
|
||||
JSON.stringify(obs.facts || []).length;
|
||||
const readTokens = Math.ceil(obsSize / 4);
|
||||
|
||||
const discoveryTokens = obs.discovery_tokens || 0;
|
||||
const discoveryDisplay = discoveryTokens > 0
|
||||
? `🔍 ${discoveryTokens.toLocaleString()}`
|
||||
: '-';
|
||||
|
||||
output += `| #${obs.id} | ${time} | ${emoji} | ${title} | ~${readTokens} | ${discoveryDisplay} |\n`;
|
||||
}
|
||||
```
|
||||
|
||||
**Section 3: Add Footer Explanation** (after table)
|
||||
|
||||
```typescript
|
||||
output += `\n💡 **Column Key**:\n`;
|
||||
output += ` - **Read**: Tokens to read this observation (cost to learn it now)\n`;
|
||||
output += ` - **Discovery**: Tokens Previous Claude spent exploring/researching this topic\n`;
|
||||
output += `\n**ROI**: Reading these learnings instead of re-discovering saves ${savingsPercent}% tokens\n`;
|
||||
```
|
||||
|
||||
**Edge Case**: Handle old observations without discovery_tokens (show '-' or 0)
|
||||
|
||||
---
|
||||
|
||||
### Phase 8: Update Chroma Sync (Optional)
|
||||
|
||||
**File**: `src/services/sync/ChromaSync.ts`
|
||||
|
||||
**Changes**: Include discovery_tokens in vector metadata
|
||||
|
||||
```typescript
|
||||
// Around line ~100, syncObservation metadata
|
||||
metadata: {
|
||||
session_id: sessionId,
|
||||
project: project,
|
||||
type: observation.type,
|
||||
title: observation.title,
|
||||
prompt_number: promptNumber,
|
||||
discovery_tokens: discoveryTokens, // NEW
|
||||
created_at_epoch: createdAtEpoch,
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
**Why?**: Enables semantic search to factor in discovery cost for relevance scoring (future enhancement)
|
||||
|
||||
---
|
||||
|
||||
## Testing Plan
|
||||
|
||||
### Unit Tests
|
||||
|
||||
1. **Token Capture Test**:
|
||||
- Mock Agent SDK response with usage data
|
||||
- Verify ActiveSession.cumulativeTokens increments correctly
|
||||
- Test cache token handling (creation counts, read doesn't)
|
||||
|
||||
2. **Storage Test**:
|
||||
- Create observation with discovery_tokens
|
||||
- Verify database stores correctly
|
||||
- Query back and verify field present
|
||||
|
||||
3. **Display Test**:
|
||||
- Create test observations with varying discovery costs
|
||||
- Run context-hook
|
||||
- Verify metrics calculate correctly
|
||||
- Verify table displays both Read and Discovery columns
|
||||
|
||||
### Integration Tests
|
||||
|
||||
1. **Full Session Flow**:
|
||||
- Start new session
|
||||
- Trigger multiple tool executions
|
||||
- Generate observations
|
||||
- Verify cumulative tokens accumulate
|
||||
- Check context displays metrics
|
||||
|
||||
2. **Migration Test**:
|
||||
- Backup existing database
|
||||
- Run migration #8
|
||||
- Verify columns added
|
||||
- Verify existing data intact (discovery_tokens = 0)
|
||||
- Test new observations store correctly
|
||||
|
||||
### Manual Testing
|
||||
|
||||
1. **Real Usage Scenario**:
|
||||
- Start fresh Claude Code session
|
||||
- Perform research task (read files, search codebase)
|
||||
- Generate observations via claude-mem
|
||||
- Check database for discovery_tokens values
|
||||
- Start new session, verify context shows metrics
|
||||
|
||||
2. **YC Demo Data**:
|
||||
- Run 5 sessions on same topic
|
||||
- Collect token data for each session
|
||||
- Calculate actual ROI (Session 1 cost vs Sessions 2-5)
|
||||
- Screenshot metrics for YC application
|
||||
|
||||
---
|
||||
|
||||
## Rollout Plan
|
||||
|
||||
### Phase 1: Data Collection (Week 1)
|
||||
- Deploy migration and token capture
|
||||
- Run without displaying metrics yet
|
||||
- Verify data quality and accuracy
|
||||
- Fix any issues with token tracking
|
||||
|
||||
### Phase 2: Display Metrics (Week 2)
|
||||
- Enable context hook display
|
||||
- Gather user feedback
|
||||
- Iterate on presentation format
|
||||
- Document any edge cases
|
||||
|
||||
### Phase 3: YC Application (Week 2-3)
|
||||
- Collect empirical data from real usage
|
||||
- Generate charts/graphs showing ROI
|
||||
- Write case study with actual numbers
|
||||
- Amend YC application with proof
|
||||
|
||||
### Phase 4: Public Launch (Week 4)
|
||||
- Blog post explaining the feature
|
||||
- Update README with ROI metrics
|
||||
- Submit to HN/Reddit with data
|
||||
- Reach out to Anthropic with findings
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
**Technical Success**:
|
||||
- ✅ Token capture accuracy: >95% of SDK responses captured
|
||||
- ✅ Database migration: 0 data loss, all observations migrated
|
||||
- ✅ Display accuracy: Metrics match raw data within 5%
|
||||
|
||||
**Business Success**:
|
||||
- ✅ Demonstrate 50-75% token reduction across 10+ sessions
|
||||
- ✅ YC application strengthened with empirical data
|
||||
- ✅ User/Claude understanding of ROI improves (survey/feedback)
|
||||
|
||||
**Strategic Success**:
|
||||
- ✅ Proof that memory optimization reduces infrastructure needs
|
||||
- ✅ Data compelling enough for Anthropic partnership discussion
|
||||
- ✅ Foundation for enterprise licensing ROI calculator
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Token Attribution**:
|
||||
- Should each observation get cumulative session tokens, or split proportionally?
|
||||
- **Decision**: Use cumulative (simpler, shows total cost at that point)
|
||||
|
||||
2. **Cache Tokens**:
|
||||
- How to handle cache_read_input_tokens in ROI calculation?
|
||||
- **Decision**: Don't count cache reads as discovery (they're already discovered)
|
||||
|
||||
3. **Display Format**:
|
||||
- Show raw token counts or human-readable format (K, M)?
|
||||
- **Decision**: Use toLocaleString() for readability (e.g., "4,000" not "4K")
|
||||
|
||||
4. **Pricing Display**:
|
||||
- Should we show dollar costs too, or just tokens?
|
||||
- **Decision**: Tokens only initially. Pricing varies by model/plan, adds complexity
|
||||
|
||||
5. **Historical Data**:
|
||||
- What to do with old observations without discovery_tokens?
|
||||
- **Decision**: Show as 0 or '-', document limitation
|
||||
|
||||
---
|
||||
|
||||
## Files Modified Summary
|
||||
|
||||
**Core Implementation**:
|
||||
- `src/services/worker/SDKAgent.ts` - Capture usage, pass to storage
|
||||
- `src/services/worker-types.ts` - Add cumulative token fields
|
||||
- `src/services/sqlite/migrations.ts` - Migration #8 for discovery_tokens
|
||||
- `src/services/sqlite/SessionStore.ts` - Store discovery tokens
|
||||
- `src/services/sqlite/types.ts` - Update interfaces
|
||||
- `src/services/sqlite/SessionSearch.ts` - Include in queries
|
||||
- `src/hooks/context-hook.ts` - Display metrics
|
||||
|
||||
**Optional**:
|
||||
- `src/services/sync/ChromaSync.ts` - Include in vector metadata
|
||||
- `src/services/worker/SessionManager.ts` - Initialize cumulative tokens
|
||||
|
||||
**Documentation**:
|
||||
- `CLAUDE.md` - Update with new feature
|
||||
- `README.md` - Add ROI metrics section
|
||||
- Issue #104 - Track implementation progress
|
||||
|
||||
---
|
||||
|
||||
## Timeline Estimate
|
||||
|
||||
**Day 1** (Tomorrow):
|
||||
- [ ] Create branch ✅
|
||||
- [ ] Write implementation plan ✅
|
||||
- [ ] Phase 1: Capture token usage (2 hours)
|
||||
- [ ] Phase 2: Update types (30 min)
|
||||
- [ ] Phase 3: Database migration (1 hour)
|
||||
|
||||
**Day 2**:
|
||||
- [ ] Phase 4: Update SessionStore (1 hour)
|
||||
- [ ] Phase 5: Update types (30 min)
|
||||
- [ ] Phase 6: Update search queries (1 hour)
|
||||
- [ ] Testing: Unit tests (2 hours)
|
||||
|
||||
**Day 3**:
|
||||
- [ ] Phase 7: Update context hook display (2 hours)
|
||||
- [ ] Testing: Integration tests (2 hours)
|
||||
- [ ] Manual testing and iteration (2 hours)
|
||||
|
||||
**Day 4**:
|
||||
- [ ] Collect real usage data (ongoing throughout day)
|
||||
- [ ] Generate YC metrics/charts (2 hours)
|
||||
- [ ] Amend YC application (2 hours)
|
||||
- [ ] Documentation updates (1 hour)
|
||||
|
||||
**Total**: ~20 hours of development over 4 days
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
**Risk 1**: Agent SDK usage data incomplete or missing
|
||||
**Mitigation**: Default to 0, log warnings, don't break existing functionality
|
||||
|
||||
**Risk 2**: Migration fails on large databases
|
||||
**Mitigation**: Test on database copy first, add rollback mechanism
|
||||
|
||||
**Risk 3**: Token estimates inaccurate
|
||||
**Mitigation**: Document methodology, provide "rough estimate" disclaimer
|
||||
|
||||
**Risk 4**: Display too noisy/overwhelming
|
||||
**Mitigation**: Make display configurable via settings, start collapsed
|
||||
|
||||
**Risk 5**: YC data not compelling enough
|
||||
**Mitigation**: Run on diverse projects, cherry-pick best examples, be honest about limitations
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ✅ Create branch `enhancement/roi`
|
||||
2. ✅ Write implementation plan
|
||||
3. Start Phase 1: Implement token capture in SDKAgent.ts
|
||||
4. Run manual test to verify usage data captured
|
||||
5. Continue through phases sequentially
|
||||
6. Collect data for YC application by end of week
|
||||
|
||||
---
|
||||
|
||||
## Notes for Tomorrow
|
||||
|
||||
**Start here**: `src/services/worker/SDKAgent.ts` line 64-86
|
||||
**Key insight**: `message.message.usage` contains the token data
|
||||
**Don't forget**: Initialize cumulative tokens to 0 in SessionManager
|
||||
**Test with**: Simple session that reads a few files and creates 1-2 observations
|
||||
|
||||
**The goal**: By end of week, have real numbers showing 50-75% token savings to prove the hypothesis and strengthen YC application.
|
||||
|
||||
---
|
||||
|
||||
*This plan represents ~20 hours of focused development. Prioritize getting Phase 1-7 working correctly over perfection. The YC data is the critical deliverable.*
|
||||
@@ -17,7 +17,7 @@
|
||||
<img src="https://img.shields.io/badge/License-AGPL%203.0-blue.svg" alt="License">
|
||||
</a>
|
||||
<a href="package.json">
|
||||
<img src="https://img.shields.io/badge/version-6.0.0-green.svg" alt="Version">
|
||||
<img src="https://img.shields.io/badge/version-6.3.0-green.svg" alt="Version">
|
||||
</a>
|
||||
<a href="package.json">
|
||||
<img src="https://img.shields.io/badge/node-%3E%3D18.0.0-brightgreen.svg" alt="Node">
|
||||
@@ -73,6 +73,7 @@ Restart Claude Code. Context from previous sessions will automatically appear in
|
||||
- 🖥️ **Web Viewer UI** - Real-time memory stream at http://localhost:37777
|
||||
- 🤖 **Automatic Operation** - No manual intervention required
|
||||
- 🔗 **Citations** - Reference past decisions with `claude-mem://` URIs
|
||||
- 🧪 **Beta Channel** - Try experimental features like Endless Mode via version switching
|
||||
|
||||
---
|
||||
|
||||
@@ -89,31 +90,31 @@ npx mintlify dev
|
||||
|
||||
### Getting Started
|
||||
|
||||
- **[Installation Guide](docs/installation.mdx)** - Quick start & advanced installation
|
||||
- **[Usage Guide](docs/usage/getting-started.mdx)** - How Claude-Mem works automatically
|
||||
- **[Search Tools](docs/usage/search-tools.mdx)** - Query your project history with natural language
|
||||
- **[Installation Guide](https://docs.claude-mem.ai/installation)** - Quick start & advanced installation
|
||||
- **[Usage Guide](https://docs.claude-mem.ai/usage/getting-started)** - How Claude-Mem works automatically
|
||||
- **[Search Tools](https://docs.claude-mem.ai/usage/search-tools)** - Query your project history with natural language
|
||||
- **[Beta Features](https://docs.claude-mem.ai/beta-features)** - Try experimental features like Endless Mode
|
||||
|
||||
### Best Practices
|
||||
|
||||
- **[Context Engineering](docs/context-engineering.mdx)** - AI agent context optimization principles
|
||||
- **[Progressive Disclosure](docs/progressive-disclosure.mdx)** - Philosophy behind Claude-Mem's context priming strategy
|
||||
- **[Context Engineering](https://docs.claude-mem.ai/context-engineering)** - AI agent context optimization principles
|
||||
- **[Progressive Disclosure](https://docs.claude-mem.ai/progressive-disclosure)** - Philosophy behind Claude-Mem's context priming strategy
|
||||
|
||||
### Architecture
|
||||
|
||||
- **[Overview](docs/architecture/overview.mdx)** - System components & data flow
|
||||
- **[Architecture Evolution](docs/architecture-evolution.mdx)** - The journey from v3 to v5
|
||||
- **[Hooks Architecture](docs/hooks-architecture.mdx)** - How Claude-Mem uses lifecycle hooks
|
||||
- **[Hooks Reference](docs/architecture/hooks.mdx)** - 7 hook scripts explained
|
||||
- **[Worker Service](docs/architecture/worker-service.mdx)** - HTTP API & PM2 management
|
||||
- **[Database](docs/architecture/database.mdx)** - SQLite schema & FTS5 search
|
||||
- **[MCP Search](docs/architecture/mcp-search.mdx)** - 9 search tools & examples
|
||||
- **[Viewer UI](docs/VIEWER.md)** - Web-based memory stream visualization
|
||||
- **[Overview](https://docs.claude-mem.ai/architecture/overview)** - System components & data flow
|
||||
- **[Architecture Evolution](https://docs.claude-mem.ai/architecture-evolution)** - The journey from v3 to v5
|
||||
- **[Hooks Architecture](https://docs.claude-mem.ai/hooks-architecture)** - How Claude-Mem uses lifecycle hooks
|
||||
- **[Hooks Reference](https://docs.claude-mem.ai/architecture/hooks)** - 7 hook scripts explained
|
||||
- **[Worker Service](https://docs.claude-mem.ai/architecture/worker-service)** - HTTP API & PM2 management
|
||||
- **[Database](https://docs.claude-mem.ai/architecture/database)** - SQLite schema & FTS5 search
|
||||
- **[Search Architecture](https://docs.claude-mem.ai/architecture/search-architecture)** - Hybrid search with Chroma vector database
|
||||
|
||||
### Configuration & Development
|
||||
|
||||
- **[Configuration](docs/configuration.mdx)** - Environment variables & settings
|
||||
- **[Development](docs/development.mdx)** - Building, testing, contributing
|
||||
- **[Troubleshooting](docs/troubleshooting.mdx)** - Common issues & solutions
|
||||
- **[Configuration](https://docs.claude-mem.ai/configuration)** - Environment variables & settings
|
||||
- **[Development](https://docs.claude-mem.ai/development)** - Building, testing, contributing
|
||||
- **[Troubleshooting](https://docs.claude-mem.ai/troubleshooting)** - Common issues & solutions
|
||||
|
||||
---
|
||||
|
||||
@@ -150,7 +151,7 @@ npx mintlify dev
|
||||
5. **mem-search Skill** - Natural language queries with progressive disclosure (~2,250 token savings vs MCP)
|
||||
6. **Chroma Vector Database** - Hybrid semantic + keyword search for intelligent context retrieval
|
||||
|
||||
See [Architecture Overview](docs/architecture/overview.mdx) for details.
|
||||
See [Architecture Overview](https://docs.claude-mem.ai/architecture/overview) for details.
|
||||
|
||||
---
|
||||
|
||||
@@ -186,7 +187,45 @@ Claude-Mem provides intelligent search through the mem-search skill that auto-in
|
||||
"What was happening when we added the viewer UI?"
|
||||
```
|
||||
|
||||
See [Search Tools Guide](docs/usage/search-tools.mdx) for detailed examples.
|
||||
See [Search Tools Guide](https://docs.claude-mem.ai/usage/search-tools) for detailed examples.
|
||||
|
||||
---
|
||||
|
||||
## Beta Features & Endless Mode
|
||||
|
||||
Claude-Mem offers a **beta channel** with experimental features. Switch between stable and beta versions directly from the web viewer UI.
|
||||
|
||||
### How to Try Beta
|
||||
|
||||
1. Open http://localhost:37777
|
||||
2. Click Settings (gear icon)
|
||||
3. In **Version Channel**, click "Try Beta (Endless Mode)"
|
||||
4. Wait for the worker to restart
|
||||
|
||||
Your memory data is preserved when switching versions.
|
||||
|
||||
### Endless Mode (Beta)
|
||||
|
||||
The flagship beta feature is **Endless Mode** - a biomimetic memory architecture that dramatically extends session length:
|
||||
|
||||
**The Problem**: Standard Claude Code sessions hit context limits after ~50 tool uses. Each tool adds 1-10k+ tokens, and Claude re-synthesizes all previous outputs on every response (O(N²) complexity).
|
||||
|
||||
**The Solution**: Endless Mode compresses tool outputs into ~500-token observations and transforms the transcript in real-time:
|
||||
|
||||
```
|
||||
Working Memory (Context): Compressed observations (~500 tokens each)
|
||||
Archive Memory (Disk): Full tool outputs preserved for recall
|
||||
```
|
||||
|
||||
**Expected Results**:
|
||||
- ~95% token reduction in context window
|
||||
- ~20x more tool uses before context exhaustion
|
||||
- Linear O(N) scaling instead of quadratic O(N²)
|
||||
- Full transcripts preserved for perfect recall
|
||||
|
||||
**Caveats**: Adds latency (60-90s per tool for observation generation), still experimental.
|
||||
|
||||
See [Beta Features Documentation](https://docs.claude-mem.ai/beta-features) for details.
|
||||
|
||||
---
|
||||
|
||||
@@ -278,7 +317,7 @@ See [CHANGELOG.md](CHANGELOG.md) for complete version history.
|
||||
- `CLAUDE_MEM_WORKER_PORT` - Worker port (default: 37777)
|
||||
- `CLAUDE_MEM_DATA_DIR` - Data directory override (dev only)
|
||||
|
||||
See [Configuration Guide](docs/configuration.mdx) for details.
|
||||
See [Configuration Guide](https://docs.claude-mem.ai/configuration) for details.
|
||||
|
||||
---
|
||||
|
||||
@@ -301,7 +340,7 @@ npm run worker:start
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
See [Development Guide](docs/development.mdx) for detailed instructions.
|
||||
See [Development Guide](https://docs.claude-mem.ai/development) for detailed instructions.
|
||||
|
||||
---
|
||||
|
||||
@@ -318,7 +357,7 @@ If you're experiencing issues, describe the problem to Claude and the troublesho
|
||||
- Database issues → `sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;"`
|
||||
- Search not working → Check FTS5 tables exist
|
||||
|
||||
See [Troubleshooting Guide](docs/troubleshooting.mdx) for complete solutions.
|
||||
See [Troubleshooting Guide](https://docs.claude-mem.ai/troubleshooting) for complete solutions.
|
||||
|
||||
---
|
||||
|
||||
@@ -332,7 +371,7 @@ Contributions are welcome! Please:
|
||||
4. Update documentation
|
||||
5. Submit a Pull Request
|
||||
|
||||
See [Development Guide](docs/development.mdx) for contribution workflow.
|
||||
See [Development Guide](https://docs.claude-mem.ai/development) for contribution workflow.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -148,16 +148,19 @@ When Claude invokes the skill:
|
||||
|
||||
## Search Architecture
|
||||
|
||||
### Hybrid Search System
|
||||
### 3-Layer Hybrid Search System
|
||||
|
||||
claude-mem uses a **hybrid search architecture** combining:
|
||||
claude-mem uses a **3-layer sequential search architecture** that mimics human long-term memory:
|
||||
|
||||
1. **SQLite FTS5 (Full-Text Search)** - Keyword-based search
|
||||
2. **ChromaDB (Vector Search)** - Semantic similarity search
|
||||
**Storage Flow (Write Path):**
|
||||
1. **SQLite First** - Data written synchronously to SQLite (fast, immediate access)
|
||||
2. **ChromaDB Background Sync** - Worker asynchronously generates embeddings and syncs to ChromaDB
|
||||
|
||||
**Search Flow (Read Path - Sequential, NOT parallel):**
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Search Request Flow │
|
||||
│ 3-Layer Sequential Search Flow │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
@@ -166,61 +169,70 @@ claude-mem uses a **hybrid search architecture** combining:
|
||||
│ /api/search/* │
|
||||
└─────────────────────────┘
|
||||
│
|
||||
┌─────────────┴─────────────┐
|
||||
▼ ▼
|
||||
┌──────────────────────────┐ ┌──────────────────────────┐
|
||||
│ SessionSearch (FTS5) │ │ ChromaSync (Vector DB) │
|
||||
│ │ │ │
|
||||
│ Full-text keyword │ │ Semantic similarity │
|
||||
│ search on: │ │ search on: │
|
||||
│ - titles │ │ - narratives │
|
||||
│ - narratives │ │ - facts │
|
||||
│ - facts │ │ - file content │
|
||||
│ - concepts │ │ │
|
||||
│ │ │ Embeddings: │
|
||||
│ SQLite DB: │ │ - text-embedding-3-small│
|
||||
│ observations_fts │ │ - 90-day recency filter │
|
||||
│ sessions_fts │ │ │
|
||||
│ prompts_fts │ │ ChromaDB: │
|
||||
│ │ │ observations collection │
|
||||
└──────────────────────────┘ └──────────────────────────┘
|
||||
│ │
|
||||
└─────────────┬─────────────┘
|
||||
▼
|
||||
┌─────────────────────────┐
|
||||
│ Merged Results │
|
||||
│ - Deduplicated │
|
||||
│ - Sorted by relevance │
|
||||
│ - Formatted (index/full)│
|
||||
└─────────────────────────┘
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ LAYER 1: Semantic Retrieval (ChromaDB) │
|
||||
│ ───────────────────────────────────────────────────────── │
|
||||
│ Vector similarity search finds semantically relevant items │
|
||||
│ Returns: observation IDs in index format (~50-100 tokens) │
|
||||
│ Filter: 90-day recency prioritizes recent work │
|
||||
│ Output: List of relevant observation IDs │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ LAYER 2: Temporal Ordering (SQLite) │
|
||||
│ ───────────────────────────────────────────────────────── │
|
||||
│ Takes observation IDs from Layer 1 │
|
||||
│ Sorts by created_at timestamp (fast SQLite temporal query) │
|
||||
│ Identifies: MOST RECENT relevant observation │
|
||||
│ Why: ChromaDB doesn't easily query by date range sorted │
|
||||
│ Output: Top observation ID by time │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ LAYER 3: Instant Context Timeline (SQLite) │
|
||||
│ ───────────────────────────────────────────────────────── │
|
||||
│ Uses top observation ID from Layer 2 as anchor │
|
||||
│ Retrieves N observations BEFORE and AFTER that point │
|
||||
│ Provides: "what led here" + "what happened next" context │
|
||||
│ This is the KILLER FEATURE: mimics human memory │
|
||||
│ Output: Timeline with temporal context │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Why This Architecture Exists:**
|
||||
|
||||
The problem: LLMs don't experience time linearly like humans do. Finding semantically relevant information isn't enough—you need temporal context.
|
||||
|
||||
The solution:
|
||||
- **ChromaDB** for "what's relevant" (semantic understanding)
|
||||
- **SQLite** for "when did it happen" (temporal ordering with fast date-range queries)
|
||||
- **Timeline** for "what was the context" (before/after observations)
|
||||
|
||||
Together, they mimic how humans recall: "I did X, which led to Y, then Z happened."
|
||||
|
||||
**Human Memory Analogy:**
|
||||
|
||||
Humans don't just remember isolated facts. They remember sequences: what they did before something, what happened after. The instant context timeline gives LLMs this same temporal awareness that humans experience naturally.
|
||||
|
||||
### Search Types
|
||||
|
||||
#### 1. Full-Text Search (FTS5)
|
||||
#### 1. Vector Search (ChromaDB) - PRIMARY Search Layer
|
||||
|
||||
**How it works:**
|
||||
- Uses SQLite FTS5 virtual tables for instant keyword matching
|
||||
- Supports boolean operators: `AND`, `OR`, `NOT`, `NEAR`, `*` (wildcard)
|
||||
- Ranks results by BM25 relevance scoring
|
||||
- Sub-100ms performance on 8,000+ observations
|
||||
|
||||
**Example query:**
|
||||
```sql
|
||||
-- User asks: "How did we implement JWT authentication?"
|
||||
SELECT * FROM observations_fts
|
||||
WHERE observations_fts MATCH 'JWT AND authentication'
|
||||
ORDER BY rank
|
||||
LIMIT 20;
|
||||
```
|
||||
|
||||
#### 2. Vector Search (ChromaDB)
|
||||
**Role:** Layer 1 - Semantic Retrieval
|
||||
|
||||
**How it works:**
|
||||
- Text is embedded using OpenAI's `text-embedding-3-small` model
|
||||
- Vector similarity search finds semantically related content
|
||||
- Vector similarity search finds semantically related content, not just keyword matches
|
||||
- 90-day recency filter prioritizes recent work
|
||||
- Combined with keyword search for hybrid results
|
||||
- Returns observation IDs for temporal processing in Layer 2
|
||||
|
||||
**Why it's primary:**
|
||||
- Understands meaning, not just keywords ("auth flow" matches "JWT implementation")
|
||||
- Finds relevant work even when you don't know exact terms used
|
||||
- Semantic understanding crucial for LLM memory retrieval
|
||||
|
||||
**Example query:**
|
||||
```python
|
||||
@@ -230,6 +242,37 @@ collection.query(
|
||||
n_results=20,
|
||||
where={"created_at": {"$gte": ninety_days_ago}}
|
||||
)
|
||||
# Returns: observation IDs semantically related to login/auth
|
||||
```
|
||||
|
||||
#### 2. Full-Text Search (FTS5) - Supporting Layer
|
||||
|
||||
**Role:** Layer 2 & 3 - Temporal Ordering and Timeline Context
|
||||
|
||||
**How it works:**
|
||||
- Uses SQLite FTS5 virtual tables for instant keyword matching
|
||||
- Supports boolean operators: `AND`, `OR`, `NOT`, `NEAR`, `*` (wildcard)
|
||||
- Fast temporal queries with date-range sorting
|
||||
- Sub-100ms performance on 8,000+ observations
|
||||
|
||||
**Why it's supporting:**
|
||||
- ChromaDB handles semantic "what's relevant"
|
||||
- SQLite/FTS5 handles temporal "when did it happen" and "what came before/after"
|
||||
- Optimized for timeline queries and date-based sorting
|
||||
|
||||
**Example query:**
|
||||
```sql
|
||||
-- Takes observation IDs from ChromaDB, sorts by time
|
||||
SELECT * FROM observations
|
||||
WHERE id IN (/* IDs from ChromaDB */)
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT 1;
|
||||
|
||||
-- Then retrieves timeline context around that observation
|
||||
SELECT * FROM observations
|
||||
WHERE created_at_epoch < anchor_timestamp
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT 10; -- "what led here"
|
||||
```
|
||||
|
||||
#### 3. Structured Filters
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
@everyone
|
||||
|
||||
**Endless Mode: Breaking Claude's Context Limits**
|
||||
|
||||
## The Problem
|
||||
|
||||
Ever hit 67% context usage mid-session and had to restart Claude Code? Context window limits are the #1 killer of long coding sessions. When you're deep in a complex refactor or debugging session, the last thing you want is to lose all that built-up context.
|
||||
|
||||
## The Solution: Endless Mode
|
||||
|
||||
Endless Mode compresses tool outputs **in real-time** as you work. Instead of storing the full 500-line file you just read, it stores a compact observation like:
|
||||
|
||||
> "Read package.json - found 47 dependencies including React 18, TypeScript 5.2, and custom build scripts"
|
||||
|
||||
**The result: 70-84% token reduction** on tool outputs, letting you work indefinitely without hitting context limits.
|
||||
|
||||
## The Numbers (Real Test Results)
|
||||
|
||||
We analyzed **500 transcripts** containing **1,884 tool uses**:
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Tool uses analyzed | 1,884 |
|
||||
| Observations matched | 868 |
|
||||
| Eligible for compression | 406 |
|
||||
| Compression rate (facts-only) | **84%** |
|
||||
| Characters saved | 887,783 of 1,056,285 |
|
||||
|
||||
**Which tools benefit most:**
|
||||
- **Bash output**: 236 compressible (command outputs -> facts)
|
||||
- **Read file contents**: 98 compressible (file contents -> summaries)
|
||||
- **Grep results**: 42 compressible (search results -> key matches)
|
||||
|
||||
**Key insight**: We only compress tool **outputs**, never inputs. Inputs contain semantic meaning (the actual diff, the query, the code you wrote). Outputs are verbose results that can be summarized without losing meaning.
|
||||
|
||||
## The Journey (69 observations over 10 days)
|
||||
|
||||
**Nov 16 - The Vision**
|
||||
Decided to build Endless Mode as an *optional* feature to avoid mandatory architectural refactoring. The idea: let users opt-in to experimental compression without breaking anything for those who don't.
|
||||
|
||||
**Nov 19-20 - Implementation Begins**
|
||||
Hit our first bug immediately: duplicate observations appearing on the 2nd prompt of each session. Classic regression - the endless mode changes broke something that was already working. Fixed it, kept going.
|
||||
|
||||
**Nov 21 - The Big Switch**
|
||||
Made a critical architectural change: switched from **deferred** (async, 5-second timeout) to **synchronous** transformation (blocking, 90-second timeout). Endless Mode needs to wait for compression to complete before continuing - otherwise you'd read uncompressed data.
|
||||
|
||||
Multiple rounds of experimental release preparation. Documented all dependencies. Critical bugs kept appearing.
|
||||
|
||||
**Nov 22 - Validation**
|
||||
Endpoints verified. Toggle working. Documentation reviewed. Things looking stable.
|
||||
|
||||
**Nov 23 - The Setback**
|
||||
**Disabled endless mode.** It was causing everything to hang. The 90-second synchronous blocking was too aggressive - when compression took too long, the whole system locked up. Had to prioritize stability.
|
||||
|
||||
25 sessions had successfully used it before this point.
|
||||
|
||||
**Nov 25 - The Solution**
|
||||
Created a **beta branch strategy**: Endless Mode lives on `beta/7.0`, isolated from main. Added Version Channel UI so users can safely try it without affecting stable users. Easy rollback if issues occur.
|
||||
|
||||
Built analysis scripts to measure *actual* compression rates instead of theoretical. Validated 84% savings on real transcripts.
|
||||
|
||||
## How to Try It
|
||||
|
||||
**v6.3.1** added a Version Channel switcher:
|
||||
|
||||
1. Open http://localhost:37777
|
||||
2. Find **"Version Channel"** in Settings sidebar
|
||||
3. Click **"Try Beta (Endless Mode)"**
|
||||
4. Refresh the UI after switching
|
||||
|
||||
**Safe to try**: Your memory data lives in `~/.claude-mem/` - completely separate from the plugin code. Switching branches won't touch your data. Easy rollback with "Switch to Stable" button.
|
||||
|
||||
**Current beta branch**: `beta/7.0`
|
||||
|
||||
---
|
||||
|
||||
This has been a real engineering journey - vision, implementation, bugs, setbacks, and creative solutions. The beta branch approach lets us keep iterating on stability while giving adventurous users access to the feature.
|
||||
@@ -0,0 +1,427 @@
|
||||
# Endless Mode: Real-Time Context Compression Plan
|
||||
|
||||
## Executive Summary
|
||||
|
||||
"Endless Mode" is an optional feature that enables Claude sessions to run indefinitely by transparently compressing tool use transcripts in real-time. Using an in-memory transformation layer in the worker service, heavy tool outputs are dynamically replaced with lightweight observations during session resume—without modifying the immutable source transcripts. This allows sessions to continue for weeks or months without hitting context window limits, while preserving full conversation history and maintaining zero risk of data corruption.
|
||||
|
||||
---
|
||||
|
||||
## Problem Statement
|
||||
|
||||
### Current Behavior
|
||||
|
||||
Claude sessions accumulate full tool transcripts in the context window:
|
||||
- File reads: 5k-10k tokens per read
|
||||
- Bash outputs: 1k-5k tokens per command
|
||||
- Search results: 2k-8k tokens per search
|
||||
- Total context limit: ~200k tokens
|
||||
|
||||
When the context window fills, users must start a new session, losing conversational continuity.
|
||||
|
||||
### What Happens Today
|
||||
|
||||
1. Tool executes during session
|
||||
2. PostToolUse hook captures tool data
|
||||
3. Worker creates compressed observation (~200-500 tokens)
|
||||
4. **But**: Full tool transcript stays in Claude's context window
|
||||
5. **Observation only helps next session** via SessionStart injection
|
||||
|
||||
### The Gap
|
||||
|
||||
Observations exist and are created in real-time, but they're not used to compress the **current** session's context. We have the compressed data, we just don't apply it to the active session.
|
||||
|
||||
---
|
||||
|
||||
## Proposed Solution: Endless Mode
|
||||
|
||||
### Core Concept
|
||||
|
||||
When a session resumes (either after restart or during continuation), **transform messages in memory** by replacing heavy tool use content with lightweight observations before feeding them to the Agent SDK. The source transcript remains immutable on disk.
|
||||
|
||||
### Architecture Principle
|
||||
|
||||
**Immutable Storage + Ephemeral Transform = Safe Compression**
|
||||
|
||||
```
|
||||
Disk (never modified) Memory (transform) Agent SDK
|
||||
────────────────────── ────────────────────── ────────────────
|
||||
transcript.jsonl Load messages Resume session
|
||||
tool_use_abc → Look up observation → with compressed
|
||||
tool_use_def Replace content context
|
||||
tool_use_xyz Feed to SDK
|
||||
```
|
||||
|
||||
### Key Properties
|
||||
|
||||
1. **Immutable**: Original transcripts never modified
|
||||
2. **Non-destructive**: Full history preserved on disk
|
||||
3. **No duplication**: No forks, no copies
|
||||
4. **Transparent**: User sees same conversation, compression is under the hood
|
||||
5. **Optional**: Feature flag allows users to opt-in/out
|
||||
6. **Reversible**: Can always read original transcript
|
||||
|
||||
---
|
||||
|
||||
## How It Works
|
||||
|
||||
### Session Resume Flow (Endless Mode Enabled)
|
||||
|
||||
```
|
||||
1. User continues session / Claude Code restarts
|
||||
↓
|
||||
2. Worker service intercepts resume request
|
||||
↓
|
||||
3. Load transcript JSONL from disk (immutable)
|
||||
↓
|
||||
4. Transform Loop:
|
||||
For each message in transcript:
|
||||
- If tool_use message:
|
||||
- Query SQLite: SELECT observation WHERE tool_use_id = ?
|
||||
- Replace tool content with observation (facts, narrative, concepts)
|
||||
- If other message type:
|
||||
- Pass through unchanged
|
||||
↓
|
||||
5. Feed transformed messages to Agent SDK
|
||||
↓
|
||||
6. Agent SDK resumes session with compressed context
|
||||
↓
|
||||
7. New tool uses append to original transcript (normal flow)
|
||||
↓
|
||||
8. Next resume: Loop repeats, new tool uses also get compressed
|
||||
```
|
||||
|
||||
### Session Resume Flow (Endless Mode Disabled)
|
||||
|
||||
```
|
||||
1. User continues session
|
||||
↓
|
||||
2. Load transcript JSONL from disk
|
||||
↓
|
||||
3. Feed messages directly to Agent SDK (no transformation)
|
||||
↓
|
||||
4. Session resumes with full tool transcripts (current behavior)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: Foundation (Week 1)
|
||||
|
||||
**Goal**: Set up infrastructure for transformation layer
|
||||
|
||||
Tasks:
|
||||
1. Add `tool_use_id` column to observations table (SQLite schema migration)
|
||||
2. Update PostToolUse hook to capture and store tool_use_id
|
||||
3. Create `TransformLayer` class in worker service
|
||||
4. Add `CLAUDE_MEM_ENDLESS_MODE` environment variable (default: false)
|
||||
5. Write tests for observation lookup by tool_use_id
|
||||
|
||||
**Deliverable**: Database schema updated, tool_use_ids being captured
|
||||
|
||||
### Phase 2: Transform Logic (Week 2)
|
||||
|
||||
**Goal**: Build message transformation engine
|
||||
|
||||
Tasks:
|
||||
1. Implement `TransformLayer.transformMessages(messages)` function
|
||||
2. Tool use detection logic (identify tool_use messages in transcript)
|
||||
3. Observation lookup and replacement logic
|
||||
4. Fallback handling (if observation missing, keep original content)
|
||||
5. Message serialization/deserialization
|
||||
|
||||
**Deliverable**: Working transform function that compresses messages in memory
|
||||
|
||||
### Phase 3: Agent SDK Integration (Week 2-3)
|
||||
|
||||
**Goal**: Wire transform layer into session resume flow
|
||||
|
||||
Tasks:
|
||||
1. Identify where worker service resumes Agent SDK sessions
|
||||
2. Inject transform layer before session resume
|
||||
3. Add feature flag check (only transform if endless mode enabled)
|
||||
4. Logging and instrumentation (track compression ratios, transform time)
|
||||
5. Error handling and graceful degradation
|
||||
|
||||
**Deliverable**: Worker service can resume sessions with compressed context
|
||||
|
||||
### Phase 4: Testing & Validation (Week 3-4)
|
||||
|
||||
**Goal**: Verify endless mode works correctly
|
||||
|
||||
Tasks:
|
||||
1. Create test session with 50+ tool uses
|
||||
2. Enable endless mode and resume session
|
||||
3. Verify context window usage (should be dramatically lower)
|
||||
4. Test conversation quality (does Claude have enough context?)
|
||||
5. Measure performance (transform latency, lookup speed)
|
||||
6. Edge case testing (missing observations, malformed transcripts)
|
||||
|
||||
**Deliverable**: Endless mode working in test environment
|
||||
|
||||
### Phase 5: Beta Release (Week 4+)
|
||||
|
||||
**Goal**: Release to power users for feedback
|
||||
|
||||
Tasks:
|
||||
1. Documentation (how to enable, what to expect, how to disable)
|
||||
2. Add endless mode toggle to viewer UI
|
||||
3. Monitoring and observability (track usage, failures, compression stats)
|
||||
4. Collect feedback from beta users
|
||||
5. Iterate based on real-world usage
|
||||
|
||||
**Deliverable**: Endless mode available as opt-in beta feature
|
||||
|
||||
---
|
||||
|
||||
## Technical Requirements
|
||||
|
||||
### Database Schema
|
||||
|
||||
```sql
|
||||
-- Add to observations table
|
||||
ALTER TABLE observations ADD COLUMN tool_use_id TEXT UNIQUE;
|
||||
CREATE INDEX idx_observations_tool_use_id ON observations(tool_use_id);
|
||||
```
|
||||
|
||||
### Worker Service API
|
||||
|
||||
```typescript
|
||||
interface TransformLayerConfig {
|
||||
enabled: boolean; // CLAUDE_MEM_ENDLESS_MODE
|
||||
fallbackToOriginal: boolean; // If observation missing, use full content
|
||||
maxLookupTime: number; // Timeout for SQLite queries
|
||||
}
|
||||
|
||||
class TransformLayer {
|
||||
constructor(config: TransformLayerConfig, db: SessionStore);
|
||||
|
||||
// Main transform function
|
||||
async transformMessages(messages: Message[]): Promise<Message[]>;
|
||||
|
||||
// Helper functions
|
||||
private async lookupObservation(toolUseId: string): Promise<Observation | null>;
|
||||
private replaceToolContent(message: Message, observation: Observation): Message;
|
||||
private isToolUseMessage(message: Message): boolean;
|
||||
}
|
||||
```
|
||||
|
||||
### Agent SDK Integration Point
|
||||
|
||||
```typescript
|
||||
// In worker service session resume logic
|
||||
async function resumeSession(sessionId: string, transcriptPath: string) {
|
||||
const messages = await loadTranscript(transcriptPath);
|
||||
|
||||
// Transform layer (only if endless mode enabled)
|
||||
const transformedMessages = config.endlessMode
|
||||
? await transformLayer.transformMessages(messages)
|
||||
: messages;
|
||||
|
||||
// Resume with transformed (or original) messages
|
||||
return await agentSDK.resumeSession({
|
||||
sessionId,
|
||||
messages: transformedMessages
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
### Risk 1: Information Loss
|
||||
|
||||
**Risk**: Compressed observations may lose critical details that Claude needs to reference later.
|
||||
|
||||
**Mitigation**:
|
||||
- Make endless mode optional (users can disable if quality degrades)
|
||||
- Improve observation quality (better prompts, more comprehensive facts)
|
||||
- Hybrid approach: Keep recent N tool uses in full, compress older ones
|
||||
- Monitor conversation quality metrics
|
||||
|
||||
### Risk 2: Transform Performance
|
||||
|
||||
**Risk**: Looking up observations for 100+ tool uses during resume could be slow.
|
||||
|
||||
**Mitigation**:
|
||||
- Index tool_use_id in SQLite (O(log n) lookups)
|
||||
- Batch queries (single SELECT with IN clause)
|
||||
- Measure and optimize (target <100ms for typical session)
|
||||
- Cache observations in memory during session
|
||||
|
||||
### Risk 3: Missing Observations
|
||||
|
||||
**Risk**: Tool use executed but observation not yet created (async worker lag).
|
||||
|
||||
**Mitigation**:
|
||||
- Fallback to original content if observation missing
|
||||
- Log when fallback occurs (helps identify worker performance issues)
|
||||
- Allow observations to be created retroactively
|
||||
- Consider synchronous observation creation for critical tools
|
||||
|
||||
### Risk 4: Transcript Corruption
|
||||
|
||||
**Risk**: Bug in transform layer could corrupt user conversations.
|
||||
|
||||
**Mitigation**:
|
||||
- **Never modify source transcripts** (read-only)
|
||||
- Transform happens in memory only
|
||||
- Extensive testing before beta release
|
||||
- Feature flag allows instant disable if issues found
|
||||
- Keep full audit trail in logs
|
||||
|
||||
### Risk 5: Agent SDK Compatibility
|
||||
|
||||
**Risk**: Agent SDK updates could break transform layer integration.
|
||||
|
||||
**Mitigation**:
|
||||
- Document exact Agent SDK version requirements
|
||||
- Monitor Agent SDK release notes
|
||||
- Test against new SDK versions before upgrading
|
||||
- Graceful degradation if SDK changes detected
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
### Proof of Concept Success
|
||||
|
||||
- [ ] Transform layer successfully compresses a 50-tool-use session
|
||||
- [ ] Context window usage reduced by 80%+ compared to uncompressed
|
||||
- [ ] Session resumes without errors
|
||||
- [ ] Conversation quality remains high (subjective evaluation)
|
||||
|
||||
### Beta Release Success
|
||||
|
||||
- [ ] 10+ users running endless mode without issues
|
||||
- [ ] Average context savings: 85%+ across all sessions
|
||||
- [ ] Transform latency: <200ms for typical resume
|
||||
- [ ] Zero transcript corruption incidents
|
||||
- [ ] Positive user feedback on conversation continuity
|
||||
|
||||
### Production Success
|
||||
|
||||
- [ ] Endless mode becomes default setting
|
||||
- [ ] Sessions running for weeks/months without context issues
|
||||
- [ ] Context window exhaustion becomes rare edge case
|
||||
- [ ] User-reported "session too long" issues drop to near zero
|
||||
- [ ] Transform layer performance scales to 1000+ tool use sessions
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```bash
|
||||
# Enable endless mode (default: false)
|
||||
CLAUDE_MEM_ENDLESS_MODE=true
|
||||
|
||||
# Fallback behavior if observation missing (default: true)
|
||||
CLAUDE_MEM_TRANSFORM_FALLBACK=true
|
||||
|
||||
# Max time to wait for observation lookup (default: 500ms)
|
||||
CLAUDE_MEM_TRANSFORM_TIMEOUT=500
|
||||
|
||||
# Keep recent N tool uses uncompressed (default: 0, compress all)
|
||||
CLAUDE_MEM_TRANSFORM_KEEP_RECENT=0
|
||||
```
|
||||
|
||||
### User Controls
|
||||
|
||||
```typescript
|
||||
// Future: UI toggle in viewer
|
||||
interface EndlessModeSettings {
|
||||
enabled: boolean;
|
||||
keepRecentToolUses: number; // Hybrid mode
|
||||
fallbackToOriginal: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Context Economics: Before vs. After
|
||||
|
||||
### Example Session (50 tool uses)
|
||||
|
||||
**Before (Endless Mode OFF):**
|
||||
```
|
||||
File reads: 10 × 8,000 tokens = 80,000 tokens
|
||||
Bash outputs: 20 × 2,000 tokens = 40,000 tokens
|
||||
Searches: 15 × 4,000 tokens = 60,000 tokens
|
||||
Other tools: 5 × 1,000 tokens = 5,000 tokens
|
||||
──────────────────────────────────────────────────
|
||||
Total: 185,000 tokens
|
||||
Context remaining: 15,000 tokens (92% full)
|
||||
```
|
||||
|
||||
**After (Endless Mode ON):**
|
||||
```
|
||||
File reads: 10 × 300 tokens = 3,000 tokens
|
||||
Bash outputs: 20 × 250 tokens = 5,000 tokens
|
||||
Searches: 15 × 400 tokens = 6,000 tokens
|
||||
Other tools: 5 × 200 tokens = 1,000 tokens
|
||||
──────────────────────────────────────────────────
|
||||
Total: 15,000 tokens
|
||||
Context remaining: 185,000 tokens (7.5% full)
|
||||
|
||||
Savings: 170,000 tokens (92% reduction)
|
||||
```
|
||||
|
||||
**Session Longevity:**
|
||||
- Before: ~50 tool uses before context full
|
||||
- After: ~600+ tool uses before context full
|
||||
- **12x longer sessions**
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
### Immediate Actions (This Week)
|
||||
|
||||
1. **Database Migration**: Add tool_use_id column to observations table
|
||||
2. **Hook Update**: Modify PostToolUse hook to capture tool_use_id from Agent SDK
|
||||
3. **Architecture Validation**: Confirm where Agent SDK session resume happens in worker service
|
||||
4. **Prototype**: Build minimal TransformLayer class with observation lookup
|
||||
|
||||
### Short Term (Next 2 Weeks)
|
||||
|
||||
1. Implement complete transform logic
|
||||
2. Wire into worker service resume flow
|
||||
3. Add endless mode feature flag
|
||||
4. Test with real sessions
|
||||
|
||||
### Medium Term (Next Month)
|
||||
|
||||
1. Beta release to power users
|
||||
2. Gather feedback and iterate
|
||||
3. Performance optimization
|
||||
4. Documentation and user guides
|
||||
|
||||
### Long Term (Future)
|
||||
|
||||
1. Make endless mode default
|
||||
2. Hybrid sliding window (keep recent tools uncompressed)
|
||||
3. Selective compression by tool type
|
||||
4. Auto-tune compression based on context usage patterns
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Tool Use ID Format**: What does the Agent SDK's tool_use_id look like? Is it UUID, hash, or sequential?
|
||||
2. **Transcript Format**: What's the exact JSONL schema for tool_use messages? Where is the content we'll replace?
|
||||
3. **Resume Hook Point**: Where exactly in the worker service does session resume happen? Is there a clear integration point?
|
||||
4. **Observation Delay**: How long between PostToolUse firing and observation being available in SQLite? Does this affect resume?
|
||||
5. **Feature Flag Storage**: Environment variable, or persist user preference in database?
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
Endless Mode transforms claude-mem from a "memory between sessions" system into a "continuous compression engine" that enables truly infinite sessions. By leveraging the observations we're already creating in real-time and applying them as an ephemeral transformation layer during resume, we can extend session longevity by 10-12x without any risk to user data.
|
||||
|
||||
The key architectural insight is **immutability**: by never modifying source transcripts and performing all compression in memory, we get the benefits of context window optimization without the risks of data corruption or loss. Combined with the optional nature of the feature, this provides a safe, reversible path to fundamentally better session continuity.
|
||||
|
||||
This is the natural evolution of claude-mem: from remembering what happened before, to making it possible to never stop.
|
||||
@@ -0,0 +1,144 @@
|
||||
---
|
||||
title: "Beta Features"
|
||||
description: "Try experimental features like Endless Mode before they're released"
|
||||
---
|
||||
|
||||
# Beta Features
|
||||
|
||||
Claude-Mem offers a beta channel for users who want to try experimental features before they're released to the stable channel.
|
||||
|
||||
## Version Channel Switching
|
||||
|
||||
You can switch between stable and beta versions directly from the web viewer UI at http://localhost:37777.
|
||||
|
||||
### How to Access
|
||||
|
||||
1. Open the Claude-Mem viewer at http://localhost:37777
|
||||
2. Click the **Settings** gear icon in the top-right
|
||||
3. Find the **Version Channel** section
|
||||
4. Click **Try Beta (Endless Mode)** to switch to beta, or **Switch to Stable** to return
|
||||
|
||||
### What Happens When You Switch
|
||||
|
||||
When switching versions:
|
||||
|
||||
1. **Local changes are discarded** - Any modifications in the plugin directory are reset
|
||||
2. **Git fetch and checkout** - The installed plugin switches to the target branch
|
||||
3. **Dependencies reinstall** - `npm install` runs to ensure correct dependencies
|
||||
4. **Worker restarts automatically** - The background service restarts with the new version
|
||||
|
||||
**Your memory data is always preserved.** The database at `~/.claude-mem/claude-mem.db` is not affected by version switching. All your observations, sessions, and summaries remain intact.
|
||||
|
||||
### Version Indicators
|
||||
|
||||
The Version Channel section shows your current status:
|
||||
|
||||
- **Stable** (green badge) - You're running the production release
|
||||
- **Beta** (orange badge) - You're running the beta with experimental features
|
||||
|
||||
You'll also see the exact branch name (e.g., `main` for stable, `beta/7.0` for beta).
|
||||
|
||||
## Endless Mode (Beta)
|
||||
|
||||
The flagship experimental feature in beta is **Endless Mode** - a biomimetic memory architecture that dramatically extends how long Claude can maintain context in a session.
|
||||
|
||||
### The Problem Endless Mode Solves
|
||||
|
||||
In standard Claude Code sessions:
|
||||
|
||||
- Tool outputs (file reads, bash output, search results) accumulate in the context window
|
||||
- Each tool can add 1-10k+ tokens to the context
|
||||
- After ~50 tool uses, the context window fills up (~200k tokens)
|
||||
- You're forced to start a new session, losing conversational continuity
|
||||
|
||||
Worse, Claude **re-synthesizes all previous tool outputs** on every response. This is O(N²) complexity - quadratically growing both in tokens and compute.
|
||||
|
||||
### How Endless Mode Works
|
||||
|
||||
Endless Mode applies a biomimetic memory architecture inspired by how human memory works:
|
||||
|
||||
**Two-Tier Memory System:**
|
||||
|
||||
```
|
||||
Working Memory (Context Window):
|
||||
→ Compressed observations only (~500 tokens each)
|
||||
→ Fast, efficient, manageable
|
||||
|
||||
Archive Memory (Transcript File):
|
||||
→ Full tool outputs preserved on disk
|
||||
→ Perfect recall, searchable
|
||||
```
|
||||
|
||||
**The Key Innovation**: After each tool use, Endless Mode:
|
||||
1. Waits for the worker to generate a compressed observation (blocking)
|
||||
2. Transforms the transcript file on disk
|
||||
3. Replaces the full tool output with the compressed observation
|
||||
4. Claude resumes with the compressed context
|
||||
|
||||
This transforms O(N²) scaling into O(N) - linear instead of quadratic.
|
||||
|
||||
### Expected Results
|
||||
|
||||
Based on analysis of real sessions:
|
||||
|
||||
- **Token savings**: ~95% reduction in context window usage
|
||||
- **Efficiency gain**: ~20x more tool uses before context exhaustion
|
||||
- **Quality preservation**: Observations cache the synthesis result, so no information is lost
|
||||
|
||||
### Caveats
|
||||
|
||||
Endless Mode is experimental:
|
||||
|
||||
- **Adds latency** - Blocking hooks wait for observation generation (60-90s per tool use)
|
||||
- **Requires working database** - Observations must save successfully for transformation
|
||||
- **New architecture** - Less battle-tested than standard mode
|
||||
|
||||
### When to Use Beta
|
||||
|
||||
Consider switching to beta if you:
|
||||
|
||||
- Frequently hit context window limits
|
||||
- Work on long, complex sessions with many tool uses
|
||||
- Want to help test and provide feedback on new features
|
||||
- Are comfortable with experimental software
|
||||
|
||||
### When to Stay on Stable
|
||||
|
||||
Stay on stable if you:
|
||||
|
||||
- Need maximum reliability for critical work
|
||||
- Prefer battle-tested, production-ready features
|
||||
- Don't frequently hit context limits
|
||||
- Want the smoothest, fastest experience
|
||||
|
||||
## Checking for Updates
|
||||
|
||||
While on beta (or stable), you can check for updates:
|
||||
|
||||
1. Open Settings in the viewer
|
||||
2. In the Version Channel section, click **Check for Updates**
|
||||
3. The plugin will pull the latest changes and restart
|
||||
|
||||
## Switching Back
|
||||
|
||||
If you encounter issues on beta:
|
||||
|
||||
1. Open Settings in the viewer
|
||||
2. Click **Switch to Stable**
|
||||
3. Wait for the worker to restart
|
||||
|
||||
Your memory data is preserved, and you'll be back on the stable release.
|
||||
|
||||
## Providing Feedback
|
||||
|
||||
If you encounter bugs or have feedback about beta features:
|
||||
|
||||
- Open an issue at [GitHub Issues](https://github.com/thedotmack/claude-mem/issues)
|
||||
- Include your branch (`beta/7.0` etc.) in the report
|
||||
- Describe what you expected vs. what happened
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Configuration](configuration) - Customize other Claude-Mem settings
|
||||
- [Troubleshooting](troubleshooting) - Common issues and solutions
|
||||
- [Architecture Overview](architecture/overview) - Understand how Claude-Mem works
|
||||
@@ -151,6 +151,26 @@ Search operations are now provided via:
|
||||
- **HTTP API**: 10 endpoints on worker service port 37777
|
||||
- **Progressive Disclosure**: Full instructions loaded on-demand only when needed
|
||||
|
||||
## Version Channel
|
||||
|
||||
Claude-Mem supports switching between stable and beta versions via the web viewer UI.
|
||||
|
||||
### Accessing Version Channel
|
||||
|
||||
1. Open the viewer at http://localhost:37777
|
||||
2. Click the Settings gear icon
|
||||
3. Find the **Version Channel** section
|
||||
|
||||
### Switching Versions
|
||||
|
||||
- **Try Beta**: Click "Try Beta (Endless Mode)" to switch to the beta branch with experimental features
|
||||
- **Switch to Stable**: Click "Switch to Stable" to return to the production release
|
||||
- **Check for Updates**: Pull the latest changes for your current branch
|
||||
|
||||
**Your memory data is preserved** when switching versions. Only the plugin code changes.
|
||||
|
||||
See [Beta Features](beta-features) for details on what's available in beta.
|
||||
|
||||
## PM2 Configuration
|
||||
|
||||
Worker service is managed by PM2 via `ecosystem.config.cjs`:
|
||||
|
||||
@@ -36,7 +36,8 @@
|
||||
"introduction",
|
||||
"installation",
|
||||
"usage/getting-started",
|
||||
"usage/search-tools"
|
||||
"usage/search-tools",
|
||||
"beta-features"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "5.5.1",
|
||||
"version": "6.0.9",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "claude-mem",
|
||||
"version": "5.5.1",
|
||||
"version": "6.0.9",
|
||||
"license": "AGPL-3.0",
|
||||
"dependencies": {
|
||||
"@anthropic-ai/claude-agent-sdk": "^0.1.27",
|
||||
|
||||
+3
-2
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "6.0.1",
|
||||
"version": "6.3.2",
|
||||
"description": "Memory compression system for Claude Code - persist context across sessions",
|
||||
"keywords": [
|
||||
"claude",
|
||||
@@ -39,7 +39,8 @@
|
||||
"worker:start": "pm2 start ecosystem.config.cjs",
|
||||
"worker:stop": "pm2 stop claude-mem-worker",
|
||||
"worker:restart": "pm2 restart claude-mem-worker",
|
||||
"worker:logs": "pm2 logs claude-mem-worker",
|
||||
"worker:logs": "pm2 flush claude-mem-worker && pm2 logs claude-mem-worker --lines 100 --nostream",
|
||||
"worker:logs:no-flush": "pm2 logs claude-mem-worker --lines 100 --nostream",
|
||||
"changelog:generate": "node scripts/generate-changelog.js",
|
||||
"usage:analyze": "node scripts/analyze-usage.js",
|
||||
"usage:today": "node scripts/analyze-usage.js $(date +%Y-%m-%d)"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "6.0.1",
|
||||
"version": "6.3.2",
|
||||
"description": "Persistent memory system for Claude Code - seamlessly preserve context across sessions",
|
||||
"author": {
|
||||
"name": "Alex Newman"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
import{stdin as I}from"process";import w from"better-sqlite3";import{join as E,dirname as k,basename as W}from"path";import{homedir as O}from"os";import{existsSync as K,mkdirSync as x}from"fs";import{fileURLToPath as U}from"url";function M(){return typeof __dirname<"u"?__dirname:k(U(import.meta.url))}var q=M(),l=process.env.CLAUDE_MEM_DATA_DIR||E(O(),".claude-mem"),R=process.env.CLAUDE_CONFIG_DIR||E(O(),".claude"),J=E(l,"archives"),Q=E(l,"logs"),z=E(l,"trash"),Z=E(l,"backups"),ee=E(l,"settings.json"),f=E(l,"claude-mem.db"),se=E(l,"vector-db"),te=E(R,"settings.json"),re=E(R,"commands"),ne=E(R,"CLAUDE.md");function L(p){x(p,{recursive:!0})}var h=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(h||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=h[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=h[e].padEnd(5),d=s.padEnd(6),_="";r?.correlationId?_=`[${r.correlationId}] `:r?.sessionId&&(_=`[session-${r.sessionId}] `);let u="";n!=null&&(this.level===0&&typeof n=="object"?u=`
|
||||
`+JSON.stringify(n,null,2):u=" "+this.formatData(n));let T="";if(r){let{sessionId:m,sdkSessionId:S,correlationId:c,...a}=r;Object.keys(a).length>0&&(T=` {${Object.entries(a).map(([y,D])=>`${y}=${D}`).join(", ")}}`)}let b=`[${o}] [${i}] [${d}] ${_}${t}${T}${u}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},A=new N;var g=class{db;constructor(){L(l),this.db=new w(f),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
import{stdin as I}from"process";import w from"better-sqlite3";import{join as m,dirname as k,basename as W}from"path";import{homedir as O}from"os";import{existsSync as K,mkdirSync as x}from"fs";import{fileURLToPath as U}from"url";function M(){return typeof __dirname<"u"?__dirname:k(U(import.meta.url))}var q=M(),E=process.env.CLAUDE_MEM_DATA_DIR||m(O(),".claude-mem"),R=process.env.CLAUDE_CONFIG_DIR||m(O(),".claude"),J=m(E,"archives"),Q=m(E,"logs"),z=m(E,"trash"),Z=m(E,"backups"),ee=m(E,"settings.json"),f=m(E,"claude-mem.db"),se=m(E,"vector-db"),te=m(R,"settings.json"),re=m(R,"commands"),ne=m(R,"CLAUDE.md");function A(c){x(c,{recursive:!0})}var h=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(h||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=h[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=h[e].padEnd(5),d=s.padEnd(6),_="";r?.correlationId?_=`[${r.correlationId}] `:r?.sessionId&&(_=`[session-${r.sessionId}] `);let T="";n!=null&&(this.level===0&&typeof n=="object"?T=`
|
||||
`+JSON.stringify(n,null,2):T=" "+this.formatData(n));let u="";if(r){let{sessionId:l,sdkSessionId:S,correlationId:p,...a}=r;Object.keys(a).length>0&&(u=` {${Object.entries(a).map(([y,D])=>`${y}=${D}`).join(", ")}}`)}let b=`[${o}] [${i}] [${d}] ${_}${t}${u}${T}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},L=new N;var g=class{db;constructor(){A(E),this.db=new w(f),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable(),this.ensureDiscoveryTokensColumn()}initializeSchema(){try{this.db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS schema_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
version INTEGER UNIQUE NOT NULL,
|
||||
@@ -166,7 +166,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}ensureDiscoveryTokensColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(11))return;this.db.pragma("table_info(observations)").some(o=>o.name==="discovery_tokens")||(this.db.exec("ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to observations table")),this.db.pragma("table_info(session_summaries)").some(o=>o.name==="discovery_tokens")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(11,new Date().toISOString())}catch(e){throw console.error("[SessionStore] Discovery tokens migration error:",e.message),e}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
files_read, files_edited, notes, prompt_number, created_at
|
||||
@@ -293,13 +293,17 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT OR IGNORE INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, user_prompt, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,t,r.toISOString(),n);return i.lastInsertRowid===0||i.changes===0?this.db.prepare(`
|
||||
`).run(e,e,s,t,r.toISOString(),n);return i.lastInsertRowid===0||i.changes===0?(s&&s.trim()!==""&&this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET project = ?, user_prompt = ?
|
||||
WHERE claude_session_id = ?
|
||||
`).run(s,t,e),this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE claude_session_id = ? LIMIT 1
|
||||
`).get(e).id:i.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
|
||||
`).get(e).id):i.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET sdk_session_id = ?
|
||||
WHERE id = ? AND sdk_session_id IS NULL
|
||||
`).run(s,e).changes===0?(A.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
|
||||
`).run(s,e).changes===0?(L.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET worker_port = ?
|
||||
WHERE id = ?
|
||||
@@ -312,29 +316,29 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO user_prompts
|
||||
(claude_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r,n=0){let o=new Date,i=o.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
`).run(e,e,s,o.toISOString(),i),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(u.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n,o.toISOString(),i);return{id:Number(u.lastInsertRowid),createdAtEpoch:i}}storeSummary(e,s,t,r,n=0){let o=new Date,i=o.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
`).run(e,e,s,o.toISOString(),i),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
(sdk_session_id, project, request, investigated, learned, completed,
|
||||
next_steps, notes, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(u.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
next_steps, notes, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n,o.toISOString(),i);return{id:Number(u.lastInsertRowid),createdAtEpoch:i}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
@@ -357,7 +361,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
WHERE up.id IN (${i})
|
||||
ORDER BY up.created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],d,_;if(e!==null){let m=`
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],d,_;if(e!==null){let l=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id <= ? ${o}
|
||||
@@ -369,7 +373,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
WHERE id >= ? ${o}
|
||||
ORDER BY id ASC
|
||||
LIMIT ?
|
||||
`;try{let c=this.db.prepare(m).all(e,...i,t+1),a=this.db.prepare(S).all(e,...i,r+1);if(c.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=c.length>0?c[c.length-1].created_at_epoch:s,_=a.length>0?a[a.length-1].created_at_epoch:s}catch(c){return console.error("[SessionStore] Error getting boundary observations:",c.message),{observations:[],sessions:[],prompts:[]}}}else{let m=`
|
||||
`;try{let p=this.db.prepare(l).all(e,...i,t+1),a=this.db.prepare(S).all(e,...i,r+1);if(p.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=p.length>0?p[p.length-1].created_at_epoch:s,_=a.length>0?a[a.length-1].created_at_epoch:s}catch(p){return console.error("[SessionStore] Error getting boundary observations:",p.message),{observations:[],sessions:[],prompts:[]}}}else{let l=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch <= ? ${o}
|
||||
@@ -381,12 +385,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
WHERE created_at_epoch >= ? ${o}
|
||||
ORDER BY created_at_epoch ASC
|
||||
LIMIT ?
|
||||
`;try{let c=this.db.prepare(m).all(s,...i,t),a=this.db.prepare(S).all(s,...i,r+1);if(c.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=c.length>0?c[c.length-1].created_at_epoch:s,_=a.length>0?a[a.length-1].created_at_epoch:s}catch(c){return console.error("[SessionStore] Error getting boundary timestamps:",c.message),{observations:[],sessions:[],prompts:[]}}}let u=`
|
||||
`;try{let p=this.db.prepare(l).all(s,...i,t),a=this.db.prepare(S).all(s,...i,r+1);if(p.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=p.length>0?p[p.length-1].created_at_epoch:s,_=a.length>0?a[a.length-1].created_at_epoch:s}catch(p){return console.error("[SessionStore] Error getting boundary timestamps:",p.message),{observations:[],sessions:[],prompts:[]}}}let T=`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,T=`
|
||||
`,u=`
|
||||
SELECT *
|
||||
FROM session_summaries
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
@@ -397,5 +401,5 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${o.replace("project","s.project")}
|
||||
ORDER BY up.created_at_epoch ASC
|
||||
`;try{let m=this.db.prepare(u).all(d,_,...i),S=this.db.prepare(T).all(d,_,...i),c=this.db.prepare(b).all(d,_,...i);return{observations:m,sessions:S.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:c.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(m){return console.error("[SessionStore] Error querying timeline records:",m.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};import X from"path";import{homedir as F}from"os";import{existsSync as B,readFileSync as j}from"fs";function C(){try{let p=X.join(F(),".claude-mem","settings.json");if(B(p)){let e=JSON.parse(j(p,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function v(p){console.error("[claude-mem cleanup] Hook fired",{input:p?{session_id:p.session_id,cwd:p.cwd,reason:p.reason}:null}),p||(console.log("No input provided - this script is designed to run as a Claude Code SessionEnd hook"),console.log(`
|
||||
Expected input format:`),console.log(JSON.stringify({session_id:"string",cwd:"string",transcript_path:"string",hook_event_name:"SessionEnd",reason:"exit"},null,2)),process.exit(0));let{session_id:e,reason:s}=p;console.error("[claude-mem cleanup] Searching for active SDK session",{session_id:e,reason:s});let t=new g,r=t.findActiveSDKSession(e);r||(console.error("[claude-mem cleanup] No active SDK session found",{session_id:e}),t.close(),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)),console.error("[claude-mem cleanup] Active SDK session found",{session_id:r.id,sdk_session_id:r.sdk_session_id,project:r.project,worker_port:r.worker_port}),t.markSessionCompleted(r.id),console.error("[claude-mem cleanup] Session marked as completed in database"),t.close();try{let n=r.worker_port||C();await fetch(`http://127.0.0.1:${n}/sessions/${r.id}/complete`,{method:"POST",signal:AbortSignal.timeout(1e3)}),console.error("[claude-mem cleanup] Worker notified to stop processing indicator")}catch(n){console.error("[claude-mem cleanup] Failed to notify worker (non-critical):",n)}console.error("[claude-mem cleanup] Cleanup completed successfully"),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)}if(I.isTTY)v(void 0);else{let p="";I.on("data",e=>p+=e),I.on("end",async()=>{let e=p?JSON.parse(p):void 0;await v(e)})}
|
||||
`;try{let l=this.db.prepare(T).all(d,_,...i),S=this.db.prepare(u).all(d,_,...i),p=this.db.prepare(b).all(d,_,...i);return{observations:l,sessions:S.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:p.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(l){return console.error("[SessionStore] Error querying timeline records:",l.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};import F from"path";import{homedir as X}from"os";import{existsSync as B,readFileSync as H}from"fs";function v(){try{let c=F.join(X(),".claude-mem","settings.json");if(B(c)){let e=JSON.parse(H(c,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function C(c){console.error("[claude-mem cleanup] Hook fired",{input:c?{session_id:c.session_id,cwd:c.cwd,reason:c.reason}:null}),c||(console.log("No input provided - this script is designed to run as a Claude Code SessionEnd hook"),console.log(`
|
||||
Expected input format:`),console.log(JSON.stringify({session_id:"string",cwd:"string",transcript_path:"string",hook_event_name:"SessionEnd",reason:"exit"},null,2)),process.exit(0));let{session_id:e,reason:s}=c;console.error("[claude-mem cleanup] Searching for active SDK session",{session_id:e,reason:s});let t=new g,r=t.findActiveSDKSession(e);r||(console.error("[claude-mem cleanup] No active SDK session found",{session_id:e}),t.close(),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)),console.error("[claude-mem cleanup] Active SDK session found",{session_id:r.id,sdk_session_id:r.sdk_session_id,project:r.project,worker_port:r.worker_port}),t.markSessionCompleted(r.id),console.error("[claude-mem cleanup] Session marked as completed in database"),t.close();try{let n=r.worker_port||v();await fetch(`http://127.0.0.1:${n}/sessions/${r.id}/complete`,{method:"POST",signal:AbortSignal.timeout(1e3)}),console.error("[claude-mem cleanup] Worker notified to stop processing indicator")}catch(n){console.error("[claude-mem cleanup] Failed to notify worker (non-critical):",n)}console.error("[claude-mem cleanup] Cleanup completed successfully"),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)}if(I.isTTY)C(void 0);else{let c="";I.on("data",e=>c+=e),I.on("end",async()=>{let e=c?JSON.parse(c):void 0;await C(e)})}
|
||||
|
||||
File diff suppressed because one or more lines are too long
+50
-44
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
import Y from"path";import{stdin as D}from"process";import F from"better-sqlite3";import{join as E,dirname as U,basename as J}from"path";import{homedir as f}from"os";import{existsSync as ee,mkdirSync as M}from"fs";import{fileURLToPath as w}from"url";function X(){return typeof __dirname<"u"?__dirname:U(w(import.meta.url))}var te=X(),m=process.env.CLAUDE_MEM_DATA_DIR||E(f(),".claude-mem"),h=process.env.CLAUDE_CONFIG_DIR||E(f(),".claude"),re=E(m,"archives"),ne=E(m,"logs"),oe=E(m,"trash"),ie=E(m,"backups"),ae=E(m,"settings.json"),L=E(m,"claude-mem.db"),pe=E(m,"vector-db"),de=E(h,"settings.json"),ce=E(h,"commands"),_e=E(h,"CLAUDE.md");function A(p){M(p,{recursive:!0})}var N=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(N||{}),O=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=N[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=N[e].padEnd(5),d=s.padEnd(6),u="";r?.correlationId?u=`[${r.correlationId}] `:r?.sessionId&&(u=`[session-${r.sessionId}] `);let c="";n!=null&&(this.level===0&&typeof n=="object"?c=`
|
||||
`+JSON.stringify(n,null,2):c=" "+this.formatData(n));let l="";if(r){let{sessionId:T,sdkSessionId:b,correlationId:_,...a}=r;Object.keys(a).length>0&&(l=` {${Object.entries(a).map(([k,x])=>`${k}=${x}`).join(", ")}}`)}let S=`[${o}] [${i}] [${d}] ${u}${t}${l}${c}`;e===3?console.error(S):console.log(S)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},C=new O;var g=class{db;constructor(){A(m),this.db=new F(L),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
import re from"path";import{stdin as M}from"process";import W from"better-sqlite3";import{join as m,dirname as P,basename as ae}from"path";import{homedir as L}from"os";import{existsSync as _e,mkdirSync as H}from"fs";import{fileURLToPath as B}from"url";function $(){return typeof __dirname<"u"?__dirname:P(B(import.meta.url))}var j=$(),l=process.env.CLAUDE_MEM_DATA_DIR||m(L(),".claude-mem"),h=process.env.CLAUDE_CONFIG_DIR||m(L(),".claude"),me=m(l,"archives"),Ee=m(l,"logs"),le=m(l,"trash"),Te=m(l,"backups"),be=m(l,"settings.json"),A=m(l,"claude-mem.db"),Se=m(l,"vector-db"),ge=m(h,"settings.json"),Re=m(h,"commands"),he=m(h,"CLAUDE.md");function y(a){H(a,{recursive:!0})}function v(){return m(j,"..","..")}var N=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(N||{}),f=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=N[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let i=new Date().toISOString().replace("T"," ").substring(0,23),o=N[e].padEnd(5),p=s.padEnd(6),_="";r?.correlationId?_=`[${r.correlationId}] `:r?.sessionId&&(_=`[session-${r.sessionId}] `);let c="";n!=null&&(this.level===0&&typeof n=="object"?c=`
|
||||
`+JSON.stringify(n,null,2):c=" "+this.formatData(n));let E="";if(r){let{sessionId:T,sdkSessionId:S,correlationId:u,...d}=r;Object.keys(d).length>0&&(E=` {${Object.entries(d).map(([F,X])=>`${F}=${X}`).join(", ")}}`)}let b=`[${i}] [${o}] [${p}] ${_}${t}${E}${c}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},C=new f;var g=class{db;constructor(){y(l),this.db=new W(A),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable(),this.ensureDiscoveryTokensColumn()}initializeSchema(){try{this.db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS schema_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
version INTEGER UNIQUE NOT NULL,
|
||||
@@ -63,7 +63,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_sdk_session ON session_summaries(sdk_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_project ON session_summaries(project);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
|
||||
`),this.db.prepare("INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)").run(4,new Date().toISOString()),console.error("[SessionStore] Migration004 applied successfully"))}catch(e){throw console.error("[SessionStore] Schema initialization error:",e.message),e}}ensureWorkerPortColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(5))return;this.db.pragma("table_info(sdk_sessions)").some(r=>r.name==="worker_port")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN worker_port INTEGER"),console.error("[SessionStore] Added worker_port column to sdk_sessions table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(5,new Date().toISOString())}catch(e){console.error("[SessionStore] Migration error:",e.message)}}ensurePromptTrackingColumns(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(6))return;this.db.pragma("table_info(sdk_sessions)").some(d=>d.name==="prompt_counter")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN prompt_counter INTEGER DEFAULT 0"),console.error("[SessionStore] Added prompt_counter column to sdk_sessions table")),this.db.pragma("table_info(observations)").some(d=>d.name==="prompt_number")||(this.db.exec("ALTER TABLE observations ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to observations table")),this.db.pragma("table_info(session_summaries)").some(d=>d.name==="prompt_number")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(6,new Date().toISOString())}catch(e){console.error("[SessionStore] Prompt tracking migration error:",e.message)}}removeSessionSummariesUniqueConstraint(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;if(!this.db.pragma("index_list(session_summaries)").some(r=>r.unique===1)){this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString());return}console.error("[SessionStore] Removing UNIQUE constraint from session_summaries.sdk_session_id..."),this.db.exec("BEGIN TRANSACTION");try{this.db.exec(`
|
||||
`),this.db.prepare("INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)").run(4,new Date().toISOString()),console.error("[SessionStore] Migration004 applied successfully"))}catch(e){throw console.error("[SessionStore] Schema initialization error:",e.message),e}}ensureWorkerPortColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(5))return;this.db.pragma("table_info(sdk_sessions)").some(r=>r.name==="worker_port")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN worker_port INTEGER"),console.error("[SessionStore] Added worker_port column to sdk_sessions table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(5,new Date().toISOString())}catch(e){console.error("[SessionStore] Migration error:",e.message)}}ensurePromptTrackingColumns(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(6))return;this.db.pragma("table_info(sdk_sessions)").some(p=>p.name==="prompt_counter")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN prompt_counter INTEGER DEFAULT 0"),console.error("[SessionStore] Added prompt_counter column to sdk_sessions table")),this.db.pragma("table_info(observations)").some(p=>p.name==="prompt_number")||(this.db.exec("ALTER TABLE observations ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to observations table")),this.db.pragma("table_info(session_summaries)").some(p=>p.name==="prompt_number")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(6,new Date().toISOString())}catch(e){console.error("[SessionStore] Prompt tracking migration error:",e.message)}}removeSessionSummariesUniqueConstraint(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;if(!this.db.pragma("index_list(session_summaries)").some(r=>r.unique===1)){this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString());return}console.error("[SessionStore] Removing UNIQUE constraint from session_summaries.sdk_session_id..."),this.db.exec("BEGIN TRANSACTION");try{this.db.exec(`
|
||||
CREATE TABLE session_summaries_new (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
sdk_session_id TEXT NOT NULL,
|
||||
@@ -166,7 +166,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}ensureDiscoveryTokensColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(11))return;this.db.pragma("table_info(observations)").some(i=>i.name==="discovery_tokens")||(this.db.exec("ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to observations table")),this.db.pragma("table_info(session_summaries)").some(i=>i.name==="discovery_tokens")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(11,new Date().toISOString())}catch(e){throw console.error("[SessionStore] Discovery tokens migration error:",e.message),e}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
files_read, files_edited, notes, prompt_number, created_at
|
||||
@@ -244,12 +244,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE id = ?
|
||||
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE id IN (${i})
|
||||
WHERE id IN (${o})
|
||||
ORDER BY created_at_epoch ${n}
|
||||
${o}
|
||||
${i}
|
||||
`).all(...e)}getSummaryForSession(e){return this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
@@ -262,7 +262,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT files_read, files_modified
|
||||
FROM observations
|
||||
WHERE sdk_session_id = ?
|
||||
`).all(e),r=new Set,n=new Set;for(let o of t){if(o.files_read)try{let i=JSON.parse(o.files_read);Array.isArray(i)&&i.forEach(d=>r.add(d))}catch{}if(o.files_modified)try{let i=JSON.parse(o.files_modified);Array.isArray(i)&&i.forEach(d=>n.add(d))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
|
||||
`).all(e),r=new Set,n=new Set;for(let i of t){if(i.files_read)try{let o=JSON.parse(i.files_read);Array.isArray(o)&&o.forEach(p=>r.add(p))}catch{}if(i.files_modified)try{let o=JSON.parse(i.files_modified);Array.isArray(o)&&o.forEach(p=>n.add(p))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
|
||||
SELECT id, claude_session_id, sdk_session_id, project, user_prompt
|
||||
FROM sdk_sessions
|
||||
WHERE id = ?
|
||||
@@ -289,13 +289,17 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
|
||||
`).get(e)?.prompt_counter||1}getPromptCounter(e){return this.db.prepare(`
|
||||
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
|
||||
`).get(e)?.prompt_counter||0}createSDKSession(e,s,t){let r=new Date,n=r.getTime(),i=this.db.prepare(`
|
||||
`).get(e)?.prompt_counter||0}createSDKSession(e,s,t){let r=new Date,n=r.getTime(),o=this.db.prepare(`
|
||||
INSERT OR IGNORE INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, user_prompt, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,t,r.toISOString(),n);return i.lastInsertRowid===0||i.changes===0?this.db.prepare(`
|
||||
`).run(e,e,s,t,r.toISOString(),n);return o.lastInsertRowid===0||o.changes===0?(s&&s.trim()!==""&&this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET project = ?, user_prompt = ?
|
||||
WHERE claude_session_id = ?
|
||||
`).run(s,t,e),this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE claude_session_id = ? LIMIT 1
|
||||
`).get(e).id:i.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
|
||||
`).get(e).id):o.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET sdk_session_id = ?
|
||||
WHERE id = ? AND sdk_session_id IS NULL
|
||||
@@ -312,29 +316,29 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO user_prompts
|
||||
(claude_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r,n=0){let i=new Date,o=i.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
|
||||
`).run(e,e,s,i.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(c.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n,i.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r,n=0){let i=new Date,o=i.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
|
||||
`).run(e,e,s,i.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
(sdk_session_id, project, request, investigated, learned, completed,
|
||||
next_steps, notes, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(c.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
next_steps, notes, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n,i.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
@@ -342,62 +346,64 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT * FROM session_summaries
|
||||
WHERE id IN (${i})
|
||||
WHERE id IN (${o})
|
||||
ORDER BY created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
${i}
|
||||
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT
|
||||
up.*,
|
||||
s.project,
|
||||
s.sdk_session_id
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.id IN (${i})
|
||||
WHERE up.id IN (${o})
|
||||
ORDER BY up.created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],d,u;if(e!==null){let T=`
|
||||
${i}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let i=n?"AND project = ?":"",o=n?[n]:[],p,_;if(e!==null){let T=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id <= ? ${o}
|
||||
WHERE id <= ? ${i}
|
||||
ORDER BY id DESC
|
||||
LIMIT ?
|
||||
`,b=`
|
||||
`,S=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id >= ? ${o}
|
||||
WHERE id >= ? ${i}
|
||||
ORDER BY id ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(T).all(e,...i,t+1),a=this.db.prepare(b).all(e,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
|
||||
`;try{let u=this.db.prepare(T).all(e,...o,t+1),d=this.db.prepare(S).all(e,...o,r+1);if(u.length===0&&d.length===0)return{observations:[],sessions:[],prompts:[]};p=u.length>0?u[u.length-1].created_at_epoch:s,_=d.length>0?d[d.length-1].created_at_epoch:s}catch(u){return console.error("[SessionStore] Error getting boundary observations:",u.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch <= ? ${o}
|
||||
WHERE created_at_epoch <= ? ${i}
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT ?
|
||||
`,b=`
|
||||
`,S=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? ${o}
|
||||
WHERE created_at_epoch >= ? ${i}
|
||||
ORDER BY created_at_epoch ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(T).all(s,...i,t),a=this.db.prepare(b).all(s,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let c=`
|
||||
`;try{let u=this.db.prepare(T).all(s,...o,t),d=this.db.prepare(S).all(s,...o,r+1);if(u.length===0&&d.length===0)return{observations:[],sessions:[],prompts:[]};p=u.length>0?u[u.length-1].created_at_epoch:s,_=d.length>0?d[d.length-1].created_at_epoch:s}catch(u){return console.error("[SessionStore] Error getting boundary timestamps:",u.message),{observations:[],sessions:[],prompts:[]}}}let c=`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${i}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,l=`
|
||||
`,E=`
|
||||
SELECT *
|
||||
FROM session_summaries
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${i}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,S=`
|
||||
`,b=`
|
||||
SELECT up.*, s.project, s.sdk_session_id
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${o.replace("project","s.project")}
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${i.replace("project","s.project")}
|
||||
ORDER BY up.created_at_epoch ASC
|
||||
`;try{let T=this.db.prepare(c).all(d,u,...i),b=this.db.prepare(l).all(d,u,...i),_=this.db.prepare(S).all(d,u,...i);return{observations:T,sessions:b.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:_.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function P(p,e,s){return p==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:p==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:p==="UserPromptSubmit"||p==="PostToolUse"?{continue:!0,suppressOutput:!0}:p==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function v(p,e,s={}){let t=P(p,e,s);return JSON.stringify(t)}import H from"path";import{homedir as B}from"os";import{existsSync as j,readFileSync as $}from"fs";var W=100;function R(){try{let p=H.join(B(),".claude-mem","settings.json");if(j(p)){let e=JSON.parse($(p,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function G(){try{let p=R();return(await fetch(`http://127.0.0.1:${p}/health`,{signal:AbortSignal.timeout(W)})).ok}catch{return!1}}async function y(){if(await G())return;let p=R();throw new Error(`Worker service is not responding on port ${p}.
|
||||
`;try{let T=this.db.prepare(c).all(p,_,...o),S=this.db.prepare(E).all(p,_,...o),u=this.db.prepare(b).all(p,_,...o);return{observations:T,sessions:S.map(d=>({id:d.id,sdk_session_id:d.sdk_session_id,project:d.project,request:d.request,completed:d.completed,next_steps:d.next_steps,created_at:d.created_at,created_at_epoch:d.created_at_epoch})),prompts:u.map(d=>({id:d.id,claude_session_id:d.claude_session_id,project:d.project,prompt:d.prompt_text,created_at:d.created_at,created_at_epoch:d.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function G(a,e,s){return a==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:a==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:a==="UserPromptSubmit"||a==="PostToolUse"?{continue:!0,suppressOutput:!0}:a==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function D(a,e,s={}){let t=G(a,e,s);return JSON.stringify(t)}import k from"path";import{homedir as Y}from"os";import{existsSync as x,readFileSync as K}from"fs";import{execSync as V}from"child_process";var q=100,J=500,Q=10;function R(){try{let a=k.join(Y(),".claude-mem","settings.json");if(x(a)){let e=JSON.parse(K(a,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function U(){try{let a=R();return(await fetch(`http://127.0.0.1:${a}/health`,{signal:AbortSignal.timeout(q)})).ok}catch{return!1}}async function z(){try{let a=v(),e=k.join(a,"ecosystem.config.cjs");if(!x(e))throw new Error(`Ecosystem config not found at ${e}`);V(`pm2 start "${e}"`,{cwd:a,stdio:"pipe",encoding:"utf-8"});for(let s=0;s<Q;s++)if(await new Promise(t=>setTimeout(t,J)),await U())return!0;return!1}catch{return!1}}async function w(){if(await U())return;if(!await z()){let e=R();throw new Error(`Worker service failed to start on port ${e}.
|
||||
|
||||
If you just updated the plugin, PM2's watch mode should restart automatically.
|
||||
If the problem persists, run: pm2 restart claude-mem-worker`)}async function K(p){if(!p)throw new Error("newHook requires input");let{session_id:e,cwd:s,prompt:t}=p,r=Y.basename(s);await y();let n=new g,o=n.createSDKSession(e,r,t),i=n.incrementPromptCounter(o);n.saveUserPrompt(e,i,t),console.error(`[new-hook] Session ${o}, prompt #${i}`),n.close();let d=R(),u=t.startsWith("/")?t.substring(1):t;try{let c=await fetch(`http://127.0.0.1:${d}/sessions/${o}/init`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({project:r,userPrompt:u,promptNumber:i}),signal:AbortSignal.timeout(5e3)});if(!c.ok){let l=await c.text();throw new Error(`Failed to initialize session: ${c.status} ${l}`)}}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(v("UserPromptSubmit",!0))}var I="";D.on("data",p=>I+=p);D.on("end",async()=>{let p=I?JSON.parse(I):void 0;await K(p)});
|
||||
Try manually running: pm2 start ecosystem.config.cjs
|
||||
Or restart: pm2 restart claude-mem-worker`)}}import{appendFileSync as Z}from"fs";import{homedir as ee}from"os";import{join as se}from"path";var te=se(ee(),".claude-mem","silent.log");function O(a,e,s=""){let t=new Date().toISOString(),o=((new Error().stack||"").split(`
|
||||
`)[2]||"").match(/at\s+(?:.*\s+)?\(?([^:]+):(\d+):(\d+)\)?/),p=o?`${o[1].split("/").pop()}:${o[2]}`:"unknown",_=`[${t}] [${p}] ${a}`;if(e!==void 0)try{_+=` ${JSON.stringify(e)}`}catch(c){_+=` [stringify error: ${c}]`}_+=`
|
||||
`;try{Z(te,_)}catch(c){console.error("[silent-debug] Failed to write to log:",c)}return s}async function ne(a){if(!a)throw new Error("newHook requires input");let{session_id:e,cwd:s,prompt:t}=a;O("[new-hook] Input received",{session_id:e,cwd:s,cwd_type:typeof s,cwd_length:s?.length,has_cwd:!!s,prompt_length:t?.length});let r=re.basename(s);O("[new-hook] Project extracted",{project:r,project_type:typeof r,project_length:r?.length,is_empty:r==="",cwd_was:s}),await w();let n=new g,i=n.createSDKSession(e,r,t),o=n.incrementPromptCounter(i);n.saveUserPrompt(e,o,t),console.error(`[new-hook] Session ${i}, prompt #${o}`),n.close();let p=R(),_=t.startsWith("/")?t.substring(1):t;try{let c=await fetch(`http://127.0.0.1:${p}/sessions/${i}/init`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({project:r,userPrompt:_,promptNumber:o}),signal:AbortSignal.timeout(5e3)});if(!c.ok){let E=await c.text();throw new Error(`Failed to initialize session: ${c.status} ${E}`)}}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(D("UserPromptSubmit",!0))}var I="";M.on("data",a=>I+=a);M.on("end",async()=>{let a=I?JSON.parse(I):void 0;await ne(a)});
|
||||
|
||||
+50
-46
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
import{stdin as D}from"process";import X from"better-sqlite3";import{join as m,dirname as U,basename as J}from"path";import{homedir as A}from"os";import{existsSync as ee,mkdirSync as M}from"fs";import{fileURLToPath as w}from"url";function F(){return typeof __dirname<"u"?__dirname:U(w(import.meta.url))}var te=F(),l=process.env.CLAUDE_MEM_DATA_DIR||m(A(),".claude-mem"),N=process.env.CLAUDE_CONFIG_DIR||m(A(),".claude"),re=m(l,"archives"),ne=m(l,"logs"),oe=m(l,"trash"),ie=m(l,"backups"),ae=m(l,"settings.json"),C=m(l,"claude-mem.db"),de=m(l,"vector-db"),pe=m(N,"settings.json"),ce=m(N,"commands"),_e=m(N,"CLAUDE.md");function v(d){M(d,{recursive:!0})}var O=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(O||{}),I=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=O[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=O[e].padEnd(5),p=s.padEnd(6),u="";r?.correlationId?u=`[${r.correlationId}] `:r?.sessionId&&(u=`[session-${r.sessionId}] `);let E="";n!=null&&(this.level===0&&typeof n=="object"?E=`
|
||||
`+JSON.stringify(n,null,2):E=" "+this.formatData(n));let c="";if(r){let{sessionId:T,sdkSessionId:g,correlationId:_,...a}=r;Object.keys(a).length>0&&(c=` {${Object.entries(a).map(([k,x])=>`${k}=${x}`).join(", ")}}`)}let b=`[${o}] [${i}] [${p}] ${u}${t}${c}${E}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},S=new I;var R=class{db;constructor(){v(l),this.db=new X(C),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
import{stdin as M}from"process";import W from"better-sqlite3";import{join as m,dirname as X,basename as te}from"path";import{homedir as A}from"os";import{existsSync as ie,mkdirSync as H}from"fs";import{fileURLToPath as P}from"url";function B(){return typeof __dirname<"u"?__dirname:X(P(import.meta.url))}var j=B(),T=process.env.CLAUDE_MEM_DATA_DIR||m(A(),".claude-mem"),N=process.env.CLAUDE_CONFIG_DIR||m(A(),".claude"),de=m(T,"archives"),pe=m(T,"logs"),ce=m(T,"trash"),_e=m(T,"backups"),ue=m(T,"settings.json"),v=m(T,"claude-mem.db"),me=m(T,"vector-db"),Ee=m(N,"settings.json"),le=m(N,"commands"),Te=m(N,"CLAUDE.md");function y(a){H(a,{recursive:!0})}function C(){return m(j,"..","..")}var O=(o=>(o[o.DEBUG=0]="DEBUG",o[o.INFO=1]="INFO",o[o.WARN=2]="WARN",o[o.ERROR=3]="ERROR",o[o.SILENT=4]="SILENT",o))(O||{}),f=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=O[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,o){if(e<this.level)return;let i=new Date().toISOString().replace("T"," ").substring(0,23),n=O[e].padEnd(5),p=s.padEnd(6),u="";r?.correlationId?u=`[${r.correlationId}] `:r?.sessionId&&(u=`[session-${r.sessionId}] `);let E="";o!=null&&(this.level===0&&typeof o=="object"?E=`
|
||||
`+JSON.stringify(o,null,2):E=" "+this.formatData(o));let c="";if(r){let{sessionId:b,sdkSessionId:R,correlationId:_,...d}=r;Object.keys(d).length>0&&(c=` {${Object.entries(d).map(([w,F])=>`${w}=${F}`).join(", ")}}`)}let l=`[${i}] [${n}] [${p}] ${u}${t}${c}${E}`;e===3?console.error(l):console.log(l)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},S=new f;var g=class{db;constructor(){y(T),this.db=new W(v),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable(),this.ensureDiscoveryTokensColumn()}initializeSchema(){try{this.db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS schema_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
version INTEGER UNIQUE NOT NULL,
|
||||
@@ -166,7 +166,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}ensureDiscoveryTokensColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(11))return;this.db.pragma("table_info(observations)").some(i=>i.name==="discovery_tokens")||(this.db.exec("ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to observations table")),this.db.pragma("table_info(session_summaries)").some(i=>i.name==="discovery_tokens")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(11,new Date().toISOString())}catch(e){throw console.error("[SessionStore] Discovery tokens migration error:",e.message),e}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
files_read, files_edited, notes, prompt_number, created_at
|
||||
@@ -244,12 +244,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE id = ?
|
||||
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,o=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",n=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE id IN (${i})
|
||||
ORDER BY created_at_epoch ${n}
|
||||
${o}
|
||||
WHERE id IN (${n})
|
||||
ORDER BY created_at_epoch ${o}
|
||||
${i}
|
||||
`).all(...e)}getSummaryForSession(e){return this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
@@ -262,7 +262,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT files_read, files_modified
|
||||
FROM observations
|
||||
WHERE sdk_session_id = ?
|
||||
`).all(e),r=new Set,n=new Set;for(let o of t){if(o.files_read)try{let i=JSON.parse(o.files_read);Array.isArray(i)&&i.forEach(p=>r.add(p))}catch{}if(o.files_modified)try{let i=JSON.parse(o.files_modified);Array.isArray(i)&&i.forEach(p=>n.add(p))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
|
||||
`).all(e),r=new Set,o=new Set;for(let i of t){if(i.files_read)try{let n=JSON.parse(i.files_read);Array.isArray(n)&&n.forEach(p=>r.add(p))}catch{}if(i.files_modified)try{let n=JSON.parse(i.files_modified);Array.isArray(n)&&n.forEach(p=>o.add(p))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(o)}}getSessionById(e){return this.db.prepare(`
|
||||
SELECT id, claude_session_id, sdk_session_id, project, user_prompt
|
||||
FROM sdk_sessions
|
||||
WHERE id = ?
|
||||
@@ -289,13 +289,17 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
|
||||
`).get(e)?.prompt_counter||1}getPromptCounter(e){return this.db.prepare(`
|
||||
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
|
||||
`).get(e)?.prompt_counter||0}createSDKSession(e,s,t){let r=new Date,n=r.getTime(),i=this.db.prepare(`
|
||||
`).get(e)?.prompt_counter||0}createSDKSession(e,s,t){let r=new Date,o=r.getTime(),n=this.db.prepare(`
|
||||
INSERT OR IGNORE INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, user_prompt, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,t,r.toISOString(),n);return i.lastInsertRowid===0||i.changes===0?this.db.prepare(`
|
||||
`).run(e,e,s,t,r.toISOString(),o);return n.lastInsertRowid===0||n.changes===0?(s&&s.trim()!==""&&this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET project = ?, user_prompt = ?
|
||||
WHERE claude_session_id = ?
|
||||
`).run(s,t,e),this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE claude_session_id = ? LIMIT 1
|
||||
`).get(e).id:i.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
|
||||
`).get(e).id):n.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET sdk_session_id = ?
|
||||
WHERE id = ? AND sdk_session_id IS NULL
|
||||
@@ -308,33 +312,33 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
FROM sdk_sessions
|
||||
WHERE id = ?
|
||||
LIMIT 1
|
||||
`).get(e)?.worker_port||null}saveUserPrompt(e,s,t){let r=new Date,n=r.getTime();return this.db.prepare(`
|
||||
`).get(e)?.worker_port||null}saveUserPrompt(e,s,t){let r=new Date,o=r.getTime();return this.db.prepare(`
|
||||
INSERT INTO user_prompts
|
||||
(claude_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
`).run(e,s,t,r.toISOString(),o).lastInsertRowid}storeObservation(e,s,t,r,o=0){let i=new Date,n=i.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
`).run(e,e,s,i.toISOString(),n),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,o,i.toISOString(),n);return{id:Number(c.lastInsertRowid),createdAtEpoch:n}}storeSummary(e,s,t,r,o=0){let i=new Date,n=i.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
`).run(e,e,s,i.toISOString(),n),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
(sdk_session_id, project, request, investigated, learned, completed,
|
||||
next_steps, notes, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
next_steps, notes, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,o,i.toISOString(),n);return{id:Number(c.lastInsertRowid),createdAtEpoch:n}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
@@ -342,62 +346,62 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,o=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",n=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT * FROM session_summaries
|
||||
WHERE id IN (${i})
|
||||
ORDER BY created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
WHERE id IN (${n})
|
||||
ORDER BY created_at_epoch ${o}
|
||||
${i}
|
||||
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,o=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",n=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT
|
||||
up.*,
|
||||
s.project,
|
||||
s.sdk_session_id
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.id IN (${i})
|
||||
ORDER BY up.created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],p,u;if(e!==null){let T=`
|
||||
WHERE up.id IN (${n})
|
||||
ORDER BY up.created_at_epoch ${o}
|
||||
${i}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,o){let i=o?"AND project = ?":"",n=o?[o]:[],p,u;if(e!==null){let b=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id <= ? ${o}
|
||||
WHERE id <= ? ${i}
|
||||
ORDER BY id DESC
|
||||
LIMIT ?
|
||||
`,g=`
|
||||
`,R=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id >= ? ${o}
|
||||
WHERE id >= ? ${i}
|
||||
ORDER BY id ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(T).all(e,...i,t+1),a=this.db.prepare(g).all(e,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
|
||||
`;try{let _=this.db.prepare(b).all(e,...n,t+1),d=this.db.prepare(R).all(e,...n,r+1);if(_.length===0&&d.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=d.length>0?d[d.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let b=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch <= ? ${o}
|
||||
WHERE created_at_epoch <= ? ${i}
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT ?
|
||||
`,g=`
|
||||
`,R=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? ${o}
|
||||
WHERE created_at_epoch >= ? ${i}
|
||||
ORDER BY created_at_epoch ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(T).all(s,...i,t),a=this.db.prepare(g).all(s,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let E=`
|
||||
`;try{let _=this.db.prepare(b).all(s,...n,t),d=this.db.prepare(R).all(s,...n,r+1);if(_.length===0&&d.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=d.length>0?d[d.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let E=`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${i}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,c=`
|
||||
SELECT *
|
||||
FROM session_summaries
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${i}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,b=`
|
||||
`,l=`
|
||||
SELECT up.*, s.project, s.sdk_session_id
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${o.replace("project","s.project")}
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${i.replace("project","s.project")}
|
||||
ORDER BY up.created_at_epoch ASC
|
||||
`;try{let T=this.db.prepare(E).all(p,u,...i),g=this.db.prepare(c).all(p,u,...i),_=this.db.prepare(b).all(p,u,...i);return{observations:T,sessions:g.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:_.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function H(d,e,s){return d==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:d==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:d==="UserPromptSubmit"||d==="PostToolUse"?{continue:!0,suppressOutput:!0}:d==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function f(d,e,s={}){let t=H(d,e,s);return JSON.stringify(t)}import P from"path";import{homedir as B}from"os";import{existsSync as j,readFileSync as $}from"fs";var W=100;function h(){try{let d=P.join(B(),".claude-mem","settings.json");if(j(d)){let e=JSON.parse($(d,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function G(){try{let d=h();return(await fetch(`http://127.0.0.1:${d}/health`,{signal:AbortSignal.timeout(W)})).ok}catch{return!1}}async function y(){if(await G())return;let d=h();throw new Error(`Worker service is not responding on port ${d}.
|
||||
`;try{let b=this.db.prepare(E).all(p,u,...n),R=this.db.prepare(c).all(p,u,...n),_=this.db.prepare(l).all(p,u,...n);return{observations:b,sessions:R.map(d=>({id:d.id,sdk_session_id:d.sdk_session_id,project:d.project,request:d.request,completed:d.completed,next_steps:d.next_steps,created_at:d.created_at,created_at_epoch:d.created_at_epoch})),prompts:_.map(d=>({id:d.id,claude_session_id:d.claude_session_id,project:d.project,prompt:d.prompt_text,created_at:d.created_at,created_at_epoch:d.created_at_epoch}))}}catch(b){return console.error("[SessionStore] Error querying timeline records:",b.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function $(a,e,s){return a==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:a==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:a==="UserPromptSubmit"||a==="PostToolUse"?{continue:!0,suppressOutput:!0}:a==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function I(a,e,s={}){let t=$(a,e,s);return JSON.stringify(t)}import D from"path";import{homedir as G}from"os";import{existsSync as k,readFileSync as Y}from"fs";import{execSync as K}from"child_process";var V=100,q=500,J=10;function h(){try{let a=D.join(G(),".claude-mem","settings.json");if(k(a)){let e=JSON.parse(Y(a,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function x(){try{let a=h();return(await fetch(`http://127.0.0.1:${a}/health`,{signal:AbortSignal.timeout(V)})).ok}catch{return!1}}async function Q(){try{let a=C(),e=D.join(a,"ecosystem.config.cjs");if(!k(e))throw new Error(`Ecosystem config not found at ${e}`);K(`pm2 start "${e}"`,{cwd:a,stdio:"pipe",encoding:"utf-8"});for(let s=0;s<J;s++)if(await new Promise(t=>setTimeout(t,q)),await x())return!0;return!1}catch{return!1}}async function U(){if(await x())return;if(!await Q()){let e=h();throw new Error(`Worker service failed to start on port ${e}.
|
||||
|
||||
If you just updated the plugin, PM2's watch mode should restart automatically.
|
||||
If the problem persists, run: pm2 restart claude-mem-worker`)}var Y=new Set(["ListMcpResourcesTool","SlashCommand","Skill","TodoWrite","AskUserQuestion"]);async function K(d){if(!d)throw new Error("saveHook requires input");let{session_id:e,cwd:s,tool_name:t,tool_input:r,tool_response:n}=d;if(Y.has(t)){console.log(f("PostToolUse",!0));return}await y();let o=new R,i=o.createSDKSession(e,"",""),p=o.getPromptCounter(i);o.close();let u=S.formatTool(t,r),E=h();S.dataIn("HOOK",`PostToolUse: ${u}`,{sessionId:i,workerPort:E});try{let c=await fetch(`http://127.0.0.1:${E}/sessions/${i}/observations`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({tool_name:t,tool_input:r!==void 0?JSON.stringify(r):"{}",tool_response:n!==void 0?JSON.stringify(n):"{}",prompt_number:p,cwd:s||""}),signal:AbortSignal.timeout(2e3)});if(!c.ok){let b=await c.text();throw S.failure("HOOK","Failed to send observation",{sessionId:i,status:c.status},b),new Error(`Failed to send observation to worker: ${c.status} ${b}`)}S.debug("HOOK","Observation sent successfully",{sessionId:i,toolName:t})}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(f("PostToolUse",!0))}var L="";D.on("data",d=>L+=d);D.on("end",async()=>{let d=L?JSON.parse(L):void 0;await K(d)});
|
||||
Try manually running: pm2 start ecosystem.config.cjs
|
||||
Or restart: pm2 restart claude-mem-worker`)}}var z=new Set(["ListMcpResourcesTool","SlashCommand","Skill","TodoWrite","AskUserQuestion"]);async function Z(a){if(!a)throw new Error("saveHook requires input");let{session_id:e,cwd:s,tool_name:t,tool_input:r,tool_response:o}=a;if(z.has(t)){console.log(I("PostToolUse",!0));return}await U();let i=new g,n=i.createSDKSession(e,"",""),p=i.getPromptCounter(n);i.close();let u=S.formatTool(t,r),E=h();S.dataIn("HOOK",`PostToolUse: ${u}`,{sessionId:n,workerPort:E});try{let c=await fetch(`http://127.0.0.1:${E}/sessions/${n}/observations`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({tool_name:t,tool_input:r!==void 0?JSON.stringify(r):"{}",tool_response:o!==void 0?JSON.stringify(o):"{}",prompt_number:p,cwd:s||""}),signal:AbortSignal.timeout(2e3)});if(!c.ok){let l=await c.text();throw S.failure("HOOK","Failed to send observation",{sessionId:n,status:c.status},l),new Error(`Failed to send observation to worker: ${c.status} ${l}`)}S.debug("HOOK","Observation sent successfully",{sessionId:n,toolName:t})}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(I("PostToolUse",!0))}var L="";M.on("data",a=>L+=a);M.on("end",async()=>{let a=L?JSON.parse(L):void 0;await Z(a)});
|
||||
|
||||
Executable
+652
File diff suppressed because one or more lines are too long
+152
-103
File diff suppressed because one or more lines are too long
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
import{stdin as k}from"process";import{readFileSync as x,existsSync as U}from"fs";import B from"better-sqlite3";import{join as m,dirname as F,basename as ne}from"path";import{homedir as A}from"os";import{existsSync as de,mkdirSync as X}from"fs";import{fileURLToPath as H}from"url";function j(){return typeof __dirname<"u"?__dirname:F(H(import.meta.url))}var ce=j(),E=process.env.CLAUDE_MEM_DATA_DIR||m(A(),".claude-mem"),f=process.env.CLAUDE_CONFIG_DIR||m(A(),".claude"),ue=m(E,"archives"),_e=m(E,"logs"),me=m(E,"trash"),le=m(E,"backups"),Ee=m(E,"settings.json"),y=m(E,"claude-mem.db"),Te=m(E,"vector-db"),ge=m(f,"settings.json"),Se=m(f,"commands"),be=m(f,"CLAUDE.md");function C(i){X(i,{recursive:!0})}var O=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(O||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=O[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let a=new Date().toISOString().replace("T"," ").substring(0,23),o=O[e].padEnd(5),d=s.padEnd(6),c="";r?.correlationId?c=`[${r.correlationId}] `:r?.sessionId&&(c=`[session-${r.sessionId}] `);let u="";n!=null&&(this.level===0&&typeof n=="object"?u=`
|
||||
`+JSON.stringify(n,null,2):u=" "+this.formatData(n));let l="";if(r){let{sessionId:T,sdkSessionId:S,correlationId:_,...p}=r;Object.keys(p).length>0&&(l=` {${Object.entries(p).map(([M,w])=>`${M}=${w}`).join(", ")}}`)}let b=`[${a}] [${o}] [${d}] ${c}${t}${l}${u}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},g=new N;var R=class{db;constructor(){C(E),this.db=new B(y),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
import{stdin as w}from"process";import{readFileSync as F,existsSync as X}from"fs";import Y from"better-sqlite3";import{join as m,dirname as B,basename as pe}from"path";import{homedir as y}from"os";import{existsSync as Ee,mkdirSync as j}from"fs";import{fileURLToPath as $}from"url";function W(){return typeof __dirname<"u"?__dirname:B($(import.meta.url))}var G=W(),l=process.env.CLAUDE_MEM_DATA_DIR||m(y(),".claude-mem"),f=process.env.CLAUDE_CONFIG_DIR||m(y(),".claude"),Te=m(l,"archives"),ge=m(l,"logs"),Se=m(l,"trash"),be=m(l,"backups"),Re=m(l,"settings.json"),A=m(l,"claude-mem.db"),he=m(l,"vector-db"),fe=m(f,"settings.json"),Oe=m(f,"commands"),Ne=m(f,"CLAUDE.md");function v(a){j(a,{recursive:!0})}function C(){return m(G,"..","..")}var O=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(O||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=O[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let i=new Date().toISOString().replace("T"," ").substring(0,23),o=O[e].padEnd(5),d=s.padEnd(6),p="";r?.correlationId?p=`[${r.correlationId}] `:r?.sessionId&&(p=`[session-${r.sessionId}] `);let u="";n!=null&&(this.level===0&&typeof n=="object"?u=`
|
||||
`+JSON.stringify(n,null,2):u=" "+this.formatData(n));let E="";if(r){let{sessionId:T,sdkSessionId:b,correlationId:_,...c}=r;Object.keys(c).length>0&&(E=` {${Object.entries(c).map(([H,P])=>`${H}=${P}`).join(", ")}}`)}let S=`[${i}] [${o}] [${d}] ${p}${t}${E}${u}`;e===3?console.error(S):console.log(S)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},g=new N;var R=class{db;constructor(){v(l),this.db=new Y(A),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable(),this.ensureDiscoveryTokensColumn()}initializeSchema(){try{this.db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS schema_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
version INTEGER UNIQUE NOT NULL,
|
||||
@@ -166,7 +166,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}ensureDiscoveryTokensColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(11))return;this.db.pragma("table_info(observations)").some(i=>i.name==="discovery_tokens")||(this.db.exec("ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to observations table")),this.db.pragma("table_info(session_summaries)").some(i=>i.name==="discovery_tokens")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(11,new Date().toISOString())}catch(e){throw console.error("[SessionStore] Discovery tokens migration error:",e.message),e}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
files_read, files_edited, notes, prompt_number, created_at
|
||||
@@ -244,12 +244,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE id = ?
|
||||
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",a=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE id IN (${o})
|
||||
ORDER BY created_at_epoch ${n}
|
||||
${a}
|
||||
${i}
|
||||
`).all(...e)}getSummaryForSession(e){return this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
@@ -262,7 +262,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT files_read, files_modified
|
||||
FROM observations
|
||||
WHERE sdk_session_id = ?
|
||||
`).all(e),r=new Set,n=new Set;for(let a of t){if(a.files_read)try{let o=JSON.parse(a.files_read);Array.isArray(o)&&o.forEach(d=>r.add(d))}catch{}if(a.files_modified)try{let o=JSON.parse(a.files_modified);Array.isArray(o)&&o.forEach(d=>n.add(d))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
|
||||
`).all(e),r=new Set,n=new Set;for(let i of t){if(i.files_read)try{let o=JSON.parse(i.files_read);Array.isArray(o)&&o.forEach(d=>r.add(d))}catch{}if(i.files_modified)try{let o=JSON.parse(i.files_modified);Array.isArray(o)&&o.forEach(d=>n.add(d))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
|
||||
SELECT id, claude_session_id, sdk_session_id, project, user_prompt
|
||||
FROM sdk_sessions
|
||||
WHERE id = ?
|
||||
@@ -293,9 +293,13 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT OR IGNORE INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, user_prompt, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,t,r.toISOString(),n);return o.lastInsertRowid===0||o.changes===0?this.db.prepare(`
|
||||
`).run(e,e,s,t,r.toISOString(),n);return o.lastInsertRowid===0||o.changes===0?(s&&s.trim()!==""&&this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET project = ?, user_prompt = ?
|
||||
WHERE claude_session_id = ?
|
||||
`).run(s,t,e),this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE claude_session_id = ? LIMIT 1
|
||||
`).get(e).id:o.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
|
||||
`).get(e).id):o.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET sdk_session_id = ?
|
||||
WHERE id = ? AND sdk_session_id IS NULL
|
||||
@@ -312,29 +316,29 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO user_prompts
|
||||
(claude_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r){let n=new Date,a=n.getTime();this.db.prepare(`
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r,n=0){let i=new Date,o=i.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),a),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
`).run(e,e,s,i.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),a);return{id:Number(u.lastInsertRowid),createdAtEpoch:a}}storeSummary(e,s,t,r){let n=new Date,a=n.getTime();this.db.prepare(`
|
||||
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n,i.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r,n=0){let i=new Date,o=i.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),a),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
`).run(e,e,s,i.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
(sdk_session_id, project, request, investigated, learned, completed,
|
||||
next_steps, notes, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),a);return{id:Number(u.lastInsertRowid),createdAtEpoch:a}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
next_steps, notes, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n,i.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
@@ -342,12 +346,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",a=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT * FROM session_summaries
|
||||
WHERE id IN (${o})
|
||||
ORDER BY created_at_epoch ${n}
|
||||
${a}
|
||||
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",a=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
${i}
|
||||
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT
|
||||
up.*,
|
||||
s.project,
|
||||
@@ -356,63 +360,63 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.id IN (${o})
|
||||
ORDER BY up.created_at_epoch ${n}
|
||||
${a}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let a=n?"AND project = ?":"",o=n?[n]:[],d,c;if(e!==null){let T=`
|
||||
${i}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let i=n?"AND project = ?":"",o=n?[n]:[],d,p;if(e!==null){let T=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id <= ? ${a}
|
||||
WHERE id <= ? ${i}
|
||||
ORDER BY id DESC
|
||||
LIMIT ?
|
||||
`,S=`
|
||||
`,b=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id >= ? ${a}
|
||||
WHERE id >= ? ${i}
|
||||
ORDER BY id ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(T).all(e,...o,t+1),p=this.db.prepare(S).all(e,...o,r+1);if(_.length===0&&p.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,c=p.length>0?p[p.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
|
||||
`;try{let _=this.db.prepare(T).all(e,...o,t+1),c=this.db.prepare(b).all(e,...o,r+1);if(_.length===0&&c.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,p=c.length>0?c[c.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch <= ? ${a}
|
||||
WHERE created_at_epoch <= ? ${i}
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT ?
|
||||
`,S=`
|
||||
`,b=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? ${a}
|
||||
WHERE created_at_epoch >= ? ${i}
|
||||
ORDER BY created_at_epoch ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(T).all(s,...o,t),p=this.db.prepare(S).all(s,...o,r+1);if(_.length===0&&p.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,c=p.length>0?p[p.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let u=`
|
||||
`;try{let _=this.db.prepare(T).all(s,...o,t),c=this.db.prepare(b).all(s,...o,r+1);if(_.length===0&&c.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,p=c.length>0?c[c.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let u=`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${a}
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${i}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,l=`
|
||||
`,E=`
|
||||
SELECT *
|
||||
FROM session_summaries
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${a}
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${i}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,b=`
|
||||
`,S=`
|
||||
SELECT up.*, s.project, s.sdk_session_id
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${a.replace("project","s.project")}
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${i.replace("project","s.project")}
|
||||
ORDER BY up.created_at_epoch ASC
|
||||
`;try{let T=this.db.prepare(u).all(d,c,...o),S=this.db.prepare(l).all(d,c,...o),_=this.db.prepare(b).all(d,c,...o);return{observations:T,sessions:S.map(p=>({id:p.id,sdk_session_id:p.sdk_session_id,project:p.project,request:p.request,completed:p.completed,next_steps:p.next_steps,created_at:p.created_at,created_at_epoch:p.created_at_epoch})),prompts:_.map(p=>({id:p.id,claude_session_id:p.claude_session_id,project:p.project,prompt:p.prompt_text,created_at:p.created_at,created_at_epoch:p.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function P(i,e,s){return i==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:i==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:i==="UserPromptSubmit"||i==="PostToolUse"?{continue:!0,suppressOutput:!0}:i==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function v(i,e,s={}){let t=P(i,e,s);return JSON.stringify(t)}import $ from"path";import{homedir as W}from"os";import{existsSync as G,readFileSync as Y}from"fs";var K=100;function h(){try{let i=$.join(W(),".claude-mem","settings.json");if(G(i)){let e=JSON.parse(Y(i,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function q(){try{let i=h();return(await fetch(`http://127.0.0.1:${i}/health`,{signal:AbortSignal.timeout(K)})).ok}catch{return!1}}async function D(){if(await q())return;let i=h();throw new Error(`Worker service is not responding on port ${i}.
|
||||
`;try{let T=this.db.prepare(u).all(d,p,...o),b=this.db.prepare(E).all(d,p,...o),_=this.db.prepare(S).all(d,p,...o);return{observations:T,sessions:b.map(c=>({id:c.id,sdk_session_id:c.sdk_session_id,project:c.project,request:c.request,completed:c.completed,next_steps:c.next_steps,created_at:c.created_at,created_at_epoch:c.created_at_epoch})),prompts:_.map(c=>({id:c.id,claude_session_id:c.claude_session_id,project:c.project,prompt:c.prompt_text,created_at:c.created_at,created_at_epoch:c.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function K(a,e,s){return a==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:a==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:a==="UserPromptSubmit"||a==="PostToolUse"?{continue:!0,suppressOutput:!0}:a==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function D(a,e,s={}){let t=K(a,e,s);return JSON.stringify(t)}import k from"path";import{homedir as q}from"os";import{existsSync as x,readFileSync as V}from"fs";import{execSync as J}from"child_process";var Q=100,z=500,Z=10;function h(){try{let a=k.join(q(),".claude-mem","settings.json");if(x(a)){let e=JSON.parse(V(a,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function U(){try{let a=h();return(await fetch(`http://127.0.0.1:${a}/health`,{signal:AbortSignal.timeout(Q)})).ok}catch{return!1}}async function ee(){try{let a=C(),e=k.join(a,"ecosystem.config.cjs");if(!x(e))throw new Error(`Ecosystem config not found at ${e}`);J(`pm2 start "${e}"`,{cwd:a,stdio:"pipe",encoding:"utf-8"});for(let s=0;s<Z;s++)if(await new Promise(t=>setTimeout(t,z)),await U())return!0;return!1}catch{return!1}}async function M(){if(await U())return;if(!await ee()){let e=h();throw new Error(`Worker service failed to start on port ${e}.
|
||||
|
||||
If you just updated the plugin, PM2's watch mode should restart automatically.
|
||||
If the problem persists, run: pm2 restart claude-mem-worker`)}import{appendFileSync as V}from"fs";import{homedir as J}from"os";import{join as Q}from"path";var z=Q(J(),".claude-mem","silent.log");function I(i,e,s=""){let t=new Date().toISOString(),o=((new Error().stack||"").split(`
|
||||
`)[2]||"").match(/at\s+(?:.*\s+)?\(?([^:]+):(\d+):(\d+)\)?/),d=o?`${o[1].split("/").pop()}:${o[2]}`:"unknown",c=`[${t}] [${d}] ${i}`;if(e!==void 0)try{c+=` ${JSON.stringify(e)}`}catch(u){c+=` [stringify error: ${u}]`}c+=`
|
||||
`;try{V(z,c)}catch(u){console.error("[silent-debug] Failed to write to log:",u)}return s}function Z(i){if(!i||!U(i))return"";try{let e=x(i,"utf-8").trim();if(!e)return"";let s=e.split(`
|
||||
Try manually running: pm2 start ecosystem.config.cjs
|
||||
Or restart: pm2 restart claude-mem-worker`)}}import{appendFileSync as se}from"fs";import{homedir as te}from"os";import{join as re}from"path";var ne=re(te(),".claude-mem","silent.log");function I(a,e,s=""){let t=new Date().toISOString(),o=((new Error().stack||"").split(`
|
||||
`)[2]||"").match(/at\s+(?:.*\s+)?\(?([^:]+):(\d+):(\d+)\)?/),d=o?`${o[1].split("/").pop()}:${o[2]}`:"unknown",p=`[${t}] [${d}] ${a}`;if(e!==void 0)try{p+=` ${JSON.stringify(e)}`}catch(u){p+=` [stringify error: ${u}]`}p+=`
|
||||
`;try{se(ne,p)}catch(u){console.error("[silent-debug] Failed to write to log:",u)}return s}function oe(a){if(!a||!X(a))return"";try{let e=F(a,"utf-8").trim();if(!e)return"";let s=e.split(`
|
||||
`);for(let t=s.length-1;t>=0;t--)try{let r=JSON.parse(s[t]);if(r.type==="user"&&r.message?.content){let n=r.message.content;if(typeof n=="string")return n;if(Array.isArray(n))return n.filter(o=>o.type==="text").map(o=>o.text).join(`
|
||||
`)}}catch{continue}}catch(e){g.error("HOOK","Failed to read transcript",{transcriptPath:i},e)}return""}function ee(i){if(!i||!U(i))return"";try{let e=x(i,"utf-8").trim();if(!e)return"";let s=e.split(`
|
||||
`);for(let t=s.length-1;t>=0;t--)try{let r=JSON.parse(s[t]);if(r.type==="assistant"&&r.message?.content){let n="",a=r.message.content;return typeof a=="string"?n=a:Array.isArray(a)&&(n=a.filter(d=>d.type==="text").map(d=>d.text).join(`
|
||||
`)}}catch{continue}}catch(e){g.error("HOOK","Failed to read transcript",{transcriptPath:a},e)}return""}function ie(a){if(!a||!X(a))return"";try{let e=F(a,"utf-8").trim();if(!e)return"";let s=e.split(`
|
||||
`);for(let t=s.length-1;t>=0;t--)try{let r=JSON.parse(s[t]);if(r.type==="assistant"&&r.message?.content){let n="",i=r.message.content;return typeof i=="string"?n=i:Array.isArray(i)&&(n=i.filter(d=>d.type==="text").map(d=>d.text).join(`
|
||||
`)),n=n.replace(/<system-reminder>[\s\S]*?<\/system-reminder>/g,""),n=n.replace(/\n{3,}/g,`
|
||||
|
||||
`).trim(),n}}catch{continue}}catch(e){g.error("HOOK","Failed to read transcript",{transcriptPath:i},e)}return""}async function se(i){if(!i)throw new Error("summaryHook requires input");let{session_id:e}=i;await D();let s=new R,t=s.createSDKSession(e,"",""),r=s.getPromptCounter(t),n=s.db.prepare(`
|
||||
`).trim(),n}}catch{continue}}catch(e){g.error("HOOK","Failed to read transcript",{transcriptPath:a},e)}return""}async function ae(a){if(!a)throw new Error("summaryHook requires input");let{session_id:e}=a;await M();let s=new R,t=s.createSDKSession(e,"",""),r=s.getPromptCounter(t),n=s.db.prepare(`
|
||||
SELECT id, claude_session_id, sdk_session_id, project
|
||||
FROM sdk_sessions WHERE id = ?
|
||||
`).get(t),a=s.db.prepare(`
|
||||
`).get(t),i=s.db.prepare(`
|
||||
SELECT COUNT(*) as count
|
||||
FROM observations
|
||||
WHERE sdk_session_id = ?
|
||||
`).get(n?.sdk_session_id);I("[summary-hook] Session diagnostics",{claudeSessionId:e,sessionDbId:t,sdkSessionId:n?.sdk_session_id,project:n?.project,promptNumber:r,observationCount:a?.count||0,transcriptPath:i.transcript_path}),s.close();let o=h(),d=Z(i.transcript_path||""),c=ee(i.transcript_path||"");I("[summary-hook] Extracted messages",{hasLastUserMessage:!!d,hasLastAssistantMessage:!!c,lastAssistantPreview:c.substring(0,200),lastAssistantLength:c.length}),g.dataIn("HOOK","Stop: Requesting summary",{sessionId:t,workerPort:o,promptNumber:r,hasLastUserMessage:!!d,hasLastAssistantMessage:!!c});try{let u=await fetch(`http://127.0.0.1:${o}/sessions/${t}/summarize`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({prompt_number:r,last_user_message:d,last_assistant_message:c}),signal:AbortSignal.timeout(2e3)});if(!u.ok){let l=await u.text();throw g.failure("HOOK","Failed to generate summary",{sessionId:t,status:u.status},l),new Error(`Failed to request summary from worker: ${u.status} ${l}`)}g.debug("HOOK","Summary request sent successfully",{sessionId:t})}catch(u){throw u.cause?.code==="ECONNREFUSED"||u.name==="TimeoutError"||u.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):u}finally{await fetch(`http://127.0.0.1:${o}/api/processing`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({isProcessing:!1})})}console.log(v("Stop",!0))}var L="";k.on("data",i=>L+=i);k.on("end",async()=>{let i=L?JSON.parse(L):void 0;await se(i)});
|
||||
`).get(n?.sdk_session_id);I("[summary-hook] Session diagnostics",{claudeSessionId:e,sessionDbId:t,sdkSessionId:n?.sdk_session_id,project:n?.project,promptNumber:r,observationCount:i?.count||0,transcriptPath:a.transcript_path}),s.close();let o=h(),d=oe(a.transcript_path||""),p=ie(a.transcript_path||"");I("[summary-hook] Extracted messages",{hasLastUserMessage:!!d,hasLastAssistantMessage:!!p,lastAssistantPreview:p.substring(0,200),lastAssistantLength:p.length}),g.dataIn("HOOK","Stop: Requesting summary",{sessionId:t,workerPort:o,promptNumber:r,hasLastUserMessage:!!d,hasLastAssistantMessage:!!p});try{let u=await fetch(`http://127.0.0.1:${o}/sessions/${t}/summarize`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({prompt_number:r,last_user_message:d,last_assistant_message:p}),signal:AbortSignal.timeout(2e3)});if(!u.ok){let E=await u.text();throw g.failure("HOOK","Failed to generate summary",{sessionId:t,status:u.status},E),new Error(`Failed to request summary from worker: ${u.status} ${E}`)}g.debug("HOOK","Summary request sent successfully",{sessionId:t})}catch(u){throw u.cause?.code==="ECONNREFUSED"||u.name==="TimeoutError"||u.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):u}finally{await fetch(`http://127.0.0.1:${o}/api/processing`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({isProcessing:!1})})}console.log(D("Stop",!0))}var L="";w.on("data",a=>L+=a);w.on("end",async()=>{let a=L?JSON.parse(L):void 0;await ae(a)});
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env node
|
||||
import{execSync as p}from"child_process";import{join as r}from"path";import{homedir as s}from"os";import{existsSync as u}from"fs";import i from"path";import{homedir as a}from"os";import{existsSync as c,readFileSync as l}from"fs";function n(){try{let e=i.join(a(),".claude-mem","settings.json");if(c(e)){let t=JSON.parse(l(e,"utf-8")),o=parseInt(t.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(o))return o}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}var m=r(s(),".claude","plugins","marketplaces","thedotmack"),d=r(m,"node_modules");u(d)||(console.error(`
|
||||
import{execSync as _}from"child_process";import{join as i}from"path";import{homedir as p}from"os";import{existsSync as x}from"fs";import l from"path";import{homedir as f}from"os";import{existsSync as g,readFileSync as h}from"fs";import{join as t,dirname as m,basename as y}from"path";import{homedir as c}from"os";import{fileURLToPath as u}from"url";function d(){return typeof __dirname<"u"?__dirname:m(u(import.meta.url))}var A=d(),e=process.env.CLAUDE_MEM_DATA_DIR||t(c(),".claude-mem"),s=process.env.CLAUDE_CONFIG_DIR||t(c(),".claude"),P=t(e,"archives"),w=t(e,"logs"),C=t(e,"trash"),I=t(e,"backups"),b=t(e,"settings.json"),v=t(e,"claude-mem.db"),U=t(e,"vector-db"),M=t(s,"settings.json"),O=t(s,"commands"),L=t(s,"CLAUDE.md");function a(){try{let o=l.join(f(),".claude-mem","settings.json");if(g(o)){let n=JSON.parse(h(o,"utf-8")),r=parseInt(n.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(r))return r}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}var D=i(p(),".claude","plugins","marketplaces","thedotmack"),k=i(D,"node_modules");x(k)||(console.error(`
|
||||
---
|
||||
\u{1F389} Note: This appears under Plugin Hook Error, but it's not an error. That's the only option for
|
||||
user messages in Claude Code UI until a better method is provided.
|
||||
@@ -17,12 +17,15 @@ Dependencies have been installed in the background. This only happens once.
|
||||
Thank you for installing Claude-Mem!
|
||||
|
||||
This message was not added to your startup context, so you can continue working as normal.
|
||||
`),process.exit(3));try{let e=r(s(),".claude","plugins","marketplaces","thedotmack","plugin","scripts","context-hook.js"),t=p(`node "${e}" --colors`,{encoding:"utf8"}),o=n();console.error(`
|
||||
`),process.exit(3));try{let o=i(p(),".claude","plugins","marketplaces","thedotmack","plugin","scripts","context-hook.js"),n=_(`node "${o}" --colors`,{encoding:"utf8"}),r=a();console.error(`
|
||||
|
||||
\u{1F4DD} Claude-Mem Context Loaded
|
||||
\u2139\uFE0F Note: This appears as stderr but is informational only
|
||||
|
||||
`+t+`
|
||||
`+n+`
|
||||
|
||||
\u{1F4FA} Watch live in browser http://localhost:${o}/ (New! v5.1)
|
||||
`)}catch(e){console.error(`\u274C Failed to load context display: ${e}`)}process.exit(3);
|
||||
\u{1F4AC} Feedback & Support
|
||||
https://github.com/thedotmack/claude-mem/discussions/110
|
||||
|
||||
\u{1F4FA} Watch live in browser http://localhost:${r}/
|
||||
`)}catch(o){console.error(`\u274C Failed to load context display: ${o}`)}process.exit(3);
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -1,196 +1,123 @@
|
||||
---
|
||||
name: mem-search
|
||||
description: Search claude-mem's persistent cross-session memory database to find work from previous conversations days, weeks, or months ago. Access past session summaries, bug fixes, feature implementations, and decisions that are NOT in the current conversation context. Use when user asks "did we already solve this?", "how did we do X last time?", "what happened in last week's session?", or needs information from previous sessions stored in the PM2-managed database. Searches observations, session summaries, and user prompts across entire project history.
|
||||
description: Search claude-mem's persistent cross-session memory database. Use when user asks "did we already solve this?", "how did we do X last time?", or needs work from previous sessions.
|
||||
---
|
||||
|
||||
# Memory Search
|
||||
|
||||
Access claude-mem's persistent cross-session memory through HTTP API. Find past work, understand context across sessions, and learn from previous decisions.
|
||||
Search past work across all sessions. Simple workflow: search → get IDs → fetch details by ID.
|
||||
|
||||
## When to Use This Skill
|
||||
## When to Use
|
||||
|
||||
**Use when users ask about work from PREVIOUS sessions** (not current conversation):
|
||||
Use when users ask about PREVIOUS sessions (not current conversation):
|
||||
- "Did we already fix this?"
|
||||
- "How did we solve X last time?"
|
||||
- "What happened last week?"
|
||||
|
||||
### Temporal Triggers (Key Indicators)
|
||||
- "Did we **already** fix this bug?" or "Have we seen this error **before**?"
|
||||
- "How did we solve X **last time**?" or "What approach did we take **previously**?"
|
||||
- "What did we do in **yesterday's/last week's/last month's** session?"
|
||||
- "**When** did we last work on this?" or "What's the **history** of this file?"
|
||||
## The Workflow
|
||||
|
||||
### Cross-Session Queries
|
||||
- "Show me all authentication-related changes **across all sessions**"
|
||||
- "What features did we add **last month**?" (not "today" or "this session")
|
||||
- "Why did we choose this approach **before**?" (decisions from past sessions)
|
||||
- "What files did we modify **when we added X**?" (historical context)
|
||||
**ALWAYS follow this exact flow:**
|
||||
|
||||
**Do NOT use** for current session work, future planning, or questions Claude can answer from current conversation context.
|
||||
1. **Search** - Get an index of results with IDs
|
||||
2. **Review** - Look at titles/dates, pick relevant IDs
|
||||
3. **Fetch** - Get full details ONLY for those IDs
|
||||
|
||||
## Common Trigger Phrases
|
||||
### Step 1: Search Everything
|
||||
|
||||
This skill activates when detecting phrases about **cross-session history**:
|
||||
```bash
|
||||
curl "http://localhost:37777/api/search?query=authentication&format=index&limit=5"
|
||||
```
|
||||
|
||||
- "Did we already solve this?" / "Have we done this before?"
|
||||
- "How did we implement X last time?"
|
||||
- "What did we work on yesterday/last week/last month?"
|
||||
- "Show me the history of [file/feature/decision]"
|
||||
- "When did we fix/add/change X?"
|
||||
- "What was happening around [date/time]?"
|
||||
- "Catch me up on recent sessions" / "What have we been doing?"
|
||||
- "What changes to [filename] across all sessions?"
|
||||
**Required parameters:**
|
||||
- `query` - Search term
|
||||
- `format=index` - ALWAYS start with index (lightweight)
|
||||
- `limit=5` - Start small (3-5 results)
|
||||
|
||||
**Unique identifiers:** claude-mem, persistent memory, cross-session database, session history, PM2-managed database
|
||||
**Returns:**
|
||||
```
|
||||
1. [feature] Added JWT authentication
|
||||
Date: 11/17/2025, 3:48:45 PM
|
||||
ID: 11131
|
||||
|
||||
## Available Operations
|
||||
2. [bugfix] Fixed auth token expiration
|
||||
Date: 11/16/2025, 2:15:22 PM
|
||||
ID: 10942
|
||||
```
|
||||
|
||||
### Full-Text Search
|
||||
- **observations** - Search all observations by keyword (bugs, features, decisions, discoveries, changes)
|
||||
- Use when: "How did we implement X?" or "What bugs did we fix?"
|
||||
- Example: Search for "authentication JWT" to find auth-related work
|
||||
### Step 2: Pick IDs
|
||||
|
||||
- **sessions** - Search session summaries to find what was accomplished when
|
||||
- Use when: "What did we accomplish last time?" or "What was the goal of that session?"
|
||||
- Example: Find sessions where "added login feature"
|
||||
Review the index results. Identify which IDs are actually relevant. Discard the rest.
|
||||
|
||||
- **prompts** - Find what users have asked about in previous sessions
|
||||
- Use when: "Did I ask about this before?" or "What did I request last week?"
|
||||
- Example: Search for "database migration" in past user prompts
|
||||
### Step 3: Fetch by ID
|
||||
|
||||
### Filtered Search
|
||||
- **by-type** - Filter observations by type (bugfix, feature, refactor, decision, discovery, change)
|
||||
- Use when: "Show me all bug fixes" or "List features we added"
|
||||
- Example: Get all observations with type=bugfix from last month
|
||||
For each relevant ID, fetch full details:
|
||||
|
||||
- **by-concept** - Find observations tagged with specific concepts (problem-solution, how-it-works, gotcha)
|
||||
- Use when: "What patterns did we discover?" or "Show me gotchas"
|
||||
- Example: Find all observations tagged with concept "gotcha"
|
||||
```bash
|
||||
# Fetch observation
|
||||
curl "http://localhost:37777/api/observation/11131"
|
||||
|
||||
- **by-file** - Find all work related to a specific file path across all sessions
|
||||
- Use when: "What changes to auth.ts?" or "History of this file"
|
||||
- Example: Get all work related to "src/auth/login.ts"
|
||||
# Fetch session
|
||||
curl "http://localhost:37777/api/session/2005"
|
||||
|
||||
### Timeline & Context
|
||||
- **recent-context** - Get last N sessions with summaries and observations
|
||||
- Use when: "What's been happening?" or "Catch me up on recent work"
|
||||
- Example: Get last 3 sessions with limit=3
|
||||
# Fetch prompt
|
||||
curl "http://localhost:37777/api/prompt/5421"
|
||||
```
|
||||
|
||||
- **timeline** - Get chronological context around a specific point in time (before/after window)
|
||||
- Use when: "What was happening around [date]?" or "Show me context from that time"
|
||||
- Example: Timeline around session 123 with depth 5 before and after
|
||||
**ID formats:**
|
||||
- Observations: Just the number (11131)
|
||||
- Sessions: Just the number (2005) from "S2005"
|
||||
- Prompts: Just the number (5421)
|
||||
|
||||
- **timeline-by-query** - Search first, then get timeline context around best match
|
||||
- Use when: "When did we implement auth?" combined with "show me context around that time"
|
||||
- Example: Search for "OAuth implementation" then get surrounding timeline
|
||||
## Search Parameters
|
||||
|
||||
For detailed instructions on any operation, read the corresponding file in [operations/](operations/).
|
||||
**Basic:**
|
||||
- `query` - What to search for (required)
|
||||
- `format` - "index" or "full" (always use "index" first)
|
||||
- `limit` - How many results (default 5, max 100)
|
||||
|
||||
## Quick Decision Guide
|
||||
**Filters (optional):**
|
||||
- `type` - Filter to "observations", "sessions", or "prompts"
|
||||
- `project` - Filter by project name
|
||||
- `dateRange[start]` - Start date (YYYY-MM-DD)
|
||||
- `dateRange[end]` - End date (YYYY-MM-DD)
|
||||
- `obs_type` - Filter observations by: bugfix, feature, decision, discovery, change
|
||||
|
||||
**What is the user asking about?**
|
||||
## Examples
|
||||
|
||||
1. **Recent work** (last 3-5 sessions) → Use **recent-context** with limit=3-5
|
||||
2. **Specific topic/keyword** (bugs, features, decisions) → Use **observations** search
|
||||
3. **Specific file history** (changes to a file) → Use **by-file** search
|
||||
4. **Timeline/chronology** (what happened when) → Use **timeline** or **timeline-by-query**
|
||||
5. **Type-specific** (all bug fixes, all features) → Use **by-type** filter
|
||||
**Find recent bug fixes:**
|
||||
```bash
|
||||
curl "http://localhost:37777/api/search?query=bug&type=observations&obs_type=bugfix&format=index&limit=5"
|
||||
```
|
||||
|
||||
**Most common:** Use **observations** search for general "how did we..." questions.
|
||||
**Find what happened last week:**
|
||||
```bash
|
||||
curl "http://localhost:37777/api/search?query=&type=observations&dateRange[start]=2025-11-11&format=index&limit=10"
|
||||
```
|
||||
|
||||
## Progressive Disclosure Workflow (Token Efficiency)
|
||||
**Search everything:**
|
||||
```bash
|
||||
curl "http://localhost:37777/api/search?query=database+migration&format=index&limit=5"
|
||||
```
|
||||
|
||||
**Core Principle**: Find high-signal items in index format FIRST (~50-100 tokens each), then request full details ONLY for relevant items (~500-1000 tokens each).
|
||||
## Why This Workflow?
|
||||
|
||||
**4-Step Workflow:**
|
||||
**Token efficiency:**
|
||||
- Index format: ~50-100 tokens per result
|
||||
- Full format: ~500-1000 tokens per result
|
||||
- **10x difference** - only fetch full when you know it's relevant
|
||||
|
||||
1. **Start with Index Format**
|
||||
- Always use `format=index` initially
|
||||
- Set `limit=3-5` (not 10-20)
|
||||
- Review titles and dates to assess relevance
|
||||
- Token cost: ~50-100 per result
|
||||
|
||||
2. **Identify Relevant Items**
|
||||
- Scan index results
|
||||
- Discard irrelevant items from list
|
||||
- Keep only 1-3 most relevant
|
||||
|
||||
3. **Request Full Details Selectively**
|
||||
- Use `format=full` ONLY for specific relevant items
|
||||
- Token cost: ~500-1000 per result
|
||||
- **10x cost difference** - be selective
|
||||
|
||||
4. **Refine with Filters**
|
||||
- Use type, dateRange, concepts, files filters
|
||||
- Paginate with offset if needed
|
||||
- Narrow scope before expanding limits
|
||||
|
||||
**DO:**
|
||||
- ✅ Start with `format=index` and `limit=3-5`
|
||||
- ✅ Use filters (type, dateRange, concepts, files) to narrow results
|
||||
- ✅ Request `format=full` ONLY for specific relevant items
|
||||
- ✅ Use offset for pagination instead of large limits
|
||||
|
||||
**DON'T:**
|
||||
- ❌ Jump straight to `format=full`
|
||||
- ❌ Request `limit=20` without assessing index results first
|
||||
- ❌ Load full details for all results upfront
|
||||
- ❌ Skip index format to "save a step" (costs 10x more tokens)
|
||||
|
||||
See [principles/progressive-disclosure.md](principles/progressive-disclosure.md) for complete workflow with examples and token calculations.
|
||||
|
||||
## Quick Reference Table
|
||||
|
||||
| Need | Operation | Key Parameters |
|
||||
|------|-----------|----------------|
|
||||
| Recent context | recent-context | limit=3-5 |
|
||||
| Search observations | observations | query, format=index, limit=5 |
|
||||
| Search sessions | sessions | query, format=index, limit=5 |
|
||||
| Find by type | by-type | type=(bugfix\|feature\|decision), format=index |
|
||||
| Find by file | by-file | filePath, format=index |
|
||||
| Timeline around event | timeline | anchor=(sessionDbId), depth_before=5, depth_after=5 |
|
||||
| Search + timeline | timeline-by-query | query, mode=auto |
|
||||
|
||||
## Common Workflows
|
||||
|
||||
For step-by-step guides on typical user requests, see [operations/common-workflows.md](operations/common-workflows.md):
|
||||
- Understanding past work from previous sessions
|
||||
- Finding specific bug fixes from history
|
||||
- Understanding file history across sessions
|
||||
- Timeline investigation workflows
|
||||
- Search composition patterns
|
||||
|
||||
## Response Formatting
|
||||
|
||||
For guidelines on presenting search results to users, see [operations/formatting.md](operations/formatting.md):
|
||||
- Index format responses (compact lists with titles/dates)
|
||||
- Full format responses (complete observation details)
|
||||
- Timeline responses (chronologically grouped)
|
||||
- Error handling and user-friendly messages
|
||||
|
||||
## Technical Notes
|
||||
|
||||
- **Port:** Default 37777 (configurable via `CLAUDE_MEM_WORKER_PORT`)
|
||||
- **Response format:** Always JSON
|
||||
- **Search engine:** FTS5 full-text search + structured filters
|
||||
- **All operations:** HTTP GET with query parameters
|
||||
- **Worker:** PM2-managed background process
|
||||
**Clarity:**
|
||||
- See everything first
|
||||
- Pick what matters
|
||||
- Get details only for what you need
|
||||
|
||||
## Error Handling
|
||||
|
||||
If HTTP request fails:
|
||||
1. Inform user the claude-mem search service isn't available
|
||||
2. Suggest checking if worker is running: `pm2 list` or `pm2 status claude-mem-worker`
|
||||
3. Offer to help troubleshoot using the troubleshoot skill
|
||||
|
||||
## Resources
|
||||
|
||||
**Principles:**
|
||||
- [principles/progressive-disclosure.md](principles/progressive-disclosure.md) - Complete 4-step workflow with token calculations
|
||||
- [principles/anti-patterns.md](principles/anti-patterns.md) - 5 anti-patterns to avoid with LLM behavior insights
|
||||
|
||||
**Operations:**
|
||||
- [operations/](operations/) - Detailed instructions for each operation (9 operations + help)
|
||||
- [operations/common-workflows.md](operations/common-workflows.md) - Step-by-step workflow guides
|
||||
- [operations/formatting.md](operations/formatting.md) - Response formatting templates
|
||||
If search fails, tell the user the worker isn't available and suggest:
|
||||
```bash
|
||||
pm2 list # Check if worker is running
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Remember:** This skill searches **cross-session persistent memory**, NOT current conversation. Start with index format for token efficiency. Use temporal triggers to differentiate from native Claude memory.
|
||||
**Remember:** ALWAYS search with format=index first. ALWAYS fetch by ID for details. The IDs are there for a reason - USE THEM.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Search Observations (Full-Text)
|
||||
# Search Observations (Semantic + Full-Text Hybrid)
|
||||
|
||||
Search all observations using natural language queries.
|
||||
|
||||
@@ -17,10 +17,16 @@ curl -s "http://localhost:37777/api/search/observations?query=authentication&for
|
||||
|
||||
## Parameters
|
||||
|
||||
- **query** (required): Search terms (e.g., "authentication", "bug fix", "database migration")
|
||||
- **query** (optional): Natural language search query - uses semantic search (ChromaDB) for ranking with SQLite FTS5 fallback (e.g., "authentication", "bug fix", "database migration"). Can be omitted for filter-only searches.
|
||||
- **format**: "index" (summary) or "full" (complete details). Default: "full"
|
||||
- **limit**: Number of results (default: 20, max: 100)
|
||||
- **project**: Filter by project name (optional)
|
||||
- **dateRange**: Filter by date range (optional) - `dateRange[start]` and/or `dateRange[end]`
|
||||
- **obs_type**: Filter by observation type: bugfix, feature, refactor, decision, discovery, change (optional)
|
||||
- **concepts**: Filter by concept tags (optional)
|
||||
- **files**: Filter by file paths (optional)
|
||||
|
||||
**Important**: When omitting `query`, you MUST provide at least one filter (project, dateRange, obs_type, concepts, or files)
|
||||
|
||||
## When to Use Each Format
|
||||
|
||||
@@ -76,13 +82,28 @@ Found 5 results for "authentication":
|
||||
|
||||
For complete formatting guidelines, see formatting.md (documentation coming soon).
|
||||
|
||||
## Filter-Only Examples
|
||||
|
||||
Search without query text (direct SQLite filtering):
|
||||
|
||||
```bash
|
||||
# Get all observations from November 2025
|
||||
curl -s "http://localhost:37777/api/search?type=observations&dateRange[start]=2025-11-01&format=index"
|
||||
|
||||
# Get all bug fixes from a specific project
|
||||
curl -s "http://localhost:37777/api/search?type=observations&obs_type=bugfix&project=api-server&format=index"
|
||||
|
||||
# Get all observations from last 7 days
|
||||
curl -s "http://localhost:37777/api/search?type=observations&dateRange[start]=2025-11-11&format=index"
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
**Missing query parameter:**
|
||||
**Missing query and filters:**
|
||||
```json
|
||||
{"error": "Missing required parameter: query"}
|
||||
{"error": "Either query or filters required for search"}
|
||||
```
|
||||
Fix: Add the query parameter
|
||||
Fix: Provide either a query parameter OR at least one filter (project, dateRange, obs_type, concepts, files)
|
||||
|
||||
**No results found:**
|
||||
```json
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -376,6 +376,36 @@
|
||||
animation: spin 1.5s linear infinite;
|
||||
}
|
||||
|
||||
.queue-bubble {
|
||||
position: absolute;
|
||||
top: -8px;
|
||||
right: -8px;
|
||||
background: var(--color-accent-primary);
|
||||
color: var(--color-text-button);
|
||||
font-size: 10px;
|
||||
font-weight: 600;
|
||||
font-family: 'Monaspace Radon', monospace;
|
||||
min-width: 18px;
|
||||
height: 18px;
|
||||
border-radius: 9px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
padding: 0 5px;
|
||||
box-shadow: 0 2px 4px rgba(0, 0, 0, 0.2);
|
||||
animation: pulse 2s ease-in-out infinite;
|
||||
z-index: 10;
|
||||
}
|
||||
|
||||
@keyframes pulse {
|
||||
0%, 100% {
|
||||
transform: scale(1);
|
||||
}
|
||||
50% {
|
||||
transform: scale(1.1);
|
||||
}
|
||||
}
|
||||
|
||||
.logo-text {
|
||||
font-family: 'Monaspace Radon', monospace;
|
||||
font-weight: 100;
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
# Postmortem: Worker Debug Failure - 2025-11-17
|
||||
|
||||
## Incident Summary
|
||||
Attempted to fix broken worker service. Worker was in crash loop with 225 restarts, failing with "MCP error -32000: Connection closed". Debug attempt failed and changes were reverted.
|
||||
|
||||
## What Went Wrong
|
||||
|
||||
### 1. **Jumped to Symptoms, Not Root Cause**
|
||||
- Saw "MCP connection failed" errors in logs
|
||||
- Immediately focused on MCP/Chroma connection code
|
||||
- Never asked: "Why is this suddenly broken when it worked before?"
|
||||
- Classic symptom chasing instead of root cause analysis
|
||||
|
||||
### 2. **Ignored the Build Pipeline**
|
||||
- Worker file wasn't in the expected location (`plugin/worker-service.cjs` vs `plugin/scripts/worker-service.cjs`)
|
||||
- Build output existed but search server was producing corrupted/error output
|
||||
- Never investigated: "Is the build system broken?"
|
||||
- Should have compared built artifacts between main and current branch
|
||||
|
||||
### 3. **Tried to Fix by Disabling Instead of Understanding**
|
||||
- Final approach: comment out Chroma, comment out search server
|
||||
- This is the opposite of debugging - it's just making things "work" by removing functionality
|
||||
- User called this out as "duct tape around 5 things unrelated to the problem"
|
||||
- Violated YAGNI/KISS by adding defensive complexity instead of fixing the actual issue
|
||||
|
||||
### 4. **Didn't Compare Working vs Broken State**
|
||||
- User specifically said "we fixed this before"
|
||||
- Should have immediately: `git diff main src/services/worker-service.ts`
|
||||
- Did this eventually but didn't follow through on the findings
|
||||
- The diff showed only search-everything additions - the core worker code was UNCHANGED
|
||||
- This should have been a huge red flag: "If the code is the same, why is it broken?"
|
||||
|
||||
### 5. **Overcomplicated the Investigation**
|
||||
- Started reading through ChromaSync implementation
|
||||
- Traced through MCP connection code
|
||||
- Analyzed startup sequences
|
||||
- All of this was unnecessary if the root cause was a build issue
|
||||
|
||||
## What Should Have Happened
|
||||
|
||||
### Correct Debug Sequence:
|
||||
1. ✅ Check worker status (`pm2 list`) - DONE
|
||||
2. ✅ Check error logs - DONE
|
||||
3. ❌ **Compare current code to main branch** - SKIPPED INITIALLY
|
||||
4. ❌ **Check if built files are correct** - SKIPPED
|
||||
5. ❌ **Test the build pipeline** - NEVER DONE
|
||||
6. ❌ **Verify dependencies are installed** - NEVER CHECKED
|
||||
|
||||
### The Real Questions:
|
||||
- Is this a code change or a build issue?
|
||||
- What changed between working state and broken state?
|
||||
- Are the built artifacts corrupted?
|
||||
- Is the search server build actually valid?
|
||||
- Are there missing dependencies in plugin/scripts/node_modules?
|
||||
|
||||
## Likely Root Causes (Untested)
|
||||
|
||||
Based on evidence:
|
||||
1. **Build artifacts are corrupted** - search-server.mjs threw syntax errors when run
|
||||
2. **Node modules missing/outdated** - plugin/scripts/node_modules may be stale
|
||||
3. **ESM/CJS bundling issue** - esbuild may have produced invalid output
|
||||
4. **search-everything branch has broken build config** - scripts/build-hooks.js may have issues
|
||||
|
||||
## Key Lessons
|
||||
|
||||
### KISS/DRY/YAGNI Violations
|
||||
- Added complexity (disabling features) instead of removing it
|
||||
- Tried to work around symptoms instead of fixing root cause
|
||||
- Ignored the principle: "If it worked before and code is same, it's environment/build"
|
||||
|
||||
### Debugging Anti-Patterns
|
||||
1. **Symptom Chasing**: Following error messages down rabbit holes
|
||||
2. **Defensive Coding**: Commenting out "broken" features instead of fixing them
|
||||
3. **Ignoring History**: Not comparing working vs broken states
|
||||
4. **Build Blindness**: Assuming built artifacts are correct without verification
|
||||
|
||||
### What Good Debugging Looks Like
|
||||
1. Compare working state (main) vs broken state (current branch)
|
||||
2. Identify what actually changed (code? deps? build?)
|
||||
3. Test the simplest hypothesis first (build issue vs code issue)
|
||||
4. Never disable features to "fix" things - that's not fixing
|
||||
|
||||
## Action Items for Next Attempt
|
||||
|
||||
### Before Writing Any Code:
|
||||
- [ ] `git diff main` for all modified files
|
||||
- [ ] Check if `plugin/scripts/` artifacts are valid JavaScript
|
||||
- [ ] Compare build process: `npm run build` output on main vs current branch
|
||||
- [ ] Verify `plugin/scripts/node_modules` exists and is current
|
||||
- [ ] Test search-server.mjs in isolation: `node plugin/scripts/search-server.mjs`
|
||||
|
||||
### If Build is Broken:
|
||||
- [ ] Check scripts/build-hooks.js for recent changes
|
||||
- [ ] Verify esbuild configuration
|
||||
- [ ] Test build on main branch, then on current branch
|
||||
- [ ] Don't modify source code until build is proven working
|
||||
|
||||
### If Code is Broken:
|
||||
- [ ] Create minimal repro (which specific change broke it?)
|
||||
- [ ] Fix the actual bug, don't add workarounds
|
||||
- [ ] Test the fix in isolation
|
||||
|
||||
## Conclusion
|
||||
|
||||
This failure exemplifies "debugging by making changes" instead of "debugging by understanding". The instinct to fix symptoms (MCP errors) instead of investigating root cause (why is it broken now?) led to wasted effort and ultimately no solution.
|
||||
|
||||
The user's frustration was justified - I was adding defensive duct tape instead of finding and fixing the real problem. This is exactly what KISS/DRY/YAGNI principles are meant to prevent.
|
||||
|
||||
**Next time: Compare, verify, understand, THEN fix. Never disable features to make errors go away.**
|
||||
@@ -0,0 +1,119 @@
|
||||
Unified Search API Consolidation Plan
|
||||
|
||||
Overview
|
||||
|
||||
Consolidate 10 search endpoints into 6 powerful, semantic endpoints with intelligent aliasing for backward compatibility.
|
||||
|
||||
New Endpoint Structure
|
||||
|
||||
1. /search - Unified Cross-Type Search
|
||||
|
||||
- Searches all record types (observations + sessions + prompts) via Chroma multi-collection search
|
||||
- Optional type filter to narrow down
|
||||
- Replaces: search_observations, search_sessions, search_user_prompts
|
||||
- Params: query, response=[index|full], type, project, dateRange, limit, offset, orderBy
|
||||
|
||||
2. /timeline - Unified Timeline Tool
|
||||
|
||||
- Supports both anchor-based and query-based modes via params
|
||||
- If anchor → direct timeline lookup (like get_context_timeline)
|
||||
- If query → search-first then timeline (like get_timeline_by_query)
|
||||
- Params: anchor OR query, depth_before, depth_after, response, mode, project
|
||||
|
||||
3. /decisions - Decision Observations
|
||||
|
||||
- Metadata-first search for type=decision observations
|
||||
- Uses specialized search logic for precision
|
||||
- Params: response, project, dateRange, limit, offset, orderBy
|
||||
|
||||
4. /changes - Change Observations
|
||||
|
||||
- Metadata-first search for type=change observations
|
||||
- Same pattern as /decisions
|
||||
|
||||
5. /how-it-works - How-It-Works Concept
|
||||
|
||||
- Metadata-first search for concept=how-it-works observations
|
||||
- Same pattern as concept endpoints
|
||||
|
||||
6. /contextualize - Intelligent Context Builder
|
||||
|
||||
- Complex hybrid endpoint:
|
||||
a. Get 7 latest decisions + 7 latest changes + 3 latest how-it-works
|
||||
b. Find newest date across all results
|
||||
c. Get timeline (7 before + 7 after) around that date
|
||||
d. Merge & re-sort into single timeline (newest → oldest)
|
||||
e. Return timeline + narratives of each concept's latest result
|
||||
- Params: query (for contextualization), project
|
||||
|
||||
Implementation Phases
|
||||
|
||||
Phase 1: Core Unified Search (search-server.ts)
|
||||
|
||||
- Create search tool with Chroma multi-collection query
|
||||
- Add type filtering support
|
||||
- Alias old tools: search_observations → search(type=['observations'])
|
||||
|
||||
Phase 2: Unified Timeline (search-server.ts)
|
||||
|
||||
- Merge get_context_timeline + get_timeline_by_query logic
|
||||
- Support both anchor and query params (mutually exclusive)
|
||||
- Alias old timeline tools to new unified implementation
|
||||
|
||||
Phase 3: Specialized Concept Endpoints (search-server.ts)
|
||||
|
||||
- Create decisions, changes, how_it_works tools
|
||||
- Use metadata-first search strategy
|
||||
- Update find_by_type and find_by_concept to call these internally
|
||||
|
||||
Phase 4: Contextualize Endpoint (search-server.ts)
|
||||
|
||||
- Implement parallel fetching (7 decisions, 7 changes, 3 how-it-works)
|
||||
- Find newest date, get timeline around it
|
||||
- Merge, re-sort, extract narratives
|
||||
- Return structured response with timeline + narratives
|
||||
|
||||
Phase 5: HTTP API Routes (worker-service.ts)
|
||||
|
||||
- Add 6 new routes: /api/search, /api/timeline, /api/decisions, /api/changes, /api/how-it-works, /api/contextualize
|
||||
- Update old routes to alias new implementations
|
||||
- Maintain backward compatibility
|
||||
|
||||
Phase 6: Chroma Multi-Collection Search (ChromaSync.ts)
|
||||
|
||||
- Add searchAll() method to query all collections in parallel
|
||||
- Include source collection metadata in results
|
||||
- Merge and rank by similarity score
|
||||
|
||||
Phase 7: SQLite Fallback (SessionSearch.ts)
|
||||
|
||||
- Add searchAll() for FTS5 fallback when Chroma unavailable
|
||||
- Merge results from all three FTS5 tables
|
||||
|
||||
Phase 8: Documentation & Skill Updates
|
||||
|
||||
- Update mem-search skill with new endpoints
|
||||
- Update CLAUDE.md and README.md
|
||||
- Add examples and migration guide
|
||||
|
||||
Phase 9: Testing & Deployment
|
||||
|
||||
- Unit tests for all new tools
|
||||
- Integration tests for aliasing
|
||||
- Manual testing via mem-search skill
|
||||
- Build → sync → worker restart
|
||||
|
||||
Key Design Decisions
|
||||
|
||||
✅ Aliasing Strategy: Old endpoints call new implementations internally (zero breaking changes)
|
||||
✅ Unified Search: Chroma multi-collection search for true cross-type queries
|
||||
✅ Flexible Timeline: Single tool supports both direct and query-based modes
|
||||
✅ Specialized Shortcuts: /decisions, /changes, /how-it-works for common queries
|
||||
✅ Intelligent Context: /contextualize auto-builds rich context with narratives
|
||||
|
||||
Migration Impact
|
||||
|
||||
- Users: Zero breaking changes, old endpoints work via aliasing
|
||||
- Codebase: Simplified from 10 conceptual endpoints to 6
|
||||
- Performance: Improved via Chroma multi-collection search
|
||||
- Developer UX: Cleaner, more semantic API
|
||||
@@ -0,0 +1,410 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import fs from 'fs';
|
||||
import Database from 'better-sqlite3';
|
||||
import readline from 'readline';
|
||||
import path from 'path';
|
||||
import { homedir } from 'os';
|
||||
import { globSync } from 'glob';
|
||||
|
||||
// =============================================================================
|
||||
// TOOL REPLACEMENT DECISION TABLE
|
||||
// =============================================================================
|
||||
//
|
||||
// KEY INSIGHT: Observations are the SEMANTIC SYNTHESIS of tool results.
|
||||
// They contain what Claude LEARNED, which is what future Claude needs.
|
||||
//
|
||||
// Tool | Replace OUTPUT? | Reason
|
||||
// ------------------|-----------------|----------------------------------------
|
||||
// Read | ✅ YES | Observation = what was learned from file
|
||||
// Bash | ✅ YES | Observation = what command revealed
|
||||
// Grep | ✅ YES | Observation = what search found
|
||||
// Task | ✅ YES | Observation = what agent discovered
|
||||
// WebFetch | ✅ YES | Observation = what page contained
|
||||
// Glob | ⚠️ MAYBE | File lists are often small already
|
||||
// WebSearch | ⚠️ MAYBE | Results are moderate size
|
||||
// Edit | ❌ NO | OUTPUT is tiny ("success"), INPUT is ground truth
|
||||
// Write | ❌ NO | OUTPUT is tiny, INPUT is the file content
|
||||
// NotebookEdit | ❌ NO | OUTPUT is tiny, INPUT is the code
|
||||
// TodoWrite | ❌ NO | Both tiny
|
||||
// AskUserQuestion | ❌ NO | Both small, user input matters
|
||||
// mcp__* | ⚠️ MAYBE | Varies by tool
|
||||
//
|
||||
// NEVER REPLACE INPUT - it contains the action (diff, command, query, path)
|
||||
// ONLY REPLACE OUTPUT - swap raw results for semantic synthesis (observation)
|
||||
//
|
||||
// REPLACEMENT FORMAT:
|
||||
// Original output gets replaced with:
|
||||
// "[Strategically Omitted by Claude-Mem to save tokens]
|
||||
//
|
||||
// [Observation: Title here]
|
||||
// Facts: ...
|
||||
// Concepts: ..."
|
||||
// =============================================================================
|
||||
|
||||
// Configuration
|
||||
const DB_PATH = path.join(homedir(), '.claude-mem', 'claude-mem.db');
|
||||
const MAX_TRANSCRIPTS = parseInt(process.env.MAX_TRANSCRIPTS || '500', 10);
|
||||
|
||||
// Find transcript files (most recent first)
|
||||
const TRANSCRIPT_DIR = path.join(homedir(), '.claude/projects/-Users-alexnewman-Scripts-claude-mem');
|
||||
const allTranscriptFiles = globSync(path.join(TRANSCRIPT_DIR, '*.jsonl'));
|
||||
|
||||
// Sort by modification time (most recent first), take MAX_TRANSCRIPTS
|
||||
const transcriptFiles = allTranscriptFiles
|
||||
.map(f => ({ path: f, mtime: fs.statSync(f).mtime }))
|
||||
.sort((a, b) => b.mtime - a.mtime)
|
||||
.slice(0, MAX_TRANSCRIPTS)
|
||||
.map(f => f.path);
|
||||
|
||||
console.log(`Config: MAX_TRANSCRIPTS=${MAX_TRANSCRIPTS}`);
|
||||
console.log(`Using ${transcriptFiles.length} most recent transcript files (of ${allTranscriptFiles.length} total)\n`);
|
||||
|
||||
// Map to store original content from transcript (both inputs and outputs)
|
||||
const originalContent = new Map();
|
||||
|
||||
// Track contaminated (already transformed) transcripts
|
||||
let skippedTranscripts = 0;
|
||||
|
||||
// Marker for already-transformed content (endless mode replacement format)
|
||||
const TRANSFORMATION_MARKER = '**Key Facts:**';
|
||||
|
||||
// Auto-discover agent transcripts linked to main session
|
||||
async function discoverAgentFiles(mainTranscriptPath) {
|
||||
console.log('Discovering linked agent transcripts...');
|
||||
|
||||
const agentIds = new Set();
|
||||
const fileStream = fs.createReadStream(mainTranscriptPath);
|
||||
const rl = readline.createInterface({
|
||||
input: fileStream,
|
||||
crlfDelay: Infinity
|
||||
});
|
||||
|
||||
for await (const line of rl) {
|
||||
if (!line.includes('agentId')) continue;
|
||||
|
||||
try {
|
||||
const obj = JSON.parse(line);
|
||||
|
||||
// Check for agentId in toolUseResult
|
||||
if (obj.toolUseResult?.agentId) {
|
||||
agentIds.add(obj.toolUseResult.agentId);
|
||||
}
|
||||
} catch (e) {
|
||||
// Skip malformed lines
|
||||
}
|
||||
}
|
||||
|
||||
// Build agent file paths
|
||||
const directory = path.dirname(mainTranscriptPath);
|
||||
const agentFiles = Array.from(agentIds).map(id =>
|
||||
path.join(directory, `agent-${id}.jsonl`)
|
||||
).filter(filePath => fs.existsSync(filePath));
|
||||
|
||||
console.log(` → Found ${agentIds.size} agent IDs`);
|
||||
console.log(` → ${agentFiles.length} agent files exist on disk\n`);
|
||||
|
||||
return agentFiles;
|
||||
}
|
||||
|
||||
// Parse transcript to get BOTH tool_use (inputs) and tool_result (outputs) content
|
||||
// Returns true if transcript is clean, false if contaminated (already transformed)
|
||||
async function loadOriginalContentFromFile(filePath, fileLabel) {
|
||||
const fileStream = fs.createReadStream(filePath);
|
||||
const rl = readline.createInterface({
|
||||
input: fileStream,
|
||||
crlfDelay: Infinity
|
||||
});
|
||||
|
||||
let count = 0;
|
||||
let isContaminated = false;
|
||||
const toolUseIdsFromThisFile = new Set();
|
||||
|
||||
for await (const line of rl) {
|
||||
if (!line.includes('toolu_')) continue;
|
||||
|
||||
try {
|
||||
const obj = JSON.parse(line);
|
||||
|
||||
if (obj.message?.content) {
|
||||
for (const item of obj.message.content) {
|
||||
// Capture tool_use (inputs)
|
||||
if (item.type === 'tool_use' && item.id) {
|
||||
const existing = originalContent.get(item.id) || { input: '', output: '', name: '' };
|
||||
existing.input = JSON.stringify(item.input || {});
|
||||
existing.name = item.name;
|
||||
originalContent.set(item.id, existing);
|
||||
toolUseIdsFromThisFile.add(item.id);
|
||||
count++;
|
||||
}
|
||||
|
||||
// Capture tool_result (outputs)
|
||||
if (item.type === 'tool_result' && item.tool_use_id) {
|
||||
const content = typeof item.content === 'string' ? item.content : JSON.stringify(item.content);
|
||||
|
||||
// Check for transformation marker - if found, transcript is contaminated
|
||||
if (content.includes(TRANSFORMATION_MARKER)) {
|
||||
isContaminated = true;
|
||||
}
|
||||
|
||||
const existing = originalContent.get(item.tool_use_id) || { input: '', output: '', name: '' };
|
||||
existing.output = content;
|
||||
originalContent.set(item.tool_use_id, existing);
|
||||
toolUseIdsFromThisFile.add(item.tool_use_id);
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
// Skip malformed lines
|
||||
}
|
||||
}
|
||||
|
||||
// If contaminated, remove all data from this file and report
|
||||
if (isContaminated) {
|
||||
for (const id of toolUseIdsFromThisFile) {
|
||||
originalContent.delete(id);
|
||||
}
|
||||
console.log(` ⚠️ Skipped ${fileLabel} (already transformed)`);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (count > 0) {
|
||||
console.log(` → Found ${count} tool uses in ${fileLabel}`);
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
async function loadOriginalContent() {
|
||||
console.log('Loading original content from transcripts...');
|
||||
console.log(` → Scanning ${transcriptFiles.length} transcript files...\n`);
|
||||
|
||||
let cleanTranscripts = 0;
|
||||
|
||||
// Load from all transcript files
|
||||
for (const transcriptFile of transcriptFiles) {
|
||||
const filename = path.basename(transcriptFile);
|
||||
const isClean = await loadOriginalContentFromFile(transcriptFile, filename);
|
||||
if (isClean) {
|
||||
cleanTranscripts++;
|
||||
} else {
|
||||
skippedTranscripts++;
|
||||
}
|
||||
}
|
||||
|
||||
// Also check for any agent files not already included
|
||||
for (const transcriptFile of transcriptFiles) {
|
||||
if (transcriptFile.includes('agent-')) continue; // Already an agent file
|
||||
const agentFiles = await discoverAgentFiles(transcriptFile);
|
||||
for (const agentFile of agentFiles) {
|
||||
if (transcriptFiles.includes(agentFile)) continue; // Already processed
|
||||
const filename = path.basename(agentFile);
|
||||
const isClean = await loadOriginalContentFromFile(agentFile, `agent transcript (${filename})`);
|
||||
if (!isClean) {
|
||||
skippedTranscripts++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
console.log(`\nTotal: Loaded original content for ${originalContent.size} tool uses (inputs + outputs)`);
|
||||
if (skippedTranscripts > 0) {
|
||||
console.log(`⚠️ Skipped ${skippedTranscripts} transcripts (already transformed with endless mode)`);
|
||||
}
|
||||
console.log();
|
||||
}
|
||||
|
||||
// Strip __N suffix from tool_use_id to get base ID
|
||||
function getBaseToolUseId(id) {
|
||||
return id ? id.replace(/__\d+$/, '') : id;
|
||||
}
|
||||
|
||||
// Query observations from database using tool_use_ids found in transcripts
|
||||
// Handles suffixed IDs like toolu_abc__1, toolu_abc__2 matching transcript's toolu_abc
|
||||
function queryObservations() {
|
||||
// Get tool_use_ids from the loaded transcript content
|
||||
const toolUseIds = Array.from(originalContent.keys());
|
||||
|
||||
if (toolUseIds.length === 0) {
|
||||
console.log('No tool use IDs found in transcripts\n');
|
||||
return [];
|
||||
}
|
||||
|
||||
console.log(`Querying observations for ${toolUseIds.length} tool use IDs from transcripts...`);
|
||||
|
||||
const db = new Database(DB_PATH, { readonly: true });
|
||||
|
||||
// Build LIKE clauses to match both exact IDs and suffixed variants (toolu_abc, toolu_abc__1, etc)
|
||||
const likeConditions = toolUseIds.map(() => 'tool_use_id LIKE ?').join(' OR ');
|
||||
const likeParams = toolUseIds.map(id => `${id}%`);
|
||||
|
||||
const query = `
|
||||
SELECT
|
||||
id,
|
||||
tool_use_id,
|
||||
type,
|
||||
narrative,
|
||||
title,
|
||||
facts,
|
||||
concepts,
|
||||
LENGTH(COALESCE(facts,'')) as facts_len,
|
||||
LENGTH(COALESCE(title,'')) + LENGTH(COALESCE(facts,'')) as title_facts_len,
|
||||
LENGTH(COALESCE(title,'')) + LENGTH(COALESCE(facts,'')) + LENGTH(COALESCE(concepts,'')) as compact_len,
|
||||
LENGTH(COALESCE(narrative,'')) as narrative_len,
|
||||
LENGTH(COALESCE(title,'')) + LENGTH(COALESCE(narrative,'')) + LENGTH(COALESCE(facts,'')) + LENGTH(COALESCE(concepts,'')) as full_obs_len
|
||||
FROM observations
|
||||
WHERE ${likeConditions}
|
||||
ORDER BY created_at DESC
|
||||
`;
|
||||
|
||||
const observations = db.prepare(query).all(...likeParams);
|
||||
db.close();
|
||||
|
||||
console.log(`Found ${observations.length} observations matching tool use IDs (including suffixed variants)\n`);
|
||||
|
||||
return observations;
|
||||
}
|
||||
|
||||
// Tools eligible for OUTPUT replacement (observation = semantic synthesis of result)
|
||||
const REPLACEABLE_TOOLS = new Set(['Read', 'Bash', 'Grep', 'Task', 'WebFetch', 'Glob', 'WebSearch']);
|
||||
|
||||
// Analyze OUTPUT-only replacement for eligible tools
|
||||
function analyzeTransformations(observations) {
|
||||
console.log('='.repeat(110));
|
||||
console.log('OUTPUT REPLACEMENT ANALYSIS (Eligible Tools Only)');
|
||||
console.log('='.repeat(110));
|
||||
console.log();
|
||||
console.log('Eligible tools:', Array.from(REPLACEABLE_TOOLS).join(', '));
|
||||
console.log();
|
||||
|
||||
// Group observations by BASE tool_use_id (strip __N suffix)
|
||||
// This groups toolu_abc, toolu_abc__1, toolu_abc__2 together
|
||||
const obsByToolId = new Map();
|
||||
observations.forEach(obs => {
|
||||
const baseId = getBaseToolUseId(obs.tool_use_id);
|
||||
if (!obsByToolId.has(baseId)) {
|
||||
obsByToolId.set(baseId, []);
|
||||
}
|
||||
obsByToolId.get(baseId).push(obs);
|
||||
});
|
||||
|
||||
// Define strategies to test
|
||||
const strategies = [
|
||||
{ name: 'facts_only', field: 'facts_len', desc: 'Facts only (~400 chars)' },
|
||||
{ name: 'title_facts', field: 'title_facts_len', desc: 'Title + Facts (~450 chars)' },
|
||||
{ name: 'compact', field: 'compact_len', desc: 'Title + Facts + Concepts (~500 chars)' },
|
||||
{ name: 'narrative', field: 'narrative_len', desc: 'Narrative only (~700 chars)' },
|
||||
{ name: 'full', field: 'full_obs_len', desc: 'Full observation (~1200 chars)' }
|
||||
];
|
||||
|
||||
// Track results per strategy
|
||||
const results = {};
|
||||
strategies.forEach(s => {
|
||||
results[s.name] = {
|
||||
transforms: 0,
|
||||
noTransform: 0,
|
||||
saved: 0,
|
||||
totalOriginal: 0
|
||||
};
|
||||
});
|
||||
|
||||
// Track stats
|
||||
let eligible = 0;
|
||||
let ineligible = 0;
|
||||
let noTranscript = 0;
|
||||
const toolCounts = {};
|
||||
|
||||
// Analyze each tool use
|
||||
obsByToolId.forEach((obsArray, toolUseId) => {
|
||||
const original = originalContent.get(toolUseId);
|
||||
const toolName = original?.name || 'unknown';
|
||||
const outputLen = original?.output?.length || 0;
|
||||
|
||||
// Skip if no transcript data
|
||||
if (!original || outputLen === 0) {
|
||||
noTranscript++;
|
||||
return;
|
||||
}
|
||||
|
||||
// Skip if tool not eligible for replacement
|
||||
if (!REPLACEABLE_TOOLS.has(toolName)) {
|
||||
ineligible++;
|
||||
return;
|
||||
}
|
||||
|
||||
eligible++;
|
||||
toolCounts[toolName] = (toolCounts[toolName] || 0) + 1;
|
||||
|
||||
// Sum lengths across ALL observations for this tool use (handles multiple obs per tool_use_id)
|
||||
// Test each strategy - OUTPUT replacement only
|
||||
strategies.forEach(strategy => {
|
||||
const obsLen = obsArray.reduce((sum, obs) => sum + (obs[strategy.field] || 0), 0);
|
||||
const r = results[strategy.name];
|
||||
|
||||
r.totalOriginal += outputLen;
|
||||
|
||||
if (obsLen > 0 && obsLen < outputLen) {
|
||||
r.transforms++;
|
||||
r.saved += (outputLen - obsLen);
|
||||
} else {
|
||||
r.noTransform++;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// Print results
|
||||
console.log('TOOL BREAKDOWN:');
|
||||
Object.entries(toolCounts).sort((a, b) => b[1] - a[1]).forEach(([tool, count]) => {
|
||||
console.log(` ${tool}: ${count}`);
|
||||
});
|
||||
console.log();
|
||||
console.log('-'.repeat(100));
|
||||
console.log(`Eligible tool uses: ${eligible}`);
|
||||
console.log(`Ineligible (Edit/Write/etc): ${ineligible}`);
|
||||
console.log(`No transcript data: ${noTranscript}`);
|
||||
console.log('-'.repeat(100));
|
||||
console.log();
|
||||
console.log('Strategy Transforms No Transform Chars Saved Original Size Savings %');
|
||||
console.log('-'.repeat(100));
|
||||
|
||||
strategies.forEach(strategy => {
|
||||
const r = results[strategy.name];
|
||||
const pct = r.totalOriginal > 0 ? ((r.saved / r.totalOriginal) * 100).toFixed(1) : '0.0';
|
||||
console.log(
|
||||
`${strategy.desc.padEnd(35)} ${String(r.transforms).padStart(10)} ${String(r.noTransform).padStart(12)} ${String(r.saved.toLocaleString()).padStart(13)} ${String(r.totalOriginal.toLocaleString()).padStart(15)} ${pct.padStart(8)}%`
|
||||
);
|
||||
});
|
||||
|
||||
console.log('-'.repeat(100));
|
||||
console.log();
|
||||
|
||||
// Find best strategy
|
||||
let bestStrategy = null;
|
||||
let bestSavings = 0;
|
||||
strategies.forEach(strategy => {
|
||||
if (results[strategy.name].saved > bestSavings) {
|
||||
bestSavings = results[strategy.name].saved;
|
||||
bestStrategy = strategy;
|
||||
}
|
||||
});
|
||||
|
||||
if (bestStrategy) {
|
||||
const r = results[bestStrategy.name];
|
||||
const pct = ((r.saved / r.totalOriginal) * 100).toFixed(1);
|
||||
console.log(`BEST STRATEGY: ${bestStrategy.desc}`);
|
||||
console.log(` - Transforms ${r.transforms} of ${eligible} eligible tool uses (${((r.transforms/eligible)*100).toFixed(1)}%)`);
|
||||
console.log(` - Saves ${r.saved.toLocaleString()} of ${r.totalOriginal.toLocaleString()} chars (${pct}% reduction)`);
|
||||
}
|
||||
|
||||
console.log();
|
||||
}
|
||||
|
||||
// Main execution
|
||||
async function main() {
|
||||
await loadOriginalContent();
|
||||
const observations = queryObservations();
|
||||
analyzeTransformations(observations);
|
||||
}
|
||||
|
||||
main().catch(error => {
|
||||
console.error('Fatal error:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
+31
-11
@@ -26,12 +26,10 @@ const WORKER_SERVICE = {
|
||||
source: 'src/services/worker-service.ts'
|
||||
};
|
||||
|
||||
// DEPRECATED: MCP search server replaced by skill-based search
|
||||
// Keeping source file for reference: src/servers/search-server.ts
|
||||
// const SEARCH_SERVER = {
|
||||
// name: 'search-server',
|
||||
// source: 'src/servers/search-server.ts'
|
||||
// };
|
||||
const SEARCH_SERVER = {
|
||||
name: 'search-server',
|
||||
source: 'src/servers/search-server.ts'
|
||||
};
|
||||
|
||||
async function buildHooks() {
|
||||
console.log('🔨 Building claude-mem hooks and worker service...\n');
|
||||
@@ -94,6 +92,31 @@ async function buildHooks() {
|
||||
const workerStats = fs.statSync(`${hooksDir}/${WORKER_SERVICE.name}.cjs`);
|
||||
console.log(`✓ worker-service built (${(workerStats.size / 1024).toFixed(2)} KB)`);
|
||||
|
||||
// Build search server
|
||||
console.log(`\n🔧 Building search server...`);
|
||||
await build({
|
||||
entryPoints: [SEARCH_SERVER.source],
|
||||
bundle: true,
|
||||
platform: 'node',
|
||||
target: 'node18',
|
||||
format: 'cjs',
|
||||
outfile: `${hooksDir}/${SEARCH_SERVER.name}.cjs`,
|
||||
minify: true,
|
||||
logLevel: 'error',
|
||||
external: ['better-sqlite3'],
|
||||
define: {
|
||||
'__DEFAULT_PACKAGE_VERSION__': `"${version}"`
|
||||
},
|
||||
banner: {
|
||||
js: '#!/usr/bin/env node'
|
||||
}
|
||||
});
|
||||
|
||||
// Make search server executable
|
||||
fs.chmodSync(`${hooksDir}/${SEARCH_SERVER.name}.cjs`, 0o755);
|
||||
const searchServerStats = fs.statSync(`${hooksDir}/${SEARCH_SERVER.name}.cjs`);
|
||||
console.log(`✓ search-server built (${(searchServerStats.size / 1024).toFixed(2)} KB)`);
|
||||
|
||||
// Build each hook
|
||||
for (const hook of HOOKS) {
|
||||
console.log(`\n🔧 Building ${hook.name}...`);
|
||||
@@ -126,14 +149,11 @@ async function buildHooks() {
|
||||
console.log(`✓ ${hook.name} built (${sizeInKB} KB)`);
|
||||
}
|
||||
|
||||
// DEPRECATED: MCP search server no longer built (replaced by skill-based search)
|
||||
// Search functionality now provided via HTTP API + search skill
|
||||
// Source file kept for reference: src/servers/search-server.ts
|
||||
|
||||
console.log('\n✅ All hooks and worker service built successfully!');
|
||||
console.log('\n✅ All hooks, worker service, and search server built successfully!');
|
||||
console.log(` Output: ${hooksDir}/`);
|
||||
console.log(` - Hooks: *-hook.js`);
|
||||
console.log(` - Worker: worker-service.cjs`);
|
||||
console.log(` - Search Server: search-server.cjs`);
|
||||
console.log(` - Skills: plugin/skills/`);
|
||||
console.log('\n💡 Note: Dependencies will be auto-installed on first hook execution');
|
||||
|
||||
|
||||
@@ -0,0 +1,273 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Endless Mode Token Economics Calculator
|
||||
*
|
||||
* Simulates the recursive/cumulative token savings from Endless Mode by
|
||||
* "playing the tape through" with real observation data from SQLite.
|
||||
*
|
||||
* Key Insight:
|
||||
* - Discovery tokens are ALWAYS spent (creating observations)
|
||||
* - But Endless Mode feeds compressed observations as context instead of full tool outputs
|
||||
* - Savings compound recursively - each tool benefits from ALL previous compressions
|
||||
*/
|
||||
|
||||
const observationsData = [{"id":10136,"type":"decision","title":"Token Accounting Function for Recursive Continuation Pattern","discovery_tokens":4037,"created_at_epoch":1763360747429,"compressed_size":1613},
|
||||
{"id":10135,"type":"discovery","title":"Sequential Thinking Analysis of Token Economics Calculator","discovery_tokens":1439,"created_at_epoch":1763360651617,"compressed_size":1812},
|
||||
{"id":10134,"type":"discovery","title":"Recent Context Query Execution","discovery_tokens":1273,"created_at_epoch":1763360646273,"compressed_size":1228},
|
||||
{"id":10133,"type":"discovery","title":"Token Data Query Execution and Historical Context","discovery_tokens":11878,"created_at_epoch":1763360642485,"compressed_size":1924},
|
||||
{"id":10132,"type":"discovery","title":"Token Data Query and Script Validation Request","discovery_tokens":4167,"created_at_epoch":1763360628269,"compressed_size":903},
|
||||
{"id":10131,"type":"discovery","title":"Endless Mode Token Economics Analysis Output: Complete Infrastructure Impact","discovery_tokens":2458,"created_at_epoch":1763360553238,"compressed_size":2166},
|
||||
{"id":10130,"type":"change","title":"Integration of Actual Compute Savings Analysis into Main Execution Flow","discovery_tokens":11031,"created_at_epoch":1763360545347,"compressed_size":1032},
|
||||
{"id":10129,"type":"discovery","title":"Prompt Caching Economics: User Cost vs. Anthropic Compute Cost Divergence","discovery_tokens":20059,"created_at_epoch":1763360540854,"compressed_size":1802},
|
||||
{"id":10128,"type":"discovery","title":"Token Caching Cost Analysis Across AI Model Providers","discovery_tokens":3506,"created_at_epoch":1763360478133,"compressed_size":1245},
|
||||
{"id":10127,"type":"discovery","title":"Endless Mode Token Economics Calculator Successfully Integrated Prompt Caching Cost Model","discovery_tokens":3481,"created_at_epoch":1763360384055,"compressed_size":2444},
|
||||
{"id":10126,"type":"bugfix","title":"Fix Return Statement Variable Names in playTheTapeThrough Function","discovery_tokens":8326,"created_at_epoch":1763360374566,"compressed_size":1250},
|
||||
{"id":10125,"type":"change","title":"Redesign Timeline Display to Show Fresh/Cached Token Breakdown and Real Dollar Costs","discovery_tokens":12999,"created_at_epoch":1763360368843,"compressed_size":2004},
|
||||
{"id":10124,"type":"change","title":"Replace Estimated Cost Model with Actual Caching-Based Costs in Anthropic Scale Analysis","discovery_tokens":12867,"created_at_epoch":1763360361147,"compressed_size":2064},
|
||||
{"id":10123,"type":"change","title":"Pivot Session Length Comparison Table from Token to Cost Metrics","discovery_tokens":9746,"created_at_epoch":1763360352992,"compressed_size":1652},
|
||||
{"id":10122,"type":"change","title":"Add Dual Reporting: Token Count vs Actual Cost in Comparison Output","discovery_tokens":9602,"created_at_epoch":1763360346495,"compressed_size":1640},
|
||||
{"id":10121,"type":"change","title":"Apply Prompt Caching Cost Model to Endless Mode Calculation Function","discovery_tokens":9963,"created_at_epoch":1763360339238,"compressed_size":2003},
|
||||
{"id":10120,"type":"change","title":"Integrate Prompt Caching Cost Calculations into Without-Endless-Mode Function","discovery_tokens":8652,"created_at_epoch":1763360332046,"compressed_size":1701},
|
||||
{"id":10119,"type":"change","title":"Display Prompt Caching Pricing in Initial Calculator Output","discovery_tokens":6669,"created_at_epoch":1763360325882,"compressed_size":1188},
|
||||
{"id":10118,"type":"change","title":"Add Prompt Caching Pricing Model to Token Economics Calculator","discovery_tokens":10433,"created_at_epoch":1763360320552,"compressed_size":1264},
|
||||
{"id":10117,"type":"discovery","title":"Claude API Prompt Caching Cost Optimization Factor","discovery_tokens":3439,"created_at_epoch":1763360210175,"compressed_size":1142},
|
||||
{"id":10116,"type":"discovery","title":"Endless Mode Token Economics Verified at Scale","discovery_tokens":2855,"created_at_epoch":1763360144039,"compressed_size":2184},
|
||||
{"id":10115,"type":"feature","title":"Token Economics Calculator for Endless Mode Sessions","discovery_tokens":13468,"created_at_epoch":1763360134068,"compressed_size":1858},
|
||||
{"id":10114,"type":"decision","title":"Token Accounting for Recursive Session Continuations","discovery_tokens":3550,"created_at_epoch":1763360052317,"compressed_size":1478},
|
||||
{"id":10113,"type":"discovery","title":"Performance and Token Optimization Impact Analysis for Endless Mode","discovery_tokens":3464,"created_at_epoch":1763359862175,"compressed_size":1259},
|
||||
{"id":10112,"type":"change","title":"Endless Mode Blocking Hooks & Transcript Transformation Plan Document Created","discovery_tokens":17312,"created_at_epoch":1763359465307,"compressed_size":2181},
|
||||
{"id":10111,"type":"change","title":"Plan Document Creation for Morning Implementation","discovery_tokens":3652,"created_at_epoch":1763359347166,"compressed_size":843},
|
||||
{"id":10110,"type":"decision","title":"Blocking vs Non-Blocking Behavior by Mode","discovery_tokens":3652,"created_at_epoch":1763359347165,"compressed_size":797},
|
||||
{"id":10109,"type":"decision","title":"Tool Use and Observation Processing Architecture: Non-Blocking vs Blocking","discovery_tokens":3472,"created_at_epoch":1763359247045,"compressed_size":1349},
|
||||
{"id":10108,"type":"feature","title":"SessionManager.getMessageIterator implements event-driven async generator with graceful abort handling","discovery_tokens":2417,"created_at_epoch":1763359189299,"compressed_size":2016},
|
||||
{"id":10107,"type":"feature","title":"SessionManager implements event-driven session lifecycle with auto-initialization and zero-latency queue notifications","discovery_tokens":4734,"created_at_epoch":1763359165608,"compressed_size":2781},
|
||||
{"id":10106,"type":"discovery","title":"Two distinct uses of transcript data: live data flow vs session initialization","discovery_tokens":2933,"created_at_epoch":1763359156448,"compressed_size":2015},
|
||||
{"id":10105,"type":"discovery","title":"Transcript initialization pattern identified for compressed context on session resume","discovery_tokens":2933,"created_at_epoch":1763359156447,"compressed_size":2536},
|
||||
{"id":10104,"type":"feature","title":"SDKAgent implements event-driven message generator with continuation prompt logic and Endless Mode integration","discovery_tokens":6148,"created_at_epoch":1763359140399,"compressed_size":3241},
|
||||
{"id":10103,"type":"discovery","title":"Endless Mode architecture documented with phased implementation plan and context economics","discovery_tokens":5296,"created_at_epoch":1763359127954,"compressed_size":3145},
|
||||
{"id":10102,"type":"feature","title":"Save hook enhanced to extract and forward tool_use_id for Endless Mode linking","discovery_tokens":3294,"created_at_epoch":1763359115848,"compressed_size":2125},
|
||||
{"id":10101,"type":"feature","title":"TransformLayer implements Endless Mode context compression via observation substitution","discovery_tokens":4637,"created_at_epoch":1763359108317,"compressed_size":2629},
|
||||
{"id":10100,"type":"feature","title":"EndlessModeConfig implemented for loading Endless Mode settings from files and environment","discovery_tokens":2313,"created_at_epoch":1763359099972,"compressed_size":2125},
|
||||
{"id":10098,"type":"change","title":"User prompts wrapped with semantic XML structure in buildInitPrompt and buildContinuationPrompt","discovery_tokens":7806,"created_at_epoch":1763359091460,"compressed_size":1585},
|
||||
{"id":10099,"type":"discovery","title":"Session persistence mechanism relies on SDK internal state without context reload","discovery_tokens":7806,"created_at_epoch":1763359091460,"compressed_size":1883},
|
||||
{"id":10097,"type":"change","title":"Worker service session init now extracts userPrompt and promptNumber from request body","discovery_tokens":7806,"created_at_epoch":1763359091459,"compressed_size":1148},
|
||||
{"id":10096,"type":"feature","title":"SessionManager enhanced to accept dynamic userPrompt updates during multi-turn conversations","discovery_tokens":7806,"created_at_epoch":1763359091457,"compressed_size":1528},
|
||||
{"id":10095,"type":"discovery","title":"Five lifecycle hooks integrate claude-mem at critical session boundaries","discovery_tokens":6625,"created_at_epoch":1763359074808,"compressed_size":1570},
|
||||
{"id":10094,"type":"discovery","title":"PostToolUse hook is real-time observation creation point, not delayed processing","discovery_tokens":6625,"created_at_epoch":1763359074807,"compressed_size":2371},
|
||||
{"id":10093,"type":"discovery","title":"PostToolUse hook timing and compression integration options explored","discovery_tokens":1696,"created_at_epoch":1763359062088,"compressed_size":1605},
|
||||
{"id":10092,"type":"discovery","title":"Transcript transformation strategy for endless mode identified","discovery_tokens":6112,"created_at_epoch":1763359057563,"compressed_size":1968},
|
||||
{"id":10091,"type":"decision","title":"Finalized Transcript Compression Implementation Strategy","discovery_tokens":1419,"created_at_epoch":1763358943803,"compressed_size":1556},
|
||||
{"id":10090,"type":"discovery","title":"UserPromptSubmit Hook as Compression Integration Point","discovery_tokens":1546,"created_at_epoch":1763358931936,"compressed_size":1621},
|
||||
{"id":10089,"type":"decision","title":"Hypothesis 5 Selected: UserPromptSubmit Hook for Transcript Compression","discovery_tokens":1465,"created_at_epoch":1763358920209,"compressed_size":1918}];
|
||||
|
||||
// Estimate original tool output size from discovery tokens
|
||||
// Heuristic: discovery_tokens roughly correlates with original content size
|
||||
// Assumption: If it took 10k tokens to analyze, original was probably 15-30k tokens
|
||||
function estimateOriginalToolOutputSize(discoveryTokens) {
|
||||
// Conservative multiplier: 2x (original content was 2x the discovery cost)
|
||||
// This accounts for: reading the tool output + analyzing it + generating observation
|
||||
return discoveryTokens * 2;
|
||||
}
|
||||
|
||||
// Convert compressed_size (character count) to approximate token count
|
||||
// Rough heuristic: 1 token ≈ 4 characters for English text
|
||||
function charsToTokens(chars) {
|
||||
return Math.ceil(chars / 4);
|
||||
}
|
||||
|
||||
/**
|
||||
* Simulate session WITHOUT Endless Mode (current behavior)
|
||||
* Each continuation carries ALL previous full tool outputs in context
|
||||
*/
|
||||
function calculateWithoutEndlessMode(observations) {
|
||||
let cumulativeContextTokens = 0;
|
||||
let totalDiscoveryTokens = 0;
|
||||
let totalContinuationTokens = 0;
|
||||
const timeline = [];
|
||||
|
||||
observations.forEach((obs, index) => {
|
||||
const toolNumber = index + 1;
|
||||
const originalToolSize = estimateOriginalToolOutputSize(obs.discovery_tokens);
|
||||
|
||||
// Discovery cost (creating observation from full tool output)
|
||||
const discoveryCost = obs.discovery_tokens;
|
||||
totalDiscoveryTokens += discoveryCost;
|
||||
|
||||
// Continuation cost: Re-process ALL previous tool outputs + current one
|
||||
// This is the key recursive cost
|
||||
cumulativeContextTokens += originalToolSize;
|
||||
const continuationCost = cumulativeContextTokens;
|
||||
totalContinuationTokens += continuationCost;
|
||||
|
||||
timeline.push({
|
||||
tool: toolNumber,
|
||||
obsId: obs.id,
|
||||
title: obs.title.substring(0, 60),
|
||||
originalSize: originalToolSize,
|
||||
discoveryCost,
|
||||
contextSize: cumulativeContextTokens,
|
||||
continuationCost,
|
||||
totalCostSoFar: totalDiscoveryTokens + totalContinuationTokens
|
||||
});
|
||||
});
|
||||
|
||||
return {
|
||||
totalDiscoveryTokens,
|
||||
totalContinuationTokens,
|
||||
totalTokens: totalDiscoveryTokens + totalContinuationTokens,
|
||||
timeline
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Simulate session WITH Endless Mode
|
||||
* Each continuation carries ALL previous COMPRESSED observations in context
|
||||
*/
|
||||
function calculateWithEndlessMode(observations) {
|
||||
let cumulativeContextTokens = 0;
|
||||
let totalDiscoveryTokens = 0;
|
||||
let totalContinuationTokens = 0;
|
||||
const timeline = [];
|
||||
|
||||
observations.forEach((obs, index) => {
|
||||
const toolNumber = index + 1;
|
||||
const originalToolSize = estimateOriginalToolOutputSize(obs.discovery_tokens);
|
||||
const compressedSize = charsToTokens(obs.compressed_size);
|
||||
|
||||
// Discovery cost (same as without Endless Mode - still need to create observation)
|
||||
const discoveryCost = obs.discovery_tokens;
|
||||
totalDiscoveryTokens += discoveryCost;
|
||||
|
||||
// KEY DIFFERENCE: Add COMPRESSED size to context, not original size
|
||||
cumulativeContextTokens += compressedSize;
|
||||
const continuationCost = cumulativeContextTokens;
|
||||
totalContinuationTokens += continuationCost;
|
||||
|
||||
const compressionRatio = ((originalToolSize - compressedSize) / originalToolSize * 100).toFixed(1);
|
||||
|
||||
timeline.push({
|
||||
tool: toolNumber,
|
||||
obsId: obs.id,
|
||||
title: obs.title.substring(0, 60),
|
||||
originalSize: originalToolSize,
|
||||
compressedSize,
|
||||
compressionRatio: `${compressionRatio}%`,
|
||||
discoveryCost,
|
||||
contextSize: cumulativeContextTokens,
|
||||
continuationCost,
|
||||
totalCostSoFar: totalDiscoveryTokens + totalContinuationTokens
|
||||
});
|
||||
});
|
||||
|
||||
return {
|
||||
totalDiscoveryTokens,
|
||||
totalContinuationTokens,
|
||||
totalTokens: totalDiscoveryTokens + totalContinuationTokens,
|
||||
timeline
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Play the tape through - show token-by-token progression
|
||||
*/
|
||||
function playTheTapeThrough(observations) {
|
||||
console.log('\n' + '='.repeat(100));
|
||||
console.log('ENDLESS MODE TOKEN ECONOMICS CALCULATOR');
|
||||
console.log('Playing the tape through with REAL observation data');
|
||||
console.log('='.repeat(100) + '\n');
|
||||
|
||||
console.log(`📊 Dataset: ${observations.length} observations from live sessions\n`);
|
||||
|
||||
// Calculate both scenarios
|
||||
const without = calculateWithoutEndlessMode(observations);
|
||||
const withMode = calculateWithEndlessMode(observations);
|
||||
|
||||
// Show first 10 tools from each scenario side by side
|
||||
console.log('🎬 TAPE PLAYBACK: First 10 Tools\n');
|
||||
console.log('WITHOUT Endless Mode (Current) | WITH Endless Mode (Proposed)');
|
||||
console.log('-'.repeat(100));
|
||||
|
||||
for (let i = 0; i < Math.min(10, observations.length); i++) {
|
||||
const w = without.timeline[i];
|
||||
const e = withMode.timeline[i];
|
||||
|
||||
console.log(`\nTool #${w.tool}: ${w.title}`);
|
||||
console.log(` Original: ${w.originalSize.toLocaleString()}t | Compressed: ${e.compressedSize.toLocaleString()}t (${e.compressionRatio} saved)`);
|
||||
console.log(` Context: ${w.contextSize.toLocaleString()}t | Context: ${e.contextSize.toLocaleString()}t`);
|
||||
console.log(` Total: ${w.totalCostSoFar.toLocaleString()}t | Total: ${e.totalCostSoFar.toLocaleString()}t`);
|
||||
}
|
||||
|
||||
// Summary table
|
||||
console.log('\n' + '='.repeat(100));
|
||||
console.log('📈 FINAL TOTALS\n');
|
||||
|
||||
console.log('WITHOUT Endless Mode (Current):');
|
||||
console.log(` Discovery tokens: ${without.totalDiscoveryTokens.toLocaleString()}t (creating observations)`);
|
||||
console.log(` Continuation tokens: ${without.totalContinuationTokens.toLocaleString()}t (context accumulation)`);
|
||||
console.log(` TOTAL TOKENS: ${without.totalTokens.toLocaleString()}t`);
|
||||
|
||||
console.log('\nWITH Endless Mode:');
|
||||
console.log(` Discovery tokens: ${withMode.totalDiscoveryTokens.toLocaleString()}t (same - still create observations)`);
|
||||
console.log(` Continuation tokens: ${withMode.totalContinuationTokens.toLocaleString()}t (COMPRESSED context)`);
|
||||
console.log(` TOTAL TOKENS: ${withMode.totalTokens.toLocaleString()}t`);
|
||||
|
||||
const tokensSaved = without.totalTokens - withMode.totalTokens;
|
||||
const percentSaved = (tokensSaved / without.totalTokens * 100).toFixed(1);
|
||||
|
||||
console.log('\n💰 SAVINGS:');
|
||||
console.log(` Tokens saved: ${tokensSaved.toLocaleString()}t`);
|
||||
console.log(` Percentage saved: ${percentSaved}%`);
|
||||
console.log(` Efficiency gain: ${(without.totalTokens / withMode.totalTokens).toFixed(2)}x`);
|
||||
|
||||
// Anthropic scale calculation
|
||||
console.log('\n' + '='.repeat(100));
|
||||
console.log('🌍 ANTHROPIC SCALE IMPACT\n');
|
||||
|
||||
// Conservative assumptions
|
||||
const activeUsers = 100000; // Claude Code users
|
||||
const sessionsPerWeek = 10; // Per user
|
||||
const toolsPerSession = observations.length; // Use our actual data
|
||||
const weeklyToolUses = activeUsers * sessionsPerWeek * toolsPerSession;
|
||||
|
||||
const avgTokensPerToolWithout = without.totalTokens / observations.length;
|
||||
const avgTokensPerToolWith = withMode.totalTokens / observations.length;
|
||||
|
||||
const weeklyTokensWithout = weeklyToolUses * avgTokensPerToolWithout;
|
||||
const weeklyTokensWith = weeklyToolUses * avgTokensPerToolWith;
|
||||
const weeklyTokensSaved = weeklyTokensWithout - weeklyTokensWith;
|
||||
|
||||
console.log('Assumptions:');
|
||||
console.log(` Active Claude Code users: ${activeUsers.toLocaleString()}`);
|
||||
console.log(` Sessions per user/week: ${sessionsPerWeek}`);
|
||||
console.log(` Tools per session: ${toolsPerSession}`);
|
||||
console.log(` Weekly tool uses: ${weeklyToolUses.toLocaleString()}`);
|
||||
|
||||
console.log('\nWeekly Compute:');
|
||||
console.log(` Without Endless Mode: ${(weeklyTokensWithout / 1e9).toFixed(2)} billion tokens`);
|
||||
console.log(` With Endless Mode: ${(weeklyTokensWith / 1e9).toFixed(2)} billion tokens`);
|
||||
console.log(` Weekly savings: ${(weeklyTokensSaved / 1e9).toFixed(2)} billion tokens (${percentSaved}%)`);
|
||||
|
||||
const annualTokensSaved = weeklyTokensSaved * 52;
|
||||
console.log(` Annual savings: ${(annualTokensSaved / 1e12).toFixed(2)} TRILLION tokens`);
|
||||
|
||||
console.log('\n💡 What this means:');
|
||||
console.log(` • ${percentSaved}% reduction in Claude Code inference costs`);
|
||||
console.log(` • ${(without.totalTokens / withMode.totalTokens).toFixed(1)}x more users served with same infrastructure`);
|
||||
console.log(` • Massive energy/compute savings at scale`);
|
||||
console.log(` • Longer sessions = better UX without economic penalty`);
|
||||
|
||||
console.log('\n' + '='.repeat(100) + '\n');
|
||||
|
||||
return {
|
||||
without,
|
||||
withMode,
|
||||
tokensSaved,
|
||||
percentSaved,
|
||||
weeklyTokensSaved,
|
||||
annualTokensSaved
|
||||
};
|
||||
}
|
||||
|
||||
// Run the calculation
|
||||
playTheTapeThrough(observationsData);
|
||||
@@ -0,0 +1,64 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Protected sync-marketplace script
|
||||
*
|
||||
* Prevents accidental rsync overwrite when installed plugin is on beta branch.
|
||||
* If on beta, the user should use the UI to update instead.
|
||||
*/
|
||||
|
||||
const { execSync } = require('child_process');
|
||||
const { existsSync } = require('fs');
|
||||
const path = require('path');
|
||||
const os = require('os');
|
||||
|
||||
const INSTALLED_PATH = path.join(os.homedir(), '.claude', 'plugins', 'marketplaces', 'thedotmack');
|
||||
|
||||
function getCurrentBranch() {
|
||||
try {
|
||||
if (!existsSync(path.join(INSTALLED_PATH, '.git'))) {
|
||||
return null;
|
||||
}
|
||||
return execSync('git rev-parse --abbrev-ref HEAD', {
|
||||
cwd: INSTALLED_PATH,
|
||||
encoding: 'utf-8',
|
||||
stdio: ['pipe', 'pipe', 'pipe']
|
||||
}).trim();
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
const branch = getCurrentBranch();
|
||||
|
||||
if (branch && branch !== 'main') {
|
||||
console.log('');
|
||||
console.log('\x1b[33m%s\x1b[0m', `WARNING: Installed plugin is on beta branch: ${branch}`);
|
||||
console.log('\x1b[33m%s\x1b[0m', 'Running rsync would overwrite beta code.');
|
||||
console.log('');
|
||||
console.log('Options:');
|
||||
console.log(' 1. Use UI at http://localhost:37777 to update beta');
|
||||
console.log(' 2. Switch to stable in UI first, then run sync');
|
||||
console.log(' 3. Force rsync: npm run sync-marketplace:force');
|
||||
console.log('');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Normal rsync for main branch or fresh install
|
||||
console.log('Syncing to marketplace...');
|
||||
try {
|
||||
execSync(
|
||||
'rsync -av --delete --exclude=.git ./ ~/.claude/plugins/marketplaces/thedotmack/',
|
||||
{ stdio: 'inherit' }
|
||||
);
|
||||
|
||||
console.log('Running npm install in marketplace...');
|
||||
execSync(
|
||||
'cd ~/.claude/plugins/marketplaces/thedotmack/ && npm install',
|
||||
{ stdio: 'inherit' }
|
||||
);
|
||||
|
||||
console.log('\x1b[32m%s\x1b[0m', 'Sync complete!');
|
||||
} catch (error) {
|
||||
console.error('\x1b[31m%s\x1b[0m', 'Sync failed:', error.message);
|
||||
process.exit(1);
|
||||
}
|
||||
+112
-55
@@ -71,6 +71,7 @@ interface Observation {
|
||||
concepts: string | null;
|
||||
files_read: string | null;
|
||||
files_modified: string | null;
|
||||
discovery_tokens: number | null;
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
@@ -130,12 +131,6 @@ function formatDate(dateStr: string): string {
|
||||
});
|
||||
}
|
||||
|
||||
// Helper: Estimate token count for text
|
||||
function estimateTokens(text: string | null): number {
|
||||
if (!text) return 0;
|
||||
return Math.ceil(text.length / CHARS_PER_TOKEN_ESTIMATE);
|
||||
}
|
||||
|
||||
// Helper: Convert absolute paths to relative paths
|
||||
function toRelativePath(filePath: string, cwd: string): string {
|
||||
if (path.isAbsolute(filePath)) {
|
||||
@@ -154,24 +149,6 @@ function renderSummaryField(label: string, value: string | null, color: string,
|
||||
return [`**${label}**: ${value}`, ''];
|
||||
}
|
||||
|
||||
// Helper: Get all observations for given sessions
|
||||
function getObservations(db: SessionStore, sessionIds: string[]): Observation[] {
|
||||
if (sessionIds.length === 0) return [];
|
||||
|
||||
const placeholders = sessionIds.map(() => '?').join(',');
|
||||
const observations = db.db.prepare(`
|
||||
SELECT
|
||||
id, sdk_session_id, type, title, subtitle, narrative,
|
||||
facts, concepts, files_read, files_modified,
|
||||
created_at, created_at_epoch
|
||||
FROM observations
|
||||
WHERE sdk_session_id IN (${placeholders})
|
||||
ORDER BY created_at_epoch DESC
|
||||
`).all(...sessionIds) as Observation[];
|
||||
|
||||
return observations;
|
||||
}
|
||||
|
||||
/**
|
||||
* Context Hook Main Logic
|
||||
*/
|
||||
@@ -187,7 +164,7 @@ async function contextHook(input?: SessionStartInput, useColors: boolean = false
|
||||
const allObservations = db.db.prepare(`
|
||||
SELECT
|
||||
id, sdk_session_id, type, title, subtitle, narrative,
|
||||
facts, concepts, files_read, files_modified,
|
||||
facts, concepts, files_read, files_modified, discovery_tokens,
|
||||
created_at, created_at_epoch
|
||||
FROM observations
|
||||
WHERE project = ?
|
||||
@@ -239,25 +216,74 @@ async function contextHook(input?: SessionStartInput, useColors: boolean = false
|
||||
if (timelineObs.length > 0) {
|
||||
// Legend/Key
|
||||
if (useColors) {
|
||||
output.push(`${colors.dim}Legend: 🎯 session-request | 🔴 bugfix | 🟣 feature | 🔄 refactor | ✅ change | 🔵 discovery | 🧠 decision${colors.reset}`);
|
||||
output.push('');
|
||||
output.push(`${colors.dim}Legend: 🎯 session-request | 🔴 bugfix | 🟣 feature | 🔄 refactor | ✅ change | 🔵 discovery | ⚖️ decision${colors.reset}`);
|
||||
} else {
|
||||
output.push(`**Legend:** 🎯 session-request | 🔴 bugfix | 🟣 feature | 🔄 refactor | ✅ change | 🔵 discovery | 🧠 decision`);
|
||||
output.push('');
|
||||
output.push(`**Legend:** 🎯 session-request | 🔴 bugfix | 🟣 feature | 🔄 refactor | ✅ change | 🔵 discovery | ⚖️ decision`);
|
||||
}
|
||||
output.push('');
|
||||
|
||||
// Progressive Disclosure Usage Instructions
|
||||
// Column Key
|
||||
if (useColors) {
|
||||
output.push(`${colors.dim}💡 Progressive Disclosure: This index shows WHAT exists (titles) and retrieval COST (token counts).${colors.reset}`);
|
||||
output.push(`${colors.dim} → Use MCP search tools to fetch full observation details on-demand (Layer 2)${colors.reset}`);
|
||||
output.push(`${colors.dim} → Prefer searching observations over re-reading code for past decisions and learnings${colors.reset}`);
|
||||
output.push(`${colors.dim} → Critical types (🔴 bugfix, 🧠 decision) often worth fetching immediately${colors.reset}`);
|
||||
output.push(`${colors.bright}💡 Column Key${colors.reset}`);
|
||||
output.push(`${colors.dim} Read: Tokens to read this observation (cost to learn it now)${colors.reset}`);
|
||||
output.push(`${colors.dim} Work: Tokens spent on work that produced this record (🔍 research, 🛠️ building, ⚖️ deciding)${colors.reset}`);
|
||||
} else {
|
||||
output.push(`💡 **Column Key**:`);
|
||||
output.push(`- **Read**: Tokens to read this observation (cost to learn it now)`);
|
||||
output.push(`- **Work**: Tokens spent on work that produced this record (🔍 research, 🛠️ building, ⚖️ deciding)`);
|
||||
}
|
||||
output.push('');
|
||||
|
||||
// Context Index Usage Instructions
|
||||
if (useColors) {
|
||||
output.push(`${colors.dim}💡 Context Index: This semantic index (titles, types, files, tokens) is usually sufficient to understand past work.${colors.reset}`);
|
||||
output.push('');
|
||||
output.push(`${colors.dim}When you need implementation details, rationale, or debugging context:${colors.reset}`);
|
||||
output.push(`${colors.dim} - Use the mem-search skill to fetch full observations on-demand${colors.reset}`);
|
||||
output.push(`${colors.dim} - Critical types (🔴 bugfix, ⚖️ decision) often need detailed fetching${colors.reset}`);
|
||||
output.push(`${colors.dim} - Trust this index over re-reading code for past decisions and learnings${colors.reset}`);
|
||||
} else {
|
||||
output.push(`💡 **Context Index:** This semantic index (titles, types, files, tokens) is usually sufficient to understand past work.`);
|
||||
output.push('');
|
||||
output.push(`When you need implementation details, rationale, or debugging context:`);
|
||||
output.push(`- Use the mem-search skill to fetch full observations on-demand`);
|
||||
output.push(`- Critical types (🔴 bugfix, ⚖️ decision) often need detailed fetching`);
|
||||
output.push(`- Trust this index over re-reading code for past decisions and learnings`);
|
||||
}
|
||||
output.push('');
|
||||
|
||||
// Section 1: Aggregate ROI Metrics
|
||||
const totalObservations = observations.length;
|
||||
const totalReadTokens = observations.reduce((sum, obs) => {
|
||||
// Estimate read tokens from observation size
|
||||
const obsSize = (obs.title?.length || 0) +
|
||||
(obs.subtitle?.length || 0) +
|
||||
(obs.narrative?.length || 0) +
|
||||
JSON.stringify(obs.facts || []).length;
|
||||
return sum + Math.ceil(obsSize / CHARS_PER_TOKEN_ESTIMATE);
|
||||
}, 0);
|
||||
const totalDiscoveryTokens = observations.reduce((sum, obs) => sum + (obs.discovery_tokens || 0), 0);
|
||||
const savings = totalDiscoveryTokens - totalReadTokens;
|
||||
const savingsPercent = totalDiscoveryTokens > 0
|
||||
? Math.round((savings / totalDiscoveryTokens) * 100)
|
||||
: 0;
|
||||
|
||||
// Display Context Economics section
|
||||
if (useColors) {
|
||||
output.push(`${colors.bright}${colors.cyan}📊 Context Economics${colors.reset}`);
|
||||
output.push(`${colors.dim} Loading: ${totalObservations} observations (${totalReadTokens.toLocaleString()} tokens to read)${colors.reset}`);
|
||||
output.push(`${colors.dim} Work investment: ${totalDiscoveryTokens.toLocaleString()} tokens spent on research, building, and decisions${colors.reset}`);
|
||||
if (totalDiscoveryTokens > 0) {
|
||||
output.push(`${colors.green} Your savings: ${savings.toLocaleString()} tokens (${savingsPercent}% reduction from reuse)${colors.reset}`);
|
||||
}
|
||||
output.push('');
|
||||
} else {
|
||||
output.push(`💡 **Progressive Disclosure:** This index shows WHAT exists (titles) and retrieval COST (token counts).`);
|
||||
output.push(`- Use MCP search tools to fetch full observation details on-demand (Layer 2)`);
|
||||
output.push(`- Prefer searching observations over re-reading code for past decisions and learnings`);
|
||||
output.push(`- Critical types (🔴 bugfix, 🧠 decision) often worth fetching immediately`);
|
||||
output.push(`📊 **Context Economics**:`);
|
||||
output.push(`- Loading: ${totalObservations} observations (${totalReadTokens.toLocaleString()} tokens to read)`);
|
||||
output.push(`- Work investment: ${totalDiscoveryTokens.toLocaleString()} tokens spent on research, building, and decisions`);
|
||||
if (totalDiscoveryTokens > 0) {
|
||||
output.push(`- Your savings: ${savings.toLocaleString()} tokens (${savingsPercent}% reduction from reuse)`);
|
||||
}
|
||||
output.push('');
|
||||
}
|
||||
|
||||
@@ -380,8 +406,8 @@ async function contextHook(input?: SessionStartInput, useColors: boolean = false
|
||||
|
||||
// Table header (markdown only)
|
||||
if (!useColors) {
|
||||
output.push(`| ID | Time | T | Title | Tokens |`);
|
||||
output.push(`|----|------|---|-------|--------|`);
|
||||
output.push(`| ID | Time | T | Title | Read | Work |`);
|
||||
output.push(`|----|------|---|-------|------|------|`);
|
||||
}
|
||||
|
||||
currentFile = file;
|
||||
@@ -389,10 +415,11 @@ async function contextHook(input?: SessionStartInput, useColors: boolean = false
|
||||
lastTime = '';
|
||||
}
|
||||
|
||||
// Render observation row
|
||||
let icon = '•';
|
||||
const time = formatTime(obs.created_at);
|
||||
const title = obs.title || 'Untitled';
|
||||
|
||||
// Map observation type to emoji
|
||||
// Map observation type to emoji icon
|
||||
let icon = '•';
|
||||
switch (obs.type) {
|
||||
case 'bugfix':
|
||||
icon = '🔴';
|
||||
@@ -410,15 +437,40 @@ async function contextHook(input?: SessionStartInput, useColors: boolean = false
|
||||
icon = '🔵';
|
||||
break;
|
||||
case 'decision':
|
||||
icon = '🧠';
|
||||
icon = '⚖️';
|
||||
break;
|
||||
default:
|
||||
icon = '•';
|
||||
}
|
||||
|
||||
const time = formatTime(obs.created_at);
|
||||
const title = obs.title || 'Untitled';
|
||||
const tokens = estimateTokens(obs.narrative);
|
||||
// Section 2: Calculate read tokens (estimate from observation size)
|
||||
const obsSize = (obs.title?.length || 0) +
|
||||
(obs.subtitle?.length || 0) +
|
||||
(obs.narrative?.length || 0) +
|
||||
JSON.stringify(obs.facts || []).length;
|
||||
const readTokens = Math.ceil(obsSize / CHARS_PER_TOKEN_ESTIMATE);
|
||||
|
||||
// Get discovery tokens (handle old observations without this field)
|
||||
const discoveryTokens = obs.discovery_tokens || 0;
|
||||
|
||||
// Map observation type to work emoji
|
||||
let workEmoji = '🔍'; // default to research/discovery
|
||||
switch (obs.type) {
|
||||
case 'discovery':
|
||||
workEmoji = '🔍'; // research/exploration
|
||||
break;
|
||||
case 'change':
|
||||
case 'feature':
|
||||
case 'bugfix':
|
||||
case 'refactor':
|
||||
workEmoji = '🛠️'; // building/modifying
|
||||
break;
|
||||
case 'decision':
|
||||
workEmoji = '⚖️'; // decision-making
|
||||
break;
|
||||
}
|
||||
|
||||
const discoveryDisplay = discoveryTokens > 0 ? `${workEmoji} ${discoveryTokens.toLocaleString()}` : '-';
|
||||
|
||||
const showTime = time !== lastTime;
|
||||
const timeDisplay = showTime ? time : '';
|
||||
@@ -426,10 +478,11 @@ async function contextHook(input?: SessionStartInput, useColors: boolean = false
|
||||
|
||||
if (useColors) {
|
||||
const timePart = showTime ? `${colors.dim}${time}${colors.reset}` : ' '.repeat(time.length);
|
||||
const tokensPart = tokens > 0 ? `${colors.dim}(~${tokens}t)${colors.reset}` : '';
|
||||
output.push(` ${colors.dim}#${obs.id}${colors.reset} ${timePart} ${icon} ${title} ${tokensPart}`);
|
||||
const readPart = readTokens > 0 ? `${colors.dim}(~${readTokens}t)${colors.reset}` : '';
|
||||
const discoveryPart = discoveryTokens > 0 ? `${colors.dim}(${workEmoji} ${discoveryTokens.toLocaleString()}t)${colors.reset}` : '';
|
||||
output.push(` ${colors.dim}#${obs.id}${colors.reset} ${timePart} ${icon} ${title} ${readPart} ${discoveryPart}`);
|
||||
} else {
|
||||
output.push(`| #${obs.id} | ${timeDisplay || '″'} | ${icon} | ${title} | ~${tokens} |`);
|
||||
output.push(`| #${obs.id} | ${timeDisplay || '″'} | ${icon} | ${title} | ~${readTokens} | ${discoveryDisplay} |`);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -456,11 +509,15 @@ async function contextHook(input?: SessionStartInput, useColors: boolean = false
|
||||
output.push(...renderSummaryField('Next Steps', mostRecentSummary.next_steps, colors.magenta, useColors));
|
||||
}
|
||||
|
||||
// Footer with MCP search instructions
|
||||
if (useColors) {
|
||||
output.push(`${colors.dim}Use claude-mem MCP search to access records with the given ID${colors.reset}`);
|
||||
} else {
|
||||
output.push(`*Use claude-mem MCP search to access records with the given ID*`);
|
||||
// Footer with token savings message
|
||||
if (totalDiscoveryTokens > 0 && savings > 0) {
|
||||
const workTokensK = Math.round(totalDiscoveryTokens / 1000);
|
||||
output.push('');
|
||||
if (useColors) {
|
||||
output.push(`${colors.dim}💰 Access ${workTokensK}k tokens of past research & decisions for just ${totalReadTokens.toLocaleString()}t. Use claude-mem search to access memories by ID instead of re-reading files.${colors.reset}`);
|
||||
} else {
|
||||
output.push(`💰 Access ${workTokensK}k tokens of past research & decisions for just ${totalReadTokens.toLocaleString()}t. Use claude-mem search to access memories by ID instead of re-reading files.`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -38,6 +38,7 @@ import { stdin } from 'process';
|
||||
import { SessionStore } from '../services/sqlite/SessionStore.js';
|
||||
import { createHookResponse } from './hook-response.js';
|
||||
import { ensureWorkerRunning, getWorkerPort } from '../shared/worker-utils.js';
|
||||
import { silentDebug } from '../utils/silent-debug.js';
|
||||
|
||||
export interface UserPromptSubmitInput {
|
||||
session_id: string;
|
||||
@@ -55,8 +56,27 @@ async function newHook(input?: UserPromptSubmitInput): Promise<void> {
|
||||
}
|
||||
|
||||
const { session_id, cwd, prompt } = input;
|
||||
|
||||
// Debug: Log what we received
|
||||
silentDebug('[new-hook] Input received', {
|
||||
session_id,
|
||||
cwd,
|
||||
cwd_type: typeof cwd,
|
||||
cwd_length: cwd?.length,
|
||||
has_cwd: !!cwd,
|
||||
prompt_length: prompt?.length
|
||||
});
|
||||
|
||||
const project = path.basename(cwd);
|
||||
|
||||
silentDebug('[new-hook] Project extracted', {
|
||||
project,
|
||||
project_type: typeof project,
|
||||
project_length: project?.length,
|
||||
is_empty: project === '',
|
||||
cwd_was: cwd
|
||||
});
|
||||
|
||||
// Ensure worker is running
|
||||
await ensureWorkerRunning();
|
||||
|
||||
|
||||
@@ -52,7 +52,8 @@ try {
|
||||
"\n\n📝 Claude-Mem Context Loaded\n" +
|
||||
" ℹ️ Note: This appears as stderr but is informational only\n\n" +
|
||||
output +
|
||||
`\n\n📺 Watch live in browser http://localhost:${port}/ (New! v5.1)\n`
|
||||
"\n\n💬 Feedback & Support\nhttps://github.com/thedotmack/claude-mem/discussions/110\n" +
|
||||
`\n📺 Watch live in browser http://localhost:${port}/\n`
|
||||
);
|
||||
|
||||
} catch (error) {
|
||||
|
||||
+943
-51
File diff suppressed because it is too large
Load Diff
@@ -13,7 +13,8 @@ import {
|
||||
|
||||
/**
|
||||
* Search interface for session-based memory
|
||||
* Provides FTS5 full-text search and structured queries for sessions, observations, and summaries
|
||||
* Provides filter-only structured queries for sessions, observations, and user prompts
|
||||
* Vector search is handled by ChromaDB - this class only supports filtering without query text
|
||||
*/
|
||||
export class SessionSearch {
|
||||
private db: Database.Database;
|
||||
@@ -31,7 +32,18 @@ export class SessionSearch {
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure FTS5 tables exist (inline migration)
|
||||
* Ensure FTS5 tables exist (backward compatibility only - no longer used for search)
|
||||
*
|
||||
* FTS5 tables are maintained for backward compatibility but not used for search.
|
||||
* Vector search (Chroma) is now the primary search mechanism.
|
||||
*
|
||||
* Retention Rationale:
|
||||
* - Prevents breaking existing installations with FTS5 tables
|
||||
* - Allows graceful migration path for users
|
||||
* - Tables maintained but search paths removed
|
||||
* - Triggers still fire to keep tables synchronized
|
||||
*
|
||||
* TODO: Remove FTS5 infrastructure in future major version (v7.0.0)
|
||||
*/
|
||||
private ensureFTSTables(): void {
|
||||
try {
|
||||
@@ -134,22 +146,6 @@ export class SessionSearch {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Escape FTS5 special characters in user input
|
||||
*
|
||||
* FTS5 uses double quotes for phrase searches and treats certain characters
|
||||
* as operators (*, AND, OR, NOT, parentheses, etc.). To prevent injection,
|
||||
* we wrap user input in double quotes and escape internal quotes by doubling them.
|
||||
* This converts any user input into a safe phrase search.
|
||||
*
|
||||
* @param text - User input to escape for FTS5 MATCH queries
|
||||
* @returns Safely escaped FTS5 query string
|
||||
*/
|
||||
private escapeFTS5(text: string): string {
|
||||
// Escape internal double quotes by doubling them (FTS5 standard)
|
||||
// Then wrap the entire string in double quotes for phrase search
|
||||
return `"${text.replace(/"/g, '""')}"`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build WHERE clause for structured filters
|
||||
@@ -243,116 +239,78 @@ export class SessionSearch {
|
||||
}
|
||||
|
||||
/**
|
||||
* Search observations using FTS5 full-text search
|
||||
* Search observations using filter-only direct SQLite query.
|
||||
* Vector search is handled by ChromaDB - this only supports filtering without query text.
|
||||
*/
|
||||
searchObservations(query: string, options: SearchOptions = {}): ObservationSearchResult[] {
|
||||
searchObservations(query: string | undefined, options: SearchOptions = {}): ObservationSearchResult[] {
|
||||
const params: any[] = [];
|
||||
const { limit = 50, offset = 0, orderBy = 'relevance', ...filters } = options;
|
||||
|
||||
// Build FTS5 match query
|
||||
const ftsQuery = this.escapeFTS5(query);
|
||||
params.push(ftsQuery);
|
||||
// FILTER-ONLY PATH: When no query text, query table directly
|
||||
// This enables date filtering which Chroma cannot do (requires direct SQLite access)
|
||||
if (!query) {
|
||||
const filterClause = this.buildFilterClause(filters, params, 'o');
|
||||
if (!filterClause) {
|
||||
throw new Error('Either query or filters required for search');
|
||||
}
|
||||
|
||||
// Build filter conditions
|
||||
const filterClause = this.buildFilterClause(filters, params, 'o');
|
||||
const whereClause = filterClause ? `AND ${filterClause}` : '';
|
||||
const orderClause = this.buildOrderClause(orderBy, false);
|
||||
|
||||
// Build ORDER BY
|
||||
const orderClause = this.buildOrderClause(orderBy, true);
|
||||
const sql = `
|
||||
SELECT o.*, o.discovery_tokens
|
||||
FROM observations o
|
||||
WHERE ${filterClause}
|
||||
${orderClause}
|
||||
LIMIT ? OFFSET ?
|
||||
`;
|
||||
|
||||
// Main query with FTS5
|
||||
const sql = `
|
||||
SELECT
|
||||
o.*,
|
||||
observations_fts.rank as rank
|
||||
FROM observations o
|
||||
JOIN observations_fts ON o.id = observations_fts.rowid
|
||||
WHERE observations_fts MATCH ?
|
||||
${whereClause}
|
||||
${orderClause}
|
||||
LIMIT ? OFFSET ?
|
||||
`;
|
||||
|
||||
params.push(limit, offset);
|
||||
|
||||
const results = this.db.prepare(sql).all(...params) as ObservationSearchResult[];
|
||||
|
||||
// Normalize rank to score (0-1, higher is better)
|
||||
if (results.length > 0) {
|
||||
const minRank = Math.min(...results.map(r => r.rank || 0));
|
||||
const maxRank = Math.max(...results.map(r => r.rank || 0));
|
||||
const range = maxRank - minRank || 1;
|
||||
|
||||
results.forEach(r => {
|
||||
if (r.rank !== undefined) {
|
||||
// Invert rank (lower rank = better match) and normalize to 0-1
|
||||
r.score = 1 - ((r.rank - minRank) / range);
|
||||
}
|
||||
});
|
||||
params.push(limit, offset);
|
||||
return this.db.prepare(sql).all(...params) as ObservationSearchResult[];
|
||||
}
|
||||
|
||||
return results;
|
||||
// Vector search with query text should be handled by ChromaDB
|
||||
// This method only supports filter-only queries (query=undefined)
|
||||
console.warn('[SessionSearch] Text search not supported - use ChromaDB for vector search');
|
||||
return [];
|
||||
}
|
||||
|
||||
/**
|
||||
* Search session summaries using FTS5 full-text search
|
||||
* Search session summaries using filter-only direct SQLite query.
|
||||
* Vector search is handled by ChromaDB - this only supports filtering without query text.
|
||||
*/
|
||||
searchSessions(query: string, options: SearchOptions = {}): SessionSummarySearchResult[] {
|
||||
searchSessions(query: string | undefined, options: SearchOptions = {}): SessionSummarySearchResult[] {
|
||||
const params: any[] = [];
|
||||
const { limit = 50, offset = 0, orderBy = 'relevance', ...filters } = options;
|
||||
|
||||
// Build FTS5 match query
|
||||
const ftsQuery = this.escapeFTS5(query);
|
||||
params.push(ftsQuery);
|
||||
// FILTER-ONLY PATH: When no query text, query session_summaries table directly
|
||||
if (!query) {
|
||||
const filterOptions = { ...filters };
|
||||
delete filterOptions.type;
|
||||
const filterClause = this.buildFilterClause(filterOptions, params, 's');
|
||||
if (!filterClause) {
|
||||
throw new Error('Either query or filters required for search');
|
||||
}
|
||||
|
||||
// Build filter conditions (without type filter - not applicable to summaries)
|
||||
const filterOptions = { ...filters };
|
||||
delete filterOptions.type;
|
||||
const filterClause = this.buildFilterClause(filterOptions, params, 's');
|
||||
const whereClause = filterClause ? `AND ${filterClause}` : '';
|
||||
const orderClause = orderBy === 'date_asc'
|
||||
? 'ORDER BY s.created_at_epoch ASC'
|
||||
: 'ORDER BY s.created_at_epoch DESC';
|
||||
|
||||
// Note: session_summaries don't have files_read/files_modified in the same way
|
||||
// We'll need to adjust the filter clause
|
||||
const adjustedWhereClause = whereClause.replace(/files_read/g, 'files_read').replace(/files_modified/g, 'files_edited');
|
||||
const sql = `
|
||||
SELECT s.*, s.discovery_tokens
|
||||
FROM session_summaries s
|
||||
WHERE ${filterClause}
|
||||
${orderClause}
|
||||
LIMIT ? OFFSET ?
|
||||
`;
|
||||
|
||||
// Build ORDER BY
|
||||
const orderClause = orderBy === 'relevance'
|
||||
? 'ORDER BY session_summaries_fts.rank ASC'
|
||||
: orderBy === 'date_asc'
|
||||
? 'ORDER BY s.created_at_epoch ASC'
|
||||
: 'ORDER BY s.created_at_epoch DESC';
|
||||
|
||||
// Main query with FTS5
|
||||
const sql = `
|
||||
SELECT
|
||||
s.*,
|
||||
session_summaries_fts.rank as rank
|
||||
FROM session_summaries s
|
||||
JOIN session_summaries_fts ON s.id = session_summaries_fts.rowid
|
||||
WHERE session_summaries_fts MATCH ?
|
||||
${adjustedWhereClause}
|
||||
${orderClause}
|
||||
LIMIT ? OFFSET ?
|
||||
`;
|
||||
|
||||
params.push(limit, offset);
|
||||
|
||||
const results = this.db.prepare(sql).all(...params) as SessionSummarySearchResult[];
|
||||
|
||||
// Normalize rank to score
|
||||
if (results.length > 0) {
|
||||
const minRank = Math.min(...results.map(r => r.rank || 0));
|
||||
const maxRank = Math.max(...results.map(r => r.rank || 0));
|
||||
const range = maxRank - minRank || 1;
|
||||
|
||||
results.forEach(r => {
|
||||
if (r.rank !== undefined) {
|
||||
r.score = 1 - ((r.rank - minRank) / range);
|
||||
}
|
||||
});
|
||||
params.push(limit, offset);
|
||||
return this.db.prepare(sql).all(...params) as SessionSummarySearchResult[];
|
||||
}
|
||||
|
||||
return results;
|
||||
// Vector search with query text should be handled by ChromaDB
|
||||
// This method only supports filter-only queries (query=undefined)
|
||||
console.warn('[SessionSearch] Text search not supported - use ChromaDB for vector search');
|
||||
return [];
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -368,7 +326,7 @@ export class SessionSearch {
|
||||
const orderClause = this.buildOrderClause(orderBy, false);
|
||||
|
||||
const sql = `
|
||||
SELECT o.*
|
||||
SELECT o.*, o.discovery_tokens
|
||||
FROM observations o
|
||||
WHERE ${filterClause}
|
||||
${orderClause}
|
||||
@@ -396,7 +354,7 @@ export class SessionSearch {
|
||||
const orderClause = this.buildOrderClause(orderBy, false);
|
||||
|
||||
const observationsSql = `
|
||||
SELECT o.*
|
||||
SELECT o.*, o.discovery_tokens
|
||||
FROM observations o
|
||||
WHERE ${filterClause}
|
||||
${orderClause}
|
||||
@@ -440,7 +398,7 @@ export class SessionSearch {
|
||||
sessionParams.push(`%${filePath}%`, `%${filePath}%`);
|
||||
|
||||
const sessionsSql = `
|
||||
SELECT s.*
|
||||
SELECT s.*, s.discovery_tokens
|
||||
FROM session_summaries s
|
||||
WHERE ${baseConditions.join(' AND ')}
|
||||
ORDER BY s.created_at_epoch DESC
|
||||
@@ -470,7 +428,7 @@ export class SessionSearch {
|
||||
const orderClause = this.buildOrderClause(orderBy, false);
|
||||
|
||||
const sql = `
|
||||
SELECT o.*
|
||||
SELECT o.*, o.discovery_tokens
|
||||
FROM observations o
|
||||
WHERE ${filterClause}
|
||||
${orderClause}
|
||||
@@ -483,16 +441,13 @@ export class SessionSearch {
|
||||
}
|
||||
|
||||
/**
|
||||
* Search user prompts with full-text search
|
||||
* Search user prompts using filter-only direct SQLite query.
|
||||
* Vector search is handled by ChromaDB - this only supports filtering without query text.
|
||||
*/
|
||||
searchUserPrompts(query: string, options: SearchOptions = {}): UserPromptSearchResult[] {
|
||||
searchUserPrompts(query: string | undefined, options: SearchOptions = {}): UserPromptSearchResult[] {
|
||||
const params: any[] = [];
|
||||
const { limit = 20, offset = 0, orderBy = 'relevance', ...filters } = options;
|
||||
|
||||
// Build FTS5 match query
|
||||
const ftsQuery = this.escapeFTS5(query);
|
||||
params.push(ftsQuery);
|
||||
|
||||
// Build filter conditions (join with sdk_sessions for project filtering)
|
||||
const baseConditions: string[] = [];
|
||||
if (filters.project) {
|
||||
@@ -514,47 +469,34 @@ export class SessionSearch {
|
||||
}
|
||||
}
|
||||
|
||||
const whereClause = baseConditions.length > 0 ? `AND ${baseConditions.join(' AND ')}` : '';
|
||||
// FILTER-ONLY PATH: When no query text, query user_prompts table directly
|
||||
if (!query) {
|
||||
if (baseConditions.length === 0) {
|
||||
throw new Error('Either query or filters required for search');
|
||||
}
|
||||
|
||||
// Build ORDER BY
|
||||
const orderClause = orderBy === 'relevance'
|
||||
? 'ORDER BY user_prompts_fts.rank ASC'
|
||||
: orderBy === 'date_asc'
|
||||
? 'ORDER BY up.created_at_epoch ASC'
|
||||
: 'ORDER BY up.created_at_epoch DESC';
|
||||
const whereClause = `WHERE ${baseConditions.join(' AND ')}`;
|
||||
const orderClause = orderBy === 'date_asc'
|
||||
? 'ORDER BY up.created_at_epoch ASC'
|
||||
: 'ORDER BY up.created_at_epoch DESC';
|
||||
|
||||
// Main query with FTS5 (join sdk_sessions for project filtering)
|
||||
const sql = `
|
||||
SELECT
|
||||
up.*,
|
||||
user_prompts_fts.rank as rank
|
||||
FROM user_prompts up
|
||||
JOIN user_prompts_fts ON up.id = user_prompts_fts.rowid
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE user_prompts_fts MATCH ?
|
||||
${whereClause}
|
||||
${orderClause}
|
||||
LIMIT ? OFFSET ?
|
||||
`;
|
||||
const sql = `
|
||||
SELECT up.*
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
${whereClause}
|
||||
${orderClause}
|
||||
LIMIT ? OFFSET ?
|
||||
`;
|
||||
|
||||
params.push(limit, offset);
|
||||
|
||||
const results = this.db.prepare(sql).all(...params) as UserPromptSearchResult[];
|
||||
|
||||
// Normalize rank to score
|
||||
if (results.length > 0) {
|
||||
const minRank = Math.min(...results.map(r => r.rank || 0));
|
||||
const maxRank = Math.max(...results.map(r => r.rank || 0));
|
||||
const range = maxRank - minRank || 1;
|
||||
|
||||
results.forEach(r => {
|
||||
if (r.rank !== undefined) {
|
||||
r.score = 1 - ((r.rank - minRank) / range);
|
||||
}
|
||||
});
|
||||
params.push(limit, offset);
|
||||
return this.db.prepare(sql).all(...params) as UserPromptSearchResult[];
|
||||
}
|
||||
|
||||
return results;
|
||||
// Vector search with query text should be handled by ChromaDB
|
||||
// This method only supports filter-only queries (query=undefined)
|
||||
console.warn('[SessionSearch] Text search not supported - use ChromaDB for vector search');
|
||||
return [];
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -28,6 +28,7 @@ export class SessionStore {
|
||||
this.addObservationHierarchicalFields();
|
||||
this.makeObservationsTextNullable();
|
||||
this.createUserPromptsTable();
|
||||
this.ensureDiscoveryTokensColumn();
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -492,6 +493,43 @@ export class SessionStore {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure discovery_tokens column exists (migration 11)
|
||||
* CRITICAL: This migration was incorrectly using version 7 (which was already taken by removeSessionSummariesUniqueConstraint)
|
||||
* The duplicate version number may have caused migration tracking issues in some databases
|
||||
*/
|
||||
private ensureDiscoveryTokensColumn(): void {
|
||||
try {
|
||||
// Check if migration already applied to avoid unnecessary re-runs
|
||||
const applied = this.db.prepare('SELECT version FROM schema_versions WHERE version = ?').get(11) as {version: number} | undefined;
|
||||
if (applied) return;
|
||||
|
||||
// Check if discovery_tokens column exists in observations table
|
||||
const observationsInfo = this.db.pragma('table_info(observations)');
|
||||
const obsHasDiscoveryTokens = (observationsInfo as any[]).some((col: any) => col.name === 'discovery_tokens');
|
||||
|
||||
if (!obsHasDiscoveryTokens) {
|
||||
this.db.exec('ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0');
|
||||
console.error('[SessionStore] Added discovery_tokens column to observations table');
|
||||
}
|
||||
|
||||
// Check if discovery_tokens column exists in session_summaries table
|
||||
const summariesInfo = this.db.pragma('table_info(session_summaries)');
|
||||
const sumHasDiscoveryTokens = (summariesInfo as any[]).some((col: any) => col.name === 'discovery_tokens');
|
||||
|
||||
if (!sumHasDiscoveryTokens) {
|
||||
this.db.exec('ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0');
|
||||
console.error('[SessionStore] Added discovery_tokens column to session_summaries table');
|
||||
}
|
||||
|
||||
// Record migration only after successful column verification/addition
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(11, new Date().toISOString());
|
||||
} catch (error: any) {
|
||||
console.error('[SessionStore] Discovery tokens migration error:', error.message);
|
||||
throw error; // Re-throw to prevent silent failures
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get recent session summaries for a project
|
||||
*/
|
||||
@@ -975,6 +1013,17 @@ export class SessionStore {
|
||||
|
||||
// If lastInsertRowid is 0, insert was ignored (session exists), so fetch existing ID
|
||||
if (result.lastInsertRowid === 0 || result.changes === 0) {
|
||||
// Session exists - UPDATE project and user_prompt if we have non-empty values
|
||||
// This fixes the bug where SAVE hook creates session with empty project,
|
||||
// then NEW hook can't update it because INSERT OR IGNORE skips the insert
|
||||
if (project && project.trim() !== '') {
|
||||
this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET project = ?, user_prompt = ?
|
||||
WHERE claude_session_id = ?
|
||||
`).run(project, userPrompt, claudeSessionId);
|
||||
}
|
||||
|
||||
const selectStmt = this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE claude_session_id = ? LIMIT 1
|
||||
`);
|
||||
@@ -1074,7 +1123,8 @@ export class SessionStore {
|
||||
files_read: string[];
|
||||
files_modified: string[];
|
||||
},
|
||||
promptNumber?: number
|
||||
promptNumber?: number,
|
||||
discoveryTokens: number = 0
|
||||
): { id: number; createdAtEpoch: number } {
|
||||
const now = new Date();
|
||||
const nowEpoch = now.getTime();
|
||||
@@ -1105,8 +1155,8 @@ export class SessionStore {
|
||||
const stmt = this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
const result = stmt.run(
|
||||
@@ -1121,6 +1171,7 @@ export class SessionStore {
|
||||
JSON.stringify(observation.files_read),
|
||||
JSON.stringify(observation.files_modified),
|
||||
promptNumber || null,
|
||||
discoveryTokens,
|
||||
now.toISOString(),
|
||||
nowEpoch
|
||||
);
|
||||
@@ -1146,7 +1197,8 @@ export class SessionStore {
|
||||
next_steps: string;
|
||||
notes: string | null;
|
||||
},
|
||||
promptNumber?: number
|
||||
promptNumber?: number,
|
||||
discoveryTokens: number = 0
|
||||
): { id: number; createdAtEpoch: number } {
|
||||
const now = new Date();
|
||||
const nowEpoch = now.getTime();
|
||||
@@ -1177,8 +1229,8 @@ export class SessionStore {
|
||||
const stmt = this.db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
(sdk_session_id, project, request, investigated, learned, completed,
|
||||
next_steps, notes, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
next_steps, notes, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
const result = stmt.run(
|
||||
@@ -1191,6 +1243,7 @@ export class SessionStore {
|
||||
summary.next_steps,
|
||||
summary.notes,
|
||||
promptNumber || null,
|
||||
discoveryTokens,
|
||||
now.toISOString(),
|
||||
nowEpoch
|
||||
);
|
||||
|
||||
@@ -471,6 +471,30 @@ export const migration006: Migration = {
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Migration 007 - Add discovery_tokens column for ROI metrics
|
||||
* Tracks token cost of discovering/creating each observation and summary
|
||||
*/
|
||||
export const migration007: Migration = {
|
||||
version: 7,
|
||||
up: (db: Database) => {
|
||||
// Add discovery_tokens to observations table
|
||||
db.run(`ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0`);
|
||||
|
||||
// Add discovery_tokens to session_summaries table
|
||||
db.run(`ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0`);
|
||||
|
||||
console.log('✅ Added discovery_tokens columns for ROI tracking');
|
||||
},
|
||||
|
||||
down: (db: Database) => {
|
||||
// Note: SQLite doesn't support DROP COLUMN in all versions
|
||||
// In production, would need to recreate tables without these columns
|
||||
console.log('⚠️ Warning: SQLite ALTER TABLE DROP COLUMN not fully supported');
|
||||
console.log('⚠️ To rollback, manually recreate the observations and session_summaries tables');
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* All migrations in order
|
||||
*/
|
||||
@@ -480,5 +504,6 @@ export const migrations: Migration[] = [
|
||||
migration003,
|
||||
migration004,
|
||||
migration005,
|
||||
migration006
|
||||
migration006,
|
||||
migration007
|
||||
];
|
||||
@@ -215,6 +215,7 @@ export interface ObservationRow {
|
||||
files_read: string | null; // JSON array
|
||||
files_modified: string | null; // JSON array
|
||||
prompt_number: number | null;
|
||||
discovery_tokens: number; // ROI metrics: tokens spent discovering this observation
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
@@ -232,6 +233,7 @@ export interface SessionSummaryRow {
|
||||
files_edited: string | null; // JSON array
|
||||
notes: string | null;
|
||||
prompt_number: number | null;
|
||||
discovery_tokens: number; // ROI metrics: cumulative tokens spent in this session
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
|
||||
@@ -36,6 +36,7 @@ interface StoredObservation {
|
||||
files_read: string | null; // JSON
|
||||
files_modified: string | null; // JSON
|
||||
prompt_number: number;
|
||||
discovery_tokens: number; // ROI metrics
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
@@ -51,6 +52,7 @@ interface StoredSummary {
|
||||
next_steps: string | null;
|
||||
notes: string | null;
|
||||
prompt_number: number;
|
||||
discovery_tokens: number; // ROI metrics
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
@@ -345,7 +347,8 @@ export class ChromaSync {
|
||||
project: string,
|
||||
obs: ParsedObservation,
|
||||
promptNumber: number,
|
||||
createdAtEpoch: number
|
||||
createdAtEpoch: number,
|
||||
discoveryTokens: number = 0
|
||||
): Promise<void> {
|
||||
// Convert ParsedObservation to StoredObservation format
|
||||
const stored: StoredObservation = {
|
||||
@@ -362,6 +365,7 @@ export class ChromaSync {
|
||||
files_read: JSON.stringify(obs.files_read),
|
||||
files_modified: JSON.stringify(obs.files_modified),
|
||||
prompt_number: promptNumber,
|
||||
discovery_tokens: discoveryTokens,
|
||||
created_at: new Date(createdAtEpoch * 1000).toISOString(),
|
||||
created_at_epoch: createdAtEpoch
|
||||
};
|
||||
@@ -387,7 +391,8 @@ export class ChromaSync {
|
||||
project: string,
|
||||
summary: ParsedSummary,
|
||||
promptNumber: number,
|
||||
createdAtEpoch: number
|
||||
createdAtEpoch: number,
|
||||
discoveryTokens: number = 0
|
||||
): Promise<void> {
|
||||
// Convert ParsedSummary to StoredSummary format
|
||||
const stored: StoredSummary = {
|
||||
@@ -401,6 +406,7 @@ export class ChromaSync {
|
||||
next_steps: summary.next_steps,
|
||||
notes: summary.notes,
|
||||
prompt_number: promptNumber,
|
||||
discovery_tokens: discoveryTokens,
|
||||
created_at: new Date(createdAtEpoch * 1000).toISOString(),
|
||||
created_at_epoch: createdAtEpoch
|
||||
};
|
||||
|
||||
@@ -29,6 +29,7 @@ import { SSEBroadcaster } from './worker/SSEBroadcaster.js';
|
||||
import { SDKAgent } from './worker/SDKAgent.js';
|
||||
import { PaginationHelper } from './worker/PaginationHelper.js';
|
||||
import { SettingsManager } from './worker/SettingsManager.js';
|
||||
import { getBranchInfo, switchBranch, pullUpdates, type BranchInfo, type SwitchResult } from './worker/BranchManager.js';
|
||||
|
||||
export class WorkerService {
|
||||
private app: express.Application;
|
||||
@@ -155,6 +156,12 @@ export class WorkerService {
|
||||
this.app.get('/api/observations', this.handleGetObservations.bind(this));
|
||||
this.app.get('/api/summaries', this.handleGetSummaries.bind(this));
|
||||
this.app.get('/api/prompts', this.handleGetPrompts.bind(this));
|
||||
|
||||
// Fetch by ID
|
||||
this.app.get('/api/observation/:id', this.handleGetObservationById.bind(this));
|
||||
this.app.get('/api/session/:id', this.handleGetSessionById.bind(this));
|
||||
this.app.get('/api/prompt/:id', this.handleGetPromptById.bind(this));
|
||||
|
||||
this.app.get('/api/stats', this.handleGetStats.bind(this));
|
||||
this.app.get('/api/processing-status', this.handleGetProcessingStatus.bind(this));
|
||||
this.app.post('/api/processing', this.handleSetProcessing.bind(this));
|
||||
@@ -167,7 +174,20 @@ export class WorkerService {
|
||||
this.app.get('/api/mcp/status', this.handleGetMcpStatus.bind(this));
|
||||
this.app.post('/api/mcp/toggle', this.handleToggleMcp.bind(this));
|
||||
|
||||
// Branch switching (beta toggle)
|
||||
this.app.get('/api/branch/status', this.handleGetBranchStatus.bind(this));
|
||||
this.app.post('/api/branch/switch', this.handleSwitchBranch.bind(this));
|
||||
this.app.post('/api/branch/update', this.handleUpdateBranch.bind(this));
|
||||
|
||||
// Search API endpoints (for skill-based search)
|
||||
// Unified endpoints (new consolidated API)
|
||||
this.app.get('/api/search', this.handleUnifiedSearch.bind(this));
|
||||
this.app.get('/api/timeline', this.handleUnifiedTimeline.bind(this));
|
||||
this.app.get('/api/decisions', this.handleDecisions.bind(this));
|
||||
this.app.get('/api/changes', this.handleChanges.bind(this));
|
||||
this.app.get('/api/how-it-works', this.handleHowItWorks.bind(this));
|
||||
|
||||
// Backward compatibility endpoints (use /api/search with type param instead)
|
||||
this.app.get('/api/search/observations', this.handleSearchObservations.bind(this));
|
||||
this.app.get('/api/search/sessions', this.handleSearchSessions.bind(this));
|
||||
this.app.get('/api/search/prompts', this.handleSearchPrompts.bind(this));
|
||||
@@ -180,15 +200,50 @@ export class WorkerService {
|
||||
this.app.get('/api/search/help', this.handleSearchHelp.bind(this));
|
||||
}
|
||||
|
||||
/**
|
||||
* Cleanup orphaned MCP server processes (uvx/chroma) from previous sessions
|
||||
*/
|
||||
private async cleanupOrphanedProcesses(): Promise<void> {
|
||||
try {
|
||||
const { execSync } = await import('child_process');
|
||||
|
||||
// Find orphaned uvx processes (which spawn chroma servers)
|
||||
try {
|
||||
const processes = execSync('pgrep -fl uvx', { encoding: 'utf-8', stdio: 'pipe' }).trim();
|
||||
if (processes) {
|
||||
const processCount = processes.split('\n').length;
|
||||
logger.info('WORKER', 'Cleaning up orphaned MCP processes', { count: processCount });
|
||||
|
||||
// Kill the processes
|
||||
execSync('pkill -f uvx', { stdio: 'pipe' });
|
||||
logger.success('WORKER', `Cleaned up ${processCount} orphaned MCP server processes`);
|
||||
}
|
||||
} catch (error: any) {
|
||||
// pgrep returns exit code 1 if no processes found (not an error)
|
||||
if (error.status === 1) {
|
||||
logger.debug('WORKER', 'No orphaned MCP processes to clean up');
|
||||
} else {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
// Don't fail startup if cleanup fails
|
||||
logger.warn('WORKER', 'Failed to cleanup orphaned processes (non-fatal)', {}, error as Error);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Start the worker service
|
||||
*/
|
||||
async start(): Promise<void> {
|
||||
// Cleanup orphaned processes from previous sessions
|
||||
await this.cleanupOrphanedProcesses();
|
||||
|
||||
// Initialize database (once, stays open)
|
||||
await this.dbManager.initialize();
|
||||
|
||||
// Connect to MCP search server
|
||||
const searchServerPath = path.join(__dirname, '..', '..', 'plugin', 'scripts', 'search-server.mjs');
|
||||
const searchServerPath = path.join(__dirname, '..', '..', 'plugin', 'scripts', 'search-server.cjs');
|
||||
const transport = new StdioClientTransport({
|
||||
command: 'node',
|
||||
args: [searchServerPath],
|
||||
@@ -215,6 +270,16 @@ export class WorkerService {
|
||||
// Shutdown all active sessions
|
||||
await this.sessionManager.shutdownAll();
|
||||
|
||||
// Close MCP client connection (terminates search server process)
|
||||
if (this.mcpClient) {
|
||||
try {
|
||||
await this.mcpClient.close();
|
||||
logger.info('SYSTEM', 'MCP client closed');
|
||||
} catch (error) {
|
||||
logger.error('SYSTEM', 'Failed to close MCP client', {}, error as Error);
|
||||
}
|
||||
}
|
||||
|
||||
// Close HTTP server
|
||||
if (this.server) {
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
@@ -222,7 +287,7 @@ export class WorkerService {
|
||||
});
|
||||
}
|
||||
|
||||
// Close database connection
|
||||
// Close database connection (includes ChromaSync cleanup)
|
||||
await this.dbManager.close();
|
||||
|
||||
logger.info('SYSTEM', 'Worker shutdown complete');
|
||||
@@ -277,9 +342,11 @@ export class WorkerService {
|
||||
|
||||
// Send initial processing status (based on queue depth + active generators)
|
||||
const isProcessing = this.sessionManager.isAnySessionProcessing();
|
||||
const queueDepth = this.sessionManager.getTotalActiveWork(); // Includes queued + actively processing
|
||||
this.sseBroadcaster.broadcast({
|
||||
type: 'processing_status',
|
||||
isProcessing
|
||||
isProcessing,
|
||||
queueDepth
|
||||
});
|
||||
}
|
||||
|
||||
@@ -612,6 +679,87 @@ export class WorkerService {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get observation by ID
|
||||
* GET /api/observation/:id
|
||||
*/
|
||||
private handleGetObservationById(req: Request, res: Response): void {
|
||||
try {
|
||||
const id = parseInt(req.params.id, 10);
|
||||
if (isNaN(id)) {
|
||||
res.status(400).json({ error: 'Invalid observation ID' });
|
||||
return;
|
||||
}
|
||||
|
||||
const store = this.dbManager.getSessionStore();
|
||||
const observation = store.getObservationById(id);
|
||||
|
||||
if (!observation) {
|
||||
res.status(404).json({ error: `Observation #${id} not found` });
|
||||
return;
|
||||
}
|
||||
|
||||
res.json(observation);
|
||||
} catch (error) {
|
||||
logger.failure('WORKER', 'Get observation by ID failed', {}, error as Error);
|
||||
res.status(500).json({ error: (error as Error).message });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get session by ID
|
||||
* GET /api/session/:id
|
||||
*/
|
||||
private handleGetSessionById(req: Request, res: Response): void {
|
||||
try {
|
||||
const id = parseInt(req.params.id, 10);
|
||||
if (isNaN(id)) {
|
||||
res.status(400).json({ error: 'Invalid session ID' });
|
||||
return;
|
||||
}
|
||||
|
||||
const store = this.dbManager.getSessionStore();
|
||||
const sessions = store.getSessionSummariesByIds([id]);
|
||||
|
||||
if (sessions.length === 0) {
|
||||
res.status(404).json({ error: `Session #${id} not found` });
|
||||
return;
|
||||
}
|
||||
|
||||
res.json(sessions[0]);
|
||||
} catch (error) {
|
||||
logger.failure('WORKER', 'Get session by ID failed', {}, error as Error);
|
||||
res.status(500).json({ error: (error as Error).message });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get user prompt by ID
|
||||
* GET /api/prompt/:id
|
||||
*/
|
||||
private handleGetPromptById(req: Request, res: Response): void {
|
||||
try {
|
||||
const id = parseInt(req.params.id, 10);
|
||||
if (isNaN(id)) {
|
||||
res.status(400).json({ error: 'Invalid prompt ID' });
|
||||
return;
|
||||
}
|
||||
|
||||
const store = this.dbManager.getSessionStore();
|
||||
const prompts = store.getUserPromptsByIds([id]);
|
||||
|
||||
if (prompts.length === 0) {
|
||||
res.status(404).json({ error: `Prompt #${id} not found` });
|
||||
return;
|
||||
}
|
||||
|
||||
res.json(prompts[0]);
|
||||
} catch (error) {
|
||||
logger.failure('WORKER', 'Get prompt by ID failed', {}, error as Error);
|
||||
res.status(500).json({ error: (error as Error).message });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get database statistics (with worker metadata)
|
||||
*/
|
||||
@@ -761,11 +909,12 @@ export class WorkerService {
|
||||
}
|
||||
|
||||
/**
|
||||
* Get processing status (for viewer UI spinner)
|
||||
* Get processing status (for viewer UI spinner and queue indicator)
|
||||
*/
|
||||
private handleGetProcessingStatus(req: Request, res: Response): void {
|
||||
const isProcessing = this.sessionManager.isAnySessionProcessing();
|
||||
res.json({ isProcessing });
|
||||
const queueDepth = this.sessionManager.getTotalActiveWork(); // Includes queued + actively processing
|
||||
res.json({ isProcessing, queueDepth });
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
@@ -778,7 +927,7 @@ export class WorkerService {
|
||||
*/
|
||||
broadcastProcessingStatus(): void {
|
||||
const isProcessing = this.sessionManager.isAnySessionProcessing();
|
||||
const queueDepth = this.sessionManager.getTotalQueueDepth();
|
||||
const queueDepth = this.sessionManager.getTotalActiveWork(); // Includes queued + actively processing
|
||||
const activeSessions = this.sessionManager.getActiveSessionCount();
|
||||
|
||||
logger.info('WORKER', 'Broadcasting processing status', {
|
||||
@@ -789,7 +938,8 @@ export class WorkerService {
|
||||
|
||||
this.sseBroadcaster.broadcast({
|
||||
type: 'processing_status',
|
||||
isProcessing
|
||||
isProcessing,
|
||||
queueDepth
|
||||
});
|
||||
}
|
||||
|
||||
@@ -892,11 +1042,188 @@ export class WorkerService {
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Search API Handlers (for skill-based search)
|
||||
// Branch Switching Handlers (Beta Toggle)
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Search observations
|
||||
* GET /api/branch/status - Get current branch information
|
||||
*/
|
||||
private handleGetBranchStatus(req: Request, res: Response): void {
|
||||
try {
|
||||
const info = getBranchInfo();
|
||||
res.json(info);
|
||||
} catch (error) {
|
||||
logger.failure('WORKER', 'Failed to get branch status', {}, error as Error);
|
||||
res.status(500).json({ error: (error as Error).message });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /api/branch/switch - Switch to a different branch
|
||||
* Body: { branch: "main" | "beta/7.0" }
|
||||
*/
|
||||
private async handleSwitchBranch(req: Request, res: Response): Promise<void> {
|
||||
try {
|
||||
const { branch } = req.body;
|
||||
|
||||
if (!branch) {
|
||||
res.status(400).json({ success: false, error: 'Missing branch parameter' });
|
||||
return;
|
||||
}
|
||||
|
||||
// Validate branch name
|
||||
const allowedBranches = ['main', 'beta/7.0'];
|
||||
if (!allowedBranches.includes(branch)) {
|
||||
res.status(400).json({
|
||||
success: false,
|
||||
error: `Invalid branch. Allowed: ${allowedBranches.join(', ')}`
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
logger.info('WORKER', 'Branch switch requested', { branch });
|
||||
|
||||
const result = await switchBranch(branch);
|
||||
|
||||
if (result.success) {
|
||||
// Schedule worker restart after response is sent
|
||||
setTimeout(() => {
|
||||
logger.info('WORKER', 'Restarting worker after branch switch');
|
||||
process.exit(0); // PM2 will restart the worker
|
||||
}, 1000);
|
||||
}
|
||||
|
||||
res.json(result);
|
||||
} catch (error) {
|
||||
logger.failure('WORKER', 'Branch switch failed', {}, error as Error);
|
||||
res.status(500).json({ success: false, error: (error as Error).message });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /api/branch/update - Pull latest updates for current branch
|
||||
*/
|
||||
private async handleUpdateBranch(req: Request, res: Response): Promise<void> {
|
||||
try {
|
||||
logger.info('WORKER', 'Branch update requested');
|
||||
|
||||
const result = await pullUpdates();
|
||||
|
||||
if (result.success) {
|
||||
// Schedule worker restart after response is sent
|
||||
setTimeout(() => {
|
||||
logger.info('WORKER', 'Restarting worker after branch update');
|
||||
process.exit(0); // PM2 will restart the worker
|
||||
}, 1000);
|
||||
}
|
||||
|
||||
res.json(result);
|
||||
} catch (error) {
|
||||
logger.failure('WORKER', 'Branch update failed', {}, error as Error);
|
||||
res.status(500).json({ success: false, error: (error as Error).message });
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Search API Handlers (for skill-based search)
|
||||
// ============================================================================
|
||||
|
||||
// ============================================================================
|
||||
// Unified Search API Handlers (New Consolidated API)
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Unified search across all memory types (observations, sessions, prompts)
|
||||
* GET /api/search?query=...&format=index&limit=20
|
||||
*/
|
||||
private async handleUnifiedSearch(req: Request, res: Response): Promise<void> {
|
||||
try {
|
||||
const result = await this.mcpClient.callTool({
|
||||
name: 'search',
|
||||
arguments: req.query
|
||||
});
|
||||
res.json(result.content);
|
||||
} catch (error) {
|
||||
logger.failure('WORKER', 'Unified search failed', {}, error as Error);
|
||||
res.status(500).json({ error: (error as Error).message });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Unified timeline (anchor or query-based)
|
||||
* GET /api/timeline?anchor=123 OR GET /api/timeline?query=...
|
||||
*/
|
||||
private async handleUnifiedTimeline(req: Request, res: Response): Promise<void> {
|
||||
try {
|
||||
const result = await this.mcpClient.callTool({
|
||||
name: 'timeline',
|
||||
arguments: req.query
|
||||
});
|
||||
res.json(result.content);
|
||||
} catch (error) {
|
||||
logger.failure('WORKER', 'Unified timeline failed', {}, error as Error);
|
||||
res.status(500).json({ error: (error as Error).message });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Semantic shortcut for finding decision observations
|
||||
* GET /api/decisions?format=index&limit=20
|
||||
*/
|
||||
private async handleDecisions(req: Request, res: Response): Promise<void> {
|
||||
try {
|
||||
const result = await this.mcpClient.callTool({
|
||||
name: 'decisions',
|
||||
arguments: req.query
|
||||
});
|
||||
res.json(result.content);
|
||||
} catch (error) {
|
||||
logger.failure('WORKER', 'Decisions search failed', {}, error as Error);
|
||||
res.status(500).json({ error: (error as Error).message });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Semantic shortcut for finding change-related observations
|
||||
* GET /api/changes?format=index&limit=20
|
||||
*/
|
||||
private async handleChanges(req: Request, res: Response): Promise<void> {
|
||||
try {
|
||||
const result = await this.mcpClient.callTool({
|
||||
name: 'changes',
|
||||
arguments: req.query
|
||||
});
|
||||
res.json(result.content);
|
||||
} catch (error) {
|
||||
logger.failure('WORKER', 'Changes search failed', {}, error as Error);
|
||||
res.status(500).json({ error: (error as Error).message });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Semantic shortcut for finding "how it works" explanations
|
||||
* GET /api/how-it-works?format=index&limit=20
|
||||
*/
|
||||
private async handleHowItWorks(req: Request, res: Response): Promise<void> {
|
||||
try {
|
||||
const result = await this.mcpClient.callTool({
|
||||
name: 'how_it_works',
|
||||
arguments: req.query
|
||||
});
|
||||
res.json(result.content);
|
||||
} catch (error) {
|
||||
logger.failure('WORKER', 'How it works search failed', {}, error as Error);
|
||||
res.status(500).json({ error: (error as Error).message });
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Backward Compatibility API Handlers
|
||||
// All functionality available via /api/search with type/obs_type/concepts/files params
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Search observations (use /api/search?type=observations instead)
|
||||
* GET /api/search/observations?query=...&format=index&limit=20&project=...
|
||||
*/
|
||||
private async handleSearchObservations(req: Request, res: Response): Promise<void> {
|
||||
|
||||
@@ -19,6 +19,8 @@ export interface ActiveSession {
|
||||
generatorPromise: Promise<void> | null;
|
||||
lastPromptNumber: number;
|
||||
startTime: number;
|
||||
cumulativeInputTokens: number; // Track input tokens for discovery cost
|
||||
cumulativeOutputTokens: number; // Track output tokens for discovery cost
|
||||
}
|
||||
|
||||
export interface PendingMessage {
|
||||
|
||||
@@ -0,0 +1,247 @@
|
||||
/**
|
||||
* BranchManager: Git branch detection and switching for beta feature toggle
|
||||
*
|
||||
* Enables users to switch between stable (main) and beta branches via the UI.
|
||||
* The installed plugin at ~/.claude/plugins/marketplaces/thedotmack/ is a git repo.
|
||||
*/
|
||||
|
||||
import { execSync } from 'child_process';
|
||||
import { existsSync, unlinkSync } from 'fs';
|
||||
import { homedir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { logger } from '../../utils/logger.js';
|
||||
|
||||
const INSTALLED_PLUGIN_PATH = join(homedir(), '.claude', 'plugins', 'marketplaces', 'thedotmack');
|
||||
|
||||
export interface BranchInfo {
|
||||
branch: string | null;
|
||||
isBeta: boolean;
|
||||
isGitRepo: boolean;
|
||||
isDirty: boolean;
|
||||
canSwitch: boolean;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
export interface SwitchResult {
|
||||
success: boolean;
|
||||
branch?: string;
|
||||
message?: string;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute git command in installed plugin directory
|
||||
*/
|
||||
function execGit(command: string): string {
|
||||
return execSync(`git ${command}`, {
|
||||
cwd: INSTALLED_PLUGIN_PATH,
|
||||
encoding: 'utf-8',
|
||||
timeout: 30000
|
||||
}).trim();
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute shell command in installed plugin directory
|
||||
*/
|
||||
function execShell(command: string, timeoutMs: number = 60000): string {
|
||||
return execSync(command, {
|
||||
cwd: INSTALLED_PLUGIN_PATH,
|
||||
encoding: 'utf-8',
|
||||
timeout: timeoutMs
|
||||
}).trim();
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current branch information
|
||||
*/
|
||||
export function getBranchInfo(): BranchInfo {
|
||||
// Check if git repo exists
|
||||
const gitDir = join(INSTALLED_PLUGIN_PATH, '.git');
|
||||
if (!existsSync(gitDir)) {
|
||||
return {
|
||||
branch: null,
|
||||
isBeta: false,
|
||||
isGitRepo: false,
|
||||
isDirty: false,
|
||||
canSwitch: false,
|
||||
error: 'Installed plugin is not a git repository'
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
// Get current branch
|
||||
const branch = execGit('rev-parse --abbrev-ref HEAD');
|
||||
|
||||
// Check if dirty (has uncommitted changes)
|
||||
const status = execGit('status --porcelain');
|
||||
const isDirty = status.length > 0;
|
||||
|
||||
// Determine if on beta branch
|
||||
const isBeta = branch.startsWith('beta');
|
||||
|
||||
return {
|
||||
branch,
|
||||
isBeta,
|
||||
isGitRepo: true,
|
||||
isDirty,
|
||||
canSwitch: true // We can always switch (will discard local changes)
|
||||
};
|
||||
} catch (error) {
|
||||
logger.error('BRANCH', 'Failed to get branch info', {}, error as Error);
|
||||
return {
|
||||
branch: null,
|
||||
isBeta: false,
|
||||
isGitRepo: true,
|
||||
isDirty: false,
|
||||
canSwitch: false,
|
||||
error: (error as Error).message
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Switch to a different branch
|
||||
*
|
||||
* Steps:
|
||||
* 1. Discard local changes (from rsync syncs)
|
||||
* 2. Fetch latest from origin
|
||||
* 3. Checkout target branch
|
||||
* 4. Pull latest
|
||||
* 5. Clear install marker and run npm install
|
||||
* 6. Restart worker (handled by caller after response)
|
||||
*/
|
||||
export async function switchBranch(targetBranch: string): Promise<SwitchResult> {
|
||||
const info = getBranchInfo();
|
||||
|
||||
if (!info.isGitRepo) {
|
||||
return {
|
||||
success: false,
|
||||
error: 'Installed plugin is not a git repository. Please reinstall.'
|
||||
};
|
||||
}
|
||||
|
||||
if (info.branch === targetBranch) {
|
||||
return {
|
||||
success: true,
|
||||
branch: targetBranch,
|
||||
message: `Already on branch ${targetBranch}`
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
logger.info('BRANCH', 'Starting branch switch', {
|
||||
from: info.branch,
|
||||
to: targetBranch
|
||||
});
|
||||
|
||||
// 1. Discard local changes (safe - user data is at ~/.claude-mem/)
|
||||
logger.debug('BRANCH', 'Discarding local changes');
|
||||
execGit('checkout -- .');
|
||||
execGit('clean -fd'); // Remove untracked files too
|
||||
|
||||
// 2. Fetch latest
|
||||
logger.debug('BRANCH', 'Fetching from origin');
|
||||
execGit('fetch origin');
|
||||
|
||||
// 3. Checkout target branch
|
||||
logger.debug('BRANCH', 'Checking out branch', { branch: targetBranch });
|
||||
try {
|
||||
execGit(`checkout ${targetBranch}`);
|
||||
} catch {
|
||||
// Branch might not exist locally, try tracking remote
|
||||
execGit(`checkout -b ${targetBranch} origin/${targetBranch}`);
|
||||
}
|
||||
|
||||
// 4. Pull latest
|
||||
logger.debug('BRANCH', 'Pulling latest');
|
||||
execGit(`pull origin ${targetBranch}`);
|
||||
|
||||
// 5. Clear install marker and run npm install
|
||||
const installMarker = join(INSTALLED_PLUGIN_PATH, '.install-version');
|
||||
if (existsSync(installMarker)) {
|
||||
unlinkSync(installMarker);
|
||||
}
|
||||
|
||||
logger.debug('BRANCH', 'Running npm install');
|
||||
execShell('npm install', 120000); // 2 minute timeout for npm
|
||||
|
||||
logger.success('BRANCH', 'Branch switch complete', {
|
||||
branch: targetBranch
|
||||
});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
branch: targetBranch,
|
||||
message: `Switched to ${targetBranch}. Worker will restart automatically.`
|
||||
};
|
||||
} catch (error) {
|
||||
logger.error('BRANCH', 'Branch switch failed', { targetBranch }, error as Error);
|
||||
|
||||
// Try to recover by checking out original branch
|
||||
try {
|
||||
if (info.branch) {
|
||||
execGit(`checkout ${info.branch}`);
|
||||
}
|
||||
} catch {
|
||||
// Recovery failed, user needs manual intervention
|
||||
}
|
||||
|
||||
return {
|
||||
success: false,
|
||||
error: `Branch switch failed: ${(error as Error).message}`
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Pull latest updates for current branch
|
||||
*/
|
||||
export async function pullUpdates(): Promise<SwitchResult> {
|
||||
const info = getBranchInfo();
|
||||
|
||||
if (!info.isGitRepo || !info.branch) {
|
||||
return {
|
||||
success: false,
|
||||
error: 'Cannot pull updates: not a git repository'
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
logger.info('BRANCH', 'Pulling updates', { branch: info.branch });
|
||||
|
||||
// Discard local changes first
|
||||
execGit('checkout -- .');
|
||||
|
||||
// Fetch and pull
|
||||
execGit('fetch origin');
|
||||
execGit(`pull origin ${info.branch}`);
|
||||
|
||||
// Clear install marker and reinstall
|
||||
const installMarker = join(INSTALLED_PLUGIN_PATH, '.install-version');
|
||||
if (existsSync(installMarker)) {
|
||||
unlinkSync(installMarker);
|
||||
}
|
||||
execShell('npm install', 120000);
|
||||
|
||||
logger.success('BRANCH', 'Updates pulled', { branch: info.branch });
|
||||
|
||||
return {
|
||||
success: true,
|
||||
branch: info.branch,
|
||||
message: `Updated ${info.branch}. Worker will restart automatically.`
|
||||
};
|
||||
} catch (error) {
|
||||
logger.error('BRANCH', 'Pull failed', {}, error as Error);
|
||||
return {
|
||||
success: false,
|
||||
error: `Pull failed: ${(error as Error).message}`
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get installed plugin path (for external use)
|
||||
*/
|
||||
export function getInstalledPluginPath(): string {
|
||||
return INSTALLED_PLUGIN_PATH;
|
||||
}
|
||||
@@ -30,16 +30,28 @@ export class DatabaseManager {
|
||||
// Initialize ChromaSync
|
||||
this.chromaSync = new ChromaSync('claude-mem');
|
||||
|
||||
// Start background backfill (fire-and-forget)
|
||||
this.chromaSync.ensureBackfilled().catch(() => {});
|
||||
// Start background backfill (fire-and-forget, with error logging)
|
||||
this.chromaSync.ensureBackfilled().catch((error) => {
|
||||
logger.error('DB', 'Chroma backfill failed (non-fatal)', {}, error);
|
||||
});
|
||||
|
||||
logger.info('DB', 'Database initialized');
|
||||
}
|
||||
|
||||
/**
|
||||
* Close database connection
|
||||
* Close database connection and cleanup all resources
|
||||
*/
|
||||
async close(): Promise<void> {
|
||||
// Close ChromaSync first (terminates uvx/python processes)
|
||||
if (this.chromaSync) {
|
||||
try {
|
||||
await this.chromaSync.close();
|
||||
this.chromaSync = null;
|
||||
} catch (error) {
|
||||
logger.error('DB', 'Failed to close ChromaSync', {}, error as Error);
|
||||
}
|
||||
}
|
||||
|
||||
if (this.sessionStore) {
|
||||
this.sessionStore.close();
|
||||
this.sessionStore = null;
|
||||
|
||||
@@ -85,6 +85,34 @@ export class SDKAgent {
|
||||
|
||||
const responseSize = textContent.length;
|
||||
|
||||
// Capture token state BEFORE updating (for delta calculation)
|
||||
const tokensBeforeResponse = session.cumulativeInputTokens + session.cumulativeOutputTokens;
|
||||
|
||||
// Extract and track token usage
|
||||
const usage = message.message.usage;
|
||||
if (usage) {
|
||||
session.cumulativeInputTokens += usage.input_tokens || 0;
|
||||
session.cumulativeOutputTokens += usage.output_tokens || 0;
|
||||
|
||||
// Cache creation counts as discovery, cache read doesn't
|
||||
if (usage.cache_creation_input_tokens) {
|
||||
session.cumulativeInputTokens += usage.cache_creation_input_tokens;
|
||||
}
|
||||
|
||||
logger.debug('SDK', 'Token usage captured', {
|
||||
sessionId: session.sessionDbId,
|
||||
inputTokens: usage.input_tokens,
|
||||
outputTokens: usage.output_tokens,
|
||||
cacheCreation: usage.cache_creation_input_tokens || 0,
|
||||
cacheRead: usage.cache_read_input_tokens || 0,
|
||||
cumulativeInput: session.cumulativeInputTokens,
|
||||
cumulativeOutput: session.cumulativeOutputTokens
|
||||
});
|
||||
}
|
||||
|
||||
// Calculate discovery tokens (delta for this response only)
|
||||
const discoveryTokens = (session.cumulativeInputTokens + session.cumulativeOutputTokens) - tokensBeforeResponse;
|
||||
|
||||
// Only log non-empty responses (filter out noise)
|
||||
if (responseSize > 0) {
|
||||
const truncatedResponse = responseSize > 100
|
||||
@@ -95,8 +123,8 @@ export class SDKAgent {
|
||||
promptNumber: session.lastPromptNumber
|
||||
}, truncatedResponse);
|
||||
|
||||
// Parse and process response
|
||||
await this.processSDKResponse(session, textContent, worker);
|
||||
// Parse and process response with discovery token delta
|
||||
await this.processSDKResponse(session, textContent, worker, discoveryTokens);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -218,8 +246,9 @@ export class SDKAgent {
|
||||
|
||||
/**
|
||||
* Process SDK response text (parse XML, save to database, sync to Chroma)
|
||||
* @param discoveryTokens - Token cost for discovering this response (delta, not cumulative)
|
||||
*/
|
||||
private async processSDKResponse(session: ActiveSession, text: string, worker?: any): Promise<void> {
|
||||
private async processSDKResponse(session: ActiveSession, text: string, worker: any | undefined, discoveryTokens: number): Promise<void> {
|
||||
// Parse observations
|
||||
const observations = parseObservations(text, session.claudeSessionId);
|
||||
|
||||
@@ -229,7 +258,8 @@ export class SDKAgent {
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
obs,
|
||||
session.lastPromptNumber
|
||||
session.lastPromptNumber,
|
||||
discoveryTokens
|
||||
);
|
||||
|
||||
// Log observation details
|
||||
@@ -253,7 +283,8 @@ export class SDKAgent {
|
||||
session.project,
|
||||
obs,
|
||||
session.lastPromptNumber,
|
||||
createdAtEpoch
|
||||
createdAtEpoch,
|
||||
discoveryTokens
|
||||
).then(() => {
|
||||
const chromaDuration = Date.now() - chromaStart;
|
||||
logger.debug('CHROMA', 'Observation synced', {
|
||||
@@ -305,7 +336,8 @@ export class SDKAgent {
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
summary,
|
||||
session.lastPromptNumber
|
||||
session.lastPromptNumber,
|
||||
discoveryTokens
|
||||
);
|
||||
|
||||
// Log summary details
|
||||
@@ -326,7 +358,8 @@ export class SDKAgent {
|
||||
session.project,
|
||||
summary,
|
||||
session.lastPromptNumber,
|
||||
createdAtEpoch
|
||||
createdAtEpoch,
|
||||
discoveryTokens
|
||||
).then(() => {
|
||||
const chromaDuration = Date.now() - chromaStart;
|
||||
logger.debug('CHROMA', 'Summary synced', {
|
||||
|
||||
@@ -38,6 +38,19 @@ export class SessionManager {
|
||||
// Check if already active
|
||||
let session = this.sessions.get(sessionDbId);
|
||||
if (session) {
|
||||
// Refresh project from database in case it was updated by new-hook
|
||||
// This fixes the bug where sessions created with empty project get updated
|
||||
// in the database but the in-memory session still has the stale empty value
|
||||
const dbSession = this.dbManager.getSessionById(sessionDbId);
|
||||
if (dbSession.project && dbSession.project !== session.project) {
|
||||
silentDebug('[SessionManager] Updating project from database', {
|
||||
sessionDbId,
|
||||
oldProject: session.project,
|
||||
newProject: dbSession.project
|
||||
});
|
||||
session.project = dbSession.project;
|
||||
}
|
||||
|
||||
// Update userPrompt for continuation prompts
|
||||
if (currentUserPrompt) {
|
||||
silentDebug('[SessionManager] Updating userPrompt for continuation', {
|
||||
@@ -89,7 +102,9 @@ export class SessionManager {
|
||||
abortController: new AbortController(),
|
||||
generatorPromise: null,
|
||||
lastPromptNumber: promptNumber || this.dbManager.getSessionStore().getPromptCounter(sessionDbId),
|
||||
startTime: Date.now()
|
||||
startTime: Date.now(),
|
||||
cumulativeInputTokens: 0,
|
||||
cumulativeOutputTokens: 0
|
||||
};
|
||||
|
||||
this.sessions.set(sessionDbId, session);
|
||||
@@ -254,6 +269,23 @@ export class SessionManager {
|
||||
return total;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get total active work (queued + currently processing)
|
||||
* Counts both pending messages and items actively being processed by SDK agents
|
||||
*/
|
||||
getTotalActiveWork(): number {
|
||||
let total = 0;
|
||||
for (const session of this.sessions.values()) {
|
||||
// Count queued messages
|
||||
total += session.pendingMessages.length;
|
||||
// Count currently processing item (1 per active generator)
|
||||
if (session.generatorPromise !== null) {
|
||||
total += 1;
|
||||
}
|
||||
}
|
||||
return total;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if any session is actively processing (has pending messages OR active generator)
|
||||
* Used for activity indicator to prevent spinner from stopping while SDK is processing
|
||||
|
||||
@@ -1,9 +1,13 @@
|
||||
import path from "path";
|
||||
import { homedir } from "os";
|
||||
import { existsSync, readFileSync } from "fs";
|
||||
import { execSync } from "child_process";
|
||||
import { getPackageRoot } from "./paths.js";
|
||||
|
||||
// Named constants for health checks
|
||||
const HEALTH_CHECK_TIMEOUT_MS = 100;
|
||||
const WORKER_STARTUP_WAIT_MS = 500;
|
||||
const WORKER_STARTUP_RETRIES = 10;
|
||||
|
||||
/**
|
||||
* Get the worker port number
|
||||
@@ -38,20 +42,61 @@ async function isWorkerHealthy(): Promise<boolean> {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Start the worker using PM2
|
||||
*/
|
||||
async function startWorker(): Promise<boolean> {
|
||||
try {
|
||||
// Find the ecosystem config file (built version in plugin/)
|
||||
const pluginRoot = getPackageRoot();
|
||||
const ecosystemPath = path.join(pluginRoot, 'ecosystem.config.cjs');
|
||||
|
||||
if (!existsSync(ecosystemPath)) {
|
||||
throw new Error(`Ecosystem config not found at ${ecosystemPath}`);
|
||||
}
|
||||
|
||||
// Start using PM2 with the ecosystem config
|
||||
// CRITICAL: Must set cwd to pluginRoot so PM2 starts from marketplace directory
|
||||
execSync(`pm2 start "${ecosystemPath}"`, {
|
||||
cwd: pluginRoot,
|
||||
stdio: 'pipe',
|
||||
encoding: 'utf-8'
|
||||
});
|
||||
|
||||
// Wait for worker to become healthy
|
||||
for (let i = 0; i < WORKER_STARTUP_RETRIES; i++) {
|
||||
await new Promise(resolve => setTimeout(resolve, WORKER_STARTUP_WAIT_MS));
|
||||
if (await isWorkerHealthy()) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
} catch (error) {
|
||||
// Failed to start worker
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure worker service is running
|
||||
* Checks health and fails with instructions if not healthy
|
||||
* PM2's watch mode handles auto-restarts automatically
|
||||
* Checks health and auto-starts if not running
|
||||
*/
|
||||
export async function ensureWorkerRunning(): Promise<void> {
|
||||
// Check if already healthy
|
||||
if (await isWorkerHealthy()) {
|
||||
return;
|
||||
}
|
||||
|
||||
const port = getWorkerPort();
|
||||
throw new Error(
|
||||
`Worker service is not responding on port ${port}.\n\n` +
|
||||
`If you just updated the plugin, PM2's watch mode should restart automatically.\n` +
|
||||
`If the problem persists, run: pm2 restart claude-mem-worker`
|
||||
);
|
||||
// Try to start the worker
|
||||
const started = await startWorker();
|
||||
|
||||
if (!started) {
|
||||
const port = getWorkerPort();
|
||||
throw new Error(
|
||||
`Worker service failed to start on port ${port}.\n\n` +
|
||||
`Try manually running: pm2 start ecosystem.config.cjs\n` +
|
||||
`Or restart: pm2 restart claude-mem-worker`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -376,6 +376,36 @@
|
||||
animation: spin 1.5s linear infinite;
|
||||
}
|
||||
|
||||
.queue-bubble {
|
||||
position: absolute;
|
||||
top: -8px;
|
||||
right: -8px;
|
||||
background: var(--color-accent-primary);
|
||||
color: var(--color-text-button);
|
||||
font-size: 10px;
|
||||
font-weight: 600;
|
||||
font-family: 'Monaspace Radon', monospace;
|
||||
min-width: 18px;
|
||||
height: 18px;
|
||||
border-radius: 9px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
padding: 0 5px;
|
||||
box-shadow: 0 2px 4px rgba(0, 0, 0, 0.2);
|
||||
animation: pulse 2s ease-in-out infinite;
|
||||
z-index: 10;
|
||||
}
|
||||
|
||||
@keyframes pulse {
|
||||
0%, 100% {
|
||||
transform: scale(1);
|
||||
}
|
||||
50% {
|
||||
transform: scale(1.1);
|
||||
}
|
||||
}
|
||||
|
||||
.logo-text {
|
||||
font-family: 'Monaspace Radon', monospace;
|
||||
font-weight: 100;
|
||||
|
||||
@@ -17,7 +17,7 @@ export function App() {
|
||||
const [paginatedSummaries, setPaginatedSummaries] = useState<Summary[]>([]);
|
||||
const [paginatedPrompts, setPaginatedPrompts] = useState<UserPrompt[]>([]);
|
||||
|
||||
const { observations, summaries, prompts, projects, isProcessing, isConnected } = useSSE();
|
||||
const { observations, summaries, prompts, projects, isProcessing, queueDepth, isConnected } = useSSE();
|
||||
const { settings, saveSettings, isSaving, saveStatus } = useSettings();
|
||||
const { stats, refreshStats } = useStats();
|
||||
const { preference, resolvedTheme, setThemePreference } = useTheme();
|
||||
@@ -96,6 +96,7 @@ export function App() {
|
||||
onSettingsToggle={toggleSidebar}
|
||||
sidebarOpen={sidebarOpen}
|
||||
isProcessing={isProcessing}
|
||||
queueDepth={queueDepth}
|
||||
themePreference={preference}
|
||||
onThemeChange={setThemePreference}
|
||||
/>
|
||||
|
||||
@@ -10,6 +10,7 @@ interface HeaderProps {
|
||||
onSettingsToggle: () => void;
|
||||
sidebarOpen: boolean;
|
||||
isProcessing: boolean;
|
||||
queueDepth: number;
|
||||
themePreference: ThemePreference;
|
||||
onThemeChange: (theme: ThemePreference) => void;
|
||||
}
|
||||
@@ -22,13 +23,21 @@ export function Header({
|
||||
onSettingsToggle,
|
||||
sidebarOpen,
|
||||
isProcessing,
|
||||
queueDepth,
|
||||
themePreference,
|
||||
onThemeChange
|
||||
}: HeaderProps) {
|
||||
return (
|
||||
<div className="header">
|
||||
<h1>
|
||||
<img src="claude-mem-logomark.webp" alt="" className={`logomark ${isProcessing ? 'spinning' : ''}`} />
|
||||
<div style={{ position: 'relative', display: 'inline-block' }}>
|
||||
<img src="claude-mem-logomark.webp" alt="" className={`logomark ${isProcessing ? 'spinning' : ''}`} />
|
||||
{queueDepth > 0 && (
|
||||
<div className="queue-bubble">
|
||||
{queueDepth}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
<span className="logo-text">claude-mem</span>
|
||||
</h1>
|
||||
<div className="status">
|
||||
|
||||
@@ -26,6 +26,19 @@ export function Sidebar({ isOpen, settings, stats, isSaving, saveStatus, isConne
|
||||
const [mcpToggling, setMcpToggling] = useState(false);
|
||||
const [mcpStatus, setMcpStatus] = useState('');
|
||||
|
||||
// Branch switching state
|
||||
interface BranchInfo {
|
||||
branch: string | null;
|
||||
isBeta: boolean;
|
||||
isGitRepo: boolean;
|
||||
isDirty: boolean;
|
||||
canSwitch: boolean;
|
||||
error?: string;
|
||||
}
|
||||
const [branchInfo, setBranchInfo] = useState<BranchInfo | null>(null);
|
||||
const [branchSwitching, setBranchSwitching] = useState(false);
|
||||
const [branchStatus, setBranchStatus] = useState('');
|
||||
|
||||
// Update settings form state when settings change
|
||||
useEffect(() => {
|
||||
setModel(settings.CLAUDE_MEM_MODEL || DEFAULT_SETTINGS.CLAUDE_MEM_MODEL);
|
||||
@@ -41,6 +54,14 @@ export function Sidebar({ isOpen, settings, stats, isSaving, saveStatus, isConne
|
||||
.catch(error => console.error('Failed to load MCP status:', error));
|
||||
}, []);
|
||||
|
||||
// Fetch branch status on mount
|
||||
useEffect(() => {
|
||||
fetch('/api/branch/status')
|
||||
.then(res => res.json())
|
||||
.then(data => setBranchInfo(data))
|
||||
.catch(error => console.error('Failed to load branch status:', error));
|
||||
}, []);
|
||||
|
||||
// Refresh stats when sidebar opens
|
||||
useEffect(() => {
|
||||
if (isOpen) {
|
||||
@@ -85,6 +106,67 @@ export function Sidebar({ isOpen, settings, stats, isSaving, saveStatus, isConne
|
||||
}
|
||||
};
|
||||
|
||||
const handleBranchSwitch = async (targetBranch: string) => {
|
||||
setBranchSwitching(true);
|
||||
setBranchStatus('Switching branches...');
|
||||
|
||||
try {
|
||||
const response = await fetch('/api/branch/switch', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ branch: targetBranch })
|
||||
});
|
||||
|
||||
const result = await response.json();
|
||||
|
||||
if (result.success) {
|
||||
setBranchStatus(`✓ ${result.message}`);
|
||||
// Worker will restart, page will refresh
|
||||
setTimeout(() => {
|
||||
setBranchStatus('Restarting worker...');
|
||||
}, 1000);
|
||||
} else {
|
||||
setBranchStatus(`✗ Error: ${result.error}`);
|
||||
setTimeout(() => setBranchStatus(''), 5000);
|
||||
setBranchSwitching(false);
|
||||
}
|
||||
} catch (error) {
|
||||
setBranchStatus(`✗ Error: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
setTimeout(() => setBranchStatus(''), 5000);
|
||||
setBranchSwitching(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleBranchUpdate = async () => {
|
||||
setBranchSwitching(true);
|
||||
setBranchStatus('Checking for updates...');
|
||||
|
||||
try {
|
||||
const response = await fetch('/api/branch/update', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' }
|
||||
});
|
||||
|
||||
const result = await response.json();
|
||||
|
||||
if (result.success) {
|
||||
setBranchStatus(`✓ ${result.message}`);
|
||||
// Worker will restart, page will refresh
|
||||
setTimeout(() => {
|
||||
setBranchStatus('Restarting worker...');
|
||||
}, 1000);
|
||||
} else {
|
||||
setBranchStatus(`✗ Error: ${result.error}`);
|
||||
setTimeout(() => setBranchStatus(''), 5000);
|
||||
setBranchSwitching(false);
|
||||
}
|
||||
} catch (error) {
|
||||
setBranchStatus(`✗ Error: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
setTimeout(() => setBranchStatus(''), 5000);
|
||||
setBranchSwitching(false);
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<div className={`sidebar ${isOpen ? 'open' : ''}`}>
|
||||
<div className="sidebar-header">
|
||||
@@ -193,6 +275,94 @@ export function Sidebar({ isOpen, settings, stats, isSaving, saveStatus, isConne
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="settings-section">
|
||||
<h3>Version Channel</h3>
|
||||
<div className="form-group">
|
||||
{branchInfo ? (
|
||||
<>
|
||||
<div style={{ display: 'flex', alignItems: 'center', gap: '8px', marginBottom: '8px' }}>
|
||||
<span style={{
|
||||
padding: '2px 8px',
|
||||
borderRadius: '4px',
|
||||
fontSize: '11px',
|
||||
fontWeight: 600,
|
||||
background: branchInfo.isBeta ? '#6b4500' : '#1a4d1a',
|
||||
color: branchInfo.isBeta ? '#ffb84d' : '#4ade80',
|
||||
border: `1px solid ${branchInfo.isBeta ? '#ffb84d' : '#4ade80'}`
|
||||
}}>
|
||||
{branchInfo.isBeta ? 'Beta' : 'Stable'}
|
||||
</span>
|
||||
<span style={{ fontSize: '12px', opacity: 0.7 }}>
|
||||
{branchInfo.branch || 'main'}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{branchInfo.isBeta ? (
|
||||
<>
|
||||
<div className="setting-description" style={{ marginBottom: '12px' }}>
|
||||
You're running the beta with Endless Mode. Your memory data is preserved when switching versions.
|
||||
</div>
|
||||
<div style={{ display: 'flex', gap: '8px', flexWrap: 'wrap' }}>
|
||||
<button
|
||||
onClick={() => handleBranchSwitch('main')}
|
||||
disabled={branchSwitching}
|
||||
style={{
|
||||
background: '#2a2a2a',
|
||||
border: '1px solid #404040',
|
||||
padding: '8px 16px',
|
||||
cursor: branchSwitching ? 'not-allowed' : 'pointer',
|
||||
opacity: branchSwitching ? 0.5 : 1
|
||||
}}
|
||||
>
|
||||
Switch to Stable
|
||||
</button>
|
||||
<button
|
||||
onClick={handleBranchUpdate}
|
||||
disabled={branchSwitching}
|
||||
style={{
|
||||
background: '#2a2a2a',
|
||||
border: '1px solid #404040',
|
||||
padding: '8px 16px',
|
||||
cursor: branchSwitching ? 'not-allowed' : 'pointer',
|
||||
opacity: branchSwitching ? 0.5 : 1
|
||||
}}
|
||||
>
|
||||
Check for Updates
|
||||
</button>
|
||||
</div>
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<div className="setting-description" style={{ marginBottom: '12px' }}>
|
||||
Try the beta to access experimental features like Endless Mode. Your memory data is preserved when switching.
|
||||
</div>
|
||||
<button
|
||||
onClick={() => handleBranchSwitch('beta/7.0')}
|
||||
disabled={branchSwitching}
|
||||
style={{
|
||||
background: '#4a3500',
|
||||
border: '1px solid #ffb84d',
|
||||
color: '#ffb84d',
|
||||
padding: '8px 16px',
|
||||
cursor: branchSwitching ? 'not-allowed' : 'pointer',
|
||||
opacity: branchSwitching ? 0.5 : 1
|
||||
}}
|
||||
>
|
||||
Try Beta (Endless Mode)
|
||||
</button>
|
||||
</>
|
||||
)}
|
||||
|
||||
{branchStatus && (
|
||||
<div className="save-status" style={{ marginTop: '8px' }}>{branchStatus}</div>
|
||||
)}
|
||||
</>
|
||||
) : (
|
||||
<div style={{ fontSize: '12px', opacity: 0.5 }}>Loading branch info...</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="settings-section">
|
||||
<h3>Worker Stats</h3>
|
||||
<div className="stats-grid">
|
||||
|
||||
@@ -10,6 +10,7 @@ export function useSSE() {
|
||||
const [projects, setProjects] = useState<string[]>([]);
|
||||
const [isConnected, setIsConnected] = useState(false);
|
||||
const [isProcessing, setIsProcessing] = useState(false);
|
||||
const [queueDepth, setQueueDepth] = useState(0);
|
||||
const eventSourceRef = useRef<EventSource | null>(null);
|
||||
const reconnectTimeoutRef = useRef<NodeJS.Timeout>();
|
||||
|
||||
@@ -83,8 +84,9 @@ export function useSSE() {
|
||||
|
||||
case 'processing_status':
|
||||
if (typeof data.isProcessing === 'boolean') {
|
||||
console.log('[SSE] Processing status:', data.isProcessing);
|
||||
console.log('[SSE] Processing status:', data.isProcessing, 'Queue depth:', data.queueDepth);
|
||||
setIsProcessing(data.isProcessing);
|
||||
setQueueDepth(data.queueDepth || 0);
|
||||
}
|
||||
break;
|
||||
}
|
||||
@@ -107,5 +109,5 @@ export function useSSE() {
|
||||
};
|
||||
}, []);
|
||||
|
||||
return { observations, summaries, prompts, projects, isProcessing, isConnected };
|
||||
return { observations, summaries, prompts, projects, isProcessing, queueDepth, isConnected };
|
||||
}
|
||||
|
||||
Executable
+86
@@ -0,0 +1,86 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Comprehensive Search API Test Suite
|
||||
# Tests all endpoints and parameter combinations
|
||||
|
||||
API_URL="http://localhost:37777"
|
||||
RESULTS_DIR="test-results"
|
||||
|
||||
echo "🔍 Starting comprehensive search API tests..."
|
||||
echo ""
|
||||
|
||||
# SEMANTIC QUERIES - Understanding how things work
|
||||
echo "📚 Running semantic queries..."
|
||||
curl -s "$API_URL/api/search?type=observations&query=worker%20service%20startup&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-01-worker-service-startup.json"
|
||||
curl -s "$API_URL/api/search?type=observations&query=SQLite%20FTS5%20implementation&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-02-sqlite-fts5-implementation.json"
|
||||
curl -s "$API_URL/api/search?type=observations&query=hook%20lifecycle%20flow&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-03-hook-lifecycle-flow.json"
|
||||
curl -s "$API_URL/api/search?type=observations&query=build%20pipeline%20process&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-04-build-pipeline-process.json"
|
||||
echo "✅ Semantic queries complete (4 tests)"
|
||||
|
||||
# DECISION QUERIES - Architectural choices
|
||||
echo "⚖️ Running decision queries..."
|
||||
curl -s "$API_URL/api/search?type=observations&obs_type=decision&query=PM2%20instead%20of%20direct%20process&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-05-pm2-decision.json"
|
||||
curl -s "$API_URL/api/search?type=observations&obs_type=decision&query=search%20architecture%20guidelines&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-06-search-architecture-decision.json"
|
||||
curl -s "$API_URL/api/search?type=observations&obs_type=decision&query=MCP%20as%20DRY%20source&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-07-mcp-dry-decision.json"
|
||||
echo "✅ Decision queries complete (3 tests)"
|
||||
|
||||
# TROUBLESHOOTING QUERIES - Finding bugfixes
|
||||
echo "🔴 Running troubleshooting queries..."
|
||||
curl -s "$API_URL/api/search?type=observations&obs_type=bugfix&query=worker%20service%20debugging&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-08-worker-debugging.json"
|
||||
curl -s "$API_URL/api/search?type=observations&query=hook%20timeout%20problems&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-09-hook-timeout.json"
|
||||
curl -s "$API_URL/api/search?type=observations&query=database%20migration%20issues&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-10-database-migration.json"
|
||||
echo "✅ Troubleshooting queries complete (3 tests)"
|
||||
|
||||
# FILE-SPECIFIC QUERIES - Tracking file changes
|
||||
echo "📁 Running file-specific queries..."
|
||||
curl -s "$API_URL/api/search?type=observations&files=search-server.ts&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-11-search-server-changes.json"
|
||||
curl -s "$API_URL/api/search?type=observations&files=context-hook&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-12-context-hook-changes.json"
|
||||
curl -s "$API_URL/api/search?type=observations&files=worker-service&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-13-worker-service-changes.json"
|
||||
echo "✅ File-specific queries complete (3 tests)"
|
||||
|
||||
# CONCEPT-BASED QUERIES - Patterns, gotchas, discoveries
|
||||
echo "🏷️ Running concept-based queries..."
|
||||
curl -s "$API_URL/api/search?type=observations&concepts=pattern&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-14-patterns.json"
|
||||
curl -s "$API_URL/api/search?type=observations&concepts=gotcha&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-15-gotchas.json"
|
||||
curl -s "$API_URL/api/search?type=observations&concepts=discovery&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-16-discoveries.json"
|
||||
echo "✅ Concept-based queries complete (3 tests)"
|
||||
|
||||
# TYPE-FILTERED QUERIES - Bugfixes, features, decisions
|
||||
echo "🔖 Running type-filtered queries..."
|
||||
curl -s "$API_URL/api/search?type=observations&obs_type=bugfix&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-17-all-bugfixes.json"
|
||||
curl -s "$API_URL/api/search?type=observations&obs_type=feature&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-18-all-features.json"
|
||||
curl -s "$API_URL/api/search?type=observations&obs_type=decision&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-19-all-decisions.json"
|
||||
echo "✅ Type-filtered queries complete (3 tests)"
|
||||
|
||||
# SESSION QUERIES - Testing session search
|
||||
echo "📝 Running session queries..."
|
||||
curl -s "$API_URL/api/search?type=sessions&query=search%20architecture&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-20-session-search.json"
|
||||
echo "✅ Session queries complete (1 test)"
|
||||
|
||||
# USER PROMPT QUERIES - Testing prompt search
|
||||
echo "💬 Running user prompt queries..."
|
||||
curl -s "$API_URL/api/search?type=prompts&query=build%20and%20deploy&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-21-prompt-search.json"
|
||||
echo "✅ User prompt queries complete (1 test)"
|
||||
|
||||
# DEDICATED ENDPOINTS - Timeline and semantic shortcuts
|
||||
echo "🎯 Running dedicated endpoint tests..."
|
||||
curl -s "$API_URL/api/decisions?format=full&limit=5" > "$RESULTS_DIR/test-22-decisions-endpoint.json"
|
||||
curl -s "$API_URL/api/changes?format=full&limit=5" > "$RESULTS_DIR/test-23-changes-endpoint.json"
|
||||
curl -s "$API_URL/api/how-it-works?format=full&limit=5" > "$RESULTS_DIR/test-24-how-it-works-endpoint.json"
|
||||
curl -s "$API_URL/api/contextualize?format=full" > "$RESULTS_DIR/test-25-contextualize-endpoint.json"
|
||||
echo "✅ Dedicated endpoint tests complete (4 tests)"
|
||||
|
||||
# TIMELINE QUERY - Get context around a specific observation
|
||||
echo "⏱️ Running timeline query..."
|
||||
curl -s "$API_URL/api/timeline?anchor=10630&depth_before=3&depth_after=3&format=full" > "$RESULTS_DIR/test-26-timeline-around-observation.json"
|
||||
echo "✅ Timeline query complete (1 test)"
|
||||
|
||||
# MULTI-PARAMETER COMBO - Test complex query combinations
|
||||
echo "🎛️ Running multi-parameter combination tests..."
|
||||
curl -s "$API_URL/api/search?type=observations&obs_type=decision&concepts=pattern&query=search&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-27-multi-param-combo.json"
|
||||
curl -s "$API_URL/api/search?type=observations&files=search-server&obs_type=feature&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-28-file-type-combo.json"
|
||||
echo "✅ Multi-parameter tests complete (2 tests)"
|
||||
|
||||
echo ""
|
||||
echo "✨ All tests complete! 28 total queries executed."
|
||||
echo "📊 Results saved to $RESULTS_DIR/"
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
|
||||
[{"type":"text","text":"## User Prompt #5\n*Source: claude-mem://user-prompt/2735*\n\nAre you ok? You seem a little excited and it's hard to do this project. Can you tell me what's going on?\n\n---\nDate: 11/17/2025, 9:38:06 PM\n\n---\n\n## User Prompt #2\n*Source: claude-mem://user-prompt/2719*\n\nedit the build script, we need it to always do this.\n\n---\nDate: 11/17/2025, 7:49:17 PM\n\n---\n\n## User Prompt #3\n*Source: claude-mem://user-prompt/2662*\n\nthose built files, the js, the cjs, if the source files don't have merge issues, then these are just way outdated build files.... is that the case?\n\n---\nDate: 11/17/2025, 3:03:13 PM\n\n---\n\n## User Prompt #12\n*Source: claude-mem://user-prompt/2651*\n\nI FUCKING DID BUILD AND SYNC AND DELETE. I would not have told you to fucking check pm2 info had I not done that\n\n---\nDate: 11/17/2025, 2:25:18 PM\n\n---\n\n## User Prompt #7\n*Source: claude-mem://user-prompt/2646*\n\nCan you fix it so that it runs from the marketplace folder so it doesn't break on EVERYONES SYSTEM WHO DOESNT FUCKING BUILD MANUALLY\n\n---\nDate: 11/17/2025, 2:19:29 PM"}]
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
|
||||
[{"type":"text","text":"# Project Context: claude-mem\n\nNo activity found for this project."}]
|
||||
@@ -0,0 +1 @@
|
||||
[{"type":"text","text":"# Timeline around anchor: 10630\n**Window:** 3 records before → 3 records after | **Items:** 3 (3 obs, 0 sessions, 0 prompts)\n\n**Legend:** 🎯 session-request | 🔴 bugfix | 🟣 feature | 🔄 refactor | ✅ change | 🔵 discovery | 🧠 decision\n\n### Nov 17, 2025\n\n**General**\n| ID | Time | T | Title | Tokens |\n|----|------|---|-------|--------|\n| #10756 | 11:50 PM | 🔴 | Fixed Incorrect Parameter Array in searchUserPrompts FTS5 Path | ~213 |\n| #10757 | ″ | 🔵 | Unified search handler implements Chroma-first with FTS5 fallback on zero results | ~224 |\n| #10758 | 11:51 PM | ✅ | Build and sync claude-mem plugin to marketplace location | ~189 |\n"}]
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,95 @@
|
||||
#!/bin/bash
|
||||
# Test script to verify process cleanup
|
||||
# This script tests that uvx/python processes are properly cleaned up
|
||||
|
||||
set -e
|
||||
|
||||
echo "=== Process Cleanup Test ==="
|
||||
echo ""
|
||||
|
||||
# Function to count uvx/python processes
|
||||
count_processes() {
|
||||
local count=$(ps aux | grep -E "(uvx|python.*chroma)" | grep -v grep | wc -l)
|
||||
echo "$count"
|
||||
}
|
||||
|
||||
# Initial count
|
||||
echo "1. Initial process count:"
|
||||
initial=$(count_processes)
|
||||
echo " uvx/python/chroma processes: $initial"
|
||||
echo ""
|
||||
|
||||
# Start a node process that creates ChromaSync
|
||||
echo "2. Starting test process that creates ChromaSync..."
|
||||
cat > /tmp/test-chroma-cleanup.mjs << 'EOF'
|
||||
import { ChromaSync } from './src/services/sync/ChromaSync.js';
|
||||
|
||||
const sync = new ChromaSync('test-project');
|
||||
|
||||
console.log('[TEST] ChromaSync created, connecting...');
|
||||
|
||||
// Try to connect (this spawns uvx process)
|
||||
try {
|
||||
await sync.ensureBackfilled();
|
||||
console.log('[TEST] Backfill started');
|
||||
} catch (error) {
|
||||
console.log('[TEST] Backfill failed (expected if no data):', error.message);
|
||||
}
|
||||
|
||||
// Wait a bit for process to start
|
||||
await new Promise(resolve => setTimeout(resolve, 2000));
|
||||
|
||||
const countBefore = parseInt(process.env.COUNT_BEFORE || '0');
|
||||
const countAfter = process.argv[2];
|
||||
|
||||
console.log('[TEST] Process count before:', countBefore);
|
||||
|
||||
// Close the sync (should terminate uvx process)
|
||||
console.log('[TEST] Closing ChromaSync...');
|
||||
await sync.close();
|
||||
|
||||
// Wait for process to terminate
|
||||
await new Promise(resolve => setTimeout(resolve, 1000));
|
||||
|
||||
console.log('[TEST] ChromaSync closed, process should be terminated');
|
||||
process.exit(0);
|
||||
EOF
|
||||
|
||||
# Run test
|
||||
COUNT_BEFORE=$initial node /tmp/test-chroma-cleanup.mjs 2>&1 &
|
||||
TEST_PID=$!
|
||||
|
||||
# Wait for process to spawn
|
||||
sleep 3
|
||||
|
||||
# Count during execution
|
||||
during=$(count_processes)
|
||||
echo " During execution: $during processes"
|
||||
echo ""
|
||||
|
||||
# Wait for test to complete
|
||||
wait $TEST_PID 2>/dev/null || true
|
||||
|
||||
# Wait a bit for cleanup
|
||||
sleep 2
|
||||
|
||||
# Final count
|
||||
echo "3. Final process count:"
|
||||
final=$(count_processes)
|
||||
echo " uvx/python/chroma processes: $final"
|
||||
echo ""
|
||||
|
||||
# Check if we leaked processes
|
||||
leaked=$((final - initial))
|
||||
if [ $leaked -gt 0 ]; then
|
||||
echo "❌ FAIL: Leaked $leaked process(es)"
|
||||
echo ""
|
||||
echo "Current processes:"
|
||||
ps aux | grep -E "(uvx|python.*chroma)" | grep -v grep
|
||||
exit 1
|
||||
else
|
||||
echo "✅ PASS: No process leaks detected"
|
||||
fi
|
||||
|
||||
# Cleanup
|
||||
rm -f /tmp/test-chroma-cleanup.mjs
|
||||
Reference in New Issue
Block a user