Compare commits
14 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Alex Newman
|
798dec972e | ||
|
Alex Newman
|
286343fef6 | ||
|
Alex Newman
|
9285826547 | ||
|
Alex Newman
|
ce3b3733fa | ||
|
Alex Newman
|
cf1c966409 | ||
|
Alex Newman
|
02fef487e7 | ||
|
Alex Newman
|
20d45006c0 | ||
|
Alex Newman
|
4f1cd309fd | ||
|
Copilot
|
c46e4a341a | ||
|
Alex Newman
|
60d5f8fbf1 | ||
|
Alex Newman
|
c0778bef00 | ||
|
Alex Newman
|
3cbc041c8b | ||
|
Copilot
|
0f96476987 | ||
|
Alex Newman
|
cd6f883020 |
@@ -10,7 +10,7 @@
|
||||
"plugins": [
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "6.0.2",
|
||||
"version": "6.0.6",
|
||||
"source": "./plugin",
|
||||
"description": "Persistent memory system for Claude Code - context compression across sessions"
|
||||
}
|
||||
|
||||
+57
-6
@@ -4,6 +4,57 @@ All notable changes to this project will be documented in this file.
|
||||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
||||
|
||||
## [6.0.5] - 2025-11-16
|
||||
|
||||
### Added
|
||||
- Automatic cleanup of orphaned MCP server processes on worker startup
|
||||
- Self-healing maintenance runs on every worker restart
|
||||
|
||||
### Improved
|
||||
- Removed manual cleanup notice from session context
|
||||
- Streamlined worker initialization process
|
||||
- Prevents orphaned process accumulation and resource leaks
|
||||
|
||||
### Fixed
|
||||
- Memory leaks from orphaned uvx/python processes are now prevented automatically
|
||||
- Workers self-heal on every restart without manual intervention
|
||||
|
||||
## [6.0.4] - 2025-11-17
|
||||
|
||||
**Patch Release**
|
||||
|
||||
Fixes memory leaks from orphaned uvx/python processes that could accumulate during ChromaDB operations.
|
||||
|
||||
**Changes:**
|
||||
- Fixed process cleanup in ChromaDB sync operations to prevent orphaned processes
|
||||
- Improved resource management for external process spawning
|
||||
|
||||
**Full Changelog:** https://github.com/thedotmack/claude-mem/compare/v6.0.3...v6.0.4
|
||||
|
||||
## [6.0.3] - 2025-11-16
|
||||
|
||||
## What's Changed
|
||||
|
||||
Documentation alignment release - merged PR #116 fixing hybrid search architecture documentation.
|
||||
|
||||
### Documentation Updates
|
||||
- Added comprehensive guide
|
||||
- Updated technical architecture documentation to reflect hybrid ChromaDB + SQLite + timeline context flow
|
||||
- Fixed skill operation guides to accurately describe semantic search capabilities
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v6.0.2...v6.0.3
|
||||
|
||||
## [6.0.2] - 2025-11-14
|
||||
|
||||
## Changes
|
||||
|
||||
- Updated user message hook with Claude-Mem community discussion link for better user engagement and support
|
||||
|
||||
## What's Changed
|
||||
- Enhanced startup context messaging with community connection information
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v6.0.1...v6.0.2
|
||||
|
||||
## [6.0.1] - 2025-11-14
|
||||
|
||||
## UI Enhancements
|
||||
@@ -1066,12 +1117,12 @@ None (patch version)
|
||||
|
||||
## [4.3.0] - 2025-10-25
|
||||
|
||||
## What's Changed
|
||||
* feat: Enhanced context hook with session observations and cross-platform improvements by @thedotmack in https://github.com/thedotmack/claude-mem/pull/25
|
||||
|
||||
## New Contributors
|
||||
* @thedotmack made their first contribution in https://github.com/thedotmack/claude-mem/pull/25
|
||||
|
||||
## What's Changed
|
||||
* feat: Enhanced context hook with session observations and cross-platform improvements by @thedotmack in https://github.com/thedotmack/claude-mem/pull/25
|
||||
|
||||
## New Contributors
|
||||
* @thedotmack made their first contribution in https://github.com/thedotmack/claude-mem/pull/25
|
||||
|
||||
**Full Changelog**: https://github.com/thedotmack/claude-mem/compare/v4.2.11...v4.3.0
|
||||
|
||||
## [4.2.10] - 2025-10-25
|
||||
|
||||
@@ -6,7 +6,7 @@ Claude-mem is a Claude Code plugin providing persistent memory across sessions.
|
||||
|
||||
**Your Role**: You are working on the plugin itself. When users interact with Claude Code with this plugin installed, your observations get captured and become their persistent memory.
|
||||
|
||||
**Current Version**: 6.0.2
|
||||
**Current Version**: 6.0.6
|
||||
|
||||
## IMPORTANT: Skills Are Auto-Invoked
|
||||
|
||||
|
||||
@@ -1,614 +0,0 @@
|
||||
# Implementation Plan: ROI Metrics & Discovery Cost Tracking
|
||||
|
||||
**Feature**: Display token discovery costs alongside observations to demonstrate knowledge reuse ROI
|
||||
**Branch**: `enhancement/roi`
|
||||
**Issue**: #104
|
||||
**Priority**: HIGH (needed for YC application amendment)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Capture token usage from Agent SDK, store as "discovery cost" with each observation, and display metrics in SessionStart context to prove that claude-mem reduces token consumption by 50-75% through knowledge reuse.
|
||||
|
||||
### The Value Proposition
|
||||
|
||||
**Session 1**: Claude spends 4,000 tokens discovering "how Stop hooks work"
|
||||
**Sessions 2-5**: Claude reads 163-token observation instead of re-discovering
|
||||
**Savings**: 15,348 tokens (77% reduction) over 5 sessions
|
||||
|
||||
This feature makes that ROI **visible and measurable** for both users and Claude.
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
Agent SDK Messages (with usage)
|
||||
↓
|
||||
SDKAgent captures usage data
|
||||
↓
|
||||
ActiveSession tracks cumulative tokens
|
||||
↓
|
||||
Observations stored with discovery_tokens
|
||||
↓
|
||||
Context hook displays metrics
|
||||
↓
|
||||
User/Claude sees ROI
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Capture Token Usage from Agent SDK
|
||||
|
||||
**File**: `src/services/worker/SDKAgent.ts`
|
||||
|
||||
**Changes**:
|
||||
1. Extract usage data from assistant messages (lines 64-86)
|
||||
2. Track cumulative session tokens in ActiveSession
|
||||
3. Pass cumulative tokens when storing observations
|
||||
|
||||
**Code Changes**:
|
||||
|
||||
```typescript
|
||||
// Line ~70: After extracting textContent, add:
|
||||
const usage = message.message.usage;
|
||||
if (usage) {
|
||||
session.cumulativeInputTokens += usage.input_tokens || 0;
|
||||
session.cumulativeOutputTokens += usage.output_tokens || 0;
|
||||
|
||||
// Cache creation counts as discovery, cache read doesn't
|
||||
if (usage.cache_creation_input_tokens) {
|
||||
session.cumulativeInputTokens += usage.cache_creation_input_tokens;
|
||||
}
|
||||
|
||||
logger.debug('SDK', 'Token usage captured', {
|
||||
sessionId: session.sessionDbId,
|
||||
inputTokens: usage.input_tokens,
|
||||
outputTokens: usage.output_tokens,
|
||||
cumulativeInput: session.cumulativeInputTokens,
|
||||
cumulativeOutput: session.cumulativeOutputTokens
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
```typescript
|
||||
// Line ~213-218: Pass discovery tokens when storing
|
||||
const { id: obsId, createdAtEpoch } = this.dbManager.getSessionStore().storeObservation(
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
obs,
|
||||
session.lastPromptNumber,
|
||||
session.cumulativeInputTokens + session.cumulativeOutputTokens // Add discovery cost
|
||||
);
|
||||
```
|
||||
|
||||
**Edge Cases**:
|
||||
- Handle missing usage data (default to 0)
|
||||
- Cache tokens: `cache_creation_input_tokens` counts as discovery, `cache_read_input_tokens` doesn't
|
||||
- Multiple observations per response: Each gets snapshot of cumulative tokens at creation time
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Update ActiveSession Type
|
||||
|
||||
**File**: `src/services/worker-types.ts`
|
||||
|
||||
**Changes**: Add token tracking fields to ActiveSession interface
|
||||
|
||||
```typescript
|
||||
export interface ActiveSession {
|
||||
sessionDbId: number;
|
||||
sdkSessionId: string | null;
|
||||
claudeSessionId: string;
|
||||
project: string;
|
||||
userPrompt: string;
|
||||
lastPromptNumber: number;
|
||||
pendingMessages: PendingMessage[];
|
||||
abortController: AbortController;
|
||||
startTime: number;
|
||||
cumulativeInputTokens: number; // NEW: Track input tokens
|
||||
cumulativeOutputTokens: number; // NEW: Track output tokens
|
||||
}
|
||||
```
|
||||
|
||||
**Initialization**: When creating new session in SessionManager.initializeSession, set:
|
||||
```typescript
|
||||
cumulativeInputTokens: 0,
|
||||
cumulativeOutputTokens: 0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Database Schema Migration
|
||||
|
||||
**File**: `src/services/sqlite/migrations.ts`
|
||||
|
||||
**Add Migration**: Create migration #8 (next available number)
|
||||
|
||||
```typescript
|
||||
{
|
||||
version: 8,
|
||||
name: 'add_discovery_tokens',
|
||||
up: (db: Database) => {
|
||||
// Add discovery_tokens to observations
|
||||
db.exec(`
|
||||
ALTER TABLE observations
|
||||
ADD COLUMN discovery_tokens INTEGER DEFAULT 0;
|
||||
`);
|
||||
|
||||
// Add discovery_tokens to summaries
|
||||
db.exec(`
|
||||
ALTER TABLE summaries
|
||||
ADD COLUMN discovery_tokens INTEGER DEFAULT 0;
|
||||
`);
|
||||
|
||||
logger.info('DB', 'Migration 8: Added discovery_tokens columns');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Why summaries too?**: Summaries represent accumulated session work, so they should also show total discovery cost.
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Update SessionStore
|
||||
|
||||
**File**: `src/services/sqlite/SessionStore.ts`
|
||||
|
||||
**Changes**:
|
||||
|
||||
1. Update `storeObservation` signature (around line ~1000):
|
||||
```typescript
|
||||
storeObservation(
|
||||
sessionId: string,
|
||||
project: string,
|
||||
observation: ParsedObservation,
|
||||
promptNumber: number,
|
||||
discoveryTokens: number = 0 // NEW parameter
|
||||
): { id: number; createdAtEpoch: number }
|
||||
```
|
||||
|
||||
2. Update INSERT statement to include discovery_tokens:
|
||||
```typescript
|
||||
const stmt = this.db.prepare(`
|
||||
INSERT INTO observations (
|
||||
session_id,
|
||||
project,
|
||||
type,
|
||||
title,
|
||||
subtitle,
|
||||
narrative,
|
||||
facts,
|
||||
concepts,
|
||||
files_read,
|
||||
files_modified,
|
||||
prompt_number,
|
||||
discovery_tokens, -- NEW
|
||||
created_at_epoch
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
const result = stmt.run(
|
||||
sessionId,
|
||||
project,
|
||||
observation.type,
|
||||
observation.title,
|
||||
observation.subtitle || '',
|
||||
observation.narrative || '',
|
||||
JSON.stringify(observation.facts || []),
|
||||
JSON.stringify(observation.concepts || []),
|
||||
JSON.stringify(observation.files || []),
|
||||
JSON.stringify([]),
|
||||
promptNumber,
|
||||
discoveryTokens, // NEW
|
||||
createdAtEpoch
|
||||
);
|
||||
```
|
||||
|
||||
3. Update `storeSummary` similarly (around line ~1150):
|
||||
```typescript
|
||||
storeSummary(
|
||||
sessionId: string,
|
||||
project: string,
|
||||
summary: ParsedSummary,
|
||||
promptNumber: number,
|
||||
discoveryTokens: number = 0 // NEW parameter
|
||||
): { id: number; createdAtEpoch: number }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Update Database Types
|
||||
|
||||
**File**: `src/services/sqlite/types.ts`
|
||||
|
||||
**Changes**: Add discovery_tokens to DBObservation and DBSummary interfaces
|
||||
|
||||
```typescript
|
||||
export interface DBObservation {
|
||||
id: number;
|
||||
session_id: string;
|
||||
project: string;
|
||||
type: 'decision' | 'bugfix' | 'feature' | 'refactor' | 'discovery' | 'change';
|
||||
title: string;
|
||||
subtitle: string;
|
||||
narrative: string | null;
|
||||
facts: string; // JSON array
|
||||
concepts: string; // JSON array
|
||||
files_read: string; // JSON array
|
||||
files_modified: string; // JSON array
|
||||
prompt_number: number;
|
||||
discovery_tokens: number; // NEW
|
||||
created_at_epoch: number;
|
||||
}
|
||||
|
||||
export interface DBSummary {
|
||||
id: number;
|
||||
session_id: string;
|
||||
request: string;
|
||||
investigated: string | null;
|
||||
learned: string | null;
|
||||
completed: string | null;
|
||||
next_steps: string | null;
|
||||
notes: string | null;
|
||||
project: string;
|
||||
prompt_number: number;
|
||||
discovery_tokens: number; // NEW
|
||||
created_at_epoch: number;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 6: Update Search Queries
|
||||
|
||||
**File**: `src/services/sqlite/SessionSearch.ts`
|
||||
|
||||
**Changes**: Ensure all SELECT queries include discovery_tokens
|
||||
|
||||
Example (around line ~50, searchObservations):
|
||||
```typescript
|
||||
SELECT
|
||||
o.id,
|
||||
o.session_id,
|
||||
o.project,
|
||||
o.type,
|
||||
o.title,
|
||||
o.subtitle,
|
||||
o.narrative,
|
||||
o.facts,
|
||||
o.concepts,
|
||||
o.files_read,
|
||||
o.files_modified,
|
||||
o.prompt_number,
|
||||
o.discovery_tokens, -- NEW
|
||||
o.created_at_epoch,
|
||||
...
|
||||
```
|
||||
|
||||
**Affected methods**:
|
||||
- `searchObservations`
|
||||
- `getRecentObservations`
|
||||
- `getObservationsByType`
|
||||
- `getObservationsByConcept`
|
||||
- `getObservationsByFile`
|
||||
- All other observation query methods
|
||||
|
||||
---
|
||||
|
||||
### Phase 7: Update Context Hook Display
|
||||
|
||||
**File**: `src/hooks/context-hook.ts`
|
||||
|
||||
**Changes**: Display discovery costs and ROI metrics in SessionStart context
|
||||
|
||||
**Section 1: Add Aggregate Metrics** (insert after line ~250, before observation table)
|
||||
|
||||
```typescript
|
||||
// Calculate aggregate metrics
|
||||
const totalObservations = observations.length;
|
||||
const totalReadTokens = observations.reduce((sum, obs) => {
|
||||
// Estimate read tokens from observation size
|
||||
const obsSize = (obs.title?.length || 0) +
|
||||
(obs.subtitle?.length || 0) +
|
||||
(obs.narrative?.length || 0) +
|
||||
JSON.stringify(obs.facts || []).length;
|
||||
return sum + Math.ceil(obsSize / 4); // Rough token estimate
|
||||
}, 0);
|
||||
const totalDiscoveryTokens = observations.reduce((sum, obs) => sum + (obs.discovery_tokens || 0), 0);
|
||||
const savings = totalDiscoveryTokens - totalReadTokens;
|
||||
const savingsPercent = totalDiscoveryTokens > 0
|
||||
? Math.round((savings / totalDiscoveryTokens) * 100)
|
||||
: 0;
|
||||
|
||||
output += `\n📊 **Context Economics**:\n`;
|
||||
output += ` - Loading: ${totalObservations} observations (${totalReadTokens.toLocaleString()} tokens to read)\n`;
|
||||
output += ` - Discovery investment: ${totalDiscoveryTokens.toLocaleString()} tokens spent by previous sessions\n`;
|
||||
output += ` - Your savings: ${savings.toLocaleString()} tokens (${savingsPercent}% reduction from reuse)\n\n`;
|
||||
```
|
||||
|
||||
**Section 2: Update Table Format** (around line ~280)
|
||||
|
||||
```typescript
|
||||
output += `| ID | Time | T | Title | Read | Discovery |\n`;
|
||||
output += `|----|------|---|-------|------|-----------||\n`;
|
||||
|
||||
for (const obs of observations) {
|
||||
const time = formatTime(obs.created_at_epoch);
|
||||
const emoji = getTypeEmoji(obs.type);
|
||||
const title = truncate(obs.title, 50);
|
||||
|
||||
// Estimate read tokens (observation size in tokens)
|
||||
const obsSize = (obs.title?.length || 0) +
|
||||
(obs.subtitle?.length || 0) +
|
||||
(obs.narrative?.length || 0) +
|
||||
JSON.stringify(obs.facts || []).length;
|
||||
const readTokens = Math.ceil(obsSize / 4);
|
||||
|
||||
const discoveryTokens = obs.discovery_tokens || 0;
|
||||
const discoveryDisplay = discoveryTokens > 0
|
||||
? `🔍 ${discoveryTokens.toLocaleString()}`
|
||||
: '-';
|
||||
|
||||
output += `| #${obs.id} | ${time} | ${emoji} | ${title} | ~${readTokens} | ${discoveryDisplay} |\n`;
|
||||
}
|
||||
```
|
||||
|
||||
**Section 3: Add Footer Explanation** (after table)
|
||||
|
||||
```typescript
|
||||
output += `\n💡 **Column Key**:\n`;
|
||||
output += ` - **Read**: Tokens to read this observation (cost to learn it now)\n`;
|
||||
output += ` - **Discovery**: Tokens Previous Claude spent exploring/researching this topic\n`;
|
||||
output += `\n**ROI**: Reading these learnings instead of re-discovering saves ${savingsPercent}% tokens\n`;
|
||||
```
|
||||
|
||||
**Edge Case**: Handle old observations without discovery_tokens (show '-' or 0)
|
||||
|
||||
---
|
||||
|
||||
### Phase 8: Update Chroma Sync (Optional)
|
||||
|
||||
**File**: `src/services/sync/ChromaSync.ts`
|
||||
|
||||
**Changes**: Include discovery_tokens in vector metadata
|
||||
|
||||
```typescript
|
||||
// Around line ~100, syncObservation metadata
|
||||
metadata: {
|
||||
session_id: sessionId,
|
||||
project: project,
|
||||
type: observation.type,
|
||||
title: observation.title,
|
||||
prompt_number: promptNumber,
|
||||
discovery_tokens: discoveryTokens, // NEW
|
||||
created_at_epoch: createdAtEpoch,
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
**Why?**: Enables semantic search to factor in discovery cost for relevance scoring (future enhancement)
|
||||
|
||||
---
|
||||
|
||||
## Testing Plan
|
||||
|
||||
### Unit Tests
|
||||
|
||||
1. **Token Capture Test**:
|
||||
- Mock Agent SDK response with usage data
|
||||
- Verify ActiveSession.cumulativeTokens increments correctly
|
||||
- Test cache token handling (creation counts, read doesn't)
|
||||
|
||||
2. **Storage Test**:
|
||||
- Create observation with discovery_tokens
|
||||
- Verify database stores correctly
|
||||
- Query back and verify field present
|
||||
|
||||
3. **Display Test**:
|
||||
- Create test observations with varying discovery costs
|
||||
- Run context-hook
|
||||
- Verify metrics calculate correctly
|
||||
- Verify table displays both Read and Discovery columns
|
||||
|
||||
### Integration Tests
|
||||
|
||||
1. **Full Session Flow**:
|
||||
- Start new session
|
||||
- Trigger multiple tool executions
|
||||
- Generate observations
|
||||
- Verify cumulative tokens accumulate
|
||||
- Check context displays metrics
|
||||
|
||||
2. **Migration Test**:
|
||||
- Backup existing database
|
||||
- Run migration #8
|
||||
- Verify columns added
|
||||
- Verify existing data intact (discovery_tokens = 0)
|
||||
- Test new observations store correctly
|
||||
|
||||
### Manual Testing
|
||||
|
||||
1. **Real Usage Scenario**:
|
||||
- Start fresh Claude Code session
|
||||
- Perform research task (read files, search codebase)
|
||||
- Generate observations via claude-mem
|
||||
- Check database for discovery_tokens values
|
||||
- Start new session, verify context shows metrics
|
||||
|
||||
2. **YC Demo Data**:
|
||||
- Run 5 sessions on same topic
|
||||
- Collect token data for each session
|
||||
- Calculate actual ROI (Session 1 cost vs Sessions 2-5)
|
||||
- Screenshot metrics for YC application
|
||||
|
||||
---
|
||||
|
||||
## Rollout Plan
|
||||
|
||||
### Phase 1: Data Collection (Week 1)
|
||||
- Deploy migration and token capture
|
||||
- Run without displaying metrics yet
|
||||
- Verify data quality and accuracy
|
||||
- Fix any issues with token tracking
|
||||
|
||||
### Phase 2: Display Metrics (Week 2)
|
||||
- Enable context hook display
|
||||
- Gather user feedback
|
||||
- Iterate on presentation format
|
||||
- Document any edge cases
|
||||
|
||||
### Phase 3: YC Application (Week 2-3)
|
||||
- Collect empirical data from real usage
|
||||
- Generate charts/graphs showing ROI
|
||||
- Write case study with actual numbers
|
||||
- Amend YC application with proof
|
||||
|
||||
### Phase 4: Public Launch (Week 4)
|
||||
- Blog post explaining the feature
|
||||
- Update README with ROI metrics
|
||||
- Submit to HN/Reddit with data
|
||||
- Reach out to Anthropic with findings
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
**Technical Success**:
|
||||
- ✅ Token capture accuracy: >95% of SDK responses captured
|
||||
- ✅ Database migration: 0 data loss, all observations migrated
|
||||
- ✅ Display accuracy: Metrics match raw data within 5%
|
||||
|
||||
**Business Success**:
|
||||
- ✅ Demonstrate 50-75% token reduction across 10+ sessions
|
||||
- ✅ YC application strengthened with empirical data
|
||||
- ✅ User/Claude understanding of ROI improves (survey/feedback)
|
||||
|
||||
**Strategic Success**:
|
||||
- ✅ Proof that memory optimization reduces infrastructure needs
|
||||
- ✅ Data compelling enough for Anthropic partnership discussion
|
||||
- ✅ Foundation for enterprise licensing ROI calculator
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Token Attribution**:
|
||||
- Should each observation get cumulative session tokens, or split proportionally?
|
||||
- **Decision**: Use cumulative (simpler, shows total cost at that point)
|
||||
|
||||
2. **Cache Tokens**:
|
||||
- How to handle cache_read_input_tokens in ROI calculation?
|
||||
- **Decision**: Don't count cache reads as discovery (they're already discovered)
|
||||
|
||||
3. **Display Format**:
|
||||
- Show raw token counts or human-readable format (K, M)?
|
||||
- **Decision**: Use toLocaleString() for readability (e.g., "4,000" not "4K")
|
||||
|
||||
4. **Pricing Display**:
|
||||
- Should we show dollar costs too, or just tokens?
|
||||
- **Decision**: Tokens only initially. Pricing varies by model/plan, adds complexity
|
||||
|
||||
5. **Historical Data**:
|
||||
- What to do with old observations without discovery_tokens?
|
||||
- **Decision**: Show as 0 or '-', document limitation
|
||||
|
||||
---
|
||||
|
||||
## Files Modified Summary
|
||||
|
||||
**Core Implementation**:
|
||||
- `src/services/worker/SDKAgent.ts` - Capture usage, pass to storage
|
||||
- `src/services/worker-types.ts` - Add cumulative token fields
|
||||
- `src/services/sqlite/migrations.ts` - Migration #8 for discovery_tokens
|
||||
- `src/services/sqlite/SessionStore.ts` - Store discovery tokens
|
||||
- `src/services/sqlite/types.ts` - Update interfaces
|
||||
- `src/services/sqlite/SessionSearch.ts` - Include in queries
|
||||
- `src/hooks/context-hook.ts` - Display metrics
|
||||
|
||||
**Optional**:
|
||||
- `src/services/sync/ChromaSync.ts` - Include in vector metadata
|
||||
- `src/services/worker/SessionManager.ts` - Initialize cumulative tokens
|
||||
|
||||
**Documentation**:
|
||||
- `CLAUDE.md` - Update with new feature
|
||||
- `README.md` - Add ROI metrics section
|
||||
- Issue #104 - Track implementation progress
|
||||
|
||||
---
|
||||
|
||||
## Timeline Estimate
|
||||
|
||||
**Day 1** (Tomorrow):
|
||||
- [ ] Create branch ✅
|
||||
- [ ] Write implementation plan ✅
|
||||
- [ ] Phase 1: Capture token usage (2 hours)
|
||||
- [ ] Phase 2: Update types (30 min)
|
||||
- [ ] Phase 3: Database migration (1 hour)
|
||||
|
||||
**Day 2**:
|
||||
- [ ] Phase 4: Update SessionStore (1 hour)
|
||||
- [ ] Phase 5: Update types (30 min)
|
||||
- [ ] Phase 6: Update search queries (1 hour)
|
||||
- [ ] Testing: Unit tests (2 hours)
|
||||
|
||||
**Day 3**:
|
||||
- [ ] Phase 7: Update context hook display (2 hours)
|
||||
- [ ] Testing: Integration tests (2 hours)
|
||||
- [ ] Manual testing and iteration (2 hours)
|
||||
|
||||
**Day 4**:
|
||||
- [ ] Collect real usage data (ongoing throughout day)
|
||||
- [ ] Generate YC metrics/charts (2 hours)
|
||||
- [ ] Amend YC application (2 hours)
|
||||
- [ ] Documentation updates (1 hour)
|
||||
|
||||
**Total**: ~20 hours of development over 4 days
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
**Risk 1**: Agent SDK usage data incomplete or missing
|
||||
**Mitigation**: Default to 0, log warnings, don't break existing functionality
|
||||
|
||||
**Risk 2**: Migration fails on large databases
|
||||
**Mitigation**: Test on database copy first, add rollback mechanism
|
||||
|
||||
**Risk 3**: Token estimates inaccurate
|
||||
**Mitigation**: Document methodology, provide "rough estimate" disclaimer
|
||||
|
||||
**Risk 4**: Display too noisy/overwhelming
|
||||
**Mitigation**: Make display configurable via settings, start collapsed
|
||||
|
||||
**Risk 5**: YC data not compelling enough
|
||||
**Mitigation**: Run on diverse projects, cherry-pick best examples, be honest about limitations
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ✅ Create branch `enhancement/roi`
|
||||
2. ✅ Write implementation plan
|
||||
3. Start Phase 1: Implement token capture in SDKAgent.ts
|
||||
4. Run manual test to verify usage data captured
|
||||
5. Continue through phases sequentially
|
||||
6. Collect data for YC application by end of week
|
||||
|
||||
---
|
||||
|
||||
## Notes for Tomorrow
|
||||
|
||||
**Start here**: `src/services/worker/SDKAgent.ts` line 64-86
|
||||
**Key insight**: `message.message.usage` contains the token data
|
||||
**Don't forget**: Initialize cumulative tokens to 0 in SessionManager
|
||||
**Test with**: Simple session that reads a few files and creates 1-2 observations
|
||||
|
||||
**The goal**: By end of week, have real numbers showing 50-75% token savings to prove the hypothesis and strengthen YC application.
|
||||
|
||||
---
|
||||
|
||||
*This plan represents ~20 hours of focused development. Prioritize getting Phase 1-7 working correctly over perfection. The YC data is the critical deliverable.*
|
||||
@@ -89,31 +89,30 @@ npx mintlify dev
|
||||
|
||||
### Getting Started
|
||||
|
||||
- **[Installation Guide](docs/installation.mdx)** - Quick start & advanced installation
|
||||
- **[Usage Guide](docs/usage/getting-started.mdx)** - How Claude-Mem works automatically
|
||||
- **[Search Tools](docs/usage/search-tools.mdx)** - Query your project history with natural language
|
||||
- **[Installation Guide](https://docs.claude-mem.ai/installation)** - Quick start & advanced installation
|
||||
- **[Usage Guide](https://docs.claude-mem.ai/usage/getting-started)** - How Claude-Mem works automatically
|
||||
- **[Search Tools](https://docs.claude-mem.ai/usage/search-tools)** - Query your project history with natural language
|
||||
|
||||
### Best Practices
|
||||
|
||||
- **[Context Engineering](docs/context-engineering.mdx)** - AI agent context optimization principles
|
||||
- **[Progressive Disclosure](docs/progressive-disclosure.mdx)** - Philosophy behind Claude-Mem's context priming strategy
|
||||
- **[Context Engineering](https://docs.claude-mem.ai/context-engineering)** - AI agent context optimization principles
|
||||
- **[Progressive Disclosure](https://docs.claude-mem.ai/progressive-disclosure)** - Philosophy behind Claude-Mem's context priming strategy
|
||||
|
||||
### Architecture
|
||||
|
||||
- **[Overview](docs/architecture/overview.mdx)** - System components & data flow
|
||||
- **[Architecture Evolution](docs/architecture-evolution.mdx)** - The journey from v3 to v5
|
||||
- **[Hooks Architecture](docs/hooks-architecture.mdx)** - How Claude-Mem uses lifecycle hooks
|
||||
- **[Hooks Reference](docs/architecture/hooks.mdx)** - 7 hook scripts explained
|
||||
- **[Worker Service](docs/architecture/worker-service.mdx)** - HTTP API & PM2 management
|
||||
- **[Database](docs/architecture/database.mdx)** - SQLite schema & FTS5 search
|
||||
- **[MCP Search](docs/architecture/mcp-search.mdx)** - 9 search tools & examples
|
||||
- **[Viewer UI](docs/VIEWER.md)** - Web-based memory stream visualization
|
||||
- **[Overview](https://docs.claude-mem.ai/architecture/overview)** - System components & data flow
|
||||
- **[Architecture Evolution](https://docs.claude-mem.ai/architecture-evolution)** - The journey from v3 to v5
|
||||
- **[Hooks Architecture](https://docs.claude-mem.ai/hooks-architecture)** - How Claude-Mem uses lifecycle hooks
|
||||
- **[Hooks Reference](https://docs.claude-mem.ai/architecture/hooks)** - 7 hook scripts explained
|
||||
- **[Worker Service](https://docs.claude-mem.ai/architecture/worker-service)** - HTTP API & PM2 management
|
||||
- **[Database](https://docs.claude-mem.ai/architecture/database)** - SQLite schema & FTS5 search
|
||||
- **[Search Architecture](https://docs.claude-mem.ai/architecture/search-architecture)** - Hybrid search with Chroma vector database
|
||||
|
||||
### Configuration & Development
|
||||
|
||||
- **[Configuration](docs/configuration.mdx)** - Environment variables & settings
|
||||
- **[Development](docs/development.mdx)** - Building, testing, contributing
|
||||
- **[Troubleshooting](docs/troubleshooting.mdx)** - Common issues & solutions
|
||||
- **[Configuration](https://docs.claude-mem.ai/configuration)** - Environment variables & settings
|
||||
- **[Development](https://docs.claude-mem.ai/development)** - Building, testing, contributing
|
||||
- **[Troubleshooting](https://docs.claude-mem.ai/troubleshooting)** - Common issues & solutions
|
||||
|
||||
---
|
||||
|
||||
@@ -150,7 +149,7 @@ npx mintlify dev
|
||||
5. **mem-search Skill** - Natural language queries with progressive disclosure (~2,250 token savings vs MCP)
|
||||
6. **Chroma Vector Database** - Hybrid semantic + keyword search for intelligent context retrieval
|
||||
|
||||
See [Architecture Overview](docs/architecture/overview.mdx) for details.
|
||||
See [Architecture Overview](https://docs.claude-mem.ai/architecture/overview) for details.
|
||||
|
||||
---
|
||||
|
||||
@@ -186,7 +185,7 @@ Claude-Mem provides intelligent search through the mem-search skill that auto-in
|
||||
"What was happening when we added the viewer UI?"
|
||||
```
|
||||
|
||||
See [Search Tools Guide](docs/usage/search-tools.mdx) for detailed examples.
|
||||
See [Search Tools Guide](https://docs.claude-mem.ai/usage/search-tools) for detailed examples.
|
||||
|
||||
---
|
||||
|
||||
@@ -278,7 +277,7 @@ See [CHANGELOG.md](CHANGELOG.md) for complete version history.
|
||||
- `CLAUDE_MEM_WORKER_PORT` - Worker port (default: 37777)
|
||||
- `CLAUDE_MEM_DATA_DIR` - Data directory override (dev only)
|
||||
|
||||
See [Configuration Guide](docs/configuration.mdx) for details.
|
||||
See [Configuration Guide](https://docs.claude-mem.ai/configuration) for details.
|
||||
|
||||
---
|
||||
|
||||
@@ -301,7 +300,7 @@ npm run worker:start
|
||||
npm run worker:logs
|
||||
```
|
||||
|
||||
See [Development Guide](docs/development.mdx) for detailed instructions.
|
||||
See [Development Guide](https://docs.claude-mem.ai/development) for detailed instructions.
|
||||
|
||||
---
|
||||
|
||||
@@ -318,7 +317,7 @@ If you're experiencing issues, describe the problem to Claude and the troublesho
|
||||
- Database issues → `sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;"`
|
||||
- Search not working → Check FTS5 tables exist
|
||||
|
||||
See [Troubleshooting Guide](docs/troubleshooting.mdx) for complete solutions.
|
||||
See [Troubleshooting Guide](https://docs.claude-mem.ai/troubleshooting) for complete solutions.
|
||||
|
||||
---
|
||||
|
||||
@@ -332,7 +331,7 @@ Contributions are welcome! Please:
|
||||
4. Update documentation
|
||||
5. Submit a Pull Request
|
||||
|
||||
See [Development Guide](docs/development.mdx) for contribution workflow.
|
||||
See [Development Guide](https://docs.claude-mem.ai/development) for contribution workflow.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -148,16 +148,19 @@ When Claude invokes the skill:
|
||||
|
||||
## Search Architecture
|
||||
|
||||
### Hybrid Search System
|
||||
### 3-Layer Hybrid Search System
|
||||
|
||||
claude-mem uses a **hybrid search architecture** combining:
|
||||
claude-mem uses a **3-layer sequential search architecture** that mimics human long-term memory:
|
||||
|
||||
1. **SQLite FTS5 (Full-Text Search)** - Keyword-based search
|
||||
2. **ChromaDB (Vector Search)** - Semantic similarity search
|
||||
**Storage Flow (Write Path):**
|
||||
1. **SQLite First** - Data written synchronously to SQLite (fast, immediate access)
|
||||
2. **ChromaDB Background Sync** - Worker asynchronously generates embeddings and syncs to ChromaDB
|
||||
|
||||
**Search Flow (Read Path - Sequential, NOT parallel):**
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Search Request Flow │
|
||||
│ 3-Layer Sequential Search Flow │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
@@ -166,61 +169,70 @@ claude-mem uses a **hybrid search architecture** combining:
|
||||
│ /api/search/* │
|
||||
└─────────────────────────┘
|
||||
│
|
||||
┌─────────────┴─────────────┐
|
||||
▼ ▼
|
||||
┌──────────────────────────┐ ┌──────────────────────────┐
|
||||
│ SessionSearch (FTS5) │ │ ChromaSync (Vector DB) │
|
||||
│ │ │ │
|
||||
│ Full-text keyword │ │ Semantic similarity │
|
||||
│ search on: │ │ search on: │
|
||||
│ - titles │ │ - narratives │
|
||||
│ - narratives │ │ - facts │
|
||||
│ - facts │ │ - file content │
|
||||
│ - concepts │ │ │
|
||||
│ │ │ Embeddings: │
|
||||
│ SQLite DB: │ │ - text-embedding-3-small│
|
||||
│ observations_fts │ │ - 90-day recency filter │
|
||||
│ sessions_fts │ │ │
|
||||
│ prompts_fts │ │ ChromaDB: │
|
||||
│ │ │ observations collection │
|
||||
└──────────────────────────┘ └──────────────────────────┘
|
||||
│ │
|
||||
└─────────────┬─────────────┘
|
||||
▼
|
||||
┌─────────────────────────┐
|
||||
│ Merged Results │
|
||||
│ - Deduplicated │
|
||||
│ - Sorted by relevance │
|
||||
│ - Formatted (index/full)│
|
||||
└─────────────────────────┘
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ LAYER 1: Semantic Retrieval (ChromaDB) │
|
||||
│ ───────────────────────────────────────────────────────── │
|
||||
│ Vector similarity search finds semantically relevant items │
|
||||
│ Returns: observation IDs in index format (~50-100 tokens) │
|
||||
│ Filter: 90-day recency prioritizes recent work │
|
||||
│ Output: List of relevant observation IDs │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ LAYER 2: Temporal Ordering (SQLite) │
|
||||
│ ───────────────────────────────────────────────────────── │
|
||||
│ Takes observation IDs from Layer 1 │
|
||||
│ Sorts by created_at timestamp (fast SQLite temporal query) │
|
||||
│ Identifies: MOST RECENT relevant observation │
|
||||
│ Why: ChromaDB doesn't easily query by date range sorted │
|
||||
│ Output: Top observation ID by time │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ LAYER 3: Instant Context Timeline (SQLite) │
|
||||
│ ───────────────────────────────────────────────────────── │
|
||||
│ Uses top observation ID from Layer 2 as anchor │
|
||||
│ Retrieves N observations BEFORE and AFTER that point │
|
||||
│ Provides: "what led here" + "what happened next" context │
|
||||
│ This is the KILLER FEATURE: mimics human memory │
|
||||
│ Output: Timeline with temporal context │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Why This Architecture Exists:**
|
||||
|
||||
The problem: LLMs don't experience time linearly like humans do. Finding semantically relevant information isn't enough—you need temporal context.
|
||||
|
||||
The solution:
|
||||
- **ChromaDB** for "what's relevant" (semantic understanding)
|
||||
- **SQLite** for "when did it happen" (temporal ordering with fast date-range queries)
|
||||
- **Timeline** for "what was the context" (before/after observations)
|
||||
|
||||
Together, they mimic how humans recall: "I did X, which led to Y, then Z happened."
|
||||
|
||||
**Human Memory Analogy:**
|
||||
|
||||
Humans don't just remember isolated facts. They remember sequences: what they did before something, what happened after. The instant context timeline gives LLMs this same temporal awareness that humans experience naturally.
|
||||
|
||||
### Search Types
|
||||
|
||||
#### 1. Full-Text Search (FTS5)
|
||||
#### 1. Vector Search (ChromaDB) - PRIMARY Search Layer
|
||||
|
||||
**How it works:**
|
||||
- Uses SQLite FTS5 virtual tables for instant keyword matching
|
||||
- Supports boolean operators: `AND`, `OR`, `NOT`, `NEAR`, `*` (wildcard)
|
||||
- Ranks results by BM25 relevance scoring
|
||||
- Sub-100ms performance on 8,000+ observations
|
||||
|
||||
**Example query:**
|
||||
```sql
|
||||
-- User asks: "How did we implement JWT authentication?"
|
||||
SELECT * FROM observations_fts
|
||||
WHERE observations_fts MATCH 'JWT AND authentication'
|
||||
ORDER BY rank
|
||||
LIMIT 20;
|
||||
```
|
||||
|
||||
#### 2. Vector Search (ChromaDB)
|
||||
**Role:** Layer 1 - Semantic Retrieval
|
||||
|
||||
**How it works:**
|
||||
- Text is embedded using OpenAI's `text-embedding-3-small` model
|
||||
- Vector similarity search finds semantically related content
|
||||
- Vector similarity search finds semantically related content, not just keyword matches
|
||||
- 90-day recency filter prioritizes recent work
|
||||
- Combined with keyword search for hybrid results
|
||||
- Returns observation IDs for temporal processing in Layer 2
|
||||
|
||||
**Why it's primary:**
|
||||
- Understands meaning, not just keywords ("auth flow" matches "JWT implementation")
|
||||
- Finds relevant work even when you don't know exact terms used
|
||||
- Semantic understanding crucial for LLM memory retrieval
|
||||
|
||||
**Example query:**
|
||||
```python
|
||||
@@ -230,6 +242,37 @@ collection.query(
|
||||
n_results=20,
|
||||
where={"created_at": {"$gte": ninety_days_ago}}
|
||||
)
|
||||
# Returns: observation IDs semantically related to login/auth
|
||||
```
|
||||
|
||||
#### 2. Full-Text Search (FTS5) - Supporting Layer
|
||||
|
||||
**Role:** Layer 2 & 3 - Temporal Ordering and Timeline Context
|
||||
|
||||
**How it works:**
|
||||
- Uses SQLite FTS5 virtual tables for instant keyword matching
|
||||
- Supports boolean operators: `AND`, `OR`, `NOT`, `NEAR`, `*` (wildcard)
|
||||
- Fast temporal queries with date-range sorting
|
||||
- Sub-100ms performance on 8,000+ observations
|
||||
|
||||
**Why it's supporting:**
|
||||
- ChromaDB handles semantic "what's relevant"
|
||||
- SQLite/FTS5 handles temporal "when did it happen" and "what came before/after"
|
||||
- Optimized for timeline queries and date-based sorting
|
||||
|
||||
**Example query:**
|
||||
```sql
|
||||
-- Takes observation IDs from ChromaDB, sorts by time
|
||||
SELECT * FROM observations
|
||||
WHERE id IN (/* IDs from ChromaDB */)
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT 1;
|
||||
|
||||
-- Then retrieves timeline context around that observation
|
||||
SELECT * FROM observations
|
||||
WHERE created_at_epoch < anchor_timestamp
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT 10; -- "what led here"
|
||||
```
|
||||
|
||||
#### 3. Structured Filters
|
||||
|
||||
@@ -0,0 +1,427 @@
|
||||
# Endless Mode: Real-Time Context Compression Plan
|
||||
|
||||
## Executive Summary
|
||||
|
||||
"Endless Mode" is an optional feature that enables Claude sessions to run indefinitely by transparently compressing tool use transcripts in real-time. Using an in-memory transformation layer in the worker service, heavy tool outputs are dynamically replaced with lightweight observations during session resume—without modifying the immutable source transcripts. This allows sessions to continue for weeks or months without hitting context window limits, while preserving full conversation history and maintaining zero risk of data corruption.
|
||||
|
||||
---
|
||||
|
||||
## Problem Statement
|
||||
|
||||
### Current Behavior
|
||||
|
||||
Claude sessions accumulate full tool transcripts in the context window:
|
||||
- File reads: 5k-10k tokens per read
|
||||
- Bash outputs: 1k-5k tokens per command
|
||||
- Search results: 2k-8k tokens per search
|
||||
- Total context limit: ~200k tokens
|
||||
|
||||
When the context window fills, users must start a new session, losing conversational continuity.
|
||||
|
||||
### What Happens Today
|
||||
|
||||
1. Tool executes during session
|
||||
2. PostToolUse hook captures tool data
|
||||
3. Worker creates compressed observation (~200-500 tokens)
|
||||
4. **But**: Full tool transcript stays in Claude's context window
|
||||
5. **Observation only helps next session** via SessionStart injection
|
||||
|
||||
### The Gap
|
||||
|
||||
Observations exist and are created in real-time, but they're not used to compress the **current** session's context. We have the compressed data, we just don't apply it to the active session.
|
||||
|
||||
---
|
||||
|
||||
## Proposed Solution: Endless Mode
|
||||
|
||||
### Core Concept
|
||||
|
||||
When a session resumes (either after restart or during continuation), **transform messages in memory** by replacing heavy tool use content with lightweight observations before feeding them to the Agent SDK. The source transcript remains immutable on disk.
|
||||
|
||||
### Architecture Principle
|
||||
|
||||
**Immutable Storage + Ephemeral Transform = Safe Compression**
|
||||
|
||||
```
|
||||
Disk (never modified) Memory (transform) Agent SDK
|
||||
────────────────────── ────────────────────── ────────────────
|
||||
transcript.jsonl Load messages Resume session
|
||||
tool_use_abc → Look up observation → with compressed
|
||||
tool_use_def Replace content context
|
||||
tool_use_xyz Feed to SDK
|
||||
```
|
||||
|
||||
### Key Properties
|
||||
|
||||
1. **Immutable**: Original transcripts never modified
|
||||
2. **Non-destructive**: Full history preserved on disk
|
||||
3. **No duplication**: No forks, no copies
|
||||
4. **Transparent**: User sees same conversation, compression is under the hood
|
||||
5. **Optional**: Feature flag allows users to opt-in/out
|
||||
6. **Reversible**: Can always read original transcript
|
||||
|
||||
---
|
||||
|
||||
## How It Works
|
||||
|
||||
### Session Resume Flow (Endless Mode Enabled)
|
||||
|
||||
```
|
||||
1. User continues session / Claude Code restarts
|
||||
↓
|
||||
2. Worker service intercepts resume request
|
||||
↓
|
||||
3. Load transcript JSONL from disk (immutable)
|
||||
↓
|
||||
4. Transform Loop:
|
||||
For each message in transcript:
|
||||
- If tool_use message:
|
||||
- Query SQLite: SELECT observation WHERE tool_use_id = ?
|
||||
- Replace tool content with observation (facts, narrative, concepts)
|
||||
- If other message type:
|
||||
- Pass through unchanged
|
||||
↓
|
||||
5. Feed transformed messages to Agent SDK
|
||||
↓
|
||||
6. Agent SDK resumes session with compressed context
|
||||
↓
|
||||
7. New tool uses append to original transcript (normal flow)
|
||||
↓
|
||||
8. Next resume: Loop repeats, new tool uses also get compressed
|
||||
```
|
||||
|
||||
### Session Resume Flow (Endless Mode Disabled)
|
||||
|
||||
```
|
||||
1. User continues session
|
||||
↓
|
||||
2. Load transcript JSONL from disk
|
||||
↓
|
||||
3. Feed messages directly to Agent SDK (no transformation)
|
||||
↓
|
||||
4. Session resumes with full tool transcripts (current behavior)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: Foundation (Week 1)
|
||||
|
||||
**Goal**: Set up infrastructure for transformation layer
|
||||
|
||||
Tasks:
|
||||
1. Add `tool_use_id` column to observations table (SQLite schema migration)
|
||||
2. Update PostToolUse hook to capture and store tool_use_id
|
||||
3. Create `TransformLayer` class in worker service
|
||||
4. Add `CLAUDE_MEM_ENDLESS_MODE` environment variable (default: false)
|
||||
5. Write tests for observation lookup by tool_use_id
|
||||
|
||||
**Deliverable**: Database schema updated, tool_use_ids being captured
|
||||
|
||||
### Phase 2: Transform Logic (Week 2)
|
||||
|
||||
**Goal**: Build message transformation engine
|
||||
|
||||
Tasks:
|
||||
1. Implement `TransformLayer.transformMessages(messages)` function
|
||||
2. Tool use detection logic (identify tool_use messages in transcript)
|
||||
3. Observation lookup and replacement logic
|
||||
4. Fallback handling (if observation missing, keep original content)
|
||||
5. Message serialization/deserialization
|
||||
|
||||
**Deliverable**: Working transform function that compresses messages in memory
|
||||
|
||||
### Phase 3: Agent SDK Integration (Week 2-3)
|
||||
|
||||
**Goal**: Wire transform layer into session resume flow
|
||||
|
||||
Tasks:
|
||||
1. Identify where worker service resumes Agent SDK sessions
|
||||
2. Inject transform layer before session resume
|
||||
3. Add feature flag check (only transform if endless mode enabled)
|
||||
4. Logging and instrumentation (track compression ratios, transform time)
|
||||
5. Error handling and graceful degradation
|
||||
|
||||
**Deliverable**: Worker service can resume sessions with compressed context
|
||||
|
||||
### Phase 4: Testing & Validation (Week 3-4)
|
||||
|
||||
**Goal**: Verify endless mode works correctly
|
||||
|
||||
Tasks:
|
||||
1. Create test session with 50+ tool uses
|
||||
2. Enable endless mode and resume session
|
||||
3. Verify context window usage (should be dramatically lower)
|
||||
4. Test conversation quality (does Claude have enough context?)
|
||||
5. Measure performance (transform latency, lookup speed)
|
||||
6. Edge case testing (missing observations, malformed transcripts)
|
||||
|
||||
**Deliverable**: Endless mode working in test environment
|
||||
|
||||
### Phase 5: Beta Release (Week 4+)
|
||||
|
||||
**Goal**: Release to power users for feedback
|
||||
|
||||
Tasks:
|
||||
1. Documentation (how to enable, what to expect, how to disable)
|
||||
2. Add endless mode toggle to viewer UI
|
||||
3. Monitoring and observability (track usage, failures, compression stats)
|
||||
4. Collect feedback from beta users
|
||||
5. Iterate based on real-world usage
|
||||
|
||||
**Deliverable**: Endless mode available as opt-in beta feature
|
||||
|
||||
---
|
||||
|
||||
## Technical Requirements
|
||||
|
||||
### Database Schema
|
||||
|
||||
```sql
|
||||
-- Add to observations table
|
||||
ALTER TABLE observations ADD COLUMN tool_use_id TEXT UNIQUE;
|
||||
CREATE INDEX idx_observations_tool_use_id ON observations(tool_use_id);
|
||||
```
|
||||
|
||||
### Worker Service API
|
||||
|
||||
```typescript
|
||||
interface TransformLayerConfig {
|
||||
enabled: boolean; // CLAUDE_MEM_ENDLESS_MODE
|
||||
fallbackToOriginal: boolean; // If observation missing, use full content
|
||||
maxLookupTime: number; // Timeout for SQLite queries
|
||||
}
|
||||
|
||||
class TransformLayer {
|
||||
constructor(config: TransformLayerConfig, db: SessionStore);
|
||||
|
||||
// Main transform function
|
||||
async transformMessages(messages: Message[]): Promise<Message[]>;
|
||||
|
||||
// Helper functions
|
||||
private async lookupObservation(toolUseId: string): Promise<Observation | null>;
|
||||
private replaceToolContent(message: Message, observation: Observation): Message;
|
||||
private isToolUseMessage(message: Message): boolean;
|
||||
}
|
||||
```
|
||||
|
||||
### Agent SDK Integration Point
|
||||
|
||||
```typescript
|
||||
// In worker service session resume logic
|
||||
async function resumeSession(sessionId: string, transcriptPath: string) {
|
||||
const messages = await loadTranscript(transcriptPath);
|
||||
|
||||
// Transform layer (only if endless mode enabled)
|
||||
const transformedMessages = config.endlessMode
|
||||
? await transformLayer.transformMessages(messages)
|
||||
: messages;
|
||||
|
||||
// Resume with transformed (or original) messages
|
||||
return await agentSDK.resumeSession({
|
||||
sessionId,
|
||||
messages: transformedMessages
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
### Risk 1: Information Loss
|
||||
|
||||
**Risk**: Compressed observations may lose critical details that Claude needs to reference later.
|
||||
|
||||
**Mitigation**:
|
||||
- Make endless mode optional (users can disable if quality degrades)
|
||||
- Improve observation quality (better prompts, more comprehensive facts)
|
||||
- Hybrid approach: Keep recent N tool uses in full, compress older ones
|
||||
- Monitor conversation quality metrics
|
||||
|
||||
### Risk 2: Transform Performance
|
||||
|
||||
**Risk**: Looking up observations for 100+ tool uses during resume could be slow.
|
||||
|
||||
**Mitigation**:
|
||||
- Index tool_use_id in SQLite (O(log n) lookups)
|
||||
- Batch queries (single SELECT with IN clause)
|
||||
- Measure and optimize (target <100ms for typical session)
|
||||
- Cache observations in memory during session
|
||||
|
||||
### Risk 3: Missing Observations
|
||||
|
||||
**Risk**: Tool use executed but observation not yet created (async worker lag).
|
||||
|
||||
**Mitigation**:
|
||||
- Fallback to original content if observation missing
|
||||
- Log when fallback occurs (helps identify worker performance issues)
|
||||
- Allow observations to be created retroactively
|
||||
- Consider synchronous observation creation for critical tools
|
||||
|
||||
### Risk 4: Transcript Corruption
|
||||
|
||||
**Risk**: Bug in transform layer could corrupt user conversations.
|
||||
|
||||
**Mitigation**:
|
||||
- **Never modify source transcripts** (read-only)
|
||||
- Transform happens in memory only
|
||||
- Extensive testing before beta release
|
||||
- Feature flag allows instant disable if issues found
|
||||
- Keep full audit trail in logs
|
||||
|
||||
### Risk 5: Agent SDK Compatibility
|
||||
|
||||
**Risk**: Agent SDK updates could break transform layer integration.
|
||||
|
||||
**Mitigation**:
|
||||
- Document exact Agent SDK version requirements
|
||||
- Monitor Agent SDK release notes
|
||||
- Test against new SDK versions before upgrading
|
||||
- Graceful degradation if SDK changes detected
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
### Proof of Concept Success
|
||||
|
||||
- [ ] Transform layer successfully compresses a 50-tool-use session
|
||||
- [ ] Context window usage reduced by 80%+ compared to uncompressed
|
||||
- [ ] Session resumes without errors
|
||||
- [ ] Conversation quality remains high (subjective evaluation)
|
||||
|
||||
### Beta Release Success
|
||||
|
||||
- [ ] 10+ users running endless mode without issues
|
||||
- [ ] Average context savings: 85%+ across all sessions
|
||||
- [ ] Transform latency: <200ms for typical resume
|
||||
- [ ] Zero transcript corruption incidents
|
||||
- [ ] Positive user feedback on conversation continuity
|
||||
|
||||
### Production Success
|
||||
|
||||
- [ ] Endless mode becomes default setting
|
||||
- [ ] Sessions running for weeks/months without context issues
|
||||
- [ ] Context window exhaustion becomes rare edge case
|
||||
- [ ] User-reported "session too long" issues drop to near zero
|
||||
- [ ] Transform layer performance scales to 1000+ tool use sessions
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```bash
|
||||
# Enable endless mode (default: false)
|
||||
CLAUDE_MEM_ENDLESS_MODE=true
|
||||
|
||||
# Fallback behavior if observation missing (default: true)
|
||||
CLAUDE_MEM_TRANSFORM_FALLBACK=true
|
||||
|
||||
# Max time to wait for observation lookup (default: 500ms)
|
||||
CLAUDE_MEM_TRANSFORM_TIMEOUT=500
|
||||
|
||||
# Keep recent N tool uses uncompressed (default: 0, compress all)
|
||||
CLAUDE_MEM_TRANSFORM_KEEP_RECENT=0
|
||||
```
|
||||
|
||||
### User Controls
|
||||
|
||||
```typescript
|
||||
// Future: UI toggle in viewer
|
||||
interface EndlessModeSettings {
|
||||
enabled: boolean;
|
||||
keepRecentToolUses: number; // Hybrid mode
|
||||
fallbackToOriginal: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Context Economics: Before vs. After
|
||||
|
||||
### Example Session (50 tool uses)
|
||||
|
||||
**Before (Endless Mode OFF):**
|
||||
```
|
||||
File reads: 10 × 8,000 tokens = 80,000 tokens
|
||||
Bash outputs: 20 × 2,000 tokens = 40,000 tokens
|
||||
Searches: 15 × 4,000 tokens = 60,000 tokens
|
||||
Other tools: 5 × 1,000 tokens = 5,000 tokens
|
||||
──────────────────────────────────────────────────
|
||||
Total: 185,000 tokens
|
||||
Context remaining: 15,000 tokens (92% full)
|
||||
```
|
||||
|
||||
**After (Endless Mode ON):**
|
||||
```
|
||||
File reads: 10 × 300 tokens = 3,000 tokens
|
||||
Bash outputs: 20 × 250 tokens = 5,000 tokens
|
||||
Searches: 15 × 400 tokens = 6,000 tokens
|
||||
Other tools: 5 × 200 tokens = 1,000 tokens
|
||||
──────────────────────────────────────────────────
|
||||
Total: 15,000 tokens
|
||||
Context remaining: 185,000 tokens (7.5% full)
|
||||
|
||||
Savings: 170,000 tokens (92% reduction)
|
||||
```
|
||||
|
||||
**Session Longevity:**
|
||||
- Before: ~50 tool uses before context full
|
||||
- After: ~600+ tool uses before context full
|
||||
- **12x longer sessions**
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
### Immediate Actions (This Week)
|
||||
|
||||
1. **Database Migration**: Add tool_use_id column to observations table
|
||||
2. **Hook Update**: Modify PostToolUse hook to capture tool_use_id from Agent SDK
|
||||
3. **Architecture Validation**: Confirm where Agent SDK session resume happens in worker service
|
||||
4. **Prototype**: Build minimal TransformLayer class with observation lookup
|
||||
|
||||
### Short Term (Next 2 Weeks)
|
||||
|
||||
1. Implement complete transform logic
|
||||
2. Wire into worker service resume flow
|
||||
3. Add endless mode feature flag
|
||||
4. Test with real sessions
|
||||
|
||||
### Medium Term (Next Month)
|
||||
|
||||
1. Beta release to power users
|
||||
2. Gather feedback and iterate
|
||||
3. Performance optimization
|
||||
4. Documentation and user guides
|
||||
|
||||
### Long Term (Future)
|
||||
|
||||
1. Make endless mode default
|
||||
2. Hybrid sliding window (keep recent tools uncompressed)
|
||||
3. Selective compression by tool type
|
||||
4. Auto-tune compression based on context usage patterns
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Tool Use ID Format**: What does the Agent SDK's tool_use_id look like? Is it UUID, hash, or sequential?
|
||||
2. **Transcript Format**: What's the exact JSONL schema for tool_use messages? Where is the content we'll replace?
|
||||
3. **Resume Hook Point**: Where exactly in the worker service does session resume happen? Is there a clear integration point?
|
||||
4. **Observation Delay**: How long between PostToolUse firing and observation being available in SQLite? Does this affect resume?
|
||||
5. **Feature Flag Storage**: Environment variable, or persist user preference in database?
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
Endless Mode transforms claude-mem from a "memory between sessions" system into a "continuous compression engine" that enables truly infinite sessions. By leveraging the observations we're already creating in real-time and applying them as an ephemeral transformation layer during resume, we can extend session longevity by 10-12x without any risk to user data.
|
||||
|
||||
The key architectural insight is **immutability**: by never modifying source transcripts and performing all compression in memory, we get the benefits of context window optimization without the risks of data corruption or loss. Combined with the optional nature of the feature, this provides a safe, reversible path to fundamentally better session continuity.
|
||||
|
||||
This is the natural evolution of claude-mem: from remembering what happened before, to making it possible to never stop.
|
||||
+10
-2
@@ -31,8 +31,16 @@ module.exports = {
|
||||
'*.log',
|
||||
'*.db',
|
||||
'*.db-*',
|
||||
'.git'
|
||||
]
|
||||
'.git',
|
||||
'vector-db', // Ignore Chroma vector DB files
|
||||
'.claude-mem' // Ignore data directory
|
||||
],
|
||||
// Allow extra time for graceful shutdown (cleanup of child processes)
|
||||
kill_timeout: 5000,
|
||||
// Wait before restarting to allow full cleanup
|
||||
wait_ready: true,
|
||||
// Shutdown signal (SIGTERM for graceful shutdown)
|
||||
kill_signal: 'SIGTERM'
|
||||
}
|
||||
]
|
||||
};
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "5.5.1",
|
||||
"version": "6.0.3",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "claude-mem",
|
||||
"version": "5.5.1",
|
||||
"version": "6.0.3",
|
||||
"license": "AGPL-3.0",
|
||||
"dependencies": {
|
||||
"@anthropic-ai/claude-agent-sdk": "^0.1.27",
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "6.0.2",
|
||||
"version": "6.0.6",
|
||||
"description": "Memory compression system for Claude Code - persist context across sessions",
|
||||
"keywords": [
|
||||
"claude",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-mem",
|
||||
"version": "6.0.2",
|
||||
"version": "6.0.6",
|
||||
"description": "Persistent memory system for Claude Code - seamlessly preserve context across sessions",
|
||||
"author": {
|
||||
"name": "Alex Newman"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
import{stdin as I}from"process";import w from"better-sqlite3";import{join as E,dirname as k,basename as W}from"path";import{homedir as O}from"os";import{existsSync as K,mkdirSync as x}from"fs";import{fileURLToPath as U}from"url";function M(){return typeof __dirname<"u"?__dirname:k(U(import.meta.url))}var q=M(),l=process.env.CLAUDE_MEM_DATA_DIR||E(O(),".claude-mem"),R=process.env.CLAUDE_CONFIG_DIR||E(O(),".claude"),J=E(l,"archives"),Q=E(l,"logs"),z=E(l,"trash"),Z=E(l,"backups"),ee=E(l,"settings.json"),f=E(l,"claude-mem.db"),se=E(l,"vector-db"),te=E(R,"settings.json"),re=E(R,"commands"),ne=E(R,"CLAUDE.md");function L(p){x(p,{recursive:!0})}var h=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(h||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=h[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=h[e].padEnd(5),d=s.padEnd(6),_="";r?.correlationId?_=`[${r.correlationId}] `:r?.sessionId&&(_=`[session-${r.sessionId}] `);let u="";n!=null&&(this.level===0&&typeof n=="object"?u=`
|
||||
`+JSON.stringify(n,null,2):u=" "+this.formatData(n));let T="";if(r){let{sessionId:m,sdkSessionId:S,correlationId:c,...a}=r;Object.keys(a).length>0&&(T=` {${Object.entries(a).map(([y,D])=>`${y}=${D}`).join(", ")}}`)}let b=`[${o}] [${i}] [${d}] ${_}${t}${T}${u}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},A=new N;var g=class{db;constructor(){L(l),this.db=new w(f),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
import{stdin as I}from"process";import w from"better-sqlite3";import{join as E,dirname as k,basename as G}from"path";import{homedir as O}from"os";import{existsSync as K,mkdirSync as x}from"fs";import{fileURLToPath as U}from"url";function M(){return typeof __dirname<"u"?__dirname:k(U(import.meta.url))}var q=M(),m=process.env.CLAUDE_MEM_DATA_DIR||E(O(),".claude-mem"),R=process.env.CLAUDE_CONFIG_DIR||E(O(),".claude"),J=E(m,"archives"),Q=E(m,"logs"),z=E(m,"trash"),Z=E(m,"backups"),ee=E(m,"settings.json"),f=E(m,"claude-mem.db"),se=E(m,"vector-db"),te=E(R,"settings.json"),re=E(R,"commands"),ne=E(R,"CLAUDE.md");function L(c){x(c,{recursive:!0})}var h=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(h||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=h[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=h[e].padEnd(5),d=s.padEnd(6),_="";r?.correlationId?_=`[${r.correlationId}] `:r?.sessionId&&(_=`[session-${r.sessionId}] `);let T="";n!=null&&(this.level===0&&typeof n=="object"?T=`
|
||||
`+JSON.stringify(n,null,2):T=" "+this.formatData(n));let u="";if(r){let{sessionId:l,sdkSessionId:S,correlationId:p,...a}=r;Object.keys(a).length>0&&(u=` {${Object.entries(a).map(([y,D])=>`${y}=${D}`).join(", ")}}`)}let b=`[${o}] [${i}] [${d}] ${_}${t}${u}${T}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},A=new N;var g=class{db;constructor(){L(m),this.db=new w(f),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable(),this.ensureDiscoveryTokensColumn()}initializeSchema(){try{this.db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS schema_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
version INTEGER UNIQUE NOT NULL,
|
||||
@@ -166,7 +166,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}ensureDiscoveryTokensColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;this.db.pragma("table_info(observations)").some(o=>o.name==="discovery_tokens")||(this.db.exec("ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to observations table")),this.db.pragma("table_info(session_summaries)").some(o=>o.name==="discovery_tokens")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString())}catch(e){console.error("[SessionStore] Discovery tokens migration error:",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
files_read, files_edited, notes, prompt_number, created_at
|
||||
@@ -312,29 +312,29 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO user_prompts
|
||||
(claude_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r,n=0){let o=new Date,i=o.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
`).run(e,e,s,o.toISOString(),i),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(u.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n,o.toISOString(),i);return{id:Number(u.lastInsertRowid),createdAtEpoch:i}}storeSummary(e,s,t,r,n=0){let o=new Date,i=o.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
`).run(e,e,s,o.toISOString(),i),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
(sdk_session_id, project, request, investigated, learned, completed,
|
||||
next_steps, notes, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(u.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
next_steps, notes, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n,o.toISOString(),i);return{id:Number(u.lastInsertRowid),createdAtEpoch:i}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
@@ -357,7 +357,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
WHERE up.id IN (${i})
|
||||
ORDER BY up.created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],d,_;if(e!==null){let m=`
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],d,_;if(e!==null){let l=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id <= ? ${o}
|
||||
@@ -369,7 +369,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
WHERE id >= ? ${o}
|
||||
ORDER BY id ASC
|
||||
LIMIT ?
|
||||
`;try{let c=this.db.prepare(m).all(e,...i,t+1),a=this.db.prepare(S).all(e,...i,r+1);if(c.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=c.length>0?c[c.length-1].created_at_epoch:s,_=a.length>0?a[a.length-1].created_at_epoch:s}catch(c){return console.error("[SessionStore] Error getting boundary observations:",c.message),{observations:[],sessions:[],prompts:[]}}}else{let m=`
|
||||
`;try{let p=this.db.prepare(l).all(e,...i,t+1),a=this.db.prepare(S).all(e,...i,r+1);if(p.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=p.length>0?p[p.length-1].created_at_epoch:s,_=a.length>0?a[a.length-1].created_at_epoch:s}catch(p){return console.error("[SessionStore] Error getting boundary observations:",p.message),{observations:[],sessions:[],prompts:[]}}}else{let l=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch <= ? ${o}
|
||||
@@ -381,12 +381,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
WHERE created_at_epoch >= ? ${o}
|
||||
ORDER BY created_at_epoch ASC
|
||||
LIMIT ?
|
||||
`;try{let c=this.db.prepare(m).all(s,...i,t),a=this.db.prepare(S).all(s,...i,r+1);if(c.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=c.length>0?c[c.length-1].created_at_epoch:s,_=a.length>0?a[a.length-1].created_at_epoch:s}catch(c){return console.error("[SessionStore] Error getting boundary timestamps:",c.message),{observations:[],sessions:[],prompts:[]}}}let u=`
|
||||
`;try{let p=this.db.prepare(l).all(s,...i,t),a=this.db.prepare(S).all(s,...i,r+1);if(p.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=p.length>0?p[p.length-1].created_at_epoch:s,_=a.length>0?a[a.length-1].created_at_epoch:s}catch(p){return console.error("[SessionStore] Error getting boundary timestamps:",p.message),{observations:[],sessions:[],prompts:[]}}}let T=`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,T=`
|
||||
`,u=`
|
||||
SELECT *
|
||||
FROM session_summaries
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
@@ -397,5 +397,5 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${o.replace("project","s.project")}
|
||||
ORDER BY up.created_at_epoch ASC
|
||||
`;try{let m=this.db.prepare(u).all(d,_,...i),S=this.db.prepare(T).all(d,_,...i),c=this.db.prepare(b).all(d,_,...i);return{observations:m,sessions:S.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:c.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(m){return console.error("[SessionStore] Error querying timeline records:",m.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};import X from"path";import{homedir as F}from"os";import{existsSync as B,readFileSync as j}from"fs";function C(){try{let p=X.join(F(),".claude-mem","settings.json");if(B(p)){let e=JSON.parse(j(p,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function v(p){console.error("[claude-mem cleanup] Hook fired",{input:p?{session_id:p.session_id,cwd:p.cwd,reason:p.reason}:null}),p||(console.log("No input provided - this script is designed to run as a Claude Code SessionEnd hook"),console.log(`
|
||||
Expected input format:`),console.log(JSON.stringify({session_id:"string",cwd:"string",transcript_path:"string",hook_event_name:"SessionEnd",reason:"exit"},null,2)),process.exit(0));let{session_id:e,reason:s}=p;console.error("[claude-mem cleanup] Searching for active SDK session",{session_id:e,reason:s});let t=new g,r=t.findActiveSDKSession(e);r||(console.error("[claude-mem cleanup] No active SDK session found",{session_id:e}),t.close(),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)),console.error("[claude-mem cleanup] Active SDK session found",{session_id:r.id,sdk_session_id:r.sdk_session_id,project:r.project,worker_port:r.worker_port}),t.markSessionCompleted(r.id),console.error("[claude-mem cleanup] Session marked as completed in database"),t.close();try{let n=r.worker_port||C();await fetch(`http://127.0.0.1:${n}/sessions/${r.id}/complete`,{method:"POST",signal:AbortSignal.timeout(1e3)}),console.error("[claude-mem cleanup] Worker notified to stop processing indicator")}catch(n){console.error("[claude-mem cleanup] Failed to notify worker (non-critical):",n)}console.error("[claude-mem cleanup] Cleanup completed successfully"),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)}if(I.isTTY)v(void 0);else{let p="";I.on("data",e=>p+=e),I.on("end",async()=>{let e=p?JSON.parse(p):void 0;await v(e)})}
|
||||
`;try{let l=this.db.prepare(T).all(d,_,...i),S=this.db.prepare(u).all(d,_,...i),p=this.db.prepare(b).all(d,_,...i);return{observations:l,sessions:S.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:p.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(l){return console.error("[SessionStore] Error querying timeline records:",l.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};import F from"path";import{homedir as X}from"os";import{existsSync as B,readFileSync as H}from"fs";function v(){try{let c=F.join(X(),".claude-mem","settings.json");if(B(c)){let e=JSON.parse(H(c,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function C(c){console.error("[claude-mem cleanup] Hook fired",{input:c?{session_id:c.session_id,cwd:c.cwd,reason:c.reason}:null}),c||(console.log("No input provided - this script is designed to run as a Claude Code SessionEnd hook"),console.log(`
|
||||
Expected input format:`),console.log(JSON.stringify({session_id:"string",cwd:"string",transcript_path:"string",hook_event_name:"SessionEnd",reason:"exit"},null,2)),process.exit(0));let{session_id:e,reason:s}=c;console.error("[claude-mem cleanup] Searching for active SDK session",{session_id:e,reason:s});let t=new g,r=t.findActiveSDKSession(e);r||(console.error("[claude-mem cleanup] No active SDK session found",{session_id:e}),t.close(),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)),console.error("[claude-mem cleanup] Active SDK session found",{session_id:r.id,sdk_session_id:r.sdk_session_id,project:r.project,worker_port:r.worker_port}),t.markSessionCompleted(r.id),console.error("[claude-mem cleanup] Session marked as completed in database"),t.close();try{let n=r.worker_port||v();await fetch(`http://127.0.0.1:${n}/sessions/${r.id}/complete`,{method:"POST",signal:AbortSignal.timeout(1e3)}),console.error("[claude-mem cleanup] Worker notified to stop processing indicator")}catch(n){console.error("[claude-mem cleanup] Failed to notify worker (non-critical):",n)}console.error("[claude-mem cleanup] Cleanup completed successfully"),console.log('{"continue": true, "suppressOutput": true}'),process.exit(0)}if(I.isTTY)C(void 0);else{let c="";I.on("data",e=>c+=e),I.on("end",async()=>{let e=c?JSON.parse(c):void 0;await C(e)})}
|
||||
|
||||
File diff suppressed because one or more lines are too long
+25
-25
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
import Y from"path";import{stdin as D}from"process";import F from"better-sqlite3";import{join as E,dirname as U,basename as J}from"path";import{homedir as f}from"os";import{existsSync as ee,mkdirSync as M}from"fs";import{fileURLToPath as w}from"url";function X(){return typeof __dirname<"u"?__dirname:U(w(import.meta.url))}var te=X(),m=process.env.CLAUDE_MEM_DATA_DIR||E(f(),".claude-mem"),h=process.env.CLAUDE_CONFIG_DIR||E(f(),".claude"),re=E(m,"archives"),ne=E(m,"logs"),oe=E(m,"trash"),ie=E(m,"backups"),ae=E(m,"settings.json"),L=E(m,"claude-mem.db"),pe=E(m,"vector-db"),de=E(h,"settings.json"),ce=E(h,"commands"),_e=E(h,"CLAUDE.md");function A(p){M(p,{recursive:!0})}var N=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(N||{}),O=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=N[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=N[e].padEnd(5),d=s.padEnd(6),u="";r?.correlationId?u=`[${r.correlationId}] `:r?.sessionId&&(u=`[session-${r.sessionId}] `);let c="";n!=null&&(this.level===0&&typeof n=="object"?c=`
|
||||
`+JSON.stringify(n,null,2):c=" "+this.formatData(n));let l="";if(r){let{sessionId:T,sdkSessionId:b,correlationId:_,...a}=r;Object.keys(a).length>0&&(l=` {${Object.entries(a).map(([k,x])=>`${k}=${x}`).join(", ")}}`)}let S=`[${o}] [${i}] [${d}] ${u}${t}${l}${c}`;e===3?console.error(S):console.log(S)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},C=new O;var g=class{db;constructor(){A(m),this.db=new F(L),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
import Y from"path";import{stdin as D}from"process";import X from"better-sqlite3";import{join as m,dirname as U,basename as J}from"path";import{homedir as f}from"os";import{existsSync as ee,mkdirSync as M}from"fs";import{fileURLToPath as w}from"url";function F(){return typeof __dirname<"u"?__dirname:U(w(import.meta.url))}var te=F(),l=process.env.CLAUDE_MEM_DATA_DIR||m(f(),".claude-mem"),h=process.env.CLAUDE_CONFIG_DIR||m(f(),".claude"),re=m(l,"archives"),ne=m(l,"logs"),oe=m(l,"trash"),ie=m(l,"backups"),ae=m(l,"settings.json"),L=m(l,"claude-mem.db"),de=m(l,"vector-db"),pe=m(h,"settings.json"),ce=m(h,"commands"),_e=m(h,"CLAUDE.md");function A(d){M(d,{recursive:!0})}var N=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(N||{}),O=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=N[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=N[e].padEnd(5),p=s.padEnd(6),u="";r?.correlationId?u=`[${r.correlationId}] `:r?.sessionId&&(u=`[session-${r.sessionId}] `);let c="";n!=null&&(this.level===0&&typeof n=="object"?c=`
|
||||
`+JSON.stringify(n,null,2):c=" "+this.formatData(n));let E="";if(r){let{sessionId:T,sdkSessionId:S,correlationId:_,...a}=r;Object.keys(a).length>0&&(E=` {${Object.entries(a).map(([k,x])=>`${k}=${x}`).join(", ")}}`)}let b=`[${o}] [${i}] [${p}] ${u}${t}${E}${c}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},v=new O;var R=class{db;constructor(){A(l),this.db=new X(L),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable(),this.ensureDiscoveryTokensColumn()}initializeSchema(){try{this.db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS schema_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
version INTEGER UNIQUE NOT NULL,
|
||||
@@ -63,7 +63,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_sdk_session ON session_summaries(sdk_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_project ON session_summaries(project);
|
||||
CREATE INDEX IF NOT EXISTS idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
|
||||
`),this.db.prepare("INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)").run(4,new Date().toISOString()),console.error("[SessionStore] Migration004 applied successfully"))}catch(e){throw console.error("[SessionStore] Schema initialization error:",e.message),e}}ensureWorkerPortColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(5))return;this.db.pragma("table_info(sdk_sessions)").some(r=>r.name==="worker_port")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN worker_port INTEGER"),console.error("[SessionStore] Added worker_port column to sdk_sessions table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(5,new Date().toISOString())}catch(e){console.error("[SessionStore] Migration error:",e.message)}}ensurePromptTrackingColumns(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(6))return;this.db.pragma("table_info(sdk_sessions)").some(d=>d.name==="prompt_counter")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN prompt_counter INTEGER DEFAULT 0"),console.error("[SessionStore] Added prompt_counter column to sdk_sessions table")),this.db.pragma("table_info(observations)").some(d=>d.name==="prompt_number")||(this.db.exec("ALTER TABLE observations ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to observations table")),this.db.pragma("table_info(session_summaries)").some(d=>d.name==="prompt_number")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(6,new Date().toISOString())}catch(e){console.error("[SessionStore] Prompt tracking migration error:",e.message)}}removeSessionSummariesUniqueConstraint(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;if(!this.db.pragma("index_list(session_summaries)").some(r=>r.unique===1)){this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString());return}console.error("[SessionStore] Removing UNIQUE constraint from session_summaries.sdk_session_id..."),this.db.exec("BEGIN TRANSACTION");try{this.db.exec(`
|
||||
`),this.db.prepare("INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)").run(4,new Date().toISOString()),console.error("[SessionStore] Migration004 applied successfully"))}catch(e){throw console.error("[SessionStore] Schema initialization error:",e.message),e}}ensureWorkerPortColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(5))return;this.db.pragma("table_info(sdk_sessions)").some(r=>r.name==="worker_port")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN worker_port INTEGER"),console.error("[SessionStore] Added worker_port column to sdk_sessions table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(5,new Date().toISOString())}catch(e){console.error("[SessionStore] Migration error:",e.message)}}ensurePromptTrackingColumns(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(6))return;this.db.pragma("table_info(sdk_sessions)").some(p=>p.name==="prompt_counter")||(this.db.exec("ALTER TABLE sdk_sessions ADD COLUMN prompt_counter INTEGER DEFAULT 0"),console.error("[SessionStore] Added prompt_counter column to sdk_sessions table")),this.db.pragma("table_info(observations)").some(p=>p.name==="prompt_number")||(this.db.exec("ALTER TABLE observations ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to observations table")),this.db.pragma("table_info(session_summaries)").some(p=>p.name==="prompt_number")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN prompt_number INTEGER"),console.error("[SessionStore] Added prompt_number column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(6,new Date().toISOString())}catch(e){console.error("[SessionStore] Prompt tracking migration error:",e.message)}}removeSessionSummariesUniqueConstraint(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;if(!this.db.pragma("index_list(session_summaries)").some(r=>r.unique===1)){this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString());return}console.error("[SessionStore] Removing UNIQUE constraint from session_summaries.sdk_session_id..."),this.db.exec("BEGIN TRANSACTION");try{this.db.exec(`
|
||||
CREATE TABLE session_summaries_new (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
sdk_session_id TEXT NOT NULL,
|
||||
@@ -166,7 +166,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}ensureDiscoveryTokensColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;this.db.pragma("table_info(observations)").some(o=>o.name==="discovery_tokens")||(this.db.exec("ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to observations table")),this.db.pragma("table_info(session_summaries)").some(o=>o.name==="discovery_tokens")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString())}catch(e){console.error("[SessionStore] Discovery tokens migration error:",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
files_read, files_edited, notes, prompt_number, created_at
|
||||
@@ -262,7 +262,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT files_read, files_modified
|
||||
FROM observations
|
||||
WHERE sdk_session_id = ?
|
||||
`).all(e),r=new Set,n=new Set;for(let o of t){if(o.files_read)try{let i=JSON.parse(o.files_read);Array.isArray(i)&&i.forEach(d=>r.add(d))}catch{}if(o.files_modified)try{let i=JSON.parse(o.files_modified);Array.isArray(i)&&i.forEach(d=>n.add(d))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
|
||||
`).all(e),r=new Set,n=new Set;for(let o of t){if(o.files_read)try{let i=JSON.parse(o.files_read);Array.isArray(i)&&i.forEach(p=>r.add(p))}catch{}if(o.files_modified)try{let i=JSON.parse(o.files_modified);Array.isArray(i)&&i.forEach(p=>n.add(p))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
|
||||
SELECT id, claude_session_id, sdk_session_id, project, user_prompt
|
||||
FROM sdk_sessions
|
||||
WHERE id = ?
|
||||
@@ -299,7 +299,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
UPDATE sdk_sessions
|
||||
SET sdk_session_id = ?
|
||||
WHERE id = ? AND sdk_session_id IS NULL
|
||||
`).run(s,e).changes===0?(C.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
|
||||
`).run(s,e).changes===0?(v.debug("DB","sdk_session_id already set, skipping update",{sessionId:e,sdkSessionId:s}),!1):!0}setWorkerPort(e,s){this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET worker_port = ?
|
||||
WHERE id = ?
|
||||
@@ -312,29 +312,29 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO user_prompts
|
||||
(claude_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r,n=0){let o=new Date,i=o.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
|
||||
`).run(e,e,s,o.toISOString(),i),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(c.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n,o.toISOString(),i);return{id:Number(E.lastInsertRowid),createdAtEpoch:i}}storeSummary(e,s,t,r,n=0){let o=new Date,i=o.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
|
||||
`).run(e,e,s,o.toISOString(),i),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
(sdk_session_id, project, request, investigated, learned, completed,
|
||||
next_steps, notes, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(c.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
next_steps, notes, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n,o.toISOString(),i);return{id:Number(E.lastInsertRowid),createdAtEpoch:i}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
@@ -357,47 +357,47 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
WHERE up.id IN (${i})
|
||||
ORDER BY up.created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],d,u;if(e!==null){let T=`
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],p,u;if(e!==null){let T=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id <= ? ${o}
|
||||
ORDER BY id DESC
|
||||
LIMIT ?
|
||||
`,b=`
|
||||
`,S=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id >= ? ${o}
|
||||
ORDER BY id ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(T).all(e,...i,t+1),a=this.db.prepare(b).all(e,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
|
||||
`;try{let _=this.db.prepare(T).all(e,...i,t+1),a=this.db.prepare(S).all(e,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch <= ? ${o}
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT ?
|
||||
`,b=`
|
||||
`,S=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? ${o}
|
||||
ORDER BY created_at_epoch ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(T).all(s,...i,t),a=this.db.prepare(b).all(s,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let c=`
|
||||
`;try{let _=this.db.prepare(T).all(s,...i,t),a=this.db.prepare(S).all(s,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let c=`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,l=`
|
||||
`,E=`
|
||||
SELECT *
|
||||
FROM session_summaries
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,S=`
|
||||
`,b=`
|
||||
SELECT up.*, s.project, s.sdk_session_id
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${o.replace("project","s.project")}
|
||||
ORDER BY up.created_at_epoch ASC
|
||||
`;try{let T=this.db.prepare(c).all(d,u,...i),b=this.db.prepare(l).all(d,u,...i),_=this.db.prepare(S).all(d,u,...i);return{observations:T,sessions:b.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:_.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function P(p,e,s){return p==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:p==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:p==="UserPromptSubmit"||p==="PostToolUse"?{continue:!0,suppressOutput:!0}:p==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function v(p,e,s={}){let t=P(p,e,s);return JSON.stringify(t)}import H from"path";import{homedir as B}from"os";import{existsSync as j,readFileSync as $}from"fs";var W=100;function R(){try{let p=H.join(B(),".claude-mem","settings.json");if(j(p)){let e=JSON.parse($(p,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function G(){try{let p=R();return(await fetch(`http://127.0.0.1:${p}/health`,{signal:AbortSignal.timeout(W)})).ok}catch{return!1}}async function y(){if(await G())return;let p=R();throw new Error(`Worker service is not responding on port ${p}.
|
||||
`;try{let T=this.db.prepare(c).all(p,u,...i),S=this.db.prepare(E).all(p,u,...i),_=this.db.prepare(b).all(p,u,...i);return{observations:T,sessions:S.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:_.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function H(d,e,s){return d==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:d==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:d==="UserPromptSubmit"||d==="PostToolUse"?{continue:!0,suppressOutput:!0}:d==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function C(d,e,s={}){let t=H(d,e,s);return JSON.stringify(t)}import P from"path";import{homedir as B}from"os";import{existsSync as j,readFileSync as $}from"fs";var G=100;function g(){try{let d=P.join(B(),".claude-mem","settings.json");if(j(d)){let e=JSON.parse($(d,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function W(){try{let d=g();return(await fetch(`http://127.0.0.1:${d}/health`,{signal:AbortSignal.timeout(G)})).ok}catch{return!1}}async function y(){if(await W())return;let d=g();throw new Error(`Worker service is not responding on port ${d}.
|
||||
|
||||
If you just updated the plugin, PM2's watch mode should restart automatically.
|
||||
If the problem persists, run: pm2 restart claude-mem-worker`)}async function K(p){if(!p)throw new Error("newHook requires input");let{session_id:e,cwd:s,prompt:t}=p,r=Y.basename(s);await y();let n=new g,o=n.createSDKSession(e,r,t),i=n.incrementPromptCounter(o);n.saveUserPrompt(e,i,t),console.error(`[new-hook] Session ${o}, prompt #${i}`),n.close();let d=R(),u=t.startsWith("/")?t.substring(1):t;try{let c=await fetch(`http://127.0.0.1:${d}/sessions/${o}/init`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({project:r,userPrompt:u,promptNumber:i}),signal:AbortSignal.timeout(5e3)});if(!c.ok){let l=await c.text();throw new Error(`Failed to initialize session: ${c.status} ${l}`)}}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(v("UserPromptSubmit",!0))}var I="";D.on("data",p=>I+=p);D.on("end",async()=>{let p=I?JSON.parse(I):void 0;await K(p)});
|
||||
If the problem persists, run: pm2 restart claude-mem-worker`)}async function K(d){if(!d)throw new Error("newHook requires input");let{session_id:e,cwd:s,prompt:t}=d,r=Y.basename(s);await y();let n=new R,o=n.createSDKSession(e,r,t),i=n.incrementPromptCounter(o);n.saveUserPrompt(e,i,t),console.error(`[new-hook] Session ${o}, prompt #${i}`),n.close();let p=g(),u=t.startsWith("/")?t.substring(1):t;try{let c=await fetch(`http://127.0.0.1:${p}/sessions/${o}/init`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({project:r,userPrompt:u,promptNumber:i}),signal:AbortSignal.timeout(5e3)});if(!c.ok){let E=await c.text();throw new Error(`Failed to initialize session: ${c.status} ${E}`)}}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(C("UserPromptSubmit",!0))}var I="";D.on("data",d=>I+=d);D.on("end",async()=>{let d=I?JSON.parse(I):void 0;await K(d)});
|
||||
|
||||
+43
-43
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
import{stdin as D}from"process";import X from"better-sqlite3";import{join as m,dirname as U,basename as J}from"path";import{homedir as A}from"os";import{existsSync as ee,mkdirSync as M}from"fs";import{fileURLToPath as w}from"url";function F(){return typeof __dirname<"u"?__dirname:U(w(import.meta.url))}var te=F(),l=process.env.CLAUDE_MEM_DATA_DIR||m(A(),".claude-mem"),N=process.env.CLAUDE_CONFIG_DIR||m(A(),".claude"),re=m(l,"archives"),ne=m(l,"logs"),oe=m(l,"trash"),ie=m(l,"backups"),ae=m(l,"settings.json"),C=m(l,"claude-mem.db"),de=m(l,"vector-db"),pe=m(N,"settings.json"),ce=m(N,"commands"),_e=m(N,"CLAUDE.md");function v(d){M(d,{recursive:!0})}var O=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(O||{}),I=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=O[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let o=new Date().toISOString().replace("T"," ").substring(0,23),i=O[e].padEnd(5),p=s.padEnd(6),u="";r?.correlationId?u=`[${r.correlationId}] `:r?.sessionId&&(u=`[session-${r.sessionId}] `);let E="";n!=null&&(this.level===0&&typeof n=="object"?E=`
|
||||
`+JSON.stringify(n,null,2):E=" "+this.formatData(n));let c="";if(r){let{sessionId:T,sdkSessionId:g,correlationId:_,...a}=r;Object.keys(a).length>0&&(c=` {${Object.entries(a).map(([k,x])=>`${k}=${x}`).join(", ")}}`)}let b=`[${o}] [${i}] [${p}] ${u}${t}${c}${E}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},S=new I;var R=class{db;constructor(){v(l),this.db=new X(C),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
import{stdin as D}from"process";import X from"better-sqlite3";import{join as m,dirname as U,basename as J}from"path";import{homedir as A}from"os";import{existsSync as ee,mkdirSync as M}from"fs";import{fileURLToPath as w}from"url";function F(){return typeof __dirname<"u"?__dirname:U(w(import.meta.url))}var te=F(),T=process.env.CLAUDE_MEM_DATA_DIR||m(A(),".claude-mem"),N=process.env.CLAUDE_CONFIG_DIR||m(A(),".claude"),re=m(T,"archives"),oe=m(T,"logs"),ne=m(T,"trash"),ie=m(T,"backups"),ae=m(T,"settings.json"),v=m(T,"claude-mem.db"),de=m(T,"vector-db"),pe=m(N,"settings.json"),ce=m(N,"commands"),_e=m(N,"CLAUDE.md");function C(d){M(d,{recursive:!0})}var O=(o=>(o[o.DEBUG=0]="DEBUG",o[o.INFO=1]="INFO",o[o.WARN=2]="WARN",o[o.ERROR=3]="ERROR",o[o.SILENT=4]="SILENT",o))(O||{}),I=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=O[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,o){if(e<this.level)return;let i=new Date().toISOString().replace("T"," ").substring(0,23),n=O[e].padEnd(5),p=s.padEnd(6),u="";r?.correlationId?u=`[${r.correlationId}] `:r?.sessionId&&(u=`[session-${r.sessionId}] `);let E="";o!=null&&(this.level===0&&typeof o=="object"?E=`
|
||||
`+JSON.stringify(o,null,2):E=" "+this.formatData(o));let c="";if(r){let{sessionId:b,sdkSessionId:g,correlationId:_,...a}=r;Object.keys(a).length>0&&(c=` {${Object.entries(a).map(([k,x])=>`${k}=${x}`).join(", ")}}`)}let l=`[${i}] [${n}] [${p}] ${u}${t}${c}${E}`;e===3?console.error(l):console.log(l)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},S=new I;var R=class{db;constructor(){C(T),this.db=new X(v),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable(),this.ensureDiscoveryTokensColumn()}initializeSchema(){try{this.db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS schema_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
version INTEGER UNIQUE NOT NULL,
|
||||
@@ -166,7 +166,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}ensureDiscoveryTokensColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;this.db.pragma("table_info(observations)").some(i=>i.name==="discovery_tokens")||(this.db.exec("ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to observations table")),this.db.pragma("table_info(session_summaries)").some(i=>i.name==="discovery_tokens")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString())}catch(e){console.error("[SessionStore] Discovery tokens migration error:",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
files_read, files_edited, notes, prompt_number, created_at
|
||||
@@ -244,12 +244,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE id = ?
|
||||
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,o=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",n=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE id IN (${i})
|
||||
ORDER BY created_at_epoch ${n}
|
||||
${o}
|
||||
WHERE id IN (${n})
|
||||
ORDER BY created_at_epoch ${o}
|
||||
${i}
|
||||
`).all(...e)}getSummaryForSession(e){return this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
@@ -262,7 +262,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT files_read, files_modified
|
||||
FROM observations
|
||||
WHERE sdk_session_id = ?
|
||||
`).all(e),r=new Set,n=new Set;for(let o of t){if(o.files_read)try{let i=JSON.parse(o.files_read);Array.isArray(i)&&i.forEach(p=>r.add(p))}catch{}if(o.files_modified)try{let i=JSON.parse(o.files_modified);Array.isArray(i)&&i.forEach(p=>n.add(p))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
|
||||
`).all(e),r=new Set,o=new Set;for(let i of t){if(i.files_read)try{let n=JSON.parse(i.files_read);Array.isArray(n)&&n.forEach(p=>r.add(p))}catch{}if(i.files_modified)try{let n=JSON.parse(i.files_modified);Array.isArray(n)&&n.forEach(p=>o.add(p))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(o)}}getSessionById(e){return this.db.prepare(`
|
||||
SELECT id, claude_session_id, sdk_session_id, project, user_prompt
|
||||
FROM sdk_sessions
|
||||
WHERE id = ?
|
||||
@@ -289,13 +289,13 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
|
||||
`).get(e)?.prompt_counter||1}getPromptCounter(e){return this.db.prepare(`
|
||||
SELECT prompt_counter FROM sdk_sessions WHERE id = ?
|
||||
`).get(e)?.prompt_counter||0}createSDKSession(e,s,t){let r=new Date,n=r.getTime(),i=this.db.prepare(`
|
||||
`).get(e)?.prompt_counter||0}createSDKSession(e,s,t){let r=new Date,o=r.getTime(),n=this.db.prepare(`
|
||||
INSERT OR IGNORE INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, user_prompt, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,t,r.toISOString(),n);return i.lastInsertRowid===0||i.changes===0?this.db.prepare(`
|
||||
`).run(e,e,s,t,r.toISOString(),o);return n.lastInsertRowid===0||n.changes===0?this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE claude_session_id = ? LIMIT 1
|
||||
`).get(e).id:i.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
|
||||
`).get(e).id:n.lastInsertRowid}updateSDKSessionId(e,s){return this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET sdk_session_id = ?
|
||||
WHERE id = ? AND sdk_session_id IS NULL
|
||||
@@ -308,33 +308,33 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
FROM sdk_sessions
|
||||
WHERE id = ?
|
||||
LIMIT 1
|
||||
`).get(e)?.worker_port||null}saveUserPrompt(e,s,t){let r=new Date,n=r.getTime();return this.db.prepare(`
|
||||
`).get(e)?.worker_port||null}saveUserPrompt(e,s,t){let r=new Date,o=r.getTime();return this.db.prepare(`
|
||||
INSERT INTO user_prompts
|
||||
(claude_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
`).run(e,s,t,r.toISOString(),o).lastInsertRowid}storeObservation(e,s,t,r,o=0){let i=new Date,n=i.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
`).run(e,e,s,i.toISOString(),n),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r){let n=new Date,o=n.getTime();this.db.prepare(`
|
||||
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,o,i.toISOString(),n);return{id:Number(c.lastInsertRowid),createdAtEpoch:n}}storeSummary(e,s,t,r,o=0){let i=new Date,n=i.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
`).run(e,e,s,i.toISOString(),n),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let c=this.db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
(sdk_session_id, project, request, investigated, learned, completed,
|
||||
next_steps, notes, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
next_steps, notes, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,o,i.toISOString(),n);return{id:Number(c.lastInsertRowid),createdAtEpoch:n}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
@@ -342,62 +342,62 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,o=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",n=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT * FROM session_summaries
|
||||
WHERE id IN (${i})
|
||||
ORDER BY created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",o=r?`LIMIT ${r}`:"",i=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
WHERE id IN (${n})
|
||||
ORDER BY created_at_epoch ${o}
|
||||
${i}
|
||||
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,o=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",n=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT
|
||||
up.*,
|
||||
s.project,
|
||||
s.sdk_session_id
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.id IN (${i})
|
||||
ORDER BY up.created_at_epoch ${n}
|
||||
${o}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let o=n?"AND project = ?":"",i=n?[n]:[],p,u;if(e!==null){let T=`
|
||||
WHERE up.id IN (${n})
|
||||
ORDER BY up.created_at_epoch ${o}
|
||||
${i}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,o){let i=o?"AND project = ?":"",n=o?[o]:[],p,u;if(e!==null){let b=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id <= ? ${o}
|
||||
WHERE id <= ? ${i}
|
||||
ORDER BY id DESC
|
||||
LIMIT ?
|
||||
`,g=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id >= ? ${o}
|
||||
WHERE id >= ? ${i}
|
||||
ORDER BY id ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(T).all(e,...i,t+1),a=this.db.prepare(g).all(e,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
|
||||
`;try{let _=this.db.prepare(b).all(e,...n,t+1),a=this.db.prepare(g).all(e,...n,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let b=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch <= ? ${o}
|
||||
WHERE created_at_epoch <= ? ${i}
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT ?
|
||||
`,g=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? ${o}
|
||||
WHERE created_at_epoch >= ? ${i}
|
||||
ORDER BY created_at_epoch ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(T).all(s,...i,t),a=this.db.prepare(g).all(s,...i,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let E=`
|
||||
`;try{let _=this.db.prepare(b).all(s,...n,t),a=this.db.prepare(g).all(s,...n,r+1);if(_.length===0&&a.length===0)return{observations:[],sessions:[],prompts:[]};p=_.length>0?_[_.length-1].created_at_epoch:s,u=a.length>0?a[a.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let E=`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${i}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,c=`
|
||||
SELECT *
|
||||
FROM session_summaries
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${o}
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${i}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,b=`
|
||||
`,l=`
|
||||
SELECT up.*, s.project, s.sdk_session_id
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${o.replace("project","s.project")}
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${i.replace("project","s.project")}
|
||||
ORDER BY up.created_at_epoch ASC
|
||||
`;try{let T=this.db.prepare(E).all(p,u,...i),g=this.db.prepare(c).all(p,u,...i),_=this.db.prepare(b).all(p,u,...i);return{observations:T,sessions:g.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:_.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function H(d,e,s){return d==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:d==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:d==="UserPromptSubmit"||d==="PostToolUse"?{continue:!0,suppressOutput:!0}:d==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function f(d,e,s={}){let t=H(d,e,s);return JSON.stringify(t)}import P from"path";import{homedir as B}from"os";import{existsSync as j,readFileSync as $}from"fs";var W=100;function h(){try{let d=P.join(B(),".claude-mem","settings.json");if(j(d)){let e=JSON.parse($(d,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function G(){try{let d=h();return(await fetch(`http://127.0.0.1:${d}/health`,{signal:AbortSignal.timeout(W)})).ok}catch{return!1}}async function y(){if(await G())return;let d=h();throw new Error(`Worker service is not responding on port ${d}.
|
||||
`;try{let b=this.db.prepare(E).all(p,u,...n),g=this.db.prepare(c).all(p,u,...n),_=this.db.prepare(l).all(p,u,...n);return{observations:b,sessions:g.map(a=>({id:a.id,sdk_session_id:a.sdk_session_id,project:a.project,request:a.request,completed:a.completed,next_steps:a.next_steps,created_at:a.created_at,created_at_epoch:a.created_at_epoch})),prompts:_.map(a=>({id:a.id,claude_session_id:a.claude_session_id,project:a.project,prompt:a.prompt_text,created_at:a.created_at,created_at_epoch:a.created_at_epoch}))}}catch(b){return console.error("[SessionStore] Error querying timeline records:",b.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function H(d,e,s){return d==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:d==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:d==="UserPromptSubmit"||d==="PostToolUse"?{continue:!0,suppressOutput:!0}:d==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function f(d,e,s={}){let t=H(d,e,s);return JSON.stringify(t)}import P from"path";import{homedir as B}from"os";import{existsSync as j,readFileSync as G}from"fs";var W=100;function h(){try{let d=P.join(B(),".claude-mem","settings.json");if(j(d)){let e=JSON.parse(G(d,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function $(){try{let d=h();return(await fetch(`http://127.0.0.1:${d}/health`,{signal:AbortSignal.timeout(W)})).ok}catch{return!1}}async function y(){if(await $())return;let d=h();throw new Error(`Worker service is not responding on port ${d}.
|
||||
|
||||
If you just updated the plugin, PM2's watch mode should restart automatically.
|
||||
If the problem persists, run: pm2 restart claude-mem-worker`)}var Y=new Set(["ListMcpResourcesTool","SlashCommand","Skill","TodoWrite","AskUserQuestion"]);async function K(d){if(!d)throw new Error("saveHook requires input");let{session_id:e,cwd:s,tool_name:t,tool_input:r,tool_response:n}=d;if(Y.has(t)){console.log(f("PostToolUse",!0));return}await y();let o=new R,i=o.createSDKSession(e,"",""),p=o.getPromptCounter(i);o.close();let u=S.formatTool(t,r),E=h();S.dataIn("HOOK",`PostToolUse: ${u}`,{sessionId:i,workerPort:E});try{let c=await fetch(`http://127.0.0.1:${E}/sessions/${i}/observations`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({tool_name:t,tool_input:r!==void 0?JSON.stringify(r):"{}",tool_response:n!==void 0?JSON.stringify(n):"{}",prompt_number:p,cwd:s||""}),signal:AbortSignal.timeout(2e3)});if(!c.ok){let b=await c.text();throw S.failure("HOOK","Failed to send observation",{sessionId:i,status:c.status},b),new Error(`Failed to send observation to worker: ${c.status} ${b}`)}S.debug("HOOK","Observation sent successfully",{sessionId:i,toolName:t})}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(f("PostToolUse",!0))}var L="";D.on("data",d=>L+=d);D.on("end",async()=>{let d=L?JSON.parse(L):void 0;await K(d)});
|
||||
If the problem persists, run: pm2 restart claude-mem-worker`)}var Y=new Set(["ListMcpResourcesTool","SlashCommand","Skill","TodoWrite","AskUserQuestion"]);async function K(d){if(!d)throw new Error("saveHook requires input");let{session_id:e,cwd:s,tool_name:t,tool_input:r,tool_response:o}=d;if(Y.has(t)){console.log(f("PostToolUse",!0));return}await y();let i=new R,n=i.createSDKSession(e,"",""),p=i.getPromptCounter(n);i.close();let u=S.formatTool(t,r),E=h();S.dataIn("HOOK",`PostToolUse: ${u}`,{sessionId:n,workerPort:E});try{let c=await fetch(`http://127.0.0.1:${E}/sessions/${n}/observations`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({tool_name:t,tool_input:r!==void 0?JSON.stringify(r):"{}",tool_response:o!==void 0?JSON.stringify(o):"{}",prompt_number:p,cwd:s||""}),signal:AbortSignal.timeout(2e3)});if(!c.ok){let l=await c.text();throw S.failure("HOOK","Failed to send observation",{sessionId:n,status:c.status},l),new Error(`Failed to send observation to worker: ${c.status} ${l}`)}S.debug("HOOK","Observation sent successfully",{sessionId:n,toolName:t})}catch(c){throw c.cause?.code==="ECONNREFUSED"||c.name==="TimeoutError"||c.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):c}console.log(f("PostToolUse",!0))}var L="";D.on("data",d=>L+=d);D.on("end",async()=>{let d=L?JSON.parse(L):void 0;await K(d)});
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
import{stdin as k}from"process";import{readFileSync as x,existsSync as U}from"fs";import B from"better-sqlite3";import{join as m,dirname as F,basename as ne}from"path";import{homedir as A}from"os";import{existsSync as de,mkdirSync as X}from"fs";import{fileURLToPath as H}from"url";function j(){return typeof __dirname<"u"?__dirname:F(H(import.meta.url))}var ce=j(),E=process.env.CLAUDE_MEM_DATA_DIR||m(A(),".claude-mem"),f=process.env.CLAUDE_CONFIG_DIR||m(A(),".claude"),ue=m(E,"archives"),_e=m(E,"logs"),me=m(E,"trash"),le=m(E,"backups"),Ee=m(E,"settings.json"),y=m(E,"claude-mem.db"),Te=m(E,"vector-db"),ge=m(f,"settings.json"),Se=m(f,"commands"),be=m(f,"CLAUDE.md");function C(i){X(i,{recursive:!0})}var O=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(O||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=O[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let a=new Date().toISOString().replace("T"," ").substring(0,23),o=O[e].padEnd(5),d=s.padEnd(6),c="";r?.correlationId?c=`[${r.correlationId}] `:r?.sessionId&&(c=`[session-${r.sessionId}] `);let u="";n!=null&&(this.level===0&&typeof n=="object"?u=`
|
||||
`+JSON.stringify(n,null,2):u=" "+this.formatData(n));let l="";if(r){let{sessionId:T,sdkSessionId:S,correlationId:_,...p}=r;Object.keys(p).length>0&&(l=` {${Object.entries(p).map(([M,w])=>`${M}=${w}`).join(", ")}}`)}let b=`[${a}] [${o}] [${d}] ${c}${t}${l}${u}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},g=new N;var R=class{db;constructor(){C(E),this.db=new B(y),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable()}initializeSchema(){try{this.db.exec(`
|
||||
import{stdin as k}from"process";import{readFileSync as x,existsSync as U}from"fs";import j from"better-sqlite3";import{join as m,dirname as F,basename as ne}from"path";import{homedir as y}from"os";import{existsSync as de,mkdirSync as X}from"fs";import{fileURLToPath as H}from"url";function B(){return typeof __dirname<"u"?__dirname:F(H(import.meta.url))}var ce=B(),l=process.env.CLAUDE_MEM_DATA_DIR||m(y(),".claude-mem"),f=process.env.CLAUDE_CONFIG_DIR||m(y(),".claude"),ue=m(l,"archives"),_e=m(l,"logs"),me=m(l,"trash"),Ee=m(l,"backups"),le=m(l,"settings.json"),A=m(l,"claude-mem.db"),Te=m(l,"vector-db"),ge=m(f,"settings.json"),be=m(f,"commands"),Se=m(f,"CLAUDE.md");function v(a){X(a,{recursive:!0})}var O=(n=>(n[n.DEBUG=0]="DEBUG",n[n.INFO=1]="INFO",n[n.WARN=2]="WARN",n[n.ERROR=3]="ERROR",n[n.SILENT=4]="SILENT",n))(O||{}),N=class{level;useColor;constructor(){let e=process.env.CLAUDE_MEM_LOG_LEVEL?.toUpperCase()||"INFO";this.level=O[e]??1,this.useColor=process.stdout.isTTY??!1}correlationId(e,s){return`obs-${e}-${s}`}sessionId(e){return`session-${e}`}formatData(e){if(e==null)return"";if(typeof e=="string")return e;if(typeof e=="number"||typeof e=="boolean")return e.toString();if(typeof e=="object"){if(e instanceof Error)return this.level===0?`${e.message}
|
||||
${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Object.keys(e);return s.length===0?"{}":s.length<=3?JSON.stringify(e):`{${s.length} keys: ${s.slice(0,3).join(", ")}...}`}return String(e)}formatTool(e,s){if(!s)return e;try{let t=typeof s=="string"?JSON.parse(s):s;if(e==="Bash"&&t.command){let r=t.command.length>50?t.command.substring(0,50)+"...":t.command;return`${e}(${r})`}if(e==="Read"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Edit"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}if(e==="Write"&&t.file_path){let r=t.file_path.split("/").pop()||t.file_path;return`${e}(${r})`}return e}catch{return e}}log(e,s,t,r,n){if(e<this.level)return;let i=new Date().toISOString().replace("T"," ").substring(0,23),o=O[e].padEnd(5),d=s.padEnd(6),c="";r?.correlationId?c=`[${r.correlationId}] `:r?.sessionId&&(c=`[session-${r.sessionId}] `);let u="";n!=null&&(this.level===0&&typeof n=="object"?u=`
|
||||
`+JSON.stringify(n,null,2):u=" "+this.formatData(n));let E="";if(r){let{sessionId:T,sdkSessionId:S,correlationId:_,...p}=r;Object.keys(p).length>0&&(E=` {${Object.entries(p).map(([M,w])=>`${M}=${w}`).join(", ")}}`)}let b=`[${i}] [${o}] [${d}] ${c}${t}${E}${u}`;e===3?console.error(b):console.log(b)}debug(e,s,t,r){this.log(0,e,s,t,r)}info(e,s,t,r){this.log(1,e,s,t,r)}warn(e,s,t,r){this.log(2,e,s,t,r)}error(e,s,t,r){this.log(3,e,s,t,r)}dataIn(e,s,t,r){this.info(e,`\u2192 ${s}`,t,r)}dataOut(e,s,t,r){this.info(e,`\u2190 ${s}`,t,r)}success(e,s,t,r){this.info(e,`\u2713 ${s}`,t,r)}failure(e,s,t,r){this.error(e,`\u2717 ${s}`,t,r)}timing(e,s,t,r){this.info(e,`\u23F1 ${s}`,r,{duration:`${t}ms`})}},g=new N;var R=class{db;constructor(){v(l),this.db=new j(A),this.db.pragma("journal_mode = WAL"),this.db.pragma("synchronous = NORMAL"),this.db.pragma("foreign_keys = ON"),this.initializeSchema(),this.ensureWorkerPortColumn(),this.ensurePromptTrackingColumns(),this.removeSessionSummariesUniqueConstraint(),this.addObservationHierarchicalFields(),this.makeObservationsTextNullable(),this.createUserPromptsTable(),this.ensureDiscoveryTokensColumn()}initializeSchema(){try{this.db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS schema_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
version INTEGER UNIQUE NOT NULL,
|
||||
@@ -166,7 +166,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO user_prompts_fts(rowid, prompt_text)
|
||||
VALUES (new.id, new.prompt_text);
|
||||
END;
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
`),this.db.exec("COMMIT"),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(10,new Date().toISOString()),console.error("[SessionStore] Successfully created user_prompts table with FTS5 support")}catch(t){throw this.db.exec("ROLLBACK"),t}}catch(e){console.error("[SessionStore] Migration error (create user_prompts table):",e.message)}}ensureDiscoveryTokensColumn(){try{if(this.db.prepare("SELECT version FROM schema_versions WHERE version = ?").get(7))return;this.db.pragma("table_info(observations)").some(i=>i.name==="discovery_tokens")||(this.db.exec("ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to observations table")),this.db.pragma("table_info(session_summaries)").some(i=>i.name==="discovery_tokens")||(this.db.exec("ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0"),console.error("[SessionStore] Added discovery_tokens column to session_summaries table")),this.db.prepare("INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)").run(7,new Date().toISOString())}catch(e){console.error("[SessionStore] Discovery tokens migration error:",e.message)}}getRecentSummaries(e,s=10){return this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
files_read, files_edited, notes, prompt_number, created_at
|
||||
@@ -244,12 +244,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE id = ?
|
||||
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",a=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
`).get(e)||null}getObservationsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE id IN (${o})
|
||||
ORDER BY created_at_epoch ${n}
|
||||
${a}
|
||||
${i}
|
||||
`).all(...e)}getSummaryForSession(e){return this.db.prepare(`
|
||||
SELECT
|
||||
request, investigated, learned, completed, next_steps,
|
||||
@@ -262,7 +262,7 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
SELECT files_read, files_modified
|
||||
FROM observations
|
||||
WHERE sdk_session_id = ?
|
||||
`).all(e),r=new Set,n=new Set;for(let a of t){if(a.files_read)try{let o=JSON.parse(a.files_read);Array.isArray(o)&&o.forEach(d=>r.add(d))}catch{}if(a.files_modified)try{let o=JSON.parse(a.files_modified);Array.isArray(o)&&o.forEach(d=>n.add(d))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
|
||||
`).all(e),r=new Set,n=new Set;for(let i of t){if(i.files_read)try{let o=JSON.parse(i.files_read);Array.isArray(o)&&o.forEach(d=>r.add(d))}catch{}if(i.files_modified)try{let o=JSON.parse(i.files_modified);Array.isArray(o)&&o.forEach(d=>n.add(d))}catch{}}return{filesRead:Array.from(r),filesModified:Array.from(n)}}getSessionById(e){return this.db.prepare(`
|
||||
SELECT id, claude_session_id, sdk_session_id, project, user_prompt
|
||||
FROM sdk_sessions
|
||||
WHERE id = ?
|
||||
@@ -312,29 +312,29 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
INSERT INTO user_prompts
|
||||
(claude_session_id, prompt_number, prompt_text, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r){let n=new Date,a=n.getTime();this.db.prepare(`
|
||||
`).run(e,s,t,r.toISOString(),n).lastInsertRowid}storeObservation(e,s,t,r,n=0){let i=new Date,o=i.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),a),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
`).run(e,e,s,i.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n.toISOString(),a);return{id:Number(u.lastInsertRowid),createdAtEpoch:a}}storeSummary(e,s,t,r){let n=new Date,a=n.getTime();this.db.prepare(`
|
||||
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.type,t.title,t.subtitle,JSON.stringify(t.facts),t.narrative,JSON.stringify(t.concepts),JSON.stringify(t.files_read),JSON.stringify(t.files_modified),r||null,n,i.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}storeSummary(e,s,t,r,n=0){let i=new Date,o=i.getTime();this.db.prepare(`
|
||||
SELECT id FROM sdk_sessions WHERE sdk_session_id = ?
|
||||
`).get(e)||(this.db.prepare(`
|
||||
INSERT INTO sdk_sessions
|
||||
(claude_session_id, sdk_session_id, project, started_at, started_at_epoch, status)
|
||||
VALUES (?, ?, ?, ?, ?, 'active')
|
||||
`).run(e,e,s,n.toISOString(),a),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let u=this.db.prepare(`
|
||||
`).run(e,e,s,i.toISOString(),o),console.error(`[SessionStore] Auto-created session record for session_id: ${e}`));let E=this.db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
(sdk_session_id, project, request, investigated, learned, completed,
|
||||
next_steps, notes, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n.toISOString(),a);return{id:Number(u.lastInsertRowid),createdAtEpoch:a}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
next_steps, notes, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(e,s,t.request,t.investigated,t.learned,t.completed,t.next_steps,t.notes,r||null,n,i.toISOString(),o);return{id:Number(E.lastInsertRowid),createdAtEpoch:o}}markSessionCompleted(e){let s=new Date,t=s.getTime();this.db.prepare(`
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'completed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
@@ -342,12 +342,12 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
UPDATE sdk_sessions
|
||||
SET status = 'failed', completed_at = ?, completed_at_epoch = ?
|
||||
WHERE id = ?
|
||||
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",a=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
`).run(s.toISOString(),t,e)}getSessionSummariesByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT * FROM session_summaries
|
||||
WHERE id IN (${o})
|
||||
ORDER BY created_at_epoch ${n}
|
||||
${a}
|
||||
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",a=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
${i}
|
||||
`).all(...e)}getUserPromptsByIds(e,s={}){if(e.length===0)return[];let{orderBy:t="date_desc",limit:r}=s,n=t==="date_asc"?"ASC":"DESC",i=r?`LIMIT ${r}`:"",o=e.map(()=>"?").join(",");return this.db.prepare(`
|
||||
SELECT
|
||||
up.*,
|
||||
s.project,
|
||||
@@ -356,63 +356,63 @@ ${e.stack}`:e.message;if(Array.isArray(e))return`[${e.length} items]`;let s=Obje
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.id IN (${o})
|
||||
ORDER BY up.created_at_epoch ${n}
|
||||
${a}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let a=n?"AND project = ?":"",o=n?[n]:[],d,c;if(e!==null){let T=`
|
||||
${i}
|
||||
`).all(...e)}getTimelineAroundTimestamp(e,s=10,t=10,r){return this.getTimelineAroundObservation(null,e,s,t,r)}getTimelineAroundObservation(e,s,t=10,r=10,n){let i=n?"AND project = ?":"",o=n?[n]:[],d,c;if(e!==null){let T=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id <= ? ${a}
|
||||
WHERE id <= ? ${i}
|
||||
ORDER BY id DESC
|
||||
LIMIT ?
|
||||
`,S=`
|
||||
SELECT id, created_at_epoch
|
||||
FROM observations
|
||||
WHERE id >= ? ${a}
|
||||
WHERE id >= ? ${i}
|
||||
ORDER BY id ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(T).all(e,...o,t+1),p=this.db.prepare(S).all(e,...o,r+1);if(_.length===0&&p.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,c=p.length>0?p[p.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary observations:",_.message),{observations:[],sessions:[],prompts:[]}}}else{let T=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch <= ? ${a}
|
||||
WHERE created_at_epoch <= ? ${i}
|
||||
ORDER BY created_at_epoch DESC
|
||||
LIMIT ?
|
||||
`,S=`
|
||||
SELECT created_at_epoch
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? ${a}
|
||||
WHERE created_at_epoch >= ? ${i}
|
||||
ORDER BY created_at_epoch ASC
|
||||
LIMIT ?
|
||||
`;try{let _=this.db.prepare(T).all(s,...o,t),p=this.db.prepare(S).all(s,...o,r+1);if(_.length===0&&p.length===0)return{observations:[],sessions:[],prompts:[]};d=_.length>0?_[_.length-1].created_at_epoch:s,c=p.length>0?p[p.length-1].created_at_epoch:s}catch(_){return console.error("[SessionStore] Error getting boundary timestamps:",_.message),{observations:[],sessions:[],prompts:[]}}}let u=`
|
||||
SELECT *
|
||||
FROM observations
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${a}
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${i}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,l=`
|
||||
`,E=`
|
||||
SELECT *
|
||||
FROM session_summaries
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${a}
|
||||
WHERE created_at_epoch >= ? AND created_at_epoch <= ? ${i}
|
||||
ORDER BY created_at_epoch ASC
|
||||
`,b=`
|
||||
SELECT up.*, s.project, s.sdk_session_id
|
||||
FROM user_prompts up
|
||||
JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${a.replace("project","s.project")}
|
||||
WHERE up.created_at_epoch >= ? AND up.created_at_epoch <= ? ${i.replace("project","s.project")}
|
||||
ORDER BY up.created_at_epoch ASC
|
||||
`;try{let T=this.db.prepare(u).all(d,c,...o),S=this.db.prepare(l).all(d,c,...o),_=this.db.prepare(b).all(d,c,...o);return{observations:T,sessions:S.map(p=>({id:p.id,sdk_session_id:p.sdk_session_id,project:p.project,request:p.request,completed:p.completed,next_steps:p.next_steps,created_at:p.created_at,created_at_epoch:p.created_at_epoch})),prompts:_.map(p=>({id:p.id,claude_session_id:p.claude_session_id,project:p.project,prompt:p.prompt_text,created_at:p.created_at,created_at_epoch:p.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function P(i,e,s){return i==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:i==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:i==="UserPromptSubmit"||i==="PostToolUse"?{continue:!0,suppressOutput:!0}:i==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function v(i,e,s={}){let t=P(i,e,s);return JSON.stringify(t)}import $ from"path";import{homedir as W}from"os";import{existsSync as G,readFileSync as Y}from"fs";var K=100;function h(){try{let i=$.join(W(),".claude-mem","settings.json");if(G(i)){let e=JSON.parse(Y(i,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function q(){try{let i=h();return(await fetch(`http://127.0.0.1:${i}/health`,{signal:AbortSignal.timeout(K)})).ok}catch{return!1}}async function D(){if(await q())return;let i=h();throw new Error(`Worker service is not responding on port ${i}.
|
||||
`;try{let T=this.db.prepare(u).all(d,c,...o),S=this.db.prepare(E).all(d,c,...o),_=this.db.prepare(b).all(d,c,...o);return{observations:T,sessions:S.map(p=>({id:p.id,sdk_session_id:p.sdk_session_id,project:p.project,request:p.request,completed:p.completed,next_steps:p.next_steps,created_at:p.created_at,created_at_epoch:p.created_at_epoch})),prompts:_.map(p=>({id:p.id,claude_session_id:p.claude_session_id,project:p.project,prompt:p.prompt_text,created_at:p.created_at,created_at_epoch:p.created_at_epoch}))}}catch(T){return console.error("[SessionStore] Error querying timeline records:",T.message),{observations:[],sessions:[],prompts:[]}}}close(){this.db.close()}};function P(a,e,s){return a==="PreCompact"?e?{continue:!0,suppressOutput:!0}:{continue:!1,stopReason:s.reason||"Pre-compact operation failed",suppressOutput:!0}:a==="SessionStart"?e&&s.context?{continue:!0,suppressOutput:!0,hookSpecificOutput:{hookEventName:"SessionStart",additionalContext:s.context}}:{continue:!0,suppressOutput:!0}:a==="UserPromptSubmit"||a==="PostToolUse"?{continue:!0,suppressOutput:!0}:a==="Stop"?{continue:!0,suppressOutput:!0}:{continue:e,suppressOutput:!0,...s.reason&&!e?{stopReason:s.reason}:{}}}function C(a,e,s={}){let t=P(a,e,s);return JSON.stringify(t)}import $ from"path";import{homedir as G}from"os";import{existsSync as W,readFileSync as Y}from"fs";var K=100;function h(){try{let a=$.join(G(),".claude-mem","settings.json");if(W(a)){let e=JSON.parse(Y(a,"utf-8")),s=parseInt(e.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(s))return s}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}async function q(){try{let a=h();return(await fetch(`http://127.0.0.1:${a}/health`,{signal:AbortSignal.timeout(K)})).ok}catch{return!1}}async function D(){if(await q())return;let a=h();throw new Error(`Worker service is not responding on port ${a}.
|
||||
|
||||
If you just updated the plugin, PM2's watch mode should restart automatically.
|
||||
If the problem persists, run: pm2 restart claude-mem-worker`)}import{appendFileSync as V}from"fs";import{homedir as J}from"os";import{join as Q}from"path";var z=Q(J(),".claude-mem","silent.log");function I(i,e,s=""){let t=new Date().toISOString(),o=((new Error().stack||"").split(`
|
||||
`)[2]||"").match(/at\s+(?:.*\s+)?\(?([^:]+):(\d+):(\d+)\)?/),d=o?`${o[1].split("/").pop()}:${o[2]}`:"unknown",c=`[${t}] [${d}] ${i}`;if(e!==void 0)try{c+=` ${JSON.stringify(e)}`}catch(u){c+=` [stringify error: ${u}]`}c+=`
|
||||
`;try{V(z,c)}catch(u){console.error("[silent-debug] Failed to write to log:",u)}return s}function Z(i){if(!i||!U(i))return"";try{let e=x(i,"utf-8").trim();if(!e)return"";let s=e.split(`
|
||||
If the problem persists, run: pm2 restart claude-mem-worker`)}import{appendFileSync as V}from"fs";import{homedir as J}from"os";import{join as Q}from"path";var z=Q(J(),".claude-mem","silent.log");function I(a,e,s=""){let t=new Date().toISOString(),o=((new Error().stack||"").split(`
|
||||
`)[2]||"").match(/at\s+(?:.*\s+)?\(?([^:]+):(\d+):(\d+)\)?/),d=o?`${o[1].split("/").pop()}:${o[2]}`:"unknown",c=`[${t}] [${d}] ${a}`;if(e!==void 0)try{c+=` ${JSON.stringify(e)}`}catch(u){c+=` [stringify error: ${u}]`}c+=`
|
||||
`;try{V(z,c)}catch(u){console.error("[silent-debug] Failed to write to log:",u)}return s}function Z(a){if(!a||!U(a))return"";try{let e=x(a,"utf-8").trim();if(!e)return"";let s=e.split(`
|
||||
`);for(let t=s.length-1;t>=0;t--)try{let r=JSON.parse(s[t]);if(r.type==="user"&&r.message?.content){let n=r.message.content;if(typeof n=="string")return n;if(Array.isArray(n))return n.filter(o=>o.type==="text").map(o=>o.text).join(`
|
||||
`)}}catch{continue}}catch(e){g.error("HOOK","Failed to read transcript",{transcriptPath:i},e)}return""}function ee(i){if(!i||!U(i))return"";try{let e=x(i,"utf-8").trim();if(!e)return"";let s=e.split(`
|
||||
`);for(let t=s.length-1;t>=0;t--)try{let r=JSON.parse(s[t]);if(r.type==="assistant"&&r.message?.content){let n="",a=r.message.content;return typeof a=="string"?n=a:Array.isArray(a)&&(n=a.filter(d=>d.type==="text").map(d=>d.text).join(`
|
||||
`)}}catch{continue}}catch(e){g.error("HOOK","Failed to read transcript",{transcriptPath:a},e)}return""}function ee(a){if(!a||!U(a))return"";try{let e=x(a,"utf-8").trim();if(!e)return"";let s=e.split(`
|
||||
`);for(let t=s.length-1;t>=0;t--)try{let r=JSON.parse(s[t]);if(r.type==="assistant"&&r.message?.content){let n="",i=r.message.content;return typeof i=="string"?n=i:Array.isArray(i)&&(n=i.filter(d=>d.type==="text").map(d=>d.text).join(`
|
||||
`)),n=n.replace(/<system-reminder>[\s\S]*?<\/system-reminder>/g,""),n=n.replace(/\n{3,}/g,`
|
||||
|
||||
`).trim(),n}}catch{continue}}catch(e){g.error("HOOK","Failed to read transcript",{transcriptPath:i},e)}return""}async function se(i){if(!i)throw new Error("summaryHook requires input");let{session_id:e}=i;await D();let s=new R,t=s.createSDKSession(e,"",""),r=s.getPromptCounter(t),n=s.db.prepare(`
|
||||
`).trim(),n}}catch{continue}}catch(e){g.error("HOOK","Failed to read transcript",{transcriptPath:a},e)}return""}async function se(a){if(!a)throw new Error("summaryHook requires input");let{session_id:e}=a;await D();let s=new R,t=s.createSDKSession(e,"",""),r=s.getPromptCounter(t),n=s.db.prepare(`
|
||||
SELECT id, claude_session_id, sdk_session_id, project
|
||||
FROM sdk_sessions WHERE id = ?
|
||||
`).get(t),a=s.db.prepare(`
|
||||
`).get(t),i=s.db.prepare(`
|
||||
SELECT COUNT(*) as count
|
||||
FROM observations
|
||||
WHERE sdk_session_id = ?
|
||||
`).get(n?.sdk_session_id);I("[summary-hook] Session diagnostics",{claudeSessionId:e,sessionDbId:t,sdkSessionId:n?.sdk_session_id,project:n?.project,promptNumber:r,observationCount:a?.count||0,transcriptPath:i.transcript_path}),s.close();let o=h(),d=Z(i.transcript_path||""),c=ee(i.transcript_path||"");I("[summary-hook] Extracted messages",{hasLastUserMessage:!!d,hasLastAssistantMessage:!!c,lastAssistantPreview:c.substring(0,200),lastAssistantLength:c.length}),g.dataIn("HOOK","Stop: Requesting summary",{sessionId:t,workerPort:o,promptNumber:r,hasLastUserMessage:!!d,hasLastAssistantMessage:!!c});try{let u=await fetch(`http://127.0.0.1:${o}/sessions/${t}/summarize`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({prompt_number:r,last_user_message:d,last_assistant_message:c}),signal:AbortSignal.timeout(2e3)});if(!u.ok){let l=await u.text();throw g.failure("HOOK","Failed to generate summary",{sessionId:t,status:u.status},l),new Error(`Failed to request summary from worker: ${u.status} ${l}`)}g.debug("HOOK","Summary request sent successfully",{sessionId:t})}catch(u){throw u.cause?.code==="ECONNREFUSED"||u.name==="TimeoutError"||u.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):u}finally{await fetch(`http://127.0.0.1:${o}/api/processing`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({isProcessing:!1})})}console.log(v("Stop",!0))}var L="";k.on("data",i=>L+=i);k.on("end",async()=>{let i=L?JSON.parse(L):void 0;await se(i)});
|
||||
`).get(n?.sdk_session_id);I("[summary-hook] Session diagnostics",{claudeSessionId:e,sessionDbId:t,sdkSessionId:n?.sdk_session_id,project:n?.project,promptNumber:r,observationCount:i?.count||0,transcriptPath:a.transcript_path}),s.close();let o=h(),d=Z(a.transcript_path||""),c=ee(a.transcript_path||"");I("[summary-hook] Extracted messages",{hasLastUserMessage:!!d,hasLastAssistantMessage:!!c,lastAssistantPreview:c.substring(0,200),lastAssistantLength:c.length}),g.dataIn("HOOK","Stop: Requesting summary",{sessionId:t,workerPort:o,promptNumber:r,hasLastUserMessage:!!d,hasLastAssistantMessage:!!c});try{let u=await fetch(`http://127.0.0.1:${o}/sessions/${t}/summarize`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({prompt_number:r,last_user_message:d,last_assistant_message:c}),signal:AbortSignal.timeout(2e3)});if(!u.ok){let E=await u.text();throw g.failure("HOOK","Failed to generate summary",{sessionId:t,status:u.status},E),new Error(`Failed to request summary from worker: ${u.status} ${E}`)}g.debug("HOOK","Summary request sent successfully",{sessionId:t})}catch(u){throw u.cause?.code==="ECONNREFUSED"||u.name==="TimeoutError"||u.message.includes("fetch failed")?new Error("There's a problem with the worker. If you just updated, type `pm2 restart claude-mem-worker` in your terminal to continue"):u}finally{await fetch(`http://127.0.0.1:${o}/api/processing`,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({isProcessing:!1})})}console.log(C("Stop",!0))}var L="";k.on("data",a=>L+=a);k.on("end",async()=>{let a=L?JSON.parse(L):void 0;await se(a)});
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env node
|
||||
import{execSync as u}from"child_process";import{join as r}from"path";import{homedir as s}from"os";import{existsSync as p}from"fs";import i from"path";import{homedir as a}from"os";import{existsSync as c,readFileSync as l}from"fs";function n(){try{let e=i.join(a(),".claude-mem","settings.json");if(c(e)){let o=JSON.parse(l(e,"utf-8")),t=parseInt(o.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(t))return t}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}var m=r(s(),".claude","plugins","marketplaces","thedotmack"),d=r(m,"node_modules");p(d)||(console.error(`
|
||||
import{execSync as u}from"child_process";import{join as r}from"path";import{homedir as s}from"os";import{existsSync as l}from"fs";import i from"path";import{homedir as a}from"os";import{existsSync as c,readFileSync as p}from"fs";function n(){try{let e=i.join(a(),".claude-mem","settings.json");if(c(e)){let t=JSON.parse(p(e,"utf-8")),o=parseInt(t.env?.CLAUDE_MEM_WORKER_PORT,10);if(!isNaN(o))return o}}catch{}return parseInt(process.env.CLAUDE_MEM_WORKER_PORT||"37777",10)}var m=r(s(),".claude","plugins","marketplaces","thedotmack"),d=r(m,"node_modules");l(d)||(console.error(`
|
||||
---
|
||||
\u{1F389} Note: This appears under Plugin Hook Error, but it's not an error. That's the only option for
|
||||
user messages in Claude Code UI until a better method is provided.
|
||||
@@ -17,15 +17,15 @@ Dependencies have been installed in the background. This only happens once.
|
||||
Thank you for installing Claude-Mem!
|
||||
|
||||
This message was not added to your startup context, so you can continue working as normal.
|
||||
`),process.exit(3));try{let e=r(s(),".claude","plugins","marketplaces","thedotmack","plugin","scripts","context-hook.js"),o=u(`node "${e}" --colors`,{encoding:"utf8"}),t=n();console.error(`
|
||||
`),process.exit(3));try{let e=r(s(),".claude","plugins","marketplaces","thedotmack","plugin","scripts","context-hook.js"),t=u(`node "${e}" --colors`,{encoding:"utf8"}),o=n();console.error(`
|
||||
|
||||
\u{1F4DD} Claude-Mem Context Loaded
|
||||
\u2139\uFE0F Note: This appears as stderr but is informational only
|
||||
|
||||
`+o+`
|
||||
`+t+`
|
||||
|
||||
\u{1F4AC} New! Connect with Claude-Mem
|
||||
\u{1F4AC} Feedback & Support
|
||||
https://github.com/thedotmack/claude-mem/discussions/110
|
||||
|
||||
\u{1F4FA} Watch live in browser http://localhost:${t}/
|
||||
\u{1F4FA} Watch live in browser http://localhost:${o}/
|
||||
`)}catch(e){console.error(`\u274C Failed to load context display: ${e}`)}process.exit(3);
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -169,7 +169,7 @@ For guidelines on presenting search results to users, see [operations/formatting
|
||||
|
||||
- **Port:** Default 37777 (configurable via `CLAUDE_MEM_WORKER_PORT`)
|
||||
- **Response format:** Always JSON
|
||||
- **Search engine:** FTS5 full-text search + structured filters
|
||||
- **Search engine:** ChromaDB semantic search (primary ranking) + SQLite FTS5 (fallback) + 90-day recency filter + temporal ordering (hybrid architecture)
|
||||
- **All operations:** HTTP GET with query parameters
|
||||
- **Worker:** PM2-managed background process
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Search Observations (Full-Text)
|
||||
# Search Observations (Semantic + Full-Text Hybrid)
|
||||
|
||||
Search all observations using natural language queries.
|
||||
|
||||
@@ -17,7 +17,7 @@ curl -s "http://localhost:37777/api/search/observations?query=authentication&for
|
||||
|
||||
## Parameters
|
||||
|
||||
- **query** (required): Search terms (e.g., "authentication", "bug fix", "database migration")
|
||||
- **query** (required): Natural language search query - uses semantic search (ChromaDB) for ranking with SQLite FTS5 fallback (e.g., "authentication", "bug fix", "database migration")
|
||||
- **format**: "index" (summary) or "full" (complete details). Default: "full"
|
||||
- **limit**: Number of results (default: 20, max: 100)
|
||||
- **project**: Filter by project name (optional)
|
||||
|
||||
+112
-55
@@ -71,6 +71,7 @@ interface Observation {
|
||||
concepts: string | null;
|
||||
files_read: string | null;
|
||||
files_modified: string | null;
|
||||
discovery_tokens: number | null;
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
@@ -130,12 +131,6 @@ function formatDate(dateStr: string): string {
|
||||
});
|
||||
}
|
||||
|
||||
// Helper: Estimate token count for text
|
||||
function estimateTokens(text: string | null): number {
|
||||
if (!text) return 0;
|
||||
return Math.ceil(text.length / CHARS_PER_TOKEN_ESTIMATE);
|
||||
}
|
||||
|
||||
// Helper: Convert absolute paths to relative paths
|
||||
function toRelativePath(filePath: string, cwd: string): string {
|
||||
if (path.isAbsolute(filePath)) {
|
||||
@@ -154,24 +149,6 @@ function renderSummaryField(label: string, value: string | null, color: string,
|
||||
return [`**${label}**: ${value}`, ''];
|
||||
}
|
||||
|
||||
// Helper: Get all observations for given sessions
|
||||
function getObservations(db: SessionStore, sessionIds: string[]): Observation[] {
|
||||
if (sessionIds.length === 0) return [];
|
||||
|
||||
const placeholders = sessionIds.map(() => '?').join(',');
|
||||
const observations = db.db.prepare(`
|
||||
SELECT
|
||||
id, sdk_session_id, type, title, subtitle, narrative,
|
||||
facts, concepts, files_read, files_modified,
|
||||
created_at, created_at_epoch
|
||||
FROM observations
|
||||
WHERE sdk_session_id IN (${placeholders})
|
||||
ORDER BY created_at_epoch DESC
|
||||
`).all(...sessionIds) as Observation[];
|
||||
|
||||
return observations;
|
||||
}
|
||||
|
||||
/**
|
||||
* Context Hook Main Logic
|
||||
*/
|
||||
@@ -187,7 +164,7 @@ async function contextHook(input?: SessionStartInput, useColors: boolean = false
|
||||
const allObservations = db.db.prepare(`
|
||||
SELECT
|
||||
id, sdk_session_id, type, title, subtitle, narrative,
|
||||
facts, concepts, files_read, files_modified,
|
||||
facts, concepts, files_read, files_modified, discovery_tokens,
|
||||
created_at, created_at_epoch
|
||||
FROM observations
|
||||
WHERE project = ?
|
||||
@@ -239,25 +216,74 @@ async function contextHook(input?: SessionStartInput, useColors: boolean = false
|
||||
if (timelineObs.length > 0) {
|
||||
// Legend/Key
|
||||
if (useColors) {
|
||||
output.push(`${colors.dim}Legend: 🎯 session-request | 🔴 bugfix | 🟣 feature | 🔄 refactor | ✅ change | 🔵 discovery | 🧠 decision${colors.reset}`);
|
||||
output.push('');
|
||||
output.push(`${colors.dim}Legend: 🎯 session-request | 🔴 bugfix | 🟣 feature | 🔄 refactor | ✅ change | 🔵 discovery | ⚖️ decision${colors.reset}`);
|
||||
} else {
|
||||
output.push(`**Legend:** 🎯 session-request | 🔴 bugfix | 🟣 feature | 🔄 refactor | ✅ change | 🔵 discovery | 🧠 decision`);
|
||||
output.push('');
|
||||
output.push(`**Legend:** 🎯 session-request | 🔴 bugfix | 🟣 feature | 🔄 refactor | ✅ change | 🔵 discovery | ⚖️ decision`);
|
||||
}
|
||||
output.push('');
|
||||
|
||||
// Progressive Disclosure Usage Instructions
|
||||
// Column Key
|
||||
if (useColors) {
|
||||
output.push(`${colors.dim}💡 Progressive Disclosure: This index shows WHAT exists (titles) and retrieval COST (token counts).${colors.reset}`);
|
||||
output.push(`${colors.dim} → Use MCP search tools to fetch full observation details on-demand (Layer 2)${colors.reset}`);
|
||||
output.push(`${colors.dim} → Prefer searching observations over re-reading code for past decisions and learnings${colors.reset}`);
|
||||
output.push(`${colors.dim} → Critical types (🔴 bugfix, 🧠 decision) often worth fetching immediately${colors.reset}`);
|
||||
output.push(`${colors.bright}💡 Column Key${colors.reset}`);
|
||||
output.push(`${colors.dim} Read: Tokens to read this observation (cost to learn it now)${colors.reset}`);
|
||||
output.push(`${colors.dim} Work: Tokens spent on work that produced this record (🔍 research, 🛠️ building, ⚖️ deciding)${colors.reset}`);
|
||||
} else {
|
||||
output.push(`💡 **Column Key**:`);
|
||||
output.push(`- **Read**: Tokens to read this observation (cost to learn it now)`);
|
||||
output.push(`- **Work**: Tokens spent on work that produced this record (🔍 research, 🛠️ building, ⚖️ deciding)`);
|
||||
}
|
||||
output.push('');
|
||||
|
||||
// Context Index Usage Instructions
|
||||
if (useColors) {
|
||||
output.push(`${colors.dim}💡 Context Index: This semantic index (titles, types, files, tokens) is usually sufficient to understand past work.${colors.reset}`);
|
||||
output.push('');
|
||||
output.push(`${colors.dim}When you need implementation details, rationale, or debugging context:${colors.reset}`);
|
||||
output.push(`${colors.dim} - Use the mem-search skill to fetch full observations on-demand${colors.reset}`);
|
||||
output.push(`${colors.dim} - Critical types (🔴 bugfix, ⚖️ decision) often need detailed fetching${colors.reset}`);
|
||||
output.push(`${colors.dim} - Trust this index over re-reading code for past decisions and learnings${colors.reset}`);
|
||||
} else {
|
||||
output.push(`💡 **Context Index:** This semantic index (titles, types, files, tokens) is usually sufficient to understand past work.`);
|
||||
output.push('');
|
||||
output.push(`When you need implementation details, rationale, or debugging context:`);
|
||||
output.push(`- Use the mem-search skill to fetch full observations on-demand`);
|
||||
output.push(`- Critical types (🔴 bugfix, ⚖️ decision) often need detailed fetching`);
|
||||
output.push(`- Trust this index over re-reading code for past decisions and learnings`);
|
||||
}
|
||||
output.push('');
|
||||
|
||||
// Section 1: Aggregate ROI Metrics
|
||||
const totalObservations = observations.length;
|
||||
const totalReadTokens = observations.reduce((sum, obs) => {
|
||||
// Estimate read tokens from observation size
|
||||
const obsSize = (obs.title?.length || 0) +
|
||||
(obs.subtitle?.length || 0) +
|
||||
(obs.narrative?.length || 0) +
|
||||
JSON.stringify(obs.facts || []).length;
|
||||
return sum + Math.ceil(obsSize / CHARS_PER_TOKEN_ESTIMATE);
|
||||
}, 0);
|
||||
const totalDiscoveryTokens = observations.reduce((sum, obs) => sum + (obs.discovery_tokens || 0), 0);
|
||||
const savings = totalDiscoveryTokens - totalReadTokens;
|
||||
const savingsPercent = totalDiscoveryTokens > 0
|
||||
? Math.round((savings / totalDiscoveryTokens) * 100)
|
||||
: 0;
|
||||
|
||||
// Display Context Economics section
|
||||
if (useColors) {
|
||||
output.push(`${colors.bright}${colors.cyan}📊 Context Economics${colors.reset}`);
|
||||
output.push(`${colors.dim} Loading: ${totalObservations} observations (${totalReadTokens.toLocaleString()} tokens to read)${colors.reset}`);
|
||||
output.push(`${colors.dim} Work investment: ${totalDiscoveryTokens.toLocaleString()} tokens spent on research, building, and decisions${colors.reset}`);
|
||||
if (totalDiscoveryTokens > 0) {
|
||||
output.push(`${colors.green} Your savings: ${savings.toLocaleString()} tokens (${savingsPercent}% reduction from reuse)${colors.reset}`);
|
||||
}
|
||||
output.push('');
|
||||
} else {
|
||||
output.push(`💡 **Progressive Disclosure:** This index shows WHAT exists (titles) and retrieval COST (token counts).`);
|
||||
output.push(`- Use MCP search tools to fetch full observation details on-demand (Layer 2)`);
|
||||
output.push(`- Prefer searching observations over re-reading code for past decisions and learnings`);
|
||||
output.push(`- Critical types (🔴 bugfix, 🧠 decision) often worth fetching immediately`);
|
||||
output.push(`📊 **Context Economics**:`);
|
||||
output.push(`- Loading: ${totalObservations} observations (${totalReadTokens.toLocaleString()} tokens to read)`);
|
||||
output.push(`- Work investment: ${totalDiscoveryTokens.toLocaleString()} tokens spent on research, building, and decisions`);
|
||||
if (totalDiscoveryTokens > 0) {
|
||||
output.push(`- Your savings: ${savings.toLocaleString()} tokens (${savingsPercent}% reduction from reuse)`);
|
||||
}
|
||||
output.push('');
|
||||
}
|
||||
|
||||
@@ -380,8 +406,8 @@ async function contextHook(input?: SessionStartInput, useColors: boolean = false
|
||||
|
||||
// Table header (markdown only)
|
||||
if (!useColors) {
|
||||
output.push(`| ID | Time | T | Title | Tokens |`);
|
||||
output.push(`|----|------|---|-------|--------|`);
|
||||
output.push(`| ID | Time | T | Title | Read | Work |`);
|
||||
output.push(`|----|------|---|-------|------|------|`);
|
||||
}
|
||||
|
||||
currentFile = file;
|
||||
@@ -389,10 +415,11 @@ async function contextHook(input?: SessionStartInput, useColors: boolean = false
|
||||
lastTime = '';
|
||||
}
|
||||
|
||||
// Render observation row
|
||||
let icon = '•';
|
||||
const time = formatTime(obs.created_at);
|
||||
const title = obs.title || 'Untitled';
|
||||
|
||||
// Map observation type to emoji
|
||||
// Map observation type to emoji icon
|
||||
let icon = '•';
|
||||
switch (obs.type) {
|
||||
case 'bugfix':
|
||||
icon = '🔴';
|
||||
@@ -410,15 +437,40 @@ async function contextHook(input?: SessionStartInput, useColors: boolean = false
|
||||
icon = '🔵';
|
||||
break;
|
||||
case 'decision':
|
||||
icon = '🧠';
|
||||
icon = '⚖️';
|
||||
break;
|
||||
default:
|
||||
icon = '•';
|
||||
}
|
||||
|
||||
const time = formatTime(obs.created_at);
|
||||
const title = obs.title || 'Untitled';
|
||||
const tokens = estimateTokens(obs.narrative);
|
||||
// Section 2: Calculate read tokens (estimate from observation size)
|
||||
const obsSize = (obs.title?.length || 0) +
|
||||
(obs.subtitle?.length || 0) +
|
||||
(obs.narrative?.length || 0) +
|
||||
JSON.stringify(obs.facts || []).length;
|
||||
const readTokens = Math.ceil(obsSize / CHARS_PER_TOKEN_ESTIMATE);
|
||||
|
||||
// Get discovery tokens (handle old observations without this field)
|
||||
const discoveryTokens = obs.discovery_tokens || 0;
|
||||
|
||||
// Map observation type to work emoji
|
||||
let workEmoji = '🔍'; // default to research/discovery
|
||||
switch (obs.type) {
|
||||
case 'discovery':
|
||||
workEmoji = '🔍'; // research/exploration
|
||||
break;
|
||||
case 'change':
|
||||
case 'feature':
|
||||
case 'bugfix':
|
||||
case 'refactor':
|
||||
workEmoji = '🛠️'; // building/modifying
|
||||
break;
|
||||
case 'decision':
|
||||
workEmoji = '⚖️'; // decision-making
|
||||
break;
|
||||
}
|
||||
|
||||
const discoveryDisplay = discoveryTokens > 0 ? `${workEmoji} ${discoveryTokens.toLocaleString()}` : '-';
|
||||
|
||||
const showTime = time !== lastTime;
|
||||
const timeDisplay = showTime ? time : '';
|
||||
@@ -426,10 +478,11 @@ async function contextHook(input?: SessionStartInput, useColors: boolean = false
|
||||
|
||||
if (useColors) {
|
||||
const timePart = showTime ? `${colors.dim}${time}${colors.reset}` : ' '.repeat(time.length);
|
||||
const tokensPart = tokens > 0 ? `${colors.dim}(~${tokens}t)${colors.reset}` : '';
|
||||
output.push(` ${colors.dim}#${obs.id}${colors.reset} ${timePart} ${icon} ${title} ${tokensPart}`);
|
||||
const readPart = readTokens > 0 ? `${colors.dim}(~${readTokens}t)${colors.reset}` : '';
|
||||
const discoveryPart = discoveryTokens > 0 ? `${colors.dim}(${workEmoji} ${discoveryTokens.toLocaleString()}t)${colors.reset}` : '';
|
||||
output.push(` ${colors.dim}#${obs.id}${colors.reset} ${timePart} ${icon} ${title} ${readPart} ${discoveryPart}`);
|
||||
} else {
|
||||
output.push(`| #${obs.id} | ${timeDisplay || '″'} | ${icon} | ${title} | ~${tokens} |`);
|
||||
output.push(`| #${obs.id} | ${timeDisplay || '″'} | ${icon} | ${title} | ~${readTokens} | ${discoveryDisplay} |`);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -456,11 +509,15 @@ async function contextHook(input?: SessionStartInput, useColors: boolean = false
|
||||
output.push(...renderSummaryField('Next Steps', mostRecentSummary.next_steps, colors.magenta, useColors));
|
||||
}
|
||||
|
||||
// Footer with MCP search instructions
|
||||
if (useColors) {
|
||||
output.push(`${colors.dim}Use claude-mem MCP search to access records with the given ID${colors.reset}`);
|
||||
} else {
|
||||
output.push(`*Use claude-mem MCP search to access records with the given ID*`);
|
||||
// Footer with token savings message
|
||||
if (totalDiscoveryTokens > 0 && savings > 0) {
|
||||
const workTokensK = Math.round(totalDiscoveryTokens / 1000);
|
||||
output.push('');
|
||||
if (useColors) {
|
||||
output.push(`${colors.dim}💰 Access ${workTokensK}k tokens of past research & decisions for just ${totalReadTokens.toLocaleString()}t. Use claude-mem search to access memories by ID instead of re-reading files.${colors.reset}`);
|
||||
} else {
|
||||
output.push(`💰 Access ${workTokensK}k tokens of past research & decisions for just ${totalReadTokens.toLocaleString()}t. Use claude-mem search to access memories by ID instead of re-reading files.`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -52,7 +52,7 @@ try {
|
||||
"\n\n📝 Claude-Mem Context Loaded\n" +
|
||||
" ℹ️ Note: This appears as stderr but is informational only\n\n" +
|
||||
output +
|
||||
"\n\n💬 New! Connect with Claude-Mem\nhttps://github.com/thedotmack/claude-mem/discussions/110\n" +
|
||||
"\n\n💬 Feedback & Support\nhttps://github.com/thedotmack/claude-mem/discussions/110\n" +
|
||||
`\n📺 Watch live in browser http://localhost:${port}/\n`
|
||||
);
|
||||
|
||||
|
||||
@@ -346,9 +346,9 @@ const filterSchema = z.object({
|
||||
const tools = [
|
||||
{
|
||||
name: 'search_observations',
|
||||
description: 'Search observations using full-text search across titles, narratives, facts, and concepts. IMPORTANT: Always use index format first (default) to get an overview with minimal token usage, then use format: "full" only for specific items of interest.',
|
||||
description: 'Search observations using hybrid semantic + full-text search (ChromaDB primary, SQLite FTS5 fallback). IMPORTANT: Always use index format first (default) to get an overview with minimal token usage, then use format: "full" only for specific items of interest.',
|
||||
inputSchema: z.object({
|
||||
query: z.string().describe('Search query for FTS5 full-text search'),
|
||||
query: z.string().describe('Natural language search query (semantic ranking via ChromaDB, FTS5 fallback)'),
|
||||
format: z.enum(['index', 'full']).default('index').describe('Output format: "index" for titles/dates only (default, RECOMMENDED for initial search), "full" for complete details (use only after reviewing index results)'),
|
||||
...filterSchema.shape
|
||||
}),
|
||||
@@ -434,9 +434,9 @@ const tools = [
|
||||
},
|
||||
{
|
||||
name: 'search_sessions',
|
||||
description: 'Search session summaries using full-text search across requests, completions, learnings, and notes. IMPORTANT: Always use index format first (default) to get an overview with minimal token usage, then use format: "full" only for specific items of interest.',
|
||||
description: 'Search session summaries using hybrid semantic + full-text search (ChromaDB primary, SQLite FTS5 fallback). IMPORTANT: Always use index format first (default) to get an overview with minimal token usage, then use format: "full" only for specific items of interest.',
|
||||
inputSchema: z.object({
|
||||
query: z.string().describe('Search query for FTS5 full-text search'),
|
||||
query: z.string().describe('Natural language search query (semantic ranking via ChromaDB, FTS5 fallback)'),
|
||||
format: z.enum(['index', 'full']).default('index').describe('Output format: "index" for titles/dates only (default, RECOMMENDED for initial search), "full" for complete details (use only after reviewing index results)'),
|
||||
project: z.string().optional().describe('Filter by project name'),
|
||||
dateRange: z.object({
|
||||
@@ -1000,9 +1000,9 @@ const tools = [
|
||||
},
|
||||
{
|
||||
name: 'search_user_prompts',
|
||||
description: 'Search raw user prompts with full-text search. Use this to find what the user actually said/requested across all sessions. IMPORTANT: Always use index format first (default) to get an overview with minimal token usage, then use format: "full" only for specific items of interest.',
|
||||
description: 'Search raw user prompts using hybrid semantic + full-text search (ChromaDB primary, SQLite FTS5 fallback). Use this to find what the user actually said/requested across all sessions. IMPORTANT: Always use index format first (default) to get an overview with minimal token usage, then use format: "full" only for specific items of interest.',
|
||||
inputSchema: z.object({
|
||||
query: z.string().describe('Search query for FTS5 full-text search'),
|
||||
query: z.string().describe('Natural language search query (semantic ranking via ChromaDB, FTS5 fallback)'),
|
||||
format: z.enum(['index', 'full']).default('index').describe('Output format: "index" for truncated prompts/dates (default, RECOMMENDED for initial search), "full" for complete prompt text (use only after reviewing index results)'),
|
||||
project: z.string().optional().describe('Filter by project name'),
|
||||
dateRange: z.object({
|
||||
@@ -1740,6 +1740,47 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
||||
}
|
||||
});
|
||||
|
||||
// Cleanup function to properly terminate all child processes
|
||||
async function cleanup() {
|
||||
console.error('[search-server] Shutting down...');
|
||||
|
||||
// Close Chroma client (terminates uvx/python processes)
|
||||
if (chromaClient) {
|
||||
try {
|
||||
await chromaClient.close();
|
||||
console.error('[search-server] Chroma client closed');
|
||||
} catch (error: any) {
|
||||
console.error('[search-server] Error closing Chroma client:', error.message);
|
||||
}
|
||||
}
|
||||
|
||||
// Close database connections
|
||||
if (search) {
|
||||
try {
|
||||
search.close();
|
||||
console.error('[search-server] SessionSearch closed');
|
||||
} catch (error: any) {
|
||||
console.error('[search-server] Error closing SessionSearch:', error.message);
|
||||
}
|
||||
}
|
||||
|
||||
if (store) {
|
||||
try {
|
||||
store.close();
|
||||
console.error('[search-server] SessionStore closed');
|
||||
} catch (error: any) {
|
||||
console.error('[search-server] Error closing SessionStore:', error.message);
|
||||
}
|
||||
}
|
||||
|
||||
console.error('[search-server] Shutdown complete');
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Register cleanup handlers for graceful shutdown
|
||||
process.on('SIGTERM', cleanup);
|
||||
process.on('SIGINT', cleanup);
|
||||
|
||||
// Start the server
|
||||
async function main() {
|
||||
// Start the MCP server FIRST (critical - must start before blocking operations)
|
||||
|
||||
@@ -264,6 +264,7 @@ export class SessionSearch {
|
||||
const sql = `
|
||||
SELECT
|
||||
o.*,
|
||||
o.discovery_tokens,
|
||||
observations_fts.rank as rank
|
||||
FROM observations o
|
||||
JOIN observations_fts ON o.id = observations_fts.rowid
|
||||
@@ -326,6 +327,7 @@ export class SessionSearch {
|
||||
const sql = `
|
||||
SELECT
|
||||
s.*,
|
||||
s.discovery_tokens,
|
||||
session_summaries_fts.rank as rank
|
||||
FROM session_summaries s
|
||||
JOIN session_summaries_fts ON s.id = session_summaries_fts.rowid
|
||||
@@ -368,7 +370,7 @@ export class SessionSearch {
|
||||
const orderClause = this.buildOrderClause(orderBy, false);
|
||||
|
||||
const sql = `
|
||||
SELECT o.*
|
||||
SELECT o.*, o.discovery_tokens
|
||||
FROM observations o
|
||||
WHERE ${filterClause}
|
||||
${orderClause}
|
||||
@@ -396,7 +398,7 @@ export class SessionSearch {
|
||||
const orderClause = this.buildOrderClause(orderBy, false);
|
||||
|
||||
const observationsSql = `
|
||||
SELECT o.*
|
||||
SELECT o.*, o.discovery_tokens
|
||||
FROM observations o
|
||||
WHERE ${filterClause}
|
||||
${orderClause}
|
||||
@@ -440,7 +442,7 @@ export class SessionSearch {
|
||||
sessionParams.push(`%${filePath}%`, `%${filePath}%`);
|
||||
|
||||
const sessionsSql = `
|
||||
SELECT s.*
|
||||
SELECT s.*, s.discovery_tokens
|
||||
FROM session_summaries s
|
||||
WHERE ${baseConditions.join(' AND ')}
|
||||
ORDER BY s.created_at_epoch DESC
|
||||
@@ -470,7 +472,7 @@ export class SessionSearch {
|
||||
const orderClause = this.buildOrderClause(orderBy, false);
|
||||
|
||||
const sql = `
|
||||
SELECT o.*
|
||||
SELECT o.*, o.discovery_tokens
|
||||
FROM observations o
|
||||
WHERE ${filterClause}
|
||||
${orderClause}
|
||||
|
||||
@@ -28,6 +28,7 @@ export class SessionStore {
|
||||
this.addObservationHierarchicalFields();
|
||||
this.makeObservationsTextNullable();
|
||||
this.createUserPromptsTable();
|
||||
this.ensureDiscoveryTokensColumn();
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -492,6 +493,39 @@ export class SessionStore {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure discovery_tokens column exists (migration 7)
|
||||
* CRITICAL: Always verify column existence, don't trust schema_versions alone
|
||||
* This prevents "no such column" errors when migrations were recorded but failed
|
||||
*/
|
||||
private ensureDiscoveryTokensColumn(): void {
|
||||
try {
|
||||
// Check if discovery_tokens column exists in observations table
|
||||
const observationsInfo = this.db.pragma('table_info(observations)');
|
||||
const obsHasDiscoveryTokens = (observationsInfo as any[]).some((col: any) => col.name === 'discovery_tokens');
|
||||
|
||||
if (!obsHasDiscoveryTokens) {
|
||||
this.db.exec('ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0');
|
||||
console.error('[SessionStore] Added discovery_tokens column to observations table');
|
||||
}
|
||||
|
||||
// Check if discovery_tokens column exists in session_summaries table
|
||||
const summariesInfo = this.db.pragma('table_info(session_summaries)');
|
||||
const sumHasDiscoveryTokens = (summariesInfo as any[]).some((col: any) => col.name === 'discovery_tokens');
|
||||
|
||||
if (!sumHasDiscoveryTokens) {
|
||||
this.db.exec('ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0');
|
||||
console.error('[SessionStore] Added discovery_tokens column to session_summaries table');
|
||||
}
|
||||
|
||||
// Record migration only after successful column verification/addition
|
||||
this.db.prepare('INSERT OR IGNORE INTO schema_versions (version, applied_at) VALUES (?, ?)').run(7, new Date().toISOString());
|
||||
} catch (error: any) {
|
||||
console.error('[SessionStore] Discovery tokens migration error:', error.message);
|
||||
throw error; // Re-throw to prevent silent failures
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get recent session summaries for a project
|
||||
*/
|
||||
@@ -1074,7 +1108,8 @@ export class SessionStore {
|
||||
files_read: string[];
|
||||
files_modified: string[];
|
||||
},
|
||||
promptNumber?: number
|
||||
promptNumber?: number,
|
||||
discoveryTokens: number = 0
|
||||
): { id: number; createdAtEpoch: number } {
|
||||
const now = new Date();
|
||||
const nowEpoch = now.getTime();
|
||||
@@ -1105,8 +1140,8 @@ export class SessionStore {
|
||||
const stmt = this.db.prepare(`
|
||||
INSERT INTO observations
|
||||
(sdk_session_id, project, type, title, subtitle, facts, narrative, concepts,
|
||||
files_read, files_modified, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
files_read, files_modified, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
const result = stmt.run(
|
||||
@@ -1121,6 +1156,7 @@ export class SessionStore {
|
||||
JSON.stringify(observation.files_read),
|
||||
JSON.stringify(observation.files_modified),
|
||||
promptNumber || null,
|
||||
discoveryTokens,
|
||||
now.toISOString(),
|
||||
nowEpoch
|
||||
);
|
||||
@@ -1146,7 +1182,8 @@ export class SessionStore {
|
||||
next_steps: string;
|
||||
notes: string | null;
|
||||
},
|
||||
promptNumber?: number
|
||||
promptNumber?: number,
|
||||
discoveryTokens: number = 0
|
||||
): { id: number; createdAtEpoch: number } {
|
||||
const now = new Date();
|
||||
const nowEpoch = now.getTime();
|
||||
@@ -1177,8 +1214,8 @@ export class SessionStore {
|
||||
const stmt = this.db.prepare(`
|
||||
INSERT INTO session_summaries
|
||||
(sdk_session_id, project, request, investigated, learned, completed,
|
||||
next_steps, notes, prompt_number, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
next_steps, notes, prompt_number, discovery_tokens, created_at, created_at_epoch)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
const result = stmt.run(
|
||||
@@ -1191,6 +1228,7 @@ export class SessionStore {
|
||||
summary.next_steps,
|
||||
summary.notes,
|
||||
promptNumber || null,
|
||||
discoveryTokens,
|
||||
now.toISOString(),
|
||||
nowEpoch
|
||||
);
|
||||
|
||||
@@ -471,6 +471,30 @@ export const migration006: Migration = {
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Migration 007 - Add discovery_tokens column for ROI metrics
|
||||
* Tracks token cost of discovering/creating each observation and summary
|
||||
*/
|
||||
export const migration007: Migration = {
|
||||
version: 7,
|
||||
up: (db: Database) => {
|
||||
// Add discovery_tokens to observations table
|
||||
db.run(`ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0`);
|
||||
|
||||
// Add discovery_tokens to session_summaries table
|
||||
db.run(`ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0`);
|
||||
|
||||
console.log('✅ Added discovery_tokens columns for ROI tracking');
|
||||
},
|
||||
|
||||
down: (db: Database) => {
|
||||
// Note: SQLite doesn't support DROP COLUMN in all versions
|
||||
// In production, would need to recreate tables without these columns
|
||||
console.log('⚠️ Warning: SQLite ALTER TABLE DROP COLUMN not fully supported');
|
||||
console.log('⚠️ To rollback, manually recreate the observations and session_summaries tables');
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* All migrations in order
|
||||
*/
|
||||
@@ -480,5 +504,6 @@ export const migrations: Migration[] = [
|
||||
migration003,
|
||||
migration004,
|
||||
migration005,
|
||||
migration006
|
||||
migration006,
|
||||
migration007
|
||||
];
|
||||
@@ -215,6 +215,7 @@ export interface ObservationRow {
|
||||
files_read: string | null; // JSON array
|
||||
files_modified: string | null; // JSON array
|
||||
prompt_number: number | null;
|
||||
discovery_tokens: number; // ROI metrics: tokens spent discovering this observation
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
@@ -232,6 +233,7 @@ export interface SessionSummaryRow {
|
||||
files_edited: string | null; // JSON array
|
||||
notes: string | null;
|
||||
prompt_number: number | null;
|
||||
discovery_tokens: number; // ROI metrics: cumulative tokens spent in this session
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
|
||||
@@ -36,6 +36,7 @@ interface StoredObservation {
|
||||
files_read: string | null; // JSON
|
||||
files_modified: string | null; // JSON
|
||||
prompt_number: number;
|
||||
discovery_tokens: number; // ROI metrics
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
@@ -51,6 +52,7 @@ interface StoredSummary {
|
||||
next_steps: string | null;
|
||||
notes: string | null;
|
||||
prompt_number: number;
|
||||
discovery_tokens: number; // ROI metrics
|
||||
created_at: string;
|
||||
created_at_epoch: number;
|
||||
}
|
||||
@@ -345,7 +347,8 @@ export class ChromaSync {
|
||||
project: string,
|
||||
obs: ParsedObservation,
|
||||
promptNumber: number,
|
||||
createdAtEpoch: number
|
||||
createdAtEpoch: number,
|
||||
discoveryTokens: number = 0
|
||||
): Promise<void> {
|
||||
// Convert ParsedObservation to StoredObservation format
|
||||
const stored: StoredObservation = {
|
||||
@@ -362,6 +365,7 @@ export class ChromaSync {
|
||||
files_read: JSON.stringify(obs.files_read),
|
||||
files_modified: JSON.stringify(obs.files_modified),
|
||||
prompt_number: promptNumber,
|
||||
discovery_tokens: discoveryTokens,
|
||||
created_at: new Date(createdAtEpoch * 1000).toISOString(),
|
||||
created_at_epoch: createdAtEpoch
|
||||
};
|
||||
@@ -387,7 +391,8 @@ export class ChromaSync {
|
||||
project: string,
|
||||
summary: ParsedSummary,
|
||||
promptNumber: number,
|
||||
createdAtEpoch: number
|
||||
createdAtEpoch: number,
|
||||
discoveryTokens: number = 0
|
||||
): Promise<void> {
|
||||
// Convert ParsedSummary to StoredSummary format
|
||||
const stored: StoredSummary = {
|
||||
@@ -401,6 +406,7 @@ export class ChromaSync {
|
||||
next_steps: summary.next_steps,
|
||||
notes: summary.notes,
|
||||
prompt_number: promptNumber,
|
||||
discovery_tokens: discoveryTokens,
|
||||
created_at: new Date(createdAtEpoch * 1000).toISOString(),
|
||||
created_at_epoch: createdAtEpoch
|
||||
};
|
||||
|
||||
@@ -180,10 +180,45 @@ export class WorkerService {
|
||||
this.app.get('/api/search/help', this.handleSearchHelp.bind(this));
|
||||
}
|
||||
|
||||
/**
|
||||
* Cleanup orphaned MCP server processes (uvx/chroma) from previous sessions
|
||||
*/
|
||||
private async cleanupOrphanedProcesses(): Promise<void> {
|
||||
try {
|
||||
const { execSync } = await import('child_process');
|
||||
|
||||
// Find orphaned uvx processes (which spawn chroma servers)
|
||||
try {
|
||||
const processes = execSync('pgrep -fl uvx', { encoding: 'utf-8', stdio: 'pipe' }).trim();
|
||||
if (processes) {
|
||||
const processCount = processes.split('\n').length;
|
||||
logger.info('WORKER', 'Cleaning up orphaned MCP processes', { count: processCount });
|
||||
|
||||
// Kill the processes
|
||||
execSync('pkill -f uvx', { stdio: 'pipe' });
|
||||
logger.success('WORKER', `Cleaned up ${processCount} orphaned MCP server processes`);
|
||||
}
|
||||
} catch (error: any) {
|
||||
// pgrep returns exit code 1 if no processes found (not an error)
|
||||
if (error.status === 1) {
|
||||
logger.debug('WORKER', 'No orphaned MCP processes to clean up');
|
||||
} else {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
// Don't fail startup if cleanup fails
|
||||
logger.warn('WORKER', 'Failed to cleanup orphaned processes (non-fatal)', {}, error as Error);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Start the worker service
|
||||
*/
|
||||
async start(): Promise<void> {
|
||||
// Cleanup orphaned processes from previous sessions
|
||||
await this.cleanupOrphanedProcesses();
|
||||
|
||||
// Initialize database (once, stays open)
|
||||
await this.dbManager.initialize();
|
||||
|
||||
@@ -215,6 +250,16 @@ export class WorkerService {
|
||||
// Shutdown all active sessions
|
||||
await this.sessionManager.shutdownAll();
|
||||
|
||||
// Close MCP client connection (terminates search server process)
|
||||
if (this.mcpClient) {
|
||||
try {
|
||||
await this.mcpClient.close();
|
||||
logger.info('SYSTEM', 'MCP client closed');
|
||||
} catch (error) {
|
||||
logger.error('SYSTEM', 'Failed to close MCP client', {}, error as Error);
|
||||
}
|
||||
}
|
||||
|
||||
// Close HTTP server
|
||||
if (this.server) {
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
@@ -222,7 +267,7 @@ export class WorkerService {
|
||||
});
|
||||
}
|
||||
|
||||
// Close database connection
|
||||
// Close database connection (includes ChromaSync cleanup)
|
||||
await this.dbManager.close();
|
||||
|
||||
logger.info('SYSTEM', 'Worker shutdown complete');
|
||||
|
||||
@@ -19,6 +19,8 @@ export interface ActiveSession {
|
||||
generatorPromise: Promise<void> | null;
|
||||
lastPromptNumber: number;
|
||||
startTime: number;
|
||||
cumulativeInputTokens: number; // Track input tokens for discovery cost
|
||||
cumulativeOutputTokens: number; // Track output tokens for discovery cost
|
||||
}
|
||||
|
||||
export interface PendingMessage {
|
||||
|
||||
@@ -30,16 +30,28 @@ export class DatabaseManager {
|
||||
// Initialize ChromaSync
|
||||
this.chromaSync = new ChromaSync('claude-mem');
|
||||
|
||||
// Start background backfill (fire-and-forget)
|
||||
this.chromaSync.ensureBackfilled().catch(() => {});
|
||||
// Start background backfill (fire-and-forget, with error logging)
|
||||
this.chromaSync.ensureBackfilled().catch((error) => {
|
||||
logger.error('DB', 'Chroma backfill failed (non-fatal)', {}, error);
|
||||
});
|
||||
|
||||
logger.info('DB', 'Database initialized');
|
||||
}
|
||||
|
||||
/**
|
||||
* Close database connection
|
||||
* Close database connection and cleanup all resources
|
||||
*/
|
||||
async close(): Promise<void> {
|
||||
// Close ChromaSync first (terminates uvx/python processes)
|
||||
if (this.chromaSync) {
|
||||
try {
|
||||
await this.chromaSync.close();
|
||||
this.chromaSync = null;
|
||||
} catch (error) {
|
||||
logger.error('DB', 'Failed to close ChromaSync', {}, error as Error);
|
||||
}
|
||||
}
|
||||
|
||||
if (this.sessionStore) {
|
||||
this.sessionStore.close();
|
||||
this.sessionStore = null;
|
||||
|
||||
@@ -85,6 +85,34 @@ export class SDKAgent {
|
||||
|
||||
const responseSize = textContent.length;
|
||||
|
||||
// Capture token state BEFORE updating (for delta calculation)
|
||||
const tokensBeforeResponse = session.cumulativeInputTokens + session.cumulativeOutputTokens;
|
||||
|
||||
// Extract and track token usage
|
||||
const usage = message.message.usage;
|
||||
if (usage) {
|
||||
session.cumulativeInputTokens += usage.input_tokens || 0;
|
||||
session.cumulativeOutputTokens += usage.output_tokens || 0;
|
||||
|
||||
// Cache creation counts as discovery, cache read doesn't
|
||||
if (usage.cache_creation_input_tokens) {
|
||||
session.cumulativeInputTokens += usage.cache_creation_input_tokens;
|
||||
}
|
||||
|
||||
logger.debug('SDK', 'Token usage captured', {
|
||||
sessionId: session.sessionDbId,
|
||||
inputTokens: usage.input_tokens,
|
||||
outputTokens: usage.output_tokens,
|
||||
cacheCreation: usage.cache_creation_input_tokens || 0,
|
||||
cacheRead: usage.cache_read_input_tokens || 0,
|
||||
cumulativeInput: session.cumulativeInputTokens,
|
||||
cumulativeOutput: session.cumulativeOutputTokens
|
||||
});
|
||||
}
|
||||
|
||||
// Calculate discovery tokens (delta for this response only)
|
||||
const discoveryTokens = (session.cumulativeInputTokens + session.cumulativeOutputTokens) - tokensBeforeResponse;
|
||||
|
||||
// Only log non-empty responses (filter out noise)
|
||||
if (responseSize > 0) {
|
||||
const truncatedResponse = responseSize > 100
|
||||
@@ -95,8 +123,8 @@ export class SDKAgent {
|
||||
promptNumber: session.lastPromptNumber
|
||||
}, truncatedResponse);
|
||||
|
||||
// Parse and process response
|
||||
await this.processSDKResponse(session, textContent, worker);
|
||||
// Parse and process response with discovery token delta
|
||||
await this.processSDKResponse(session, textContent, worker, discoveryTokens);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -218,8 +246,9 @@ export class SDKAgent {
|
||||
|
||||
/**
|
||||
* Process SDK response text (parse XML, save to database, sync to Chroma)
|
||||
* @param discoveryTokens - Token cost for discovering this response (delta, not cumulative)
|
||||
*/
|
||||
private async processSDKResponse(session: ActiveSession, text: string, worker?: any): Promise<void> {
|
||||
private async processSDKResponse(session: ActiveSession, text: string, worker: any | undefined, discoveryTokens: number): Promise<void> {
|
||||
// Parse observations
|
||||
const observations = parseObservations(text, session.claudeSessionId);
|
||||
|
||||
@@ -229,7 +258,8 @@ export class SDKAgent {
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
obs,
|
||||
session.lastPromptNumber
|
||||
session.lastPromptNumber,
|
||||
discoveryTokens
|
||||
);
|
||||
|
||||
// Log observation details
|
||||
@@ -253,7 +283,8 @@ export class SDKAgent {
|
||||
session.project,
|
||||
obs,
|
||||
session.lastPromptNumber,
|
||||
createdAtEpoch
|
||||
createdAtEpoch,
|
||||
discoveryTokens
|
||||
).then(() => {
|
||||
const chromaDuration = Date.now() - chromaStart;
|
||||
logger.debug('CHROMA', 'Observation synced', {
|
||||
@@ -305,7 +336,8 @@ export class SDKAgent {
|
||||
session.claudeSessionId,
|
||||
session.project,
|
||||
summary,
|
||||
session.lastPromptNumber
|
||||
session.lastPromptNumber,
|
||||
discoveryTokens
|
||||
);
|
||||
|
||||
// Log summary details
|
||||
@@ -326,7 +358,8 @@ export class SDKAgent {
|
||||
session.project,
|
||||
summary,
|
||||
session.lastPromptNumber,
|
||||
createdAtEpoch
|
||||
createdAtEpoch,
|
||||
discoveryTokens
|
||||
).then(() => {
|
||||
const chromaDuration = Date.now() - chromaStart;
|
||||
logger.debug('CHROMA', 'Summary synced', {
|
||||
|
||||
@@ -89,7 +89,9 @@ export class SessionManager {
|
||||
abortController: new AbortController(),
|
||||
generatorPromise: null,
|
||||
lastPromptNumber: promptNumber || this.dbManager.getSessionStore().getPromptCounter(sessionDbId),
|
||||
startTime: Date.now()
|
||||
startTime: Date.now(),
|
||||
cumulativeInputTokens: 0,
|
||||
cumulativeOutputTokens: 0
|
||||
};
|
||||
|
||||
this.sessions.set(sessionDbId, session);
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
#!/bin/bash
|
||||
# Test script to verify process cleanup
|
||||
# This script tests that uvx/python processes are properly cleaned up
|
||||
|
||||
set -e
|
||||
|
||||
echo "=== Process Cleanup Test ==="
|
||||
echo ""
|
||||
|
||||
# Function to count uvx/python processes
|
||||
count_processes() {
|
||||
local count=$(ps aux | grep -E "(uvx|python.*chroma)" | grep -v grep | wc -l)
|
||||
echo "$count"
|
||||
}
|
||||
|
||||
# Initial count
|
||||
echo "1. Initial process count:"
|
||||
initial=$(count_processes)
|
||||
echo " uvx/python/chroma processes: $initial"
|
||||
echo ""
|
||||
|
||||
# Start a node process that creates ChromaSync
|
||||
echo "2. Starting test process that creates ChromaSync..."
|
||||
cat > /tmp/test-chroma-cleanup.mjs << 'EOF'
|
||||
import { ChromaSync } from './src/services/sync/ChromaSync.js';
|
||||
|
||||
const sync = new ChromaSync('test-project');
|
||||
|
||||
console.log('[TEST] ChromaSync created, connecting...');
|
||||
|
||||
// Try to connect (this spawns uvx process)
|
||||
try {
|
||||
await sync.ensureBackfilled();
|
||||
console.log('[TEST] Backfill started');
|
||||
} catch (error) {
|
||||
console.log('[TEST] Backfill failed (expected if no data):', error.message);
|
||||
}
|
||||
|
||||
// Wait a bit for process to start
|
||||
await new Promise(resolve => setTimeout(resolve, 2000));
|
||||
|
||||
const countBefore = parseInt(process.env.COUNT_BEFORE || '0');
|
||||
const countAfter = process.argv[2];
|
||||
|
||||
console.log('[TEST] Process count before:', countBefore);
|
||||
|
||||
// Close the sync (should terminate uvx process)
|
||||
console.log('[TEST] Closing ChromaSync...');
|
||||
await sync.close();
|
||||
|
||||
// Wait for process to terminate
|
||||
await new Promise(resolve => setTimeout(resolve, 1000));
|
||||
|
||||
console.log('[TEST] ChromaSync closed, process should be terminated');
|
||||
process.exit(0);
|
||||
EOF
|
||||
|
||||
# Run test
|
||||
COUNT_BEFORE=$initial node /tmp/test-chroma-cleanup.mjs 2>&1 &
|
||||
TEST_PID=$!
|
||||
|
||||
# Wait for process to spawn
|
||||
sleep 3
|
||||
|
||||
# Count during execution
|
||||
during=$(count_processes)
|
||||
echo " During execution: $during processes"
|
||||
echo ""
|
||||
|
||||
# Wait for test to complete
|
||||
wait $TEST_PID 2>/dev/null || true
|
||||
|
||||
# Wait a bit for cleanup
|
||||
sleep 2
|
||||
|
||||
# Final count
|
||||
echo "3. Final process count:"
|
||||
final=$(count_processes)
|
||||
echo " uvx/python/chroma processes: $final"
|
||||
echo ""
|
||||
|
||||
# Check if we leaked processes
|
||||
leaked=$((final - initial))
|
||||
if [ $leaked -gt 0 ]; then
|
||||
echo "❌ FAIL: Leaked $leaked process(es)"
|
||||
echo ""
|
||||
echo "Current processes:"
|
||||
ps aux | grep -E "(uvx|python.*chroma)" | grep -v grep
|
||||
exit 1
|
||||
else
|
||||
echo "✅ PASS: No process leaks detected"
|
||||
fi
|
||||
|
||||
# Cleanup
|
||||
rm -f /tmp/test-chroma-cleanup.mjs
|
||||
Reference in New Issue
Block a user